Decomposition and Reformulation in Mixed-Integer Programming IMA New Directions Short Course on Mathematical Optimization Jim Luedtke Department of Industrial and Systems Engineering University of Wisconsin-Madison August 11, 2016 Jim Luedtke (UW-Madison) Decomposition Methods Lecture Notes 1 / 47 2 Huge number of variables and/or constraints Just solving LP relaxation is very time-consuming Possible solution: cutting plane approach { but doesn't address huge number of variables Another possibility: Lagrangian relaxation or column generation May be able to help with either or both of these challenges Particularly useful if enables decomposition { splitting one large problem into many smaller ones What makes integer programs hard to solve? 1 Weak LP relaxation bounds Pruning in branch-and-bound is rare { huge search trees Possible solution: use a better formulation or add (strong) valid inequalities Jim Luedtke (UW-Madison) Decomposition Methods Lecture Notes 2 / 47 Another possibility: Lagrangian relaxation or column generation May be able to help with either or both of these challenges Particularly useful if enables decomposition { splitting one large problem into many smaller ones What makes integer programs hard to solve? 1 Weak LP relaxation bounds Pruning in branch-and-bound is rare { huge search trees Possible solution: use a better formulation or add (strong) valid inequalities 2 Huge number of variables and/or constraints Just solving LP relaxation is very time-consuming Possible solution: cutting plane approach { but doesn't address huge number of variables Jim Luedtke (UW-Madison) Decomposition Methods Lecture Notes 2 / 47 What makes integer programs hard to solve? 1 Weak LP relaxation bounds Pruning in branch-and-bound is rare { huge search trees Possible solution: use a better formulation or add (strong) valid inequalities 2 Huge number of variables and/or constraints Just solving LP relaxation is very time-consuming Possible solution: cutting plane approach { but doesn't address huge number of variables Another possibility: Lagrangian relaxation or column generation May be able to help with either or both of these challenges Particularly useful if enables decomposition { splitting one large problem into many smaller ones Jim Luedtke (UW-Madison) Decomposition Methods Lecture Notes 2 / 47 Lagrangian Relaxation Outline 1 Lagrangian Relaxation Example Lagrangian Relaxation Bounds Solving the Lagrangian Dual 2 Dantzig-Wolfe Reformulation and Column Generation 3 Branch-and-Price Jim Luedtke (UW-Madison) Decomposition Methods Lecture Notes 3 / 47 If only we didn't have those pesky Dx ≤ d constraints . Lagrangian Relaxation Motivation Consider this IP: K IP X > z = max ck xk k=1 Dx ≤ d Akxk ≤ bk; k = 1;:::;K nK x = (x1; : : : ; xK ) 2 Z+ Jim Luedtke (UW-Madison) Decomposition Methods Lecture Notes 4 / 47 Lagrangian Relaxation Motivation Consider this IP: K IP X > z = max ck xk k=1 Dx ≤ d Akxk ≤ bk; k = 1;:::;K nK x = (x1; : : : ; xK ) 2 Z+ If only we didn't have those pesky Dx ≤ d constraints . Jim Luedtke (UW-Madison) Decomposition Methods Lecture Notes 4 / 47 R PK The problem decomposes: z = k=1 zk where > n zk = max ck xk : Akxk ≤ bk; xk 2 Z+ for k = 1;:::;K zR ≥ zIP , but dropping Dx ≤ d altogether is pretty severe Lagrangian Relaxation Motivation Drop the pesky constraints: K R X > z = max ck xk k=1 Akxk ≤ bk; k = 1;:::;K nK x = (x1; : : : ; xK ) 2 Z+ Jim Luedtke (UW-Madison) Decomposition Methods Lecture Notes 5 / 47 zR ≥ zIP , but dropping Dx ≤ d altogether is pretty severe Lagrangian Relaxation Motivation Drop the pesky constraints: K R X > z = max ck xk k=1 Akxk ≤ bk; k = 1;:::;K nK x = (x1; : : : ; xK ) 2 Z+ R PK The problem decomposes: z = k=1 zk where > n zk = max ck xk : Akxk ≤ bk; xk 2 Z+ for k = 1;:::;K Jim Luedtke (UW-Madison) Decomposition Methods Lecture Notes 5 / 47 Lagrangian Relaxation Motivation Drop the pesky constraints: K R X > z = max ck xk k=1 Akxk ≤ bk; k = 1;:::;K nK x = (x1; : : : ; xK ) 2 Z+ R PK The problem decomposes: z = k=1 zk where > n zk = max ck xk : Akxk ≤ bk; xk 2 Z+ for k = 1;:::;K zR ≥ zIP , but dropping Dx ≤ d altogether is pretty severe Jim Luedtke (UW-Madison) Decomposition Methods Lecture Notes 5 / 47 Lagrangian relaxation: relax the constraints Dx ≤ d by dualizing them { adding them to the objective with a penalty for violation Problem with just constraints Ax ≤ b should be easier to solve We'll discuss how to choose the constraints to dualize later n For simplicity we assume x 2 Z+ Lagrangian Relaxation Lagrangian Relaxation Simplify notation: Let A be the matrix combining all Ak submatrices, and b = (b1; : : : ; bK ), c = (c1; : : : ; cK ) zIP = max c>x Dx ≤ d Ax ≤ b n x 2 Z+ Jim Luedtke (UW-Madison) Decomposition Methods Lecture Notes 6 / 47 Problem with just constraints Ax ≤ b should be easier to solve We'll discuss how to choose the constraints to dualize later n For simplicity we assume x 2 Z+ Lagrangian Relaxation Lagrangian Relaxation Simplify notation: Let A be the matrix combining all Ak submatrices, and b = (b1; : : : ; bK ), c = (c1; : : : ; cK ) zIP = max c>x Dx ≤ d Ax ≤ b n x 2 Z+ Lagrangian relaxation: relax the constraints Dx ≤ d by dualizing them { adding them to the objective with a penalty for violation Jim Luedtke (UW-Madison) Decomposition Methods Lecture Notes 6 / 47 Lagrangian Relaxation Lagrangian Relaxation Simplify notation: Let A be the matrix combining all Ak submatrices, and b = (b1; : : : ; bK ), c = (c1; : : : ; cK ) zIP = max c>x Dx ≤ d Ax ≤ b n x 2 Z+ Lagrangian relaxation: relax the constraints Dx ≤ d by dualizing them { adding them to the objective with a penalty for violation Problem with just constraints Ax ≤ b should be easier to solve We'll discuss how to choose the constraints to dualize later n For simplicity we assume x 2 Z+ Jim Luedtke (UW-Madison) Decomposition Methods Lecture Notes 6 / 47 Why? Let x∗ be an optimal solution to zIP (zIP = c>x∗ and d − Dx∗ ≥ 0). z(u) ≥ c>x∗ + u>(d − Dx∗) ≥ zIP Lagrangian Relaxation Lagrangian Relaxation n First, re-write our IP, with X := fx 2 Z+ : Ax ≤ bg zIP = max c>x subject to Dx ≤ d x 2 X m Let u 2 R+ , and define the following Lagrangian Relaxation problem: IP (u): z(u) = maxfc>x + u>(d − Dx): x 2 Xg Theorem m IP For any u 2 R+ , z(u) ≥ z . Jim Luedtke (UW-Madison) Decomposition Methods Lecture Notes 7 / 47 Lagrangian Relaxation Lagrangian Relaxation n First, re-write our IP, with X := fx 2 Z+ : Ax ≤ bg zIP = max c>x subject to Dx ≤ d x 2 X m Let u 2 R+ , and define the following Lagrangian Relaxation problem: IP (u): z(u) = maxfc>x + u>(d − Dx): x 2 Xg Theorem m IP For any u 2 R+ , z(u) ≥ z . Why? Let x∗ be an optimal solution to zIP (zIP = c>x∗ and d − Dx∗ ≥ 0). z(u) ≥ c>x∗ + u>(d − Dx∗) ≥ zIP Jim Luedtke (UW-Madison) Decomposition Methods Lecture Notes 7 / 47 Modification with equality constraints Dx = d: variables u are free Lagrangian Relaxation Lagrangian Dual m For u 2 R+ , z(u) = maxfc>x + u>(d − Dx): x 2 Xg Definition The problem: LD m w = minfz(u): u 2 R+ g is called a Lagrangian dual. Properties wLD ≥ zIP LD m w ≤ z(u) for all u 2 R+ Jim Luedtke (UW-Madison) Decomposition Methods Lecture Notes 8 / 47 Lagrangian Relaxation Lagrangian Dual m For u 2 R+ , z(u) = maxfc>x + u>(d − Dx): x 2 Xg Definition The problem: LD m w = minfz(u): u 2 R+ g is called a Lagrangian dual. Properties wLD ≥ zIP LD m w ≤ z(u) for all u 2 R+ Modification with equality constraints Dx = d: variables u are free Jim Luedtke (UW-Madison) Decomposition Methods Lecture Notes 8 / 47 (1) ) x^ is feasible (2) ) zIP ≤ z(u) = c>x^ + u>(d − Dx^) = c>x^ Lagrangian Relaxation Lagrangian Dual IP (u): z(u) = maxfc>x + u>(d − Dx): x 2 Xg Theorem m Let u 2 R+ and x^ be an optimal solution to IP (u). If, (1) Dx^ ≤ d, and (2) u>(d − Dx^) = 0, then x^ is an optimal solution to IP. Note: The second condition is necessary: x^ could be feasible to IP, but not optimal. Jim Luedtke (UW-Madison) Decomposition Methods Lecture Notes 9 / 47 Lagrangian Relaxation Lagrangian Dual IP (u): z(u) = maxfc>x + u>(d − Dx): x 2 Xg Theorem m Let u 2 R+ and x^ be an optimal solution to IP (u). If, (1) Dx^ ≤ d, and (2) u>(d − Dx^) = 0, then x^ is an optimal solution to IP. Note: The second condition is necessary: x^ could be feasible to IP, but not optimal. (1) ) x^ is feasible (2) ) zIP ≤ z(u) = c>x^ + u>(d − Dx^) = c>x^ Jim Luedtke (UW-Madison) Decomposition Methods Lecture Notes 9 / 47 Lagrangian Relaxation Example Example: Stochastic Integer Programming Extensive form of SIP S SMIP > X > z = min c x + psqs ys s=1 s.t. Ax ≥ b Tsx + Wsys = hs s = 1;:::;S n1 p1 x 2 R+ × Z+ n2 p2 ys 2 R+ × Z+ ; s = 1;:::;S Jim Luedtke (UW-Madison) Decomposition Methods Lecture Notes 10 / 47 Lagrangian Relaxation Example Example: Stochastic Integer Programming Copy first-stage variables S SMIP X > > z = min ps(c xs + qs ys) s=1 Axs ≥ bs = 1;:::;S Tsxs + Wsys = hs s = 1;:::;S S X xs = ps0 xs0 s = 1;:::;S s0=1 n1 p1 xs 2 R+ × Z+ ;s = 1;:::;S n2 p2 ys 2 R+ × Z+ ; s = 1;:::;S Jim Luedtke (UW-Madison) Decomposition Methods Lecture Notes 10 / 47 Relax these constraints using Lagrangian Relaxation with dual vectors λ = (λ1; : : : ; λS): PS > > PS > PS L(λ) := min s=1 ps(c xs + qs ys)+ s=1 psλs xs − s0=1 ps0 xs0 Axs ≥ b s = 1;:::;S Tsxs + Wsys = hs s = 1;:::;S n1 p1 xs 2 R+ × Z+ ; s = 1;:::;S n2 p2 ys 2 R+ × Z+ ; s = 1;:::;S ¯ PS Rewrite the objective (λ = s=1 psλs): PS ¯ > > s=1 ps c + (λs − λ) xs + qs ys Lagrangian Relaxation Example Relax Nonanticipativity PS The constraints xs = s0=1 ps0 xs0 are called nonanticipativity constraints.