One more general point Reconnect ‘04 Solving Integer Programs with Branch and Node selection: when working in serial, how do you pick an active node Bound (and ) to process next?

Usual: best first • Select the node with the lowest lower bound • With the current incumbent and solution tolerances, you will have to Cynthia Phillips (Sandia National Laboratories) evaluate it anyway If you don’t have a good incumbent finder, you might start with diving: • Select the most refined node • Once you have an incumbent, switch to best first

Slide 2 Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy under contract DE-AC04-94AL85000.

Mixed- (IP) Branch and Bound

T Min c x • Bounding function: solve LP relaxation Subject to: Ax = b • Branching

l ≤ x ≤ u – Select an integer variable xi that’s fractional in the LP solution x* x x* x = (xI ,xC ) – Up child: set i ≥  i  n – Down child: set x ≤ x* xI ∈ Z (integer values) i  i  x ∈ Qn (rational values) – x* is no longer feasible in either child C € • Incumbent€ method – many €incumbent-finding methods to follow in later lectures € – Obvious: return the LP solution if it is integer • Satisfies feasibility-tester constraint

Slide 3 Slide 4 Solving Integer Programs: Branch and Bound Branching Decisions: One method

Root Problem = original Which candidate variable (of thousands or more choices) to choose? • Use structural insight (user priorities) when possible. • Use gradients (pseudocosts): x i = 0 x i = 1

bound x = 0.3 12 j x j = 0 x j = 1 x k = 0 x k = 1

xj = 1 xj = 0 15 13.4

15 −12 13.4 −12 infeasible Down: = 10 Up = = 2 fathomed 0.3 1 − 0.3

• Lower bound: LP relaxation

Slide 5 Slide 6

Using gradients Branch Variable Selection method 2: Strong Branching

• Use gradients to compute an expected bound movement for each child • “Try out” interesting subproblems for a while – Up = up gradient * upward round distance – Could be like a gradient “initialization” at every node – Down = down gradient * downward round distance – Could compute part of the subtree • Try to find a variable for which both children might move the bound • This can be expensive, but sometimes these decisions make a huge difference (especially very early) Initialization • When a variable is fractional for the first time, intialize by “pretending” to branch (better than using an average)

Slide 7 Slide 8 Branching on constraints Special Ordered Sets (SOS)

Can have one child with new constraint: T • Models selection: m a x ≤ b1 y =1 ∑i 1 i and the other with new constraint: aT x ≥ b = 2 y ∈ {0,1} • Must cover the subregion i • Pieces can be omitted, but only€ if provably have nothing potentially optimal € Example: time-indexed scheduling € • Node bounds are just a special case • xjt = 1 if job j is scheduled at time t • Ordered by time More generally, partition into many children

aT x ≤ b , b ≤ aT x ≤ b , , b ≤ x ≤ b 1 2 3 K k−1 k

€Slide 9 Slide 10

Special Ordered Sets: Restricted Set of Values (Review) Problems with Simple Branching on SOS

• Variable x can take on only values in {v1,v2, vm } Generally want both children to differ from parent substantially K – Frequently the vi are sorted For SOS • Capacity of an airplane assigned to a flight • The up child (setting a variable to 1) is very powerful € – All others in the set go to zero m • The down child will likely have an LP solution = parent’s x = ∑vi yi i=1 – “Schedule at any of these 1000 times except this one” m yi =1 ∑i=1 y ∈ {0,1}

– The y ’s are a . i €

Slide 11 Slide 12 Special Ordered Sets: Weak Down-Child Example Branching on SOS

m

• Variable x can take on only values in {v1,v2, vm } Set variables {x ,x , x } x =1 K 1 2 K m ∑ j • Plane capacities, values {50,100,200} Partition about an index i: j=1

m m x = € v y € ∑ i i • Up child has ∑ x j =1 i 1 = €j= i+1 m i yi =1 ∑i=1 • Down child has ∑ x j =1 y ∈ {0,1} j=1 € • Examples: If v2=0 (can’t use capacity 100), “fake” a 100-passenger plane by using – Schedule job j before/after time t € € 2 1 – Use a plane of capacity at least/at most s x + x 3 1 3 3

Slide 13 Slide 14 €

Branching on SOS Postponing Branching

• Good choice for partition point i is such that as nearly as possible Branching causes exponential growth of the search tree Is there a way to make progress without branching? i m 1 ∑ x j = ∑ x j = j=1 j= i+1 2

• Can compute gradients as with simple variable branching € • This is not general branching on a constraint: just setting multiple variable upper bounds simultaneously

Slide 15 Slide 16 Strengthen Linear Program with Cutting Planes Example: Maximum Independent Set

1/2 Cutting plane 1/2 1 4 (valid inequality) Original LP 1/2 1/2 1/2 2 3 7

5 6 1/2 1/2

1 if vertex i is in the MIS vi =  Integer optimal LP optimal solution  0 otherwise max v • Make LP polytope closer to integer polytope ∑ i

• Use families of constraints too large to explicitly list s.t. vi + vj ≤ 1 ∀(i, j) ∈E – Exponential, pseudopolynomial, polynomial (n4, n5)

Slide 17 Slide 18

Example: Some Maximum Independent Set Cutting Planes Value of a Good Feasible Solution Found Early

1/2 Root Problem = original 1 4

1/2 x = 0 x = 1 2 3 7 i i

5 6 1/2 x j = 0 x j = 1 x k = 0 x k = 1

vi + vj + vk ≤1 ∀ triangles i, j, k

fathomed infeasible More general clique inequalities (keep going till they’re too hard to find) • Simulate seeding with optimal (computation is a proof of optimality) • Note: proof of optimality depends on a good LP relaxation

Slide 19 Slide 20 Solving Subproblem LPs Quickly Geometry

• The LP relaxation of a child is usually closely related to its parent’s LP Optimal point of an LP is at a corner (assuming bounded) • Exploit that similarity by saving the parent’s basis

cT x opt = feasible

€

cT x = -∞

Slide 21 Slide 22

€

Linear Programming Algebra Linear Programming Algebra

What does a corner look like algebraically? We have Bx B + LxL + UxU = b

Ax=b Set all members of xL to their lower bound.

Partition A matrix into three parts Set all members of xU to their upper bound.

Let b ′ = b − Lx L − Ux U (this is a constant because bounds l and u are) Thus we have BxB = b′ B L U −1 Set xB = B b′ € €

This €setting of (xB, xL, xU) is called a basic solution where B is nonsingular (invertible, square). € • A basic solution satisfies Ax=b by construction Reorder x: (x , x , x ) If all x satisfy their bounds ( ≤ x ≤ u ), this is a basic feasible B L U B l B B B We have Bx B + LxL + Ux U = b solution (BFS)

Slide 23 Slide 24 € Basic Feasible Solutions Algebra and Geometry

We have Bx B + LxL + UxU = b A BFS corresponds to a corner of the feasible polytope:

Set all members of xL to their lower bound. Ax ≤ b

Set all members of xU to their upper bound. m inequality constraints (plus bounds) and n variables. Polytope in n-

Let b ′ = b − Lx L − Ux U (this is a constant because bounds l and u are) dimensional space. A corner has n tight constraints. −1 Set xB = B b′ €

With slacks Ax + IxS = b € In the common case m equality constraints (plus bounds) in n+m variables. A BFS has n tight l = 0, u = ∞ (x ≥ 0), €

€ we have b = b′ , xU = ∅, xL = N bound constraints (from the nonbasic variables).

xB are basic variables, N are nonbasic (xL are nonbasic at lower, xU are nonbasic€ at upper) €

Slide 25 Slide 26

Dual LP Primal/Dual Pair

(Simplified) primal LP problem is minimize cT x such that Ax ≤ b x ≥ 0 Primal feasible

The dual problem is: maximize€ yTb Opt such that yA c ≥ € Dual feasible y ≥ 0 • dual(dual(primal)) = primal • Frequently has a nice interpretation (max flow/min cut) €

Slide 27 Slide 28 Parent/Child relationship (intuition)

Parent optimal pair (x*,y*) • Branching reduces the feasible region of the child LP with respect to its parent and increases the dual feasible region • y* is feasible in the child’s dual LP and it’s close to optimal

Resolving the children using dual simplex, starting from the parent’s optimal basis can be at least an order of magnitude faster than starting from nothing.

Note: Basis can be big. Same space issues as with knapsack example.

Slide 29