Cutting plane and column generation metho ds using interior p oint metho ds John E. Mitchell Mathematical Sciences, Rensselaer Polytechnic Institute, Troy NY 12180 USA Visiting: TWI/SSOR, Delft UniversityofTechnology, 2628 CD Delft, The Netherlands email: [email protected] or [email protected] http://www.math.rpi.edu/~mitchj TWI/SSOR, TU Delft 25 Novemb er, 1997 Abstract Column generation metho ds can b e used to solveinte- ger programming problems, sto chastic programming prob- lems, crew scheduling problems, and multicommo dity net- work ow problems, among others. These metho ds solve a sequence of linear programming subproblems. The rapid increase in computational p ower in the last few years has made it p ossible to solvevery large instances. For such problems, it is often more ecient to use an interior p oint metho d to solve the linear programming subproblems than to use the simplex metho d. We survey b oth the theoretical and computational research in this area. 1 Intro duction Interested in linear programming problems with a large numb er of constraints or a large num- ber of variables. Solve problems using constraint generation or column generation metho ds. Solve linear programming subproblems using inte- rior p oint metho ds These techniques are used to solve: { Integer programming problems { Multicommo dity network ow problems { Sto chastic programming problems { Semi-in nite programming problems { Airline crew scheduling problems { Bounded error parameter estimation { Adaptive ltering 2 Solving an integer program with cutting planes Example: min 2x x 1 2 s.t. x + 2x 7 1 2 2x x 3 1 2 x ;x 0; integer 1 2 6 x 2 H H H H H * H t t 3 H A H H A H H A H H A H H A i H H A H t t t 2 A A * t t t 1 - t t 1 2 x 0 1 Linear programming relaxation Solve LP relaxation. Obtain x =2:6, x =2:2, value 7:4. 1 2 3 Cutting planes: min 2x x 1 2 s.t. x + 2x 7 1 2 2x x 3 1 2 x + x 4 1 2 x 2 1 x ;x 0 1 2 6 x 2 H H H H H H t t 3 H * H H A @ H A H H A H @ H A H H A H @i H t t t A 2 A A A A A * t t t 1 - t t 1 2 0 x 1 After adding two cuts Solve. Optimal solution is x =2,x =2,value 6. 1 2 Note: feasible in the original integer program, so optimal, since optimal for a relaxation. 4 Convex hull of integer p oints: 6 x 2 H H H H H * t t H H 3 A p p p p p p H H A p p p p p p p @ H H A p p p p p p p p H H A p p p p p p p p p @ H H A p p p p p p p p p p H H A p p p p p p p p p p p @ t t t H 2 A p p p p p p p p p p p A* p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p t t t 1 p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p - t t 0 x 1 2 1 A cutting plane example If we knew all the constraints of the convex hull of the integer program then we could solve the integer program by solving one linear program. Unfortunately, hard to get such a description. 5 Basic structure of a cutting plane algorithm 1. Solve the linear programming relaxation. 2. If the solution to the relaxation is feasible in the inte- ger programming problem, STOP with optimality. 3. Else, nd one or more cutting planes that separate the optimal solution to the relaxation from the con- vex hull of feasible integral p oints, and add a subset of these constraints to the relaxation. 4. Return to step 1. 6 Notes: Classically, rst relaxation is solved using the primal simplex algorithm. After the addition of cutting planes, the current pri- mal iterate b ecomes infeasible. However, the change to the dual problem is the addi- tion of some columns with corresp onding variables. If these extra dual variables are given the value 0 then the current dual solution is still dual feasible. Therefore, subsequent relaxations are solved using the dual simplex metho d in the classical approach. Dual simplex metho d is quick to reoptimize after ad- dition of just a few constraints. Notice that the values of the relaxations provide lower b ounds on the optimal value of the integer program. These lower b ounds can b e used to measure progress towards optimality, and to give p erformance guaran- tees on integral solutions. 7 Interior p oint metho ds for linear programs Interior p oint metho ds generally faster than simplex for large problems, ie, more than a thousand variables and/or constraints. Interior p oint metho ds try to keep iterates well- centered: good to keep x z for all com- i i p onents, where varies from iteration to iteration. z =dual slack. If x z = for all comp onents then i i the p ointisonthe central tra jectory. Duality gap is approximately n. n is number of primal variables. Harder to exploit a warm start with an interior p oint metho d. 8 A closer lo ok at adding a constraint Approximately solve primal-dual LP pair T T min c x max b y T s.t. Ax = b P s.t. A y c D x 0 Approximate solution: x , y . T Use interior p oint metho d so x > 0, A y<c. T Decide to add constraint g x h. Mo di ed primal-dual pair: T min c x s.t. Ax = b P 0 T g x x = h 0 x; x 0 0 T max b y + hy 0 T s.t. A y + gy c D 0 0 y 0 0 T Now g x <h, since cutting plane. So infeasible in P . 0 In D , y = y , y = 0 is feasible. 0 0 9 Restarting with interior p oint metho d: Have dual feasibility, but not interior dual p oint. Use ane direction to regain dual interior p oint: T 1 y = AA Ag ; y =1: 0 This direction gives interior p oint with minimal dis- T ruption to dual feasibility. Adjustmentto A y + gy 0 is pro jection of g onto null space of A. Not easy to regain primal interior feasible p oint. Options: { Infeasible primal-dual metho d. { Backup to earlier iterate. { Dual metho d. Would also liketogetawell-centered iterate. 10 Analytic Center Cutting Plane Metho d ACCPM Gon, Vial et al., late 80's onwards Originally develop ed for nondi erentiable optimiza- tion problems. Gon, Haurie, Vial 1992. Subsequently applied to sto chastic programming prob- lems, multicommo dity network ow problems. Similar to the indep endently develop ed Mitchell- To dd 1992 application of the primal pro jective al- gorithm to solveinteger programming problems with an interior p oint cutting plane metho d. How the algorithm works Find a weighted analytic center of an approximation to the set of optimal solutions. If this center is feasible, use a contour of the ob jective function for a cut. Else, cut o the center. Return to the rst step. 11 Example of ACCPM Want to nd lowest p oint on curve. Use piecewise linear approximation. 6 Ob j B B B v B Analytic center B r B Feasible region B B for relaxation B B B B B B B B B B B B B B B rr rr rr rr rr rr rr rr rr B rr rr rr rr rr rr rr rr r B rr rr rr rr rr rr rr rr B rrrrrrrrrrrrrr B rrrrrrrrrrrrr B v rrrrrrrrrrrr B rr rr rr rr rr B rr rr rr rr r B rr rr rr rr B rrrrrr B rrrrr B rrrr B rr B r B B B - x 12 Analytic center infeasible, so cut it o : 6 Ob j B B B v Analytic center B B r Feasible region B B for relaxation B B B B B B B B B B B B B B B B B rr rr rr rr rr rr rr rr rr H B rr rr rr rr rr rr rr r H v H B rr rr rr rr rr rr H H B rr rr rr rr r H H B rr rr rr H H B rr r H H B H B B B B B B B B B - x 13 Analytic center feasible, so cut on ob jective function: 6 Ob j B B B v Analytic center B B r Feasible region B B for relaxation B B B B B B B B B B B B B B B B B H B H H B H rr rr rr rr rr rr H B H rr rr rr rr r H B H rr rrv rr H B H rr r H B H H B B B B B B B B B - x Analytic center optimal for original problem, so STOP.