The practical revised simplex method Julian Hall School of Mathematics University of Edinburgh January 25th 2007 The practical revised simplex method Overview Overview Part 1: • The mathematics of linear programming Overview Part 1: • The mathematics of linear programming • The simplex method for linear programming Overview Part 1: • The mathematics of linear programming • The simplex method for linear programming ◦ The standard simplex method ◦ The revised simplex method Overview Part 1: • The mathematics of linear programming • The simplex method for linear programming ◦ The standard simplex method ◦ The revised simplex method • Sparsity Overview Part 1: • The mathematics of linear programming • The simplex method for linear programming ◦ The standard simplex method ◦ The revised simplex method • Sparsity ◦ Basic concepts ◦ Example from Gaussian elimination ◦ Sparsity in the standard simplex method Overview Part 1: • The mathematics of linear programming • The simplex method for linear programming ◦ The standard simplex method ◦ The revised simplex method • Sparsity ◦ Basic concepts ◦ Example from Gaussian elimination ◦ Sparsity in the standard simplex method Part 2: • Practical implementation of the revised simplex method Overview Part 1: • The mathematics of linear programming • The simplex method for linear programming ◦ The standard simplex method ◦ The revised simplex method • Sparsity ◦ Basic concepts ◦ Example from Gaussian elimination ◦ Sparsity in the standard simplex method Part 2: • Practical implementation of the revised simplex method • Parallel simplex Overview Part 1: • The mathematics of linear programming • The simplex method for linear programming ◦ The standard simplex method ◦ The revised simplex method • Sparsity ◦ Basic concepts ◦ Example from Gaussian elimination ◦ Sparsity in the standard simplex method Part 2: • Practical implementation of the revised simplex method • Parallel simplex • Research frontiers The practical revised simplex method 1 Solving LP problems minimize f = cT x subject to Ax = b x ≥ 0 where x ∈ IRn and b ∈ IRm. Solving LP problems minimize f = cT x subject to Ax = b x ≥ 0 where x ∈ IRn and b ∈ IRm. • The feasible region is the solution set of equations and bounds Geometrically, it is a convex polyhedron in IRn. Solving LP problems minimize f = cT x subject to Ax = b x ≥ 0 where x ∈ IRn and b ∈ IRm. • The feasible region is the solution set of equations and bounds Geometrically, it is a convex polyhedron in IRn. • At any vertex the variables may be partitioned into index sets ◦ B of m basic variables xB ≥ 0: ◦ N of n − m nonbasic variables xN = 0. Solving LP problems minimize f = cT x subject to Ax = b x ≥ 0 where x ∈ IRn and b ∈ IRm. • The feasible region is the solution set of equations and bounds Geometrically, it is a convex polyhedron in IRn. • At any vertex the variables may be partitioned into index sets ◦ B of m basic variables xB ≥ 0: ◦ N of n − m nonbasic variables xN = 0. • Components of c and columns of A are ◦ the basic costs cB and basis matrix B; Solving LP problems minimize f = cT x subject to Ax = b x ≥ 0 where x ∈ IRn and b ∈ IRm. • The feasible region is the solution set of equations and bounds Geometrically, it is a convex polyhedron in IRn. • At any vertex the variables may be partitioned into index sets ◦ B of m basic variables xB ≥ 0: ◦ N of n − m nonbasic variables xN = 0. • Components of c and columns of A are ◦ the basic costs cB and basis matrix B; ◦ the non-basic costs cN and matrix N. Solving LP problems minimize f = cT x subject to Ax = b x ≥ 0 where x ∈ IRn and b ∈ IRm. • The feasible region is the solution set of equations and bounds Geometrically, it is a convex polyhedron in IRn. • At any vertex the variables may be partitioned into index sets ◦ B of m basic variables xB ≥ 0: ◦ N of n − m nonbasic variables xN = 0. • Components of c and columns of A are ◦ the basic costs cB and basis matrix B; ◦ the non-basic costs cN and matrix N. • Results: ◦ At any vertex there is a partition {B, N} of the variables such that B is nonsingular. Solving LP problems minimize f = cT x subject to Ax = b x ≥ 0 where x ∈ IRn and b ∈ IRm. • The feasible region is the solution set of equations and bounds Geometrically, it is a convex polyhedron in IRn. • At any vertex the variables may be partitioned into index sets ◦ B of m basic variables xB ≥ 0: ◦ N of n − m nonbasic variables xN = 0. • Components of c and columns of A are ◦ the basic costs cB and basis matrix B; ◦ the non-basic costs cN and matrix N. • Results: ◦ At any vertex there is a partition {B, N} of the variables such that B is nonsingular. ◦ There is an optimal solution of the problem at a vertex. The practical revised simplex method 2 Conditions for optimality At any vertex the original problem is T T minimize f = cN xN + cBxB subject to N xN + B xB = b xN ≥ 0 xB ≥ 0. Conditions for optimality At any vertex the original problem is T T minimize f = cN xN + cBxB subject to N xN + B xB = b xN ≥ 0 xB ≥ 0. Multiply through by B−1 to form the reduced equations NˆxN + xB = bˆ Conditions for optimality At any vertex the original problem is T T minimize f = cN xN + cBxB subject to N xN + B xB = b xN ≥ 0 xB ≥ 0. Multiply through by B−1 to form the reduced equations NˆxN + xB = bˆ where the reduced non-basic matrix is Nˆ = B−1N Conditions for optimality At any vertex the original problem is T T minimize f = cN xN + cBxB subject to N xN + B xB = b xN ≥ 0 xB ≥ 0. Multiply through by B−1 to form the reduced equations NˆxN + xB = bˆ where the reduced non-basic matrix is Nˆ = B−1N and the vector of values of the basic variables is bˆ = B−1b. The practical revised simplex method 3 Conditions for optimality (cont.) Use the reduced equations to eliminate xB from the objective to give the reduced problem T ˆ minimize f = cˆN xN + f subject to Nˆ xN + I xB = bˆ xN ≥ 0 xB ≥ 0, where cˆN is the vector of reduced costs given by T T T ˆ cˆN = cN − cBN Conditions for optimality (cont.) Use the reduced equations to eliminate xB from the objective to give the reduced problem T ˆ minimize f = cˆN xN + f subject to Nˆ xN + I xB = bˆ xN ≥ 0 xB ≥ 0, where cˆN is the vector of reduced costs given by T T T ˆ cˆN = cN − cBN and the value of the objective at the vertex is ˆ T ˆ f = cBb. Conditions for optimality (cont.) Use the reduced equations to eliminate xB from the objective to give the reduced problem T ˆ minimize f = cˆN xN + f subject to Nˆ xN + I xB = bˆ xN ≥ 0 xB ≥ 0, where cˆN is the vector of reduced costs given by T T T ˆ cˆN = cN − cBN and the value of the objective at the vertex is ˆ T ˆ f = cBb. Necessary and sufficient conditions for optimality are cˆN ≥ 0. The practical revised simplex method 4 The standard simplex method (1948) N B RHS 1 . Nˆ I bˆ m T T ˆ 0 cˆN 0 −f The standard simplex method (1948) N B RHS 1 . Nˆ I bˆ m T T ˆ 0 cˆN 0 −f In each iteration: The standard simplex method (1948) N B RHS 1 . Nˆ I bˆ m T T ˆ 0 cˆN 0 −f In each iteration: • Select the pivotal column q0 of a nonbasic variable q ∈ N to be increased from zero. The standard simplex method (1948) N B RHS 1 . Nˆ I bˆ m T T ˆ 0 cˆN 0 −f In each iteration: • Select the pivotal column q0 of a nonbasic variable q ∈ N to be increased from zero. • Find the pivotal row p of the first basic variable p0 ∈ B to be zeroed. The standard simplex method (1948) N B RHS 1 . Nˆ I bˆ m T T ˆ 0 cˆN 0 −f In each iteration: • Select the pivotal column q0 of a nonbasic variable q ∈ N to be increased from zero. • Find the pivotal row p of the first basic variable p0 ∈ B to be zeroed. • Exchange indices p0 and q between sets B and N . The standard simplex method (1948) N B RHS 1 . Nˆ I bˆ m T T ˆ 0 cˆN 0 −f In each iteration: • Select the pivotal column q0 of a nonbasic variable q ∈ N to be increased from zero. • Find the pivotal row p of the first basic variable p0 ∈ B to be zeroed. • Exchange indices p0 and q between sets B and N . • Update tableau corresponding to this basis change. The practical revised simplex method 5 The standard simplex method (cont.) Advantages: • Easy to understand • Simple to implement The standard simplex method (cont.) Advantages: • Easy to understand • Simple to implement Disadvantages: • Expensive: the matrix Nˆ ‘usually’ treated as full. The standard simplex method (cont.) Advantages: • Easy to understand • Simple to implement Disadvantages: • Expensive: the matrix Nˆ ‘usually’ treated as full. ◦ Storage requirement: O(mn) memory locations. The standard simplex method (cont.) Advantages: • Easy to understand • Simple to implement Disadvantages: • Expensive: the matrix Nˆ ‘usually’ treated as full. ◦ Storage requirement: O(mn) memory locations. ◦ Computation requirement: O(mn) floating point operations per iteration. The standard simplex method (cont.) Advantages: • Easy to understand • Simple to implement Disadvantages: • Expensive: the matrix Nˆ ‘usually’ treated as full. ◦ Storage requirement: O(mn) memory locations. ◦ Computation requirement: O(mn) floating point operations per iteration. • Numerically unstable. The practical revised simplex method 6 Degeneracy and termination • A vertex is degenerate if at least one basic variable is zero • Degeneracy is very common in practice Degeneracy and termination • A vertex is degenerate if at least one basic variable is zero • Degeneracy is very common in practice • If there are no degenerate vertices then ◦ the objective increases strictly each iteration; ◦ the simplex method cannot return to a vertex.