Optimization

Lecture 08 12/06/11 The Revised Assume again that we are given an LP in canonical form, which we pad with slack variables. Let us place the corresponding columns on the left of the initial tableau:

0 0 ⋯ 0

As we have argued before, after ℓ iterations of the Simplex algorithm, we get:

⋯

2 The Revised Simplex Algorithm We call this part of the tableau in the ℓ-th iteration CARRYℓ It is enough to store this part of the tableau and the ordered set of basis columns to execute the Simplex algorithm: Simplex with (1) [Pricing] Compute relative costs ̅ one at a time until we find a positive one, say , or conclude that the solution is optimal. (2) [Column Generation] Generate column X of the ℓ-th tableau by .

Determine the , say , by the ususal ratio test min ∶ or discover that the optimal value is unbounded. (3) [Pivot] Update CARRYℓ to CARRYℓ by performing the row operations determined by column when pivoting on . (4) [Update Basis] Replace the -th element of by , the index of the new basis column. 3 The Revised Simplex Algorithm Is the revised Simplex algorithm faster than the standard implementation that updates the full tableau in each step? In the worst case, the advantage is small. However, in practice two reasons speak for the revised method: 1. The pricing operation (1) only needs to be performed until we find the first positive value. If the average number of columns we need to examine is small, this is fast. 2. Pricing uses the rows of the original tableau. If the original matrix is sparse (which it frequently is in practice), these operations are fast. In some cases, we don‘t even have to generate (and store) explicitely (e.g., adjacency matrix of some graph). 4 Complexity of the Simplex Method In most practical applications, Simplex turns out to be a very efficient algorithm. But what about provable runtime guarantees? Let us first recall some geometric facts about the canonical form LP max. s.t. , where is of dimension . Here, non-negativity constraints (if present) are part of the system . If the system has full rank, then no satisfies 1 constraints with equality.

5 Complexity of the Simplex Method In the corresponding , each vertex is then the intersection of exactly hyperplanes defining the halfspaces given by the constraints. We say that is simple.

We‘ve discussed (somewhat informally), that a pivot of the Simplex algorithm corresponds to moving from one vertex of to a neighboring one via an edge. Recall that an edge is the intersection of 1 defining hyperplanes.

Algebraically, in a simple polytope a vertex is defined by constraints which we require to be tight. A pivot step corresponds to loosening one constraint and selecting another one to become tight.

6 Complexity of the Simplex Method In a simple polytope we don‘t have to worry about the issue of degeneracy and so the Simplex pivot step is well-defined by the selection of a column to enter the basis. In our geometric view, this is equivalent to selecting the tight constraint to loosen. If constraints are indexed 1, … , , Bland‘s pivot rule corresponds to loosening the tight constraint with smallest index which will lead to an improvement of the objective function value. We are now ready to construct a polytope on which the Simplex algorithm with Bland‘s pivot rule needs an expo- nential number of pivot steps.

7 Complexity of the Simplex Method We define polytope ⊂ by:

1 2 0 1

1

2 1⋮ 2 1

2 1 ⋮ 2 1 with 1/3. is a Klee-Minty cube of dimension . Constraints are indexed 1, … , 2 as shown above. Klee-Minty cubes are perturbations of -dimensional hypercubes, which are defined by 0 1 for 1, … , . 8 Complexity of the Simplex Method Theorem 11 The Simplex algorithm with Bland‘s pivot rule visits every vertex of the Klee-Minty cube when solving

(1) max. s.t. ∈ , if started at 0,0, … , 0 or (2) min. s.t. ∈ , if started at 0, … , 0,1. Thus, the Simplex algorithm with Bland‘s rule will perform 2 1 pivot steps on an LP of dimension in the worst case.

Similar lower bounds can be constructed for all the other pivot rules we have discussed.

9 Complexity of the Simplex Method One might suspect that the lower bound in Theorem 11 can be extended to hold independently of any specific pivoting rule. However, this does not work. We say the distance between a pair of vertices of polytope is the minimum number of pivots (or edges) we need to get from one to the other. The diameter of the polytope is the maximum distance between any pair of vertices. Theorem 12 For any -dimensional polytope with defining halfspaces,

.

10 Complexity of the Simplex Method For almost 50 years, it was believed that in fact it was true that . This famous was finally disproved in 2010. No tight bounds are known at this point…

So, if we can‘t prove that all pivot rules are bad, can we prove that some pivot rule is good? The answer to this is „no“. To date, we don‘t know of any deterministic pivot rule that performs a polynomial (in the size of the input represen- tation) number of pivot steps in the worst case.

However, there are theoretical results explaining the efficiency of Simplex in practice.

11 Complexity of the Simplex Method There are several results proving that Simplex performs an expected polynomial number of pivots when the input comes from some suitable .

It is generally difficult to judge whether a given probability distribution is in any way realistic . This is particularly so as we usually don‘t know all possible application scenarios of our algorithm at the time we analyze it.

A relatively recent approach to combine the benefits of worst-case and average-case analysis is what we call .

12