Linear Programming and the Simplex Method David Gale his exposition of linear programming etc. Further, each food has a cost, so among all and the simplex method is intended such adequate meals the problem is to find one as a companion piece to the article that is least costly. in this issue on the life and work of George B. Dantzig in which the impact The Transportation Problem Tand significance of this particular achievement A certain good, say steel, is available in given are described. It is now nearly sixty years since amounts at a set of m origins and is demanded in Dantzig’s original discovery [3] opened up this specified amounts at a set of n destinations. Cor- whole new area of mathematics. The subject is responding to each origin i and destination j there now widely taught throughout the world at the is the cost of shipping one unit of steel from i to level of an advanced undergraduate course. The j. Find a shipping schedule that satisfies the given pages to follow are an attempt at a capsule pre- demands from the given supplies at minimum sentation of the material that might be covered in cost. three or four lectures in such a course. Almost all linear programming applications, including these examples, can be motivated in the Linear Programming following way. There is a given set of goods. For The subject of linear programming can be defined the diet problem these are the various nutrients. quite concisely. It is concerned with the problem For the transportation problem there are m + n of maximizing or minimizing a linear function goods, these being steel at each of the origins whose variables are required to satisfy a system and destinations. There is also a set of processes of linear constraints, a constraint being a linear or activities which are specified by amounts of equation or inequality. The subject might more the goods consumed as inputs or produced as appropriately be called linear optimization. Prob- outputs. In the diet problem, for example, the lems of this sort come up in a natural and quite carrot-consuming activity c has an output of c1 elementary way in many contexts but especially units of calories, c2 units of vitamin A, etc. For the in problems of economic planning. Here are two transportation problem the transportation activ- popular examples. ity tij has as input one unit of steel at origin i and as output one unit of steel at destination j. In gen- The Diet Problem eral, an activity a is column vector whose positive A list of foods is given and the object is to pre- entries are outputs, negative entries inputs. These scribe amounts of each food so as to provide a vectors will be denoted by boldface letters. It is meal that has preassigned amounts of various nu- assumed that activity aj may be carried out at any trients such as calories, vitamins, proteins, starch, nonnegative level xj . The constraints are given by another activity vector b, the right-hand side, David Gale is professor of mathematics at the Uni- which specifies the amounts of the different goods versity of California, Berkeley. His email address is that are to be produced or consumed. For the diet [email protected]. problem it is the list of amounts of the prescribed 364 Notices of the AMS Volume 54, Number 3 1 2 j n nutrients. For the transportation problem it is the a a ··· a ··· a 1 given supplies and demands at the origins and b x11 x12 ··· x1j ··· x1n 2 destinations. Finally, associated with each activity b x21 x22 ··· x2j ··· x2n j j . a is a cost cj . Given m goods and n activities a . i the linear programming problem (LP) is then to b xi1 xi2 ··· xij ··· xin find activity levels xj that satisfy the constraints . P . and minimize the total cost cj xj . Alternatively, j m c may be thought of as the profit generated by ac- b xm1 xm2 ··· xmj ··· xmn tivity a, in which case the problem is to maximize P i rather than minimize j cj xj . where xij is the coefficient of b in the expression The simplex method is an algorithm that finds for aj as a linear combination of the bi . solutions of LPs or shows that none exist. In the In matrix terms, X is the (unique) matrix satis- exposition to follow we will treat only the special fying case where the constraints are equations and the (1) BX = A. variables are nonnegative, but the more general (Symbols A, B are used ambiguously to stand cases are easily reduced to this case. either for a matrix or the set of its columns.) It will often be useful to include the unit vec- The Simplex Method tors ei in the tableau in which case it appears as In the following paragraphs we describe the sim- shown below. plex algorithm by showing how it can be thought 1 2 j m of as a substantial generalization of standard e e ··· e ··· e 1 Gauss-Jordan elimination of ordinary linear al- b y11 y12 ··· y1j ··· y1m 2 gebra. To gain intuition as to why the algorithm b y21 y22 ··· y2j ··· y2m . works, we will refer to the linear activity model . i of the previous section. Finally we will mention b yi1 yi2 ··· y ij ··· yim some interesting discoveries in the analysis of . the algorithm, including a tantalizing unsolved . m problem. b ym1 ym2 ··· ymj ··· ymm Pivots and Tableaus a1 a2 ··· aj ··· an The basic computational step in the simplex algo- x11 x12 ··· x1j ··· x1n rithm is the same as that in most of elementary x21 x22 ··· x2j ··· x2n linear algebra, the so-called pivot operation. This . is the operation on matrices used to solve systems xi1 xi2 ··· xij ··· xin of linear equations, to put matrices in echelon . form, to evaluate determinants, etc. Given a matrix A one chooses a nonzero pivot xm1 xm2 ··· xmj ··· xmn entry aij and adds multiples of row i to the other rows so as to obtain zeros in the jth column. The Note that from (1), Y = {yij } is the solution of −1 ith row is then normalized by dividing it by aij . BY = I so Y = B . Multiplying (1) on the left by Y For solving linear equations a pivot element can gives the important equation be any nonzero entry. By contrast, the simplex method restricts the choice of pivot entry and is (2) YA = X. completely described by giving a pair of simple It is easy to verify that if one pivots on en- rules, the entrance rule that determines the pivot 0 try xij , one obtains a new matrix X that is the column j and the exit rule that determines the tableau with respect to the new basis in which aj pivot row i (in theory a third rule may be needed has replaced bi . to take care of degenerate cases). By following these rules starting from the initial data the algo- a1 a2 ··· aj ··· an 1 0 0 0 rithm arrives at the solution of the linear program b x11 x12 ··· 0 ··· x1n 2 0 0 0 in a finite number of pivots. Our purpose here is b x21 x22 ··· 0 ··· x2n to present these rules and show why they work. We shall need one other concept. aj x0 x0 ··· 1 ··· x0 Definition. The tableau X of a set of vectors i1 i2 in . A = {a1, a2,..., an} with respect to a basis . 1 2 m m 0 0 0 B = {b , b ,..., b } is the m × n matrix b xm1 xm2 ··· 0 ··· xmn March 2007 Notices of the AMS 365 The simplex method solves linear programs ui , is the value of a variable xji whose correspond- ji i by a sequence of pivots in successive tableaus, ing column a = b . Every xj whose corresponding or, equivalently, by finding a sequence of bases, column is not in the current basis has the value where each basis differs from its predecessor by a zero. single vector. Case II. In trying to replace ek there is no avail- able pivot because xkj = 0 for all j. k Solving Linear Equations (i) If in addition uk = 0, then leave e in the ba- We start by showing how to solve systems of lin- sis and go on to replace ek+1. ear equations using the language of pivots and (ii) If uk ≠ 0, then the kth row yk of Y gives tableaus. Our fundamental result is the following: a solution of (4). To see this, note that from (2) h i h i h i Theorem 1. Exactly one of the following systems Y A b = X u . Since the kth row of X is has a solution: zero, we get ykA = 0, whereas ykb = uk ≠ 0, so (3) Ax = b yk/uk solves (4). The algebra becomes considerably more com- or plicated if one requires the solutions of (3) to be (4) yT A = 0T , yT b = 1. nonnegative. In this case we have the following (Again, boldface letters are column vectors and are existence theorem (known as Farkas’s Lemma): converted to row vectors using the superscript T .) Theorem 2. Exactly one of the following systems Note that rather than giving complicated con- has a solution. ditions for (3) not to be solvable this formulation (5) Ax = b, x ≥ 0 puts the solvable and unsolvable case on an equal footing. This duality is an essential element of the or theory, as will be seen. (6) yT A ≤ 0T , yT b > 0.