Lecture 2 Geometry of LPs∗ Last time we saw that, given a (minimizing) linear program in equational form, one of the following three possibilities is true: 1. The LP is infeasible. 2. The optimal value of the LP is −∞ (i.e., the LP does not have a bounded optimum). 3. A basic feasible solution exists that achieves the optimal value. 2.1 Finding a basic feasible solution Suppose we have an LP in equational form: minf cT x j Ax = b; x ≥ 0 g; where A is an m × n matrix of rank m: Recall how we might attempt to find a BFS to this LP: Let B ⊆ [n], with jBj = m, such that AB (the set of columns of A corresponding to the elements of B) is linearly independent. (Such a set of columns exists because A has full rank.) Let N = [n] n B be the indices of the n columns of A that are not in B. Since AB is invertible, we can define a vector x 2 R by −1 xB = AB b; xN = 0: By construction, x is a basic solution. If x is also feasible (i.e., if x ≥ 0), then it is a BFS. Fact 2.1. Every LP in equational form that is feasible has a BFS. (Note that this BFS may or may not be optimal.) Proof. Pick some feasible pointx ~ 2 Rn. (In particular, sincex ~ is feasible,x ~ ≥ 0.) Let P = f j j x~j > 0 g *Lecturer: Anupam Gupta. Scribe: Brian Kell. 1 LECTURE 2. GEOMETRY OF LPS 2 be the set of coordinates ofx ~ that are nonzero. We consider two cases, depending on whether the columns of AP are linearly independent. Case 1. The columns of AP are linearly independent. Then we may extend P to a basis B, i.e., a subset P ⊆ B ⊆ [n] with jBj = m such that the columns of AB are also linearly independent. Let N = [n] n B; thenx ~N = 0 (because P ⊆ B). In addition, since −1 Ax~ = b, we have ABx~B = b, sox ~ = AB b. Sox ~ is a basic solution; since it is feasible by −1 assumption, it is a BFS. (Note, by the way, that the equationx ~ = AB b means thatx ~ is the unique solution to Ax = b having xN = 0.) Case 2. The columns of AP are linearly dependent. Let N = [n] n P . Then, by the n definition of linear dependence, there exists a nonzero vector w 2 R with wN = 0 such that AP wP = 0. For any λ 2 R, the vectorx ~ + λw satisfies A(~x + λw) = b, because A(~x + λw) = Ax~ + λAw = b + 0 = b: Becausex ~N = 0 and wN = 0, we have (~x + λw)N = 0, sox ~ + λw has no more nonzero entries thanx ~ does. Sincex ~P > 0, for sufficiently small > 0 bothx ~ + w andx ~ − w are feasible (i.e,x ~ ± w ≥ 0). Let η = supf > 0 j x~ ± w ≥ 0 g be the largest such ; then one of x~ ± ηw has one more zero coordinate thanx ~ does. We can repeat this until we find a feasible solution with no more than m nonzero coordinates, at which point Case 1 applies and we have found a BFS. (Intuitively, for sufficiently small > 0, one ofx ~ ± w is moving toward a nonnegativity constraint, that is, toward the boundary of the nonnegative orthant. When becomes just large enough that the pointx ~ ± w reaches the boundary of the nonnegative orthant, we have made one more coordinate of the point zero.) 2.2 Geometric definitions Definition 2.2. Given points x; y 2 Rn, a point z 2 Rn is a convex combination of x and y if z = λx + (1 − λ)y for some λ 2 [0; 1]. Definition 2.3. A set X ⊆ Rn is convex if the convex combination of any two points in X is also in X; that is, for all x; y 2 X and all λ 2 [0; 1], the point λx + (1 − λ)y is in X. Definition 2.4. A function f : Rn ! R is convex if for all points x; y 2 Rn and all λ 2 [0; 1] we have fλx + (1 − λ)y ≤ λf(x) + (1 − λ)f(y): Fact 2.5. If P ⊆ Rn is a convex set and f : Rn ! R is a convex function, then, for any t 2 R, the set Q = f x 2 P j f(x) ≤ t g is also convex. LECTURE 2. GEOMETRY OF LPS 3 Proof. For all x1; x2 2 Q and all λ 2 [0; 1], we have f λx1 + (1 − λ)x2 ≤ λf(x1) + (1 − λ)f(x2) ≤ λt + (1 − λ)t = t; so λx1 + (1 − λ)x2 2 Q. Fact 2.6. The intersection of two convex sets is convex. n Proof. Let P; Q ⊆ R be convex sets, and let x1; x2 2 P \ Q. Let λ 2 [0; 1]. Because x1; x2 2 P and P is convex, we have λx1 + (1 − λ)x2 2 P ; likewise, λx1 + (1 − λ)x2 2 Q. So λx1 + (1 − λ)x2 2 P \ Q. Definition 2.7. A set S ⊆ Rn is a subspace if it is closed under addition and scalar multi- plication. Equivalently, S is a subspace if S = f x 2 Rn j Ax = 0 g for some matrix A. Definition 2.8. The dimension of a subspace S ⊆ Rn, written dim(S), is the size of the largest linearly independent set of vectors contained in S. Equivalently, dim(S) = n − rank(A). 0 n 0 Definition 2.9. A set S ⊆ R is an affine subspace if S = f x0 + y j y 2 S g for some n n 0 0 subspace S ⊆ R and some vector x0 2 R . In this case the dimension of S , written dim(S ), is defined to equal the dimension of S. Equivalently, S0 is an affine subspace if S0 = f x 2 Rn j Ax = b g for some matrix A and some vector b. Definition 2.10. The dimension of a set X ⊆ Rn, written dim(X), is the dimension of the minimal affine subspace that contains X. 0 0 Note that if S1 and S2 are two affine subspaces both containing X, then their intersec- 0 0 tion S1 \ S2 is an affine subspace containing X. Hence there is a unique minimal affine subspace that contains X, so dim(X) is well defined. Equivalently, given x0 2 X, the dimension of X is the largest number k for which there exist points x1; x2; : : : ; xk 2 X such that the set fx1 − x0; x2 − x0; : : : ; xk − x0g is linearly independent. Note that the definition of the dimension of a set X agrees with the definition of the dimension of an affine subspace if X happens to be an affine subspace, and the definition of the dimension of an affine subspace S0 agrees with the definition of the dimension of a subspace if S0 happens to be a subspace. Definition 2.11. A set H ⊆ Rn is a hyperplane if H = f x 2 Rn j aT x = b g for some nonzero a 2 Rn and some b 2 R. A hyperplane is an affine subspace of dimension n − 1. Definition 2.12. A set H0 ⊆ Rn is a (closed) halfspace if H0 = f x 2 Rn j aT ≥ b g for some nonzero a 2 Rn and some b 2 R. LECTURE 2. GEOMETRY OF LPS 4 A hyperplane can be written as the intersection of two halfspaces: n T n T n T f x 2 R j a x = b g = f x 2 R j a x ≥ b g \ f x 2 R j −a x ≥ −b g: Both hyperplanes and halfspaces are convex sets. Therefore the feasible region of an LP is convex, because it is the intersection of halfspaces and hyperplanes. The dimension of the feasible region of an LP in equational form, having n variables and m linearly independent constraints (equalities), is no greater than n − m, because it is contained in the intersection of m distinct hyperplanes, each of which is an affine subspace of dimension n − 1. (The dimension of the feasible region may be less than n − m, because of the nonnegativity constraints, for instance.) For example, the region in R3 defined by x1 + x2 + x3 = 1; x ≥ 0 is a 2-dimensional triangle; here, n − m = 3 − 1 = 2. (Note, however, that if the constraint were x1 + x2 + x3 = 0, the region would have dimension 0.) 3 Figure 2.1: The region f x 2 R j x1 + x2 + x3 = 1; x ≥ 0 g. Definition 2.13. A polyhedron in Rn is the intersection of finitely many halfspaces. For example, feasible regions of LPs are polyhedra. Definition 2.14. A polytope is a bounded polyhedron, that is, a polyhedron P for which + there exists B 2 R such that kxk2 ≤ B for all x 2 P . LECTURE 2. GEOMETRY OF LPS 5 Both polyhedra and polytopes are convex. Definition 2.15. Given a polyhedron P ⊆ Rn, a point x 2 P is a vertex of P if there exists c 2 Rn such that cT x < cT y for all y 2 P , y 6= x. Supposex ^ is a vertex of a polyhedron P ⊆ Rn. Let c be as in the definition above. Take K = cT x^. Then for all y 2 P we have cT y ≥ K, so the polyhedron P is contained in the halfspace f x 2 Rn j cT x ≥ K g, i.e., P lies entirely on one side of the hyperplane f x 2 Rn j cT x = K g.