Integer Programming ISE 418 Lecture 4 Dr. Ted Ralphs ISE 418 Lecture 4 1 Reading for This Lecture • N&W Sections I.4.1-I.4.3 1 ISE 418 Lecture 4 2 Linear Algebra Review: Linear Independence Definition 1. A finite collection of vectors x1; : : : ; xk 2 Rn is linearly Pk i independent if the unique solution to i=1 λix = 0 is λi = 0; i 2 [1::k]. Otherwise, the vectors are linearly dependent. Let A be a square matrix. Then, the following statements are equivalent: • The matrix A is invertible. • The matrix A> is invertible. • The determinant of A is nonzero. • The rows of A are linearly independent. • The columns of A are linearly independent. • For every vector b, the system Ax = b has a unique solution. • There exists some vector b for which the system Ax = b has a unique solution. 2 ISE 418 Lecture 4 3 Linear Algebra Review: Affine Independence Definition 2. A finite collection of vectors x1; : : : ; xk 2 Rn is affinely independent if the vectors x2 − x1; : : : ; xk − x1 2 Rn are linearly independent. • Linear independence implies affine independence, but not vice versa. • The property of linear independence is with respect to a given origin. • Affine independence is essentially a \coordinate-free" version of linear independence. Proposition 1. The following statements are equivalent: n 1. x1; : : : ; xk 2 R are affinely independent. 2. x2 − x1; : : : ; xk − x1 are linearly independent. n+1 3. (x1; 1);:::; (xk; 1) 2 R are linearly independent. 3 ISE 418 Lecture 4 4 Linear Algebra Review: Subspaces Definition 3. A nonempty subset H ⊆ Rn is called a subspace if αx+γy 2 H 8x; y 2 H and 8α; γ 2 R. Definition 4. A linear combination of a collection of vectors x1; : : : xk 2 Rn n Pk i k is any vector y 2 R such that y = i=1 λix for some λ 2 R . Definition 5. The span of a collection of vectors x1; : : : xk 2 Rn is the set of all linear combinations of those vectors. Definition 6. Given a subspace H ⊆ Rn, a collection of linearly independent vectors whose span is H is called a basis of H. The number of vectors in the basis is the dimension of the subspace. 4 ISE 418 Lecture 4 5 Linear Algebra Review: Subspaces and Bases • A given subspace has an infinite number of bases. • Each basis has the same number of vectors in it (the dimension). • If S and T are subspaces such that S ⊆ T ⊆ Rn, then a basis of S can be extended to a basis of T . • The span of the columns of a matrix A is a subspace called the column space or the range, denoted range(A). • The span of the rows of a matrix A is a subspace called the row space. • The dimensions of the column space and row space are always equal. We call this number rank(A). • Clearly, rank(A) ≤ minfm; ng. If rank(A) = minfm; ng, then A is said to have full rank. • The set fx 2 Rn j Ax = 0g is called the nullspace of A (denoted null(A)) and has dimension n − rank(A). 5 ISE 418 Lecture 4 6 Some Properties of Subspaces Proposition 2. The following are equivalent: 1. H ⊆ Rn is a subspace. 2. There is an m × n matrix A such that H = fx 2 Rn j Ax = 0g. 3. There is a k×n matrix B such that H = fx 2 Rn j x> = uB; u 2 Rkg. Proposition 3. If fx 2 Rn j Ax = bg 6= ;, the maximum number of affinely independent solutions of Ax = b is n + 1 − rank(A). Proposition 4. If H ⊆ Rn is a subspace, the subspace fx 2 Rn j x>y = 0 8 y 2 Hg is a subspace called the orthogonal subspace or orthognal complement and denoted H?. Proposition 5. If H = fx 2 Rn j Ax = 0g, with A being an m×n matrix, then H? = fx 2 Rn j x = A>u; u 2 Rmg and dim(H) + dim(H?) = n. 6 ISE 418 Lecture 4 7 Affine Spaces Definition 7. An affine combination of a collection of vectors x1; : : : xk 2 n n Pk i k R is any vector y 2 R such that y = i=1 λix for some λ 2 R with Pk j=1 λj = 1. Definition 8. A nonempty subset A ⊆ Rn is called an affine space if A is closed with respect to affine combination. Definition 9. A basis of an affine space A ⊆ Rn is a maximal set of affinely independent points of A. Definition 10. The inclusionwise minimal affine space containing a set S is called the affine hull of S, denoted aff(S). Definition 11. All bases of an affine space A have the same cardinality and this cardinality exceeds the dimension of the affine space by one. 7 ISE 418 Lecture 4 8 Projections Definition 12. If p 2 Rn and H is a subspace, the projection of p onto H is the vector q 2 H such that p − q 2 H?. • Note that this is a decomposition of a vector p into the sum of a vector in H and a vector in H?. • The projection of a set is the union of the projections of all its members. • Projections play a very important role in discrete optimization, as we will see later in the course. 8 ISE 418 Lecture 4 9 Polyhedra, Hyperplanes, and Half-spaces Definition 13. A polyhedron is a set of the form fx 2 Rn j Ax ≤ bg, where A 2 Rm×n and b 2 Rm. Definition 14. A polyhedron P ⊂ Rn is bounded if there exists a constant K such that jxij < K 8x 2 S; 8i 2 [1; n]. Definition 15. A bounded polyhedron is called a polytope. Definition 16. Let a 2 Rn and b 2 R be given. • The set fx 2 Rn j a>x = bg is called a hyperplane. • The set fx 2 Rn j a>x ≤ bg is called a half-space. 9 ISE 418 Lecture 4 10 Convex Sets Definition 17. A set S ⊆ Rn is convex if 8x; y 2 S; λ 2 [0; 1], we have λx + (1 − λ)y 2 S. 1 k n k > Definition 18. Let x ; : : : ; x 2 R and λ 2 R+ be given such that λ 1 = 1. Then Pk i 1 k 1. The vector i=1 λix is said to be a convex combination of x ; : : : ; x . 2. The convex hull of x1; : : : ; xk is the set of all convex combinations of these vectors. 1 j n j Definition 19. Let r ; : : : ; r 2 R and µ 2 R+ be given. Then Pj i 1 k 1. The vector i=1 µir is said to be a conic combination of r ; : : : ; r . 2. The conic hull of r1; : : : ; rj is the set of all conic combinations of these vectors. • The convex hull of two points is a line segment. • A set is convex if and only if for any two points in the set, the line segment joining those two points lies entirely in the set. • All polyhedra are convex. 10.