Lectures on Integer Programming
Total Page:16
File Type:pdf, Size:1020Kb
Lectures on Integer Programming L.E. Trotter, Jr. School of OR&IE Cornell University Ithaca, New York 14853 June 16, 2004 Abstract Lecture notes for the “Doctoral School in Discrete Systems Optimization,” E.P.F.- Lausanne, June 2004. This material stresses fundamental aspects of geometry and duality for Integer Programming. 1 Linear Spaces n We generally deal with IR = x = (x1,...,xn) : xj IR j , the n dimensional eu- clidean vector space over the reals{ IR. Sometimes attention∈ is restricted∀ } to−Qn, the rational n vectors. Familiarity with manipulation of vectors and matrices is assumed, but we review the− formal properties of vector spaces, stressing the manner in which the theory of linear equality systems extends naturally to linear inequality systems. Subspaces As a vector space, IRn must obey certain axioms. First, IRn is an abelian group with respect to (vector) addition: there is a zero element 0 = (0,..., 0) IRn, each element has an ∈ inverse, and vector addition is associative and commutative. 0+ a = a, a IRn ∀ ∈ n a +( a)=0, a IR (a + b)+ c = a +(− b + c), ∀ a,∈ b, c IRn a + b = b + a, ∀ a, b ∈IRn ∀ ∈ Next, IRn satisfies IR-module properties governing the action of IR on IRn via (scalar) mul- tiplication: there is a unit scalar 1 IR, scalar multiplication is associative, and scalar multiplication distributes over (both vector∈ and scalar) addition. 1a = a, a IRn ∀ ∈ n λ(µa)=(λµ)a, a IR ; λ,µ IR λ(a + b)= λa + λb, ∀ a,∈ b IRn∀; λ ∈IR ∀ ∈ n ∀ ∈ (λ + µ)a = λa + µa, a IR ; λ,µ IR ∀ ∈ ∀ ∈ Finally, IRn is closed with respect to vector addition and scalar multiplication. a + λb IRn, a, b IRn; λ IR ∈ ∀ ∈ ∀ ∈ Exercise 1.1 Show that the closure condition is equivalent to the requirement: λ a + + λ a IRn, m 1; λ ,...,λ IR; a ,...,a IRn. 2 1 1 ··· m m ∈ ∀ ≥ ∀ 1 m ∈ ∀ 1 m ∈ n The expression λ1a1 + +λmam is a linear combination of the ai. Any subset of IR obeying the above stipulations,··· i.e., containing 0 and closed under linear combinations, is a subspace. As with a1,...,am in the exercise, we often index the vectors in a set, denoting (ai)j, the jth component of ai, simply by aij. Moreover, when the set is finite, say A = a1,...,am , A may be identified with the m n matrix A IRm×n, whose ith row is (a ,...,a{ ); thus} × ∈ i1 in we write ai A to mean that ai is the ith row of matrix A. Whether we choose to view A as a set or as∈ a matrix (an ordered set of vectors) will always be clear from context. When matrix–vector products are involved, it will also be clear from context whether row or column vectors are intended, enabling us (usually) to avoid the use of transpose notation. 1 When all λi = 0, the linear combination λ1a1 + + λmam is trivial; otherwise, it is nontriv- ial. Of course, the result of the trivial linear combination··· is just the zero vector. And when 0 results from a nontrivial linear combination, the a1,...,am are linearly dependent. This terminology extends to sets: S IRn is linearly dependent when some finite subset, consist- ing of distinct elements of S, satisfies⊆ a linear dependence relation λ a + + λ a = 0. 1 1 ··· m m Similarly, a / S is linearly dependent on S provided a and otherwise only (distinct) members of S satisfy a∈ linear dependence relation in which the coefficient of a is nonzero; of course, we may scale so that the coefficient of a is 1 and then, provided S = , rewrite the dependence − 6 ∅ relation as an equation expressing a as a linear combination of members of S. The linear span of S IRn is L(S)= S a IRn : a is linearly dependent on S ; when T S L(T ), we say that⊆ T spans S. In particular,∪{ ∈ observe that L( )= 0 , as 0 is} linearly dependent⊆ ⊆ on ∅ { } any set, even ; thus is not a subspace. ∅ ∅ Exercise 1.2 Show that: (i) S T L(S) L(T ); (ii) S⊆ L(⇒S); ⊆ (iii) L⊆(L(S)) = L(S); (iv) S = L(S) S is a subspace. 2 ⇔ For any S IRn, it follows from the exercise that L(S) is a subspace containing S; we say that L(S) is⊆ the subspace generated by S. On the other hand, the intersection of all subspaces containing S, or equivalently, the unique minimal subspace containing S, is called the linear hull of S. Now when T is a subspace and T S, (i) and (iv) imply L(S) L(T )= T , and ⊇ ⊆ consequently L(S) also gives the linear hull of S. Thus we have two ways to think about any subspace: an interior (generator) description provided by the linear span and an exterior (constraint) description in terms of the linear hull. Below we turn these dual geometric descriptions into finite and equivalent representations for any subspace. Finite Generator Representations For matrix A, L(A) is finitely generated by the rows of A; in this case L(A) is called the row space of A. Exercise 1.3 Consider the following elementary row operations on rows ai, ak of matrix A: (i) interchange a a ; i ↔ k (ii) replace a a + a ; k ← k i (iii) replace ai λai, 0 = λ IR. Show that these operations← leave6 ∈ the row space of A unchanged. 2 We now show that every subspace has a finite set of generators. Any set of vectors which is not linearly dependent is linearly independent; in particular, the empty set is linearly ∅ independent. 2 m Proposition 1.4 Let a0 = i=1 λiai, with a1,...,am linearly independent. Then: (i) the λi are unique; P (ii) a :0 i = k is linearly independent λ =0; { i ≤ 6 } ⇔ k 6 (iii) L( a ,...,a )= L( a :0 i = k λ =0. { 1 m} { i ≤ 6 } ⇔ k 6 Proof: (i) If a = m λ a = m µ a , then 0= a a = m (λ µ )a . 0 i=1 i i i=1 i i 0 − 0 i=1 i − i i Thus linear independence of a ,...,a implies λ µ =0 i; hence λ = µ i. P P1 m i − i ∀P i i ∀ (ii) λk =0 0= a0 + i6=k λiai ai :0 i = k linearly dependent. For the converse,⇒ −a :0 i = k linearly⇒{ dependent≤ 6 } µ a = 0, and some µ = 0. { i ≤P 6 } ⇒ 0≤i6=k i i i 6 Moreover, µ0 = 0, since a1,...,am are linearly independent.P Thus a = 6 ( µ /µ )a = m λ a , so part (i) implies λ = 0. 0 1≤i6=k − i 0 i i=1 i i k (iii) First note that a L( a ,...,a ), hence L( a ,...,a )= L( a ,...,a ). P 0 ∈ { 1 P m} { 0 m} { 1 m} Now λk =0 ak = (1/λk)a0 1≤i6=k(λi/λk)ai ak L( ai : i = k )= L( a0,...,am ). If λ = 0,6 then⇒ a L( a :1 − i = k )= L( a :0⇒ i∈= k {). 6 } { } k 0 ∈ { i ≤ P6 } { i ≤ 6 } But linear independence of a ,...,a a L( a :1 i = k ) = L( a ,...,a ). 2 1 m ⇒ k 6∈ { i ≤ 6 } 6 { 1 m} Proposition 1.5 For subspace S, with a ,...,a = A S and b ,...,b = B S, { 1 m} ⊆ { 1 n} ⊆ if A is linearly independent and B spans S, then m n. ≤ Proof: Considering the a sequentially for 1 i m, if a B, we do nothing. i ≤ ≤ i ∈ If a B, then a L(B), so a = n λ b with b : λ =0 linearly independent. i 6∈ i ∈ i j=1 j j { j j 6 } For some λj = 0, we must have bj P a1,...,am , and we replace bj ai in B. Proposition 1.46 implies that this replacement6∈ { does} not change L(B)=←S. After m steps, a ,...,a b ,...,b and it follows that m n. 2 { 1 m}⊆{ 1 n} ≤ Thus the maximum number of linearly independent elements of a subspace is no larger than the minimum number of its elements needed to span. Since the unit vectors e1,...,en (i.e., n eij = 1 for i = j and eij = 0 for i = j) span IR , any linearly independent set of vectors in IRn, say a ,...,a , must satisfy6 m n. { 1 m} ≤ Exercise 1.6 For A IRm×n with m < n, show that Ax =0 has a solution x =0. 2 ∈ { } 6 A basis for a subspace S is a maximal linearly independent subset of S, i.e., a linearly independent set that is properly contained in no linearly independent subset of S. If B is a basis for S, then L(B) L(S) = S by (1.2), and maximality of B with respect to ⊆ linear independence implies that every other element of S is linearly dependent on B, hence S L(B). Thus L(B)= S; i.e., B spans S. Since a basis is both independent and spanning, we⊆ are able to strengthen the max-min relation observed following Proposition 1.5 above: in any subspace, the maximum size of an independent set equals the minimum size of a spanning set. Moreover, if B1 and B2 are bases, then B1 B2 , since B1 is independent and B2 is spanning; symmetrically, B B . It therefore| |≤| follows| that all bases of a subspace | 2|≤| 1| are the same size, i.e., are equicardinal. We summarize these observations in the following fundamental theorem of linear algebra, the Finite Basis Theorem, stating that any subspace can be represented as the set of all linear combinations of the finite set of generators given by any of its bases, i.e., is finitely generated by the elements of any basis.