DM559 Linear and Integer Programming Lecture 7 Vector Spaces Linear Independence, Bases and Dimension Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark Vector Spaces and Subspaces Linear independence Outline Bases and Dimension 1. Vector Spaces and Subspaces 2. Linear independence 3. Bases and Dimension 5 Vector Spaces and Subspaces Linear independence Outline Bases and Dimension 1. Vector Spaces and Subspaces 2. Linear independence 3. Bases and Dimension 6 Vector Spaces and Subspaces Linear independence Premise Bases and Dimension • We move to a higher level of abstraction • A vector space is a set with an addition and scalar multiplication that behave appropriately, that is, like Rn • Imagine a vector space as a class of a generic type (template) in object oriented programming, equipped with two operations. 7 Vector Spaces and Subspaces Linear independence Vector Spaces Bases and Dimension Definition (Vector Space) A (real) vector space V is a non-empty set equipped with an addition and a scalar multiplication operation such that for all α; β 2 R and all u; v; w 2 V : 1. u + v 2 V (closure under addition) 2. u + v = v + u (commutative law for addition) 3. u + (v + w) = (u + v) + w (associative law for addition) 4. there is a single member 0 of V , called the zero vector, such that for all v 2 V ; v + 0 = v 5. for every v 2 V there is an element w 2 V , written −v, called the negative of v, such that v + w = 0 6. αv 2 V (closure under scalar multiplication) 7. α(u + v) = αu + αv (distributive law) 8. (α + β)v = αv + βv (distributive law) 9. α(βv) = (αβ)v (associative law for vector multiplication) 10. 1v = v 8 Vector Spaces and Subspaces Linear independence Examples Bases and Dimension • set Rn • but the set of objects for which the vector space defined is valid are more than the vectors in Rn. • set of all functions F : R ! R. We can define an addition f + g: (f + g)(x) = f (x) + g(x) and a scalar multiplication αf : (αf )(x) = αf (x) • Example: x + x 2 and2 x. They can represent the result of the two operations. • What is −f ? and the zero vector? 9 Vector Spaces and Subspaces Linear independence Bases and Dimension The axioms given are minimum number needed. Other properties can be derived: For example: (−1)x = −x Proof: 0 = 0x = (1 + (−1))x = 1x + (−1)x = x + (−1)x Adding −x on both sides: − x = − x + 0 = −x + x + (−1)x = (−1)x which proves that −x = (−1)x. Try the same with −f . 10 Vector Spaces and Subspaces Linear independence Examples Bases and Dimension • V = f0g • the set of all m × n matrices • the set of all infinite sequences of real numbers, y = fy1; y2;:::; yn;:::; g; yi 2 R. (y = fyng, n ≥ 1) – addition of y = fy1; y2;:::; yn;:::; g and z = fz1; z2;:::; zn;:::; g then: y + z = fy1 + z1; y2 + z2;:::; yn + zn;:::; g – multiplication by a scalar α 2 R: αy = fαy1; αy2; : : : ; αyn;:::; g • set of all vectors in R3 with the third entry equal to 0 (verify closure): 82 3 9 x < = W = 4y5 x; y 2 R : 0 ; 11 Vector Spaces and Subspaces Linear independence Linear Combinations Bases and Dimension Definition (Linear Combination) For vectors v1; v2;:::; vk in a vector space V , the vector v = α1v1 + α2v2 + ::: + αk vk is called a linear combination of the vectors v1; v2;:::; vk . The scalars αi are called coefficients. • To find the coefficients that given a set of vertices express by linear combination a given vector, we solve a system of linear equations. • If F is the vector space of functions from R to R then the function f : x 7! 2x 2 + 3x + 4 can be expressed as a linear combination of: g : x 7! x 2, h : x 7! x, k : x 7! 1 that is: f = 2g + 3h + 4k • Given two vectors v1 and v2, is it possible to represent any point in the Cartesian plane? 12 Vector Spaces and Subspaces Linear independence Subspaces Bases and Dimension Definition (Subspace) A subspace W of a vector space V is a non-empty subset of V that is itself a vector space under the same operations of addition and scalar multiplication as V . Theorem Let V be a vector space. Then a non-empty subset W of V is a subspace if and only if both the following hold: • for all u; v 2 W , u + v 2 W (W is closed under addition) • for all v 2 W and α 2 R, αv 2 W (W is closed under scalar multiplication) ie, all other axioms can be derived to hold true 13 Vector Spaces and Subspaces Linear independence Bases and Dimension Example • The set of all vectors in R3 with the third entry equal to 0. • The set f0g is not empty, it is a subspace since 0 + 0 = 0 and α0 = 0 for any α 2 R. Example In R2, the lines y = 2x and y = 2x + 1 can be defined as the sets of vectors: x x S = y = 2x; x 2 R U = y = 2x + 1; x 2 R y y S = fx j x = tv; t 2 Rg U = fx j x = p + tv; t 2 Rg 1 0 v = ; p = 2 1 14 Vector Spaces and Subspaces Linear independence Bases and Dimension Example (cntd) 1. The set S is non-empty, since 0 = 0v 2 S. 2. closure under addition: 1 1 u = s 2 S; w = t 2 S; for some s; t 2 2 2 R u + w = sv + tv = (s + t)v 2 S since s + t 2 R 3. closure under scalar multiplication: 1 u = s 2 S for some s 2 ; α 2 2 R R αu = α(s(v)) = (αs)v 2 S since αs 2 R Note that: • u; w and α 2 R must be arbitrary 15 Vector Spaces and Subspaces Linear independence Bases and Dimension Example (cntd) 1. 0 62 U 2. U is not closed under addition: 0 1 0 1 1 2 U; 2 U but + = 62 U 1 3 1 3 4 3. U is not closed under scalar multiplication 0 0 0 2 U; 2 2 but 2 = 62 U 1 R 1 2 Note that: • proving just one of the above couterexamples is enough to show that U is not a subspace • it is sufficient to make them fail for particular choices • a good place to start is checking whether 0 2 S. If not then S is not a subspace 16 Vector Spaces and Subspaces Linear independence Bases and Dimension Theorem A non-empty subset W of a vector space is a subspace if and only if for all u; v 2 W and all α; β 2 R, we have αu + βv 2 W . That is, W is closed under linear combination. 17 Vector Spaces and Subspaces Linear independence Bases and Dimension Geometric interpretation: y y u w w u x x (0; 0) (0; 0) The line y = 2x + 1 is an affine subset, a „translation“ of a subspace 18 Vector Spaces and Subspaces Linear independence Null space of a Matrix is a Subspace Bases and Dimension Theorem For any m × n matrix A, N(A), ie, the solutions of Ax = 0, is a subspace of Rn Proof 1. A0 = 0 =) 0 2 N(A) 2. Suppose u; v 2 N(A), then u + v 2 N(A): A(u + v) = Au + Av = 0 + 0 = 0 3. Suppose u 2 N(A) and α 2 R, then αu 2 N(A): A(αu) = A(αu) = αAu = α0 = 0 The set of solutions S to a general system Ax = b is nota subspace of Rn because 0 62 S 19 Vector Spaces and Subspaces Linear independence Affine subsets Bases and Dimension Definition (Affine subset) If W is a subspace of a vector space V and x 2 V , then the set x + W defined by x + W = fx + w j w 2 W g is said to be an affine subset of V . The set of solutions S to a general system Ax = b is an affine subspace, indeed recall that if x0 is any solution of the system S = fx0 + z j z 2 N(A)g 20 Vector Spaces and Subspaces Linear independence Range of a Matrix is a Subspace Bases and Dimension Theorem For any m × n matrix A, R(A) = fAx j x 2 Rng is a subspace of Rm Proof 1. A0 = 0 =) 0 2 R(A) 2. Suppose u; v 2 R(A), then u + v 2 R(A): ... 3. Suppose u 2 R(A) and α 2 R, then αu 2 R(A): ... 21 Vector Spaces and Subspaces Linear independence Linear Span Bases and Dimension • If v = α1v1 + α2v2 + ::: + αk vk and w = β1v1 + β2v2 + ::: + βk vk , then v + w and sv; s 2 R are also linear combinations of the vectors v1; v2;:::; vk . • The set of all linear combinations of a given set of vectors of a vector space V forms a subspace: Definition (Linear span) Let V be a vector space and v1; v2;:::; vk 2 V . The linear span of X = fv1; v2;:::; vk g is the set of all linear combinations of the vectors v1; v2;:::; vk , denoted by Lin(X ), that is: Lin(fv1; v2;:::; vk g) = fα1v1 + α2v2 + ::: + αk vk j α1; α2; : : : ; αk 2 Rg Theorem If X = fv1; v2;:::; vk g is a set of vectors of a vector space V , then Lin(X ) is a subspace of V and is also called the subspace spanned by X .