LINEAR SUBSPACES 1. Subspaces of Rn We wish to generalize the notion of lines and planes. To that end, say a subset W ⊂ Rn is a (linear) subspace if it has the following three properties: (1) (Non-empty): ~0 2 W ; (2) (Closed under addition): ~v1;~v2 2 W ) ~v1 + ~v2 2 W ; (3) (Closed under scaling): ~v 2 W and k 2 R ) k~v 2 W . n o EXAMPLE: ~0 and Rn are subspaces of Rn for any n ≥ 1. 1 EXAMPLE: Line t : t 2 given by x = x is a subspace. 1 R 1 2 x 1 1 NON-EXAMPLE: W = : x ≥ 0; y ≥ 0 . As 2 W; but − 62 W: y 1 1 EXAMPLE: If T : Rm ! Rn, then ker(T ) is a subspace and Rm and Im (T ) is a subspace of Rn. For instance, (1) T (~0) = ~0 ) ~0 2 ker(T ); (2) ~v1;~v2 2 ker(T ) ) T (~v1 + ~v2) = T (~v1) + T (~v2) = ~0 ) ~v1 + ~v2 2 ker(T ); (3) ~v 2 ker(T ); k 2 R ! T (k~v) = kT (~v) = k~0 = ~0 ) k~v 2 ker(T ): 2. Span of vectors n Given a set of vectors ~v1; : : : ;~vm 2 R we define the span of these vectors to be W = span(~v1; : : : ;~vk) = fc1~v1 + ::: + ck~vk : c1; : : : ; cm 2 Rg In other words, ~w 2 W means ~w is a linear combination of the ~vi. If A = ~v1 j · · · j ~vm is the n × m matrix with columns ~vi, then span(~v1; : : : ;~vk) = Im (A) n and so span(~v1; : : : ;~vk) is a subspace of R . Thought of the other way around, the image of A is the span of its columns. n n Given a subspace W ⊂ R we say a set of vectors ~v1; : : : ;~vm 2 R span W if W = span(~v1; : : : ;~vm): n o EXAMPLE: span(~0) = ~0 . 2 2 EXAMPLE: If ~e1;~e2 are standard vectors in R , then span(~e1;~e2) = R . 223 02 1 3 2031 223 02 1 3 2031 EXAMPLE: 435 2 span @4 0 5 ; 415A and 415 62 span @4 0 5 ; 415A : 1 −1 1 0 −1 1 2 1 0 j 23 21 0 j 23 rref 4 0 1 j 35 = 40 1 j 35 −1 1 j 1 0 0 j 0 1 2 LINEAR SUBSPACES so 223 2 1 3 203 435 = 2 4 0 5 + 3 415 : 1 −1 1 Next compute 2 1 0 j 23 21 0 j 03 rref 4 0 1 j 15 = 40 1 j 05 −1 1 j 0 0 0 j 1 As this is inconsistent, and so there is no way to write: 223 2 1 3 203 415 = c1 4 0 5 + c2 415 0 −1 1 . 3. Linear independence Many different sets of vectors may span the same subspace. For instance, W = span(~v1;~v2;~v1 + ~v2) = span(~v1;~v2): Indeed, c1~v1 +c2~v2 +c3(~v1 +~v2) = (c1 +c3)~v1 +(c1 +c3)~v2. In other words, ~v1 +~v2 is redundant and is not needed to describe the subspace W . To formalize this, define a linear relation among ~v1; : : : ;~vm to be an equation of the form c1~v1 + ··· + cm~vm = 0: There is always such a solution with c1 = ··· cm = 0. A non-trivial relation is one in which at least one ci 6= 0 EXAMPLE: 1 1 −1 2 − 3 − = ~0 1 0 2 is a non-trivial linear relation amongst three vectors and this can be rewritten 1 2 1 1 −1 = − : 0 3 1 3 2 EXAMPLE: Suppose ~v1; : : : ;~vm admit a non-trivial relation c1~v1 + ··· + cm~vm = 0 with cm 6= 0, then span(~v1; ··· ;~vm−1) = span(~v1; ··· ;~vm). That is the vector ~vm is redundant and can be omitted. If A = ~v1 j · · · j ~vm is the matrix with columns the vectors ~vi, then a linear relation can be identified with a unique element of ker(A). Indeed, 2 3 c1 ~ 6 . 7 c1~v1 + ::: + cm~vm = 0 () 4 . 5 2 ker(A): cm Furthermore, a non-trivial relation corresponds to a non-zero entry of ker(A). We say ~v ; : : : ;~v are linearly independent if they have no non-trivial relation, 1 mn o n o that is, if ker(A) = ~0 . Put another way, ker(A) = ~0 if and only if the columns of A are linearly independent. LINEAR SUBSPACES 3 EXAMPLE: If ~v1; : : : ;~vm (m ≥ 2) are linearly independent, then span(~v1; : : : ;~vm−1) ( span(~v1; : : : ;~vm): That is, there is no redundancy for linearly independent sets of vectors. More generally none of the vectors can be omitted without making the span smaller. EXAMPLE: (m = 1) ~v1 is linear independent if and only if ~v1 6= 0. EXAMPLE: ~v1;~v2;~v1 + ~v2 are never linearly independent (c1 = c2 = 1; c3 = −1 gives a non-trivial relation). 2 EXAMPLE: ~e1;~e2 are linearly independent in R . Indeed, if c1 0 c1~e1 + c2~e2 = ~0 () = c2 0 so c1 = c2 = 0. That is, the only linear relation is the trivial one. EXAMPLE: Suppose span(~v1;~v2;~v3) = span(~v1 + ~v2;~v1 − ~v2), then ~v1;~v2;~v3 are not linearly independent. Indeed, ~v3 = c1(~v1 + ~v2) + c2(~v1 − ~v2) which gives the non-trivial linear relation −(c1 + c2)~v1 − (c1 − c2)~v2 + ~v3 = ~0: 213 2−23 2−13 EXAMPLE: Are 415 ; 4−15 ; 4 1 5 linearly independent? First compute 0 1 2 21 −2 −13 21 0 33 rref 41 −1 1 5 = 40 1 25 : 0 1 2 0 0 0 This means that the kernel of this matrix is 02−331 span @4−25A 1 That is, there is have non-trivial linear relation 213 2−23 2−13 −3 415 − 2 4−15 + 4 1 5 = ~0 0 1 2 and so the vectors are not linearly independent. 213 2−13 213 EXAMPLE: Are 405 ; 4 1 5 ; 405 linearly independent? Compute 2 0 1 21 −1 13 n o ker 40 1 05 = ~0 2 0 1 by showing that 21 −1 13 rref 40 1 05 = I3: 2 0 1 4 LINEAR SUBSPACES 4. Basis of a subspace A set of vectors which span a subspace W and which does not have any redun- dancies is clearly of particular interest. With this in mind for a subspace W ⊂ Rn, we say a set f~v1; : : : ;~vmg is a basis of W if (1) ~v1; : : : ;~vm are linearly independent (2) W = span(~v1; : : : ;~vm). This ensures that not only do the vectors span the subspace but none of them can be omitted. 2 EXAMPLE: ~e1;~e2 is a basis of R . Indeed, any vector x1 ~x = = x1~e1 + ~e2 x2 so the vectors span. They are linearly indepdent by inspection. 2 NON-EXAMPLE: ~e1 +~e2 is not a basis of R as it does not span (but is linearly independent). 2 NON-EXAMPLE: ~e1;~e2;~e1 + ~e2 is not a basis of R as the set is not linearly independent (but does span). n o EXAMPLE: The subspace ~0 has no basis. Say its basis is the empty set, ;. 1 0 −1 2 EXAMPLE: Find basis of ker . Observe this matrix is already in 0 1 1 0 RREF so it is easy to determine elements in the kernel. Indeed, there are two free variables f1 = x3 and f2 = x4. Plugging in specific values for these variables (i.e., solving for the pivot variables) gives the following two elements in the kernel: 2 1 3 2−23 6−17 6 0 7 ~v1 = 6 7 (f1 = x3 = 1; f2 = x4 = 0) and ~v2 = 6 7 (f1 = x3 = 0; f2 = x4 = 1) 4 1 5 4 0 5 0 1 Clearly, if f1 = s and f2 = t, then a general element of the kernel is of the form ~z = s~v1 + t~v2. In other words, the kernel is spanned by ~v1 and ~v2. Hence, we just need to check that ~v1;~v2 are linearly independent to see they are a basis. However, 2 3 c1 − 2c2 6 −c1 7 ~0 = c1~v1 + c2~v2 = 6 7 : 4 c1 5 c2 Hence, we must have c1 = 0 (from the third entry) and c2 = 0 (from the fourth entry). That is the only linear relation is the trivial one and so the vectors are linearly independent and so form a basis of the kernel. This example can be generalized to give a procedure for finding the basis of ker(A) for any matrix A. A basic observation is that ker(A) = ker(rref(A)). This is becaus the linear system with augmented matrix [Aj~0] has the same solutions as that of rref[Aj~0] = [rref(A)j~0]. With that in mind: (1) Compute rref(A) and use rref(A) to determine free variables. (2) Label the free variables as f1; : : : ; fp. (3) Let ~vi be solutions corresponding to fi = 1 and fj = 0 for j 6= i. That is, use rref(A) to solve for the pivot variables in terms of the specified values of the free variables. LINEAR SUBSPACES 5 (4) Basis of ker(A) is ~v1; : : : ;~vp. In order to justify this first observe that ker(A) = span(~v1; : : : ;~vp). As the element of ker(A) corresponding to f1 = t1; : : : ; fp = tp is t1~v1 + ::: + tp~vp 2 ker(A). Similarly, only ~vj has a non-zero entry at row corresponding to fj, so easy to see ~v1; : : : ;~vp are linearly independent and hence form a basis. EXAMPLE: Find a basis of Im (A) for 21 3 −13 A = 40 −2 1 5 = ~w1 j ~w2 j ~w3 : 2 0 1 We compute 2 1 3 1 0 2 1 rref(A) = 40 1 − 2 5 0 0 0 Hence, 02 1 31 02 31 − 2 −1 1 ker(A) = span @4 2 5A = span @4 1 5A 1 2 This means that there is a linear relation amongst the columns of A: 213 2 3 3 2−13 1 1 1 1 − ~w + ~w + ~w = − 0 + −2 + 2 1 = ~0 () ~w = ~w − ~w : 2 1 2 2 3 4 5 4 5 4 5 3 2 1 2 2 2 0 1 This shows that ~w3 2 span(~w1; ~w2).