3. Vector Spaces: Linear Combination, Span, Linear Dependence And

Home , Linear combination

3. Vector spaces: Linear combination, span, linear dependenceanddependence See section 4.1 - 4.2 of the textbook for definition and first examples of a vector space and subspace. In the following discussion, V is a real vector space and 0 denotes the zero element of V . Definition 3.1. Let S be a nonempty subset of V .

◦ A linear combination of vectors in S is a vector of the form (c1v1+···cnvn)wherev1, ··· ,vn ∈ S and c1, ··· ,cn ∈ R. ◦ We say that such a linear combination is nontrivial, if some ci ̸=0. ◦ The span of S is deﬁned to be set of all linear combinations of elements of S.Thespanof S is denoted by span(S). We deﬁne the span of the empty set to be {0}. ◦ We say S is linearly dependent if a nontrivial linear combination of distinct elements of S is equal to the zero vector. ◦ We say that S is a linearly independent set if S is not linearly dependent. Remark 3.2. Let S be a nonempty subset of V .Avectorv ∈ V is a linear combination of elements of S if and only if v can be written in the form

v = c1v1 + ···+ cnvn where v1, ··· ,vn ∈ S and c1, ··· ,cn ∈ R. Let u1, ··· ,un ∈ V .Ifv ∈ V is a linear combination of elements of {u1, ··· ,un},thenwesimply say that v is a linear combination of v1, ··· ,vn. Example 3.3. (1) Consider the vector space R3.Let

v1 =(0, 1, −1),v2 =(−1, 0, 1),v3 =(1, −1, 0),v4 =(3, 2, −5),v5 =(1, 1, 1).

◦ Here v4 is a linear combination of v1,v2 because we can write v4 =2v1 − 3v2. ◦ Exercise: Verify that v4 is a linear combination of v1,v3. ◦ Verify that v4 is a linear combination of v1,v2,v3,v5. ◦ Exercise: Show that v5 is not a linear combination of v1,v2,v3. ◦ Exercise: Explain geometrically what the above two exercises are saying about the vectors. (2) Consider the vector space V consisting of all continuous functions on R.Considerthe following elements of V : 2 2 f1(x)=x sin x, f2(x)=2x, f3(x)=1,f4(x)=3− x cos x.

Show that f4 is a linear combination of f1,f2,f3. (3) Exercise: Show that in example 1 above, the span of v1,v2 consists of all the vectors in 3 the plane S = {(x, y, z) ∈ R : x + y + z =0}.Verifythatv3 and v4 lie in S but v5 does not. What is the span of v1,v2,v3,v4?

Theorem 3.4. Let v1, ··· ,vn be elements of V and let S be the span of v1, ··· ,vn. Then S is the smallest subspace of V that contains v1, ··· ,vn. Definition 3.5. Let S be a subset of V .WesaythatS spans V or that S is a spanning set for V if span(S)=V .LetA be a subset of V .WesaythatA is a minimal spanning set for V if A spans V and no proper subset of A span V . Definition 3.6. Let A be an m × n matrix. The span of the rows of A is a subspace of Rn.This subspace is called the row space of A .SotherowspaceofA consists of all linear combinations of the rows of A.Similarlyonedefinesthe column space of A. Exercise: If B is a matrix obtained from A by performing a finite sequence of row operations, then row space of B is equal to the row space of A. 4 Suppose A is a m × n matrix. Often, we can use the above exercise to write down a simple set of vectors spanning row space of A.ApplyGauss-JordanalgorithmtoconvertA to a reduced row echelon matrix B.Nowthenon-zerorowsofB give a set of vectors whose span is row space of A (by the above exercise). 3.7. Dependence relation: Let S be a non-empty subset of V .recallthatS is linearly dependent if and only if there is some relation of the form

c1s1 + ···+ ···cnsn =0 where n ≥ 1issomenaturalnumber,s1, ··· ,sn are distinct elements of S,andc1, ··· ,cn are real numbers at least one of which is nonzero. A relation as above iscalledanontrivial dependence relation among elements of S.SoS is linearly dependence if and only if there exists a nontrivial dependent relation among elements of S.Notethat,withoutlossofgenerality,wemayassumethat in the above dependence relation each cj ̸=0.Simplydropthethosetermswherethecoeﬃcients are equal to 0. Note also that if T ⊆ S and T is linearly dependent, then so is S,sinceanontrivialdependence relation between the elements of T is also a nontrivial dependence relation between the elements of S. There are several alternative ways of stating the deﬁnition of linear independence. We collect some of these in the following exercise: 3.8. Exercise: Let V be a vector space and let S be a subset of V . Then the following are equivalent: (1) S is linearly independent. (2) If T is a proper subset of S,thenthereexistsv ∈ S such that v/∈ span(T ). (3) There is no vector v ∈ S such that v ∈ span(S −{v}). (4) every element of span(S) can be uniquely written as a linear combination of elements of S. Proof. Let’s prove (1) implies (2). Assume (1). Let T be a proper subset of S.ThenS − T is non-empty. Pick v ∈ (S − T ). If uppose v ∈ Span(T ), then that would mean v = c1t1 + ···cntn for some cj ∈ R and tj ∈ T which would give us a nontrivial dependence relation

v +(−c1)t1 +(−c2)t2 + ···+(−cn)tn =0 among elements of S contradicting the linear independence of S.Sov/∈ Span(S). Notice that the dependence relation is nontrivial because the coeﬃcient of v is equal to 1 and all the tj’s are distinct from v since v/∈ T while t1, ··· ,tn ∈ T . Let’s prove (2) implies (3). Assume (2). Let v be any element of S.LetT = S −{v}.Then(2) implies that v/∈ Span(T )=Span(S −{v}). Let’s prove (3) implies (4). Assume (3). Let v ∈ Span(S). If possible, suppose v is written as alinearcombinationofelementsofS in two ways. Let s1, ··· ,sn be all the distinct elements of S that occur while writing these two linear combinations. Thenwehavetwoexpressionsforv of the n n n form v = j=1 cjsj = j=1 djsj for some scalars c1, ··· ,cn,d1, ··· ,dn.So j=1(cj − dj)sj =0. ! ! −1 ! If possible suppose ci ̸= di for some i.Thensi =(ci − di) (dj − dj)sj proving that si ∈ !j≠ i Span(S −{si}), thereby contradicting (3). So we must have cj = dj for all j.Thuswehaveargued that every element of span(S)canbewrittenasacombinationofelementsofS only in one way. Let’s prove (4) implies (1). Assume (4). Suppose S is linearly dependent, then there exists a nontrivial dependence relation among the elements of S of the form c1s1 + ···+ cnsn =0where n ≥ 1, s1, ··· ,sn are distinct elements of S and c1, ··· ,cn are real number, at least one of which is nonzero. Pick a j such that cj ̸=0.Thenwehave

(−cj)sj = c1s1 + ···+ cj−1sj−1 + cj+1sj+1 + ···+ cnsn 5 Since s1, ··· ,sn are distinct, the above equation shows that the element (−cj)sj ∈ span(S)canbe written as a linear combination of distinct elements of S in more than one way, thus contradicting (4). ! we get the following characterization of linear independence of ﬁnite sets from the third condition above: Asubset{v1, ··· ,vn} is linearly independent if and only if v1 ̸=0and vj ∈/ n span{v1, ··· ,vj−1} for all j =2, 3, ··· ,n.Letu1, ··· ,um ∈ R .

Proof. If S = {v1, ··· ,vn} is linearly independent, then the condition holds, (use (1) implies (3) of the lemma above). Suppose now S is linearly dependent. Then either v1 =0,orbylookingat anontrivialdependencerelationbetweenelementsofS we get vj ∈/ span{v1, ··· ,vj−1} for some j. !

This characterization lets us check if {u1, ··· ,un} are linearly independent. Write down a matrix A whose rows are u1, ··· ,un.ConvertA to a matridx B in row echelon form. If every row of B has a nonzero pivot then u1, ··· ,un are linearly independent. Otherwise not. 3.9. AfewExercises (1) Let V be a vector space and let S be a nonempty subset of V . (a) Let W be any subspace of V such that S ⊆ W .Showthatspan(S) ⊆ W . (b)Conclude that span(S)isthesmallestsubspaceofV that contains S. (2) Let V be the vector space of all infinitely differentiable functions on R.Letv ∈ V be the element v(x)=x2ex.LetW =span{v, v′,v′′, ··· ,v(n), ···} where v(n) denotes the n-th derivative of v.Findthreeelementsu1,u2,u3 ∈ V such that W =span{u1,u2,u3}. (3) Let V be the vector space in the previous problem and let n be any natural number. Show that V cannot be spanned by any n elements of V . (4) Let A be a m × n matrix. Recall that the row space of S is defined to be the span of the rows of A.LetR(A)denotetherowspaceofA. (a) If B is any k × m matrix, then show that R(BA) ⊆R(A). (b) If B is an invertible k × m matrix, then show that R(BA)=R(A). (c) Let C be a matrix obtained from A by performing finitely many row operations. Show that R(C)=R(A). (d) Let A be the following matrix: 100 0 ⎛201 2⎞ A = ⎜320−2⎟ ⎜ ⎟ ⎜433 3⎟ ⎜ ⎟ ⎝754 3⎠ 4 Find v1,v2,v3 ∈ R such that R(A)=span{v1,v2,v3}.