Lecture 20: 5.5 Row Space, Column Space, and Nullspace

Wei-Ta Chu

2008/12/3 Row Space and Column Space

 Definition n  If A is an mn matrix, then the subspace of R spanned by the row vectors of A is called the row space (列空間) of A, and the subspace of Rm spanned by the column vectors is called the column space (行空間) of A.

 The solution space of the homogeneous system of equation Ax = 0, which is a subspace of Rn, is called the nullspace ( 零核空間) of A.

2008/12/3 Elementary Linear Algebra 2 Remarks

 In this section we will be concerned with two questions

 What relationships exist between the solutions of a linear system Ax=b and the row space, column space, and nullspace of A.

 What relationships exist among the row space, column space, and nullspace of a matrix.

2008/12/3 Elementary Linear Algebra 3 Remarks

 It follows from Formula(10) of Section 1.3

 We conclude that Ax=b is consistent if and only if b is expressible as a linear combination of the column vectors of A or, equivalently, if and only if b is in the column space of A.

2008/12/3 Elementary Linear Algebra 4 Theorem 5.5.1

 Theorem 5.5.1

 A system of linear equations Ax = b is consistent if and only if b is in the column space of A.

2008/12/3 Elementary Linear Algebra 5 Example

1 3 2x1   1  1 2 3x  9  Let Ax = b be the linear system  2    2 1 2x3   3 Show that b is in the column space of A, and express b as a linear combination of the column vectors of A.

 Solution:

 Solving the system by Gaussian elimination yields

x1 = 2, x2 = -1, x3 = 3  Since the system is consistent, b is in the column space of A.  Moreover, it follows that 1  3  2  1  2 1  2 3 3  9    2  1  2  3

2008/12/3 Elementary Linear Algebra 6 General and Particular Solutions

 Theorem 5.5.2

 If x0 denotes any single solution of a consistent linear system Ax = b, and if v1, v2, …, vk form a basis for the nullspace of A, (that is, the solution space of the homogeneous system Ax = 0), then every solution of Ax = b can be expressed in the form

x = x0 + c1v1 + c2v2 + · · · + ckvk Conversely, for all choices of scalars c1, c2, …, ck, the vector x in this formula is a solution of Ax = b.

2008/12/3 Elementary Linear Algebra 7 Proof of Theorem 5.5.2

 Assume that x0 is any fixed solution of Ax=b and that x is an arbitrary solution. Then Ax0 = b and Ax = b.  Subtracting these equations yields

Ax –Ax0 = 0 or A(x-x0)=0

 Which shows that x-x0 is a solution of the homogeneous system Ax = 0.

 Since v1, v2, …, vk is a basis for the solution space of this system, we can express x-x0 as a linear combination of these vectors, say x-x0 = c1v1+c2v2+…+ckvk. Thus, x=x0+c1v1+c2v2+…+ckvk.

2008/12/3 Elementary Linear Algebra 8 Proof of Theorem 5.5.2

 Conversely, for all choices of the scalars c1,c2,…,ck, we have

Ax = A(x0+c1v1+c2v2+…+ckvk)

Ax = Ax0 + c1(Av1) + c2(Av2) + … + ck(Avk)

 But x0 is a solution of the nonhomogeneous system, and v1, v2, …, vk are solutions of the homogeneous system, so the last equation implies that Ax = b + 0 + 0 + … + 0 = b  Which shows that x is a solution of Ax = b.

2008/12/3 Elementary Linear Algebra 9 Remark

 Remark

 The vector x0 is called a particular solution (特解) of Ax = b.

 The expression x0 + c1v1 + · · · + ckvk is called the general solution (通解) of Ax = b, the expression c1v1 + · · · + ckvk is called the general solution of Ax = 0.  The general solution of Ax = b is the sum of any particular solution of Ax = b and the general solution of Ax = 0.

2008/12/3 Elementary Linear Algebra 10 Example (General Solution of Ax = b)

 The solution to the x  3r 4s 2t 0  3 4 2 nonhomogeneous system 1 x   r  0  1  0  0  2            x3   2s  0  0  2 0  x + 3x –2x + 2x = 0    r s t  1 2 3 5 x s 0 0 1 0 2x + 6x –5x –2x + 4x –3x = -1 4            1 2 3 4 5 6             x5 t 0 0 0 1 5x3 + 10x4 + 15x6 = 5             x6   1/ 3  1/ 3 0  0  0  2x1 + 5x2 + 8x4 + 4x5 + 18x6 = 6   x0 x is which is the general solution.

 The vector x0 is a particular x = -3r - 4s - 2t, x = r, 1 2 solution of nonhomogeneous x3 = -2s, x4 = s, x = t, x = 1/3 system, and the linear 5 6 combination x is the general solution of the homogeneous  The result can be written in vector form as system.

2008/12/3 Elementary Linear Algebra 11 Elementary Row Operation

 Performing an elementary row operation on an augmented matrix does not change the solution set of the corresponding linear system.  It follows that applying an elementary row operation to a matrix A does not change the solution set of the corresponding linear system Ax=0, or stated another way, it does not change the nullspace of A.

The solution space of the homogeneous system of equation Ax = 0, which is a subspace of Rn, is called the nullspace of A.

2008/12/3 Elementary Linear Algebra 12 Example 2 2 1 0 1  1 1 2 3 1   Find a basis for the nullspace of A   1 1 2 0 1   0 0 1 1 1   Solution  The nullspace of A is the solution space of the homogeneous system

2x1 + 2x2 – x3 + x5 = 0 -x1 –x2 –2 x3 –3x4 + x5 = 0 x1 + x2 –2 x3 –x5 = 0 x3 + x4 + x5 = 0  In Example 10 of Section 5.4 we showed that the vectors 1  1 1  0      v10and v 2  1     0  0  0  1  form a basis for the nullspace.

2008/12/3 Elementary Linear Algebra 13 Theorems 5.5.3 and 5.5.4

 Theorem 5.5.3  Elementary row operations do not change the nullspace of a matrix.

 Theorem 5.5.4  Elementary row operations do not change the row space of a matrix.

2008/12/3 Elementary Linear Algebra 14 Proof of Theorem 5.5.4

 Suppose that the row vectors of a matrix A are r1,r2,…,rm, and let B be obtained from A by performing an elementary row operation.  We shall show that every vector in the row space of B is also in that of A, and that every vector in the row space of A is in that of B.  If the row operation is a row interchange, then B and A have the same row vectors and consequently have the same row space.

2008/12/3 Elementary Linear Algebra 15 Proof of Theorem 5.5.4

 If the row operation is multiplication of a row by a nonzero scalar or a multiple of one row to another, then

the row vector r1’,r2’,…,rm’ of B are linear combination of r1,r2,…,rm; thus they lie in the row space of A.  Since a vector space is closed under addition and scalar

multiplication, all linear combination of r1’,r2’,…,rm’ will also lie in the row space of A. Therefore, each vector in the row space of B is in the row space of A.

2008/12/3 Elementary Linear Algebra 16 Proof of Theorem 5.5.4

 Since B is obtained from A by performing a row operation, A can be obtained from B by performing the inverse operation (Sec. 1.5).  Thus the argument above shows that the row space of A is contained in the row space of B.

2008/12/3 Elementary Linear Algebra 17 Remarks

 Do elementary row operations change the column space?

 Yes!  The second column is a scalar multiple of the first, so the column space of A consists of all scalar multiplies of the first column vector.

Add -2 times the first row to the second

 Again, the second column is a scalar multiple of the first, so the column space of B consists of all scalar multiples of the first column vector. This is not the same as the column space of A.

2008/12/3 Elementary Linear Algebra 18 Theorem 5.5.5

 Theorem 5.5.5  If A and B are row equivalent matrices, then:  A given set of column vectors of A is linearly independent if and only if the corresponding column vectors of B are linearly independent.  A given set of column vectors of A forms a basis for the column space of A if and only if the corresponding column vectors of B form a basis for the column space of B.

2008/12/3 Elementary Linear Algebra 19 Theorem 5.5.6

 Theorem 5.5.6  If a matrix R is in row echelon form, then the row vectors with the leading 1’s (i.e., the nonzero row vectors) form a basis for the row space of R, and the column vectors with the leading 1’s of the row vectors form a basis for the column space of R.

2008/12/3 Elementary Linear Algebra 20 Bases for Row and Column Spaces T he matrix 1 2 5 0 3 0 1 3 0 0  R   0 0 0 1 0    0 0 0 0 0  is in row-echelon form. From Theorem 5.5.6 the vectors

r1 [1 -2 5 0 3]

r2 [0 1 3 0 0]

r3 [0 0 0 1 0] form a basis for the row space of R, and the vectors 1  2  0  0  1  0  c,, c  c  10 2 0 4  1      0  0  0  form a basis for the column space of R.

2008/12/3 Elementary Linear Algebra 21 Example

 Find bases for the row and column spaces of 1 3 4 2 5 4  2 6 9 1 8 2  A   2 6 9 1 9 7     Solution: 1 3 4 2 5 4

 Since elementary row operations do not change the row space of a matrix, we can find a basis for the row space of A by finding a basis that of any row-echelon form of A.

 Reducing A to row-echelon form we obtain

1 3 4 2 5 4  0 0 1 3 2 6 R   0 0 0 0 1 5    0 0 0 0 0 0 

2008/12/3 Elementary Linear Algebra 22 1 3 4 2 5 4  1 3 4 2 5 4  Example 2 6 9 1 8 2  0 0 1 3 2 6 A   R   2 6 9 1 9 7  0 0 0 0 1 5      1 3 4 2 5 4 0 0 0 0 0 0   The basis vectors for the row space of R and A

r1 = [1 -3 4 -2 5 4]

r2 = [0 0 1 3 -2 -6]

r3 = [0 0 0 0 1 5]  Keeping in mind that A and R may have different column spaces, we cannot find a basis for the column space of A directly from the column vectors of R.  From Theorem 5.5.5b, we can find the basis for the column space of R, then the corresponding column vectors of A will form a basis for the column space of A. 1  4  5  2  9  8  c ,, c  c   12 3 9 5  9        1  4  5

2008/12/3 Elementary Linear Algebra 23 Example (Basis for a Vector Space Using Row Operations )

 Find a basis for the space spanned by the vectors

v1= (1, -2, 0, 0, 3), v2 = (2, -5, -3, -2, 6), v3 = (0, 5, 15, 10, 0), v4 = (2,6, 18, 8, 6).  Except for a variation in notation, the space spanned by these vectors is the row space of the matrix

1 2 0 0 3 1 2 0 0 3 2 5 3 2 6     0 1 3 2 0 0 5 15 10 0 0 0 1 1 0     2 6 18 8 6 0 0 0 0 0  The nonzero row vectors in this matrix are

w1= (1, -2, 0, 0, 3), w2 =(0,1,3,2,0), w3 =(0,0,1,1,0)  These vectors form a basis for the row space and consequently form a 5 basis for the subspace of R spanned by v1, v2, v3, and v4.

2008/12/3 Elementary Linear Algebra 24 Remarks

 Keeping in mind that A and R may have different column spaces, we cannot find a basis for the column space of A directly from the column vectors of R.  However, it follows from Theorem 5.5.5b that if we can find a set of column vectors of R that forms a basis for the column space of R, then the corresponding column vectors of A will form a basis for the column space of A.

 In the previous example, the basis vectors obtained for the column space of A consisted of column vectors of A, but the basis vectors obtained for the row space of A were not all vectors of A.  Transpose of the matrix can be used to solve this problem.

2008/12/3 Elementary Linear Algebra 25 Example (Basis for the Row Space of a Matrix )

T  Find a basis for the row space of  The column space of A are 1 2 0 0 3   1  2  2  2 5 3 2 6  2  5  6  A         0 5 15 10 0 c10, c 2 3 , and c 4  18         2 6 18 8 6 0  2  8  consisting entirely of row vectors 3  6  6  from A.  Thus, the basis vectors for the row space of A are  Solution: r1 = [1 -2 0 0 3] 1 2 0 2  1 2 0 2  r = [2 -5 -3 -2 6] 2 5 5 6  0 1 5 10 2     r = [2 6 18 8 6] AT 0 3 15 18 0 0 0 1  3     0 2 10 8  0 0 0 0  3 6 0 6  0 0 0 0 

2008/12/3 Elementary Linear Algebra 26 Example (Basis and Linear Combinations)

 (a) Find a subset of the vectors v1 = (1, -2, 0, 3), v2 = (2, -5, -3, 6), v3 =(0, 1, 3, 0), v4 = (2, -1, 4, -7), v5 = (5, -8, 1, 2) that forms a basis for the space spanned by these vectors.  (b) Express each vector not in the basis as a linear combination of the basis vectors.

 Solution (a): 1 2 0 2 5  1 0 2 0 1   0 1 1 0 1 2 5 1 1 8   0 3 3 4 1  0 0 0 1 1     3 6 0 7 2  0 0 0 0 0          

v1 v 2 v 3 v 4 v 5 w1 w 2 w3 w 4 w5

 Thus, {v1, v2, v4} is a basis for the column space of the matrix.

2008/12/3 Elementary Linear Algebra 27 Example

 Solution (b):

 We can express w3 as a linear combination of w1 and w2, express w5 as a linear combination of w1, w2, and w4 (Why?). By inspection, these linear combination are

w3 = 2w1 –w2

w5 = w1 + w2 + w4  We call these the dependency equations. The corresponding relationships in the original vectors are

v3 = 2v1 –v2

v5 = v1 + v2 + v4

2008/12/3 Elementary Linear Algebra 28 Exercises

 Sec. 5.3: 3(c), 3(d), 4(b), 4(d), 11, 15 , 20(d) , 20(e) , 24  Sec. 5.4: 3(d), 4(b), 4(d), 9, 14, 19, 21  Sec. 5.5: 4, 6(c)(d), 8, 9, 12(c), 16, 18  Sec. 5.6: 4, 7, 16, 17, 18, 19

2008/12/3 Elementary Linear Algebra 29 Lecture 20: 5.6 Rank and Nullity

Wei-Ta Chu

2008/12/3 Four Fundamental Matrix Spaces

 Consider a matrix A and its transpose AT together, then there are six vector spaces of interest: T  row space of A, row space of A T  column space of A, column space of A T  null space of A, null space of A  However, the fundamental matrix spaces associated with A are  row space of A, column space of A T  null space of A, null space of A  If A is an mn matrix, then the row space of A and nullspace of A are subspaces of Rn and the column space of A and the nullspace of AT are subspace of Rm  What is the relationship between the dimensions of these four vector spaces?

2008/12/3 Elementary Linear Algebra 31 Dimension and Rank

 Theorem 5.6.1

 If A is any matrix, then the row space and column space of A have the same dimension.  Proof: Let R be any row-echelon form of A. It follows from Theorem 5.5.4 that dim(row space of A) = dim(row space of R).  It follows from Theorem 5.5.5b that dim(column space of A) = dim(column space of R)  The dimension of the row space of R is the number of nonzero rows = number of leading 1’s = dimension of the column space of R

2008/12/3 Elementary Linear Algebra 32 Rank and Nullity

 Definition

 The common dimension of the row and column space of a matrix A is called the rank (秩) of A and is denoted by rank(A); the dimension of the nullspace of a is called the nullity (零核維數) of A and is denoted by nullity(A).

2008/12/3 Elementary Linear Algebra 33 Example (Rank and Nullity)

 Find the rank and nullity of the matrix 1 2 0 4 5 3 3 7 2 0 1 4  A   2 5 2 4 6 1    4 9 2 4 4 7 

 Solution:  The reduced row-echelon form of A is 1 0 4 28 37 13 0 1 2 12 16 5       0 0 0 0 0 0    0 0 0 0 0 0   Since there are two nonzero rows (two leading 1’s), the row space and column space are both two-dimensional, so rank(A) = 2.

2008/12/3 Elementary Linear Algebra 34 Example (Rank and Nullity)

 To find the nullity of A, we must find the dimension of the solution space of the linear system Ax=0.  The corresponding system of equations will be

x1 –4x3 –28x4 –37x5 + 13x6 = 0 x2 –2x3 –12x4 –16 x5+ 5 x6 = 0  It follows that the general solution of the system is

x1 = 4r + 28s + 37t –13u, x2 = 2r + 12s + 16t –5u, x3 = r, x4 = s, x5 = t, x6 = u or x1  4  28  37  13 x  2  12  16  5  2         x3  1  0  0  0  Thus, nullity(A) = 4.  r s  t  u   x4  0  1  0  0  x  0  0  1  0  5         x6  0  0  0  1 

2008/12/3 Elementary Linear Algebra 35