Algorithms to Compute Bases and the Rank of a Matrix Subspaces

Algorithms to Compute Bases and the Rank of a Matrix

Subspaces associated to a matrix Suppose that A is an m × n matrix. The row space of A is the subspace of Rn spanned m by the rows of A. The column space of A is the subspace of R spanned by the columns of A. The null space of A is the subspace n ~ N(A) = {~x ∈ R | A~x = 0m} n n of R . N(A) is a subspace of R by the Subspace Theorem (Theorem 1 of Lecture Note 4). The row rank of A is the dimension of the row space of A. The column rank of A is the dimension of the column space of A. The nullity of A is the dimension of the null space N(A).

More on row equivalence We give some theorems on row equivalence, showing that some subspaces constructed from matrices do not change under elementary row operations. This allows us to ﬁnd important information about these subspaces from the RRE form of a matrix. These interpretations will be given later in this note. Theorem 0.1. Suppose that the m × n matrices     A1 B1  .   .  A =  .  and B =  .  Am Bm are row equivalent (with A1,...,Am,B1,...,Bm ∈ Rn). Then the subspaces Span(A1,...,Am) and Span(B1,...,Bm) of Rn are equal. To prove Theorem 0.1, we need only verify it for a single elementary row operation. Theorem 0.2. Suppose that A and B are row equivalent matrices. Then N(A) = N(B). Theorem 0.2 is just a restatement of Theorem 6 of Lecture Note 1. Theorem 0.3. Suppose that m × n matrices A = (A1,...,An) and B = (B1,...,Bn) 1 n 1 n m with A ,...,A ,B ,...,B ∈ R are row equivalent. Then the subspaces T n 1 n ~ {(x1, . . . , xn) ∈ R | x1A + ··· + xnA = 0} and T n 1 n ~ {(x1, . . . , xn) ∈ R | x1B + ··· + xnB = 0} n 1 n 1 n of R are equal. That is, A ,...,A and B ,...,B have the same dependence relations. Theorem 0.3 follows from Theorem 0.2 and the identities   x1  .  1 n A  .  = x1A + ··· + xnA xn and   x1  .  1 n B  .  = x1B + ··· + xnB xn 1 of formula (1) of Lecture Note 2.

Algorithm to compute a basis of the row space of a matrix. Suppose that   A1  A2    A =  .   .  Am is an m × n matrix. Transform A into its reduced row echelon form B. Then the nonzero rows of B form a basis of the row space Span(A1,...,Am) of A.

To establish this algorithm, apply Theorem 0.1 to conclude that the span of the rows of B is equal to the row space of A. Using the fact that every nonzero row of the RRE form B contains a leading one, it follows from the deﬁnition of linear independence that the nonzero rows of B are linearly independent.

Example 0.4. Let

u1 = (1, 2, 1, 5), u2 = (2, 4, 1, 8), u3 = (3, 6, 2, 13)

Find a basis of the subspace X = Span(u1, u2, u3) of R4.

Solution: Let     u1 1 2 1 5 A =  u2  =  2 4 1 8  . u3 3 6 2 13 We compute the RRE form of A which is  1 2 0 3   0 0 1 2  . 0 0 0 0 The ﬁrst two rows are nonzero. Thus {(1, 2, 0, 3), (0, 0, 1, 2)} is a basis of X

Algorithm to reﬁne the columns of a matrix to a basis of its column space. Sup- pose that A = (A1,A2, ··· ,An) is an m×n matrix. Transform A into its reduced row echelon form B = (B1,B2, ··· ,Bn). Let Bi1 ,Bi2 ,...,Bir be the columns of B which contain a leading 1. Then {Ai1 ,Ai2 ,...,Air } is a basis of the column space Span(A1,A2,...,An) of A.

To establish this algorithm, observe using Theorem 0.3 that the RRE form B of A gives the standard form solutions to the equation 1 n x1A + ··· + xnA = ~0.

i i ir Substituting the standard form solution for the xi, we obtain ﬁrst that A 1 ,A 2 ,...,A are linearly independent, and then that they span the column space of A. 2 Example 0.5. Find a subset of the vectors  1   2   1   5  v1 =  2  , v2 =  4  , v3 =  1  , v4 =  8  3 6 2 13 3 which is a basis of the subspace W = Span(v1, v2, v3, v4) of R .

Solution: Let  1 2 1 5  A = (v1, v2, v2, v4) =  2 4 1 8  . 3 6 2 13 We compute the RRE form of A which is  1 2 0 3   0 0 1 2  . 0 0 0 0 There are leading ones in the ﬁrst and third columns. Thus       1 1  v1 =  2  , v3 =  1   3 2  is a basis of W

Algorithm to compute a basis of the null space of a matrix. Let A = (aij) be an m × n matrix, and   x1  x2    ~x =  .   .  xn be a n × 1 vector of indeterminates. Let N(A) be the null space of the matrix A. By Theorem 0.2, a basis for N(A) can be found by solving the system

A~x = ~0m using Gaussian elimination to ﬁnd the standard form solution, putting the standard form solution into a column vector and expanding with indeterminate coeﬃcients. The vectors in this expansion are a basis of N(A).

Example 0.6. Find a basis of N(A) if  1 2 3 4  A =  1 3 5 6  . 2 5 8 10 Solution: We compute that the RRE form of A is  1 0 −1 0   0 1 2 2  . 0 0 0 0 3 We write the standard form solution as         x1 s 1 0  x2   −2s − 2t   −2   −2    =   = s   + t    x3   s   1   0  x4 t 0 1 with s, t ∈ R. Thus  1   0     −2   −2    ,    1   0   0 1  is a basis of N(A).

Algorithm to extend a set of linearly independent row vectors to a basis of Rn. Suppose that w1, . . . , wm ∈ Rn are linearly independent. Let {e1, . . . , en} be the standard basis of Rn. Transform the m × n matrix   w1  w2     .   .  wm into a reduced row echelon form B. Let Bi1 ,Bi2 ,...,Bin−m be the indices of the columns of B which do not contain a leading 1. Then {w1, . . . , wm, ei1 , . . . , ein−m } is a basis of Rn.

To establish this algorithm, recall that the span of the rows of the RRE form of B is equal to Span(w1, . . . , wm) by Theorem 0.1. Since w1, . . . , wm are linearly independent,

B contains m leading ones. One then checks that the rows of B and ei1 , . . . , ein−m are linearly independent. We then have that w1, . . . , wm, ei1 , . . . , ein−m span an n-dimensional subspace of Rn, so they are a basis of Rn by 2) of Theorem 4.10 of Lecture Note 4.

The rank of a matrix Theorem 0.7. rowrank(A) = columnrank(A). Proof. Let r be the number of leading ones in the RRE form of A. By the algorithm to compute a basis of a row space of a matrix, a basis of the row space of A has r vectors. By the algorithm to refine the columns of a matrix to a basis of its column space, a basis of the column space of A also has r vectors. Thus the row space and column space of A have the same dimension. We define the rank of A to be the row rank of A (which is also the column rank of A). Theorem 0.8. (Rank Nullity Theorem) Suppose that A is an m × n matrix. Then rank(A) + nullity(A) = n. Proof. Let r be the number of leading ones in the RRE form of A. Then rank(A) = r (by the algorithm to compute a basis of the row space of a matrix or the algorithm to refine the columns of a matrix to a basis of its column space). The nullity of A is n − r, by the algorithm to compute a basis of the null space of a matrix, since there are n − r free ~ variables in the general solution to A~x = 0m. 4 As explained in Theorem 2 of Lecture Note 5, we have the following theorem. Theorem 0.9. Suppose that A is an n × n matrix. Then the columns of A are linearly independent if and only if Det(A) 6= 0. Suppose that A is an m × n matrix. An r × r submatrix of A is a matrix obtained from A by removing m − r rows and n − r columns. Corollary 0.10. Suppose that A is an m × n matrix. Then rank(A) = r if and only if Det(M) = 0 for all (r+1)×(r+1) submatrices M of A and there exists an r×r submatrix M of A such that Det(M) 6= 0. Computing determinants has a high degree of complexity, so the best practical way to compute rank(A) of a particular matrix A is to compute the RRE form B of A, and then observe that rank(A) = the number of leading ones of B = the number of nonzero rows of B.

More applications of the Algorithms We can use coordinate vectors with respect to a convenient basis (say a standard basis) to apply the above algorithms to an arbitrary vector space. Example 0.11. Find a subset of the matrices 1 2 2 4 1 3 v = , v = , v = , 1 2 1 2 4 2 3 1 5 which is a basis of V = Span(v1, v2, v3). Solution: We use the algorithm to refine the column vectors of a matrix to a basis of its ∗ 2×2 column space. Let β = {e11, e12, e21, e22} be the standard basis of R . Form the matrix  1 2 1   2 4 3  A = ((v1)β∗ , (v2)β∗ , (v3)β∗ ) =   .  2 4 1  1 2 5 We compute that the RRE form of A is  1 2 0   0 0 1    ,  0 0 0  0 0 0 which has leading ones in the first and third columns. Thus  1   1     2   3    ,    2   1   1 5  is a basis of the column space of A (the first and third columns of A). Since  1   1   2   3  (v1)β∗ =   and (v3)β∗ =   ,  2   1  1 5 5 we have that 1 2 1 3 v = , v = 1 2 1 3 1 5 is a basis of V .

Example 0.12. Find a basis of the subspace

U = {f ∈ P4 | f(1) = f(2) = 0} of P4. Solution: We apply the algorithm to compute a basis of the null space of a matrix. We will ﬁnd the linear conditions on the coeﬃcients a0, a1, a2, a3 of 2 3 (1) f(x) = a0 + a1x + a2x + a3x ∈ P4 for f to be in U. These linear conditions are

f(1) = a0 + a1 + a2 + a3 = 0 f(3) = a0 + 2a1 + 4a2 + 8a3 = 0 The coeﬃcient matrix of this homogeneous system of equations is 1 1 1 1 A = 1 2 4 8 We compute that the RRE form of A is 1 0 −2 −6 . 0 1 3 7 We write the standard form solution as         a0 2t1 + 6t2 2 6  a1   −3t1 − 7t2   −3   −7    =   = t1   + t2    a2   t1   1   0  a3 t2 0 1 with t1, t2 ∈ R. Substituting the solutions         a0 2 a0 6  a1   −3   a1   −7    =   and   =    a2   1   a2   0  a3 0 a3 1 back into (1), we obtain that 2 − 3x + x2, 6 − 7x + x3 is a basis of U. (We are really working with coordinate vectors with respect to the basis ∗ 2 3 β = {1, x, x , x } of P4).