The Fundamental Subspaces Learning Goal: to identify the four fundamental subspaces of a matrix, and find their dimensions and how they fit together. The four fundamental subspaces We have already encountered three subspaces attached to a matrix A: its column space C(A), its nullspace N(A), and its row space C(AT). Let’s complete the cycle with the missing combination, N(AT). This is called the left nullspace. Let A be an m × n matrix with rank r. Let’s look at these spaces and what we already know about them, one by one: 1) The column space. The column space is all combinations of the columns. That is, all vectors of the form Ax for any vector x. Thus it is a subspace of Rm. We have seen that it has dimension r, and a basis consists of those columns of A that end up having pivots in them when we do row reduction. 2) The row space. This is the set of all linear combinations of the rows. Thus is a subspace of Rn. Since the rows of A are the columns of AT, we could find its dimension and basis as above. But it is more convenient to work with A directly. Row reduction doesn’t change the span of the rows, and we can ignore the zero rows we may get. So a basis for it is the rows of U (or R) we get after reduction, or the rows that became those rows from A itself. There are r rows that end up with pivots in them, so the dimension of the row space is also r. Note that this shows that if we row- reduce AT we will end up with exactly as many pivot rows and pivot columns! Row rank = column rank. This doesn’t seem obvious at first blush, but in retrospect it seems pretty simple! 3) The nullspace. This is the set of all solutions, x, to Ax = 0. As such, it is a subspace of Rn. We have discovered that a basis for it consists of the “special” null vectors. There is one of these for each free variable, and there are n – r (columns – pivots) free variables. So the null space has dimension n – r. 4) The left nullspace. Solving ATx = 0, we could transpose to get xTA = 0T. So the left nullspace is exactly those vectors that could multiply A on the left to give the zero row. Since there are m columns in the matrix AT, the left nullspace must have dimension m – r. We could find a basis for it by working with AT and finding its special null vectors, but there is another nifty trick. Perform Gaussian (or better, Gauss-Jordan) elimination on [A | I] to produce [U | B] (or [R | C]). Claim: the last m – r rows of B (which equal those of C because once we get zeroes there’s no more work to do in those rows) form a basis for the left nullspace of A. Why? Each such row is in the left nullspace, because we know that BA = U and that CA = R. Thus, looking at matrix multiplication a row at a time, the last rows or U (or R) come from the last rows of B (or C) times the entire matrix A. But these rows in U are zero, so the last rows of B times A give zero—they are in the left nullspace. They are also linearly independent, because they come from the matrix I by elementary row operations. And there are the right number, m – r, of them. So they must be a basis! There is another method for computing a basis for the left nullspace that requires doing elimination on just one extra column, but it is full of unknowns and not explicit numbers. See problems 18 – 19 of section 3.6. Finally, notice that the left null vectors give combinations of the entries of a right-hand side b that must work out for the vector to be in the column space—and this was exactly our method of finding those conditions all along! We have proved: Theorem: (Fundamental Theorem of Linear Algebra, Part I) The row and column spaces both have dimension r, and bases for them can be found among the rows and columns of A itself. The nullspaces have dimenions n – r and m – r. Another curiosity is to look at C(AT) ∩ N(A). I claim this consists only of the zero vector. For let x be a vector in both the row space and the nullspace. Since it is in the row space, x = a1(row 1) + a2(row 2) + ! + am(row m). But since it is in the nullspace, Ax = 0, so the dot product of any row with x is 0. But then x⋅x = 0! This means that x must be the zero vector. Thus C(AT) ∩ N(A) = Z, the zero dimensional space. By the subspace arithmetic theorem we proved a short while ago, we find that dim(C(AT) ∩ N(A)) + dim(C(AT) + N(A)) = dim(C(AT)) + dim(N(A)). This comes to 0 + dim(C(AT) + N(A)) = r + (n – r) = n. So dim(C(AT) + N(A)) = n. In other words, the sum of the row and nullspaces must be all of Rn. n Thus, any vector in R can be written as the sum of a vector in the nullspace plus one in the row space. Not only that, but this breakdown is unique, for if v = r1 + n1 = r2 + n2, then we rearrange to get r1 – r2 = n2 – n1. The left side is in the row space, the right in the nullspace. Since these intersect at only the zero vector, we get r1 = r2 and n1 = n2. The same relationship holds, obviously, for the column and left nullspaces. In fact, even more is true about the four fundamental subspaces, but that will have to wait…. Reading: 3.6 Problems: 3.6: 1, 3, 4, 6, 7, 8, 11, 12, 15, 18, 19, 21, 22, 23, 24, 25, 26, 31 .