Matrix Invertibility

Matrix invertibility

Rank-Nullity Theorem: For any n-column matrix A,

nullity A +rankA = n

Corollary: Let A be an R C matrix. Then A is invertible if and only if R = C and the ⇥ | | | | columns of A are linearly independent. Proof: Let F be the ﬁeld. Deﬁne f : FC FR by f (x)=Ax. ! Then A is an invertible matrix if and only if f is an invertible function. The function f is invertible i↵dim Ker f =0anddimFC =dimFR i↵nullity A =0and C = R . | | | | nullity A =0 i↵ dim Null A =0 i↵Null A = 0 { } i↵the only vector x such that Ax = 0 is x = 0 i↵the columns of A are linearly independent. QED Matrix invertibility examples

123 is not square so cannot be invertible. 456  12 is square and its columns are linearly independent so it is invertible. 34  1 1 2 2 1 3 is square but columns not linearly independent so it is not invertible. 2 3 3 1 4 4 5 Transpose of invertible matrix is invertible

Theorem: The transpose of an invertible matrix is invertible.

a1 A = v v = . AT = a a 2 1 ··· n 3 2 . 3 2 1 ··· n 3 a 6 n 7 4 5 4 5 4 5 Proof: Suppose A is invertible. Then A is square and its columns are linearly independent. Let n be the number of columns. Then rank A = n. Because A is square, it has n rows. By Rank Theorem, rows are linearly independent. Columns of transpose AT are rows of A, so columns of AT are linearly independent. Since AT is square and columns are linearly independent, AT is invertible. QED More matrix invertibility 1 1 Earlier we proved: If A has an inverse A then AA is identity matrix Converse: If BA is identity matrix then A and B are inverses? Not always true. Theorem: Suppose A and B are square matrices such that BA is an identity matrix 1.ThenA and B are inverses of each other. Proof: To show that A is invertible, need to show its columns are linearly independent. Let u be any vector such that Au = 0. Then B(Au)=B0 = 0. On the other hand, (BA)u = 1u = u, so u = 0. 1 1 1 This shows A has an inverse A . Now must show B = A . We know AA = 1. BA = 1 1 1 1 (BA)A = 1A by multiplying on the right by A 1 1 (BA)A = A 1 1 B(AA ) = A by associativity of matrix-matrix mult 1 B 1 = A 1 B = A QED Representations of vector spaces

Two important ways to represent a vector space: As the solution set of homogeneous linear system As Span b ,...,b { 1 k } a1 x =0,...,am x =0 · · Equivalently,

a1 Equivalently, Null . 2 . 3 Col 2 b b 3 1 ··· k am 6 7 6 7 6 7 4 5 6 7 6 7 4 5 Conversions between the two representations

[x, y, z]: { [4, 1, 1] [x, y, z]=0, Span [1, 2, 2] · { } [0, 1, 1] [x, y, z]=0 · }

Span [4, 1, 1], [0, 1, 1] [x, y, z]:[1, 2, 2] [x, y, z]=0 { } { · } Conversions for ane spaces?

I From representation as solution set of linear system to representation as ane hull

I From representation as ane hull to representation as solution set of linear system Conversions for ane spaces? From representation as solution set of linear system to representation as ane hull

I input: linear system Ax = b I output: vectors whose ane hull is the solution set of the linear system. x 121 1 y = 122 2 3 2  z  1 Let u be one solution to the linear system. u =[ 0.5, 0.754, 0] 5 x 121 0 2 Consider the corresponding homogeneous system Ax = 0. y = 122 2 3 0  z  Its solution set, the null space of A,isavectorspace . 4 5 V 3 Let b ,...,b be generators for . b =[2, 3, 4] 1 k V 1 4 Then the solution set of the original linear system is the ane hull of u, b + u, b + u,...,b + u. [ 0.5, .75, 0] and 1 2 k [ 0.5, .75, 0] + [2, 3, 4] From representation as solution set to representation as ane hull x 121 1 One solution to equation y = is u =[ 0.5, 0.75, 0] 122 2 3 2  z  121 4 5 Solution set of equation is u +Span b1 , Null space of is Span b1 : { } 122 { } i.e. the ane hull of u and u + b  1

b 1 u+b1

u How to transform between these two representations? Problem 1 (From left to right): I input: homogeneous linear system a x =0,...,a x = 0, 1 · m · I output: basis b1,...,bk for solution set Problem 2 (From right to left):

I input: independent vectors b1,...,bk , I output: homogeneous linear system a x =0,...,a x = 0 whose solution set equals 1 · m · Span b ,...,b { 1 k }

Representations of vector spaces Two important ways to represent a vector space: As the solution set of homogeneous linear system As Span b ,...,b { 1 k } a1 x =0,...,am x =0 · · Equivalently,

a1 Equivalently, Null . 2 . 3 Col 2 b b 3 1 ··· k am 6 7 6 7 6 7 4 5 6 7 6 7 4 5 Representations of vector spaces Two important ways to represent a vector space: As the solution set of homogeneous linear system As Span b ,...,b { 1 k } a1 x =0,...,am x =0 · · Equivalently,

a1 Equivalently, Null . 2 . 3 Col 2 b b 3 1 ··· k am 6 7 6 7 6 7 4 5 6 7 6 7 How to transform between these two representations? 4 5 Problem 1 (From left to right): I input: homogeneous linear system a x =0,...,a x = 0, 1 · m · I output: basis b1,...,bk for solution set Problem 2 (From right to left):

I input: independent vectors b1,...,bk , I output: homogeneous linear system a x =0,...,a x = 0 whose solution set equals 1 · m · Span b ,...,b { 1 k } with independent rows

with independent columns

Reformulating Problem 1

I input: homogeneous linear system a x =0,...,a x = 0, 1 · m · I output: basis b1,...,bk for solution set Let’s express this in the language of matrices:

a1 . I input: matrix A = 2 . 3 a 6 m 7 4 5 I output: matrix B = b b 2 1 ··· k 3 such that Col B =NullA 4 5 Can require the rows of the input matrix A to be linearly independent. (Discarding a superﬂuous row does not change the null space of A.) with independent rows

Reformulating Problem 1

I input: homogeneous linear system a x =0,...,a x = 0, 1 · m · I output: basis b1,...,bk for solution set Let’s express this in the language of matrices:

a1 . I input: matrix A = 2 . 3 a 6 m 7 4 5 with independent columns I output: matrix B = b b 2 1 ··· k 3 such that Col B =NullA 4 5 Can require the rows of the input matrix A to be linearly independent. (Discarding a superﬂuous row does not change the null space of A.) Reformulating Problem 1

I input: homogeneous linear system a x =0,...,a x = 0, 1 · m · I output: basis b1,...,bk for solution set Let’s express this in the language of matrices:

a1 . I input: matrix A = 2 . 3 with independent rows a 6 m 7 4 5 with independent columns I output: matrix B = b b 2 1 ··· k 3 such that Col B =NullA 4 5 Can require the rows of the input matrix A to be linearly independent. (Discarding a superﬂuous row does not change the null space of A.) same as requiring number of columns of B = n m

0 0 ··· = same as requiring AB = . . ) 2 . . 3 0 0 6 ··· 7 = same as requiring number of columns4 of B =nullity5 A )

I input: m n matrix A with independent rows ⇥ I output: matrix B with n m independent columns such that AB = 0 h i

Reformulating the reformulation of Problem 1

I input: matrix A with independent rows I output: matrix B with independent columns such that Col B =NullA By Rank-Nullity Theorem, rank A +nullityA = n Because rows of A are linearly independent, rank A = m, so m +nullityA = n

Requiring Col B =NullA is the same as requiring

(i) Col B is a subspace of Null A

(ii) dim Col B =nullityA same as requiring number of columns of B = n m

= same as requiring number of columns of B =nullityA )

I input: m n matrix A with independent rows ⇥ I output: matrix B with n m independent columns such that AB = 0 h i

Reformulating the reformulation of Problem 1

Requiring Col B =NullA is the same as requiring 0 0 ··· (i) Col B is a subspace of Null A = same as requiring AB = . . ) 2 . . 3 0 0 6 ··· 7 (ii) dim Col B =nullityA 4 5 I input: m n matrix A with independent rows ⇥ I output: matrix B with n m independent columns such that AB = 0 h i

Reformulating the reformulation of Problem 1

Requiring Col B =NullA is the same as requiring 0 0 ··· (i) Col B is a subspace of Null A = same as requiring AB = . . ) 2 . . 3 0 0 6 ··· 7 (ii) dim Col B =nullityA = same as requiring number of columns4 of B =nullity5 A ) same as requiring number of columns of B = n m Reformulating the reformulation of Problem 1

Problem 1:

I input: m n matrix A with independent rows ⇥ I output: matrix B with n m independent columns such that AB = 0 h i Deﬁne procedure null space basis(M) with this spec:

I input: r n matrix M with independent rows ⇥ I output: matrix C with n r independent columns such that MC = 0 h i Reformulating Problem 2

I input: independent vectors b1,...,bk , I output: homogeneous linear system a x =0,...,a x = 0 whose solution set 1 · m · equals Span b ,...,b { 1 k } Let’s express this in the language of matrices: I input: n k matrix B with independent columns ⇥ I output: matrix A with independent rows such that Null A = Col B As before, Rank-Nullity Theorem implies number of rows of A +nullityA = number of columns of A

As before, requiring Null A = Col B is the same as requiring (i) AB = 0 (ii) numberh of rowsi of A = n k I input: n k matrix B with independent rows ⇥ I output: matrix A with n k independent rows such that AB = 0 h i Solving Problem 2 with the procedure for Problem 1 Problem 1:

I input: m n matrix A with independent rows ⇥ I output: matrix B with n m independent columns such that AB = 0 h i Deﬁne procedure null space basis(M) I input: r n matrix M with independent rows ⇥ I output: matrix C with n r independent columns such that MC = 0 Problem 2: h i I input: n k matrix B with independent rows ⇥ I output: matrix A with n k independent rows such that AB = 0 h i To solve Problem 2, call null space basis(BT ). Returns matrix AT with independent columns such that BT AT = 0 Since BT is k n matrix, AT has n k columns. h i ⇥ Therefore AB = and A has n k independent rows. Therefore A is solution to Problem 0 2. h i