L. Vandenberghe ECE133A (Spring 2021) 4. Matrix inverses • left and right inverse • linear independence • nonsingular matrices • matrices with linearly independent columns • matrices with linearly independent rows 4.1 Left and right inverse < in general, so we have to distinguish two types of inverses Left inverse: - is a left inverse of if - = is left-invertible if it has at least one left inverse Right inverse: - is a right inverse of if - = is right-invertible if it has at least one right inverse Matrix inverses 4.2 Examples 2 −3 −4 3 6 7 1 0 1 = 6 4 6 7 , = 6 7 0 1 1 6 1 1 7 4 5 • is left-invertible; the following matrices are left inverses: 1 −11 −10 16 , 0 −1/2 3 9 7 8 −11 0 1/2 −2 • is right-invertible; the following matrices are right inverses: 2 1 −1 3 2 1 0 3 2 1 −1 3 1 6 7 6 7 6 7 6 −1 1 7 , 6 0 1 7 , 6 0 0 7 2 6 7 6 7 6 7 6 1 1 7 6 0 0 7 6 0 1 7 4 5 4 5 4 5 Matrix inverses 4.3 Some immediate properties Dimensions a left or right inverse of an < × = matrix must have size = × < Left and right inverse of (conjugate) transpose ) ) • - is a left inverse of if and only if - is a right inverse of ) -) = ¹- º) = • - is a left inverse of if and only if - is a right inverse of - = ¹- º = Matrix inverses 4.4 Inverse if has a left and a right inverse, then they are equal and unique: - = , . = =) - = -¹.º = ¹- º. = . • in this case, we call - = . the inverse of (notation: −1) • is invertible if its inverse exists Example 2 −1 1 −3 3 2 2 4 1 3 6 7 − 1 6 7 = 6 1 −1 1 7 , 1 = 6 0 −2 1 7 6 7 4 6 7 6 2 2 2 7 6 −2 −2 0 7 4 5 4 5 Matrix inverses 4.5 Linear equations set of < linear equations in = variables 11G1 ¸ 12G2 ¸ · · · ¸ 1=G= = 11 21G1 ¸ 22G2 ¸ · · · ¸ 2=G= = 12 . <1G1 ¸ <2G2 ¸ · · · ¸ <=G= = 1< • in matrix form: G = 1 • may have no solution, a unique solution, infinitely many solutions Matrix inverses 4.6 Linear equations and matrix inverse Left-invertible matrix: if - is a left inverse of , then G = 1 =) G = - G = -1 there is at most one solution (if there is a solution, it must be equal to -1) Right-invertible matrix: if - is a right inverse of , then G = -1 =) G = -1 = 1 there is at least one solution (namely, G = -1) Invertible matrix: if is invertible, then G = 1 () G = −11 there is a unique solution Matrix inverses 4.7 Outline • left and right inverse • linear independence • nonsingular matrices • matrices with linearly independent columns • matrices with linearly independent rows Linear combination a linear combination of vectors 01,..., 0= is a sum of scalar-vector products G101 ¸ G202 ¸ · · · ¸ G=0= • the scalars G8 are the coefficients of the linear combination • can be written as a matrix-vector product 2 G1 3 6 7 6 G2 7 G101 ¸ G202 ¸ · · · ¸ G=0= = 01 02 ··· 0= 6 . 7 6 . 7 6 7 6 G= 7 4 5 • the trivial linear combination has coefficients G1 = ··· = G= = 0 (same definition holds for real and complex vectors/scalars) Matrix inverses 4.8 Linear dependence a collection of vectors 01, 02,..., 0= is linearly dependent if G101 ¸ G202 ¸ · · · ¸ G=0= = 0 for some scalars G1,..., G=, not all zero • the vector 0 can be written as a nontrivial linear combination of 01,..., 0= • equivalently, at least one vector 08 is a linear combination of the other vectors: G1 G8−1 G8¸1 G= 08 = − 01 − · · · − 08−1 − 08¸1 − · · · − 0= G8 G8 G8 G8 if G8 < 0 Matrix inverses 4.9 Example the vectors 2 0.2 3 2 −0.1 3 2 0 3 6 7 6 7 6 7 01 = 6 −7 7 , 02 = 6 2 7 , 03 = 6 −1 7 6 7 6 7 6 7 6 8.6 7 6 −1 7 6 2.2 7 4 5 4 5 4 5 are linearly dependent • 0 can be expressed as a nontrivial linear combination of 01, 02, 03: 0 = 01 ¸ 202 − 303 • 01 can be expressed as a linear combination of 02, 03: 01 = −202 ¸ 303 (and similarly 02 and 03) Matrix inverses 4.10 Linear independence vectors 01,..., 0= are linearly independent if they are not linearly dependent • the zero vector cannot be written as a nontrivial linear combination: G101 ¸ G202 ¸ · · · ¸ G=0= = 0 =) G1 = G2 = ··· = G= = 0 • none of the vectors 08 is a linear combination of the other vectors Matrix with linearly independent columns = 01 02 ··· 0= has linearly independent columns if G = 0 =) G = 0 Matrix inverses 4.11 Example the vectors 2 1 3 2 −1 3 2 0 3 6 7 6 7 6 7 01 = 6 −2 7 , 02 = 6 0 7 , 03 = 6 1 7 6 7 6 7 6 7 6 0 7 6 1 7 6 1 7 4 5 4 5 4 5 are linearly independent: 2 G1 − G2 3 6 7 G101 ¸ G202 ¸ G303 = 6 −2G1 ¸ G3 7 = 0 6 7 6 G2 ¸ G3 7 4 5 holds only if G1 = G2 = G3 = 0 Matrix inverses 4.12 Dimension inequality if = vectors 01, 02,..., 0= of length < are linearly independent, then = ≤ < (proof is in textbook) • if an < × = matrix has linearly independent columns then < ≥ = • if an < × = matrix has linearly independent rows then < ≤ = Matrix inverses 4.13 Outline • left and right inverse • linear independence • nonsingular matrices • matrices with linearly independent columns • matrices with linearly independent rows Nonsingular matrix for a square matrix the following four properties are equivalent 1. is left-invertible 2. the columns of are linearly independent 3. is right-invertible 4. the rows of are linearly independent a square matrix with these properties is called nonsingular Nonsingular = invertible • if properties 1 and 3 hold, then is invertible (page 4.5) • if is invertible, properties 1 and 3 hold (by definition of invertibility) Matrix inverses 4.14 Proof (a) left-invertible linearly independent columns (b’) (b) (a’) linearly independent rows right-invertible • we show that (a) holds in general • we show that (b) holds for square matrices ) • (a’) and (b’) follow from (a) and (b) applied to Matrix inverses 4.15 Part a: suppose is left-invertible • if is a left inverse of (satisfies = ), then G = 0 =) G = 0 =) G = 0 • this means that the columns of are linearly independent: if = 01 02 ··· 0= then G101 ¸ G202 ¸ · · · ¸ G=0= = 0 holds only for the trivial linear combination G1 = G2 = ··· = G= = 0 Matrix inverses 4.16 Part b: suppose is square with linearly independent columns 01,..., 0= • for every =-vector 1 the vectors 01,..., 0=, 1 are linearly dependent (from dimension inequality on page 4.13) • hence for every 1 there exists a nontrivial linear combination G101 ¸ G202 ¸ · · · ¸ G=0= ¸ G=¸11 = 0 • we must have G=¸1 < 0 because 01,..., 0= are linearly independent • hence every 1 can be written as a linear combination of 01,..., 0= • in particular, there exist =-vectors 21,..., 2= such that 21 = 41, 22 = 42, . , 2= = 4=, • the matrix = 21 22 ··· 2= is a right inverse of : 21 22 ··· 2= = 41 42 ··· 4= = Matrix inverses 4.17 Examples 2 1 −1 1 −1 3 2 1 −1 1 3 6 7 6 7 6 −1 1 −1 1 7 = 6 −1 1 1 7 , = 6 7 6 7 6 1 1 −1 −1 7 6 1 1 −1 7 6 7 4 5 6 −1 −1 1 1 7 4 5 • is nonsingular because its columns are linearly independent: G1 − G2 ¸ G3 = 0, −G1 ¸ G2 ¸ G3 = 0,G1 ¸ G2 − G3 = 0 is only possible if G1 = G2 = G3 = 0 • is singular because its columns are linearly dependent: G = 0 for G = ¹1, 1, 1, 1º Matrix inverses 4.18 Example: Vandermonde matrix C C2 ··· C=−1 2 1 1 1 1 3 6 2 =−1 7 6 1 C2 C ··· C 7 = 6 . .2 . 2. 7 with C8 < C 9 for 8 < 9 6 . 7 6 2 =−1 7 6 1 C= C= ··· C= 7 4 5 we show that is nonsingular by showing that G = 0 only if G = 0 • G = 0 means ?¹C1º = ?¹C2º = ··· = ?¹C=º = 0 where 2 =−1 ?¹Cº = G1 ¸ G2C ¸ G3C ¸ · · · ¸ G=C ?¹Cº is a polynomial of degree = − 1 or less • if G < 0, then ?¹Cº can not have more than = − 1 distinct real roots • therefore ?¹C1º = ··· = ?¹C=º = 0 is only possible if G = 0 Matrix inverses 4.19 Inverse of transpose and product Transpose and conjugate transpose ) if is nonsingular, then and are nonsingular and ¹)º−1 = ¹−1º), ¹º−1 = ¹−1º ) we write these as − and − Product if and are nonsingular and of equal size, then is nonsingular with ¹º−1 = −1−1 Matrix inverses 4.20 Outline • left and right inverse • linear independence • nonsingular matrices • matrices with linearly independent columns • matrices with linearly independent rows Gram matrix the Gram matrix associated with a matrix = 01 02 ··· 0= is the matrix of column inner products • for real matrices: ) ) ) 2 0 01 0 02 ··· 0 0= 3 6 )1 )1 )1 7 ) 6 0 01 0 02 ··· 0 0= 7 = 6 2. 2. 2. 7 6 . 7 6 ) ) ) 7 6 0= 0 0= 0 ··· 0= 0= 7 4 1 2 5 • for complex matrices 2 0 01 0 02 ··· 0 0= 3 6 1 1 1 7 6 0 01 0 02 ··· 0 0= 7 = 6 2.