Matrices, transposes, and inverses
Math 40, Introduction to Linear Algebra Wednesday, February 1, 2012
Matrix-vector multiplication: two views
• 1st perspective: A x is linear combination of columns of A 44 11 232 3 11 22 33 4 −− 33 ==44 ++33 −− ++22 = 215215 22 11 55 21 22 A x
• 2nd perspective: A x is computed as dot product of rows of A with vector x
4 4 1 23 4 − 3 = 2 4 = 2 1 5 dot product of 1 and 3 21 2 5 2 A x
Notice that # of columns of A = # of rows of x . This is a requirement in order for matrix multiplication to be defined. Matrix multiplication
What sizes of matrices can be multiplied together?
the matrix product AB For m x n matrix A and n x p matrix B, is an m x p matrix. m x n n x p “inner” parameters must match “outer” parameters become parameters of matrix AB
If A is a square matrix and k is a positive integer, we define Ak = A A A · ··· k factors
Properties of matrix multiplication
Most of the properties that we expect to hold for matrix multiplication do....
A(B + C)=AB + AC (AB)C = A(BC) k(AB)=(kA)B = A(kB) for scalar k
.... except commutativity!!
In general, AB = BA. Matrix multiplication not commutative
Problems with hoping AB and BA are equal: In general, • BA may not be well-defined. AB = BA. (e.g., A is 2 x 3 matrix, B is 3 x 5 matrix)
• Even if AB and BA are both defined, AB and BA may not be the same size. (e.g., A is 2 x 3 matrix, B is 3 x 2 matrix)
• Even if AB and BA are both defined and of the same size, they still may not be equal.
11 12 24 33 12 11 = = = 11 12 24 33 12 11
Truth or fiction?
Question 1 For n x n matrices A and B, is A2 B2 =(A B)(A + B)? − −
No!! (A B)(A + B)=A2 + AB BA B2 − − − =0 2 2 2 Question 2 For n x n matrices A and B, is (AB) = A B ?
No!! (AB)2 = ABAB = AABB = A2B2 Matrix transpose
Definition The transpose of an m x n matrix A is the n x m matrix AT obtained by interchanging rows and columns of A, i.e., (AT ) = A i, j. ij ji ∀
Example 15 135 2 A = T 33 532− 1 A = 52 21 Transpose operation can be viewed as − flipping entries about the diagonal.
Definition A square matrix A is symmetric if AT = A.
Properties of transpose
apply twice -- get back (1) (AT )T = A to where you started
(2) (A + B)T = AT + BT
(3) For a scalar c,(cA)T = cAT
(4) (AB)T = BT AT To prove this, we show that T [(AB) ]ij = Exercise . T . Prove that for any matrix A, A A is symmetric. T T =[(B A )]ij Special matrices
Definition A matrix with all zero entries is called 0000 A = 0000 a zero matrix and is denoted 0. 0000 Definition A square matrix is upper-triangular 1 2 4 5 if all entries below main diagonal are zero. A = 06 0 00 3 analogous definition for a lower-triangular matrix − 3 8 000 −0 200 Definition A square matrix whose off-diagonal A = 00− 40 entries are all zero is called a diagonal matrix. − 0001 100 Definition The identity matrix, denoted In, is the I3 = 010 n x n diagonal matrix with all ones on the diagonal. 001
Identity matrix
100 Definition The identity matrix, denoted In, is the I3 = 010 n x n diagonal matrix with all ones on the diagonal. 001
Important property of identity matrix If A is an m x n matrix, then ImA = A and AIn = A. If A is a square matrix, then IA = A = AI. TheTheThe inverse notion inverse of aof of matrix inverse a matrix Exploration Let’s think about inverses first in the context of real num- ExplorationExploration ConsiderLet’s think the about set of inverses real numbers, first in theand context say that of realwe have num- bers. Saybers. we Say have we equation have equation the equation 3x = 2 3x =23x =2 and weand want we towant solve to forsolvex.Todoso,multiplybothsidesby for x. What1 to do1 obtain we do? and we want to solve for x.Todoso,multiplybothsidesby3 3 to obtain 1 1 12 We multiply both(3 sidesx)=1 of (2)the 1equation = x by= 3 .to obtain2 3 (33x)= (2)⇒ = 3x = . 13 31 ⇒ 3 2 For , 1 is the1 multiplicative(3 inversex)= of(2) 3 since= 1(3) =x1 = 1. . R For3 R, 3 is the multiplicative3 inverse3 of 3 since3 3(3) = 1. multiplicative inverse ⇒ 3 1 of 3 since (3) = 1 Now considerNow consider the3 following the following system system of ofNoticeNotice that thatwe can we rewrite can rewrite these these equationsTheNow,equationsThe inverse consider inverse of the a linear of matrix a system matrix equationsNoticeequations that as we as can rewrite equations as 3x 35x =65x =6 3 35 x51 x1 6 6 ExplorationExploration1 Let’s12 thinkLet’s2 aboutthink about inverses inverses first in first the in context the− context= of real= of realnum- num- − − − 23 x 1 bers. Say we2x have+32 equationx1 +3= x12 = 1 23 x2 2 1 bers. Say1 − we have2 equation− − − − − − − A 3x =23x =2 A x x b b which wewhich want we to want solve to for solvex1 forandx1xand2. x2. and we want to solveHow for dox .Todoso,multiplybothsidesbywe isolate the vector x by itself on1 LHS?to1 obtain and we want to solve for x.Todoso,multiplybothsidesby x 3 to obtain How do we isolate the vector xx1= 1 by itself on the3 LHS? How do we isolate1 the1 vector1 x1= byx itself2 on2 the LHS? (3x)=(3x(2))= (2) = x2 =x =2x. = . 3 3 3 3 ⇒ ⇒ 3 3 1 1 3 5 For ,Multiplyis the1 multiplicative both sides of matrix inverse equation of 3 since by3 5(3)−2 − =13 1.: MultiplyR For3 bothR, sidesis the of multiplicative matrix equation inverse by of− 32 − since33−: − (3) = 1. 3 − − 3 3 5 3 5 x1 3 5 6 The notion3 −5 of− inverse3 5 −x1 =3 −5 − 6 Now considerNow consider− the following− the2 following3 system− system23 of ofNoticex=2 Notice− that− thatwe2 can3 we rewrite can1 rewrite these these 2 −3 − 23− x2 2 −3 − 1 − equationsNow,equations consider− − the linear− system equationsNoticeequations− that as− we as −can rewrite equations as 10 x1 13 10 x1 =13 − 3x1 35x12 =65x2 =6 01 =x2 −3 359 x51 x1 6 6 − − 01 x2 9−− − = = 2323x x2 1 1 2x1 +32x12 +3= x12 = 1 I − 2 − − I − − − − − − A x1 A 13 x x b b which wewhich want we to want solve to for solvex1 forandx1xand2. xx1 2. =13 − How do we isolate the vector=x2 − x by itself9 on LHS? x2 x 9 − How do we isolate the vector xx1= −1 by itself on the LHS? How do we isolate the vector3 x =5 x1byx itself on the LHS? 6 ? − x2 2= ? Thus, the solution to ()is23x1 = 13x and2 x2 = 9. 1 Thus, the solution to ()isx1 =− 13 and−x2 = 9.3 −5 − Multiply both sides of matrix equation by3 5− − : Multiply both sides of matrix equation− by − −− 2: 3 want this equal to identity matrix, I 2 3− − Lecture 7 Math 40, Spring ’12,− Prof.− Kindred Page 1 3 5 3 5 x1 3 5 6 Lecture 73 −5 Math− 3 40, Spring5 − ’12,x1 Prof.3 Kindred5=3 −5 − 6 Page 1 − − 2 3 − 23− x=2 −− − 2 3 1 2 −3 − 23− x2 2 3 2 −3 − 1 − − − − − −− − − 10 x1 13 10 x1 =13 − x1 10 x011 =x2 −3 5 9 6 13 = 01 x2= − 9−− = − x2 01 x2I 2− 3 1 9 I − − − − x1 13 x1 =13 − =x2 − 9 x 9 − 2 −
Thus, the solution to ()isx1 = 13 and x2 = 9. Thus, the solution to ()isx = 13 and− x = 9. − 1 − 2 − Lecture 7 Math 40, Spring ’12, Prof. Kindred Page 1 Lecture 7 Math 40, Spring ’12, Prof. Kindred Page 1 Matrix inverses
Definition AsquarematrixA is invertible (or nonsingular)if matrix ∃ B such that AB = I and BA = I.(WesayB is an inverse of A.)
Example 27 4 7 A = is invertible because for B = − , 14 12 − 27 4 7 10 we have AB = − = = I 14 12 01 − 4 7 27 10 and likewise BA = − = = I. 12 14 01 − The notion of an inverse matrix only applies to square matrices. - For rectangular matrices of full rank, there are one-sided inverses.
- For matrices in general, there are pseudoinverses, which are a generalization to matrix inverses.
11 Example Find the inverse of A = .Wehave 11 11 ab 10 a + cb+ d 10 = = = 11 cd 01 ⇒ a + cb+ d 01 = a + c =1anda + c =0 Impossible! ⇒ 11 Therefore, A = is not invertible (or singular). 11
Take-home message: Not all square matrices are invertible.
Lecture 7 Math 40, Spring ’12, Prof. Kindred Page 1 Important questions: When does a square matrix have an inverse? • If it does have an inverse, how do we compute it? • Can a matrix have more than one inverse? •
Theorem. If A is invertible, then its inverse is unique. Proof. Assume A is invertible. Suppose, by way of contradiction, that the inverse of A is not unique, i.e., let B and C be two distinct inverses of A. Then, by def’n of inverse, we have BA = I = AB (1) and CA = I = AC. (2) It follows that B = BI by def’n of identity matrix = B(AC)by(2)above =(BA)C by associativity of matrix mult. = IC by (1) above = C. by def’n of identity matrix Thus, B = C,whichcontradictsthepreviousassumptionthatB = C. So it must be that case that the inverse of A is unique. ⇒⇐
The inverse of a matrix A is unique, Take-home message: 1 and we denote it A− .
Theorem (Properties of matrix inverse).
1 1 1 (a) If A is invertible, then A− is itself invertible and (A− )− = A.
Lecture 7 Math 40, Spring ’12, Prof. Kindred Page 2 (b) If A is invertible and c =0is a scalar, then cA is invertible and 1 1 1 (cA)− = c A− .
(c) If A and B are both n n invertible matrices, then AB is invertible 1 1 1× and (AB)− = B− A− . “socks and shoes rule” – similar to transpose of AB generalization to product of n matrices
T T 1 1 T (d) If A is invertible, then A is invertible and (A )− =(A− ) .
To prove (d), we need to show that there is some matrix such that AT = I and AT = I.
1 Proof of (d). Assume A is invertible. Then A− exists and we have
1 T T 1 T T (A− ) A =(AA− ) = I = I and T 1 T 1 T T A (A− ) =(A− A) = I = I. T T 1 1 T So A is invertible and (A )− =(A− ) .
Question: If A and B are invertible n n matrices, × what can we say about A + B?
There is no guarantee A + B is invertible even if A and B themselves are 1 1 1 invertible! In other words, we cannot say that (A + B)− = A− + B− .
How do we compute the inverse of a matrix, if it exists?
Lecture 7 Math 40, Spring ’12, Prof. Kindred Page 3 Inverse of a 2 2 matrix: Consider the special case where A is a 2 2 × × matrix with A =[ab]. If ad bc =0,thenA is invertible and its inverse is cd − 1 1 d b A− = − . ad bc ca − −
1 10 1 Exercise: Check that AA− =[01]=A− A. 21 Example For A = − ,wehave 3 3 − 1 1 1 3 1 1 3 A− = − − = − − . 3 3 2 1 2 − − − −3 We can easily check that 1 1 21 1 3 10 AA− = − − − = 3 3 1 2 01 − − −3 and 1 1 1 3 21 10 A− A = − − − = . 1 2 3 3 01 − −3 −
How do we find inverses of matrices that are larger than 2 2matrices? ×
Theorem. If some EROs reduce a square matrix A to the identity matrix 1 I, then the same EROs transform I to A− .
EROs -1 A I I A
1 If we can transform A into I,thenwewillobtainA− .Ifwecannotdoso, then A is not invertible.
Lecture 7 Math 40, Spring ’12, Prof. Kindred Page 4 1 31 Example: Find the inverse of the matrix A = −360− . 101 1 31100 1 31100 − − R2+3R1 − − 360010 0 33310 −−−−→R3+R1 − 101001 0 32101 − 13 1 100 R1 − − − 0 33 310 −−R3 − −→R2 − − 00 1 2 11 − − − 10221 0 R +R 1 2 0 3331 0 −− −R −→3 − − 00121 1 − 10 2 210 1R −3 2 01 1 1 1 0 −− −→ − − −3 00 1 21 1 − 100 2 12 R1 2R3 − − − 2 010 1 3 1 −−−−→R2+R3 − 001 21 1 − Thus, A is invertible and its inverse is 2 12 1 − −2 A− = 1 1 . 3 − 21 1 −
Why does this work? = discussion next class ⇒
Lecture 7 Math 40, Spring ’12, Prof. Kindred Page 5