Matrix Algebra I N THIS APPENDIX We Review Important Concepts From
Total Page:16
File Type:pdf, Size:1020Kb
ApPENDIX A I Matrix Algebra IN THIS APPENDIX we review important concepts from matrix algebra. A.I EIGENVALUES AND EIGENVECTORS Definition A.I LetA be an m X m matrix and x a nonzero m-dimensional real or complex vector. If Ax = Ax for some real or complex number A, then A is called an eigenvalue of A and x the corresponding eigenvector. For example, the matrix A = (~ ~) has an eigenvector (~), and cor responding eigenvalue 4. 557 MATRIX ALGEBRA Eigenvalues are found as roots A of the characteristic polynomial det(A - AI). If A is an eigenvalue of A, then any nonzero vector in the nullspace of A - AI is an eigenvector corresponding to A. For this example, det(A - AI) = det e; A 2 ~ A) = (A - l)(A - 2) - 6 = (A - 4)(A + 1), (A1) so the eigenvalues are A = 4, -1. The eigenvectors corresponding to A = 4 are found in the nullspace of = (A2) A- 41 (-32 -23) and so consist of all nonzero multiples of ( ~ ) . Similarly, the eigenvectors corre sponding to A = -1 are all nonzero multiples of ( _ ~ ) . If A and Bare m X m matrices, then the set of eigenvalues of the product matrix AB is the same as the set of eigenvalues of the product BA In fact, let A be an eigenvalue of AB, so that ABx = Ax for some x '* O. If A = 0, then 0= det(AB - AI) = det(AB) = (detA)(detB) = det(BA - AI), so A = 0 is also an eigenvalue of BA If A '* 0, then ABx = Ax implies BA(Bx) = ABx. Note that Bx '* 0 because A '* 0 and x '* O. Therefore Bx is an eigenvector of the matrix BA with eigenvalue A. A generalization of this fact is Lemma A2. Lemma A.2 If AI, ... , An are m X m matrices, then Al ... An and the cyclic permutation A,.+ I ... AnAl' .. AT have the same set of eigenvalues, where 1 ~ r ~ n. This follows from the previous paragraph by setting A = Al ... AT and B = A,.+I ... An· Definition A.3 The m X m matrices Al and Az are similar, denoted Al ~ Az, ifthere exists an invertiblem X m matrix S such that Al = SAzS-I. Similar matrices have identical eigenvalues, because their characteristic polynomials are identical: (A.3) 558 A.I EIGENVALUES AND EIGENVECTORS implies that det{AI - AI) = (detS)det{Az - AI)detS-1 = det{Az - AI). (AA) If a matrix A has eigenvectors that form a basis for Rm, then A is similar to a diagonal matrix, and A is called diagonalizable. In fact, assume Axj = Ajxj for i = 1, ... , m, and define the matrix Then one can check that the matrix equation AI AS = S ( J (A.S) holds. The matrix S is invertible because its columns span Rm. Therefore A is similar to the diagonal matrix containing its eigenvalues. Not all matrices are diagonalizable, even in the 2 X 2 case. In fact, all 2 X 2 matrices are similar to one of the following three types: 1. Al = (~ ~) 2. Az = (~ !) 3. A3 = (: -!) Remember that eigenvalues are identical for similar matrices. A matrix is similar to case 1 if there are two eigenvectors that span ~z; a matrix is similar to case 2 if there is a repeated eigenvalue with only one dimensional space of eigenvectors; and to case 3 if it has a complex pair of eigenvalues. The proof of the fact that the three types suffice for 2 X 2 matrices fol lows from the Cayley-Hamilton Theorem, which states that a matrix satisfies its characteristic equation. Theorem A.4 (Cayley-Hamilton) If P{A) is the characteristic polynomial of the matrix A, then P{A) = 0 (as a matrix equation). SS9 MATRIX ALGEBRA There are three parts to the above classification. Let A be a 2 X 2 real matrix. Fact 1. If A has real distinct eigenvalues a and b, or if A = aI, then A ~ (~ ~ ) . Proof: If A = aI we are done and a = b. If the eigenvalues are distinct, then A is diagonalizable. To see this, choose eigenvectors v and x satisfying Av = av and Ax = bx. Note that v and x are not multiples of one another since a =/= b, so that the matrix whose columns are v and x is invertible. Then Now A is of form SDS-l, where D = diag{a, b}. Fact 2. If A has a repeated eigenvalue A a and A =/= aI, then A ~ 1 = ( ao a). Proof: (A - aI)2 = O. Since A =/= aI, there exists a vector x such that v = (A - aI)x =/= O. Then (A - aI)v = (A - aI) 2x = 0, so v is an eigenvector of A. Note that v and x are not linearly dependent, since v is an eigenvector of A and x is not. The facts Ax = ax + v and Av = av can be written Fact 3. If A has eigenvalues a ± bi, with b =/= 0, then A ~ (~ -~). Proof: (A - (a + bi)I)(A - (a - bi)I) = 0 can be rewritten as (A - aI)2 = -b2I. Let x be a (real) nonzero vector and define v = t(A - aI)x, so that (A - aI)v = -bx. Since b =/= 0, v and x are not linearly dependent because x is not an eigenvector of A. The equations Ax = bv + ax and Av = av - bx can be rewritten The similarity equivalence classes for m > 2 become a little more com plicated. Look up Jordan canonical form in a linear algebra book to investigate further. 560 A.2 COORDINATE CHANGES A.2 COORDINATE CHANGES We work in two dimensions, although almost everything we say extends to higher dimensions with minor changes. A vector in ~2 can be represented in many dif ferent ways, depending on the coordinate system chosen. Choosing a coordinate system is equivalent to choosing a basis of ~2; then the coordinates of a vector are simply the coefficients that express the vector in that basis. Consider the standard basis and another basis The coefficients of a general vector (:~) are Xl and X2 in the basis B 1, and are Xl - X2 and X2 in the basis Bz. This is because or in matrix terms, (A.7) This gives us a convenient rule of thumb. To get coordinates of a vector in the second coordinate system, multiply the original coordinates by the matrix S-l, where S is a matrix whose columns are the coordinates of the second basis vectors written in terms of the first basis. Therefore in retrospect, we could have computed (A.8) as the coordinates, in the second coordinate system Bz, of the vector (:~) in the original coordinate system B1 . For example, the new coordinates of ( ~) are 561 MATRIX ALGEBRA S-I G) ( ~ ) , which is equivalent to the statement that (A.9) Now let F be a linear map on !R2• For a fixed basis (coordinate system), we find a matrix representation for F by building a matrix whose columns are the images of the basis vectors under F, expressed in that coordinate system. For example, let F be the linear map that reflects vectors through the diagonal line y = x. In the coordinate system BI the map F is represented by A= (0 1) (A. 10) I 1 ° since F (~) = (~) and F (~) = (~). In the second coordinate system Bz, the vector F (~) = (~) has coordinates and F fixes the other basis vector (~), so the map F is represented by = (A.ll) A2 (-11 0).1 The matrix representation of the linear map F depends on the coordinate system being used. What is the relation between the two representations? If x is a vector in the first coordinate system, then S-Ix gives the coordinates in the second system. Then A2S- I X applies the map F, and SA2S-lx returns to the original coordinate system. Since we could more directly accomplish this by multiplying x by AI> we have discovered that (A.12) or in other words, that Al and A2 are similar matrices. Thus similar matrices are those that represent the same map in different coordinate systems. 562 A.3 MATRIX TIMES CIRCLE EQUALS ELLIPSE AJ MATRIX TIMES CIRCLE EQUALS ELLIPSE In this section we show that the image of the unit sphere in IRm under a linear map is an ellipse, and we show how to find that ellipse. Definition A.5 Let A be an m X n matrix. The transpose of A, denoted AT, is the n X m matrix formed by changing the rows of A into columns. A square matrix is symmetric if AT = A. In terms of matrix entries, AJ = Aji• In particular, if the matrices are column vectors x and y, then x T y, using standard matrix multiplication, is the dot product, or scalar product, of x and y. It can be checked that (AB)T = BT AT. It is a standard fact found in elementary matrix algebra books that for any symmetric m X m matrix A with real entries, there is an orthonormal eigenbasis, meaning that there exist m real-valued eigenvectors WI, ... , Wm of A satisfying wTwj = °ifi * j, and WTWi = 1, for 1:::; i,j:::; m. Now assume that A is an m X m matrix that is not necessarily symmetric. The product AT A is symmetric (since (ATA)T = AT(AT)T = ATA), so it has an orthogonal basis of eigenvectors. It turns out that the corresponding eigenvalues must be nonnegative. Lemma A.6 Let A be an m X m matrix. The eigenvalues of AT A are nonnegative. Proof: Let v be a unit eigenvector of AT A, and AT Av = Av.