Matrix Analysis (Lecture 1) Yikun Zhang∗ March 29, 2018 Abstract Linear algebra and matrix analysis have long played a fundamen- tal and indispensable role in fields of science. Many modern research projects in applied science rely heavily on the theory of matrix. Mean- while, matrix analysis and its ramifications are active fields for re- search in their own right. In this course, we aim to study some fun- damental topics in matrix theory, such as eigen-pairs and equivalence relations of matrices, scrutinize the proofs of essential results, and dabble in some up-to-date applications of matrix theory. Our lecture will maintain a cohesive organization with the main stream of Horn's book[1] and complement some necessary background knowledge omit- ted by the textbook sporadically. 1 Introduction We begin our lectures with Chapter 1 of Horn's book[1], which focuses on eigenvalues, eigenvectors, and similarity of matrices. In this lecture, we re- view the concepts of eigenvalues and eigenvectors with which we are familiar in linear algebra, and investigate their connections with coefficients of the characteristic polynomial. Here we first outline the main concepts in Chap- ter 1 (Eigenvalues, Eigenvectors, and Similarity). ∗School of Mathematics, Sun Yat-sen University 1 Matrix Analysis and its Applications, Spring 2018 (L1) Yikun Zhang 1.1 Change of basis and similarity (Page 39-40, 43) Let V be an n-dimensional vector space over the field F, which can be R, C, or even Z(p) (the integers modulo a specified prime number p), and let the list B1 = fv1; v2; :::; vng be a basis for V. Any vector x 2 V can be represented as x = α1v1 +···+αnvn because B1 spans V . This representation of x in B1 is unique, since if there were some other representation of x = β1v1 +···+βnvn in the same basis, then 0 = x − x = (α1 − β1)v1 + ··· + (αn − βn)vn from which it follows that αi − βi = 0 because the list B1 is independent. Thus, given the basis B1, the linear mapping 2 3 α1 x ! [x] = 6 . 7 ; in which x = α v + ··· + α v B1 4 . 5 1 1 n n αn from V to Fn is well-defined, one-to-one, and onto. The scalars αi are called the coordinates of x with respect to the basis B1, and the column vector [x]B1 is the unique B1-coordinate representation of x. We now move on to talk about linear transformations on different bases and properties of change-of-basis matrices. Let T : V ! V be a given linear transformation. The action of T on any x 2 V is determined by its action on the basis B1, i.e., the n vectors T v1; :::; T vn. This is because any x 2 V has a unique representation x = α1v1 + ··· + αnvn and T x = T (α1v1 + ··· + αnvn) = α1T v1 + ··· + αnT vn by linearity of T . Thus the value of T x is clear once [x]B1 is known. What happen if we change the basis of V ? Can we explicitly uncover the representation of T in terms of two given bases? Let B2 = fw1; :::; wng also be a basis for V (either different from or the same as B1) and suppose that the B2-coordinate representation of T vj is 2 3 t1j [T v ] = 6 . 7 ; where j = 1; 2; :::; n: j B2 4 . 5 tnj 2 Matrix Analysis and its Applications, Spring 2018 (L1) Yikun Zhang Then, for any x 2 V , we have 2 3 2 3 2 3 n n n t1j t11 : : : t1n α1 X X X [T x] = α T v = α [T v ] = α 6 . 7 = 6 . .. 7 6 . 7 B2 j j j j B2 j 4 . 5 4 . 5 4 . 5 j=1 B2 j=1 j=1 tnj tn1 : : : tnn αn What we have shown is that [T x]B2 =B2 [T ]B1 [x]B1 . The n-by-n array [tij] depends on T and on the choice of the bases B1 and B2, but it does not depend on x. Definition 1.1. The B1-B2 basis representation of T is defined to be 2 3 t11 : : : t1n [T ] = 6 . .. 7 = [T v ] ::: [T v ] : B2 B1 4 . 5 1 B2 n B2 tn1 : : : tnn In the important special case where B2 = B1, we have B1 [T ]B1 , which is called the B1 basis representation of T . Consider the identity linear transformation I : V ! V defined by Ix = x for all x. Definition 1.2. The matrix B2 [I]B1 is called the B1 − B2 change of basis matrix, where I is the identity linear transformation. Now we are well-prepared to prove the main result of this section, which is stated as a proposition. Proposition 1.3. Every invertible matrix is a change-of-basis matrix, and every change-of-basis matrix is invertible. Proof. Given the identity transformation I : V ! V , [x]B2 = [Ix]B2 = B2 [I]B1 [x]B1 = B2 [I]B1 [Ix]B1 = B2 [I]B1 B1 [I]B2 [x]B2 for all x 2 V . By successively choosing x = w1; :::; wn and using the fact that 203 . 6.7 6.7 [w ] = 617 i; where i = 1; 2; :::; n; i B2 6 7 6.7 4.5 0 3 Matrix Analysis and its Applications, Spring 2018 (L1) Yikun Zhang this identity permits us to identify each column of B2 [I]B1 B1 [I]B2 and shows that B2 [I]B1 B1 [I]B2 = In Similarly, if we do the same computation starting with [x]B1 = [Ix]B1 = ··· , we obtain that B1 [I]B2 B2 [I]B1 = In Thus, every matrix of the form B2 [I]B1 is invertible and B1 [I]B2 is its inverse. Conversely, every invertible matrix S = [s1 s2 : : : sn] 2 Mn(F) has the form B1 [I]B for some basis B. We may take B to be the vectors fs~1; :::; s~ng defined by 203 . 6.7 6.7 [~s ] = [I] [~s ] = s s ··· s 617 = s ; where i = 1; 2; :::; n: i B1 B1 B i B 1 2 n 6 7 i 6.7 4.5 0 The list B is independent, because if a1s~1 + ··· + ans~n = 0, then 2 3 a1 0 = [0] = [a s~ + ··· + a s~ ] = s ··· s 6 . 7 B1 1 1 n n B1 1 n 4 . 5 an Due to the fact that S is invertible, we find that ai = 0, i = 1; 2; :::; n. Notice that B2 [I]B1 = [Iv1]B2 ··· [Ivn]B2 = [v1]B2 ··· [vn]B2 ; so B2 [I]B1 describes how the elements of the basis B1 are represented in terms of elements of the basis B2. Now let x 2 V and compute B2 [T ]B2 [x]B2 = [T x]B2 = [I(T x)]B2 = B2 [I]B1 [T x]B1 = B2 [I]B1 B1 [T ]B1 [x]B1 = B2 [I]B1 B1 [T ]B1 [Ix]B1 = B2 [I]B1 B1 [T ]B1 B1 [I]B2 [x]B2 By choosing x = w1; :::; wn successively, we conclude that B2 [T ]B2 = B2 [I]B1 B1 [T ]B1 B1 [I]B2 (1) 4 Matrix Analysis and its Applications, Spring 2018 (L1) Yikun Zhang This identity shows how the B1 basis representation of T changes if the basis is changed to B2. That is why we define the matrix B2 [I]B1 to be the B1 − B2 change of basis matrix. Therefore, if B1 is a given basis of a vector space V , if T is a given linear transformation on V , and if A = B1 [T ]B1 is the B1 basis representation of T , the set of all possible basis representation of T is fB2 [I]B1 B1 [T ]B1 B1 [I]B2 : B2 is a basis of V g −1 = fS AS : S 2 Mn(F) is invertibleg This, at the same time, indicates the set of all matrices that are similar to the given matrix A. Remark. Similar but not identical matrices are just different basis repre- sentations of a single linear transformation. 1.2 Constrained extrema and eigenvalues (Page 43-44, Appendix E) Nonzero vectors x such that Ax is a scalar multiple of x play a major role in analyzing the structure of a matrix or linear transformation, but such vectors arise in the more elementary context of maximizing (or minimizing) a real symmetric quadratic form subject to a geometric constraint: For a given real symmetric A 2 Mn(R), maximize xT Ax; subject to x 2 Rn; xT x = 1 (2) Commonly, we introduce a Lagrangian multiplier to convert the constrained optimization problem into an unconstrained one, L(x) = xT Ax−λxT x, whose necessary conditions for an extremum are 0 = rL = 2(Ax − λx) = 0 Thus, if a vector x 2 Rn with xT x = 1 (and hence x 6= 0) is an extremum of xT Ax, it must satisfy the equation Ax = λx. Definition 1.4. Let A 2 Mn (Mn is the set of n-by-n matrices over C. We will inherit this notation in the rest of lectures). If a scalar λ and a nonzero vector x satisfy the equation Ax = λx, x 2 Cn; x 6= 0; λ 2 C (3) 5 Matrix Analysis and its Applications, Spring 2018 (L1) Yikun Zhang then λ is called an eigenvalue of A and x is called an eigenvector of A asso- ciated with λ. The pair λ, x is an eigenpair for A. Remark. It is a key element of the definition that an eigenvector can never be the zero vector.