Linear Algebra
Total Page:16
File Type:pdf, Size:1020Kb
Linear Algebra E. Kowalski ETH Zurich¨ { Fall 2015 and Spring 2016 Version of September 15, 2016 [email protected] Contents Chapter 1. Preliminaries1 Chapter 2. Vector spaces and linear maps2 2.1. Matrices and vectors2 2.2. Matrix products3 2.3. Vector spaces and linear maps6 2.4. Subspaces 11 2.5. Generating sets 15 2.6. Linear independence and bases 18 2.7. Dimension 20 2.8. Properties of dimension 24 2.9. Matrices and linear maps 27 2.10. Solving linear equations 34 2.11. Applications 47 Chapter 3. Determinants 54 3.1. Axiomatic characterization 54 3.2. Example 57 3.3. Uniqueness and existence of the determinant 58 3.4. Properties of the determinant 61 3.5. The Vandermonde determinant 65 3.6. Permutations 67 Chapter 4. Endomorphisms 72 4.1. Sums and direct sums of vector spaces 72 4.2. Endomorphisms 77 4.3. Eigenvalues and eigenvectors 81 4.4. Some special endomorphisms 90 Chapter 5. Euclidean spaces 94 5.1. Properties of the transpose 94 5.2. Bilinear forms 94 5.3. Euclidean scalar products 98 5.4. Orthogonal bases, I 102 5.5. Orthogonal complement 106 5.6. Adjoint, I 108 5.7. Self-adjoint endomorphisms 110 5.8. Orthogonal endomorphisms 112 5.9. Quadratic forms 116 5.10. Singular values decomposition 120 Chapter 6. Unitary spaces 123 6.1. Hermitian forms 123 ii 6.2. Orthogonal bases, II 131 6.3. Orthogonal complement, II 134 6.4. Adjoint, II 136 6.5. Unitary endomorphisms 139 6.6. Normal and self-adjoint endomorphisms, II 141 6.7. Singular values decomposition, II 145 Chapter 7. The Jordan normal form 147 7.1. Statement 147 7.2. Proof of the Jordan normal form 154 Chapter 8. Duality 164 8.1. Dual space and dual basis 164 8.2. Transpose of a linear map 170 8.3. Transpose and matrix transpose 173 Chapter 9. Fields 175 9.1. Definition 175 9.2. Characteristic of a field 177 9.3. Linear algebra over arbitrary fields 178 9.4. Polynomials over a field 179 Chapter 10. Quotient vector spaces 183 10.1. Motivation 183 10.2. General definition and properties 186 10.3. Examples 188 Chapter 11. Tensor products and multilinear algebra 195 11.1. The tensor product of vector spaces 195 11.2. Examples 200 11.3. Exterior algebra 207 Appendix: dictionary 215 iii CHAPTER 1 Preliminaries We assume knowledge of: ‚ Proof by induction and by contradiction ‚ Complex numbers ‚ Basic set-theoretic definitions and notation ‚ Definitions of maps between sets, and especially injective, surjective and bijective maps ‚ Finite sets and cardinality of finite sets (in particular with respect to maps between finite sets). We denote by N “ t1; 2;:::u the set of all natural numbers. (In particuler, 0 R N). 1 CHAPTER 2 Vector spaces and linear maps Before the statement of the formal definition of a field, a field K is either Q, R, or C. 2.1. Matrices and vectors Consider a system of linear equations a11x1 ` ¨ ¨ ¨ ` a1nxn “ b1 $¨ ¨ ¨ &'am1x1 ` ¨ ¨ ¨ ` amnxn “ bm with n unknowns px1; : : : ; xn%'q in K and coefficients aij in K, bi in K. It is represented concisely by the equation fApxq “ b where A “ paijq1¤i¤m is the matrix 1¤j¤n a11 ¨ ¨ ¨ a1n A “ ¨ ¨ ¨ ¨ ¨ ¨ ¨ ¨ ¨ ¨ ˛ am1 ¨ ¨ ¨ amn ˝ ‚ with m rows and n columns, b is the column vector b1 . b “ . ; ¨ ˛ bm ˝ ‚ n m x is the column vector with coefficients x1,..., xn, and fA is the map fA : K Ñ K defined by x1 a x ` ¨ ¨ ¨ ` a x . 11 1 1n n fA . “ ¨ ¨ ¨ : ¨ ˛ ¨a x ` ¨ ¨ ¨ ` a x ˛ ´ xn ¯ m1 1 mn n ˝ ‚ ˝ ‚ We use the notation Mm;npKq for the set of all matrices with m rows and n columns n and coefficients in K, and K or Mn;1pKq for the set of all columns vectors with n rows and coefficients in K. We will also use the notation Kn for the space of row vectors with n columns. We want to study the equation fApxq “ b by composing with other maps: if fApxq “ b, m then gpfApxqq “ gpbq for any map g defined on K . If g is bijective, then conversely if ´1 gpfApxqq “ gpbq, we obtain fApxq “ b by applying the inverse map g to both sides. We do this with g also defined using a matrix. This leads to matrix products. 2 2.2. Matrix products Theorem 2.2.1. Let m, n, p be natural numbers. Let A P Mm;npKq be a matrix with m rows and n columns, and let B P Mp;mpKq be a matrix with p rows and m columns. Write A “ paijq1¤i¤m;B “ pbkiq1¤k¤p : 1¤j¤n 1¤i¤m Consider the map f obtained by composition f f Kn ÝÑA Km ÝÑB Kp; that is, f “ fB ˝ fA. Then we have f “ fC where C P Mp;npKq is the matrix C “ pckjq1¤k¤p with 1¤j¤n ckj “ bk1a1j ` bk2a2j ` ¨ ¨ ¨ ` bkmamj m “ bkiaij: i“1 ¸ n Proof. Let x “ pxjq1¤j¤n P K . We compute fpxq and check that this is the same as fC pxq. First we get by definition fApxq “ y; where y “ pyiq1¤i¤m is the row vector such that n yi “ ai1x1 ` ¨ ¨ ¨ ` ainxn “ aijxj: j“1 ¸ Then we get fpxq “ fBpyq, which is the row vector pzkq1¤k¤p with m zk “ bk1y1 ` ¨ ¨ ¨ ` bkmym “ bkiyi: i“1 ¸ Inserting the value of yi in this expression, this is m n n zk “ bki aijxj “ ckjxj i“1 j“1 j“1 ¸ ¸ ¸ where m ckj “ bk1a1j ` ¨ ¨ ¨ ` bkmamj “ bkiaij: i“1 ¸ a b x y Exercise 2.2.2. Take A “ and B “ and check the computation c d z t completely. ˆ ˙ ˆ ˙ Definition 2.2.3 (Matrix product). For A and B as above, the matrix C is called the matrix product of B and A, and is denoted C “ BA. n Example 2.2.4. (1) For A P Mm;npKq and x P K , if we view x as a column vector with n rows, we can compute the product Ax corresponding to the composition f f K ÝÑx Kn ÝÑA Km: 3 Using the formula defining fx and fA and the matrix product, we see that a11x1 ` ¨ ¨ ¨ ` a1nxn Ax “ ¨ ¨ ¨ “ fApxq: ¨ ˛ am1x1 ` ¨ ¨ ¨ ` amnxn ˝ ‚ This means that fA can also be interpreted as the map that maps a vector x to the matrix product Ax. (2) Consider B P Mk;mpKq, A P Mm;npKq. Let C “ BA, and write A “ paijq, B “ pbkjq, C “ pckiq. Consider an integer j, 1 ¤ j ¤ n and let Aj be the j-th column of A: a1j . Aj “ . : ¨ ˛ anj ˝ ‚ k We can then compute the matrix product BAj, which is an element of Mk;1pKq “ K : b11a1j ` b12a2j ` ¨ ¨ ¨ ` b1nanj . BAj “ . : ¨ ˛ bp1a1j ` bp2a2j ` ¨ ¨ ¨ ` bpnanj Comparing with the definition of˝ C, we see that ‚ c1j . BAj “ . ¨ ˛ cpj is the j-th column of C. So the columns of the˝ matrix‚ product are obtained by products of matrices with column vectors. Proposition 2.2.5 (Properties of the matrix product). (1) Given positive integers m, n and matrices A and B in Mm;npKq, we have fA “ fB if and only if A “ B. In n m particular, if a map f : K Ñ K is of the form f “ fA for some matrix A P Mm;npKq, then this matrix is unique. (2) Given positive integers m, n, p and q, and matrices A “ paijq1¤i¤m;B “ pbkiq1¤k¤p ;C “ pclkq 1¤l¤q ; 1¤j¤n 1¤i¤m 1¤k¤p defining maps f f f Kn ÝÑA Km ÝÑB Kp ÝÑC Kq; we have the equality of matrix products CpBAq “ pCBqA: In particular, for any n ¥ 1, the product of matrices is an operation on Mn;npKq (the product of matrices A and B which have both n rows and n columns is a matrix of the same size), and it is associative: ApBCq “ pABqC for all matrices A, B, C in Mn;npKq. Proof. (1) For 1 ¤ i ¤ n, consider the particular vector ei with all coefficients 0, except that the i-th coefficient is 1: 1 0 0 . 0 1 . e1 “ ¨.˛ ; e2 “ ¨ ˛ ; ¨ ¨ ¨ ; en “ ¨ ˛ : . 0 . 0 ˚0‹ ˚.‹ ˚1‹ ˚ ‹ ˚ ‹ ˚ ‹ ˝ ‚ ˝ ‚4 ˝ ‚ Computing the matrix product fApeiq “ Aei, we find that a1i . (2.1) fApeiq “ . ; ¨ ˛ ami ˝ ‚ which is the i-th column of the matrix A. Therefore, if fA “ fB, the i-column of A and B are the same (since fApeiq “ fBpeiq), which means that A and B are the same (since this is true for all columns). (3) Since composition of maps is associative, we get pfC ˝ fBq ˝ fA “ fC ˝ pfB ˝ fAq; or fCB ˝ fA “ fC ˝ fBA, or even fpCBqA “ fCpBAq, which by (1) means that pCBqA “ CpBAq. Exercise 2.2.6. Check directly using the formula for the matrix product that CpBAq “ pCBqA. Now we define two additional operations on matrices and vector: (1) addition; (2) multiplication by an element t P K. Definition 2.2.7 (Operations and special matrices). (1) For m, n natural numbers and A “ paijq1¤i¤m, B “ pbijq1¤i¤m matrices in Mm;npKq, the sum A ` B is the matrix 1¤j¤n 1¤j¤n A ` B “ paij ` bijq1¤i¤m P Mm;npKq: 1¤j¤n (2) For t P K, for m, n natural numbers and A “ paijq1¤i¤m a matrix in Mm;npKq, 1¤j¤n the product tA is the matrix tA “ ptaijq1¤i¤m P Mm;npKq: 1¤j¤n (3) For m, n natural numbers, the zero matrix 0mn is the matrix in Mm;npKq with all coefficients 0.