Discover Linear Algebra Incomplete Preliminary Draft Date: November 28, 2017 L´aszl´oBabai in collaboration with Noah Halford All rights reserved. Approved for instructional use only. Commercial distribution prohibited. c 2016 L´aszl´oBabai. Last updated: November 10, 2016 Preface TO BE WRITTEN. Babai: Discover Linear Algebra. ii This chapter last updated August 21, 2016 c 2016 L´aszl´oBabai. Contents Notation ix I Matrix Theory 1 Introduction to Part I 2 1 (F, R) Column Vectors 3 1.1 (F) Column vector basics . 3 1.1.1 The domain of scalars . 4 1.2 (F) Subspaces and span . 8 1.3 (F) Linear independence and the First Miracle of Linear Algebra . 11 1.4 (F) Dot product . 16 1.5 (R) Dot product over R ................................. 18 1.6 (F) Additional exercises . 19 2 (F) Matrices 20 2.1 Matrix basics . 20 2.2 Matrix multiplication . 24 2.3 Arithmetic of diagonal and triangular matrices . 31 2.4 Permutation Matrices . 33 2.5 Additional exercises . 36 3 (F) Matrix Rank 39 3.1 Column and row rank . 39 iii iv CONTENTS 3.2 Elementary operations and Gaussian elimination . 40 3.3 Invariance of column and row rank, the Second Miracle of Linear Algebra . 43 3.4 Matrix rank and invertibility . 45 3.5 Codimension (optional) . 47 3.6 Additional exercises . 48 4 (F) Theory of Systems of Linear Equations I: Qualitative Theory 51 4.1 Homogeneous systems of linear equations . 51 4.2 General systems of linear equations . 54 5 (F, R) Affine and Convex Combinations (optional) 56 5.1 (F) Affine combinations . 56 5.2 (F) Hyperplanes . 60 5.3 (R) Convex combinations . 60 5.4 (R) Helly's Theorem . 62 5.5 (F, R) Additional exercises . 63 6 (F) The Determinant 64 6.1 Permutations . 65 6.2 Defining the determinant . 68 6.3 Cofactor expansion . 73 6.4 Matrix inverses and the determinant . 75 6.5 Additional exercises . 76 7 (F) Theory of Systems of Linear Equations II: Cramer's Rule 77 7.1 Cramer's Rule . 77 8 (F) Eigenvectors and Eigenvalues 79 8.1 Eigenvector and eigenvalue basics . 79 8.2 Similar matrices and diagonalizability . 82 8.3 Polynomial basics . 83 8.4 The characteristic polynomial . 84 8.5 The Cayley-Hamilton Theorem . 87 8.6 Additional exercises . 89 CONTENTS v 9 (R) Orthogonal Matrices 90 9.1 Orthogonal matrices . 90 9.2 Orthogonal similarity . 91 9.3 Additional exercises . 92 10 (R) The Spectral Theorem 93 10.1 Statement of the Spectral Theorem . 93 10.2 Applications of the Spectral Theorem . 94 11 (F, R) Bilinear and Quadratic Forms 96 11.1 (F) Linear and bilinear forms . 96 11.2 (F) Multivariate polynomials . 98 11.3 (R) Quadratic forms . 100 11.4 (F) Geometric algebra (optional) . 103 12 (C) Complex Matrices 106 12.1 Complex numbers . 106 12.2 Hermitian dot product in Cn .............................. 107 12.3 Hermitian and unitary matrices . 109 12.4 Normal matrices and unitary similarity . 110 12.5 Additional exercises . 112 13 (C, R) Matrix Norms 113 13.1 (R) Operator norm . 113 13.2 (R) Frobenius norm . 114 13.3 (C) Complex Matrices . 115 II Linear Algebra of Vector Spaces 116 Introduction to Part II 117 14 (F) Preliminaries 118 14.1 Modular arithmetic . 118 14.2 Abelian groups . 123 14.3 Fields . 126 vi CONTENTS 14.4 Polynomials . 127 15 (F) Vector Spaces: Basic Concepts 138 15.1 Vector spaces . 138 15.2 Subspaces and span . 141 15.3 Linear independence and bases . 143 15.4 The First Miracle of Linear Algebra . 147 15.5 Direct sums . 150 16 (F) Linear Maps 152 16.1 Linear map basics . 152 16.2 Isomorphisms . 154 16.3 The Rank-Nullity Theorem . 155 16.4 Linear transformations . 156 16.4.1 Eigenvectors, eigenvalues, eigenspaces . 158 16.4.2 Invariant subspaces . 159 16.5 Coordinatization . 163 16.6 Change of basis . 166 17 (F) Block Matrices (optional) 168 17.1 Block matrix basics . 168 17.2 Arithmetic of block-diagonal and block-triangular matrices . 170 18 (F) Minimal Polynomials of Matrices and Linear Transformations (optional) 172 18.1 The minimal polynomial . 172 18.2 Minimal polynomials of linear transformations . 174 19 (R) Euclidean Spaces 177 19.1 Inner products . 177 19.2 Gram-Schmidt orthogonalization . 181 19.3 Isometries and orthogonal transformations . 183 19.4 First proof of the Spectral Theorem . 186 20 (C) Hermitian Spaces 189 20.1 Hermitian spaces . 189 20.2 Hermitian transformations . 194 CONTENTS vii 20.3 Unitary transformations . 195 20.4 Adjoint transformations in Hermitian spaces . 196 20.5 Normal transformations . 196 20.6 The Complex Spectral Theorem for normal transformations . 197 21 (R, C) The Singular Value Decomposition 198 21.1 The Singular Value Decomposition . 198 21.2 Low-rank approximation . 199 22 (R) Finite Markov Chains 202 22.1 Stochastic matrices . 202 22.2 Finite Markov Chains . 203 22.3 Digraphs . 205 22.4 Digraphs and Markov Chains . 205 22.5 Finite Markov Chains and undirected graphs . 205 22.6 Additional exercises . 206 23 More Chapters 207 24 Hints 208 24.1 Column Vectors . ..