More Linear Algebra1 Chris Godsil Combinatorics & Optimization University of Waterloo ©2019 1version: January 29, 2021 ii Preface These notes are meant to provide an introduction to fundamental parts of lin- ear algebra, as might be applied to problems in combinatorics. I assume the reader has had a first course in linear algebra, and is familiar with determinants. iii To Do 1. Interlacing, via Courant-Fischer and by rational functions. Need equi- table partitions for this. 2. Walk modules. Controllable graphs. 3. Quadrics. 4. Matrix exponential. 5. Lie algebras, sl(2) repns. 6. Perron-Frobenius, symbolic dynamics. 7. Perturbation theory. iv Contents 1 Spaces and Subspaces 1 1.1 Vector Spaces . 1 1.2 Subspaces . 3 1.3 Linear Mappings . 4 1.4 Duals and Adjoints . 6 1.5 Bilinear Forms . 8 1.6 Counting . 9 1.7 Normal Forms . 11 1.8 Groebner Bases . 13 1.9 Codes . 15 2 Primary Decomposition 19 2.1 Modules . 19 2.2 Control Theory . 20 2.3 Sums .................................... 22 2.4 Invariant Sums . 23 2.5 Minimal Polynomials . 25 2.6 Primary Decomposition . 27 2.7 The Degree of the Minimal Polynomial . 29 2.8 Root Spaces . 30 2.9 Examples of Root Spaces . 31 2.10 Differential Equations . 33 2.11 Linear Recurrence Equations . 35 2.12 Diagonalizability . 36 3 Frobenius Normal Form 39 3.1 Companion Matrices . 39 3.2 Transposes . 41 v vi CONTENTS 3.3 Eigenvectors for Companion Matrices . 42 3.4 Inverses of Companion Matrices . 44 3.5 Cycles.................................... 46 3.6 Circulants and Cyclic Codes . 47 3.7 Frobenius Normal Form . 49 3.8 Applications . 52 3.9 Nilpotent Matrices . 53 3.10 A Similarity Condition . 55 3.11 Triangular Maps . 57 3.12 Triangulations . 59 3.13 The “Fundamental” “Theorem of Algebra” . 59 3.14 The Kronecker Product . 63 4 Orthogonality 67 4.1 Properties of Projections . 67 4.2 Matrices Representing Projections . 68 4.3 Least Squares . 70 4.4 Orthogonal Polynomials . 71 4.5 The Three-Term Recurrence . 72 4.6 Numerical Integration . 73 5 Eigenthings 77 5.1 Self-Adjoint Operators . 77 5.2 Diagonalizability . 78 5.3 Diagonalizability, Again . 79 5.4 Eigenvectors and Optimization . 80 5.5 The Singular Value Decomposition . 81 5.6 Least Squares . 83 5.7 Legendre Polynomials . 85 5.8 Computing Eigenvalues . 87 5.9 Jacobi: An Example . 89 6 Spectral Decomposition 91 6.1 Self-Adjoint Operators . 91 6.2 Commutative Algebras . 92 6.3 Normal Operators . 94 CONTENTS vii 7 Norms 97 7.1 Convexity . 97 7.2 Extreme Points . 99 7.3 Norms . 100 7.4 Dual Norms . 102 7.5 Matrix Norms . 103 7.6 Examples . 105 7.7 Matrix Functions . 106 7.8 Powers ................................... 108 7.9 Contractions . 110 7.10 Projections . 112 7.11 Contractions . 114 7.12 Perron . 116 8 Geometry 121 8.1 Semilinear Forms . 121 8.2 The Classification of Forms . 122 8.3 Gram Matrices . 123 8.4 Equiangular Lines . 124 8.5 Tight Frames . 125 8.6 Another Gram Matrix . 126 8.7 The Orthogonal Group . 127 8.8 Skew-Symmetric to Orthogonal . 128 8.9 Reflections . 129 9 Positive Semidefinite Matrices 133 9.1 Factorizing Positive Semidefinite Matrices . 133 9.2 Computing Cholesky . 135 9.3 Polynomial Examples . 136 9.4 Positive Semidefinite Matrices . 138 10 Tensors 141 10.1 Tensor Products . 141 10.2 Quadratic Tensors . 142 10.3 Cubic Tensors . 144 10.4 Multiplication . 146 10.5 Semifields . 147 viii CONTENTS 11 Control 149 11.1 Buffalos . 149 11.2 Burgers . 151 11.3 Controllability . 152 11.4 Observability . 155 11.5 Feedback and Controllability . 156 11.6 Canonical Forms . 158 11.7 Eigenvalues and Controllability . 159 11.8 Observers . 161 11.9 Transfer Matrices . 162 12 The Smith Normal Form 165 12.1 Domains . 165 12.2 Localization . 167 12.3 Fitting . 168 12.4 Hermite . 169 12.5 Smith Normal Form . 171 13 Polynomial and Rational Matrices 175 13.1 Series . 176 13.2 Polynomial Matrices . 177 13.3 Paraunitary Matrices . 180 13.4 Division . 182 13.5 Cayley-Hamilton . 183 13.6 Greatest Common Divisors . 185 13.7 An Identity . 187 13.8 Resolvents . 189 14 Determinants 195 14.1 Permutations . 195 14.2 The Sign of a Permutation . 197 14.3 Permutation Matrices . 198 14.4 Definition of the Determinant . ..