Discover Linear Algebra Incomplete Preliminary Draft Date: November 28, 2017 L´aszl´oBabai in collaboration with Noah Halford All rights reserved. Approved for instructional use only. Commercial distribution prohibited. c 2016 L´aszl´oBabai. Last updated: November 10, 2016 Preface TO BE WRITTEN. Babai: Discover Linear Algebra. ii This chapter last updated August 21, 2016 c 2016 L´aszl´oBabai. Contents Notation ix I Matrix Theory 1 Introduction to Part I 2 1 (F, R) Column Vectors 3 1.1 (F) Column vector basics . 3 1.1.1 The domain of scalars . 3 1.2 (F) Subspaces and span . 6 1.3 (F) Linear independence and the First Miracle of Linear Algebra . 8 1.4 (F) Dot product . 12 1.5 (R) Dot product over R ................................. 14 1.6 (F) Additional exercises . 14 2 (F) Matrices 15 2.1 Matrix basics . 15 2.2 Matrix multiplication . 18 2.3 Arithmetic of diagonal and triangular matrices . 22 2.4 Permutation Matrices . 24 2.5 Additional exercises . 26 3 (F) Matrix Rank 28 3.1 Column and row rank . 28 iii iv CONTENTS 3.2 Elementary operations and Gaussian elimination . 29 3.3 Invariance of column and row rank, the Second Miracle of Linear Algebra . 31 3.4 Matrix rank and invertibility . 33 3.5 Codimension (optional) . 34 3.6 Additional exercises . 35 4 (F) Theory of Systems of Linear Equations I: Qualitative Theory 38 4.1 Homogeneous systems of linear equations . 38 4.2 General systems of linear equations . 40 5 (F, R) Affine and Convex Combinations (optional) 42 5.1 (F) Affine combinations . 42 5.2 (F) Hyperplanes . 44 5.3 (R) Convex combinations . 45 5.4 (R) Helly's Theorem . 46 5.5 (F, R) Additional exercises . 47 6 (F) The Determinant 48 6.1 Permutations . 49 6.2 Defining the determinant . 51 6.3 Cofactor expansion . 54 6.4 Matrix inverses and the determinant . 55 6.5 Additional exercises . 56 7 (F) Theory of Systems of Linear Equations II: Cramer's Rule 57 7.1 Cramer's Rule . 57 8 (F) Eigenvectors and Eigenvalues 59 8.1 Eigenvector and eigenvalue basics . 59 8.2 Similar matrices and diagonalizability . 61 8.3 Polynomial basics . 61 8.4 The characteristic polynomial . 63 8.5 The Cayley-Hamilton Theorem . 64 8.6 Additional exercises . 66 CONTENTS v 9 (R) Orthogonal Matrices 67 9.1 Orthogonal matrices . 67 9.2 Orthogonal similarity . 67 9.3 Additional exercises . 68 10 (R) The Spectral Theorem 69 10.1 Statement of the Spectral Theorem . 69 10.2 Applications of the Spectral Theorem . 69 11 (F, R) Bilinear and Quadratic Forms 71 11.1 (F) Linear and bilinear forms . 71 11.2 (F) Multivariate polynomials . 72 11.3 (R) Quadratic forms . 74 11.4 (F) Geometric algebra (optional) . 76 12 (C) Complex Matrices 78 12.1 Complex numbers . 78 12.2 Hermitian dot product in Cn .............................. 78 12.3 Hermitian and unitary matrices . 80 12.4 Normal matrices and unitary similarity . 81 12.5 Additional exercises . 82 13 (C, R) Matrix Norms 83 13.1 (R) Operator norm . 83 13.2 (R) Frobenius norm . 84 13.3 (C) Complex Matrices . 84 II Linear Algebra of Vector Spaces 85 Introduction to Part II 86 14 (F) Preliminaries 87 14.1 Modular arithmetic . 87 14.2 Abelian groups . 90 14.3 Fields . 93 vi CONTENTS 14.4 Polynomials . 94 15 (F) Vector Spaces: Basic Concepts 102 15.1 Vector spaces . 102 15.2 Subspaces and span . 104 15.3 Linear independence and bases . 105 15.4 The First Miracle of Linear Algebra . 109 15.5 Direct sums . 111 16 (F) Linear Maps 112 16.1 Linear map basics . 112 16.2 Isomorphisms . 113 16.3 The Rank-Nullity Theorem . 114 16.4 Linear transformations . 115 16.4.1 Eigenvectors, eigenvalues, eigenspaces . 116 16.4.2 Invariant subspaces . 117 16.5 Coordinatization . 120 16.6 Change of basis . 122 17 (F) Block Matrices (optional) 124 17.1 Block matrix basics . 124 17.2 Arithmetic of block-diagonal and block-triangular matrices . 125 18 (F) Minimal Polynomials of Matrices and Linear Transformations (optional) 127 18.1 The minimal polynomial . 127 18.2 Minimal polynomials of linear transformations . 128 19 (R) Euclidean Spaces 131 19.1 Inner products . 131 19.2 Gram-Schmidt orthogonalization . 134 19.3 Isometries and orthogonal transformations . 136 19.4 First proof of the Spectral Theorem . 138 20 (C) Hermitian Spaces 141 20.1 Hermitian spaces . 141 20.2 Hermitian transformations . 144 CONTENTS vii 20.3 Unitary transformations . 145 20.4 Adjoint transformations in Hermitian spaces . 146 20.5 Normal transformations . 146 20.6 The Complex Spectral Theorem for normal transformations . 147 21 (R, C) The Singular Value Decomposition 148 21.1 The Singular Value Decomposition . 148 21.2 Low-rank approximation . 149 22 (R) Finite Markov Chains 151 22.1 Stochastic matrices . 151 22.2 Finite Markov Chains . 151 22.3 Digraphs . 153 22.4 Digraphs and Markov Chains . 153 22.5 Finite Markov Chains and undirected graphs . 153 22.6 Additional exercises . 153 23 More Chapters 155 24 Hints 156 24.1 Column Vectors . ..