Fundamentals of Linear Algebra and Optimization CIS515, Some Slides Jean Gallier Department of Computer and Information Science University of Pennsylvania Philadelphia, PA 19104, USA e-mail:
[email protected] c Jean Gallier November 5, 2013 2 Contents 1 Basics of Linear Algebra 9 1.1 Motivations: Linear Combinations, Linear Independence, Rank . 9 1.2 Vector Spaces . 27 1.3 Linear Independence, Subspaces . 37 1.4 Bases of a Vector Space . 47 1.5 LinearMaps ................ 57 1.6 Matrices .................. 68 1.7 Haar Basis Vectors; a Glimpse at Wavelets 108 1.8 TheE↵ect of a Change of Bases on Matrices138 1.9 AffineMaps ................ 149 1.10 Direct Products, Sums, and Direct Sums . 164 1.11 The Dual Space E⇤ and Linear Forms . 178 1.12 Hyperplanes and Linear Forms . 199 1.13 Transpose of a Linear Map and of a Matrix 200 1.14 The Four Fundamental Subspaces . 207 2 Gaussian Elimination, LU, Cholesky, Re- 3 4 CONTENTS duced Echelon 217 2.1 Motivating Example: Curve Interpolation . 217 2.2 Gaussian Elimination and LU-Factorization226 2.3 Gaussian Elimination of Tridiagonal Ma- trices . 269 2.4 SPD Matrices and the Cholesky Decompo- sition . 276 2.5 Reduced Row Echelon Form . 280 3 Determinants 305 3.1 Permutations, Signature of a Permutation 305 3.2 AlternatingMultilinearMaps . 311 3.3 Definition of a Determinant . 319 3.4 Inverse Matrices and Determinants . 330 3.5 Systems of Linear Equations and Determi- nants . 334 3.6 DeterminantofaLinearMap . 335 3.7 The Cayley–Hamilton Theorem . 337 3.8 Further Readings . 344 4 Vector Norms and Matrix Norms 345 4.1 Normed Vector Spaces .