Lecture Notes for Linear Algebra James S. Cook Liberty University Department of Mathematics and Physics Fall 2009 2 introduction and motivations for these notes These notes should cover what is said in lecture at a minimum. However, I’m still learning so I may have a new thought or two in the middle of the semester, especially if there are interesting questions. We will have a number of short quizzes and the solutions and/or commentary about those is not in these notes for the most part. It is thus important that you never miss class. The text is good, but it does not quite fit the lecture plan this semester. I probably would have changed it if I realized earlier but alas it is too late. Fortunately, these notes are for all intents and purposes a text for the course. But, these notes lack exercises, hence the required text. The text does have a good assortment of exercises but please bear in mind that the ordering of the exercises assigned matches my lecture plan for the course and not the text’s. Finally, there are a couple topics missing from the text which we will cover and I will write up some standard homework problems for those sections. As usual there are many things in lecture which you will not really understand until later. I will regularly give quizzes on the material we covered in previous lectures. I expect you to keep up with the course. Procrastinating until the test to study will not work in this course. The difficulty and predictability of upcoming quizzes will be a function of how I percieve the class is following along. Doing the homework is doing the course. I cannot overemphasize the importance of thinking through the homework. I would be happy if you left this course with a working knowledge of: ✓ how to solve a system of linear equations ✓ Gaussian Elimination and how to interpret the rref (A) ✓ concrete and abstract matrix calculations ✓ determinants ✓ real vector spaces both abstract and concrete ✓ subspaces of vector space ✓ how to test for linear independence ✓ how to prove a set spans a space ✓ coordinates and bases ✓ column, row and null spaces for a matrix ✓ basis of an abstract vector space ✓ linear transformations 3 ✓ matrix of linear transformation ✓ change of basis on vector space ✓ geometry of Euclidean Space ✓ orthogonal bases and the Gram-Schmidt algorthim ✓ least squares fitting of experimental data ✓ best fit trigonmetric polynomials (Fourier Analysis) ✓ Eigenvalues and Eigenvectors ✓ Diagonalization ✓ how to solve a system of linear differential equations ✓ principle axis theorems for conic sections and quadric surfaces I hope that I have struck a fair balance between pure theory and application. The treatment of systems of differential equations is somewhat unusual for a first course in linear algebra. No apolo- gies though, I love the example because it shows nontrivial applications of a large swath of the theory in the course while at the same time treating problems that are simply impossible to solve without theory. Generally speaking, I tried to spread out the applications so that if you hate the theoretical part then there is still something fun in every chapter. If you don’t like the applications then you’ll just have to deal (as my little sister says) Before we begin, I should warn you that I assume quite a few things from the reader. These notes are intended for someone who has already grappled with the problem of constructing proofs. I assume you know the difference between ⇒ and ⇔. I assume the phrase ”iff” is known to you. I assume you are ready and willing to do a proof by induction, strong or weak. I assume you know what ℝ, ℂ, ℚ, ℕ and ℤ denote. I assume you know what a subset of a set is. I assume you know how to prove two sets are equal. I assume you are familar with basic set operations such as union and intersection (although we don’t use those much). More importantly, I assume you have started to appreciate that mathematics is more than just calculations. Calculations without context, without theory, are doomed to failure. At a minimum theory and proper mathematics allows you to communicate analytical concepts to other like-educated individuals. Some of the most seemingly basic objects in mathematics are insidiously complex. We’ve been taught they’re simple since our childhood, but as adults, mathematical adults, we find the actual definitions of such objects as ℝ or ℂ are rather involved. I will not attempt to provide foundational arguments to build numbers from basic set theory. I believe it is possible, I think it’s well-thought- out mathematics, but we take the existence of the real numbers as an axiom for these notes. We assume that ℝ exists and that the real numbers possess all their usual properties. In fact, I assume ℝ, ℂ, ℚ, ℕ and ℤ all exist complete with their standard properties. In short, I assume we have numbers to work with. We leave the rigorization of numbers to a different course. 4 Contents 1 Gauss-Jordon elimination 9 1.1 systems of linear equations . 9 1.2 Gauss-Jordon algorithm . 13 1.3 classification of solutions . 18 1.4 applications to curve fitting and circuits . 23 1.5 conclusions ........................................ 27 2 matrix arithmetic 29 2.1 basic terminology and notation . 29 2.2 addition and multiplication by scalars . 31 2.3 matrixmultiplication................................ 33 2.4 elementarymatrices ................................. 38 2.5 invertiblematrices ................................ 40 2.6 systems of linear equations revisited . 44 2.6.1 concatenation for solving many systems at once . 45 2.7 how to calculate the inverse of a matrix . 46 2.8 all your base are belong to us ( ei and Eij thatis).................... 47 2.9 matriceswithnames .................................. 51 2.9.1 symmetric and antisymmetric matrices . 52 2.9.2 exponent laws for matrices . 54 2.9.3 diagonal and triangular matrices . 55 2.10 applications . 57 2.11conclusions ........................................ 60 3 determinants 61 3.1 determinants and geometry . 61 3.2 cofactor expansion for the determinant . 63 3.3 adjointmatrix....................................... 69 3.4 Kramer’sRule ...................................... 71 3.5 propertiesofdeterminants. 73 3.6 examplesofdeterminants .............................. 78 3.7 applications........................................ 80 5 6 CONTENTS 3.8 conclusions ........................................ 82 4 linear algebra 83 4.1 definitionandexamples ................................ 84 4.2 subspaces ......................................... 87 4.3 spanningsetsandsubspaces. 90 4.4 linearindependence................................ 98 4.5 basesanddimension ................................... 106 4.5.1 how to calculate a basis for a span of row or column vectors . 109 4.5.2 calculating basis of a solution set . 113 4.5.3 what is dimension? . 115 4.6 general theory of linear systems . 122 4.7 applications........................................ 125 4.8 conclusions ........................................ 127 5 linear transformations 129 5.1 examples of linear transformations . 129 5.2 matrix of a linear operator . 132 5.3 composition of linear transformations . 133 5.4 isomorphism of vector spaces . 136 5.5 change we can believe in (no really, no joke) . 138 5.6 applications........................................ 143 5.7 conclusions ........................................ 143 6 linear geometry 145 6.1 Euclidean geometry of ℝn .................................146 6.2 orthogonality in ℝ n×1 ...................................150 6.3 orthogonal complements and projections . 158 6.4 theclosestvectorproblem ............................. 164 6.5 inconsistentequations ............................... 166 6.6 least squares analysis . 167 6.6.1 linear least squares problem . 167 6.6.2 nonlinear least squares . 169 6.7 orthogonal transformations . 174 6.8 innerproducts .................................... 177 6.8.1 examplesofinner-products . 179 6.8.2 Fourier analysis . 183 7 eigenvalues and eigenvectors 185 7.1 geometry of linear transformations . 186 7.2 definition and properties of eigenvalues . 188 7.2.1 eigenwarning . 190 7.3 eigenvalueproperties................................ 190 CONTENTS 7 7.4 real eigenvector examples . 192 7.5 complex eigenvector examples . 196 7.6 linear independendence of real eigenvectors . 199 7.7 linear independendence of complex eigenvectors . 203 7.8 determinants and eigenvalues . 205 7.9 diagonalization . 206 7.10 calculus of matrices . 209 7.11 introduction to systems of linear differential equations . 210 7.12 the matrix exponential . 213 7.12.1 analysis for matrices . 214 7.12.2 formulas for the matrix exponential . 215 7.13 solutions for systems of DEqns with real eigenvalues . 219 7.14 solutions for systems of DEqns with complex eigenvalues . 224 7.15 geometry and difference equations revisited . 225 7.16 conic sections and quadric surfaces . 226 8 appendix on finite sums 227 8 CONTENTS Chapter 1 Gauss-Jordon elimination Gauss-Jordon elimination is an optimal method for solving a system of linear equations. Logically it may be equivalent to methods you are already familar with but the matrix notation is by far the most efficient method. This is important since throughout this course we will be faced with the problem of solving linear equations. Additionally, the Gauss-Jordon produces the reduced row echelon form (rref) of the matrix. Given a particular matrix the rref is unique. This is of particular use in theoretical applications. 1.1 systems of linear equations Let me begin with a few examples before I state the general definition. Example 1.1.1. Consider the following system of 2 equations and 2 unknowns, x + y = 2 x − y = 0 Adding equations reveals 2x = 2 hence x = 1 . Then substitute that into either equation to deduce y = 1 . Hence the solution (1 , 1) is unique Example 1.1.2. Consider the following system of 2 equations and 2 unknowns, x + y = 2 3x + 3 y = 6 We can multiply the second equation by 1/3 to see that it is equivalent to x + y = 2 thus our two equations are in fact the same equation.