MATH 2210Q Applied Linear Algebra Notes Arthur J. Parzygnat These are my personal notes. This is not a substitute for Lay's book. I will frequently reference both recent versions of this book. The 4th edition will henceforth be referred to as [2] while the 5th edition will be [3]. In case comments apply to both versions, these two books will both be referred to as [Lay]. You will not be responsible for any Remarks in these notes. However, everything else, including what is in [Lay] (even if it's not here), is fair game for homework, quizzes, and exams. At the end of each lecture, I provide a list of recommended exercise problems that should be done after that lecture. Some of these exercises will appear on homework, quizzes, or exams! I also provide additional exercises throughout the notes which I believe are good to know. You should also browse other books and do other problems as well to get better at writing proofs and understanding the material. Notes in light red are for the reader. Notes in light green are reminders for me. When a word or phrase is underlined, that typically means the definition of this word or phrase is being given. Contents 1 Linear systems, row operations, and examples3 2 Vectors and span 21 3 Solution sets of linear systems 27 4 Linear independence and dimension of solution sets 36 5 Subspaces, bases, and linear manifolds 47 6 Convex spaces and linear programming 57 7 Linear transformations and their matrices 67 8 Visualizing linear transformations 78 9 Subspaces associated to linear transformations 83 1 10 Iterating linear transformations|matrix multiplication 92 11 Hamming's error correcting code 100 12 Inverses of linear transformations 112 13 The signed volume scale of a linear transformation 123 14 The determinant and the formula for the inverse of a matrix 136 15 Orthogonality 146 16 The Gram-Schmidt procedure 157 17 Least squares approximation 169 18 Decision making and support vector machines* 178 19 Markov chains and complex networks* 197 20 Eigenvalues and eigenvectors 207 21 Diagonalizable matrices 217 22 Spectral decomposition and the Stern-Gerlach experiment* 225 23 Solving ordinary differential equations 232 24 Vector spaces and linear transformations 245 25 Differential operators 255 26 Bases and matrices for linear transformations* 265 27 Change of basis* 277 Sections with a * at the end are additional topics which can be covered if time permits. Acknowledgments I'd like to thank Christian Carmellini, Philip Parzygnat, Zachariah Pittman, Benjamin Russo, Xing Su, Yun Yang, and George Zoghbi for many helpful suggestions and comments. 2 1 Linear systems, row operations, and examples Before saying what one studies in linear algebra, let us consider the following examples. These examples will illustrate the important concept of a mathematical object known as the matrix. Figure 1: Memes \What if I told you Linear Algebra is all about the matrix" http: //www.quickmeme.com/meme/3qwkiq and \What if I told you saying \enter the matrix" in linear algebra isn't funny" http://www.quickmeme.com/meme/36c7p8, respectively. Accessed on December 21, 2017. Example 1.1. Queens, New York has several one-way streets throughout its many neighborhoods. Figure2 shows an intersection in Middle Village, New York. We can represent the flow of traffic Figure 2: An intersection in Middle Village, New York in the borough of Queens. This image is obtained from Map data c 2017 Google https: //www.google.com/maps/place/Middle+Village,+Queens,+NY/@40.7281297,-73. 8802503,18.72z/data=!4m5!3m4!1s0x89c25e6887df03e7:0xef1a62f95c745138! 8m2!3d40.717372!4d-73.87425 around 81st and 82nd streets diagrammatically as 3 Queens Midtown Expy 58th Ave 81st St 82nd St 58th Ave Imagine that we send out detectors (such as scouts) to record the average number of cars per hour along each street. What is the smallest number of scouts we will need to determine the traffic flow on every street? To answer this question, we first point out one important assumption that appears in several different contexts: The net flow into an intersection equals the net flow out of an intersection. From this, it actually follows that the net flow into the network itself is equal to the net flow out of the network. Each edge connecting any two intersections represents an unknown and each fact above provides an equation. Hence, this system has 9 unknowns and 4 equations. Therefore, one expects that the minimum number of scouts needed is 9 − 4 = 5: However, this is certainly not a proof because some of these equations might be redundant! Furthermore, even if 5 is the minimum number of scouts needed, it does not mean that you can place these scouts anywhere and determine the entire traffic flow. For example, if you place the 5 scouts on the following streets Queens Midtown Expy þ ÿ 58th Ave 81st St þ 82nd St 58th Ave ÿ þ then you still don't know the traffic flow leaving 58th Ave and 81st Street on the bottom left. Let's see what happens explicitly by first sending out 3 scouts, which observe the following traffic flow per hour 4 Queens Midtown Expy 100 x1 58th Ave x5 70 81st St 82nd St x2 58th Ave x6 x4 30 x3 The unknown traffic flows have been labelled by the variables x1; x2; x3; x4; x5; x6; which is where we did not send out any scouts. The equations for the “flow in" equals “flow out" are given by (they are written going clockwise starting at the top left) 100 = x1 + x5 x1 + x2 = 70 (1.2) x3 = x2 + x4 x4 + x5 = 30 + x6 This system of linear equations can be rearranged in the following way x1 +x5 = 100 x +x = 70 1 2 (1.3) x2 −x3 +x4 = 0 x4 +x5 −x6 = 30 which makes it easier to see how to manipulate these expressions algebraically by adding or sub- tracting multiples of different rolls. When adding these rows, all we ever add are the coefficients and the variables are just there to remind us of our organization. We can therefore replace these equations with the augmented matrix 2 3 1 0 0 0 1 0 100 61 1 0 0 0 0 70 7 6 7 6 7 : (1.4) 40 1 −1 1 0 0 0 5 0 0 0 1 1 −1 30 Adding and subtracting rows here corresponds to the same operations for the equations. For example, subtract row 1 from row 2 to get 2 3 1 0 0 0 1 0 100 60 1 0 0 −1 0 −307 6 7 6 7 (1.5) 40 1 −1 1 0 0 0 5 0 0 0 1 1 −1 30 5 The result corresponds to the system of equations x1 +x5 = 100 x −x = −30 2 5 (1.6) x2 −x3 +x4 = 0 x4 +x5 −x6 = 30 As we first learn about these operations, we will perform them one at a time and show what happens to them explicitly by the following notation 2 3 2 3 1 0 0 0 1 0 100 1 0 0 0 1 0 100 61 1 0 0 0 0 70 7 60 1 0 0 −1 0 −307 6 7 R27!R2−R1 6 7 6 7 −−−−−−−! 6 7 (1.7) 40 1 −1 1 0 0 0 5 40 1 −1 1 0 0 0 5 0 0 0 1 1 −1 30 0 0 0 1 1 −1 30 which is read as \row 2 becomes row 2 minus row 1." We implicitly understand that all the other rows remain unchanged unless explicitly written otherwise. Subtract row 2 from row 3 to get 2 3 2 3 1 0 0 0 1 0 100 1 0 0 0 1 0 100 60 1 0 0 −1 0 −307 60 1 0 0 −1 0 −307 6 7 R37!R3−R2 6 7 6 7 −−−−−−−! 6 7 (1.8) 40 1 −1 1 0 0 0 5 40 0 −1 1 1 0 30 5 0 0 0 1 1 −1 30 0 0 0 1 1 −1 30 Subtract row 4 from row 3 to get 2 3 2 3 1 0 0 0 1 0 100 1 0 0 0 1 0 100 60 1 0 0 −1 0 −307 60 1 0 0 −1 0 −307 6 7 R37!R3−R4 6 7 6 7 −−−−−−−! 6 7 (1.9) 40 0 −1 1 1 0 30 5 40 0 −1 0 0 1 0 5 0 0 0 1 1 −1 30 0 0 0 1 1 −1 30 Multiply row 3 by −1 to get rid of the negative coefficient for the x3 variable (this step is not necessary and is mostly just for the A E S T H E T I C S) 2 3 2 3 1 0 0 0 1 0 100 1 0 0 0 1 0 100 60 1 0 0 −1 0 −307 60 1 0 0 −1 0 −307 6 7 R37→−R3 6 7 6 7 −−−−−! 6 7 (1.10) 40 0 −1 0 0 1 0 5 40 0 1 0 0 −1 0 5 0 0 0 1 1 −1 30 0 0 0 1 1 −1 30 This tells us that the initial system of linear equations is equivalent to x1 +x5 = 100 x −x = −30 2 5 (1.11) x3 −x6 = 0 x4 +x5 −x6 = 30 or x1 = 100 − x5 x2 = x5 − 30 (1.12) x3 = x6 x4 = 30 + x6 − x5 6 so that all of the traffic flows are expressed in terms of just x5 and x6: This choice is arbitrary and we could have expressed another four traffic flows in terms of the other two (again, not any four, but some).