A taste of matrix theory SM242 Fall 2019 1 Vectors A vector is an array of numbers such as v = 7 −2 1 7 : In this example, we would write v 2 Z4, because the entries of v are all integers, and the dimension of v (i.e., the number of entries) is 4. More generally, if S is a set and n is a non-negative integer, then Sn is the set of all length-n vectors with entries from S. We will write the ith entry of vector v as vi, always starting from index i = 1 and going up to the vector dimension n. So in the above example, v4 = 7. You have probably seen vectors in calculus class, physics, or elsewhere. Vectors can be used to represent many things: a physical force with direction and magnitude, a single point in n-dimensional space, or the list of partial derivatives of a multivariable function. In this class, vectors may mean any of these things, or none of these things. It will be simplest to think of a vector just as an array of numbers from some underlying set. 2 Vector operations We can do arithmetic with vectors, although it sometimes works differently than arithmetic with individual numbers. Consider, for example, a store which sells Snickers bars, M&Ms, and Twix for $1.00, $.75, and $1.50 each, respectively. We can represent this information in a vector a 2 Q3: a = 1 :75 1:5 2.1 Dot product Now let's say I want to buy 3 Snickers and 2 Twix. I can represent my purchase amounts as another vector b 2 Z3: 1 b = 3 0 2 To compute how much it will cost in total to complete my purchase, I multiply the corresponding entries from a and b and add them up. This is called a dot product or inner product: a · b = 1 × 3 + :75 × 0 + 1:5 × 2 = 6 Because a · b = 6, the transaction in this example costs $6.00. n Pn In general, the dot product of two vectors u; v 2 S is the single scalar value i=1 uivi. 2.2 Vector addition Now let's say my friend Cheryl asks me to buy her a pack of M&M's and a Twix. Cheryl's preferences are represented by another vector c 2 Z3: c = 0 1 1 How much will both of our purchases cost? To answer this we will perform the addition of two vectors, which can be computed by adding up the corresponding entries into a new vector: 233 203 233 b + c = 405 + 415 = 415 2 1 3 In general, the addition of two vectors u; v 2 Sn is another vector w 2 Sn defined by wi = ui + vi for each 1 ≤ i ≤ n. Notice that I wrote these vectors standing up rather than laying down. They are the same vectors either way, but when standing up we call them column vectors and when laying down we call them row vectors. (Some other authors prefer to write b| to indicate a row vector, which will make sense to you when we learn what the transpose of a matrix is shortly.) 2.3 Vector times scalar Finally, imagine that I repeat my order 52 times, every week of the year. How many of each candy bar do I buy in total? This is called a scalar product of a vector times a number: 233 21563 52b = 52 405 = 4 0 5 2 104 In general, for a scalar x 2 S and a vector v 2 Sn, the scalar product xv is another n vector w 2 S defined by wi = xvi for each 1 ≤ i ≤ n. 2 How do these three operations of dot product, addition, and scalar multiplication inter- act? Pretty much how you would expect. Let u; v; w 2 Sn be any n-dimensional vectors, and let x; y 2 S be any scalar value. Then vector arithmetic follows these rules, which are pretty much what you should expect: • Dot products are commutative: u · v = v · u • Vector addition is commutative and associative: (u + v) + w = u + (v + w) = (w + v) + u • Scalar multiplication is distributive: (x + y)(u + v) = x(u + v) + y(u + v) = (x + y)u + (x + y)v = xu + yu + xv + yv • Dot products are distributive too: u · (v + w) = u · v + u · w • Finally, scalar and dot products are associative with each other: x(u · v) = (xu) · v The important thing when dealing with vector arithmetic is to make sure the types and dimensions always match. The definitions don't make any sense to compute the dot product or addition of two vectors with different dimensions. Similarly, we can't compute the scalar product of two vectors, or a dot product of a number times a vector. When you are doing math with vectors, make sure you always know what is a vector and what is a scalar, and what are the dimensions of any vectors you are dealing with. 3 Matrices A matrix is a rectangular array 2 3 a1;1 a1;2 ··· a1;n 6 a a ··· a 7 6 2;1 2;2 2;n 7 A = 6 . .. 7 4 . 5 am;1 am;2 ··· am;n m×n If each entry ai;j of A is contained in the same set S, then we say A 2 S . That is, the set Sm×n is the set of matrices with entries from S organized into m rows and n columns. Importantly, notice that a matrix is just an array of vectors! If you understand what we just learned about vectors, then most facts about matrices will start to make sense. 3 For any row index 1 ≤ i ≤ m, the ith row vector of matrix A is ai;1 ··· ai;n : And for any column index 1 ≤ j ≤ m, the jth column vector of A is 2 3 a1;j 6 . 7 4 . 5 : an;j 3.1 Matrix addition and scalar multiplication From what we know about vector arithmetic, two kinds of matrix arithmetic should be no surprise: • Matrix addition: To add two matrices A; B 2 Sm×n, create a new matrix with the same dimensions, formed by adding each entry of A to the corresponding entry in B. For example, 5 6 5 4 1 8 9 7 13 + = 2 0 6 3 2 5 5 2 11 • Scalar multiplication: To multiply a matrix A times a scalar value x 2 S, just multiply each entry of A times x. For example, 3 2 5 −6 −4 −10 −2 = 0 8 8 0 −16 −16 4 Matrix products Returning to our example from the first section, we said that one store sells Snickers, M&M's and Twix for $1,00, $.75, and $1.50 each, respectively. Imagine a second store opens up and sells all three candy bars for $1.15. This gives us a 2 × 3 matrix A 2 Q2×3: 1 :75 1:5 A = 1:15 1:15 1:15 4.1 Matrix-vector multiplication If I now want to calculate how much my purchase of 3 Snickers and 2 Twix will be at each of these two stores, I need to calculate two dot products, each store's vector of prices (a row in the matrix) times my vector of purchase amounts. This is a matrix-vector product: 233 1 :75 1:5 3 × 1 + 0 × :75 + 2 × 1:5 6 0 = = 1:15 1:15 1:15 4 5 3 × 1:15 + 0 × 1:15 + 2 × 1:15 5:75 2 4 What this tells us is that my purchase will cost $6.00 at the first store, but only $5.75 at the second store. Notice that column dimension of the matrix must match up with the dimension of the vector | in this case, 3 kinds of candy. And the row dimension of the matrix will be the same as the dimension of the resulting product vector | in this case, 2 different stores. More generally, the matrix-vector product of A 2 Sm×n and u 2 Sn is a vector v 2 Sm, Pn written Au = v, where the ith entry of v, for every 1 ≤ i ≤ m, is defined by vi = j=1 ai;juj. 4.2 Matrix-matrix multiplication Let's extend this further. Remember again from the original example that my friend Cheryl wants to purchase 1 M&M's and 1 Twix, corresponding to the vector 203 415 : 1 Now we ask the question: how much will each of our purchases cost, at each of the stores in town? For this we will need four dot products, each store's price vector times each of these two purchase vectors. This is a matrix-matrix product: 23 03 1 :75 1:5 6 2:25 0 2 = 1:15 1:15 1:15 4 5 5:75 2:30 1 1 Notice that the output entry in row i and column j is computed as the dot product of the ith row of the first matrix, times the jth column in the second matrix. For example, the price $2.25 that Cheryl will spend at the first store equals the first row of 1 :75 1:5 1:15 1:15 1:15 times the second column of 23 03 40 15 1 1 computed as 203 1 :75 1:5 · 415 = 2:25 1 To compute a matrix product, you have to do a different dot product for every entry in the output matrix.