D.G. Bonett (6/2018) Matrix Notation and Operations Matrix Notation An r × c matrix is a rectangular array of elements with r rows and c columns. An r × c matrix is said to be of order r × c. A matrix is usually denoted by a capital letter printed in a boldface font (e.g., A, B, X). The elements of the matrix are represented by lower case letters with a double subscript (e.g., 푎푗푘, 푏푗푘, 푥푗푘). For instance, in the matrix X, 푥13 is the element in the first row and the third column. A 4 × 3 matrix X is shown below. 푥11 푥12 푥13 푥 푥 푥 X = [ 21 22 23] 푥31 푥32 푥33 푥41 푥42 푥43 A matrix with a single row is called a row vector and a matrix with a single column is called a column vector. Vectors are usually represented by lower case letters printed in a boldface font (e.g., a, b, x). The elements of the vector are represented by lower case letters with a single subscript (e.g., 푎푗, 푏푗, 푥푗). A 3 × 1 column vector y and a 1 × 4 row vector h are shown below. 푦1 y = [푦2] 푦3 h = [ℎ1 ℎ2 ℎ3 ℎ4 ]. It is sometimes necessary to refer to a particular row or column of a matrix. These row or column vectors are represented by a subscripted lower case letter in a boldface font. For th instance, the j row vector in the above 4 × 3 matrix X would be noted as 퐱푗. A square matrix has the same number of rows as columns. A square matrix where the diagonal elements (the elements where the two subscripts are equal) are nonzero and the off-diagonal elements are zero is called a diagonal matrix. A 3 × 3 diagonal matrix D is shown below. 1 D.G. Bonett (6/2018) 푑11 0 0 D = [ 0 푑22 0 ] 0 0 푑33 The identity matrix is a special type of diagonal matrix where all diagonal elements are equal to 1. The identity matrix is usually represented as I or In where n is the order of the identity matrix. A square matrix where the jkth element is equal to the kjth element is called a symmetric matrix. A symmetric 3 × 3 matrix is shown below. 14 5 2 S = [ 5 20 8 ] 2 8 11 A one vector is a row or column vector in which every element is equal to 1 and is represented as the number one printed in a boldface font. A 1 × 3 one vector is shown below. 1 = [1 1 1] Matrix Operations The transpose of a matrix X is represented as X' (or XT). The transpose of a matrix is obtained by interchanging the rows and columns of the matrix. For instance, if 4 6 X = [7 1] 3 9 then 4 7 3 X' = [ ] . 6 1 9 Note that the jkth element in X is equal to the kjth element in X'. Most vectors in statistical formulas are assumed to be column vectors. Row vectors, when needed, are obtained by taking the transpose of a column vector. 2 D.G. Bonett (6/2018) If two matrices A and B are of the same order, the two matrices are then conformable for th addition or subtraction, and A + B is a matrix with element 푎푗푘 + 푏푗푘 in the j row and the kth column, as illustrated below for the sum of two 3 × 2 matrices. 푎11 푎12 푎13 푏 푏 푏 A + B = [ ] + [ 11 12 13] 푎21 푎22 푎23 푏21 푏22 푏23 푎11 + 푏11 푎12 + 푏12 푎13 + 푏13 = [ ] 푎21 + 푎11 푎22 + 푏22 푎23 + 푏23 th th Likewise, A – B is a matrix with element 푎푗푘 – 푏푗푘 in the j row and the k column, as illustrated below for the difference of two 3 × 2 matrices. 푎 푎 푎 푏 푏 푏 A – B = [ 11 12 13] – [ 11 12 13] 푎21 푎22 푎23 푏21 푏22 푏23 푎 − 푏 푎 − 푏 푎 − 푏 = [ 11 11 12 12 13 13] 푎21 − 푎11 푎22 − 푏22 푎23 − 푏23 To multiply a matrix by a scalar (i.e., a single number), simply multiply each element in the matrix by the scalar. Scalars are represented by italicized lower case letters in a non- boldface font. To illustrate, if b = 2 and 4 7 3 A = [ ] 6 1 9 then 8 14 6 bA = [ ]. 12 2 18 Some statistical formulas involve the subtraction of a scalar from an n x 1 vector. The result is obtained by first multiplying the scalar by an n x 1 one vector and then taking the difference of the two vectors. For instance, if y is a 3 × 1 vector and m is a scalar, then y – m is 푦1 1 푦1 − 푚 y – m1 = [푦2] − 푚 [1] = [푦2 − 푚] 푦3 1 푦3 − 푚 3 D.G. Bonett (6/2018) The dot product of a n × 1 vector a with a n × 1 vector b is a'b = 푎1푏1 + 푎2푏2 + … + 푎푛푏푛 Note that a'b = b'a. For instance, if a' = [4 3 2] and b' = [6 1 4], then a'b = 4(6) + 3(1) + 2(4) = 35. Two n × 1 vectors, a and b, are said to be orthogonal if a'b = 0. For instance, if a' = [.5 .5 -1] and b' = [1 -1 0], then a and b are orthogonal because 퐚'b = (.5)(1) + (.5)(-1) + (-1)(0) = 0. Two matrices A and B can be multiplied if they are conformable for multiplication. To compute the matrix product AB, the number of columns of A must equal the number of rows of B. In general, if A is r × n and B is n × c, then the matrix product AB is an r × c th matrix. The jk element in the r × c product matrix is equal to the dot product 퐚푗퐛푘 where th th 퐚푗 is the j row vector of matrix A and 퐛푘 is the k column vector of matrix B. For instance, the matrices A and B shown below are conformable for computing the product AB because A is 2 × 3 and B is 3 × 4 so that the product will be a 2 × 4 matrix. 1 2 1 4 4 7 3 A = [ ] B = [5 4 3 1] 6 1 9 4 2 3 2 Each of the 2 × 4 = 8 elements of the AB matrix is a dot product. For instance, the element in row 1 and column 1 of the product AB is 1 [4 7 3] [5] = 4(1) + 7(5) + 3(4) = 51 4 and the element in row 2 and column 3 of AB is 1 [6 1 9] [3] = 6(1) + 1(3) + 9(3) = 36. 3 After computing all 8 dot products, the following result is obtained. 51 42 34 29 AB = [ ] 47 34 36 43 4 D.G. Bonett (6/2018) Unlike scalar multiplication where ab = ba, the matrix product AB does not in general equal BA. Regarding the matrix product AB, we can say that B is pre-multiplied by A or that A is post-multiplied by B. The product of matrix A with itself is denoted as A2. The transpose of a matrix product is equal to the product of the transposed matrices in reverse order. Specifically, (AB)' = B'A'. The product of three matrices ABC requires A and B to be conformable for multiplication and also requires B and C to be conformable for multiplication. The product ABC can be obtained by first computing AB and then post-multiplying the result by C, or by first computing BC and then pre-multiplying the result by A. If A is a square matrix, then the matrix inverse of A is represented as A-1. If the inverse of A exists, then AA-1 = I. This result is a generalization of scalar arithmetic where x(1/x) = 1, assuming x 0 so that the inverse of x exists. Computing a matrix inverse is tedious, and the amount of computational effort increases as the size of the matrix increases, but inverting a 2 × 2 matrix is not difficult. The inverse of a 2 × 2 matrix A is 푎 −푎 -1 22 12 A = (1/d)[ ] −푎21 푎11 where d = 푎11푎22 − 푎12푎21 is called the determinant of A. The matrix inverse does not exist unless the determinant is nonzero. Inverting a diagonal matrix D of any order is simple. The inverse of D is equal to a th diagonal matrix where the j diagonal element is equal to 1/푑푗 . The trace of a square n × n matrix A, denoted as tr(A), is defined as the sum of its n diagonal elements. tr(A) = 푎11 + 푎22 + … + 푎푛푛 5 D.G. Bonett (6/2018) For instance if 14 5 2 V = [ 9 20 8 ] 7 8 11 then tr(V) = 14 + 20 + 11 = 45. The Kronecker product of two matrices, an m × n matrix A and a p × q matrix B, is defined to be the mp × nq matrix 푎11퐁 푎12퐁 ⋯ 푎1푛퐁 푎 퐁 푎 퐁 … 푎 퐁 ⨂ [ 21 22 2푛 ] A B = ⋮ ⋮ ⋮ 푎푚1퐁 푎푚2퐁 ⋯ 푎푚푛퐁 which is obtained by replacing each element 푎푗푘 with the p × q matrix 푎푗푘B. For example, if 1 2 A = [ ] and b = [1 2 3] 3 −1 then 1 2 3 2 4 6 A ⨂ b = [ ]. 3 6 9 −1 −2 −3 The Kronecker product of an identity matrix and another matrix has the following simple form 퐁 ퟎ ⋯ ퟎ ퟎ 퐁 … ퟎ I ⨂ B = [ ] ⋮ ⋮ ⋮ ퟎ ퟎ ⋯ 퐁 where ퟎ is a matrix of zeros that has the same order as B.