Appendix A Linear Algebra This appendix surveys some of the important basic concepts in linear algebra that often appear in computer graphics literature. It is not meant to be a complete reference for linear algebra; however, it covers the concepts required to read this book. Skilled readers may skip this appendix. A.1 Matrices A matrix is an array of numbers, entries, components, terms or elements. An n × m matrix A consists of n rows and m columns and is expressed as ⎡ ⎤ a11 a12 ... a1m ⎢ ⎥ ⎢ a21 a22 ... a2m ⎥ A = ⎢ . ⎥ . (A.1) ⎣ . .. ⎦ an1 an2 ... anm A matrix is called square if the number of rows and the number of columns are equal (i.e., if n = m); otherwise, it is called rectangular (i.e., if n = m). A square matrix is orthogonal when the dot product (Sect. A.4.3.2)ofanypair of rows or any pair of columns is zero and when the columns and the rows are unit vectors (Sect. A.4.1). Also, the inverse (Sect. A.2.2) and the transpose (Sect. A.2.3) of an orthogonal matrix are equal. A.1.1 Identity Matrix An identity matrix I is a square matrix where aij = 1 when i = j and aij = 0 otherwise. Hence, an identity matrix is expressed as R. Elias, Digital Media, DOI: 10.1007/978-3-319-05137-6, 625 © Springer International Publishing Switzerland 2014 626 Appendix A: Linear Algebra ⎡ ⎤ 10... 0 ⎢ ⎥ ⎢ 01... 0 ⎥ I = ⎢ . ⎥ . (A.2) ⎣ . .. ⎦ 00... 1 As a convention, the identity matrix is always referred to by the letter I. A.1.2 Diagonal Matrix A diagonal matrix D is a square matrix with zero entries except for those of the main diagonal (i.e., where i = j). In other words, an n × n matrix D is diagonal if aij = 0 | i = j ∀ i, j ∈{1, 2,...,n}. Hence, a diagonal matrix is expressed as ⎡ ⎤ a11 0 ... 0 ⎢ ⎥ ⎢ 0 a22 ... 0 ⎥ D = ⎢ . ⎥ . (A.3) ⎣ . .. ⎦ 00... ann The same diagonal matrix may also be denoted as D = diag(a11, a22,...,ann) where a11, a22,...,ann are the elements of the main diagonal. Matrix diagonalization is converting a square matrix into a diagonal matrix. A.1.3 Symmetric Matrix A symmetric matrix is a square matrix where aij = a ji. For an n × n, it can be expressed as ⎡ ⎤ a11 a12 ... a1n ⎢ ⎥ ⎢ a12 a22 ... a2n ⎥ A = ⎢ . ⎥ . (A.4) ⎣ . .. ⎦ a1n a2n ... ann In this case, A = AT . This implies that A−1AT = I. A.1.4 Anti-Symmetric Matrix An anti-symmetric or skew symmetric matrix is a square matrix where aij =−a ji. For an n × n, it can be expressed as Appendix A: Linear Algebra 627 ⎡ ⎤ 0 a12 ... a1n ⎢ ⎥ ⎢ −a12 0 ... a2n ⎥ A = ⎢ . ⎥ . (A.5) ⎣ . .. ⎦ −a1n −a2n ... 0 An anti-symmetric matrix must have zeros on its diagonal. In this case, A =−AT . This implies that −A−1AT = I. A.2 Matrix Operations Some matrix operations are applied to single matrices (e.g., determinant, inverse, transpose and scalar multiplication) while others are applied to pairs of matrices (e.g., addition, subtraction and multiplication). In the following sections, we will talk about some of the most important matrix operations. A.2.1 Matrix Determinant A matrix determinant is defined for a square matrix. A 2× 2 matrix determinant |A| is defined as a11 a12 a11 a12 |A|≡det ≡ a21 a22 a21 a22 (A.6) = a11a22 − a12a21. A3× 3 matrix determinant |A| is defined as ⎡ ⎤ a11 a12 a13 a11 a12 a13 ⎣ ⎦ |A|≡det a21 a22 a23 ≡ a21 a22 a23 a31 a32 a33 a31 a32 a33 a22 a23 a21 a23 a21 a22 = a11 − a12 + a13 a32 a33 a31 a33 a31 a32 = a11(a22a33 − a23a32) − a12(a21a33 − a23a31) + a13(a21a32 − a22a31). (A.7) A.2.2 Matrix Inverse − − The inverse of a square matrix A is another square matrix A 1 such that AA 1 = I. a a If A = 11 12 then the inverse A−1 is calculated as a21 a22 628 Appendix A: Linear Algebra − −1 = 1 a22 a12 A |A| −a21 a11 − (A.8) = 1 a22 a12 . a a − a a 11 22 12 21 −a21 a11 For a 3 × 3matrixA,theinverseA−1 is defined as ⎡ ⎤ a22 a23 a13 a12 a12 a13 ⎢ ⎥ ⎢ a32 a33 a33 a32 a22 a23 ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ − 1 ⎢ a23 a21 a11 a13 a13 a11 ⎥ A 1 = ⎢ ⎥ . (A.9) |A| ⎢ a33 a31 a31 a33 a23 a21 ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ ⎦ a21 a22 a12 a11 a11 a12 a31 a32 a32 a31 a21 a22 Example A.1 [Matrices–inverse] Compute the inverse for the following matrix: ⎡ ⎤ 111 A = ⎣ 331⎦ . 411 Solution A.1 There are two steps: 1. Get the determinant |A| by applying Eq. (A.7). Thus, |A|=1 × (3 × 1 − 1 × 1) − 1 × (3 × 1 − 4 × 1) + 1 × (3 × 1 − 3 × 4) =−6. 2. Get the inverse A−1 by applying Eq. (A.9)as ⎡ ⎤ 31 11 11 ⎢ ⎥ ⎢ 11 11 31 ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ − 1 ⎢ 13 11 11 ⎥ A 1 = ⎢ ⎥ −6 ⎢ 14 41 13 ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ ⎦ 33 11 11 41 14 33 ⎡ ⎤ 20−2 1 = ⎣ 1 −32⎦ − 6 −93 0 Appendix A: Linear Algebra 629 ⎡ ⎤ − 1 0 1 ⎢ 3 3 ⎥ = ⎢ − 1 1 − 1 ⎥ . ⎣ 6 2 3 ⎦ 1 − 1 1 2 2 0 A.2.3 Matrix Transpose The transpose of a matrix A is the matrix obtained by replacing all elements aij with a ji (i.e., columns become rows and rows become columns). The transpose of A is denoted by AT . Thus, we can write (AT )T = A. Example A.2 [Matrices–transpose] 123 Get the transpose of A where A = . 456 Solution A.2 The inverse can be expressed as ⎡ ⎤ T 14 123 AT = = ⎣ 25⎦ . 456 36 A.2.4 Addition and Subtraction Matrices of the same size can be added together (or subtracted from each other) by adding (or subtracting) the corresponding elements. Thus, for two matrices A and B of size n × m, we may have ⎡ ⎤ ⎡ ⎤ a11 a12 ... a1m b11 b12 ... b1m ⎢ ⎥ ⎢ ⎥ ⎢ a21 a22 ... a2m ⎥ ⎢ b21 b22 ... b2m ⎥ A + B = ⎢ . ⎥ + ⎢ . ⎥ ⎣ . .. ⎦ ⎣ . .. ⎦ an1 an2 ... anm bn1 bn2 ... bnm ⎡ ⎤ (A.10) a11 + b11 a12 + b12 ... a1m + b1m ⎢ ⎥ ⎢ a21 + b21 a22 + b22 ... a2m + b2m ⎥ = ⎢ . ⎥ ⎣ . .. ⎦ an1 + bn1 an2 + bn2 ... anm + bnm or 630 Appendix A: Linear Algebra ⎡ ⎤ ⎡ ⎤ a11 a12 ... a1m b11 b12 ... b1m ⎢ ⎥ ⎢ ⎥ ⎢ a21 a22 ... a2m ⎥ ⎢ b21 b22 ... b2m ⎥ A − B = ⎢ . ⎥ − ⎢ . ⎥ ⎣ . .. ⎦ ⎣ . .. ⎦ an1 an2 ... anm bn1 bn2 ... bnm ⎡ ⎤ (A.11) a11 − b11 a12 − b12 ... a1m − b1m ⎢ ⎥ ⎢ a21 − b21 a22 − b22 ... a2m − b2m ⎥ = ⎢ . ⎥ . ⎣ . .. ⎦ an1 − bn1 an2 − bn2 ... anm − bnm Example A.3 [Matrices–addition and subtraction ] 1234 355 6 Consider two matrices A and B where A = and B = . 5677 78910 Calculate the addition A+B and the subtractions A−B and B−A. Solution A.3 The addition is performed by adding corresponding terms as 1234 355 6 A + B = + 5677 78910 + + + + = 1 32 53 54 6 5 + 76+ 87+ 97+ 10 = 47810. 12 14 16 17 The subtraction is performed by subtracting corresponding terms as 1234 355 6 A − B = − 5677 78910 − − − − = 1 32 53 54 6 5 − 76− 87− 97− 10 − − − − = 2 3 2 2 −2 −2 −2 −3 355 6 1234 B − A = − 78910 5677 − − − − = 3 15 25 36 4 7 − 58− 69− 710− 7 = 2322 . 2223 Notice that B − A =−[A − B]. Appendix A: Linear Algebra 631 Example A.4 [Matrices–addition and subtraction ] 123 35 Consider two matrices A and B where A = and B = . Calculate 567 78 the addition A+B and the subtractions A−B and B−A. Solution A.4 Since the sizes of the matrices are different, none of the operations can be performed. A.2.5 Scalar Multiplication Scalar multiplication is an operation performed by multiplying each of the elements of a matrix by a scalar value. Thus, for a n × m matrix A and a scalar value s,we can write ⎡ ⎤ ⎡ ⎤ a11 a12 ... a1m sa11 sa12 ... sa1m ⎢ ⎥ ⎢ ⎥ ⎢ a21 a22 ... a2m ⎥ ⎢ sa21 sa22 ... sa2m ⎥ sA = s ⎢ . ⎥ = ⎢ . ⎥ . (A.12) ⎣ . .. ⎦ ⎣ . .. ⎦ an1 an2 ... anm san1 san2 ... sanm Example A.5 [Matrices–scalar multiplication ] 123 If a matrix A is defined as A = , get the result of sA where s = 2.5. 456 Solution A.5 The scalar value is multiplied by each matrix term individually. Thus, 123 2.557.5 sA = 2.5 = . 456 10 12.515 Example A.6 [Matrices–scalar multiplication and addition/subtraction ] 1234 355 6 Consider two matrices A and B where A = and B = . 5677 78910 Calculate A+2B and 2A−B. Solution A.6 1234 355 6 A + 2B = + 2 5677 78910 = 1234 + 6101012 5677 14 16 18 20 + + + + = 1 62 10 3 10 4 12 5 + 14 6 + 16 7 + 18 7 + 20 = 7121316 . 19 22 25 27 632 Appendix A: Linear Algebra 1234 355 6 2A − B = 2 − 5677 78910 = 2468 − 355 6 10 12 14 14 78910 − − − − = 2 34 56 58 6 10 − 712− 814− 914− 10 − − = 1 112 . 3454 A.2.6 Matrix Multiplication In order to multiply two matrices A and B, the number of columns of the first matrix A must be equal to the number of rows of the second B. The resulting matrix has the same number of rows as the first and the same number of columns as the second. Thus, if A is of size n × m and B is of size m × k, the resulting matrix C will be of size n × k. This can be written as AB = ⎡C ⎤ c11 c12 ... c1k ⎢ ⎥ ⎢ c21 c22 ... c2k ⎥ = ⎢ . ⎥ ⎣ . .. ⎦ cn1 cn2 ... cnk ⎡ ⎤ i< =m i< =m i< =m ⎢ a i bi a i bi ... a i bik ⎥ (A.13) ⎢ 1 1 1 2 1 ⎥ ⎢ i=1 i=1 i=1 ⎥ ⎢ i< =m i< =m i< =m ⎥ ⎢ ... ⎥ ⎢ a2i bi1 a2i bi2 a2i bik ⎥ = ⎢ i=1 i=1 i=1 ⎥ , ⎢ . ⎥ ⎢ . .. ⎥ ⎢ ⎥ ⎣ i< =m i< =m i< =m ⎦ anibi1 anibi2 ..