Matrix Multiplication Let A be l × m matrix, and B be m × n matrix. Then C = A × B is given by m X cij = aik × bkj ; (1) k=1 where aik ; bkj ; cij are defined as before. Zheng-Liang Lu 312 / 354 For example, Pm Note that if n = 1, then cij = k=1 aik × bkj becomes the matrix A times the vector B. Zheng-Liang Lu 313 / 354 Example l×m m×n Let A 2 R and B 2 R . Please implement a program that A multiplies B and compare your result to A × B provided in MATLAB. For simplicity, the dimensions l; m; n are the inputs to generate random matrices by rand. Input: l; m; n for [A]l×m and [B]m×n Output: A × B Zheng-Liang Lu 314 / 354 1 clear; clc; 2 % main 3 l = randi(5, 1);%A flxm g 4 m = randi(5, 1);%B fmxn g 5 n = randi(5, 1); 6 A = randi(10, l, m); 7 B = randi(10, m, n); 8 C = zeros(size(A, 1), size(B, 2)); 9 for i = 1 : 1 : size(A, 1) 10 for j = 1 : 1 : size(B, 2) 11 for k = 1 : 1 : size(A, 2) 12 C(i, j) = C(i, j) + A(i, k) * B(k, j); 13 end 14 end 15 end Time complexity: O(n3) Strassen (1969): O(n2:807355) Coppersmith and Winograd (2010): O(n2:375477) Zheng-Liang Lu 315 / 354 Matrix Exponentiation Raising a matrix to a power is equivalent to repeatedly multiplying the matrix by itself. For example, A2 = A × A. Note that A should be a square matrix. (Why?) In mathematics, the matrix exponential1 is a matrix function on square matrices analogous to the ordinary exponential function. That is, f (A) = eA, similar to f (x) = ex . However, raising a matrix to a matrix power, that is, AB , is undefined. 1See matrix exponential and Pauli matrices. Zheng-Liang Lu 316 / 354 Example Be aware that exp(X ) and exp(1)X are different for any matrix X . 1 clear; clc; 2 % main 3 X = [0 1; 1 0]; 4 mat exp1 = exp(1) ˆ X 5 mat exp2 = 0; 6 for i = 1 : 10% check the identity ofeˆa 7 mat exp2 = mat exp2 + X ˆ (i - 1) / ... factorial(i - 1); 8 end 9 mat exp2 10 mat exp3 = exp(X)% different? Zheng-Liang Lu 317 / 354 1 mat exp1 = 2 3 1.5431 1.1752 4 1.1752 1.5431 5 6 7 mat exp2 = 8 9 1.5431 1.1752 10 1.1752 1.5431 11 12 13 mat exp3 = 14 15 1.0000 2.7183 16 2.7183 1.0000 Zheng-Liang Lu 318 / 354 Determinant det(A) is the determinant of the square matrix A.2 a b Recall that if A = , then det(A) = ad − bc. c d det(A) can calculate the determinant of A. For example, 1 clear; clc; 2 % main 3 A = magic(3); 4 det(A) 2In math, jAj also denotes det(A). Zheng-Liang Lu 319 / 354 Actually, a general formula3 for determinant is somehow complicated. You can try a recursive algorithm to calculate the determinant for any n-by-n matrices. 3See determinant. Zheng-Liang Lu 320 / 354 Recursive Algorithm for Determinant 1 function y = myDet(X) 2 [r, c] = size(X); 3 if isequal(r,c)% checkr==c? 4 if c == 1 5 y = X; 6 elseif c == 2 7 y = X(1, 1) * X(2, 2) - X(1, 2) * X(2, 1); 8 else 9 fori=1:c 10 if i == 1 11 % recall thatc ==r 12 y = X(1, 1) * myDet(X(2 : c, 2 : c)); 13 elseif (i > 1 && i < c) 14 y = y + X(1, i) * (-1)ˆ(i + 1) *... 15 myDet(X(2 : c, [1 : i-1, i + 1 : ... c])); 16 else Zheng-Liang Lu 321 / 354 17 y = y + X(1, c) * (-1)ˆ(c + 1) * ... myDet(X(2 : c, 1 : c - 1)); 18 end 19 end 20 end 21 else 22 error('Should bea square matrix.') 23 end Obviously, there are n! terms in the calculation. So, this algorithm runs in O(en log n). Try any 10-by-10 matrix. Compare the running time with det.4 4By LU decomposition with running time O(n3). Zheng-Liang Lu 322 / 354 Inverse Matrices In linear algebra, an n-by-n square matrix A is invertible if there exists an n-by-n square matrix B such that AB = BA = In; where In denotes the n-by-n identity matrix. Note that eye is similar to zeros and returns an identity matrix. The following statements are equivalent: A is invertible det(A) 6= 0 (nonsingular) A is full rank5 5See rank. Zheng-Liang Lu 323 / 354 inv(A) returns an inverse matrix of A. Note that inv(A) may return inf if A is badly scaled or nearly singular.6. So, if A is not invertible, then det(A) = 0. It is well-known that adj(A) A−1 = ; det(A) where adj(A) refers to the adjugate matrix of A. You can try to calculate an adjugate matrix7 for any n-by-n matrix. (Try.) 6That is, det(A) is near 0. 7See adjugate matrix. Zheng-Liang Lu 324 / 354 Example 1 clear; clc 2 % main 3 A = pascal(4);% Pascal matrix 4 B = inv(A) 1 >> B = 2 3 4.0000 -6.0000 4.0000 -1.0000 4 -6.0000 14.0000 -11.0000 3.0000 5 4.0000 -11.0000 10.0000 -3.0000 6 -1.0000 3.0000 -3.0000 1.0000 Try inv(magic(3)). Zheng-Liang Lu 325 / 354 System of Linear Equations A system of linear equations is a set of linear equations involving the same set of variables. For example, 8 < 3x +2y −z = 1 x −y +2z = −1 : −2x +y −2z = 0 After doing math, we have x = 1; y = −2; and z = −2. Zheng-Liang Lu 326 / 354 A general system of m linear equations with n unknowns can be written as 8 a11x1 +a12x2 ··· +a1nxn = b1 > <> a21x1 +a22x2 ··· +a2nxn = b2 . .. > . = . > : am1x1 +am2x2 ··· +amnxn = bm where x1;:::; xn are unknowns, a11;:::; amn are the coefficients of the system, and b1;:::; bm are the constant terms. Zheng-Liang Lu 327 / 354 So, we can rewrite the system of linear equations as a matrix equation, given by Ax = b: where 2 3 a11 a12 ··· a1n 6 a21 a22 ··· a2n 7 A = 6 7 ; 6 . .. 7 4 . 5 am1 am2 ··· amn 2 3 x1 6 . 7 x = 4 . 5 ; and xn 2 3 b1 6 . 7 b = 4 . 5 : bm Zheng-Liang Lu 328 / 354 General System of Linear Equations Let x be the column vector with n independent variables and m constraints.8 If m = n9, then there exists the unique solution x0. If m > n, then there is no exact solution but there exists one least-squares error solution such that kAx0 − bk2 is minimal. Let x 0 be the least-square error solution. Then the error is = Ax 0 − b, usually not zero. So, kAx 0 − bk2 = (Ax 0 − b)(Ax 0 − b) = 2 is minimal. If m < n, then there are infinitely many solutions. 8Assume that these equations are linearly independent. 9Equivalently, rank(A) = rank([A b]). Or, see Cramer's rule. Zheng-Liang Lu 329 / 354 Quick Glance at Method of Least Squares The method of least squares is a standard approach to the approximate solution of overdetermined systems. Zheng-Liang Lu 330 / 354 Matrix Division in MATLAB Consider Ax = b for a system of linear equations. Anb is the matrix division of A into B, which is roughly the same as inv(A)b, except that it is computed in a different way. Left matrix divide (n) or mldivide is to do x =A−1b =inv(A)b =Anb: Zheng-Liang Lu 331 / 354 Unique Solution (m = n) For example, 8 < 3x +2y −z = 1 x −y +2z = −1 : −2x +y −2z = 0 1 >> A = [3 2 -1; 1 -1 2; -2 1 -2]; 2 >> b = [1; -1; 0]; 3 >> x = A n b 4 5 1 6 -2 7 -2 Zheng-Liang Lu 332 / 354 Overdetermined System (m > n) For example, 8 < 2x −y = 2 x −2y = −2 : x +y = 1 1 >> A=[2 -1; 1 -2; 1 1]; 2 >> b=[2; -2; 1]; 3 >> x = A n b 4 5 1 6 1 Zheng-Liang Lu 333 / 354 Underdetermined System (m < n) For example, x +2y +3z = 7 4x +5y +6z = 8 1 >> A = [1 2 3; 4 5 6]; 2 >> b = [7; 8]; 3 >> x = A n b 4 5 -3 6 0 7 3.333 8 9 %(Why?) Zheng-Liang Lu 334 / 354 Gauss Elimination Recall the Gauss Elimination in high school. For example, 8 < 3x +2y −z = 1 x −y +2z = −1 : −2x +y −2z = 0 It is known that x = 1; y = −2; and z = −2. Zheng-Liang Lu 335 / 354 Check if det(A) = 0, then the program terminates; otherwise, the program continues. Loop to form upper triangular matrix which looks like 2 3 1 ¯a12 ··· ¯a1n 6 0 1 ··· ¯a2n 7 A¯ = 6 7 ; 6 . 7 4 . 1 . 5 0 0 ··· 1 and 2 3 b¯1 6 b¯2 7 b¯ = 6 7 : 6 . 7 4 . 5 b¯n where ¯aij s and b¯i s are the resulting values. Zheng-Liang Lu 336 / 354 Use backward substitution to determine the solution vector x by xn = b¯n; n X xi = b¯i − ¯aij xj ; j=i+1 where i 2 f1; ··· ; n − 1g.