MATH3602 Chapter 2 Advanced Direct Methods 1 2 Advanced Direct Methods 2.1 Use of Permutation (Similarity) in Solving Block-Diagonal System • Suppose we are to" solve the following# system of linear equation: Ax = b where D11 D12 A = and Dij = Diag(Dij; ··· ;Dij) D21 D22 1 n are n × n diagonal matrices for i; j = 1; 2. Here A is a matrix of 2 × 2 block (same size n × n of diagonal matrix). • 2 3 2 3 2 3 D11 D12 x b 6 1 1 7 6 1 7 6 1 7 6 ... ... 7 6 . 7 6 . 7 6 7 6 7 6 7 6 D11 D12 7 6 x 7 6 b 7 6 n n 7 6 n 7 = 6 n 7 6 21 22 7 6 7 6 7 6 D1 D1 7 6 xn+1 7 6 bn+1 7 4 ... ... 5 4 . 5 4 . 5 21 22 Dn Dn x2n b2n • LU factorization may need O(n3) operation? MATH3602 Chapter 2 Advanced Direct Methods 2 • Any strategy you can propose apart from LU factorization? We consider the following example:2 3 6 5 0 0 2 0 0 7 6 7 6 0 4 0 0 1 0 7 6 7 6 0 0 5 0 0 2 7 6 7 · A = 6 7 = L U 6 1 0 0 4 0 0 7 6 7 4 0 3 0 0 3 0 5 0 0 1 0 0 5 2 3 2 3 6 1:00 0 0 0 0 0 7 6 5:00 0 0 2 0 0 7 6 7 6 7 6 0 1:00 0 0 0 0 7 6 0 4:00 0 0 1 0 7 6 7 6 7 6 0 0 1:00 0 0 0 7 6 0 0 5:00 0 0 2 7 · ≡ 6 7 · 6 7 L U 6 7 6 7 6 0:20 0 0 1:00 0 0 7 6 0 0 0 3:60 0 0 7 6 7 6 7 4 0 0:75 0 0 1:00 0 5 4 0 0 0 0 2:25 0 5 0 0 0:20 0 0 1:00 0 0 0 0 0 4:60 MATH3602 Chapter 2 Advanced Direct Methods 3 • One can consider a re-arrangement of both the equations and variables and get the following equivalent system of equations A~x~ = b~: 2 3 2 3 2 3 D11 D12 x b 6 1 1 7 6 1 7 6 1 7 6 21 22 7 6 7 6 7 6 D1 D1 7 6 xn+1 7 6 bn+1 7 6 11 12 7 6 7 6 7 6 D2 D2 7 6 x2 7 6 b2 7 6 7 6 7 6 7 6 D21 D22 7 6 x 7 6 b 7 6 2 2 7 6 n+2 7 6 n+2 7 6 . 7 6 . 7 = 6 . 7 6 .. 7 6 . 7 6 . 7 6 7 6 7 6 7 6 ... 7 6 . 7 6 . 7 6 7 6 7 6 7 4 11 12 5 4 5 4 5 Dn Dn xn bn 21 22 Dn Dn x2n b2n MATH3602 Chapter 2 Advanced Direct Methods 4 • It is clear to see that the linear system has a unique solution iff " #! 11 12 Di Di 6 det 21 22 = 0 for i = 1; 2; : : : ; n: Di Di • The solution is given by (i = 1; 2; : : : ; n) " # " # x 1 D22b − D12b i = i i i n+i 11 22 − 21 12 11 − 21 xn+i Di Di Di Di Di bn+i Di bi • The computational cost is O(n). • The memory cost is also O(n). • There exists a permutation matrix P such that A~ = P · A · P T where P T = P −1. MATH3602 Chapter 2 Advanced Direct Methods 5 • The following is an example when n = 2: 2 3 2 3 2 3 2 3 11 12 11 12 6 D1 D1 7 6 1 0 0 0 7 6 D1 D1 7 6 1 0 0 0 7 6 21 22 7 6 7 6 11 12 7 6 7 6 D1 D1 7 6 0 0 1 0 7 6 D2 D2 7 6 0 0 1 0 7 6 11 12 7 = 6 7 6 21 22 7 6 7 4 D2 D2 5 4 0 1 0 0 5 4 D1 D1 5 4 0 1 0 0 5 21 22 21 22 D2 D2 0 0 0 1 D2 D2 0 0 0 1 2 3 2 3 2 3 11 12 11 12 6 D1 D1 7 6 1 0 0 0 7 6 D1 D1 7 6 21 22 7 6 7 6 11 12 7 • 6 D1 D1 7 6 0 0 1 0 7 6 D2 D2 7 6 11 12 7 = 6 7 6 21 22 7 4 D2 D2 5 4 0 1 0 0 5 4 D1 D1 5 D21 D22 0 0 0 1 D21 D22 2 2 2 3 2 32 2 2 3 11 12 11 12 6 D1 D1 7 6 D1 D1 7 6 1 0 0 0 7 6 21 22 7 6 21 22 7 6 7 • 6 D1 D1 7 6 D1 D1 7 6 0 0 1 0 7 6 11 12 7 = 6 11 12 7 6 7 4 D2 D2 5 4 D2 D2 5 4 0 1 0 0 5 21 22 21 22 D2 D2 D2 D2 0 0 0 1 MATH3602 Chapter 2 Advanced Direct Methods 6 • To solve Ax = b, i.e., to solve AP~ x = P b: Stage 1 : A~y = P b (solve for y) Stage 2 : P x = y (solve for x) • The computational cost of P x can be ignored or is negligible. • The idea can be extended to a matrix of m × m block (same size of diagonal matrix). • If m << n, then the computational cost will be O(m3n). Why? • The method can be implemented in a parallel computer easily. If there are O(n) processors, then the computational cost can be reduced to O(m3). MATH3602 Chapter 2 Advanced Direct Methods 7 • LU factorization can be applied wisely in diagonal block form: 2 3 11 12 6 D1 D1 7 6 . 7 6 .. .. 7 6 7 6 D11 D12 7 6 n n 7 6 21 22 7 6 D1 D1 7 6 7 4 ... ... 5 21 22 Dn Dn 2 3 2 3 1 2 6 1 7 6 U1 U1 7 6 . 7 6 . 7 6 .. 7 6 .. .. 7 6 7 6 7 6 1 7 6 U 1 U 2 7 = 6 7 6 n n 7 6 7 6 1 7 6 L1 1 7 6 Un+1 7 6 7 6 7 4 ... ... 5 4 ... 5 1 Ln 1 U2n • The idea can be extended to a matrix of m × m block (same size of diagonal matrix). If m << n, the computational cost will be O(m3n). MATH3602 Chapter 2 Advanced Direct Methods 8 2.2 Use of Similarity in Solving Circulant Matrix System • A matrix is called a circulant matrix if it takes the following form: 2 3 ······ 6 a1 a2 an−1 an 7 6 7 6 an a1 a2 ······ an−1 7 6 . 7 6 a − a a ······ . 7 6 n 1 n 1 7 An = 6 . 7 : 6 . .. .. .. .. 7 6 . 7 4 a3 ······ .. a1 a2 5 a2 a3 ··· an−1 an a1 • We note that each row is just the previous row cycled forward for one step. • Thus an n × n circulant matrix is characterized by n coefficients only. 0 0 • In an n × n circulant matrix, we have Cij = Ci0j0 if i − j ≡ i − j (mod n). MATH3602 Chapter 2 Advanced Direct Methods 9 • In fact, we can write Xn−1 i An = ai+1Cn (2.1) i=0 where C is a permutation matrix n 2 3 0 1 0 ··· 0 6 7 6 7 6 0 0 1 0 7 6 . 7 Cn = 6 . .. .. 7 : 6 7 4 . ... ... 1 5 1 0 ······ 0 • i 0 n We note that Cn are all permutation matrices and Cn = Cn = In. • Moreover, An is diagonalized if Cn is diagonalizable. MATH3602 Chapter 2 Advanced Direct Methods 10 • i It can be shown that all Cn can be diagonalized by using the discrete Fast Fourier Transform (FFT) matrix Fn. • The discrete FFT matrix is defined2 as follows: 3 0·0 0·1 ··· 0·(n−1) 6 wn wn wn 7 6 1·0 1·1 ··· 1·(n−1) 7 6 wn wn wn 7 Fn = Fn(wn) = 6 7 4 . 5 (n−1)·0 (n−1)·1 (n−1)·(n−1) wn wn ··· wn where ( ) ( ) −2πi 2π 2π w = e n = cos − i sin : n n n • It can be shown that (Exercise) 2 3 − · − w−0·0 w−0·1 ··· w 0 (n 1) ( ) 6 n n n 7 6 −1·0 −1·1 ··· −1·(n−1) 7 −1 1 1 1 6 wn wn wn 7 Fn = Fn = 6 . 7 : n wn n 4 . 5 −(n−1)·0 −(n−1)·1 −(n−1)·(n−1) wn wn ··· wn MATH3602 Chapter 2 Advanced Direct Methods 11 • A demonstration2 3 2 3 2 3 2 3 6 0 1 0 0 7 6 0 0 1 0 7 6 0 0 0 1 7 6 1 0 0 0 7 6 7 6 7 6 7 6 7 6 0 0 1 0 7 2 6 0 0 0 1 7 3 6 1 0 0 0 7 4 6 0 1 0 0 7 C4 = 6 7 ;C = 6 7 ;C = 6 7 ;C = 6 7 ; 4 0 0 0 1 5 4 4 1 0 0 0 5 4 4 0 1 0 0 5 4 4 0 0 1 0 5 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 • 2 3 2 3 6 1 1 1 1 7 6 0:25 0:25 0:25 0:25 7 6 − − 7 6 − − 7 6 1 i 1 i 7 −1 6 0:25 0:25i 0:25 0:25i 7 F4 = 6 7 F = 6 7 : 4 1 −1 1 −1 5 4 4 0:25 −0:25 0:25 −0:25 5 1 i −1 −i 0:25 −0:25i −0:25 0:25i • · −1 · · −1 − − Here F4 F4 = I4 and F4 C4 F4 = Diag(1; i; 1; i).