The Poisson matrix and Kronecker Products Tom Lyche University of Oslo Norway The Poisson matrix and Kronecker Products – p. 1/27 Plan for the day Numerical solution of the Poisson problem The finite difference scheme Formulation as a matrix equation Formulation in standard form Ax = b. The Poisson matrix A is block tridiagonal LU factorization of a block tridiagonal matrix Solving the Poisson problem, complexity Kronecker product Relation to Poisson problem Basic properties Eigenvalues and eigenvectors Eigenpairs for the Poisson matrix The Poisson matrix and Kronecker Products – p. 2/27 Numerical solution of the Poisson Problem u=0 u=0 −(u +u ) =f u=0 xx yy u=0 m N, h = 1/(m + 1), vj,k u(jh, kh) ∈ ≈ (m+1)h j,k+1 (jh,kh) kh j-1,k j,k j+1,k 0 j,k-1 0 jh (m+1)h The Poisson matrix and Kronecker Products – p. 3/27 Discrete Equation ′′ g(t h) 2g(t)+ g(t + h) g (t) − − ≈ h2 v j,k+1 v v j-1,k vj,k j+1,k v j,k-1 From differential equation for j,k = 1, 2,...,m 2 ( vj−1,k + 2vj,k vj+1,k)+( vj,k−1 + 2vj,k vj,k+1)= h fj,k (1) − − − − From boundary conditions vj,k = 0 if j = 0, m + 1 or k = 0, m + 1 (2) The Poisson matrix and Kronecker Products – p. 4/27 Matrix Equation T V + V T = h2F 2 1 v11 v12 v11 v12 2 1 1 f11 f12 − + − = 1 2 v21 v22 v21 v22 1 2 9 f21 f22 − − m,m T := T m = tridiag ( 1, 2, 1) R m − − ∈ v11 v1m f11 f1m ··· ··· . Rm,m . Rm,m V := . F := . ∈ ∈ vm1 vmm fm1 fmm ··· ··· 2 (1), (2) TV + VT = h F ⇔ m m 2 TV + VT = T jivik + vjiT ik = l.h.s in (1) = h (F )jk jk X X i=1 i=1 The Poisson matrix and Kronecker Products – p. 5/27 Convert T V + V T = h2F to Ax = b 1,3 2,3 3,3 7 8 9 1,2 2,2 3,2 > 4 5 6 1,1 2,1 3,1 1 2 3 v in grid x in grid j,k i -v j,k+1 -x i+m - 4 -v -x -x v j-1,k vj,k j+1,k i-1 4x i i+1 --> -v j,k-1 -xi-m 4xi xi−1 xi+1 xi−m xi+m = bi − − − − The Poisson matrix and Kronecker Products – p. 6/27 Linear system Ax = b For m = 3 or n = 9 we have the following linear system: 2 4x1 x2 x4 = h f11 − − 2 x1 4x2 x3 x5 = h f21 − − − 2 x2 4x3 x6 = h f31 − − 2 x1 4x4 x5 x7 = h f12 − − − 2 x2 x4 4x5 x6 x8 = h f22 − − − − 2 x3 x5 4x6 x9 = h f32 − − − 2 x4 4x7 x8 = h f13 − − 2 x5 x7 4x8 x9 = h f23 − − − 2 x6 x8 4x9 = h f33 − − The Poisson matrix and Kronecker Products – p. 7/27 Blockstructure of A 4 1 0 1 0 0 0 0 0 − − 1 4 1 0 1 0 0 0 0 − − − 0 1 4 0 0 1 0 0 0 − − 1 0 0 4 1 0 1 0 0 − − − A = 0 1 0 1 4 1 0 1 0 − − − − 0 0 1 0 1 4 0 0 1 − − − 0 0 0 1 0 0 4 1 0 − − 0 0 0 0 1 0 1 4 1 − − − 0 0 0 0 0 1 0 1 4 − − T + 2I I 0 − A = I T + 2I I − − 0 I T + 2I − Block tridiagonal The Poisson matrix and Kronecker Products – p. 8/27 LU-factorization of a tridiagonal matrix d1 c1 1 u1 c1 a2 d2 c2 .. .. . l2 1 . .. .. .. = . .. .. . un−1 cn−1 a −1 d −1 c −1 n n n ln 1 un an dn (3) Note that L has ones on the diagonal, and that A and U have the same super-diagonal. By equating entries in (3) we find d1 = u1, ak = lkuk−1, dk = lkck−1 + uk, k = 2, 3,...,n. The Poisson matrix and Kronecker Products – p. 9/27 The Algorithm A = LU (LU-factorization) Ly = b (forward substitution) Ux = y (backward substituion) ak u1 = d1, lk = , uk = dk lkck−1, k = 2, 3,...,n. uk−1 − y1 = b1, yk = bk lkyk−1, k = 2, 3,...,n, − xn = yn/dn, xk =(yk ckxk+1)/uk, k = n 1,..., 2, 1. − − This process is well defined if A is strictly diagonally dominant. The number of arithmetic operations (flops) is O(n). The Poisson matrix and Kronecker Products – p. 10/27 LU of Block Tridiagonal Matrix D1 C1 1 U 1 C1 A2 D2 C2 L2 1 . .. .. .. .. .. = .. .. U m−1 Cm−1 Am−1 Dm−1 Cm−1 Lm 1 U m Am Dm −1 U 1 = D1, Lk = AkU − , U k = Dk LkCk−1, k = 2, 3,...,m. (4) k 1 − To solve the system Ax = b we partition b conformally with A in the form T T T b =[b1 ,..., bm]. The formulas for solving Ly = b and Ux = y are as follows: y = b1, y = bk Lky − , k = 2, 3,...,m, 1 k − k 1 −1 −1 (5) xm = U y , xk = U (y Ckxk+1), k = m 1,..., 2, 1. m m k k − − T T T The solution is then x =[x1 ,..., xm]. The Poisson matrix and Kronecker Products – p. 11/27 Poisson Problem The discrete Poisson problem can be solved writing it as a block tridiagonal matrix. We can also use Cholesky factorization of a band matrix. The matrix has bandwidth d = √n. Cholesky requires O(2nd2)= O(2n2) flops. The work using block tridiagonal solver is also O(n2). Need better methods (next time). The Poisson matrix and Kronecker Products – p. 12/27 Kronecker Product Definition 1. For any positive integers p,q,r,s we define the Kronecker product of two matrices A Rp,q and B Rr,s as a matrix C Rpr,qs given in block form as ∈ ∈ ∈ Ab1,1 Ab1,2 Ab1,s ··· Ab2,1 Ab2,2 Ab2,s ··· C = . . . . Abr,1 Abr,2 Abr,s ··· We denote the Kronecker product of A and B by C = A B. ⊗ Defined for rectangular matrices of any dimension # of rows(columns) = product of # of rows(columns) in A and B The Poisson matrix and Kronecker Products – p. 13/27 Kronecker Product Example 2 1 1 0 T = − , I = 1 2 0 1 − 2 1 0 0 − T 0 1 2 0 0 T I = = − ⊗ 0 T 0 0 2 1 − 0 0 1 2 − 2 0 1 0 − 2I I 0 2 0 1 I T = − = − ⊗ I 2I 1 0 2 0 − − 0 1 0 2 − The Poisson matrix and Kronecker Products – p. 14/27 Poisson Matrix = T I + I T ⊗ ⊗ T 2I I 0 − A = T + I 2I I = T I + I T ⊗ ⊗ − − T 0 I 2I − r,r s,s Definition 2. Let for positive integers r,s,k, A R , B R and Ik be the ∈ ∈ identity matrix of order k. The sum A Is + Ir B is known as the Kronecker sum of ⊗ ⊗ A and B. The Poisson matrix is the Kronecker sum of T with itself. The Poisson matrix and Kronecker Products – p. 15/27 Matrix equation Kronecker ↔ Given A Rr,r, B Rs,s, F Rr,s. Find V Rr,s such that ∈ ∈ ∈ ∈ AV BT = F For B Rm,n define ∈ b1 b1j b2 b2j mn vec(B) := R , bj = jth column . ∈ . . . bn bmj Lemma 1. AV BT = F (A B) vec(V ) = vec(F ) (6) ⇔ ⊗ T AV + V B = F (A Is + Ir B) vec(V ) = vec(F ). (7) ⇔ ⊗ ⊗ The Poisson matrix and Kronecker Products – p. 16/27 Proof T We partition V , F , and B by columns as V =(v1,..., vs), T F =(f 1,..., f s) and B =(b1,..., bs). Then we have (A B) vec(V ) = vec(F ) ⊗ Ab11 Ab1s v1 f1 ··· . . . = . ⇔ Abs1 Abss vs fs ··· A bij vj = fi, i = 1,...,m ⇔ X j T [AV b1,..., AV bs]= F AV B = F . ⇔ ⇔ This proves (6) The Poisson matrix and Kronecker Products – p. 17/27 Proof Continued (A Is + Ir B) vec(V ) = vec(F ) ⊗ ⊗ T T T (AV I + IrV B )= F AV + V B = F . ⇔ s ⇔ This gives a slick way to derive the Poisson matrix. Recall T AV + V B = F (A Is + Ir B) vec(V ) = vec(F ) ⇔ ⊗ ⊗ Therefore, TV + VT = h2F (T I + I T ) vec(V )= h2 vec(F ) ⇔ ⊗ ⊗ The Poisson matrix and Kronecker Products – p. 18/27 Mixed Product Rule (A B)(C D)=(AC) (BD) ⊗ ⊗ ⊗ Proof If B Rr,t and D Rt,s for some integers r, t then ∈ ∈ Ab1,1 Ab1,t Cd1,1 Cd1,s E1,1 E1,s ··· ··· ··· . . . = . , Abr,1 Abr,t Cdt,1 Cdt,s Er,1 Er,s ··· ··· ··· where for all i, j t Ei,j = bi,kdk,jAC =(AC)(BD)i,j = ((AC) (BD))i,j. X ⊗ k=1 where in the last formula i, j refers to the ij-block in the Kronecker product. The Poisson matrix and Kronecker Products – p. 19/27 Properties of Kronecker Products The usual arithmetic rules hold. Note however that (A B)T = AT BT ⊗ ⊗ − − (A B)−1 = A 1 B 1 if A and B are nonsingular ⊗ ⊗ A B = B A ⊗ 6 ⊗ (A B)(C D)=(AC) (BD) (mixed product rule) ⊗ ⊗ ⊗ The mixed product rule is valid for any matrices as long as the products AC and BD are defined.