WS 2009/2010 Numerical Linear Algebra Professor Dr. Christoph Pflaum Contents 1 Linear Equation Systems in the Numerical Solution of PDE’s 5 1.1 ExamplesofPDE’s........................ 5 1.2 Finite-Difference-Discretization of Poisson’s Equation..... 7 1.3 FD Discretization for Convection-Diffusion . 8 1.4 Irreducible and Diagonal Dominant Matrices . 9 1.5 FE (Finite Element) Discretization . 12 1.6 Discretization Error and Algebraic Error . 15 1.7 Basic Theory for LInear Iterative Solvers . 15 1.8 Effective Convergence Rate . 18 1.9 Jacobi and Gauss-Seidel Iteration . 20 1.9.1 Ideas of Both Methods . 20 1.9.2 Description of Jacobi and Gauss-Seidel Iteration by Matrices.......................... 22 1.10 Convergence Rate of Jacobi and Gauss-Seidel Iteration . 24 1.10.1 General Theory for Weak Dominant Matrices . 24 1.10.2 Special Theory for the FD-Upwind . 26 1.10.3 FE analysis, Variational approach . 30 1 1.10.4 Analysis of the Convergence of the Jacobi Method . 33 1.10.5 Iteration Method with Damping Parameter . 34 1.10.6 Damped Jacobi Method . 35 1.10.7 Analysis of the Damped Jacobi method . 35 1.10.8 Heuristic approach . 37 2 Multigrid Algorithm 38 2.1 Multigrid algorithm on a Simple Structured Grid . 38 2.1.1 Multigrid ......................... 38 2.1.2 Idea of Multigrid Algorithm . 39 2.1.3 Two–grid Multigrid Algorithm . 40 2.1.4 Restriction and Prolongation Operators . 41 2.1.5 Prolongation or Interpolation . 41 2.1.6 Pointwise Restriction . 41 2.1.7 Weighted Restriction . 42 2.2 Iteration Matrix of the Two–Grid Multigrid Algorithm . 42 2.3 Multigrid Algorithm . 43 2.4 Multigrid Algorithm for Finite Elements . 44 2.4.1 ModelProblem...................... 44 2.4.2 Example.......................... 44 2.4.3 TheNodalBasis ..................... 45 2.4.4 Prolongation Operator for Finite Elements . 45 2 2.4.5 Restriction Operator for Finite Elements . 46 2.5 Fourier Analysis of the Multigrid method . 47 2.5.1 Local Fourier analysis . 47 2.5.2 Definition ......................... 51 2.5.3 Local Fourier analysis of the smoother . 51 3 Gradient Method and cg 52 3.1 GradientMethod ......................... 52 3.2 Analysis of the Gradient Method . 53 3.3 The Method of Conjugate Directions . 55 3.4 cg-Method (Conjugate Gradient Algorithm) . 57 3.5 Analysis of the cg algorithm . 59 3.6 Preconditioned cg Algorithm . 61 4 GMRES 63 4.1 Minimalresidualmethod . 64 4.2 Solution of the Minimization Problem of GMRES . 65 4.3 Computation of QR-Decomposition with Givens Rotation . 66 4.4 TheGMRESAlgorithm . 67 4.5 Convergence of the GMRES method . 68 5 Eigenvalue Problems 69 5.1 RayleighQuotient ........................ 69 3 5.2 Method of Conjugate Gradients . 71 5.3 Simple Vector Iteration . 74 5.4 Computation of Eigenvalues using the Rayleigh Quotient . 76 5.5 Jacobi-Davidson-Algorithm . 80 5.5.1 The Jacobi-Method . 80 5.5.2 Motivation of Davidson’s Algorithm . 81 5.5.3 The concept of the Jacobi-Davidson-Algorithm . 82 5.5.4 Jacobi-Davidson-Algorithm . 84 4 1 Linear Equation Systems in the Numerical So- lution of PDE’s 1.1 Examples of PDE’s 1. Heat Equation temperature g at the boundary hom. plate Heat source f in the interior of the plate. Question: What is the temperature inside of the plate? Poisson Problem (P) Let Ω Rn open, bounded, f C(Ω), g C(δΩ). ⊂ ∈ ∈ Find u C2(Ω) such that ∈ ∆u = f on Ω − u δΩ = g ∂2 ∂2 where ∆ = + ∂x2 ∂y2 2. Convection-Diffusion-Problem Find u C2(Ω) such that ∈ ∆u +~b u + cu = f on Ω − ·∇ u δΩ = 0 2 where ~b (C(Ω)) , f,c C(Ω) ∈ ∈ 3. Navier-Stokes-Equation ∂u ∂p ∂(u2) ∂(uv) 1 + + + = ∆u ∂t ∂x ∂x ∂y Re ∂u ∂p ∂(uv) ∂(v2) 1 + + + = ∆v ∂t ∂y ∂x ∂y Re ∂u ∂v + = 0 ∂x ∂y 5 u v 4. Laser simulation mirror 1 mirror 2 M M2 1 r Γ =Γ Γ M M1 ∪ M2 2 Find u CC(Ω), λ C such that ∈ ∈ ∆u k2u = λu − − u = 0 ΓM ∂u = 0 (or boundary condition third kind) ∂~n Γ rest We apply the ansatz −ikz˜ ikz˜ u = ure + ule where k˜ is an average value of k. This leads to the equivalent eigenvalue problem: Find ur, ul, λ such that ∂u ∆u + 2ik˜ r + (k˜2 k2)u = λu − r ∂z − r r ∂u ∆u 2ik˜ l + (k˜2 k2)u = λu − l − ∂z − l l ∂ur ∂ul ur + ul = 0, = 0 ΓM ∂z − ∂z ΓM ∂ur ∂ul = = 0 ∂~n Γrest ∂~n Γrest 6 1.2 Finite-Difference-Discretization of Poisson’s Equation Assume Ω =]0, 1[2 and that an exact solution of (P) exists. We are looking for an approximate solution uh of (P) on a grid Ωh of meshsize h. Choose h = 1 where m N. m ∈ Ω = (ih, jh) i, j = 1,...,m 1 h − Ωh = (ih, jh) i, j = 0,...,m Discretization by Finite Differences: Idea: Replace second derivative by difference quotient. Let ex = (1, 0) and ey = (1, 0), ∂2u ∂2u ∆u(z)= (z)= f(z) for z Ω − −∂x2 − ∂y2 ∈ h u (z + he ) 2u (z)+ u (z he ) h x − h h − x − h2 u (z + he ) 2u (z)+ u (z he ) h y − h h − y = f(z) − h2 and u(z) = g(z) = for z Ω Ω ≈ ∈ h\ h uh(z) = g(z) This leads to a linear equation system L U = F where U = (u (z)) , h h h h h z∈Ωh L is Ω Ω matrix. The discretization can be described by the stencil h | h| × | h| 1 m m m − h2 −1,1 0,1 1,1 1 4 1 = m m m − h2 h2 − h2 −1,0 0,0 1,0 1 m m m − h2 −1,−1 0,−1 1,−1 X X X X X X X X X X X X X X X X 7 Let us abbreviate Ui,j := uh(ih, jh) and fi,j := f(ih, jh). Then, in case of g = 0, the matrix equation LhUh = Fh is equivalent to: 1 mklUi+k,j+l = fi,j k,lX=−1 1.3 FD Discretization for Convection-Diffusion Let Ω, Ωh as above. du ∆u + b = f − dx Assume that b is constant. 1. Discretization by central difference: du u (z + he ) u (z he ) (z) h x − h − x dx ≈ 2h This leads to the stencil 1 − h2 1 b 4 1 + b − h2 − 2h h2 − h2 2h 1 h2 − unstable for large b. → 2. Upwind discretization: du u (z) u (z he ) (z) h − h − x dx ≈ h This leads to the stencil 1 − h2 1 b 4 + b 1 − h2 − h h2 h − h2 1 h2 − 8 1.4 Irreducible and Diagonal Dominant Matrices Definition 1. A n n matrix A is called strong diagonal dominant, if × a > a 1 i n (1) | ii| | ij| ≤ ≤ Xi6=j A is called weak diagonal dominant, if there exists at least one i such that (1) holds and such that a a 1 i n | ii|≥ | ij| ≤ ≤ Xi6=j Definition 2. A is called reducible, if there exists a subset J 6= 1, 2,...,n , ⊂ { } J = . such that 6 ∅ a = 0 for all i J, j J ij 6∈ ∈ A not reducible matrix is called irreducible. Remark. An reducible matrix has the form A11 A12 0 A 22 The equation system separates in two parts. → Example: 1. Poisson FD: 4 diagonal: aii = h2 1 if i is N,S,W,O of j non-diagonal: a = − h2 ij 0 else A is not strong diagonal dominant, but weak diagonal dominant. • To see this, consider a point i such that j is N of i. Then 1 if i is S,W,O of j a = − h2 ij 0 else A is irreducible. • Proof: If A is reducible, then, 1, 2, ..., n is the union of two { } different sets of colored points, where one set is J. Then, there is a point j J such that one of the points i=N,W,S,E is not ∈ contained in J, but i is contained in 1, 2, ..., n . This implies { } a = 0. contradiction. j,i 6 ⇒ 9 2. Convection-Diffusion-Equation centered difference • 4 a = | ii| h2 4 1 1 b 1 b a = 2 + + + | ij| h2 · h2 h2 2h h2 − 2h i6=j X 1 b 1 b = 3 + + h2 2h h2 − 2h Thus, a a , if and only if 1 b 0. | ii|≥ i6=j | ij| h2 − 2h ≤ This shows a a , if and only if h< 2 | Pii|≥ i6=j | ij| b upwind • P 4 b a = + | ii| h2 h 4 b ≥ + a for all h,b > 0 h2 h ≥ | ij| Xi6=j Conclusion • 2 central: A is weak diagonal dominant if and only if h< b . upwind: A is weak diagonal dominant. A is irreducible in both cases. Definition 3. Let A be an n n matrix. Consider n points P1, ..., Pn. −→ × Draw an edge between Pi, Pj if ai,j = 0. The directed graph of A is this set 6 −→ of points P1, ..., Pn with these edges Pi, Pj. Definition 4. A directed graph is called strongly connected, if for every pair of disjoint points Pi, Pj there exists a directed path in the graph. This means −→ −→ −→ there exists a path Pi0 Pi1 , Pi2 Pi3 , ..., Pir−1 Pir such that Pi0 = Pi andPir = Pj. Theorem 1. A n n matrix A is irreducible, if and only if its directed × graph is connected. Proof. Let A be irreducible.