EFFICIENT \BLACK-BOX" MULTIGRID SOLVERS FOR CONVECTION-DOMINATED PROBLEMS A thesis submitted to the University of Manchester for the degree of Doctor of Philosophy in the Faculty of Engineering and Physical Science 2011 Glyn Owen Rees School of Computer Science 2 Contents Declaration 12 Copyright 13 Acknowledgement 14 1 Introduction 15 2 The Convection-Di®usion Problem 18 2.1 The continuous problem . 18 2.2 The Galerkin approximation method . 22 2.2.1 The ¯nite element method . 26 2.3 The Petrov-Galerkin (SUPG) approximation . 29 2.4 Properties of the discrete operator . 34 2.5 The Navier-Stokes problem . 37 3 Methods for Solving Linear Algebraic Systems 42 3.1 Basic iterative methods . 43 3.1.1 The Jacobi method . 45 3.1.2 The Gauss-Seidel method . 46 3.1.3 Convergence of splitting iterations . 48 3.1.4 Incomplete LU (ILU) factorisation . 49 3.2 Krylov methods . 56 3.2.1 The GMRES method . 59 3.2.2 Preconditioned GMRES method . 62 3.3 Multigrid method . 67 3.3.1 Geometric multigrid method . 67 3.3.2 Algebraic multigrid method . 75 3.3.3 Parallel AMG . 80 3.3.4 Literature summary of multigrid . 81 3.3.5 Multigrid preconditioning of Krylov solvers . 84 3 3.4 A new tILU smoother . 85 4 Two-Dimensional Case Studies 89 4.1 The di®usion problem . 90 4.2 Geometric multigrid preconditioning . 95 4.2.1 Constant uni-directional wind . 95 4.2.2 Double glazing problem - recirculating wind . 106 4.2.3 Combined uni-directional and recirculating wind . 116 4.2.4 Multiple recirculating wind . 121 4.2.5 Summary . 127 4.3 Algebraic multigrid preconditioning . 132 4.4 Parallelisation . 140 4.5 Performance pro¯le and summary . 146 4.6 Floor-driven cavity problem . 149 5 Three-Dimensional Case Studies 155 5.1 Geometric multigrid preconditioning . 156 5.1.1 Constant uni-directional wind . 156 5.1.2 Double glazing problem - recirculating wind . 160 5.1.3 Summary . 169 5.2 Algebraic multigrid preconditioning . 170 5.3 Parallelisation . 175 5.4 Performance pro¯le and summary . 178 6 Conclusion and Future Work 181 A Additional Results 184 B Iterative Linear Solvers 185 B.1 Overview . 185 B.2 List of available iterative linear solvers . 185 B.3 How to change the LinearSolver . 186 B.4 Point iterative solver . 189 B.5 Geometric multigrid . 190 B.6 Interpolation matrix . 191 C HPCx Notes 194 C.1 Download . 194 C.2 Building the library . 195 C.3 Libtool . 195 4 C.4 OOMPHLIB . 195 Bibliography 198 (Word count: 43,408) 5 List of Tables 2.1 Diagonal dominance of the coe±cient matrix as a function of the h and P e ................................... 36 3.1 Spectral properties of the coe±cient matrix and iteration count . 61 3.2 Spectral properties of the coe±cient matrix preconditioned by di®erent point iterative methods . 65 4.1 Convergence of GMRES solvers applied to the solution of the discrete Poisson problem. (a) GMG preconditioner; (b) AMG preconditioner; (c) parallel AMG preconditioner . 92 4.2 Convergence of the GMRES/GMG solvers (Case study 4.2.1) . 98 4.3 Modulus of the maximum eigenvalue for di®erent smoothers . 103 4.4 Convergence of the GMRES/GMG solvers (Case study 4.2.1), with lexicographical ordering of the unknowns in the negative x-direction . 106 4.5 Convergence of the GMRES/GMG solvers (Case study 4.2.2), with di®erent orderings of the unknowns and P e¤ = 500 . 109 4.6 Convergence of the GMRES/GMG solvers (Case study 4.2.2), with di®erent orderings of the unknowns and P e¤ = 2000 . 112 4.7 Convergence of the GMRES/GMG solvers, on stretched grids . 116 4.8 Convergence of the GMRES/GMG solvers (Case study 4.2.3) . 119 4.9 Truncation statistics of the coe±cient matrix, with uniform grid re- ¯nement . 120 4.10 Comparison of truncation statistics for Case studies 4.2.1, 4.2.2, 4.2.3 and 4.2.4 with uniform grid re¯nement . 121 4.11 Convergence of the GMRES/GMG solvers (Case study 4.2.4) . 123 4.12 Convergence of the GMRES/GMG solvers, with adaptive grid re¯ne- ment.................................... 125 4.13 Truncation statistics of the coe±cient matrix, with adaptive grid re- ¯nement . 126 4.14 Comparison of truncation statistics for Case studies 4.2.1, 4.2.2, 4.2.3 and 4.2.4 with adaptive grid re¯nement . 127 6 4.15 Convergence of the GMRES/GMG solvers, with alternative damping parameters (Case study 4.2.2) . 128 4.16 Convergence of the GMRES/GMG solvers, with alternative damping parameters (Case study 4.2.4) . 129 4.17 Convergence of the GMRES/GMG solvers, with alternative MG V- cycle smoothing parameters (Case study 4.2.2) . 130 4.18 Convergence of the GMRES/GMG solvers, with alternative MG V- cycle smoothing parameters (Case study 4.2.4) . 131 4.19 Convergence of the GMRES/AMG solvers (Case study 4.2.1) . 133 4.20 Convergence of the GMRES/AMG solvers (Case study 4.2.2) . 134 4.21 Convergence of the GMRES/AMG solvers (Case study 4.2.4) . 135 4.22 Coarsening statistics within AMG . 136 4.23 Convergence of the GMRES/AMG solvers, with alternative damping parameters (Case study 4.2.2) . 137 4.24 Convergence of the GMRES/AMG solvers, with alternative damping parameters (Case study 4.2.4) . 138 4.25 Convergence of the GMRES/AMG solvers, with alternative MG V- cycle smoothing parameters (Case study 4.2.2) . 139 4.26 Convergence of the GMRES/AMG solvers, with alternative MG V- cycle smoothing parameters (Case study 4.2.4) . 140 4.27 Parallel convergence of the GMRES preconditioned classical AMG solvers (Case study 4.2.1) . 142 4.28 Parallel convergence of the GMRES preconditioned classical AMG solvers (Case study 4.2.2) . 144 4.29 Parallel convergence of the GMRES preconditioned AMG solvers using Falgout coarsening (Case study 4.2.2) . 144 4.30 Parallel convergence of the GMRES preconditioned AMG solvers (Case study 4.2.4) . 146 4.31 Convergence of the GMRES solver applied to the discrete Navier- Stokes problem (smoother damping parameter γ = 0:5) . 152 4.32 Convergence of the GMRES solver applied to the discrete Navier- Stokes problem (smoother damping parameter γ = 1:0) . 153 5.1 Convergence of the GMRES/GMG solvers (Case study 5.1.1) . 158 5.2 Convergence of the GMRES/GMG solvers (Case study 5.1.1), with lexicographical ordering of the unknowns in the negative x-direction . 159 5.3 Convergence of the GMRES/GMG solvers (Case study 5.1.2), with di®erent orderings of the unknowns and P e¤ = 500 . 162 7 5.4 Convergence of the GMRES/GMG solvers (Case study 5.1.2), with di®erent orderings of the unknowns and P e¤ = 2000 . 164 5.5 Truncation statistics of the coe±cient matrix, with uniform grid re- ¯nement . 165 5.6 Convergence of the GMRES/GMG solvers, with adaptive grid re¯ne- ment.................................... 167 5.7 Truncation statistics of the coe±cient matrix, with adaptive grid re- ¯nement . 168 5.8 Comparison of truncation statistics for Case studies 5.1.1 and 5.1.2 with adaptive grid re¯nement . 169 5.9 Convergence of the GMRES/AMG solvers (Case study 5.1.1) . 171 5.10 Convergence of the GMRES/AMG solvers (Case study 5.1.2) . 172 5.11 Coarsening statistics within AMG . 172 5.12 Truncation ratio of smoothers within AMG preconditioning . 174 5.13 Operator complexity of tILU0 ...................... 174 5.14 Average stencil size of tILU0 ....................... 174 5.15 Parallel convergence of the GMRES preconditioned classical AMG solvers (Case study 5.1.1) . 176 5.16 Parallel convergence of the GMRES preconditioned classical AMG solvers (Case study 5.1.2) . 177 A.1 Convergence of the GMRES/AMG solvers (Case study 4.2.3) . 184 8 List of Figures 2.1 Flow through a domain showing three di®erent types of boundaries . 21 2.2 Q1 mapping from the reference element . 27 2.3 FE solution obtained from a convection-di®usion equation that repre- sents the global e®ect of layers . 29 2.4 SUPG FE solution of a convection-di®usion equation with di®erent values of stabilisation parameter . 33 3.1 Convergence history of the damped Jacobi (γ = 0:5) and Gauss-Seidel methods, on a 1D problem . 68 3.2 Fourier vectors . 68 3.3 Convergence history of the damped Jacobi (γ = 0:5) and Gauss-Seidel methods applied to the reduction of di®erent Fourier modes . 69 3.4 (a) Di®erent types of MG cycles, (b) Grid representation of a MG hierarchy with uniform grid re¯nement . 72 3.5 Tree data structure associated with adaptive re¯nement . 74 4.1 A contour plot of the solution u of the di®usion problem . 91 4.2 The convection-di®usion problem (Case study 4.2.1), with constant, uni-directional wind . 96 4.3 OOMPHLIB global tree structure nodal ordering for uniformly re¯ned grids . 97 4.4 Eigenvalues of the matrix Eamp based on di®erent smoothers . 104 4.5 The convection-di®usion problem (Case study 4.2.2), with recirculating wind . 108 4.6 Discretisation of the domain with (a) uniform, (b) stretched grid . 115 4.7 The convection-di®usion problem (Case study 4.2.3), with uniform and recirculating wind . 118 4.8 The convection-di®usion problem (Case study 4.2.4), with multiple recirculating vortices .