Multigrid Equation Solvers for Large Scale Nonlinear Finite Element Simulations Mark Francis Adams Rep ort No UCBCSD January Computer Science Division EECS University of California Berkeley California Multigrid Equation Solvers for Large Scale Nonlinear Finite Element Simulations by Mark Francis Adams BA University of California Berkeley A dissertation submitted in partial satisfaction of the requirements for the degree of Do ctor of Philosophy in Engineering Civil Engineering in the GRADUATE DIVISION of the UNIVERSITY of CALIFORNIA BERKELEY Committee in charge Professor Rob ert L Taylor Cochair Professor James W Demmel Cochair Professor Gregory L Fenves Professor Katherine Yelick The dissertation of Mark Francis Adams is approved Cochair Date Cochair Date Date Date University of California Berkeley Multigrid Equation Solvers for Large Scale Nonlinear Finite Element Simulations Copyright by Mark Francis Adams Abstract Multigrid Equation Solvers for Large Scale Nonlinear Finite Element Simulations by Mark Francis Adams Do ctor of Philosophy in Engineering Civil Engineering University of California Berkeley Professor Rob ert L Taylor Cochair Professor James W Demmel Cochair The nite element metho d has grown in the past years to b e a p opular metho d for the simulation of physical systems in science and engineering The nite element metho d is used in a wide array of industries In fact just ab out any enterprise that makes a physical pro duct can and probably do es use nite element technology The success of the nite element metho d is due in large part to its ability to allow the use of accurate formulation of partial dierential equations PDEs on arbitrarily general physical domains with complex b oundary conditions Additionally the rapid growth in the computational p ower available in to days computers for an ever more aordable price has made nite element technology more accessible to a wider variety of industries and academic disciplin es As computational resources allow p eople to pro duce ever more accurate simulation of their systems allowing for the more ecient design and safety testing of everything from automobiles to nuclear weapons to articial knee joints all asp ects of the nite element simulation pro cess are stressed The largest b ottleneck in the growth in the scale of nite element applications is the linear equation solver used in implicit time integration schemes This is due to the fact that the direct solution metho ds p opular in the nite element community as they are ecient easy to use and relatively unaected by the underlying PDE and discretization do not scale well with increasing problem size The scale of problems that are now b ecoming feasible demand that iterative meth o ds b e used The p erformance issues of iterative metho ds is very dierent from those of direct metho ds as their p erformance is highly sensitive to the underlying PDE and dis cretization the construction of robust iterative metho ds for nite element matrices is a hard problem which is currently a very active area of research We discuss the iterative metho ds commonly used to day and show that they can all b e characterized as metho ds that solve problems eciently by pro jecting the solution to a series of subspaces The goal of iterative metho d design and indeed of nite element metho d design is to select a series of subspaces that solves problems optimally solvers try to minimize solution costs and nite element formulations try to optimize accuracy of the solution The subspaces used in multigrid metho ds are highly eective in minimizing solution costs particularly on large problems Multigrid is known to b e the most eective solution metho d for some discretized PDEs however its eective use on unstructured nite element meshes is an op en problem and constitutes the theme of this study The main contribution of this dissertation is the algorithmic development and exp erimental analysis of robust and scalable techniques for the solution of the sparse ill conditioned matrices that arise from nite element simulation in D continuum mechanics We show that our multigrid formulations are eective in the linear solution of systems with large jumps in material co ecients for problems with realistic mesh conguration and ge ometries including p o orly prop ortioned elements and for problems with p o or geometric conditioning as is commonplace in structural engineering We show that the these meth o ds can b e used eectively within nonlinear simulations via Newtons metho d We solve problems with more than sixteen million degrees of freedom and parallel solver eciency of ab out on pro cessors of a Cray TE We also show that our metho ds can b e adapted and extended to the indenite matrices that arise in the simulation of problems with constraints namely contact problems formulated with Lagrange multiplier Professor Rob ert L Taylor Dissertation Committee Cochair Professor James W Demmel Dissertation Committee Cochair iii To my b est friend Nighat for constant supp ort for many years b efore my graduate work and throughout the many years of this work iv Contents List of Figures viii List of Tables xi Dissertation summary Introduction The nite element metho d Motivation Goals Dissertation outline Contributions Notation Mathematical preliminaries Introduction The nite element metho d Finite element example Linear isotropic heat equation Iterative equation solver basics Matrix splitting metho ds Krylov subspace metho ds Preconditioned Krylov subspace metho ds Krylov subspace metho ds as pro jections One level domain decomp osition Alternating Schwartz metho d Multiplicative and additive Schwarz Multilevel domain decomp osition Introduction A Simple two level metho d Multigrid Convergence of multigrid Convergence analysis of domain decomp osition Variational formulation Domain decomp osition comp onents v A convergence theory High p erformance linear equation solvers for nite element matrices Introduction Algebraic multigrid A promising algebraic metho d Geometric approach on unstructured meshes Promising geometric approaches Domain decomp osition A domain decomp osition metho d Our metho d Introduction A parallel maximal indep endent set algorithm An asynchronous distributed memory algorithm Shared memory algorithm Distributed memory algorithm Complexity of the asynchronous maximal indep endent set algorithm Numerical results Maximal indep endent set heuristics Automatic coarse grid creation with unstructured meshes Topological classication of vertices in nite element meshes A simple face identication algorithm Mo died maximal indep endent set algorithm Vertex ordering in MIS algorithm on mo died nite element graphs Coarse grid cover of ne grid Numerical results Mesh generation Finite element shap e functions Galerkin construction of coarse grid op erators Smo others Multigrid characteristics on linear problems in solid mechanics Introduction Multigrid works Large jumps in material co ecients soft section cantilever b eam Large jumps in material co ecients curved material interface Incompressible materials Poorly prop ortioned elements Conclusion Parallel architecture and algorithmic issues Introduction Parallel nite element co de structure Finite element co de structure vi Finite element parallelism Parallelism and graph partitioning Parallel computer architecture Multiple levels of partitioning Solver complexity issues A parallel nite element architecture Athena Epimetheus Prometheus AthenaEpimetheusPrometheus construction details Pro cessor sub domain agglomeration Simple sub domain agglomeration metho d Sub domain agglomeration as an