Numerical Methods for Partial Differential Equations Seongjai Kim Department of Mathematics and Statistics Mississippi State University Mississippi State, MS 39762 USA Email: [email protected] August 12, 2021 Seongjai Kim, Department of Mathematics and Statistics, Mississippi State University, Mississippi State, MS 39762-5921 USA Email: [email protected]. The work of the author is supported in part by NSF grant DMS-1228337. Prologue In the area of “Numerical Methods for Differential Equations", it seems very hard to find a textbook incorporating mathematical, physical, and engineer- ing issues of numerical methods in a synergistic fashion. So the first goal of this lecture note is to provide students a convenient textbook that addresses both physical and mathematical aspects of numerical methods for partial dif- ferential equations (PDEs). In solving PDEs numerically, the following are essential to consider: • physical laws governing the differential equations (physical understand- ing), • stability/accuracy analysis of numerical methods (mathematical under- standing), • issues/difficulties in realistic applications, and • implementation techniques (efficiency of human efforts). In organizing the lecture note, I am indebted by Ferziger and Peric [23], John- son [32], Strikwerda [64], and Varga [68], among others. Currently the lecture note is not fully grown up; other useful techniques would be soon incorporated. Any questions, suggestions, comments will be deeply appreciated. i ii Contents 1 Mathematical Preliminaries1 1.1. Taylor’s Theorem & Polynomial Fitting..............2 1.2. Finite Differences...........................8 1.2.1. Uniformly spaced grids....................8 1.2.2. General grids......................... 10 1.3. Overview of PDEs........................... 16 1.4. Difference Equations......................... 24 1.5. Homework............................... 29 2 Numerical Methods for ODEs 31 2.1. Taylor-Series Methods........................ 33 2.1.1. The Euler method....................... 34 2.1.2. Higher-order Taylor methods................ 37 2.2. Runge-Kutta Methods........................ 40 2.2.1. Second-order Runge-Kutta method............. 41 2.2.2. Fourth-order Runge-Kutta method............. 44 2.2.3. Adaptive methods....................... 46 2.3. Accuracy Comparison for One-Step Methods........... 47 2.4. Multi-step Methods.......................... 50 2.5. High-Order Equations & Systems of Differential Equations.. 52 2.6. Homework............................... 53 3 Properties of Numerical Methods 55 3.1. A Model Problem: Heat Conduction in 1D............. 56 3.2. Consistency.............................. 60 iii iv Contents 3.3. Convergence.............................. 63 3.4. Stability................................ 69 3.4.1. Approaches for proving stability.............. 70 3.4.2. The von Neumann analysis................. 72 3.4.3. Influence of lower-order terms................ 76 3.5. Boundedness – Maximum Principle................ 77 3.5.1. Convection-dominated fluid flows.............. 78 3.5.2. Stability vs. boundedness.................. 79 3.6. Conservation.............................. 80 3.7. A Central-Time Scheme....................... 81 3.8. The θ-Method............................. 82 3.8.1. Stability analysis for the θ-Method............. 84 3.8.2. Accuracy order......................... 85 3.8.3. Maximum principle...................... 87 3.8.4. Error analysis......................... 89 3.9. Homework............................... 90 4 Finite Difference Methods for Elliptic Equations 91 4.1. Finite Difference (FD) Methods................... 92 4.1.1. Constant-coefficient problems................ 93 4.1.2. General diffusion coefficients................ 96 4.1.3. FD schemes for mixed derivatives............. 98 4.1.4. L1-norm error estimates for FD schemes......... 98 4.1.5. The Algebraic System for FDM............... 105 4.2. Solution of Linear Algebraic Systems................ 109 4.2.1. Direct method: the LU factorization............ 110 4.2.2. Linear iterative methods................... 115 4.2.3. Convergence theory...................... 116 4.2.4. Relaxation methods...................... 122 4.2.5. Line relaxation methods................... 129 4.3. Krylov Subspace Methods...................... 132 4.3.1. Steepest descent method................... 133 4.3.2. Conjugate gradient (CG) method.............. 135 Contents v 4.3.3. Preconditioned CG method................. 138 4.4. Other Iterative Methods....................... 140 4.4.1. Incomplete LU-factorization................. 140 4.5. Numerical Examples with Python................. 144 4.6. Homework............................... 150 5 Finite Element Methods for Elliptic Equations 153 5.1. Finite Element (FE) Methods in 1D Space............. 154 5.1.1. Variational formulation................... 154 5.1.2. Formulation of FEMs..................... 159 5.2. The Hilbert spaces.......................... 172 5.3. An error estimate for FEM in 1D.................. 174 5.4. Other Variational Principles..................... 179 5.5. FEM for the Poisson equation.................... 180 5.5.1. Integration by parts...................... 180 5.5.2. Defining FEMs........................ 183 5.5.3. Assembly: Element stiffness matrices........... 189 5.5.4. Extension to Neumann boundary conditions....... 191 5.6. Finite Volume (FV) Method..................... 193 5.7. Average of The Diffusion Coefficient................ 198 5.8. Abstract Variational Problem.................... 200 5.9. Numerical Examples with Python................. 203 5.10.Homework............................... 206 6 FD Methods for Hyperbolic Equations 209 6.1. Introduction.............................. 210 6.2. Basic Difference Schemes...................... 213 6.2.1. Consistency.......................... 215 6.2.2. Convergence.......................... 217 6.2.3. Stability............................ 220 6.2.4. Accuracy............................ 225 6.3. Conservation Laws.......................... 228 6.3.1. Euler equations of gas dynamics.............. 228 vi Contents 6.4. Shocks and Rarefaction........................ 235 6.4.1. Characteristics........................ 235 6.4.2. Weak solutions........................ 237 6.5. Numerical Methods.......................... 239 6.5.1. Modified equations...................... 239 6.5.2. Conservative methods.................... 246 6.5.3. Consistency.......................... 250 6.5.4. Godunov’s method....................... 251 6.6. Nonlinear Stability.......................... 252 6.6.1. Total variation stability (TV-stability)........... 253 6.6.2. Total variation diminishing (TVD) methods........ 255 6.6.3. Other nonoscillatory methods................ 256 6.7. Numerical Examples with Python................. 261 6.8. Homework............................... 263 7 Domain Decomposition Methods 265 7.1. Introduction to DDMs......................... 266 7.2. Overlapping Schwarz Alternating Methods (SAMs)....... 269 7.2.1. Variational formulation................... 269 7.2.2. SAM with two subdomains.................. 270 7.2.3. Convergence analysis..................... 271 7.2.4. Coarse subspace correction................. 274 7.3. Nonoverlapping DDMs........................ 277 7.3.1. Multi-domain formulation.................. 277 7.3.2. The Steklov-Poincaré operator............... 279 7.3.3. The Schur complement matrix............... 281 7.4. Iterative DDMs Based on Transmission Conditions....... 284 7.4.1. The Dirichlet-Neumann method.............. 284 7.4.2. The Neumann-Neumann method.............. 286 7.4.3. The Robin method....................... 287 7.4.4. Remarks on DDMs of transmission conditions...... 288 7.5. Homework............................... 294 Contents vii 8 Multigrid Methods∗ 297 8.1. Introduction to Multigrid Methods................. 298 8.2. Homework............................... 299 9 Locally One-Dimensional Methods 301 9.1. Heat Conduction in 1D Space: Revisited.............. 302 9.2. Heat Equation in Two and Three Variables............ 308 9.2.1. The θ-method......................... 309 9.2.2. Convergence analysis for θ-method............. 311 9.3. LOD Methods for the Heat Equation................ 314 9.3.1. The ADI method........................ 315 9.3.2. Accuracy of the ADI: Two examples............ 321 9.3.3. The general fractional step (FS) procedure........ 324 9.3.4. Improved accuracy for LOD procedures.......... 326 9.3.5. A convergence proof for the ADI-II............. 333 9.3.6. Accuracy and efficiency of ADI-II.............. 335 9.4. Homework............................... 337 10 Special Schemes 339 10.1.Wave Propagation and Absorbing Boundary Conditions..... 340 10.1.1. Introduction to wave equations............... 340 10.1.2. Absorbing boundary conditions (ABCs).......... 341 10.1.3. Waveform ABC........................ 342 11 Projects∗ 349 11.1.High-order FEMs for PDEs of One Spacial Variable....... 349 A Basic Concepts in Fluid Dynamics 351 A.1. Conservation Principles....................... 351 A.2. Conservation of Mass......................... 353 A.3. Conservation of Momentum..................... 353 A.4. Non-dimensionalization of the Navier-Stokes Equations.... 356 A.5. Generic Transport Equations.................... 358 A.6. Homework............................... 359 viii Contents B Elliptic Partial Differential Equations 361 B.1. Regularity Estimates......................... 361 B.2. Maximum and Minimum Principles................ 363 B.3. Discrete Maximum and Minimum Principles..........