Multiphysics Simulations: Challenges and Opportunities
Total Page:16
File Type:pdf, Size:1020Kb
ANL/MCS-TM-321 Rev. 1.1 Multiphysics Simulations: Challenges and Opportunities Rev. 1.1 Mathematics and Computer Division About Argonne National Laboratory Argonne is a U.S. Department of Energy laboratory managed by UChicago Argonne, LLC under contract DE-AC02-06CH11357. The Laboratory’s main facility is outside Chicago, at 9700 South Cass Avenue, Argonne, Illinois 60439. For information about Argonne and its pioneering science and technology programs, see www.anl.gov. Availability of This Report This report is available, at no cost, at http://www.osti.gov/bridge. It is also available on paper to the U.S. Department of Energy and its contractors, for a processing fee, from: U.S. Department of Energy Office of Scientific and Technical Information P.O. Box 62 Oak Ridge, TN 37831-0062 phone (865) 576-8401 fax (865) 576-5728 [email protected] Disclaimer This report was prepared as an account of work sponsored by an agency of the United States Government. Neither the United States Government nor any agency thereof, nor UChicago Argonne, LLC, nor any of their employees or officers, makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise, does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or any agency thereof. The views and opinions of document authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof, Argonne National Laboratory, or UChicago Argonne, LLC. Multiphysics Simulations: Challenges and Opportunities 1 ANL/MCS-TM 321, Revision 1.1, October 2012 To appear as a special issue of the International Journal of High Performance Computing Applications David E. Keyes, KAUST and Columbia University Lois Curfman McInnes, Argonne National Laboratory Carol Woodward, Lawrence Livermore National Laboratory William Gropp, University of Illinois at Urbana-Champaign Eric Myra, University of Michigan Michael Pernice, Idaho National Laboratory John Bell, Lawrence Berkeley National Laboratory Jed Brown, Argonne National Laboratory Alain Clo, KAUST Jeffrey Connors, Lawrence Livermore National Laboratory Emil Constantinescu, Argonne National Laboratory Don Estep, Colorado State University Kate Evans, Oak Ridge National Laboratory Charbel Farhat, Stanford University Ammar Hakim, Princeton Plasma Physics Laboratory Glenn Hammond, Pacific Northwest National Laboratory Glen Hansen, Sandia National Laboratories Judith Hill, Oak Ridge National Laboratory Tobin Isaac, University of Texas at Austin Xiangmin Jiao, Stonybrook University Kirk Jordan, IBM Research Center Dinesh Kaushik, Argonne National Laboratory Efthimios Kaxiras, Harvard University Alice Koniges, Lawrence Berkeley National Laboratory Kihwan Lee, SLAC National Accelerator Laboratory Aaron Lott, Lawrence Livermore National Laboratory Qiming Lu, Fermi National Accelerator Laboratory John Magerlein, IBM Research Center Reed Maxwell, Colorado School of Mines Michael McCourt, Cornell University Miriam Mehl, Technische Universitat¨ Munchen¨ Roger Pawlowski, Sandia National Laboratories Amanda Peters Randles, Harvard University Daniel Reynolds, Southern Methodist University Beatrice Riviere,` Rice University Ulrich Rude,¨ University Erlangen-Nuremberg Tim Scheibe, Pacific Northwest National Laboratory John Shadid, Sandia National Laboratories Brendan Sheehan, Colorado State University Mark Shephard, Rensselaer Polytechnic Institute Andrew Siegel, Argonne National Laboratory Barry Smith, Argonne National Laboratory Xianzhu Tang, Los Alamos National Laboratory Cian Wilson, Columbia University Barbara Wohlmuth, Technische Universitat¨ Munchen¨ 1Please cite this document as follows: Multiphysics Simulations: Challenges and Opportunities, David E. Keyes, Lois Curfman McInnes, Carol Woodward, William Gropp, Eric Myra, Michael Pernice, John Bell, Jed Brown, Alain Clo, Jeffrey Connors, Emil Constantinescu, Don Estep, Kate Evans, Charbel Farhat, Ammar Hakim, Glenn Hammond, Glen Hansen, Judith Hill, Tobin Isaac, Xiaomin Jiao, Kirk Jordan, Dinesh Kaushik, Efthimios Kaxiras, Alice Koniges, Kihwan Lee, Aaron Lott, Qiming Lu, John Magerlein, Reed Maxwell, Michael McCourt, Miriam Mehl, Roger Pawlowski, Amanda Peters Randles, Daniel Reynolds, Beatrice Riviere,` Ulrich Rude,¨ Tim Scheibe, John Shadid, Brendan Sheehan, Mark Shephard, Andrew Siegel, Barry Smith, Xianzhu Tang, Cian Wilson, and Barbara Wohlmuth, Tech. Rep. ANL/MCS-TM-321, Revision 1.1, October 2012, Argonne National Laboratory. To appear as a special issue of the International Journal of High Performance Computing Applications. Abstract We consider multiphysics applications from algorithmic and architectural perspectives, where “algorithmic” includes both mathematical analysis and computational complexity and “architectural” includes both software and hardware environments. Many diverse multiphysics applications can be reduced, en route to their computational simulation, to a common algebraic coupling paradigm. Mathematical analysis of multiphysics coupling in this form is not always practical for realistic applications, but model problems representative of applications discussed herein can provide insight. A variety of software frameworks for multiphysics applications have been constructed and refined within disciplinary communities and executed on leading-edge computer systems. We examine several of these, expose some commonalities among them, and attempt to extrapolate best practices to future systems. From our study, we summarize challenges and forecast opportunities. Keywords: multiphysics, multimodel, multirate, multiscale, implicit and explicit algorithms, strong and weak coupling, loose and tight coupling Contents 1 Introduction 1 1.1 What Constitutes Multiphysics?...................................1 1.2 Prototype Algebraic Forms......................................3 1.3 Structure and Motivation for This Review..............................5 2 Practices and Perils in Multiphysics Applications7 2.1 Examples of PDE-Based Multiphysics Applications.........................7 2.1.1 Interaction of fluids and structures..............................8 2.1.2 Fission reactor fuel performance...............................9 2.1.3 Conjugate heat transfer and neutron transport coupling in reactor cores.......... 11 2.1.4 Multiscale methods in crack propagation.......................... 13 2.1.5 Multiscale methods in ultrafast DNA sequencing...................... 14 2.1.6 Magnetic confinement fusion................................. 15 2.1.7 Subsurface science...................................... 17 2.1.8 Surface and subsurface hydrology.............................. 18 2.1.9 Climate modeling....................................... 19 2.1.10 Radiation hydrodynamics.................................. 21 2.1.11 Geodynamics......................................... 22 2.1.12 Particle accelerator design.................................. 23 2.2 Crosscutting Issues in Multiphysics Applications.......................... 25 2.2.1 Choices and challenges in coupling algorithms....................... 25 2.2.2 Software engineering..................................... 26 2.2.3 Analysis and verification................................... 27 3 Algorithms for Multiphysics Coupling 28 3.1 Solver Methods............................................ 28 3.1.1 Methods for systems of linear equations........................... 28 3.1.2 Methods for systems of nonlinear equations......................... 31 3.2 Continuum-Continuum Coupling................................... 33 3.2.1 Methods for coupling multiphysics components in space.................. 34 3.2.2 Methods for coupling multiphysics components in time................... 37 3.3 Continuum-Discrete Coupling.................................... 41 3.4 Error Estimation............................................ 43 3.4.1 Operator splitting for reaction-diffusion equations...................... 44 3.4.2 Operator splitting for advection-diffusion equations..................... 46 3.4.3 Iterative solution of parabolic problems coupled through a common boundary....... 46 3.4.4 Solution of systems of elliptic equations with independent discretizations......... 48 3.5 Uncertainty Quantification...................................... 49 4 Multiphysics Software 51 4.1 Status of Software for Multiphysics................................. 51 4.1.1 Current practices....................................... 51 4.1.2 Common needs........................................ 52 4.1.3 Software successes...................................... 53 4.2 Challenges in Multiphysics Software Design............................. 57 4.2.1 Enabling introduction of new models, algorithms, and data structures........... 57 4.2.2 Sharing methods and codes among application fields.................... 59 4.2.3 Relating multiphysics spatial discretizations......................... 60 4.2.4 Timestep control....................................... 62 4.2.5 Achieving performance.................................... 62 4.2.6 Software engineering issues for multiphysics integration.................. 62 4.3 Difficulties in Collaborative Multiphysics Software......................... 63 4.3.1 Maintenance of a stable