DOCSLIB.ORG

Exploring Capabilities Within Fortrilinos by Solving the 3D Burgers Equation

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Exploring Capabilities Within Fortrilinos by Solving the 3D Burgers Equation Scientific Programming 20 (2012) 275–292 275 DOI 10.3233/SPR-2012-0353 IOS Press Exploring capabilities within ForTrilinos by solving the 3D Burgers equation Karla Morris a,∗, Damian W.I. Rouson a, M. Nicole Lemaster a and Salvatore Filippone b a Sandia National Laboratories, Livermore, CA, USA b Università di Roma “Tor Vergata”, Roma, Italy Abstract. We present the first three-dimensional, partial differential equation solver to be built atop the recently released, open-source ForTrilinos package (http://trilinos.sandia.gov/packages/fortrilinos). ForTrilinos currently provides portable, object- oriented Fortran 2003 interfaces to the C++ packages Epetra, AztecOO and Pliris in the Trilinos library and framework [ACM Trans. Math. Softw. 31(3) (2005), 397–423]. Epetra provides distributed matrix and vector storage and basic linear algebra cal- culations. Pliris provides direct solvers for dense linear systems. AztecOO provides iterative sparse linear solvers. We demon- strate how to build a parallel application that encapsulates the Message Passing Interface (MPI) without requiring the user to make direct calls to MPI except for startup and shutdown. The presented example demonstrates the level of effort required to set up a high-order, finite-difference solution on a Cartesian grid. The example employs an abstract data type (ADT) calculus [Sci. Program. 16(4) (2008), 329–339] that empowers programmers to write serial code that lower-level abstractions resolve into distributed-memory, parallel implementations. The ADT calculus uses compilable Fortran constructs that resemble the mathe- matical formulation of the partial differential equation of interest. Keywords: ForTrilinos, Trilinos, Fortran 2003/2008, object oriented programming 1. Introduction Burgers [4]. As one of the few nonlinear PDEs with known analytical solutions, the Burgers equation (in The story of modern, scientific programming is the both its 1D and 3D forms) often plays a role in theo- story of modern, mainstream programming: perfor- retical investigations as a proxy for the more compli- mance gains derive primarily from increasing levels cated 3D Navier–Stokes equations (NSE). The Burg- of parallel execution. Likewise, the story of modern, ers equation retains the unsteady, advective, and dif- scientific programming languages resembles the glob- fusive nature of the NSE without the many additional alization of modern society: mixed-language environ- complications associated with pressure gradients. The ments increasingly predominate. The ForTrilinos soft- Burgers equation also provides qualitative insights into phenomena ranging from charge density waves, vor- ware package sits at the intersection of these two trends tex lines in high-temperature superconductors, and the and was first released in the August 2010 Trilinos large-scale structure of the universe. See [5] and [6] for open-source library and framework (release 10.4.1). surveys of applications for the Burgers equation and ForTrilinos brings to the Fortran community the see [2] and [3] for exact analytical solutions of the 1D object-oriented (OO), native interfaces to C++ pack- and 3D Burgers equation, respectively. ages in the Trilinos library and framework. The goal of the current effort was to investigate the The current article presents the first attempt to level of programming effort required to build a repre- construct a three-dimensional (3D) partial differen- sentative scientific software application atop ForTrili- tial equation (PDE) solver based on ForTrilinos. We nos and provide a detailed guide for future users on present a case study that involves solving a 3D gener- ways to exploit the available capabilities. Our study alization of the classical one-dimensional (1D) PDE of employs one of the compact finite difference schemes developed by Lele [9]. In these schemes, the finite- *Corresponding author: Karla V. Morris, PhD, Combustion Re- search Facility, Sandia National Laboratories, 7011 East Avenue, difference approximations to the nodal derivatives re- MS 9055, Livermore, CA 94550, USA. Tel.: +1 925 294 3287; sult from inverting a linear system of equations with E-mail: [email protected]. the nodal function values as the right-hand side (RHS). 1058-9244/12/$27.50 © 2012 – IOS Press and the authors. All rights reserved 276 K. Morris et al. / Exploring capabilities within ForTrilinos by solving the 3D Burgers equation These schemes’ compact stencil and high-order accu- mining the Burgers equation for insights into fluid be- racy have made them popular for computing complex havior. fluid flows in geometries amenable to finite difference From a software demonstration standpoint, the Burg- approximations [12]. In the current context, the solu- ers equation offers two primary benefits. First, it mod- tion of the linear systems required by compact finite els some of the basic numerical properties of many of difference schemes provides a useful showcase for the the more complicated governing equations of interest linear solver interfaces ForTrilinos provides. to engineering and science. Second, the equation itself In the interest of brevity and to avoid repetition, (most often in the 1D form) has been used to model the remainder of this article assumes familiarity with phenomena such as gas dynamics, traffic flow, flood Fortran 2003. We refer readers who are familiar with waves in rivers, chemical reactions, shock and sound Fortran 90/95 but unfamiliar with OO programming waves and many others. (OOP) in Fortran 2003 to the scientific software de- We solve Eq. (1) in a 3D cube with periodic bound- sign text by Rouson et al. [16]. We refer readers who ary conditions in each direction, approximating all spa- are unfamiliar with Fortran 90/95 – including the ra- tial derivatives with a sixth-order-accurate finite differ- tionale for using modules and derived types – to the ence scheme from [9]. The chosen family of finite dif- modern Fortran text by Metcalf et al. [10]. Section 2 of ference schemes have the general form the current article presents the mathematical methodol- ogy employed in the current demonstration application and the software design methodology employed in For- Af = Bf,(2) Trilinos. Section 3 presents the resulting demonstration Cf = Df,(3) application and the recommended approach for access- ing the requisite ForTrilinos capabilities. Section 4 dis- where f, f and f are nodal solution values and first cusses the results. Section 5 concludes and presents the and second derivatives, respectively, and where A, B, path forward. C and D are all sparse coefficient matrices. The Ap- pendix provides the stencils that generate the coeffi- 2. Methodology cient matrices. The tridiagonal matrix structure of the corresponding 1D problem generates a band-diagonal 2.1. Mathematical model system in 3D. We employ a second-order Runge–Kutta algorithm The 3D Burgers equation describes the nonlinear ad- to advance the governing equation in time. The tempo- vection and diffusion of a vector quantity: rally discrete form of Eq. (1) is thus ∂u 2 Δ = −u ·∇u + ν∇ u,(1)n+1/2 n n n t ∂t u = u + [N(u ) + L(u )] ,(4) 2 where u is a vector field with components u, v and un+1 = un + [N(un+1/2) + L(un+1/2)]Δt,(5) w and ν is a constant parameter. The analogy to the 3D NSE lies in viewing u as a fluid velocity vec- where L is a discrete linear operator that approximates tor field with ν playing the role of the fluid’s kine- the diffusive term in (1) and N is the discrete nonlin- matic viscosity. There are some limits to this view- ear operator that approximates the advective term. We point: arriving at a NSE viscous term of the same form calculate the time step Δt based on the stability re- as the corresponding term in Eq. (1) requires apply- ing an incompressibility constraint to the velocity field quirements of the linear term, which is more restrictive and enforcing that constraint via a pressure gradient asymptotically than the stability restriction on the non- term, but Eq. (1) contains no pressure gradient and its linear term (the Courant condition) in the context of an solution therefore does not satisfy incompressibility. explicit time advancement algorithm. Due to similar reservations, [17], for example, refer to the medium in their Burgers-equation study of turbu- 2.2. Software methodology lence in a self-gravitating medium as a “sticky dust” rather than as “viscous matter”. Nonetheless, the Burg- The solver presented here demonstrates several For- ers equation’s formal similarity to the incompressible Trilinos capabilities and, in doing so, showcases sev- NSE and the existence of exact solutions to the Burg- eral object-oriented features of Fortran 2003. The cho- ers equation have inspired considerable activity around sen example is intended to inform users interested in K. Morris et al. / Exploring capabilities within ForTrilinos by solving the 3D Burgers equation 277 creating and operating on ForTrilinos objects within a The Epetra package hosts classes that support paral- fully objected-oriented application. The software de- lel and serial basic linear algebra functionality. Epe- sign philosophy employed herein closely resembles tra classes provide the foundation for all Trilinos pack- that of the ForTrilinos package. ages. AztecOO is the OO interface to the Aztec pack- A ForTrilinos-enabled application leverages a deep age, providing massively scalable iterative solvers for software stack. This stack resolves issues related to sparse linear systems. AztecOO accepts external pre- language interoperability, portability and memory conditioners. The Pliris package supports a parallel, management in ways that remove such concerns from OO interface to the solution of dense linear systems, the end user. The top layer corresponds to user code employing LU factorization on double precision data. of the kind presented in the current paper. The bottom Additionally, ForTrilinos contains the procedural in- layer comprises the Trilinos package of interest. Typ- frastructure for accessing Amesos, Galeri and Ifpack ically the Trilinos layer does all of the heavy lifting from Fortran.
Load more

Recommended publications
  • Accelerating the LOBPCG Method on Gpus Using a Blocked Sparse Matrix Vector Product
    Accelerating the LOBPCG method on GPUs using a blocked Sparse Matrix Vector Product Hartwig Anzt and Stanimire Tomov and Jack Dongarra Innovative Computing Lab University of Tennessee Knoxville, USA Email: [email protected], [email protected], [email protected] Abstract— the computing power of today’s supercomputers, often accel- erated by coprocessors like graphics processing units (GPUs), This paper presents a heterogeneous CPU-GPU algorithm design and optimized implementation for an entire sparse iter- becomes challenging. ative eigensolver – the Locally Optimal Block Preconditioned Conjugate Gradient (LOBPCG) – starting from low-level GPU While there exist numerous efforts to adapt iterative lin- data structures and kernels to the higher-level algorithmic choices ear solvers like Krylov subspace methods to coprocessor and overall heterogeneous design. Most notably, the eigensolver technology, sparse eigensolvers have so far remained out- leverages the high-performance of a new GPU kernel developed side the main focus. A possible explanation is that many for the simultaneous multiplication of a sparse matrix and a of those combine sparse and dense linear algebra routines, set of vectors (SpMM). This is a building block that serves which makes porting them to accelerators more difficult. Aside as a backbone for not only block-Krylov, but also for other from the power method, algorithms based on the Krylov methods relying on blocking for acceleration in general. The subspace idea are among the most commonly used general heterogeneous LOBPCG developed here reveals the potential of eigensolvers [1]. When targeting symmetric positive definite this type of eigensolver by highly optimizing all of its components, eigenvalue problems, the recently developed Locally Optimal and can be viewed as a benchmark for other SpMM-dependent applications.
    [Show full text]
  • Present and Future Leadership Computers at OLCF
    Present and Future Leadership Computers at OLCF Buddy Bland OLCF Project Director Presented at: SC’14 November 17-21, 2014 New Orleans ORNL is managed by UT-Battelle for the US Department of Energy Oak Ridge Leadership Computing Facility (OLCF) Mission: Deploy and operate the computational resources required to tackle global challenges Providing world-leading computational and data resources and specialized services for the most computationally intensive problems Providing stable hardware/software path of increasing scale to maximize productive applications development Providing the resources to investigate otherwise inaccessible systems at every scale: from galaxy formation to supernovae to earth systems to automobiles to nanomaterials With our partners, deliver transforming discoveries in materials, biology, climate, energy technologies, and basic science SC’14 Summit - Bland 2 Our Science requires that we continue to advance OLCF’s computational capability over the next decade on the roadmap to Exascale. Since clock-rate scaling ended in 2003, HPC Titan and beyond deliver hierarchical parallelism with performance has been achieved through increased very powerful nodes. MPI plus thread level parallelism parallelism. Jaguar scaled to 300,000 cores. through OpenACC or OpenMP plus vectors OLCF5: 5-10x Summit Summit: 5-10x Titan ~20 MW Titan: 27 PF Hybrid GPU/CPU Jaguar: 2.3 PF Hybrid GPU/CPU 10 MW Multi-core CPU 9 MW CORAL System 7 MW 2010 2012 2017 2022 3 SC’14 Summit - Bland Today’s Leadership System - Titan Hybrid CPU/GPU architecture, Hierarchical Parallelism Vendors: Cray™ / NVIDIA™ • 27 PF peak • 18,688 Compute nodes, each with – 1.45 TF peak – NVIDIA Kepler™ GPU - 1,311 GF • 6 GB GDDR5 memory – AMD Opteron™- 141 GF • 32 GB DDR3 memory – PCIe2 link between GPU and CPU • Cray Gemini 3-D Torus Interconnect • 32 PB / 1 TB/s Lustre® file system 4 SC’14 Summit - Bland Scientific Progress at all Scales Fusion Energy Liquid Crystal Film Stability A Princeton Plasma Physics Laboratory ORNL Postdoctoral fellow Trung Nguyen team led by C.S.
    [Show full text]
  • The Latest in Tpetra: Trilinos' Parallel Sparse Linear Algebra
    Photos placed in horizontal position with even amount of white space between photos The latest in Tpetra: Trilinos’ and header parallel sparse linear algebra Photos placed in horizontal position with even amount of white space Mark Hoemmen between photos and header Sandia National Laboratories 23 Apr 2019 SAND2019-4556 C (UUR) Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International, Inc. for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525. Tpetra: parallel sparse linear algebra § “Parallel”: MPI + Kokkos § Tpetra implements § Sparse graphs & matrices, & dense vecs § Parallel kernels for solving Ax=b & Ax=λx § MPI communication & (re)distribution § Tpetra supports many linear solvers § Key Tpetra features § Can manage > 2 billion (10^9) unknowns § Can pick the type of values: § Real, complex, extra precision § Automatic differentiation § Types for stochastic PDE discretizations 2 Over 15 years of Tpetra 2004-5: Stage 1 rewrite of Tpetra to use Paul 2005-10: Mid 2010: Assemble Deprecate & Sexton ill-tested I start Kokkos 2.0 new Tpetra purge for starts research- work on Kokkos team; gather Trilinos 13 Tpetra ware Tpetra 2.0 requirements 2005 2007 2009 2011 2013 2015 2017 2019 Trilinos 10.0 Trilinos 11.0 Trilinos 12.0 Trilinos 13.0 ??? Trilinos 9.0 2006 2008 2010 2012 2014 2016 2018 2020 2008: Chris Baker Fix bugs; rewrite Stage 2: purge Improve GPU+MPI
    [Show full text]
  • Warthog: a MOOSE-Based Application for the Direct Code Coupling of BISON and PROTEUS (MS-15OR04010310)
    ORNL/TM-2015/532 Warthog: A MOOSE-Based Application for the Direct Code Coupling of BISON and PROTEUS (MS-15OR04010310) Alexander J. McCaskey Approved for public release. Stuart Slattery Distribution is unlimited. Jay Jay Billings September 2015 DOCUMENT AVAILABILITY Reports produced after January 1, 1996, are generally available free via US Department of Energy (DOE) SciTech Connect. Website: http://www.osti.gov/scitech/ Reports produced before January 1, 1996, may be purchased by members of the public from the following source: National Technical Information Service 5285 Port Royal Road Springfield, VA 22161 Telephone: 703-605-6000 (1-800-553-6847) TDD: 703-487-4639 Fax: 703-605-6900 E-mail: [email protected] Website: http://www.ntis.gov/help/ordermethods.aspx Reports are available to DOE employees, DOE contractors, Energy Technology Data Ex- change representatives, and International Nuclear Information System representatives from the following source: Office of Scientific and Technical Information PO Box 62 Oak Ridge, TN 37831 Telephone: 865-576-8401 Fax: 865-576-5728 E-mail: [email protected] Website: http://www.osti.gov/contact.html This report was prepared as an account of work sponsored by an agency of the United States Government. Neither the United States Government nor any agency thereof, nor any of their employees, makes any warranty, express or implied, or assumes any legal lia- bility or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or rep- resents that its use would not infringe privately owned rights. Refer- ence herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise, does not nec- essarily constitute or imply its endorsement, recommendation, or fa- voring by the United States Government or any agency thereof.
    [Show full text]
  • Geeng Started with Trilinos
    Geng Started with Trilinos Michael A. Heroux Photos placed in horizontal position Sandia National with even amount of white space Laboratories between photos and header USA Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL85000. SAND NO. 2011-XXXXP Outline - Some (Very) Basics - Overview of Trilinos: What it can do for you. - Trilinos Software Organization. - Overview of Packages. - Package Management. - Documentation, Building, Using Trilinos. 1 Online Resource § Trilinos Website: https://trilinos.org § Portal to Trilinos resources. § Documentation. § Mail lists. § Downloads. § WebTrilinos § Build & run Trilinos examples in your web browser! § Need username & password (will give these out later) § https://code.google.com/p/trilinos/wiki/TrilinosHandsOnTutorial § Example codes: https://code.google.com/p/trilinos/w/list 2 WHY USE MATHEMATICAL LIBRARIES? 3 § A farmer had chickens and pigs. There was a total of 60 heads and 200 feet. How many chickens and how many pigs did the farmer have? § Let x be the number of chickens, y be the number of pigs. § Then: x + y = 60 2x + 4y = 200 § From first equaon x = 60 – y, so replace x in second equaon: 2(60 – y) + 4y = 200 § Solve for y: 120 – 2y + 4y = 200 2y = 80 y = 40 § Solve for x: x = 60 – 40 = 20. § The farmer has 20 chickens and 40 pigs. 4 § A restaurant owner purchased one box of frozen chicken and another box of frozen pork for $60. Later the owner purchased 2 boxes of chicken and 4 boxes of pork for $200.
    [Show full text]
  • Block Locally Optimal Preconditioned Eigenvalue Xolvers BLOPEX
    Block Locally Optimal Preconditioned Eigenvalue Xolvers I.Lashuk, E.Ovtchinnikov, M.Argentati, A.Knyazev Block Locally Optimal Preconditioned Eigenvalue Xolvers BLOPEX Ilya Lashuk, Merico Argentati, Evgenii Ovtchinnikov, Andrew Knyazev (speaker) Department of Mathematical Sciences and Center for Computational Mathematics University of Colorado at Denver and Health Sciences Center Supported by the Lawrence Livermore National Laboratory and the National Science Foundation Center for Computational Mathematics, University of Colorado at Denver Block Locally Optimal Preconditioned Eigenvalue Xolvers I.Lashuk, E.Ovtchinnikov, M.Argentati, A.Knyazev Abstract Block Locally Optimal Preconditioned Eigenvalue Xolvers (BLOPEX) is a package, written in C, that at present includes only one eigenxolver, Locally Optimal Block Preconditioned Conjugate Gradient Method (LOBPCG). BLOPEX supports parallel computations through an abstract layer. BLOPEX is incorporated in the HYPRE package from LLNL and is availabe as an external block to the PETSc package from ANL as well as a stand-alone serial library. Center for Computational Mathematics, University of Colorado at Denver Block Locally Optimal Preconditioned Eigenvalue Xolvers I.Lashuk, E.Ovtchinnikov, M.Argentati, A.Knyazev Acknowledgements Supported by the Lawrence Livermore National Laboratory, Center for Applied Scientific Computing (LLNL–CASC) and the National Science Foundation. We thank Rob Falgout, Charles Tong, Panayot Vassilevski, and other members of the Hypre Scalable Linear Solvers project team for their help and support. We thank Jose E. Roman, a member of SLEPc team, for writing the SLEPc interface to our Hypre LOBPCG solver. The PETSc team has been very helpful in adding our BLOPEX code as an external package to PETSc. Center for Computational Mathematics, University of Colorado at Denver Block Locally Optimal Preconditioned Eigenvalue Xolvers I.Lashuk, E.Ovtchinnikov, M.Argentati, A.Knyazev CONTENTS 1.
    [Show full text]
  • A New Overview of the Trilinos Project
    Scientific Programming 20 (2012) 83–88 83 DOI 10.3233/SPR-2012-0355 IOS Press A new overview of the Trilinos project Michael A. Heroux ∗ and James M. Willenbring Sandia National Laboratories, Albuquerque, NM, USA Abstract. Since An Overview of the Trilinos Project [ACM Trans. Math. Softw. 31(3) (2005), 397–423] was published in 2005, Trilinos has grown significantly. It now supports the development of a broad collection of libraries for scalable computational science and engineering applications, and a full-featured software infrastructure for rigorous lean/agile software engineering. This growth has created significant opportunities and challenges. This paper focuses on some of the most notable changes to the Trilinos project in the last few years. At the time of the writing of this article, the current release version of Trilinos was 10.12.2. Keywords: Software libraries, software frameworks 1. Introduction teams. Scientific libraries have a natural scope such that one or a few people work closely together on a The Trilinos project is a community of develop- cohesive collection of concerns. In 1999 the Acceler- ers and a collection of reusable software components ated Strategic Computing Initiative (ASCI) was fund- called packages. Trilinos provides a large and grow- ing many new algorithmic and software efforts. These ing collection of open-source software libraries for efforts included an explicit requirement to increase the scalable parallel computational science and engineer- level of formal software engineering practices and pro- ing applications, as well as a software infrastructure cesses, including efforts of small development teams. that supports a rigorous lean/agile software lifecycle Even before this time, there were numerous disjoint model [13].
    [Show full text]
  • The Deal.II Library, Version 9.3, 2021
    The deal.II Library, Version 9.3 Daniel Arndt1*, Wolfgang Bangerth2,3, Bruno Blais4, Marc Fehling2, Rene Gassmöller5, Timo Heister6, Luca Heltai7, Uwe Köcher8, Martin Kronbichler9,10, Matthias Maier11, Peter Munch9,12, Jean-Paul Pelteret13, Sebastian Proell9, Konrad Simon14, Bruno Turcksin1*, David Wells15, and Jiaqi Zhang6 1Scalable Algorithms and Coupled Physics Group, Computational Sciences and Engineering Division, Oak Ridge National Laboratory, 1 Bethel Valley Rd., TN 37831, USA. arndtd/[email protected] 2Department of Mathematics, Colorado State University, Fort Collins, CO 80523-1874, USA. bangerth/[email protected] 3Department of Geosciences, Colorado State University, Fort Collins, CO 80523, USA. 4Research Unit for Industrial Flows Processes (URPEI), Department of Chemical Engineering, Polytechnique Montréal, PO Box 6079, Stn Centre-Ville, Montréal, Québec, Canada, H3C 3A7. [email protected] 5Department of Geological Sciences, University of Florida, 1843 Stadium Road, Gainesville, FL, 32611, USA. [email protected] 6School of Mathematical and Statistical Sciences, Clemson University, Clemson, SC, 29634, USA heister/[email protected] 7SISSA, International School for Advanced Studies, Via Bonomea 265, 34136, Trieste, Italy. [email protected] 8Chair of Numerical Mathematics, Helmut-Schmidt-University, University of the Federal Armed Forces Hamburg, Holstenhofweg 85, 22043 Hamburg, Germany. [email protected] 9Institute for Computational Mechanics, Technical University of Munich, Boltzmannstr. 15, 85748 Garching, Germany. kronbichler/munch/[email protected] 10Department of Information Technology, Uppsala University, Box 337, 751 05 Uppsala, Sweden. [email protected] 11Department of Mathematics, Texas A&M University, 3368 TAMU, College Station, TX 77845, USA. [email protected] 12Institute of Material Systems Modeling, Helmholtz-Zentrum Hereon, Max-Planck-Str.
    [Show full text]
  • Trilinos Configure, Build, Test, and Install Reference Guide
    Trilinos Configure, Build, Test, and Install Reference Guide Author: Roscoe A. Bartlett ([email protected]) Contact: [email protected] Abstract This document contains reference information on how to configure, build, test, and install Trilinos using the TriBITS CMake build sys- tem. The primary audience are users of Trilinos that need to con- figure and build the software. The secondary audience are actual developers of Trilinos. Contents 1 Introduction1 2 Trilinos-specific options1 2.1 Enabling float and complex Scalar types..............2 2.2 Enabling/disabling time monitors..................2 3 Getting set up to use CMake3 3.1 Installing a binary release of CMake [casual users]........3 3.2 Installing CMake from source [developers and experienced users]3 4 Getting CMake Help3 4.1 Finding CMake help at the website.................3 4.2 Building CMake help locally.....................3 5 Configuring (Makefile Generator)4 5.1 Setting up a build directory.....................4 5.2 Basic configuration..........................4 5.3 Selecting the list of packages to enable...............6 5.3.1 Determine the list of packages that can be enabled....6 5.3.2 Print package dependencies.................7 5.3.3 Enable a set of packages...................8 5.3.4 Enable to test all effects of changing a given package(s).8 5.3.5 Enable all packages with tests and examples........8 5.3.6 Disable a package and all its dependencies.........9 5.3.7 Remove all package enables in the cache..........9 5.4 Selecting compiler and linker options................ 10 1 5.4.1 Configuring to build with default debug or release compiler flags.............................
    [Show full text]
  • Sphynx: a Parallel Multi-GPU Graph Partitioner for Distributed-Memory Systems
    Sphynx: a parallel multi-GPU graph partitioner for distributed-memory systems Seher Acer1,∗, Erik G. Boman1,∗, Christian A. Glusa1, Sivasankaran Rajamanickam1 Center for Computing Research, Sandia National Laboratories, Albuquerque, NM, U.S.A. Abstract Graph partitioning has been an important tool to partition the work among several processors to minimize the communication cost and balance the workload. While accelerator-based supercomputers are emerging to be the standard, the use of graph partitioning becomes even more important as applications are rapidly moving to these architectures. However, there is no distributed-memory- parallel, multi-GPU graph partitioner available for applications. We developed a spectral graph partitioner, Sphynx, using the portable, accelerator-friendly stack of the Trilinos framework. In Sphynx, we allow using different preconditioners and exploit their unique advantages. We use Sphynx to systematically evaluate the various algorithmic choices in spectral partitioning with a focus on the GPU performance. We perform those evaluations on two distinct classes of graphs: regular (such as meshes, matrices from finite element methods) and irregular (such as social networks and web graphs), and show that different settings and preconditioners are needed for these graph classes. The experimental results on the Summit supercomputer show that Sphynx is the fastest alternative on irregular graphs in an application-friendly setting and obtains a partitioning quality close to ParMETIS on regular graphs. When compared to nvGRAPH on a single GPU, Sphynx is faster and obtains better balance and better quality partitions. Sphynx provides a good and robust partitioning method across a wide range of graphs for applications looking for a GPU-based partitioner.
    [Show full text]
  • Solving Sparse Linear Systems Part 2: Trilinos Solver Overview
    Solving Sparse Linear Systems Part 2: Trilinos Solver Overview Michael A. Heroux Sandia National LaBoratories http://www.users.csBsju.edu/~mheroux/HartreeTutorialPart2.pdF Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, 1 for the United States Department of Energy under contract DE-AC04-94AL85000. Outline § Sources oF sparse linear systems. § Sparse solver ecosystem. § Details aBout sparse data structures. § Overview oF Trilinos. § Overview oF Trilinos linear solvers. 2 If You Know Some Linear Algebra § If this intro material is not new, visit the Trilinos Tutorial Site: w https://github.com/trilinos/Trilinos_tutorial/wiki/TrilinosHandsOnTutorial w Lots of examples. w A Web portal for compiling, linking and running Trilinos examples. w Pointers to Trilinos User Group videos: Coverage of many Trilinos topics. § If you know Linux well: w Start installing Trilinos from the instructions. w Install these packages: • Kokkos, Tpetra, Epetra, EpetraExt, AztecOO, ML, Belos, Muelu, Ifpack, Ifpack2, Zoltan, Zoltan2, Amesos • Rob Allan has already done this on Tutorial systems, so he can assist you. § If you are into Docker: w Go https://hub.docker.com/r/sjdeal/webtrilinos/ for image. § If you want to start on the Epetra, AztecOO example: w Scroll down the tutorial page and start using and studying them. 3§ Explore… Setting up Linux Environment § Commands: w module avail # (look for Trilinos install) w module load intel # (get Intel compilers) w module load trilinos/12.2.1 w env # (look for path to Trilinos) § Copy and paste sample makefile from Hands-on page w http://trilinos.org/oldsite/Export_Makefile_example.txt w Get rid of references to libmyappLib w Define: MYAPP_TRILINOS_DIR=/gpfs/stfc/local/apps/intel/trilinos/12.2.1 § Try to make.
    [Show full text]
  • Trilinos Tutorial
    Trilinos Tutorial For Trilinos Release 10.2 Sandia Report SAND2004-2189, September 2004, Unlimited Release Last updated March 2010 Marzio Sala Michael A. Heroux David M. Day James M. Willenbring (Editors) T O EN F E TM N R E A R P G E Y D U • • N A I C T I E R D E S M T A ATES OF Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Com- pany, for the United States Department of Energy’s National Nuclear Security Administration under Contract DE-AC04-94-AL85000. Approved for public release; further dissemination unlimited. Sandia National Laboratories www.sandia.gov PO BOX 5800 Albuquerque, New Mexico, USA Preface The Trilinos Project is an effort to facilitate the design, development, integration and ongoing support of mathematical software libraries. The goal of the Trilinos Project is to develop algorithms and enabling technologies within an object-oriented software framework for the solution of large-scale, complex multiphysics engineering and scientific applications. There is an emphasis is on developing robust, scalable algorithms in a software framework, using abstract interfaces for flexible interoperability of components while providing a full-featured set of concrete classes that implement all the abstract interfaces. This document introduces the use of Trilinos, Release 10.2 and subsequent minor releases 10.2.X. The presented material includes, among others, the definition of distributed matrices and vectors with Epetra, the iterative solution of linear systems with AztecOO, incomplete factorizations with IFPACK, multilevel and domain decomposition preconditioners with ML, direct solution of linear system with Amesos, eigenvalues and eigenvectors computations with Anasazi, and iterative solution of nonlinear systems with NOX.
    [Show full text]