SANDIA REPORT SAND2004-4820 Unlimited Release Printed September 2004 Amesos 2.0 Reference Guide Marzio Sala Prepared by Sandia National Laboratories Albuquerque, New Mexico 87185 and Livermore, California 94550 Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy's National Nuclear Security Administration under Contract DE-AC04-94-AL85000. Approved for public release; further dissemination unlimited. Issued by Sandia National Laboratories, operated for the United States Department of Energy by Sandia Corporation. NOTICE: This report was prepared as an account of work sponsored by an agency of the United States Government. Neither the United States Government, nor any agency thereof, nor any of their employees, nor any of their contractors, subcontractors, or their employees, make any warranty, express or implied, or assume any legal liability or re- sponsibility for the accuracy, completeness, or usefulness of any information, appara- tus, product, or process disclosed, or represent that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise, does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Govern- ment, any agency thereof, or any of their contractors or subcontractors. The views and opinions expressed herein do not necessarily state or reflect those of the United States Government, any agency thereof, or any of their contractors. Printed in the United States of America. This report has been reproduced directly from the best available copy. Available to DOE and DOE contractors from U.S. Department of Energy Office of Scientific and Technical Information P.O. Box 62 Oak Ridge, TN 37831 Telephone: (865) 576-8401 Facsimile: (865) 576-5728 E-Mail: [email protected] Online ordering: http://www.doe.gov/bridge Available to the public from U.S. Department of Commerce National Technical Information Service 5285 Port Royal Rd Springfield, VA 22161 Telephone: (800) 553-6847 Facsimile: (703) 605-6900 E-Mail: [email protected] Online ordering: http://www.ntis.gov/help/ordermethods.asp?loc=7-4-0#online NT O E F E TM N R E A R P G E Y D • • U N A I C T I E R D E S M T A ATES OF ¢¡¤£¦¥¨§ © © § © ¦ !" # § © © Printed %$&(')%* Amesos 2.0 Reference Guide Marzio Sala Computational Math & Algorithms Sandia National Laboratories P.O. Box 5800, MS 1110 Albuquerque, NM 87185-1110 Abstract This document describes the main functionalities of the version 2.0 of the AMESOS pack- age. AMESOS provides an object-oriented interface to several serial and parallel sparse direct solvers libraries for the solution of the linear system of equations +-,/.1032 (1) , where + is a real sparse, distributed matrix, defined as an Epetra RowMatrix object, and 0 and are defined as Epetra MultiVector objects. AMESOS provides a common look-and- feel for all interfaces, and insulates the user from each solver’s details, such as matrix and vector formats, and data distribution. Currently supported libraries are: LAPACK, KLU, UMFPACK, PARDISO, TAUCS, SuperLU, SuperLU DIST, MUMPS, DSCPACK. This document is organized as follows. First, Section 1 introduces the design of AMESOS. Section 2 presents the basic usage of the AMESOS package. Section 3 details how to configure and compile AMESOS. Section 4 describes the interfaces of AMESOSto the supported direct solvers. A brief note on the examples included in the distribution is reported in Section 5. 3 Acknowledgments The author would like to acknowledge the support of the ASCI and LDRD programs that funded development of AMESOS. 4 Amesos 2.0 Reference Guide Contents 1 The Design of AMESOS . 6 2 Basic Usage . 7 2.1 Supported Matrix Formats . 9 2.2 Parameters for All AMESOS Solvers . 9 3 Configuring and Installing AMESOS . 12 4 Supported Solvers . 14 4.1 Interface to LAPACK . 14 4.2 Interface to KLU . 14 4.3 Interface to UMFPACK 4.3 . 16 4.4 Interface to PARDISO 1.2.3 . 16 4.5 Interface to TAUCS 2.2 . 17 4.6 Interface to SuperLU 3.0 . 17 4.7 Interface to SuperLU DIST 2.0 . 17 4.8 Interface to MUMPS 4.3.1 . 19 4.9 Interface to DSCPACK 1.0 . 22 5 Guide to the Examples . 22 5 1 The Design of AMESOS The AMESOS package, developed by (in alphabetical order) T. Davis, M. Heroux, R. Hoekstra, M. Sala, and K. Stanley, is an effort to define a set of object-oriented, abstract interfaces for the usage of serial and parallel sparse direct solvers. Although one serial direct solver, KLU, is distributed within AMESOS, the goal of the AMESOS project is to make it easier to interface a code that makes use of EPETRA objects with direct solver libraries developed outside TRILINOS. AMESOS is written in C++, and has been designed with the following requirements: Simplicity of usage: Solving linear system (1) in a language like MATLAB is very easy, just write X = A \ b. It should not be much more difficult in a C++, production code. Flexibility: More than one algorithm must be available, to offer optimal algorithms for small and large matrices, serial and parallel. Efficiency: The solution of (1) must be as efficient as possible, using state-of-the-art algo- rithms. Besides, the overhead due to the C++ design must be minimal. To fulfill these design requirements, we split the solution of linear system (1) into the following steps: 1. Definition of the sparsity pattern of the linear system matrix; 2. Computation of the symbolic factorization; 3. Definition of the values of the linear system matrix; 4. Computation of the numeric factorization; 5. Definition of the values of the right-hand side; 6. Solution of the linear system. Steps 1, 3 and 5 depend on the matrix and vector format. To increase flexibility, AMESOS requires the matrix to be derived from the Epetra_RowMatrix format, and the solution and right-hand side vector to the defined as Epetra_MultiVector’s. Steps 2, 4 and 6 correspond to three different methods in the AMESOS classes. From an ab- stract point of view, these steps do not depend on the direct solver of choice. However, their concrete implementations does, since different libraries may require different matrix and vector formats and distribution, may have different parameters or different ways of setting the same pa- rameter. To obtain flexibility, AMESOS insulates the user from the details specific to each solver, so that generic methods can be used to manipulate all the supported interface. This design goal is ac- complished using a pure virtual class, which defines methods SymbolicFactorization(), NumericFactorization() and Solve(), plus method SetParameters() which can be used to tune the interface. To increase efficiency, all AMESOS classes are defined as light containers. Each class simply converts the matrix A from the input Epetra_RowMatrix format into the solver’s required format, and sets the parameters are defined by the user. Therefore, AMESOS interfaces are as efficient as the underline solver library. 6 2 Basic Usage A fragment of code using AMESOS is as follows. Let us suppose that A is an Epetra_RowMatrix, and X and B are two Epetra_MultiVector’s. First, we need to include the header files for AMESOS: #include "Amesos.h" #include "Amesos_BaseSolver.h" Note that these header files will not include the header files for the supported libraries (which are of course needed to compile the AMESOS library itself). Then, we need to create an linear problem, as follows: Epetra_LinearProblem Problem(&A, &X, &B); At this point, we can create an AMESOS class using the factory class Amesos: Amesos_BaseSolver* Solver; Amesos Factory; char* SolverType = "Amesos_Klu"; // uses the KLU direct solver Solver = Factory.Create(SolverType, Problem); At this point, we can perform the symbolic factorization of the linear system matrix: AMESOS_CHK_ERR(Solver->SymbolicFactorization()); This phase does not require the numerical values of A, which can therefore be changed after the call to SymbolicFactorization(). However, the nonzero pattern of A cannot be changed. AMESOS_CHK_ERR is a macro (defined in Amesos_ConfigDefs.h) that checks the return code: if not zero, the macro prints out an error message, and returns. The numeric factorization is performed by AMESOS_CHK_ERR(Solver->NumericFactorization()); NumericFactorization() accesses the values of A, but does not consider the vectors X and B. Finally, to solve the linear system, we simply write AMESOS_CHK_ERR(Solver->Solve()); In the previous example, we showed how to use the KLU solver (see Section 4.2 for more details). Other interfaces can be created using the factory class by simply changing one parameter. Note that the supported solver can be serial or parallel, dense or sparse: the user code still remains the same (except for the name of the solver); AMESOS will take care of data redistribution if required by the selected solver. The list of supported solvers is reported in Table 1. Method Factory.Query() can be used to query the factory about the availability of a given solver: char* SolverType = "Amesos_Klu"; bool IsAvailable = Factory.Query(SolverType); 7 Class Communicator Matrix type Interface to Amesos Lapack serial general LAPACK Amesos Klu serial general KLU Amesos Umfpack serial general UMFPACK 4.3 Amesos Pardiso serial/OMP general PARDISO 1.2.3 Amesos Taucs serial symmetric TAUCS 2.2 Amesos Superlu serial general SuperLU 3.0 Amesos Superludist parallel general SuperLU DIST 2.0 Amesos Mumps parallel SPD, sym, general MUMPS 4.3.1 Amesos Dscpack parallel symmetric DSCPACK 1.0 Table 1. Supported interfaces. “serial” means that the supported direct solver is serial. In this case, when solving with more than one proces- sor, the linear problem is gathered to process 0, here solved, then the solution is broadcasted to the distributed solution vector.