Numerical and Parallel Libraries

Uwe Küster

University of Stuttgart High-Performance Computing-Center Stuttgart (HLRS) www.hlrs.de Numerical Numerical ParallelLibraries and [33]

Libraries Slide 1 Höchstleistungsrechenzentrum Stuttgart Numerical Libraries

Public Domain commercial vendor specific

Libraries Uwe Küster Slide 2 of 35 Höchstleistungsrechenzentrum Stuttgart Overview

• numerical libraries for linear systems – dense – sparse

Numerical and Parallel Libraries [33] • FFT • support for parallelization

Libraries Uwe Küster Slide 3 of 35 Höchstleistungsrechenzentrum Stuttgart Public Domain

Lapack-3 linear equations, eigenproblems BLAS fast linear kernels Linpack linear equations Eispack eigenproblems Slatec old library, large functionality Quadpack numerical quadrature Itpack sparse problems pim linear systems PETSc linear systems Netlib Server best server http://www.netlib.org/utk/papers/iterative-survey/packages.html

Libraries Uwe Küster Slide 4 of 35 Höchstleistungsrechenzentrum Stuttgart netlib server

for all public domain numerical programs and libraries http://www.netlib.org http://www-unix.mcs.anl.gov/scidac-tops/Software_Page.html

Libraries Uwe Küster Slide 5 of 35 Höchstleistungsrechenzentrum Stuttgart Contents of netlib

access aicm alliant amos ampl anl-reports apollo atlas benchmark bib bibnet bihar blacs blas blast bmp c c++ cephes chammp cheney-kincaid clapack commercial confdb conformal contin control crc cumulvs ddsv dierckx diffpack domino eispack elefunt env f2c fdlibm fftpack fishpack fitpack floppy fmm fn fortran fortran-m fp gcv gmat gnu go graphics harwell hence hompack hpf hypercube ieeecss ijsa image intercom itpack jakef java kincaid-cheney la-net lanczos lanz lapack lapack++ lapack90 laso lawson-hanson linalg linpack list lp machines magic maspar mds microscope minpack misc mpfun mpi mpicl na-digest-html napack netsolve news numeralgo ode odepack odrpack opt p4 paragraph paranoia parkbench parmacs pascal pdes performance photo picl pltmg poly2 polyhedra popi port posix pppack presto problem-set pvm3 quadpack random research scalapack sched scilib seispack sequent sfmm slap slatec sminpack sodepack sparse sparse-blas sparspak specfun spin srwn stoeplitz stringsearch svdpack templates tennessee textbook toeplitz toms tomspdf transform typesetting uncon vanhuffel vfftpack vfnlib voronoi xmagic xnetlib y12m Libraries Uwe Küster Slide 6 of 35 Höchstleistungsrechenzentrum Stuttgart Links for SciDAC-TOPS

• Hypre - A family of powerful, scalable preconditioners • PARPACK - Scalable solvers for sparse generalized eigensystems • PETSc - Scalable nonlinear and linear solvers for PDEs and related problems • SLEPc , A scalable eigenvalue solvers package • SUNDIALS - Parallel ODE integrators and nonlinear solvers • SuperLU - Parallel direct sparse LU factorization • TAO - Scalable optimization software • Veltisto - Scalable PDE-constrained optimizer

Libraries Uwe Küster Slide 7 of 35 Höchstleistungsrechenzentrum Stuttgart VSIPL

Vector/Signal/Image Processing Library http://www.vsipl.org/ C,C++ FFT, histogram, density, median,convolution, gradient edge detection,morphological operators, expand, shrink, padding matrix product, -transpose, -LU, -Cholesky, QR, toeplitz solver, least squares, vector operations

Libraries Uwe Küster Slide 8 of 35 Höchstleistungsrechenzentrum Stuttgart Gnu Scientific Library (GSL)

• Complex Numbers • Integration • Roots of Polynomials • Simulated Annealing • Special Functions • Differential Equations • Vectors and Matrices • Interpolation • Permutations • Sorting • Numerical Differentiation • BLAS Support • Chebyshev Approximation • Linear Algebra • Series Acceleration • Eigensystems • Discrete Hankel Transforms • Fast Fourier Transforms • Root-Finding • Quadrature • Minimization • Random Numbers • Quasi-Random Sequences • Least-Squares Fitting • Random Distributions • Physical Constants • Statistics Histograms • IEEE Floating-Point • N-Tuples • Discrete Wavelet Transforms • Monte Carlo • Basis splines

http://www.lrz- muenchen.de/services/software/mathematik/gsl/fortran/ index.html

Libraries Uwe Küster Slide 9 of 35 Höchstleistungsrechenzentrum Stuttgart LAPACK for linear systems

• for dense problems, fast • Linear Equations, real and complex – Orthogonal Factorizations and Linear Least Squares Problems – Generalized Orthogonal Factorizations and Linear Least Squares Problems • Eigenproblems • Symmetric and non-symmetric real eigenproblems – hermitian and non-hermitian complex – Generalized Symmetric Definite Eigenproblems – Generalized Non-symmetric Eigenproblems • Singular Value Decomposition • Generalized (or Quotient) Singular Value Decomposition • based on BLAS • shared memory parallel versions

Libraries Uwe Küster Slide 10 of 35 Höchstleistungsrechenzentrum Stuttgart Basic Linear Algebra Subroutines: BLAS

•BLAS-1 vector x vector saxpy data transfer 3*n operations 2*n-1 FLOP/Word 2/3

•BLAS-2 matrix x vector data transfer n*n+2*n Numerical and Parallel Libraries [33] operations n*(2*n-1) FLOP/Word 2

•BLAS-3 matrix x matrix data transfer 3*n*n operations n*n*(2*n-1) FLOP/Word 2/3*n

Libraries Uwe Küster Slide 11 of 35 Höchstleistungsrechenzentrum Stuttgart where to get Lapack and BLAS

• search engine for Lapack: – problem definition, code, calling parameters, … – download – http://www.cs.colorado.edu/~lapack/ • Sources from Netlib • vendor tuned versions on all platforms, also parallel • for PC/Windows free Intel® Math Kernel Library http://developer.intel.com/software/products/mkl/index.htm • for AMD http://www.amd.com/us- en/Processors/DevelopWithAMD/0,,30_2252_869_2282,00.html • GOTO-Blas : http://www.tacc.utexas.edu/resources/software

Libraries Uwe Küster Slide 12 of 35 Höchstleistungsrechenzentrum Stuttgart getting BLAS by ATLAS:

generates tuned versions of e.g. BLAS code Automated Emprical Optimization of Software (AEOS) http://www.netlib.org/atlas/

available for LAPACK Fortran and C : ( ◊: S,D,C,Z) ◊GESV (lin. equ.) ◊GETRF(lin. equ. LU) ◊GETRS(lin. equ. solve) ◊POSV (sym. pos. lin. equ.) ◊POTRF(sym. pos. lin. equ. LU) ◊POTRS(sym. pos. lin. equ. solve)

Libraries Uwe Küster Slide 13 of 35 Höchstleistungsrechenzentrum Stuttgart ScaLAPACK http://www.netlib.org/scalapack/

Scalable Linear Algebra PACKage, or Scalable LAPACK handles dense and band matrices on distributed memory machines not general sparse matrices functionality is subset of LAPACK based on BLAS and BLACS

Libraries Uwe Küster Slide 14 of 35 Höchstleistungsrechenzentrum Stuttgart ScaLAPACK contents

• Linear Equations – Orthogonal Factorizations and Linear Least Squares Problems • QR Factorization • LQ Factorization • QR Factorization with Column Pivoting • Complete Orthogonal Factorization • Other Factorizations – Generalized Orthogonal Factorizations • Generalized QR Factorization • Generalized RQ factorization • Symmetric and Non-symmetric Eigenproblems – Eigenvalues, Eigenvectors, and Schur Factorization • Singular Value Decomposition • Generalized Symmetric Definite Eigenproblems

Libraries Uwe Küster Slide 15 of 35 Höchstleistungsrechenzentrum Stuttgart Commercial Libraries

•IMSL Fortran 77 Library •NAG Fortran 77 Library F90 Library Parallel Library SMP Library

Libraries Uwe Küster Slide 16 of 35 Höchstleistungsrechenzentrum Stuttgart NAG Parallel Library, commercial

• distributed memory machine (MPI, PVM) • http://www.nag.co.uk/numeric/fd/FDdescription.asp • Dense linear algebra (including ScaLapack) • Eigenvalue and singular value problems • Input/Output, data distribution • Optimisation • Quadrature • Random number generators • Sparse linear algebra • Sparse matrix solvers • Support/utility routines • contents in http://www.nag.co.uk/numeric/FD/manual/html/genint/FDlibconts.asp

Libraries Uwe Küster Slide 17 of 35 Höchstleistungsrechenzentrum Stuttgart NAG SMP Library, commercial

• Zeros of Polynomials, Roots ofTranscendental Equations • Quadrature, Numerical Differentiation, Interpolation, Approximation • Ordinary Differential Equations,Partial Differential Equations • Integral Equations • Curve and Surface Fitting • Minimizing or Maximizing a Function • Matrix Factorizations • Eigenvalues and Eigenvectors • Orthogonalisation • Linear Equations (LAPACK) • Least-squares and Eigenvalue Problems (LAPACK) • Sparse Linear Algebra • Statistics • Random Number Generators, sorting • Time Series Analysis • Operations Research

Libraries Uwe Küster Slide 18 of 35 Höchstleistungsrechenzentrum Stuttgart Vendor specific Libraries

Intel MKL AMD ACML CRAY SciLib IBM ESSL Numerical and Parallel Libraries [33] NEC ASL NEC MPACK NEC Mathkeisan

Libraries Uwe Küster Slide 19 of 35 Höchstleistungsrechenzentrum Stuttgart NEC MathKeisan (ASL) contents 1

• linear equations, eigenvalues and eigenvectors direct, iterative, parallel sparse, banded and dense multiple right hand sides real and complex (non)symmetric, (non)hermitian

Libraries Uwe Küster Slide 20 of 35 Höchstleistungsrechenzentrum Stuttgart NEC MathKeisan (ASL) contents 2

Fast Fourier Transform Wavelet Transform Interpolation, Approximation Integration Roots of equations Ordinary Differential equations Random numbers, Sorting Special functions Statistics

Libraries Uwe Küster Slide 21 of 35 Höchstleistungsrechenzentrum Stuttgart NEC MathKeisan (MPACK) contents

BLAS Basic Linear Algebra Subprograms LAPACK Linear algebra for high performance computers ScaLAPACK Scalable Linear Algebra package (contains PBLAS) BLACS Basic Linear Algebra Communication Subprograms PARBLAS Shared memory Parallel BLAS ARPACK Solution of large scale eigenvalue problems FFT HP's VECLIB and SGI/CRAY LIBSCI 3.1 interface SOLVER Direct solver for sparse SPD systems METIS Matrix/Graph ordering and partitioning library ParMETIS Parallel Matrix/Graph ordering and partition library

Libraries Uwe Küster Slide 22 of 35 Höchstleistungsrechenzentrum Stuttgart PETSc Features (http://www-fp.mcs.anl.gov/petsc)

Libraries Uwe Küster Slide 23 of 35 Höchstleistungsrechenzentrum Stuttgart PETSc components

• Vec vector operations, scatter, gather, ghostpoints for regular structures

• Mat data structures for parallel sparse matrices

• PC preconditioners (ILU(k), LU, block Jakobi, additive Schwarz)

• KSP Krylov subspace iterative methods (GMRES, CG, CGS, Bi-CG-Stab, TFQMR, CR, LSQR)

• SNES Newton-like methods for nonlinear systems

• TS (pseudo-) time evolution codes for solution of PDEs

Libraries Uwe Küster Slide 24 of 35 Höchstleistungsrechenzentrum Stuttgart ParPre

• preconditioners for large sparse iterative solvers • uses PETSc • additive Schwarz • multiplicative Schwarz • block SSOR • domain decomposition methods • multilevel • http://www.cs.utk.edu/~eijkhout/parpre.html

Libraries Uwe Küster Slide 25 of 35 Höchstleistungsrechenzentrum Stuttgart PARPACK

http://www.caam.rice.edu/software/ARPACK/ http://www.caam.rice.edu/~kristyn/parpack_home.html Parallel version of ARPACK calculates a few eigenvalues of a large sparse matrix Implicitly Restarted Lanczos Method (IRLM) symmetric, non-symmetric, and generalized eigenproblems Singular Value Decomposition.

Libraries Uwe Küster Slide 26 of 35 Höchstleistungsrechenzentrum Stuttgart European Parasol project (Mumps)

http://www.enseeiht.fr/lima/apo/mumps/European ESPRIT Activity MUMPS (MUltifrontal Massively Parallel sparse direct Solver ) • public domain • symmetric positive definite matrices; • general symmetric matrices; • general unsymmetric matrices;

Numerical and Parallel Libraries [33] • Iterative refinement and backward error analysis; •Input matrix in assembled format distributed assembled format elemental format

Libraries Uwe Küster Slide 27 of 35 Höchstleistungsrechenzentrum Stuttgart Aztec

• iterative library for solution of large distributed sparse linear systems • CG, GMRES, BiCGSTAB • for systems arising from discretization of partial differential equations • Jakobi, polynomial, LU, ILU etc. preconditioning in processor domains • distributed modified sparse row format (DMSR) • distributed variable block row format (DVBR) • matrix free interface, user defined preconditioning interface • unknown numbering may be global • free for research; license necessary • http://www.cs.sandia.gov/CRF/aztec1.html

Libraries Uwe Küster Slide 28 of 35 Höchstleistungsrechenzentrum Stuttgart trilinos

• Framework and Tools • Software engineering Technology and Integration • Discretizations • Meshes, Geometry and Load Balancing • Scalable Linear Algebra • Linear and Eigen Solvers • Embedded Nonlinear Analysis Tools • http://trilinos.sandia.gov/capability_areas.html

Libraries Uwe Küster Slide 29 of 35 Höchstleistungsrechenzentrum Stuttgart MFACT

• multifrontal direct solver • sparse symmetric positive definite linear systems • Lapack style interface • similar to CAPSS • free • http://www.cs.utk.edu/~padma/mfact.html

Libraries Uwe Küster Slide 30 of 35 Höchstleistungsrechenzentrum Stuttgart SuperLU

• solves large sparse non-symmetric systems • direct solver • SuperLU seriel • SuperLU_MT shared memory parallel • SuperLU_DIST distributed memory • free • http://www.nersc.gov/~xiaoye/SuperLU/

Libraries Uwe Küster Slide 31 of 35 Höchstleistungsrechenzentrum Stuttgart FFTW

• Discrete Fourier Transform • one and more dimension • C, wrapper routines for Fortran • free GPL • http://www.fftw.org/

Libraries Uwe Küster Slide 32 of 35 Höchstleistungsrechenzentrum Stuttgart ParMETIS

• graph partitioning • mesh partitioning • graph repartitioning • graph partition refinement • matrix reordering

Libraries Uwe Küster Slide 33 of 35 Höchstleistungsrechenzentrum Stuttgart global arrays

• shared memory programming interface for distributed memory machines • asynchronously access of logical blocks of physically distributed dense multi-dimensional arrays • cooperating with shared memory paradigm and MPI • create a distributed array, controlling alignment and distribution; duplicate an array with properties of another existing array; destroy an array; synchronize. • fetch, store and accumulate into a rectangular patch of global array; gather and scatter; atomic read and increment efficient access to local elements of global array (by a pointer).

• free • http://www.emsl.pnl.gov:2080/docs/global/ga.html

Libraries Uwe Küster Slide 34 of 35 Höchstleistungsrechenzentrum Stuttgart end

Libraries Uwe Küster Slide 35 of 35 Höchstleistungsrechenzentrum Stuttgart