LAPACK++ V. 1.0 High Performance Linear Algebra Users' Guide April 1994 Jack Dongarra Roldan Pozo David Walker Oak Ridge National Lab oratory UniversityofTennessee, Knoxville Contents 1 Intro duction 3 2 Overview 5 2.1 Contents of LAPACK++ :: ::: :::: ::: :::: ::: :::: ::: :::: ::: :::: : 5 2.2 A simple co de example :::: ::: :::: ::: :::: ::: :::: ::: :::: ::: :::: : 6 2.3 Basic Linear Algebra Subroutines BLAS++ : :::: ::: :::: ::: :::: ::: :::: : 6 2.4 Performance ::: ::: :::: ::: :::: ::: :::: ::: :::: ::: :::: ::: :::: : 7 3 LAPACK++ Matrix Ob jects 9 3.1 Fundamental Matrix Op erations : :::: ::: :::: ::: :::: ::: :::: ::: :::: : 9 3.2 General Rectangular Matrices ::: :::: ::: :::: ::: :::: ::: :::: ::: :::: : 10 3.2.1 Declarations :: :::: ::: :::: ::: :::: ::: :::: ::: :::: ::: :::: : 10 3.3 Triangular Matrices : :::: ::: :::: ::: :::: ::: :::: ::: :::: ::: :::: : 12 3.3.1 Symmetric, Hermitian and SPD Matrices ::: ::: :::: ::: :::: ::: :::: : 12 3.4 Banded Matrices ::: :::: ::: :::: ::: :::: ::: :::: ::: :::: ::: :::: : 12 3.4.1 Triangular Banded Matrices :::: ::: :::: ::: :::: ::: :::: ::: :::: : 13 3.4.2 Symmetric and Hermitian Banded Matrices :: ::: :::: ::: :::: ::: :::: : 14 3.5 Tridiagonal Matrices : :::: ::: :::: ::: :::: ::: :::: ::: :::: ::: :::: : 14 4 Driver Routines 15 4.1 Linear Equations ::: :::: ::: :::: ::: :::: ::: :::: ::: :::: ::: :::: : 15 4.2 Eigenvalue Problems : :::: ::: :::: ::: :::: ::: :::: ::: :::: ::: :::: : 18 4.3 Memory Optimizations: Factorizing in place :: :::: ::: :::: ::: :::: ::: :::: : 18 5 Factorization Classes 19 5.1 Optimizations: Factoring in Place :::: ::: :::: ::: :::: ::: :::: ::: :::: : 19 A Programming Examples 21 A.1 Polynomial data tting :::: ::: :::: ::: :::: ::: :::: ::: :::: ::: :::: : 21 B Release Notes for v. 1.0 23 B.1 How to rep ort bugs :: :::: ::: :::: ::: :::: ::: :::: ::: :::: ::: :::: : 23 B.2 Tips and Suggestions : :::: ::: :::: ::: :::: ::: :::: ::: :::: ::: :::: : 23 B.3 Reducing Compilation time : ::: :::: ::: :::: ::: :::: ::: :::: ::: :::: : 23 B.3.1 Minimize the numb er of included header les : ::: :::: ::: :::: ::: :::: : 23 B.3.2 Header le madness : ::: :::: ::: :::: ::: :::: ::: :::: ::: :::: : 23 B.4 Performance Considerations : ::: :::: ::: :::: ::: :::: ::: :::: ::: :::: : 23 B.4.1 Array Bounds Checking :: :::: ::: :::: ::: :::: ::: :::: ::: :::: : 23 B.4.2 Improving Ai; j eciency :::: ::: :::: ::: :::: ::: :::: ::: :::: : 24 B.5 Exception Handling : :::: ::: :::: ::: :::: ::: :::: ::: :::: ::: :::: : 24 1 2 CONTENTS B.6 Frequently asked questions : ::: :::: ::: :::: ::: :::: ::: :::: ::: :::: : 24 B.6.1 What is the p erformance of LAPACK++? ::: ::: :::: ::: :::: ::: :::: : 24 B.6.2 Do I need a Fortran compiler to use LAPACK++? : :::: ::: :::: ::: :::: : 24 B.6.3 Why are the LAPACK++ matrix classes not templated? : ::: :::: ::: :::: : 24 B.6.4 Can LAPACK++ work with built-in C/C++ arrays and other matrix classes? ::: : 25 B.6.5 Can the matrix and vector ob jects b e used indep endently of BLAS++ and LAPACK++? 25 B.6.6 Is there an existing standard for C++ matrix classes? ::: ::: :::: ::: :::: : 25 C Direct Interfaces to LAPACK and BLAS Routines 27 Chapter 1 Intro duction LAPACK++ is an ob ject-oriented C++ extension to the LAPACK [1] library for numerical linear algebra. This package includes state-of-the-art numerical algorithms for the more common linear algebra problems encountered in scienti c and engineering applications: solving linear equations, linear least squares, and eigenvalue problems for dense and banded systems. Traditionally, such libraries have b een available only in Fortran; however, with an increasing number of programmers using C and C++ for scienti c software development, there is a need to have high-quality numerical libraries to supp ort these platforms as well. LAPACK++ provides the sp eed and eciency comp etitive with nativeFortran co des see section 2.4, while allowing programmers to capitalize on the software engineering b ene ts of ob ject oriented programming. The overall design of LAPACK++ includes supp ort for distributed and shared memory architectures [6] .Version 1.0 includes supp ort only for unipro cessor and shared memory platforms. Distributed memory architectures will b e supp orted in Version 2.0. Replacing the Fortran 77 interface of LAPACK with an ob ject-oriented framework simpli es the co ding style and allows for a more exible and extendible software platform. The design goals of LAPACK++ include Maintain p erformance comp etitive with Fortran. Provide a simple interface that hides implementation details of various matrix storage schemes and their corresp onding factorization structures. Provide a universal interface and op en system design for integration into user-de ned data structures and third-party matrix packages. Replace static work array limitations of Fortran with more exible and typ e-safe dynamic memory allo cation schemes. Provide an ecient indexing scheme for matrix elements that has minimal overhead and can b e opti- mized for in most application co de lo ops. Utilize function and op erator overloading in C++ to simplify and reduce the number of interface entry p oints to LAPACK. Utilize exception error handling in C++ for intelligent managing of error situations without cluttering up application co des. Provide the capability to access submatrices by reference, rather than byvalue, and p erform factor- izations \in place". This is vital for implementing blo cked algorithms eciently. 3 4 CHAPTER 1. INTRODUCTION Provide more meaningful naming conventions for variables and function names. Names no longer limited to six alphanumeric characters. LAPACK++ also provides an ob ject-oriented interface to the Basic Linear Algebra Subprograms BLAS [4], [8] see section 2.3, allowing programmers to utilize these optimized computational kernels in their own C++ applications. Chapter 2 Overview The underlying philosophy of the LAPACK++ is to provide an interface which is relatively simple, yet powerful enough to express all complex and subtle tasks within LAPACK, including those which optimize p erformance and/or storage. Following the framework of LAPACK, the C++ extension contains driver routines for solving standard typ es of problems, computational routines to p erform a distinct computa- tional task, and auxiliary routines to p erform a certain subtask or common low-level computation. Each driver routine typically calls a sequence of computational routines. Taken as a whole, the computational routines can p erform a wider range of tasks than are covered by the driver routines. Currently, dense and band matrices are supp orted. General sparse matrices are handled in [7]. 2.1 Contents of LAPACK++ With over 1,000 subroutines in the original f77 LAPACK, not every routine is implemented in LAPACK++. Instead, source co de examples in the various ma jor areas are provided, allowing users to easily extend the package for their particular needs. LAPACK++ provides source co de for Algorithms { LU Factorization T { Cholesky LL Factorization { QR Factorization { Eigenvalue problems Storage Classes { rectangular matrices { symmetric and symmetric p ositive de nite SPD { banded matrices { tri/bidiagonal matrices Element Data Typ es { int, long int, oat, double, double precision complex, { arbitrary Vector data typ es via templates section B.6.3 5 6 CHAPTER 2. OVERVIEW 2.2 A simple co de example To provide a rst glimpse at how LAPACK++ simpli es the user interface, this section presents a few simple co de fragments. The examples are incomplete and are meant to merely illustrate the interface style. The next few sections will further discuss the details of matrix classes and their op erations. The rst example illustrates a co de fragment to solve a linear system Ax = b using LU factorization: include <lapack++.h> // 1 LaGenMatDouble AN,N; // 2 LaVectorDouble xN, bN; // 3 ... LaLinSolveA,x,b; // 4 Line 1 includes all of the LAPACK++ ob ject and function declarations. Line 2 declares A to b e a square N N co ecient matrix, while line 3 declares the right-hand-side and solution vectors. Finally, the LaLinSolve function in line 4 calls the underlying LAPACK driver routine SGESV for solving linear equations. Consider now solving a similar system with a tridiagonal co ecient matrix: include <lapack++.h> LaSPDMatDouble AN,N; LaVectorDouble xN, bN; ... LaLinSolveA,x,b; The only co de mo d cation is in the declaration of A. In this case LaLinSolve calls the Cholesky driver routine for solving symmetric, p ositive-de nite linear systems. The LaLinSolve function has b een over- loaded to p erform di erent tasks dep ending on the typ e of the input matrix A. If the matrix typ es are known at compile time, as in this example, then there is no runtime overhead asso ciated with this. 2.3 Basic Linear Algebra Subroutines BLAS++ The Basic Linear Algebra Subprograms BLAS [4] has b een the key to obtaining go o d p erformance on a wide variety of computer architectures. The BLAS de ne a common interface for low-level op erations often found in computational kernels. These op erations, such as matrix/matrix multiply and triangular solves, typically comprise of most of the computatioal workload found in dense and banded linear algebra algorithms. The Level 3 BLAS obtains go o d p erformance on a wide varietyofarchitectures bykeeping data used most often in the closest level of memory hierarhcy registers, cache, etc.. The BLAS++ interface simpli es many of the calling sequences to the traditional f77 BLAS interface,