Introduction to Sparse Matrices In Scilab Michael Baudin November 2011 R´esum´e The goal of this document is to present the management of sparse matrices in Scilab. We present the basic features of Scilab, which can create sparse matrices and can convert from and to dense matrices. We show how to solve sparse linear equations in Scilab, by using sparse LU decomposition and iterative methods. We present the sparse Cho- lesky decomposition. We present the functions from the UMFPACK and TAUCS modules. We briefly present the internal sparse API. We introduce to the Arnoldi package. We present the Matrix Market tool- box, which reads and writes sparse matrix files. We analyze how to solve the Poisson Partial Differential Equation with sparse matrices. We present the Imsls toolbox, a set of iterative solvers for sparse linear systems of equations. Table des mati`eres 1 Introduction4 1.1 Note on this document......................4 1.2 Overview.............................4 2 Basic features5 2.1 Creating sparse matrices.....................5 2.2 Storage format..........................8 2.3 Sparse arithmetic.........................9 1 3 Solving sparse linear equations9 3.1 Sparse LU decomposition.....................9 3.2 Sparse backslash......................... 12 3.3 Iterative methods for sparse linear equations.......... 13 4 Cholesky factorizations 14 4.1 The chfact and chsolve functions............... 15 4.2 The spchol function....................... 16 4.3 Conclusion............................. 18 5 Functions from the UMFPACK module 18 6 The UMFPACK package 19 7 The TAUCS package 21 8 The API 22 9 The ARnoldi PACKage 25 10 The Matrix Market module 26 11 Solving Poisson PDE with Sparse Matrices 29 11.1 Introduction............................ 29 11.2 The sparse matrix......................... 31 11.3 The Poisson Solver........................ 32 11.4 The benchmark.......................... 34 11.5 Conclusion............................. 37 12 The Imsls toolbox 37 Bibliography 39 2 Copyright c 2008-2011 - Consortium Scilab - Digiteo - Michael Baudin This file must be used under the terms of the Creative Commons Attribution- ShareAlike 3.0 Unported License : http://creativecommons.org/licenses/by-sa/3.0 3 1 Introduction In numerical analysis, a sparse matrix is a matrix populated primarily with zeros[13]. Huge sparse matrices often appear in science or engineering when solving partial differential equations. Scilab provides several features to manage sparse matrices and perform usual linear algebra operations on them. These operations includes all the basic linear algebra including addition, dot product, transpose and the matrix vector product. For these basic operations, we do not make any difference in a script managing a dense matrix and a script managing a sparse matrix. 1.1 Note on this document This document is an open-source project. The LATEX sources are available on the Scilab Forge : http://forge.scilab.org/index.php/p/docscisparse/ The LATEX sources are provided under the terms of the Creative Commons Attribution-ShareAlike 3.0 Unported License : http://creativecommons.org/licenses/by-sa/3.0 The Scilab scripts are provided on the Forge, inside the project, under the scripts sub-directory. The scripts are available under the CeCiLL licence : http://www.cecill.info/licences/Licence_CeCILL_V2-en.txt 1.2 Overview In this section, we make an overview of the features provided by Scilab for sparse matrices. The current set of tools available in Scilab for sparse matrices are the following : { management of sparse matrices of arbitrary size, { basic algebra on sparse matrices, including, sum, dot product, trans- pose, matrix-matrix product, { sparse LU decomposition and resolution of linear equations from this decomposition based on the Sparse package written by Kenneth S. Kun- dert and Alberto Sangiovanni-Vincentelli, 4 { sparse Cholesky decomposition and resolution of linear equations from this decomposition, based on a sparse Cholesky factorization package developed by Esmond Ng and Barry Peyton at ORNL and a multiple minimun-degree ordering package by Joseph Liu at University of Wa- terloo, { iterative methods of linear systems of equations, including Precondi- tionned Conjugate Gradient (pcg), Generalized Minimum Residual me- thod (gmres), Quasi Minimal Residual method with preconditionning (qmr), { sparse LU decomposition and resolution of linear equations from this decomposition based on the UMFPACK library, { sparse Cholesky decomposition and resolution of linear equations from this decomposition based on the TAUCS library, { sparse eigenvalue computations, based on the Arpack library, using the Implicitly Restarted Arnoldi Method. Scilab manages several file formats read and write sparse matrices. { Scilab is able to read matrices in the Harwell-Boeing format, thanks to the ReadHBSparse function of the Umfpack module. { The Matrix Market external module (available in ATOMS) provides functions to read and write matrices in the Matrix Market format. 2 Basic features In this section, we present the basic sparse matrix features. In the first part, we review how to create a sparse matrix. Then we analyze how a sparse matrix is stored internally. 2.1 Creating sparse matrices The figure1 present several functions to create sparse matrices. In the following session, we use the sprand function to create a 100×1000 sparse matrix with density 0.001. The density parameter makes so that the number of non-zero entries in the sparse matrix is approximately equal to 100 · 1000 · 0:001 = 100. Then we use the size function to compute the size of the matrix. Finally, we compute the number of non-zero entries with the nnz function. -->A=sprand(100,1000,0.001); 5 sp2adj converts sparse matrix into adjacency form speye sparse identity matrix spones sparse matrix sprand sparse random matrix spzeros sparse zero matrix full sparse to full matrix conversion sparse sparse matrix definition mtlb sparse convert sparse matrix nnz number of non zero entries in a matrix spcompack converts a compressed adjacency representation spget retrieves entries of sparse matrix Figure 1 { Basic features for sparse matrices. -->size (A) ans = 100. 1000. -->nnz (A) ans = 100. The sparse and full functions works as complementary functions. In- deed, the sparse function converts a full matrix into a sparse one, while the full function converts a sparse matrix into a full one. In the following session, we create a 3 × 5 dense matrix. Then we use the sparse function to convert it into a sparse matrix. Scilab then displays all the non-zero entries of the matrix one at a time. Finally, we use the full function to convert the sparse matrix into a dense one. -->A = [ -->1 2 0 0 0 -->3 4 5 0 0 -->0 6 7 8 0 -->] A = 1. 2. 0. 0. 0. 3. 4. 5. 0. 0. 0. 6. 7. 8. 0. -->B = sparse(A) B = ( 3, 5) sparse matrix 6 ( 1, 1) 1. ( 1, 2) 2. ( 2, 1) 3. ( 2, 2) 4. ( 2, 3) 5. ( 3, 2) 6. ( 3, 3) 7. ( 3, 4) 8. -->C = full(B) C = 1. 2. 0. 0. 0. 3. 4. 5. 0. 0. 0. 6. 7. 8. 0. Sparse matrices can be real or complex. In the following session, we define a 3-by-5 complex matrix of doubles. -->A = [ -->1 2 0 0 0 -->3 4 5 0 0 -->0 6 7 8 0 -->]; -->B = [ -->9 10 0 0 0 -->11 12 13 0 0 -->0 14 15 16 0 -->]; -->C=complex(A,B) C = 1.+9.i 2.+10.i 0 0 0 3.+11.i 4.+12.i 5.+13.i 0 0 0 6.+14.i 7.+15.i 8.+16.i 0 -->D=sparse(C) D = ( 3, 5) sparse matrix ( 1, 1) 1.+9.i ( 1, 2) 2.+10.i ( 2, 1) 3.+11.i ( 2, 2) 4.+12.i ( 2, 3) 5.+13.i ( 3, 2) 6.+14.i ( 3, 3) 7.+15.i ( 3, 4) 8.+16.i 7 2.2 Storage format There are various ways of storing the nonzero entries of a sparse matrix. In this section, we review how sparse matrices are stored internally, at the library level. As we are going to see, the format used in Scilab is very similar (but not exactly identical) to the compressed sparse row format (CSR), in which the nonzero entries are stored row-by-row. While this detail does not change the way a user user sparse matrices at the interpreter level, it does have an impact on the way the libraries are connected to Scilab. Hence, this is an important feature of sparse matrices in Scilab and it is worthwhile to know precisely what is stored. In Scilab, a sparse matrix is defined in the modules/core/includes/s- parse.h. This is a C struct with type scisparse and the following fields : { int m : the number of rows. { int n : the number of columns. { int it : is equal to 0 if the matrix is real, and 1 if the matrix is complex. { int nel : number of nonzero elements */ { int *mnel : an array with m entries. The entry mnel[i] is the number of non nul elements of the i-th row. { int *icol : an array with nel entries. The entry icol[j] is the column of the j-th nonzero element. { double *R : an array with nel entries. The entry R[j] is the real part of the j-th nonzero element. { double *I : an array with nel entries. The entry I[j] is the imaginary part of the j-th nonzero element. Consider the sparse matrix in the following example. -->A = [ -->1 2 0 -->3 4 5 -->0 6 7 -->]; -->B = sparse(A); In this case, the fields are the following. B.m: 3 B.n: 3 B.it: 0 B. nel : 7 B.mnel: [2, 3, 2] 8 B.icol: [1, 2, 1, 2, 3, 2, 3] B.R: [1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0] B.I: [undefined] This current storage is different from the CSR format, in the sense that there is no array containing the row indices corresponding to the values and no array containing the cumulated indexes where each column starts.