Sparse storage and solution methods MATH2071: Numerical Methods in Scientific Computing II http://people.sc.fsu.edu/∼jburkardt/classes/math2071 2020/sparse/sparse.pdf Let's just concentrate on the seats that are occupied! Sparse Matrices Given a linear system in which the system matrix has very few nonzero elements, find efficient methods for storing the matrix, and for solving the linear system. 1 Where do sparse matrices come from? So far, the sparse matrices we have seen have had great regularity. Diagonal and tridiagonal matrices can be thought of as sparse, but their structure is so simple that there are many efficient ways to store the values and do linear algebra operations. Matrices associated with a 2D or 3D finite difference scheme over a rectangular region are sparse, and sparse storage can be useful for them; however, these examples are not typical, since almost every row of the matrix has the same number of nonzeros, showing up at predictable locations. A more challenging example of a sparse matrix occurs in finite element methods, where an irregular region is meshed with triangles, or other polygonal shapes. Just as in finite differences, we write an equation at each node of the mesh, and that equation will involve the approximate solution values at the node, and at all the nodes which are the immediate neighbors. Unlike the finite difference example, we generally can't predict in advance the locations of these neighbors, or their number. For example, in a study of the ice sheets of Greenland, a mesh involving 900,000 nodes was created. These nodes were placed in such a way that the mesh was very fine near the coast, and near locations of rapid icesheet movement. It was then necessary to organize these nodes into triangles, using a technique known as Delaunay triangulation and then to build the finite element matrix, in a process similar to what we have done for finite element problems. 1 Greenland, and a small portion of the mesh. Consider the fact that, in the picture of the mesh, each node will generate an equation. Each neighbor node, that is joined by a connecting line, will be associated with a nonzero coefficient in that equation. Thus, each row of our matrix will have an unknown number of nonzeros, at unknown locations, which cannot be predicted in advance, until we analyze the connections in the mesh. 2 Sparse tools in MATLAB Given a matrix A stored in the usual m × n dense format, MATLAB provides us with some functions that can help us if we suspect that the matrix is sparse. Whether a matrix is stored using full or sparse storage, we can count the number of nonzeros and report the relative sparsity by the following commands: 1 count = nnz (A); 2 sparsity = count /m/ n; 3 fprintf ( 1, ' Matrix has %d nonzeros, with sparsity %gnn', count, sparsity ); To extract the values of the nonzeros in the matrix: 1 v = nonzeros (A); To determine the locations and values of the nonzeros: 1 [ i , j , v ] = find (A); To see a picture of the nonzero pattern of the matrix: 1 spy (A) 3 Exercise: Examine a matrix Use the A=dif2 matrix(m,n) function from the web page to generate a dense second difference matrix of dimension 20 × 20. We can use the MATLAB sparse tools to analyze the matrix, extract the nonzeros, and see a picture of the sparsity pattern. Now let's look at a random sparse matrix. Use MATLAB's sprand(m,n,sparsity) function, which returns a matrix in sparse format, with a sparsity percentage of the entries being nonzero. The location and values of these entries are chosen at random. 2 1 m = 1 2 ; 2 n = 9 ; 3 sparsity = 0.15; 4 A= sprand ( m, n, sparsity ); Listing 1: How to define a random sparse matrix. Determine the actual number of nonzeros, the actual sparsity, and extract the rows, columns, and values of the nonzero entries. Use spy(A) to see a picture of this matrix. Copy the file wathen sparse matrix.m, which, for a certain finite element problem, generates the system matrix in MATLAB sparse format. Create the matrix W = wathen sparse matrix(3; 3). Here the 3 × 3 is not the dimension of the grid, but a count of the number of elements used. This matrix will actually be of dimensions 40 × 40. Determine the number of nonzeros, the sparsity, and extract the rows, columns, and values of the nonzero entries for this matrix. Use spy(W) to see a picture of this matrix. 4 Creating a MATLAB sparse matrix MATLAB can work with matrices in full (dense) or sparse format. If we have a matrix in full storage format, we can convert it to sparse format: 1 Asparse = sparse (A); Similarly, if we have a matrix in sparse format, we can convert it to full format: 1 B f u l l = f u l l (B); A matrix is a collection of values Ai;j. We could think of a matrix as a list of its nonzero entries, that is, a table of values i; j; v, where each v is a value Ai;j. This is how MATLAB thinks of a sparse matrix. If we have three such vectors of rows, columns, and values, we can create the corresponding MATLAB sparse matrix: 1 A = sparse ( i , j , v ) Consider the following matrix: 2 11 0 0 14 15 3 6 21 22 0 0 0 7 A = 6 7 4 0 0 33 0 0 5 41 0 0 44 45 We can set up this matrix sparsely as: 1 i = [1,1,1,2,2,3,4,4,4]; 2 j = [1,4,5,1,2,3,1,4,5]; 3 v = [11,14,15,21,22,33,41,44,45]; 4 A = sparse ( i , j , v ) ; Listing 2: Converting i j v to a MATLAB sparse matrix. 3 5 Exercise: Set up a MATLAB sparse -1,2,-1 matrix Consider the following 4 × 4 matrix C: 2 2 −1 0 0 3 6 −1 2 −1 0 7 C = 6 7 4 0 −1 2 −1 5 0 0 −1 2 Use MATLAB commands to define the vectors i, j, v, and create a sparse matrix C. 1. Determine the number of rows and columns in the matrix; 2. Determine the number of nonzero elements in C? 3. What command will return the vectors i, j, v from C? 4. What is the inverse of C? Is it also stored as a sparse matrix? Is it sparse? 5. What happens if you use the [V,L]=eigs(C) command to get eigen information? 6 Reading a MATLAB sparse matrix from a file Our example matrix could be described by the following table: i j v ------ 1 1 11 1 4 14 1 5 15 2 1 21 2 2 22 3 3 33 4 4 44 4 5 45 Suppose we had stored this information in a file: example sparse matrix.txt. We can read this data file into MATLAB using the load() function, which results in an 8 × 3 array, whose columns are the row index, column index, and value of the nonzeros. MATLAB has an spconvert() command that will convert this data to a sparse matrix in one step: 1 data = load ( ' e x a m p l e s p a r s e matrix.txt' ); 2 A = spconvert ( data ) ; Listing 3: Read sparse data from file to create sparse matrix. 7 The Sparse Triplet (ST) storage scheme The sparse triplet storage scheme is extremely simple. We are given an m × n matrix A, which has nst nonzero elements. We can represent the matrix by storing, for each nonzero element Ai;j, the row index, column index, and value, in arrays ist(1:nst), jst(1:nst), vst(1:nst). Of course, the ST scheme only makes sense for large, sparse matrices, for which 3 ∗ nst << m ∗ n. We must also assume that, once we have replaced the dense matrix by the ST version, we can still perform any linear algebra tasks we need. We will see that matrix multiplication can still be done with this representation. However, computing the LU factorization would not be possible, because the LU factors of a sparse matrix are not necessarily sparse at all. 4 As a reminder of how the ST scheme works, consider this 4 × 5 matrix A: 2 11 0 0 14 15 3 6 21 22 0 0 0 7 A = 6 7 4 0 0 33 0 0 5 41 0 0 44 45 for which the nst = 8 nonzero entries could be stored in ST form as: ist jst vst 1 1 11 2 1 21 4 1 41 2 2 22 3 3 33 1 4 14 4 4 44 4 5 45 Consider that a \small" version of our 2d Poisson problem, using a 100 × 100 grid, involves 10,000 variables. The full matrix is therefore 100,000,000 entries. However, typically, there are at most 5 nonzero coefficients in each equation, for a total of nst ≈ 50; 000 nonzero values. Therefore, we could store this matrix using 50,000 real values (vst) and two integer arrays each of length 50,000. Our storage costs have gone down by a factor of roughly 700. 8 Matrix multiplication using ST If we want to solve a big sparse linear system, we are likely to be using an iterative method. Most iterative methods require the ability to compute matrix-vector products like y = A ∗ x or y = A0 ∗ x.