scikit-sparse Documentation Release 0.4.3+9.ge35b764

Antony Lee

Nov 07, 2018

Contents

1 Overview 3 1.1 Introduction...... 3 1.2 Download...... 3 1.3 Requirements...... 3 1.4 Installation...... 4 1.5 Contact...... 4 1.6 Developers...... 4

2 Sparse Cholesky decomposition (sksparse.cholmod)5 2.1 Overview...... 5 2.2 Quickstart...... 5 2.3 Top-level functions...... 6 2.4 Factor objects...... 8 2.5 Convenience methods...... 12 2.6 Error handling...... 12

3 Changes 15 3.1 v0.4.4...... 15 3.2 v0.4.3...... 15 3.3 v0.4.2...... 15 3.4 v0.4.1...... 15 3.5 v0.4...... 16 3.6 v0.3.1...... 16 3.7 v0.3...... 16 3.8 v0.2...... 16 3.9 v0.1...... 16

4 Indices and tables 17

Python Module Index 19

i ii scikit-sparse Documentation, Release 0.4.3+9.ge35b764

Contents:

Contents 1 scikit-sparse Documentation, Release 0.4.3+9.ge35b764

2 Contents CHAPTER 1

Overview

1.1 Introduction

The scikit-sparse package (previously known as scikits.sparse) is a companion to the scipy.sparse library for manipulation in Python. All scikit-sparse routines expect and return scipy.sparse matrices (usually in CSC format). The intent of scikit-sparse is to wrap GPL’ed code such as SuiteSparse, which cannot be included in SciPy proper. Currently our coverage is rather. . . sparse, with only a wrapper for the CHOLMOD routines for sparse Cholesky de- composition, but we hope that this will expand over time. Contributions of new wrappers are very welcome, especially if you can follow the style of the existing interfaces.

1.2 Download

The current release may be downloaded from the Python Package index at https://pypi.python.org/pypi/scikit-sparse/ Or from the homepage at https://github.com/scikit-sparse/scikit-sparse/releases Or the latest development version may be found in our Git repository:

$ git clone git://github.com/scikit-sparse/scikit-sparse.git

1.3 Requirements

Installing scikit-sparse requires: • Python

3 scikit-sparse Documentation, Release 0.4.3+9.ge35b764

• NumPy • SciPy • Cython • CHOLMOD (included in SuiteSparse) Test versions are: * Python: 3.7, 3.6 * NumPy: 1.15, 1.14, 1.13 * SciPy: 1.1, 1.0, 0.19 * SuiteSparse: 5.2 (Other versions may work but are untested.) On Debian/Ubuntu systems, the following command should suffice:

$ sudo apt-get install python-scipy libsuitesparse-dev

On Arch Linux, run:

$ sudo pacman -S suitesparse

1.4 Installation

As usual,

$ pip install --user scikit-sparse or with conda

$ conda install - conda-forge scikit-sparse

1.5 Contact

Post your suggestions and questions directly to our bug tracker.

1.6 Developers

• 2008 David Cournapeau • 2009-2015 Nathaniel Smith • 2010 Dag Sverre Seljebotn • 2014 Leon Barrett • 2015 Yuri • 2016-2017 Antony Lee • 2016 Alex Grigorievskiy • 2016-2018 Joscha Reimer

4 Chapter 1. Overview CHAPTER 2

Sparse Cholesky decomposition (sksparse.cholmod)

New in version 0.1.

2.1 Overview

This module provides efficient implementations of all the basic operations for sparse, symmetric, positive-definite matrices (as, for instance, commonly arise in least squares problems). Specifically, it exposes most of the capabilities of the CHOLMOD package, including: • Computation of the Cholesky decomposition 퐿퐿′ = 퐴 or 퐿퐷퐿′ = 퐴 (with fill-reducing permutation) for both real and complex sparse matrices 퐴, in any format supported by scipy.sparse. (However, CSC matrices will be most efficient.) • A convenient and efficient interface for using this decomposition to solve problems of the form 퐴푥 = 푏. • The ability to perform the costly fill-reduction analysis once, and then re-use it to efficiently decompose many matrices with the same pattern of non-zero entries. • In-place ‘update’ and ‘downdate’ operations, for computing the Cholesky decomposition of a rank-k update of 퐴 and of product 퐴퐴′. So, the result is the Cholesky decomposition of 퐴 + 퐶퐶′ (or 퐴퐴′ + 퐶퐶′). The last case is useful when the columns of A become available incrementally (e.g., due to memory constraints), or when many matrices with similar but non-identical columns must be factored. • Convenience functions for computing the (log) of the matrix that has been factored. • A convenience function for explicitly computing the inverse of the matrix that has been factored (though this is rarely useful).

2.2 Quickstart

If 퐴 is a sparse, symmetric, positive-definite matrix, and 푏 is a matrix or vector (either sparse or dense), then the following code solves the equation 퐴푥 = 푏:

5 scikit-sparse Documentation, Release 0.4.3+9.ge35b764

from sksparse.cholmod import cholesky factor= cholesky(A) x= factor(b)

If we just want to compute its determinant:

factor= cholesky(A) ld= factor.logdet()

(This returns the log of the determinant, rather than the determinant itself, to avoid issues with underflow/overflow. See logdet(), log().) If you have a least-squares problem to solve, minimizing ||푀푥 − 푏||2, and 푀 is a sparse matrix, the solution is 푥 = (푀 ′푀)−1푀 ′푏, which can be efficiently calculated as:

from sksparse.cholmod import cholesky_AAt # Notice that CHOLMOD computes AA' and we want M'M, so we must set A = M'! factor= cholesky_AAt(M.T) x= factor(M.T * b)

However, you should be aware that for least squares problems, the Cholesky method is usually faster but somewhat less numerically stable than QR- or SVD-based techniques.

2.3 Top-level functions

All usage of this module starts by calling one of four functions, all of which return a Factor object, documented below. Most users will want one of the cholesky functions, which perform a fill-reduction analysis and decomposition together: sksparse.cholmod.cholesky(A, beta=0, mode="auto", ordering_method="default", use_long=None) Computes the fill-reducing Cholesky decomposition of

퐴 + 훽퐼

where A is a sparse, symmetric, positive-definite matrix, preferably in CSC format, and beta is any real scalar (usually 0 or 1). (And 퐼 denotes the .) Only the lower triangular part of A is used. mode is passed to analyze(). ordering_method is passed to analyze(). use_long is passed to analyze(). Returns A Factor object represented the decomposition. sksparse.cholmod.cholesky_AAt(A, beta=0, mode="auto", ordering_method="default", use_long=None) Computes the fill-reducing Cholesky decomposition of

퐴퐴′ + 훽퐼

6 Chapter 2. Sparse Cholesky decomposition (sksparse.cholmod) scikit-sparse Documentation, Release 0.4.3+9.ge35b764

where A is a sparse matrix, preferably in CSC format, and beta is any real scalar (usually 0 or 1). (And 퐼 denotes the identity matrix.) Note that if you are solving a conventional least-squares problem, you will need to transpose your matrix before calling this function, and therefore it will be somewhat more efficient to construct your matrix in CSR format (so that its transpose will be in CSC format). mode is passed to analyze_AAt(). ordering_method is passed to analyze_AAt(). use_long is passed to analyze_AAt(). Returns A Factor object represented the decomposition. However, some users may want to break the fill-reduction analysis and actual decomposition into separate steps, and instead begin with one of the analyze functions, which perform only fill-reduction: sksparse.cholmod.analyze(A, mode="auto", ordering_method="default", use_long=None) Computes the optimal fill-reducing permutation for the A, but does not factor it (i.e., it per- forms a “symbolic Cholesky decomposition”). This function ignores the actual contents of the matrix A. All it cares about are (1) which entries are non-zero, and (2) whether A has real or complex type. Parameters • A – The matrix to be analyzed. • mode – Specifies which algorithm should be used to (eventually) compute the Cholesky decomposition – one of “simplicial”, “supernodal”, or “auto”. See the CHOLMOD docu- mentation for details on how “auto” chooses the algorithm to be used. • ordering_method – Specifies which ordering algorithm should be used to (eventually) order the matrix A – one of “natural”, “amd”, “metis”, “nesdis”, “colamd”, “default” and “best”. “natural” means no permutation. See the CHOLMOD documentation for more details. • use_long – Specifies if the long type (64 bit) or the int type (32 bit) should be used for the indices of the sparse matrices. If use_long is None try to estimate if long type is needed. Returns A Factor object representing the analysis. Many operations on this object will fail, be- cause it does not yet hold a full decomposition. Use Factor.cholesky_inplace() (or similar) to actually factor a matrix. sksparse.cholmod.analyze_AAt(A, mode="auto", ordering_method="default", use_long=None) Computes the optimal fill-reducing permutation for the symmetric matrix 퐴퐴′, but does not factor it (i.e., it performs a “symbolic Cholesky decomposition”). This function ignores the actual contents of the matrix A. All it cares about are (1) which entries are non-zero, and (2) whether A has real or complex type. Parameters • A – The matrix to be analyzed. • mode – Specifies which algorithm should be used to (eventually) compute the Cholesky decomposition – one of “simplicial”, “supernodal”, or “auto”. See the CHOLMOD docu- mentation for details on how “auto” chooses the algorithm to be used. • ordering_method – Specifies which ordering algorithm should be used to (eventually) order the matrix A – one of “natural”, “amd”, “metis”, “nesdis”, “colamd”, “default” and “best”. “natural” means no permutation. See the CHOLMOD documentation for more details. • use_long – Specifies if the long type (64 bit) or the int type (32 bit) should be used for the indices of the sparse matrices. If use_long is None try to estimate if long type is needed.

2.3. Top-level functions 7 scikit-sparse Documentation, Release 0.4.3+9.ge35b764

Returns A Factor object representing the analysis. Many operations on this object will fail, be- cause it does not yet hold a full decomposition. Use Factor.cholesky_AAt_inplace() (or similar) to actually factor a matrix.

Note: Even if you used cholesky() or cholesky_AAt(), you can still call cholesky_inplace() or cholesky_AAt_inplace() on the resulting Factor to quickly factor another matrix with the same non-zero pattern as your original matrix.

2.4 Factor objects class sksparse.cholmod.Factor A Factor object represents the Cholesky decomposition of some matrix 퐴 (or 퐴퐴′). Each Factor fixes: • A specific fill-reducing permutation • A choice of which Cholesky algorithm to use (see analyze()) • Whether we are currently working with real numbers or complex Given a Factor object, you can: • Compute new Cholesky decompositions of matrices that have the same pattern of non-zeros • Perform ‘updates’ or ‘downdates’ • Access the various Cholesky factors • Solve equations involving those factors

2.4.1 Factoring new matrices

Factor.cholesky_inplace(A, beta=0) Updates this Factor so that it represents the Cholesky decomposition of 퐴+훽퐼, rather than whatever it contained before. 퐴 must have the same pattern of non-zeros as the matrix used to create this factor originally. Factor.cholesky_AAt_inplace(A, beta=0) The same as cholesky_inplace(), except it factors 퐴퐴′ + 훽퐼 instead of 퐴 + 훽퐼. Factor.cholesky(A, beta=0) The same as cholesky_inplace() except that it first creates a copy of the current Factor and modifes the copy. Returns The new Factor object. Factor.cholesky_AAt(A, beta=0) The same as cholesky_AAt_inplace() except that it first creates a copy of the current Factor and modifes the copy. Returns The new Factor object.

2.4.2 Updating/Downdating

Factor.update_inplace(C, subtract=False) Incremental building of 퐴퐴′ decompositions.

8 Chapter 2. Sparse Cholesky decomposition (sksparse.cholmod) scikit-sparse Documentation, Release 0.4.3+9.ge35b764

Updates this factor so that instead of representing the decomposition of 퐴 (퐴퐴′), it computes the decomposi- tion of 퐴 + 퐶퐶′ (퐴퐴′ + 퐶퐶′) for subtract=False which is the default, or 퐴 − 퐶퐶′ (퐴퐴′ − 퐶퐶′) for subtract=True. This method does not require that the Factor was created with cholesky_AAt(), though that is the common case. The usual use for this is to factor AA’ when A has a large number of columns, or those columns become available incrementally. Instead of loading all of A into memory, one can load in ‘strips’ of columns and pass them to this method one at a time. Note that no fill-reduction analysis is done; whatever permutation was chosen by the initial call to analyze() will be used regardless of the pattern of non-zeros in C.

2.4.3 Accessing Cholesky factors explicitly

Note: When possible, it is generally more efficient to use the solve_... functions documented below rather than extracting the Cholesky factors explicitly.

Factor.P() Returns the fill-reducing permutation P, as a vector of indices. The decomposition 퐿퐿′ or 퐿퐷퐿′ is of:

A[P[:, np.newaxis], P[np.newaxis, :]]

(or similar for AA’). Factor.D() Converts this factorization to the style

퐿퐷퐿′ = 푃 퐴푃 ′

or

퐿퐷퐿′ = 푃 퐴퐴′푃 ′

and then returns the D as a 1d vector.

Note: This method uses an efficient implementation that extracts the diagonal D directly from CHOLMOD’s internal representation. It never makes a copy of the factor matrices, or actually con- verts a full LL’ factorization into an LDL’ factorization just to extract D.

Factor.L() If necessary, converts this factorization to the style

퐿퐿′ = 푃 퐴푃 ′

or

2.4. Factor objects 9 scikit-sparse Documentation, Release 0.4.3+9.ge35b764

퐿퐿′ = 푃 퐴퐴′푃 ′

and then returns the sparse lower- L.

Warning: The L matrix returned by this method and the one returned by L_D() are different!

Factor.LD() If necessary, converts this factorization to the style

퐿퐷퐿′ = 푃 퐴푃 ′

or

퐿퐷퐿′ = 푃 퐴퐴′푃 ′

and then returns a sparse lower-triangular matrix “LD”, which contains the D matrix on its diagonal, plus the below-diagonal part of L (the actual diagonal of L is all-ones). See L_D() for a more convenient interface. Factor.L_D() If necessary, converts this factorization to the style

퐿퐷퐿′ = 푃 퐴푃 ′

or

퐿퐷퐿′ = 푃 퐴퐴′푃 ′

and then returns the pair (L, D) where L is a sparse lower-triangular matrix and D is a sparse diagonal matrix.

Warning: The L matrix returned by this method and the one returned by L() are different!

2.4.4 Solving equations

All methods in this section accept both sparse and dense matrices (or vectors) b, and return either a sparse or dense x accordingly. All methods in this section act on 퐿퐷퐿′ factorizations by default. Thus L refers by default to the matrix returned by L_D(), not that returned by L() (though conversion is not performed unless necessary). Factor.solve_A(b) Solves a linear system.

10 Chapter 2. Sparse Cholesky decomposition (sksparse.cholmod) scikit-sparse Documentation, Release 0.4.3+9.ge35b764

Parameters b – right-hand-side Returns math:x, where 퐴푥 = 푏 (or 퐴퐴′푥 = 푏, if you used cholesky_AAt()). __call__() is an alias for this function, i.e., you can simply call the Factor object like a function to solve 퐴푥 = 푏. Factor.__call__(b) Alias for solve_A(). Factor.solve_LDLt(b) Solves a linear system. Parameters b – right-hand-side Returns math:x, where 퐿퐷퐿′푥 = 푏. (This is different from solve_A() because it does not correct for the fill-reducing permutation.) Factor.solve_LD(b) Solves a linear system. Parameters b – right-hand-side Returns math:x, where 퐿퐷푥 = 푏. Factor.solve_DLt(b) Solves a linear system. Parameters b – right-hand-side Returns math:x, where 퐷퐿′푥 = 푏. Factor.solve_L(b) Solves a linear system. Parameters • b – right-hand-side • use_LDLt_decomposition – If True, use the L of the LDL’ decomposition. If False, use the L of the LL’ decomposition. Returns math:x, where 퐿푥 = 푏. Factor.solve_Lt(b) Solves a linear system. Parameters • b – right-hand-side • use_LDLt_decomposition – If True, use the L of the LDL’ decomposition. If False, use the L of the LL’ decomposition. Returns math:x, where 퐿′푥 = 푏. Factor.solve_D(b) Returns 푥, where 퐷푥 = 푏. Factor.apply_P(b) Returns 푥, where 푥 = 푃 푏. Factor.apply_Pt(b) Returns 푥, where 푥 = 푃 ′푏.

2.4. Factor objects 11 scikit-sparse Documentation, Release 0.4.3+9.ge35b764

2.5 Convenience methods

Factor.logdet() Computes the (natural) log of the determinant of the matrix A. If f is a factor, then f.logdet() is equivalent to np.sum(np.log(f.D())). New in version 0.2. Factor.det() Computes the determinant of the matrix A. Consider using logdet() instead, for improved . (In particular, are often prone to problems with underflow or overflow.) New in version 0.2. Factor.slogdet() Computes the log-determinant of the matrix A, with the same API as numpy.linalg.slogdet(). This returns a tuple (sign, logdet), where sign is always the number 1.0 (because the determinant of a positive- definite matrix is always a positive real number), and logdet is the (natural) logarithm of the determinant of the matrix A. New in version 0.2. Factor.inv() Returns the inverse of the matrix A, as a sparse (CSC) matrix.

Warning: For most purposes, it is better to use solve() instead of computing the inverse explicitly. That is, the following two pieces of code produce identical results: x=f.solve(b) x=f.inv() * b # DON'T DO THIS! But the first line is both faster and produces more accurate results.

Sometimes, though, you really do need the inverse explicitly (e.g., for calculating standard errors in least squares regression), so if that’s your situation, here you go. New in version 0.2. Factor.copy() Copies the current Factor. Returns A new Factor object.

2.6 Error handling class sksparse.cholmod.CholmodError class sksparse.cholmod.CholmodNotPositiveDefiniteError class sksparse.cholmod.CholmodNotInstalledError class sksparse.cholmod.CholmodOutOfMemoryError class sksparse.cholmod.CholmodTooLargeError class sksparse.cholmod.CholmodNotPositiveDefiniteError

12 Chapter 2. Sparse Cholesky decomposition (sksparse.cholmod) scikit-sparse Documentation, Release 0.4.3+9.ge35b764

class sksparse.cholmod.CholmodInvalidError class sksparse.cholmod.CholmodGpuProblemError Errors detected by CHOLMOD or by our wrapper code are converted into exceptions of type CholmodError or an appropriated subclass. class sksparse.cholmod.CholmodWarning Warnings issued by CHOLMOD are converted into Python warnings of type CholmodWarning. class sksparse.cholmod.CholmodTypeConversionWarning CHOLMOD itself supports matrices in CSC form with 32-bit integer indices and ‘double’ precision floats (64-bits, or 128-bits total for complex numbers). If you pass some other sort of matrix, then the wrapper code will convert it for you before passing it to CHOLMOD, and issue a warning of type CholmodTypeConversionWarning to let you know that your efficiency is not as high as it might be.

Warning: Not all conversions currently produce warnings. This is a bug.

Child of CholmodWarning.

2.6. Error handling 13 scikit-sparse Documentation, Release 0.4.3+9.ge35b764

14 Chapter 2. Sparse Cholesky decomposition (sksparse.cholmod) CHAPTER 3

Changes

3.1 v0.4.4

• Bug in solve with dense array, where base of result is not set correctly, fixed. • Travis tests are using conda now. • Supported versions updated to: - Python: 3.7, 3.6 - NumPy: 1.15, 1.14, 1.13 - SciPy: 1.1, 1.0, 0.19 - SuiteSparse: 5.2

3.2 v0.4.3

• The method solve_L can now also use the L matrix of the LL’ decomposition. • Supported versions updated to: - Python: 3.6, 3.5 - NumPy: 1.14, 1.13 - SciPy: 1.0, 0.19

3.3 v0.4.2

• Bug where the ordering method is not taken into account is fixed. • The Factor class has now a (public) copy method.

3.4 v0.4.1

• Bug with relaxed stride checking in NumPy 1.12 fixed. • Supported versions updated to: - Python: 3.6, 3.5, 3.4, 2.7 - NumPy: 1.8 to 1.12

15 scikit-sparse Documentation, Release 0.4.3+9.ge35b764

3.5 v0.4

• 64-bit indices (type long) are now supported. • The ordering method for Cholesky decomposition is now choosable. • Specific exceptions subclasses are now thrown for each error condition. • Setup does not rely on an installed Cython anymore.

3.6 v0.3.1

• Ensure that arrays returned by the Factor.solve_...() methods are writeable.

3.7 v0.3

• Dropped deprecated Factor.solve_P() and Factor.solve_P(). • Fixed a memory leak upon garbage collection of Factor.

3.8 v0.2

• Factor solve methods now return 1d output for 1d input (just like np.dot does). • Factor.solve_P() and Factor.solve_P() deprecated; use Factor.apply_P() and Factor. apply_Pt() instead. • New methods for computing determinants of positive-definite matrices: Factor.det(), Factor. logdet(), Factor.slogdet(). • New method for explicitly computing inverse of a positive-definite matrix: Factor.inv(). • Factor.D() has much better implementation. • Build system improvements. • Wrapper code re-licensed under BSD terms.

3.9 v0.1

First public release.

16 Chapter 3. Changes CHAPTER 4

Indices and tables

• genindex • modindex • search

17 scikit-sparse Documentation, Release 0.4.3+9.ge35b764

18 Chapter 4. Indices and tables Python Module Index

s sksparse.cholmod,5

19 scikit-sparse Documentation, Release 0.4.3+9.ge35b764

20 Python Module Index Index

Symbols I __call__() (sksparse.cholmod.Factor method), 11 inv() (sksparse.cholmod.Factor method), 12 A L analyze() (in module sksparse.cholmod),7 L() (sksparse.cholmod.Factor method),9 analyze_AAt() (in module sksparse.cholmod),7 L_D() (sksparse.cholmod.Factor method), 10 apply_P() (sksparse.cholmod.Factor method), 11 LD() (sksparse.cholmod.Factor method), 10 apply_Pt() (sksparse.cholmod.Factor method), 11 logdet() (sksparse.cholmod.Factor method), 12 C P cholesky() (in module sksparse.cholmod),6 P() (sksparse.cholmod.Factor method),9 cholesky() (sksparse.cholmod.Factor method),8 cholesky_AAt() (in module sksparse.cholmod),6 S cholesky_AAt() (sksparse.cholmod.Factor method),8 sksparse.cholmod (module),5, 15 cholesky_AAt_inplace() (sksparse.cholmod.Factor slogdet() (sksparse.cholmod.Factor method), 12 method),8 solve_A() (sksparse.cholmod.Factor method), 10 cholesky_inplace() (sksparse.cholmod.Factor method),8 solve_D() (sksparse.cholmod.Factor method), 11 CholmodError (class in sksparse.cholmod), 12 solve_DLt() (sksparse.cholmod.Factor method), 11 CholmodGpuProblemError (class in sksparse.cholmod), solve_L() (sksparse.cholmod.Factor method), 11 13 solve_LD() (sksparse.cholmod.Factor method), 11 CholmodInvalidError (class in sksparse.cholmod), 12 solve_LDLt() (sksparse.cholmod.Factor method), 11 CholmodNotInstalledError (class in sksparse.cholmod), solve_Lt() (sksparse.cholmod.Factor method), 11 12 CholmodNotPositiveDefiniteError (class in U sksparse.cholmod), 12 update_inplace() (sksparse.cholmod.Factor method),8 CholmodOutOfMemoryError (class in sksparse.cholmod), 12 CholmodTooLargeError (class in sksparse.cholmod), 12 CholmodTypeConversionWarning (class in sksparse.cholmod), 13 CholmodWarning (class in sksparse.cholmod), 13 copy() (sksparse.cholmod.Factor method), 12 D D() (sksparse.cholmod.Factor method),9 det() (sksparse.cholmod.Factor method), 12 F Factor (class in sksparse.cholmod),8

21