Preconditioning techniques based on the Birkhoff–von Neumann decomposition Michele Benzi∗ Alex Potheny Bora U¸carz Abstract same property. These motivations are shared by recent We introduce a class of preconditioners for general sparse work on ILU preconditioners, where their fine-grained matrices based on the Birkhoff–von Neumann decomposition computation [8] and approximate application [1] are in- of doubly stochastic matrices. These preconditioners are vestigated for GPU-like systems. aimed primarily at solving challenging linear systems with The paper is organized as follows. We first give highly unstructured and indefinite coefficient matrices. We necessary background (Section 2) on doubly stochastic present some theoretical results and numerical experiments matrices and the BvN decomposition. We then develop on linear systems from a variety of applications. splittings for doubly stochastic matrices (Section 3), where we analyze convergence properties and discuss 1 Introduction algorithms to construct the preconditioners. Later in the same Section, we discuss how the preconditioners We consider the solution of linear systems Ax = b where can be used for arbitrary matrices by some preprocess- A = [a ] 2 n×n, b is a given vector and x is the ij R ing. Here our approach results in a generalization of the unknown vector. Our aim is to develop and investigate Birkhoff-von Neumann decomposition for matrices with preconditioners for Krylov subspace methods for solving positive and negative entries where the sum of the ab- such linear systems, where A is highly unstructured and solute values of the entries in any given row or column indefinite. is one. This generalization could be of interest in other For a given matrix A, we first preprocess it to get a areas. Then, we give experimental results (Section 4) doubly stochastic matrix (whose row and column sums with nonnegative and also arbitrary matrices, and then are one). Then using this doubly stochastic matrix, we conclude the paper. select some fraction of some of the nonzeros of A to be included in the preconditioner. Our main tools are 2 Background and definitions the well-known Birkhoff-von Neumann (BvN) decompo- sition (this will be discussed in Section 2 for complete- Here we define several properties of matrices: irre- ness), and a splitting of the input matrix in the form ducible, full indecomposable, and doubly stochastic ma- A = M − N based on its BvN decomposition. When trices. such a splitting is defined, M −1 or solvers for My = z An n × n matrix A is reducible if there exists a are required. We discuss sufficient conditions when such permutation matrix P such that a splitting is convergent and discuss specialized solvers A A for My = z when these conditions are met. In case the P AP T = 1;1 1;2 ; OA conditions become restrictive in practice, we build pre- 2;2 conditioners based on this splitting by using the LU de- where A is an r × r submatrix, A is an (n − r) × composition of M. Our motivation is that that the pre- 1;1 2;2 −1 (n−r) submatrix, and 1 ≤ r < n. If such a permutation conditioner M can be applied to vectors by a number matrix does not exist, then A is irreducible [21, Ch. 1]. of highly concurrent steps, where the number of steps When A is reducible, either A1;1 or A2;2 can be reducible is controlled by the user. Therefore, the precondition- as well, and we can recursively identify their diagonal ers (or the splittings) can be advantageous for use in blocks, until all diagonal blocks are irreducible. That many-core computing systems. In the context of split- is, we can obtain tings, the application of N to vectors can also enjoy the 2 3 ∗Department of Mathematics and Computer Science, Emory A1;1 A1;2 ··· A1;s University, Atlanta, USA ([email protected]). 6 0 A2;2 ··· A2;s7 yDepartment of Computer Science, Purdue University, West (2.1) P AP T = 6 7 ; 6 . .. 7 Lafayette, USA ([email protected]). 4 . 5 z CNRS and LIP, ENS Lyon, France ([email protected]) 0 0 ··· As;s where each Ai;i is square and irreducible. This block where upper triangular form, with square irreducible diagonal blocks is called Frobenius normal form [15, p. 532]. (3.4) M = α1P1 + ··· + αrPr; An n × n matrix A is fully indecomposable if there N = −αr+1Pr+1 − · · · − αkPk: exists a permutation matrix Q such that AQ has a zero- free diagonal and is irreducible [6, Chs. 3 and 4]. If A Note that M and −N are doubly substochastic matri- is not fully indecomposable, but nonsingular, it can be ces. permuted into the block upper triangular form Definition 1. A splitting of the form (3.3) with M and A A P AQT = 1;1 1;2 ; N given by (3.4) is said to be a doubly substochastic OA2;2 splitting. where each Ai;i is fully indecomposable or can be further Definition 2. A doubly substochastic splitting A = permuted into the block upper triangular form. If M − N of a doubly stochastic matrix A is said to be the coefficient matrix of a linear system is not fully standard if M is invertible. We will call such a splitting indecomposable, the block upper triangular form should an SDS splitting. be obtained, and only the small systems with the diagonal blocks should be factored for simplicity and In general, it is not easy to guarantee that a given efficiency [9, Ch. 6]. We therefore assume without loss doubly substochastic splitting is standard, except for of generality that matrix A is fully indecomposable. some trivial situation such as the case r = 1, in An n × n matrix A is doubly stochastic if aij ≥ 0 which case M is always invertible. We also have a for all i; j and Ae = AT e = e, where e is the vector characterization for invertible M when r = 2. of all ones. This means that the row sums and col- umn sums are equal to one. If these sums are less Theorem 3.1. Let M = α1P1 + α2P2. Then, M is than one, then the matrix A is doubly substochastic. invertible if (i) α1 6= α2, or (ii) α1 = α2 and all the A doubly stochastic matrix is fully indecomposable or fully indecomposable blocks of M have an odd number is block diagonal where each block is fully indecompos- of rows (and columns). If any such block is of even able. By Birkhoff's Theorem [3], there exist coefficients order, M is singular. α ; α ; : : : ; α 2 (0; 1) with Pk α = 1, and permuta- 1 2 k i=1 i Proof. We investigate the two cases separately. tion matrices P ;P ;:::;P such that 1 2 k Case (i): Without loss of generality assume that (2.2) A = α1P1 + α2P2 + ··· + αkPk : α1 > α2. We have T Such a representation of A as a convex combination M = α1P1 + α2P2 = P1(α1I + α2P1 P2): of permutation matrices is known as a Birkhoff–von T Neumann decomposition (BvN); in general, it is not The matrix α1I + α2P1 P2 is nonsingular. Indeed, its unique. The Marcus{Ree Theorem states that there eigenvalues are of the form α1 + α2λj, where λj is the are BvN decompositions with k ≤ n2 − 2n + 2 for generic eigenvalue of the (orthogonal, doubly stochastic) T dense matrices; Brualdi and Gibson [5] and Brualdi [4] matrix P1 P2, and since jλjj = 1 for all j and α1 > α2, show that for a fully indecomposable sparse matrix it follows that α1 + α2λj 6= 0 for all j. Thus, M is with τ nonzeros, we have BvN decompositions with invertible. k ≤ τ − 2n + 2. Case (ii): This is a consequence of the Perron{ An n × n nonnegative, fully indecomposable matrix Frobenius Theorem. To see this, observe that we A can be uniquely scaled with two positive diagonal ma- need to show that under the stated conditions the sum T trices R and C such that RAC is doubly stochastic [20]. P1 + P2 = P1(I + P1 P2) is invertible, i.e., λ = −1 T cannot be an eigenvalue of P1 P2. Since both P1 and P2 T 3 Splittings of doubly stochastic matrices are permutation matrices, P1 P2 is also a permutation T T n matrix and the Frobenius normal form T = Π(P1 P2)Π 3.1 Definition and properties. Let b 2 R be T given and consider solving the linear system Ax = b of P1 P2, i.e., the block triangular matrix where A is doubly stochastic. Hereafter we assume that 2 3 A is invertible. After finding a representation of A in T1;1 T1;2 ··· T1;s the form (2.2), pick an integer r between 1 and k − 1 6 0 T2;2 ··· T2;s7 and split A as (3.5) T = 6 7 ; 6 . .. 7 4 . 5 (3.3) A = M − N; 0 0 ··· Ts;s T 0 has Ti;j = 0 for i 6= j. The eigenvalues of P1 P2 are just any x if and only if ρ(H) < 1. Hence, we are interested the eigenvalues of the diagonal blocks Ti;i of T . Note in conditions that guarantee that the spectral radius of that there may be only one such block, corresponding the iteration matrix T to the case where P P2 is irreducible. Each diagonal 1 −1 block Ti;i is also a permutation matrix. Thus, each Ti;i H = M N = −1 is doubly stochastic, orthogonal, and irreducible. Any − (α1P1 + ··· + αrPr) (αr+1Pr+1 + ··· + αkPk) matrix of this kind corresponds to a cyclic permutation and has its eigenvalues on the unit circle.