Pivot-Free Block Matrix Inversion Stephen M. Watt Ontario Research Centre for Computer Algebra Department of Computer Science University of Western Ontario London Ontario, CANADA N6A 5B7 [email protected] Abstract Second, we wish to use block matrices as one representa- tion in the mathematical libraries for Aldor [2]. Aldor sup- We present a pivot-free deterministic algorithm for the ports generic programming through parametric polymor- inversion of block matrices. The method is based on the phism, where types must satisfy specific type categories. Moore-Penrose inverse and is applicable over certain gen- For matrices to satisfy the type category Ring they must eral classes of rings. This improves on previous methods provide a partial inverse operation. that required at least one invertible on-diagonal block, and Third, we wish to use block matrices as a test case in that otherwise required row- or column-based pivoting, dis- our work benchmarking the performance of compilers on rupting the block structure. Our method is applicable to any generic programs [3]. invertible matrix and does not require any particular blocks We consider the inversion of 2×2 block matrices. Larger to invertible. This is achieved at the cost of two additional matrices are expressed by applying this construction recur- specialized matrix multiplications and, in some cases, re- sively. Earlier methods for block matrix inversion either quires the inversion to be performed in an extended ring. require one of the blocks to be invertible or degenerate into individual row or column operations, destroying the block 1. Introduction structure. We ask whether it is possible to formulate deterministic block matrix inversion in such a way that Algorithms for block matrices have been well studied and have many benefits under certain conditions: They may 1. only operations on entire blocks are used, be used to improve memory hierarchy performance, at the level of cache, disk or network. They may be used on a 2. no case-based branching is required, quad-tree representation that allows reasonable memory use for dense, sparse or structured matrices. They allow the 3. block inverses are required only when they are guaran- generic formulation of many algorithms of linear algebra so teed to exist and that they may be applied over non-commutative domains. 4. applied recursively, the method gives inversion the This allows some some algorithms to be expressed in a same complexity as matrix multiplication? recursive formulation, leading to improved computational complexity. Block matrices have been used in numerical As with other block-oriented methods, we do not require computing, and also in symbolic computing [1]. numerical stability. This paper revisits the question of block algorithms for We are able to answer this question positively. We show matrix inversion. We are interested in this problem for sev- such a block matrix inversion that is applicable over a gen- eral reasons: eral class of rings. In particular, it may be used for block First, it is an interesting mathematical problem. While matrices with real, complex or finite field entries. many problems that are na ıvely expressed using matrix in- The rest of the paper is organized as follows: Section 2 verse (such as the solution of linear systems) are better provides some necessary background. Section 3 presents solved using other methods (such as PLU factorization), our method for block matrix inverse. The algorithm is pre- matrix inverse nevertheless remains a fundamental opera- sented first for formally real rings, then for the complex and tion. general field cases. Finally we present our conclusions. 2. Preliminaries may be invertible even if all of A, B, C and D are non- invertible. Second, this expression for the matrix inverse re- 2.1 Recursive Block Matrices quires 8 block inversions and 8 block multiplications. When applied recursively to give an algorithm for n × n matrices, 3 We consider n × n matrices with elements from a ring this leads to O(n ) coefficient ring operations. Since faster R and denote the matrix ring Rn×n. For convenience we methods are well known and practical, this formulation is require that R have unity and explicitly state any additional not attractive for a generic algorithm. properties when they are required. If n is even, we may put To avoid these problems, a more common formulation the matrices of Rn×n in a one-to-one correspondence with for block matrix inversion is as n/2×n/2 2×2 the matrices of (R ) . For an n × n matrix M we I −A−1B A−1 0 I 0 M −1 = take 0 I 0 S−1 −CA−1 I AB A M = CD " # A−1 + A−1BS−1CA−1 − A−1BS−1 n n = A A (1) with the 2 × 2 matrices A, B, C, D having elements Aij = −1 −1 −1 −SA CA SA M , B = M n , C = M n , D = M n n i,j ij i,j+ 2 ij i+ 2 ,j ij i+ 2 ,j+ 2 −1 with 1 ≤ i, j, ≤ n/2. For general n, there are two ways to where SA = D − CA B is the Schur complement of A in impose a 2×2 block structure that preserves the multiplica- M. Alternatively, the inverse may be expressed as tive properties of the matrix ring: −1 −1 n×n −1 I 0 SD 0 I −BD The first method is to embed R in a ring of larger M = −1 −1 matrices R(n+`)×(n+`) by adding ones along the diagonal −D CI 0 D 0 I " # and zeros elsewhere. If n is odd and ` is one, then we may −1 −1 −1 dlog ne S −S BD apply the 2×2 block division once. If ` = 2 2 −n (i.e. = D D . −1 −1 −1 −1 −1 −1 (2) if n+` is the next power of 2), then the construction may be −D CSD D + D CSD BD dlog ne 2×2 applied 2 times to obtain a fully recursive struc- This formulation requires only that either A or D and the ture. This method has the advantage that, at each level, the corresponding Schur complement be invertible. It also has matrix elements are from a specific ring. For the recursive the benefit of computational efficiency, requiring only 2 (2×2)k block matrix ring, we introduce the notation R . This inversions, 6 multiplications and 3 additive operations on allows certain isomorphisms to be expressed conveniently blocks. If 2 ≤ ω ≤ 3 is the exponent such that n × n matrix multiplication requires O(nω) element operations, 2k×2k ∼ 2k−1×2k−1 2×2 ∼ (2×2)k−i (2×2)i ∼ (2×2)k R =(R ) =(R ) =R . then this formulation of block matrix inverse, applied re- cursively, leads to an O(nω) algorithm for inversion. For The second method is to divide matrices unevenly. concreteness, Strassen’s matrix multiplication [4] requires The blocks A, B, C, D have elements A = M , ij i,j nlog2 7 multiplications and 6(nlog2 7 − n2) additive opera- Bij0 = Mi,j0+`, Ci0j = Mi0+`,j, Di0j0 = Mi0+`,j0+`, tions (+, −), and the above formulas give matrix inversion where ` is between 1 and n and the indices range as in (6nlog2 7 − 1)/5 multiplicative operations (×, inverse) 1 ≤ i, j ≤ ` and 1 ≤ i0, j0 ≤ n − `. This construction and (72nlog2 7 − 165n2 + 93n)/10 additive operations. may be applied repeatedly to obtain a fully recursive 2 × 2 block structure. For the complexity results given below, we n 2.3 Non-Invertible Blocks choose ` = b 2 c. Both methods may be used for the block matrix inversion While the complexity of the block matrix inversion given problem considered in this paper. Note that in both methods by equations (1) and (2) is acceptable, there remains the the blocks A and D are square. problem that it requires either A or D to be invertible. How- ever, as stated earlier, all blocks may be non-invertible, even 2.2. Na¨ıve Block Matrix Inverse for invertible M. For example, the following permutation matrix is invertible: 2 3 Perhaps the most æsthetically pleasing formulation of 1 0 0 0 6 7 the block matrix inverse is as 6 0 0 1 0 7 " # 6 7 (A − BD−1C)−1 (C − DB−1A)−1 4 0 1 0 0 5 M −1 = , (B − AC−1D)−1 (D − CA−1B)−1 0 0 0 1 which is valid over any ring, and may be applied recursively. The usual approach to deal with this problem is to aban- This formulation has two defects, however: First, it re- don the block formulation of the inverse, and use a row- quires all of A, B, C and D to be invertible. In fact, M oriented divide and conquer PLU matrix factorization to compute the inverse using pivots to avoid non-invertible el- 3. Pivot-Free Inversion ements. See, for example, [5]. This has two problems: The first problem is that if we 3.1 Preconditioning for Block Inversion are representing matrices in memory using a recursive block structure (say, as quad-trees), then a row-oriented PLU de- To apply the recursive block methods given by (1) or (2), composition either requires converting the matrix in and out we must find a way to guarantee that the required blocks of the desired representation or requires using an indexing will be invertible. We do this by preconditioning the matrix scheme that carries a high overhead. The second problem is M. For invertible M, we may choose any invertible G and that if the entries are not themselves block sub-matrices, but have members of some non-commutative base ring, then there is M −1 = (GM)−1G.