BlockMatrixFormulas.tex July 7, 2015 Block Matrix Formulas John A. Gubner Department of Electrical and Computer Engineering University of Wisconsin–Madison Abstract We derive a number of formulas for block matrices, including the block matrix inverse formulas, determinant formulas, psuedoinverse formulas, etc. If you find this writeup useful, or if you find typos or mistakes, please let me know at [email protected] 1. Preliminary Observations Given a block matrix AB F := ; CD if D is invertible, then the Schur complement of D is S := A − BD−1C: It is easy to check that I BD−1 S 0 I 0 F = : (1) 0 I 0 D D−1CI Then detF = detSdetD, and we see that given D is invertible, F is invertible if and only if S is invertible. When F is invertible, taking the inverse of (1) yields I 0S−1 0 I −BD−1 F−1 = −D−1CI 0 D−1 0 I " # S−1 −S−1BD−1 = : (2) − D−1CS−1 D−1CS−1BD−1 + D−1 Instead of assuming D is invertible, suppose A is invertible. The Schur comple- ment of A is Q := D −CA−1B; 1 BlockMatrixFormulas.tex July 7, 2015 and I 0 A 0 IA−1B F = : (3) CA−1 I 0 Q 0 I Then detF = detAdetQ, and we see that given A is invertible, F is invertible if and only if Q is invertible. If F is invertible, I −A−1BA−1 0 I 0 F−1 = 0 I 0 Q−1 −CA−1 I " # A−1 + A−1BQ−1CA−1 −A−1BQ−1 = : (4) − Q−1CA−1 Q−1 2. Results Using (1)–(4), there are several identities that can easily be found. 2.1. Determinant Formulas First, comparing (1) and (3) yields AB det = det(A)det(D −CA−1B) = det(A − BD−1C)det(D): (5) CD In particular, taking A = In and D = Ik yields det(Ik −CB) = det(In − BC): (6) 2.2. Matrix Inversion Formulas Next, comparing the upper-left blocks of (2) and (4), we see that [A − BD−1C]−1 = A−1 + A−1B[D −CA−1B]−1CA−1; (7) which is known as the Sherman–Morrison–Woodbury formula or sometimes just the Woodbury formula. The remaining corresponding blocks are also equal. For example, combining the top row of (2) with the bottom row of (4), we obtain " # AB −1 [A − BD−1C]−1 −[A − BD−1C]−1BD−1 = : (8) CD − [D −CA−1B]−1CA−1 [D −CA−1B]−1 2 BlockMatrixFormulas.tex July 7, 2015 2.3. Psuedoinverse Formulas A nice application of (8) is to the pseudoinverse of a 2 × 2 block matrix. Recall that if M is full rank, then its pseudoinverse is M† = (M∗M)−1M∗, where M∗ is the complex conjugate transpose of M. Consider the case h i G∗ M = GH and M∗ = : H∗ Then G∗GG∗H M∗M = ; H∗GH∗H and by (8), " # (G∗P?G)−1 −(G∗P?G)−1G∗H(H∗H)−1 (M∗M)−1 = H H ; (9) ∗ ? −1 ∗ ∗ −1 ∗ ? −1 − (H PG H) H G(G G) (H PG H) ? ∗ −1 ∗ ? ∗ −1 ∗ where PH := I − H(H H) H and PG := I − G(G G) G . It now easily follows that " # " # (G∗P?G)−1G∗P? (P?G)† (M∗M)−1M∗ = H H = H : (10) ∗ ? −1 ∗ ? ? † (H PG H) H PG (PG H) 2.4. Positive Semidefinite Matrices A matrix U is positive semidefinite if U∗ = U and x∗Ux ≥ 0 for all vectors x. In this case, we use the notation U 0. If U and V are Hermitian, we write U V if U −V is positive semidefinite. For real matrices, the condition U = U∗ is equivalent to U = U T, where U T denotes the transpose of U. Suppose that F is real with FT = F. Then A = AT, D = DT, and C = BT. Now rewrite (1) as A − BD−1BT 0 I −BD−1 AB I 0 = : 0 D 0 I BT D −D−1BT I Suppose the center matrix on the right is known to be positive semidefinite. Then since the matrices on either side are transposes of each other, it is easy to see that the matrix on the left-hand side is also positive semidefinite. It then follows that A − BD−1BT is positive semidefinite; in symbols, A BD−1BT. 3 BlockMatrixFormulas.tex July 7, 2015 2.4.1. Upper-Left Block Submatrices Given a 2 × 2 block matrix U, we denote its upper-left block by fUgULB. For the ∗ matrix F defined earlier, fFgULB = A. We show below that if F = F , then ∗ −1 ∗ −1 fU F UgULB fU gULBfFgULBfUgULB: More specifically, if PQ U := ; RS then ∗ −1 ∗ −1 fU F UgULB P A P: To derive the result, we first use (4) to compute the left-hand block column of F−1U. We get " # fA−1 + A−1BQ−1CA−1gP − A−1BQ−1R : − Q−1CA−1P + Q−1R It follows that " # fA−1 + A−1BQ−1CA−1gP − A−1BQ−1R fU∗F−1Ug = P∗ R∗ ULB − Q−1CA−1P + Q−1R = P∗A−1P + P∗A−1BQ−1CA−1P − P∗A−1BQ−1R − RQ−1CA−1P + R∗Q−1R = P∗A−1P + (B∗A−1P − R)∗Q−1(CA−1P − R): ∗ ∗ ∗ −1 ∗ −1 Since F = F , we have C = B , it follows that fU F UgULB P A P. Remarks. (i) The above display does not depend on Q or S. (ii) The only requirement on U is that its size be such that the required matrix multiplications are defined. The number of columns in Q and S must be the same, but this number is arbitrary, and could be zero, in which case U is a 2 × 1 block matrix. In particular, there is no requirement that U be a square matrix. References [1] W. W. Hager, “Updating the inverse of a matrix,” SIAM Review, vol. 31, no. 2, pp. 221–239, 1989. [2] Wikipedia, “Schur complement — Wikipedia, The Free Encyclopedia,” [Online]. Available: https://en.wikipedia.org/w/index.php?title=Schur_complement&oldid= 658902757, accessed July 6, 2015. [3] Wikipedia, “Woodbury matrix identity — Wikipedia, The Free Encyclopedia,” [Online]. Available: https://en.wikipedia.org/w/index.php?title=Woodbury_matrix_ identity&oldid=661757721, accessed July 6, 2015. 4.