Inequalities for Kronecker Products and Hadamard Products of Positive

Home , Kronecker product

R ESEARCH ARTICLE ScienceAsia 35 (2009): 106–110 doi: 10.2306/scienceasia1513-1874.2009.35.106 Inequalities for Kronecker products and Hadamard products of positive deﬁnite matrices Pattrawut Chansangiam∗, Patcharin Hemchote, Praiboon Pantaragphong Department of Mathematics and Computer Science, Faculty of Science, King Mongkut’s Institute of Technology Ladkrabang, Bangkok 10520, Thailand

∗ Corresponding author, e-mail: [email protected] Received 9 Oct 2008 Accepted 15 Jan 2009

ABSTRACT: The purpose of this paper is to develop inequalities for Kronecker products and Hadamard products of positive deﬁnite matrices. A number of inequalities involving powers, Kronecker powers, and Hadamard powers of linear combination of matrices are presented. In particular, Holder¨ inequalities and arithmetic mean-geometric mean inequalities for Kronecker products and Hadamard products are obtained as special cases.

KEYWORDS:Lowner¨ partial order, jointly concave map

INTRODUCTION that A is positive semidefinite. The relation “6” is reflexive, antisymmetric and transitive, i.e., “6” forms Inequalities have proved to be powerful tools in a partial order on the space of Hermitian matrices. mathematics. Matrix inequalities arise in various This is known as the Lowner¨ partial order 5. In fact, branches of mathematics and science such as system this relation does not form a total order on the space of and control theory 1, optimization 2 and semidefinite Hermitian matrices; there are Hermitian matrices A, programming 3. Matrix inequalities are also important B of same size which are not comparable, i.e., neither tools in quantum statistical inference and quantum A 6 B nor A > B holds. information theory 4. This paper is concerned with Now we introduce matrix products that differ inequalities for Kronecker products and Hadamard from the ordinary matrix multiplication. A notion products of special kinds of matrices. Let us begin which is useful in the study of matrix equations and with some terminology and notation. other applications is the Kronecker product of matri- As usual, R stands for the set of real numbers. ces. The Kronecker product of A = [aij] ∈ Mm,n and Denote the set of all m × n complex matrices by B = [bij] ∈ Mp,q is defined to be the block matrix Mm,n and abbreviate Mn,n to Mn for convenience.  a B ··· a B  The identity matrix in Mn is denoted by In or I if the 11 1n  . . .  size of matrix is clear. A matrix A ∈ Mn is Hermitian A ⊗ B := . .. . ∈ Mmp,nq. ∗ ∗   if A = A, where denotes the conjugate transpose. a B ··· a B ∗ m1 mn A ∈ Mn is positive semidefinite if x Ax > 0 for all vectors x ∈ Cn (Cn is the set of n-tuples of This matrix product has very nice properties. The complex numbers). If A is positive semidefinite and most important is the mixed-product property: invertible, then A is positive definite. An equivalent (A ⊗ B)(C ⊗ D) = AC ⊗ BD condition for A ∈ Mn to be positive definite is that A is Hermitian and all eigenvalues of A are positive real numbers. A Hermitian matrix with the positivity for matrices A, B, C, D with appropriate sizes. The property (a positive definite matrix), which plays a set of positive semidefinite matrices is closed under similar role to a positive real number, is an important the Kronecker product, i.e. kind of matrix. The set of all n × n positive definite A > 0,B > 0 =⇒ A ⊗ B > 0. matrices is represented by Pn. By inequalities for matrices we mean inequalities The Hadamard product of A, B ∈ Mm,n is the in the Lowner¨ sense. Given Hermitian matrices A and entrywise product B of the same size, A 6 B means that B − A is positive semidefinite. In particular, A > 0 indicates A ◦ B := [aijbij] ∈ Mm,n. www.scienceasia.org ScienceAsia 35 (2009) 107

The Hadamard product differs from the ordinary where α, β are scalars. A map ψ : Pn × Pn → Pm product in many ways. The most important is that is jointly concave if for any A, B, C, D ∈ Pn and any Hadamard multiplication is commutative. The most 0 < < 1, basic properties of the product is the closure of the cone of positive semidefinite matrices under the ψ A + (1 − )B, C + (1 − )D) Hadamard product (Schur product theorem 6), i.e. > ψ(A, C) + (1 − )ψ(B,D). The following fact is well-known. A > 0,B > 0 =⇒ A ◦ B > 0. Lemma 2 (Ando 7) A map Φ defined by Φ(A, B) = −1 −1 −1 Inequalities for the Kronecker product and the (A + B ) for A, B ∈ Pn is jointly concave. 6–10 Hadamard product of matrices have a long history . The following identity is used in the next section. There are many results associated with them. In this paper we study inequalities for Kronecker products Lemma 3 The following identity holds for any and Hadamard products involving powers, Kronecker A, B ∈ Pn and s > 0 : powers and Hadamard powers of linear combinations −1 (s−1A ⊗ I)−1 + (I ⊗ B)−1 of matrices. We derive some inequalities and then in- −1 vestigate results under special cases. Among these we = (A ⊗ B−1) (A ⊗ B−1) + (sI ⊗ I) (I ⊗ B). obtain Holder¨ and arithmetic mean-geometric mean (1) (AM-GM) inequalities. Proof : Since A and B are positive definite and s PRELIMINARIES is positive, all terms in the equality (1) are existent, We recall the concept of maps on matrix spaces. Let f positive definite and nonsingular. For convenience, −1 −1 be a continuous real-valued function on a real interval write X = s A ⊗ I, Y = I ⊗ B, Z = A ⊗ B and P = X + Y . It follows from the mixed-product Ω. Let A ∈ Mn be Hermitian with eigenvalues λ1, λ2, . . . , λn contained in Ω. Since A is Hermitian, property of the Kronecker product that A can be decomposed as Z + (sI ⊗ I) (s−1I ⊗ I) − (s−1Y )P −1(s−1Z)

∗ −1 −1 A = U diag[λ1, λ2, . . . , λn] U = Z(s I ⊗ I) + (sI ⊗ I)(s I ⊗ I) − Z(s−1Y )(X + Y )−1(s−1Z) where U is unitary (i.e. U ∗U = I) and −1 −1 −1 diag[λ1, λ2, . . . , λn] is the diagonal matrix with diag- − (sI ⊗ I)(s Y )(X + Y ) (s Z) onal entries λ1, λ2, . . . , λn. The functional calculus = (s−1Z) + (I ⊗ I) − X(X + Y )−1(s−1Z) for A is deﬁned as − Y (X + Y )−1(s−1Z) ∗ −1 −1 −1 f(A) = U diag[f(λ1), f(λ2), . . . , f(λn)] U . = (s Z) + In2 − (X + Y )(X + Y ) (s Z)

= In2 . For example, if A ∈ Pn and r ∈ R, then That is Ar = U diag[λr, λr, . . . , λr ] U ∗. 1 2 n (s−1I ⊗ I) − (s−1Y )(X + Y )−1(s−1Z) −1 The following property involving Kronecker products = Z + (sI ⊗ I) . of positive deﬁnite matrices can be derived from the Again, the mixed-product property yields mixed-product property. −1 Z−1X−1 + Y −1 Y −1 Lemma 1 (Horn and Johnson 6) For any A, B ∈ Pn = Z−1X − X(X + Y )−1XY −1 and r ∈ R, we have (A ⊗ B)r = Ar ⊗ Br. = (A−1 ⊗ B)X(Y −1) A map Φ: Mn → Mm is unital if Φ maps − (A−1 ⊗ B)X(X + Y )−1XY −1 unit element to unit element, i.e. Φ(In) = Im. −1 −1 −1 −1 Φ is positive if Φ maps positive element to positive = (s I ⊗ I) − (s Y )(X + Y ) (s Z) −1 element, i.e. A > 0 ⇒ Φ(A) > 0. Φ is linear if Φ = Z + (sI ⊗ I) . preserves addition and scalar multiplication, i.e., −1 Thus, (X−1 +Y −1)−1 = ZZ +(sI ⊗I) Y which Φ(αA + βB) = αΦ(A) + βΦ(B) is (1).

www.scienceasia.org 108 ScienceAsia 35 (2009)

INEQUALITIES FOR KRONECKER is jointly concave. It is well-known that the positive PRODUCTS linear combination of the jointly concave maps is In this section we derive inequalities for the Kronecker jointly concave. Hence, from the viewpoint of the Riemann integral, the integrand is also jointly con- product of positive definite matrices in the form (αA+ r 1−r βB)r ⊗ (αC + βD)s and α(Ar ⊗ Cs) + β(Br ⊗ cave, and so is A ⊗ B . This means that for any Ds) where A, B, C, D are positive definite matrices A, B, C, D ∈ Pn and scalar 0 < < 1, and α, β, r, s are positive real numbers such that r + r s s = 1. (A + (1 − )B) ⊗ (C + (1 − )D) r s r s > (A ⊗ C ) + (1 − )(B ⊗ D ) Theorem 1 For A, B, C, D ∈ Pn and α, β, r, s > 0 such that r + s = 1, for s > 0 and r + s = 1. Since 0 < α/(α + β) < 1, by setting = α/(α + β), we get (2). (αA + βB)r ⊗ (αC + βD)s From this theorem, we obtain the Holder¨ inequality for positive definite matrices as a special case. α(Ar ⊗ Cs) + β(Br ⊗ Ds). (2) > Recall that real numbers p, q are conjugate exponents Proof : Let f be a real-valued function defined by if p, q are positive and 1/p + 1/q = 1. f(t) = tr for t > 0 and 0 < r < 1. Clearly, f is Corollary 1 For A, B, C, D ∈ and conjugate continuous. Recall an integral representation of f (see Pn exponents p, q, we have Ref. 8): Z ∞ r−1 r sin rπ s t p p 1 q q 1 t = ds. (A⊗B)+(C⊗D) 6 (A +C ) p ⊗(B +D ) q . (3) π 0 s + t For convenience, write Y = I ⊗B and Z = A⊗B−1. Proof : First, set α = β = 1, r = 1/p and s = 1/q Hence, the functional calculus for A ⊗ B−1, namely, in Theorem 1. Then swap B with C. Finally, replace p q p q f(A ⊗ B−1) = (A ⊗ B−1)r can be written as A, B, C, D with A ,B ,C ,D , respectively. The case p = 2 (hence q = 2) in (3) is the ∞ sin rπ Z −1 Cauchy-Schwarz inequality. See Ref. 7 for general- (sI ⊗ I)r−1ZZ + (sI ⊗ I) ds. π 0 izations of the Holder¨ inequality for positive definite matrices. Recall that the Kronecker sum of A ∈ Mn It follows from Lemma 1 that and B ∈ Mm is defined as Ar ⊗ B1−r = (ArI) ⊗ (B−rB) A ⊕ B := (A ⊗ Im) + (In ⊗ B) ∈ Mmn. = (Ar ⊗ B−r)(I ⊗ B) −1 r With this notation, we obtain the following from the = (A ⊗ B ) (I ⊗ B). case B = C = I in Corollary 1.

Hence, by Lemma 3 we obtain Corollary 2 For A, B ∈ Pn and conjugate exponents p, q, we have Ar ⊗ B1−r ∞ p 1 q 1 sin rπ Z −1 A ⊕ B (A + I) p ⊗ (B + I) q . (4) = (sI ⊗ I)r−1ZZ + (sI ⊗ I) ds Y 6 π 0 ∞ The equality in (2) occurs if A = B, C = D. sin rπ Z −1 = sr−1ZZ + (sI ⊗ I) ds Y Now we investigate results under other special cases π 0 ∞ of Theorem 1. We consider the cases (i) A = C, B = sin rπ Z −1 = sr−1ZZ + (sI ⊗ I) Y ds D, (ii) A = D, B = C, (iii) r = s, (iv) α = β. π 0 Many inequalities can be obtained via combining ∞ sin rπ Z −1 these cases. For A, B, C, D ∈ and α, β, r, s > 0 = sr−1(s−1A ⊗ I)−1 + Y −1 ds. Pn π 0 such that r + s = 1, we have the following results:

Since s−1A ⊗ I and I ⊗ B are positive deﬁnite, by (αA + βB)r ⊗ (αA + βB)s Φ: 2 × 2 → 2 r s r s Lemma 2 we have that the map Pn Pn Pn > α(A ⊗ A ) + β(B ⊗ B ) , (5) deﬁned by (αA + βB)r ⊗ (βA + αB)s −1 r s r s Φ(s−1A⊗I,I⊗B) = (s−1A⊗I)−1+(I⊗B)−1 > α(A ⊗ B ) + β(B ⊗ A ) , (6) www.scienceasia.org ScienceAsia 35 (2009) 109

1 (αA + βB) ⊗ (αC + βD) 2 and hence 1 1 2 2 1 1 1 1 α(A ⊗ C) + β(B ⊗ D) , (7) 2 ⊗2 2 ⊗2 2 2 > (A ) + (B ) > (A ⊗ B) + (B ⊗ A) . (A + B)r ⊗ (C + D)s A B (Ar ⊗ Cs) + (Br ⊗ Ds). (8) Similarly, if and are comparable, then (10) is > sharper than (11). Note that giving positive definite From inequalities (5)-(8), it follows that matrices A, B does not always imply A > B or A 6 B. (ii) Without the commutativity of A and B under 1 (A + B) ⊗ (C + D) 2 the Kronecker product, the inequality (16) becomes 1 1 the AM-GM inequality for real numbers when n = 1. > (A ⊗ C) 2 + (B ⊗ D) 2 , (9) (A + B)r ⊗ (A + B)s INEQUALITIES FOR HADAMARD r s r s > (A ⊗ A ) + (B ⊗ B ) , (10) PRODUCTS (A + B)r ⊗ (A + B)s In this section we derive inequalities for Hadamard r s r s products of positive definite matrices. > (A ⊗ B ) + (B ⊗ A ) , (11) 1 2 (αA + βB) ⊗ (βA + αB) Theorem 2 For A, B, C, D ∈ Pn and α, β, r, s > 0 1 1 such that r + s = 1, > α(A ⊗ B) 2 + β(B ⊗ A) 2 . (12) r s Recall that the kth Kronecker power of A ∈ Mn (αA + βB) ◦ (αC + βD) r s r s is defined inductively for any positive integer k by > α(A ◦ C ) + β(B ◦ D ). (18) A⊗1 := A and A⊗k := A ⊗ A⊗(k−1), for k > 1. It is easy to see that for any A ∈ Pn, positive integer Proof : Let us define Φ: Pn × Pn → Pn2 by k, and real number r, Φ(A, B) = Ar ⊗ Bs. Recall that the Hadamard product of matrices is the principal submatrix of the ⊗k r r ⊗k (A ) = (A ) . (13) Kronecker product of matrices. Consequently, there exists a unital positive linear map ϕ : 2 → such With this notation and property (13), we get the Pn Pn that ϕ(A ⊗ B) = A ◦ B. Hence, following results:

1 ⊗2 1 ⊗2 1 ⊗2 (ϕ ◦ Φ)(A, B) = ϕ(Φ(A, B)) (αA + βB) 2 > α(A 2 ) + β(B 2 ) , (14) = ϕ(Ar ⊗ Bs) = Ar ◦ Bs. 1 ⊗2 1 ⊗2 1 ⊗2 (A + B) 2 > (A 2 ) + (B 2 ) , (15) 1 ⊗2 1 1 Since Φ is jointly concave (by Theorem 1) and ϕ (A + B) 2 (A ⊗ B) 2 + (B ⊗ A) 2 . (16) > is positive and linear, the composition ϕ ◦ Φ is also The next result is the AM-GM inequality for matrices jointly concave. This means that for any A, B, C, D ∈ involving the Kronecker product. Pn and any scalar 0 < < 1,

Corollary 3 If A, B ∈ Pn commute under the Kro- (A + (1 − )B)r ◦ (C + (1 − )D)s necker product, then r s r s > (A ◦ C ) + (1 − )(B ◦ D ). 1 1 1 ⊗2 (A ⊗ B) 2 (A + B) 2 (17) 6 2 Since 0 < α/(α+β) < 1, by replacing with α/(α+ β), we get (18). A = B with equality if and only if . We obtain the Holder¨ inequality for positive deﬁ- Proof : The inequality (16) becomes (17) if A ⊗ B = nite matrices as a special case of Theorem 2. B ⊗ A. The equality part follows from the fact that A ⊗ B = B ⊗ A if and only if there is a scalar c such Corollary 4 For A, B, C, D ∈ Pn and conjugate exponents p, q, we have that A = cB or B = cA. p p 1 q q 1 Remark 1 (i) If A and B are comparable, then (15) (A◦B)+(C ◦D) 6 (A +C ) p ◦(B +D ) q . (19) is sharper than (16). The reason is that the closure of α = β = 1 r = 1/p s = Pn under the Kronecker product yields Proof : First set , and 1/q in (18). Then swap B with C. Finally, replace 1 1 1 1 p q p q (A 2 − B 2 ) ⊗ (A 2 − B 2 ) > 0 A, B, C, D with A ,B ,C ,D , respectively.

www.scienceasia.org 110 ScienceAsia 35 (2009)

The case p = 2 (hence q = 2) in (19) is the Remark 2 If A and B are comparable, then (30) is Cauchy-Schwarz inequality. The Hadamard sum of sharper than (31). Indeed, the closure of Pn under the matrices is defined and studied in Ref. 11. The Hadamard product implies Hadamard sum of A, B ∈ Mn is 1 1 1 1 (A 2 − B 2 ) ◦ (A 2 − B 2 ) > 0 A • B := A ◦ I + I ◦ B. and hence In particular, from Corollary 4, we get 1 (2) 1 (2) 1 1 (A 2 ) + (B 2 ) > 2(A 2 ◦ B 2 ). Corollary 5 For A, B ∈ Pn and conjugate exponents p, q, we have Similarly, if A and B are comparable, then (26) is sharper than (27). p 1 q 1 A • B (A + I) p ◦ (B + I) q . (20) 6 Now we obtain, from (31), the AM-GM inequal- The equality in (18) occurs if A = B, C = D. ity for matrices involving the Hadamard product. Now we investigate results under other special cases Corollary 6 For A, B ∈ , we have the following of Theorem 2. We consider the cases (i) A = C,B = Pn inequality: D, (ii) A = D, B = C, (iii) r = s, (iv) α = β. Many inequalities can be derived from these 1 1 1 1 (2) A 2 ◦ B 2 (A + B) 2 . (32) cases. The following inequalities hold for any 6 2 A, B, C, D ∈ Pn and α, β, r, s > 0 with r + s = 1: CONCLUSIONS (αA + βB)r ◦ (αA + βB)s We have obtained many matrix inequalities involving r s r s Kronecker products and Hadamard products of pos- > α(A ◦ A ) + β(B ◦ B ), (21) itive definite matrices using the concept of maps on (αA + βB)r ◦ (βA + αB)s matrix spaces. We believe that one can get other r s s r > α(A ◦ B ) + β(A ◦ B ), (22) inequalities by appropriate use of this concept. Our 1 1 (αA + βB) 2 ◦ (αC + βD) 2 results should be applicable in fields related to matrix 1 1 1 1 theory. > α(A 2 ◦ C 2 ) + β(B 2 ◦ D 2 ), (23) REFERENCES (A + B)r ◦ (C + D)s (Ar ◦ Cs) + (Br ◦ Ds). (24) 1. Boyd S, Ghaoui L, Feron E, Balakrishnan V (1994) > Linear Matrix Inequalities in System and Control The- From inequalities (21)-(24), it follows that ory, SIAM, Philadelphia.

1 1 2. Todd M (2001) Semidefinite optimization. Acta Numer- (A + B) 2 ◦ (C + D) 2 ica 10, 515–60. 1 1 1 1 3. Klerk E (2002) Aspects of Semidefinite Programming: > (A 2 ◦ C 2 ) + (B 2 ◦ D 2 ) , (25) r s Interior Point Algorithms and Selected Applications, (A + B) ◦ (A + B) Kluwer Academic Publishers. r s r s > (A ◦ A ) + (B ◦ B ) , (26) 4. Barndorff-Nielsen OE, Gill RD, Jupp PE (2003) On (A + B)r ◦ (A + B)s quantum statistical inference. J Roy Stat Soc B 65, r s s r 775–816. > (A ◦ B ) + (A ◦ B ) , (27) 5. Lowner¨ K (1934) Uber¨ monotone Matrixfunktionen. 1 1 (αA + βB) 2 ◦ (βA + αB) 2 Math Z 38, 177–216. 1 1 6. Horn RA, Johnson CR (1991) Topics in Matrix Analy- > (α + β)(A 2 ◦ B 2 ). (28) sis, Cambridge Univ Press, New York. Recall that the kth Hadamard power of A ∈ 7. Ando T (1979) Concavity of certain maps on positive Mn definite matrices and applications to Hadamard prod- is defined by A(k) := [ak ], k being a positive integer. ij ucts. Lin Algebra Appl 26, 203–41. (k) k (k) It is easy to see that (αA) = α A and 8. Bhatia R (1997) Matrix Analysis, Springer-Verlag, New A(k) = A ◦ A(k−1), k = 2, 3,.... York. 9. Horn RA, Johnson CR (1985) Matrix Analysis, Cam- With this notation, we have the following results: bridge Univ Press, New York. 10. Zhan X (2002) Matrix Inequalities, Springer-Verlag, 1 (2) 1 (2) 1 (2) (αA + βB) 2 > α(A 2 ) + β(B 2 ) , (29) Berlin. 1 (2) 1 (2) 1 (2) 11. Zhour Z, Kiliciman A (2006) Matrix equalities and in- (A + B) 2 (A 2 ) + (B 2 ) , (30) > equalities involving Khatri-Rao and Tracy-Singh sums. 1 (2) 1 1 (A + B) 2 > 2(A 2 ◦ B 2 ). (31) J Inequal Pure Appl Math 7, 1–17. www.scienceasia.org