Traces of Hadamard and Kronecker Products of Matrices

Math. Appl. 6 (2017), 143–150 DOI: 10.13164/ma.2017.09

TRACES OF HADAMARD AND KRONECKER PRODUCTS OF MATRICES

PANKAJ KUMAR DAS and LALIT K. VASHISHT

Abstract. We present some inequality/equality for traces of Hadamard product and Kronecker product of matrices. Some numerical examples are given to support the results.

1. Introduction

The Hadamard (or Schur) and Kronecker products are widely studied and applied in matrix theory, statistics, system theory and other areas [2]. It was Schur who initially studied algebraic and analytic properties of Hadamard product. In 1990, Horn [1] presents a widespread information focusing on the Hadamard product. Magnus and Neudecker [4] give some basic results and statistical applications involving Hadamard or Kronecker products. For basics on these two matrix products, one may refer to [3,5–7]. In [6], the authors investigated traces of Hadamard and Kronecker products of matrices and obtained some inequalities for traces of products of matrices. In this note, we present some inequality for traces of Hadamard product, Kronecker product and mixed type product of matrices. We recall the basic deﬁnitions and notations to make our presentation self- + contained. The set of all positive real numbers is denoted by R . By Mn we denote the family of n-by-n matrices over the real ﬁeld R. For A = [aij] ∈ Mn, the Pn scalar i=1 aii is called the trace of A and is usually denoted by tr(A) or traceA. The Hadamard product of two matrices A = [aij] and B = [bij] of identical size is just their element-wise product which is given by

A ◦ B = [aijbij]m×n.

Let A = [aij] be an m-by-n matrix and let B = [bij] be a p-by-q matrix. The Kronecker product of A and B is deﬁned as

  a11B a12B . . . a1nB  a21B a22B . . . a2nB  A ⊗ B =    . . .. .   . . . .  am1B am2B . . . amnB

MSC (2010): primary 15A45; secondary 15A24. Keywords: trace, Hadamard product, Kronecker product. 143 144 P. K. DAS and L. K. VASHISHT

Lemma 1.1. [6] Let A ∈ Mn and B ∈ Mm. Then, tr(A ⊗ B) = tr(B ⊗ A) = tr(A)tr(B).

Theorem 1.2. [6] If A = [aij],B = [bij] ∈ Mn, then Pn−1 Pn (1) tr(A ⊗ B) = n tr(A ◦ B) − i=1 j=i+1(aii − ajj)(bii − bjj). A+B A+B (2) tr(A ◦ B) ≤ tr 2 ◦ 2 .

2. Main results

We start with an inequality involving the trace of m times Hadamard product of a matrix and the trace of the given matrix .

Proposition 2.1. Let A ∈ Mn with positive real diagonal. Then, tr A ◦ A ◦ A ◦ · · · ◦ A ≤ (tr(A))m. | {z } m + Proof. Let A = [aij ] ∈ Mn and let aii ∈ R . Then, by the deﬁnitions of Hadamard product and trace of matrices, we have n m m X m X m tr A ◦ A ◦ A ◦ · · · ◦ A = a ≤ aii = (tr(A)) . | {z } ii m i=1 i=1 The proposition is proved.

Proposition 2.2. If A = [aij],B = [bij] ∈ Mn, then n tr (A ◦ A ◦ · · · ◦ A) ◦ (B ◦ B ◦ · · · ◦ B) | {z } | {z } n n n n n−1 n X n X n X X n n n n = aii bii − (aii − ajj)(bii − bjj). i=1 i=1 i=1 j=i+1 Proof. Since n n X X tr (A ◦ A ◦ · · · ◦ A) ⊗ (B ◦ B ◦ · · · ◦ B) = an bn , | {z } | {z } ii ii n n i=1 i=1 applying Theorem 1 in [6], we get n tr (A ◦ A ◦ · · · ◦ A) ◦ (B ◦ B ◦ · · · ◦ B) | {z } | {z } n n n n n−1 n X n X n X X n n n n = aii bii − (aii − ajj)(bii − bjj). i=1 i=1 i=1 j=i+1 The following proposition provides a relation between the trace of a matrix and Kronecker product of the given matrix. TRACES OF HADAMARD AND KRONECKER PRODUCTS OF MATRICES 145

Proposition 2.3. For any A ∈ Mn, we have tr A ⊗ A ⊗ A ⊗ · · · ⊗ A = (tr(A))m. | {z } m Proof. We prove the proposition by induction. By Lemma 1.1, the equation is true for n = 2. Assume that it is true n = k, i.e., tr A ⊗ A ⊗ A ⊗ · · · ⊗ A = (tr(A))k. (2.1) | {z } k Again using Lemma 1.1, we compute tr A ⊗ A ⊗ A ⊗ · · · ⊗ A = tr A ⊗ A ⊗ A ⊗ · · · ⊗ A tr(A) | {z } | {z } k+1 k = (tr(A))ktr(A) (by (2.1)) = (tr(A))k+1.

The proposition is proved. 3 1 Example 2.4. Let A = . 4 1 Then, 27 9 9 3 9 3 3 1 36 9 12 3 12 3 4 1   36 12 9 3 12 4 3 1   9 1 48 12 12 3 16 4 4 1 A ◦ A = ,A ⊗ A ⊗ A =   16 1 36 12 12 4 9 3 3 1   48 12 16 4 12 3 4 1   48 16 12 4 12 4 3 1 64 16 16 4 16 4 4 1 and tr(A) = 4. Now we have tr(A ◦ A) = 10 ≤ 42 = (tr(A))3. So, Proposition 2.1 is true. Also, tr(A ⊗ A ⊗ A) = 64 = (tr(A))3. This veriﬁes Proposition 2.2.

Proposition 2.5. For any A ∈ Mn, we have tr (A + At) ◦ (A + At) = 4 tr(A ◦ A).

Proof. Let A = [aij] ∈ Mn. Then n n t t X X 2 tr (A + A ) ◦ (A + A ) = (aii + aii)(aii + aii) = 4 aii = 4 tr(A ◦ A). i=1 i=1 The proposition is proved.

Proposition 2.6. For any A, B ∈ Mn, we have tr (A + B) ◦ (A − B) = tr(A ◦ A) − tr(B ◦ B). 146 P. K. DAS and L. K. VASHISHT

Proof. Let A = [aij],B = [bij] ∈ Mn be arbitrary. We compute n X tr (A + B) ◦ (A − B) = (aii + bii)(aii − bii) i=1 n X 2 2 = (aii) − (bii) i=1 = tr(A ◦ A) − tr(B ◦ B).

Proposition 2.7. For any A, B ∈ Mn, we have tr (A + B) ◦ (A + B) = tr(A ◦ A) + 2tr(A ◦ B) + tr(B ◦ B).

Proof. Similar to proof of Proposition 2.6. The following proposition gives the relationship between trace of a matrix obtained as Kronecker product of a ﬁnite sum of matrices and the traces of the matrices. We prove the result for two matrices.

Proposition 2.8. Let A, B ∈ Mn. Then tr (A + B) ⊗ (A + B) ⊗ (A + B) ⊗ · · · ⊗ (A + B) | {z } k k k−i i X k = tr(A) tr(B) . i i=1 Proof. Using Proposition 2.3, we compute tr (A + B) ⊗ (A + B) ⊗ (A + B) ⊗ · · · ⊗ (A + B) | {z } k k k = tr(A + B) = tr(A) + tr(B)

k k−i i X k = tr(A) tr(B) . i i=1 The result is proved.

Remark 2.9. Using Proposition 2.1, for any A, B ∈ Mn with positive diagonal, we can show that tr (A + B) ◦ (A + B) ◦ (A + B) ◦ · · · ◦ (A + B) | {z } n n n−i i X n ≤ tr(A) tr(B) . i i=1 TRACES OF HADAMARD AND KRONECKER PRODUCTS OF MATRICES 147

The next proposition gives a trace inequality for the Hadamard product of matrix sums.

Proposition 2.10. If A, B ∈ Mn, then 1 (tr(A ◦ B))2 ≤ tr (A + B) ◦ (A + B) ◦ (A + B) ◦ (A + B) . 16 Proof. Using Theorem 1.2 and Proposition 2.3, we compute A + B A + B A + B A + B (tr(A ◦ B))2 ≤ tr ◦ tr ◦ 2 2 2 2 A + B A + B A + B A + B = tr ◦ ◦ ◦ 2 2 2 2 1 = tr (A + B) ◦ (A + B) ◦ (A + B) ◦ (A + B) . 16

The proposition is proved. The following theorem provides a relation between the trace of matrices gener- ated by Kronecker and Hadamard product of matrices.

Theorem 2.11. If A, B, C, D ∈ Mn, then (1) (A ⊗ B) ◦ (C ⊗ D) = (A ◦ C) ⊗ (B ◦ D). Pn−1 Pn (2) tr(A⊗(B ◦C)) = n tr(A◦B ◦C)− i=1 j=i+1(aii −ajj)(biicii −bjjcjj).

Proof. Let us write A = [aij] and C = [cij]. Then, we compute (A ⊗ B) ◦ (C ⊗ D)     a11B a12B . . . a1nB c11D c12D . . . c1nD  a21B a22B . . . a2nB   c21D c22D . . . c2nD  =   ◦    . . .. .   . . .. .   . . . .   . . . .  an1B an2B . . . annB cn1D cn2D . . . cnnD   a11c11B ◦ D a12c12B ◦ D . . . a1nc1nB ◦ D  a21c21B ◦ D a22c22B ◦ D . . . a2nc2nB ◦ D  =   = (A ◦ C) ⊗ (B ◦ D).  . . .. .   . . . .  an1cn1B ◦ D an2cn2B ◦ D . . . anncnnB ◦ D Hence, (1) is proved. Let us write B = [bij]. Then, by Theorem 1.2, we have

n−1 n X X tr(A ⊗ (B ◦ C)) = n tr(A ◦ (B ◦ C)) − (aii − ajj)(biicii − bjjcjj) i=1 j=i+1 n−1 n X X = n tr(A ◦ B ◦ C) − (aii − ajj)(biicii − bjjcjj). i=1 j=i+1

Thus, (2) is proved. 148 P. K. DAS and L. K. VASHISHT

Corollary 2.12. If A, B, C, D ∈ Mn, then tr (A ⊗ B) ◦ (C ⊗ D) = n tr(A ◦ B ◦ C ◦ D)

n−1 n X X − (aiicii − ajjcjj)(biidii − bjjdjj). i=1 j=i+1 Indeed, by Theorem 2.11, we can write tr (A ⊗ B) ◦ (C ⊗ D) = tr (A ◦ C) ⊗ (B ◦ D) .

Thus, by Theorem 1.2, we have tr (A ⊗ B) ◦ (C ⊗ D) = n tr(A ◦ C ◦ B ◦ D)

n−1 n X X − (aiicii − ajjcjj)(biidii − bjjdjj). i=1 j=i+1

Remark 2.13. If A, C ∈ Mn and B,D ∈ Mk (k 6= n), then the equations given in Theorem 2.11 hold too. 3 1 4 −1 2 3 Example 2.14. Let A = , B = , C = and 4 −1 3 2 1 −1 −2 1 D = . −1 −3 Then 12 −3 4 −1 −4 2 −6 3   9 6 3 2  −2 −6 −3 −9 A ⊗ B =   ,C ⊗ D =   ; 16 −4 −4 1  −2 1 2 −1 12 8 −3 −2 −1 −3 1 3

6 3 −8 −1 A ◦ C = ,B ◦ D = ; 4 1 −3 −6 and −24 −3 −8 −1  −9 −18 −3 −6 −24 −1 A ⊗ (B ◦ D) =   ,A ◦ B ◦ D = . −32 −4 8 1  −12 6 −12 −24 3 6 It is easy to see that −48 −6 −24 −3  −18 −36 −9 −18 (A ⊗ B) ◦ (C ⊗ D) =   = (A ◦ C) ⊗ (B ◦ D). −32 −4 −8 −1  −12 −24 −3 −6 We compute n−1 n X X n tr(A ◦ B ◦ D) − (aii − ajj)(biidii − bjjdjj) i=1 j=i+1 TRACES OF HADAMARD AND KRONECKER PRODUCTS OF MATRICES 149

= 2(−18) − (−8) (here n = 2) = −28 = tr (A ⊗ (B ◦ D)). Thus, Theorem 2.11 is veriﬁed. To conclude the paper, we give a result that connects the traces of matrices obtained by Kronecker and Hadamard product of matrices in terms of the trace of Hadamard product of matrices.

Theorem 2.15. If Ai,Bi ∈ Mn for 1 ≤ i ≤ n, then n Y tr (A1 ⊗ A2 ⊗ · · · ⊗ An) ◦ (B1 ⊗ B2 ⊗ · · · ⊗ Bn) = tr(Ai ◦ Bi). i=1 Proof. We prove the theorem by induction. Clearly, the equation is true for n = 1. Assume that it is true for n = k, i.e., k Y tr (A1 ⊗ A2 ⊗ · · · ⊗ Ak) ◦ (B1 ⊗ B2 ⊗ · · · ⊗ Bk) = tr(Ai ◦ Bi). i=1 We compute tr (A1 ⊗ A2 ⊗ · · · ⊗ Ak+1) ◦ (B1 ⊗ B2 ⊗ · · · ⊗ Bk+1) = tr (A1 ⊗ A2 ⊗ · · · ⊗ Ak) ◦ (B1 ⊗ B2 ⊗ · · · ⊗ Bk) ⊗ (Ak+1 ◦ Bk+1)

(by Theorem 2.11) = tr (A1 ⊗ A2 ⊗ · · · ⊗ Ak) ◦ (B1 ⊗ B2 ⊗ · · · ⊗ Bk) tr(Ak+1 ◦ Bk+1)

(by Lemma 1.1) k Y = tr(Ai ◦ Bi)tr(Ak+1 ◦ Bk+1) i=1 k+1 Y = tr(Ai ◦ Bi). i=1 Thus, the equation is true for n = k + 1. The theorem is proved.

Remark 2.16. Theorem 2.15 is useful because the computation of Ai ◦ Bi (1 ≤ i ≤ n) is much easier than (A1 ⊗ A2 ⊗ · · · ⊗ An) ◦ (B1 ⊗ B2 ⊗ · · · ⊗ Bn).

References

[1] R. A. Horn, The Hadamard product, Proc. Symp. Appl. Math. 40 (1990), 87–169. [2] R. A. Horn and C. R. Johnson, Matrix Analysis, 2nd ed., Cambridge University Press, 2010. [3] S. Liu and G. Trenkler, Hadamard, Khatri–Rao, Kronecker and other matrix products, Int. J. Inf. Syst. Sci. 4 (2008), 160–177. [4] J. R. Magnus and H. Neudecker, Matrix Diﬀerential Calculus with Applications in Statistics and Econometrics, 2nd ed., Wiley, Chichester, UK, 1999. [5] M. Ozel,¨ Trace and determinant inequalities for Hadamard products, International Journal of Physical Sciences 7 (2012), 1–4. [6] N. Taskara and I. H. Gumus, On the traces of Hadamard and Kronecker products of matrices, Selcuk Journal of Applied Mathematics 14 (2013), 31–36. 150 P. K. DAS and L. K. VASHISHT

[7] H. Zhang and F. Ding, On the Kronecker products and their applications, J. Appl. Math. 2013 (2013), Article ID 296185, 8 pp.

Pankaj Kumar Das, Department of Mathematical Sciences, Tezpur University, Napaam, Tezpur, Assam-784028, India (On lien from Shivaji College, University of Delhi, Department of Mathematics, Raja Garden, Ring Road, New Delhi-110 027, India) e-mail: [email protected], [email protected]

Lalit K. Vashisht, Department of Mathematics, University of Delhi, Delhi-110007, India e-mail: [email protected]