General Letters in Mathematics Vol. 4, No. 1, Feb 2018, pp.13-22 e-ISSN 2519-9277, p-ISSN 2519-9269 Available online at http:// www.refaad.com
On Hadamard and Kronecker Products Over Matrix of Matrices
Z. Kishka1, M. Saleem 2, M. Abdalla3 and A. Elrawy 4
1,2 Dept. of Math., Faculty of Science, Sohag University, Sohag 82524, Egypt. 3,4 Dept. of Math., Faculty of Science, South Valley University, Qena 83523, Egypt. [email protected], 2 [email protected],3 [email protected],4 [email protected],
Abstract. In this paper, we define and study Hadmard and Kronecker products over matrix of matrices and their basic properties. Furthermore, we establish a connection the Hadamard product of matrix of matrices and the usual matrix of matrices multiplication. In addition, we show some application of the Kronecker product.
Keywords: Hadamard (Schur) product, Kronecker sum, Kronecker product, matrix of matrices. 2010 MSC No: 15A15, 15A09, 34A30, 39A10.
1 Introduction
Matrices and matrix operations play an important role in almost every branch of mathematics, computer graph- ics, communication, computational mathematics, natural and social sciences and engineering. The Hadamard and Kronecker products are studied and utilized widely in matrix theory, statistics [1, 2], physics[2], system theory and other areas; see, e.g. [3, 4, 5, 6, 7, 8, 9, 10, 11]. An equality connection between the Hadamard and Kronecker products look to be firstly used by e.g. [12, 13, 14]. In partitioned matrices, the Khatri-Rao product can be seen as a generalized Hadamard product which is discussed and used by many authors e.g. [15, 16]. Also Tracy-Singh product as a generalized Kronecker product is studied in [19, 20, 21]. Finally, the approach of this paper may not be practical conventional in all situations. In the present paper, we define and study Hadamard and Kronecker product over the matrix of matrices (in a short way; MMs) which was presented newly by Kishka et al [22]. We now give a short overview of this paper. In section 2, we define and study Hadamard product over MMs and give some properties. Then we show the association between Hadamard and MMs product in section 3. Finally, in section 4, we introduce the Kronecker product and prove a number of its properties. In addition, we introduce the notation of the vector matrices (VMs)-operator from which applications can be submitted to Kronecker product. Throughout this paper, the accompanying notations are utilized: Let K be a field and Ml (K) be the set of all l × l matrices defined on K, I and O stand for the identity matrix and the zero matrix in Ml (K), respectively. We denote by Mm×n (Ml (K)) the set of all m × n MMs over Ml (K) , the elements of Mm×n (Ml (K)) are denoted by A, B, C, D, G,.... [22] Let A ∈ Mm×n (Ml (K)) , then it can be written as a rectangular table of elements Aij; i = 1, 2, ..., m and j = 1, 2, ..., n, as follows: A11 ... A1n . .. . A = . . . . Am1 ... Amn 14 Z.Kishka et al.
t Definition 1.1. [22] Given a m × n-MMs; A = (Aij). Its transpose is the n × m-MMs A , given by
t [A ]ji = (Aij) = [A]ij, i = 1, ..., m; j = 1, ..., n, where Aij ∈ Ml (K) .
2 Hadamard Product
In this section, we give a definition of Hadamard product over MMs with some properties.
Definition 2.1. Let A = (Aij) and B = (Bij) ∈ Mm×n(Ml(K)), then
A ◦ B = (AijBij), i = 1, ..., m, j = 1, ..., n, where Aij,Bij ∈ Ml(K), this product is called Hadamard or Schur product over MMs.
Example 2.2. Let A and B ∈ M2(M2(K)) where
2 −3 2 3 −1 1 1 3 A = , 2 6 1 0 2 4 0 1 and 0 1 1 0 1 1 0 1 B = 3 3 , 2 −3 2 0 −1 1 0 2 then −1 1 2 3 1 −2 1 3 A ◦ B = 3 3 . −2 0 2 0 0 −2 0 2
Right away we might investigate some fundamentals properties of the Hadamard product over MMs.
Theorem 2.3. Let A and B ∈ Mm×n(Ml(K)), then A ◦ B = B ◦ A if Ml(K) is commutative ring.
Proof. The proof follows directly from the fact that Aij and Bij are commutative. Let A and B be m × n-MMs where A = (Aij) and B = (Bij), then
A ◦ B = (AijBij) = (BijAij) = B ◦ A.
Theorem 2.4. The identity MMs under the Hadamard product is the m × n MMs with all entries equal to I, denoted Jmn.
Proof. Let A = (Aij) ∈ Mm×n(Ml(K)), then A ◦ Jmn = (AijIij) = (Aij) and so Jmn ◦ A = (Aij). Therefore, Jmn as defined above is indeed the identity MMs under the Hadamard product.
We have denoted the Hadamard identity as Jmn to avoid confusion with the “usual” identity MMs, In.
ˆ Theorem 2.5. Let A = (Aij) ∈ Mm×n(Ml(K)), then A has a Hadamard inverse, denoted by A if and only if ˆ −1 Aij are non-singular ∀i = 1, 2, ..., m; j = 1, 2, ..., n. Furthermore, A = (Aij ). On Hadamard and Kronecker products over Matrix of Matrices 15
ˆ ˆ −1 Proof. (⇒) Let A = (Aij) ∈ Mm×n(Ml(K)) with Hadamard inverse A, then A ◦ A = AijAij = (Iij) = Jmn, which is only possible when all entries of A are invertible. In other words Aij are non-singular matrices ∀i = 1, 2, ..., m; j = 1, 2, ..., n. (⇐) Take any A = (Aij) ∈ Mm×n(Ml(K)) such that Aij are non-singular ∀i = 1, 2, ..., m; j = 1, 2, ..., n. Then there ˆ ˆ ˆ ˆ −1 −1 exists A = (Aij ) such that A ◦ A = A ◦ A = J mn, and so A has an inverse A = (Aij ).
ˆ We have denoted the Hadamard inverse as A to avoid confusion with the “usual” inverse MMs, A−1.
Theorem 2.6. Let A, B ∈ Mm×n(Ml(K)), then
(A ◦ B)t = At ◦ Bt.
Proof. Suppose that A, B ∈ Mm×n(Ml(K)) where A = (Aij) and B = (Bij), then
t t (A ◦ B) = (AijBij)
= (AjiBji) = At ◦ Bt.
t t t Remark 2.7. If Ml (K) is commutative ring with unity, then (A ◦ B) = B ◦ A .
Theorem 2.8. The set Mn(Ml(K)) of non-singular MMs is a group under Hadamard product.
Proof. Let A = (Aij), B = (Bij) and C = (Cij) ∈ Mn(Ml(K)) be MMs then it is easy to see that Mn(Ml(K)) is a group.
Theorem 2.9. Suppose that E ∈ Ml(K) and A, B and C ∈ Mn(Ml(K)), then (1) C◦ (A + B) = C ◦ A + C ◦ B. (2) E (A ◦ B) = (EA) ◦ B = A ◦ (EB) . where Ml(K) is commutative ring with unity. Proof. Part 1.
[C◦ (A + B)]ij = [C]ij [A + B]ij = [C]ij [A]ij + [B]ij
= [C]ij [A]ij + [C]ij [B]ij
= [C ◦ A]ij + [C ◦ B]ij
= [C ◦ A + C ◦ B]ij
Part 2.
[E (A ◦ B)]ij = E[A]ij[B]ij
= [EA]ij[B]ij
= [EA ◦ B]ij
= [A]ijE[B]ij
= [A]ij[EB]ij
= [A ◦ EB]ij , since Ml(K) is commutative ring with unity.
Remark 2.10. The above part 2 is also true if we replace E by c ∈ K. 16 Z.Kishka et al.
3 Connection between Hadamard and MMs products
In this section, we show connection between Hadamard and MMs multiplications. Let A = (Aij) ∈ Mm(Ml(K)) and consider the set S12...m of m cyclic permutations of 12...m given by
S12...m = {123... (m − 1) m, 23... (m − 1) m1, ..., m12... (m − 2) (m − 1)}. (1) th If Ai is the i column of the m × m MMs A, then we define
A(s) = (As1 : As2 : ... : Asm) , (2) i.e., A(s) is the MMs A with permuted columns according to the permutation s = s1s2...sm. It is easy to see that A(s) = A.I(s). Also, for B = (Bij) ∈ Mm(Ml(K)), let us define Bs11 Bs22 ··· Bsmm
Bs11 Bs22 ··· Bsmm B = . (3) [S] . . .. . . . . .
Bs11 Bs22 ··· Bsmm With these notation, we have the following theorem.
Theorem 3.1. Let A = (Aij) and B = (Bij) ∈ Mm(Ml(K)), then X X A.B = A(s) ◦ B[s] = A.I(s) ◦ B[s], s S where A(S) and B[S] are defined as in (2) and (3) , respectively, for a particular permutation s ∈ S12...m.
Proof. Suppose that A = (Aij) and B = (Bij) ∈ Mm(Ml(K)), then Pm Pm j=1 A1jBj1 ··· j=1 A1jBjm Pm Pm j=1 A2jBj1 ··· j=1 A2jBjm A.B= . .. . . . . Pm Pm j=1 AmjBj1 ··· j=1 AmjBjm m m m X X X = AijBj1 : AijBj2 : ... : AijBjm . j=1 j=1 j=1
Clearly, the usual product of MMs A.B decomposes uniquely as the sum of m MMs C(s), where s = s1s2...sm ∈ S123...m. Such a decomposition can be constructed as follows: The first MMs C(12...m) is obtained by
C12...m = (Ai1B11 : Ai2B22 : ... : AimBmm) = A(12...m) ◦ B[12...m]. The second MMs is selected according to the permutation 23...m1, i.e.,
C23...m1 = (Ai2B21 : Ai3B32 : ... : Ai1B1m) = A(23...m1) ◦ B[23...m1]. Following this procedure and taking the MMs according to the complete set of m cyclic permutations of 12...m in (1) , we obtain the (m − 1)th MMs as
C(m−1)m1...(m−3)(m−2) = A ◦ B . ((m−1)m1...(m−3)(m−2)) [(m−1)m1...(m−3)(m−2)] Finally, the MMs corresponding to the ultimate permutation m1···(m − 2)(m − 1) is formed by the remaining summands as Cm1···(m−2)(m−1) = A ◦ B . (m1···(m−2)(m−1)) [m1···(m−2)(m−1)] Thus, we have
A.B= A(12...m) ◦ B[12...m] + A(23...m1) ◦ B[23...m1] + ....+ A ◦ B + A ◦ B , ((m−1)m1...(m−3)(m−2)) [(m−1)m1...(m−3)(m−2)] (m1···(m−2)(m−1)) [m1···(m−2)(m−1)] which is the required result. On Hadamard and Kronecker products over Matrix of Matrices 17
2IO 2II Example 3.2. Let A = and B = ∈ M (M ( )), then I 3I IO 2 2 K
4I 2I A.B = 5II 2IO 2IO O 2I II = ◦ + ◦ I 3I 2IO 3II II
= A(12)◦B[12] + A(21)◦B[21].
Example 3.3. Let A = (Aij) and B = (Bij) ∈ M3(Ml(K)), then A11B11 + A12B21 + A13B31 A11B12 + A12B22 + A13B32 A11B13 + A12B23 + A13B33 A.B = A21B11 + A22B21 + A23B31 A21B12 + A22B22 + A23B32 A21B13 + A22B23 + A23B33 A31B11 + A32B21 + A33B31 A31B12 + A32B22 + A33B32 A31B13 + A32B23 + A33B33 A11 A12 A13 B11 B22 B33 A12 A13 A11 B21 B32 B13 = A21 A22 A23 ◦ B11 B22 B33 + A22 A23 A21 ◦ B21 B32 B13 A31 A32 A33 B11 B22 B33 A32 A33 A31 B21 B32 B13 A13 A11 A12 B31 B12 B23 + A23 A21 A22 ◦ B31 B12 B23 A33 A31 A32 B31 B12 B23
= A(123)◦B[123] + A(231)◦B[231] + A(312)◦B[312].
A comparable expansion of A.B when the MMs A and B are not square can be given by the following corollary.
Corollary 3.4. Let A = (Aij) ∈ Mm×k(Ml(K)) and B = (Bij) ∈ Mk×m(Ml(K)), with m ≤ k, then, X X A.B = A(s) ◦ B[s] = A.I(s) ◦ B[s], s S where the summation runs over all cyclic permutations s = s1s2...sm (consisting of all the first m indices) of S12...m....k. For a particular s = s1s2...sm ∈ S12...m....k, A(s) is as given in (2) with k replaced by m and B[s] is as in (3).
Example 3.5. Let us take B11 B12 A11 A12 A13 A = and B = B21 B22 , A21 A22 A23 B31 B33
Here, S123 = {123, 231, 312} as usual, but the permutations that we consider have only the first two parts, i.e., s = 12, 23, or 31. Then,
A B + A B + A B A B + A B + A B A.B = 11 11 12 21 13 31 11 12 12 22 13 32 A21B11 + A22B21 + A23B31 A21B12 + A22B22 + A23B32
= A(12) ◦ B[12] + A(23) ◦ B[23] + A(31) ◦ B[31] A A B B A A B B = 11 12 ◦ 11 22 + 12 13 ◦ 21 32 A21 A22 B11 B22 A22 A23 B21 B32 A A B B + 13 11 ◦ 31 12 . A23 A21 B31 B12
4 Kronecker Product
In this section, we give a definition of Kronecker product over MMs, and some properties. 18 Z.Kishka et al.
Definition 4.1. Let A = (Aij) ∈ Mn×m(Ml(K)) and B = (Bij) ∈ Mp×q(Ml(K)), the np × mq MMs A11B A12B ... A1mB A21B A22B ... A2mB A ⊗ B = , . . . . . . . . An1B An2B ... AnmB is called Kroncker product of A and B.
In order to explore the variety of applications of the Kronecker product we introduce the notation of the VMs –operator.
Definition 4.2. For any A ∈ Mm×n(Ml(K)), the VMs-operator is defined as
VMs (A) = (A11, ..., Am1, ..., A1n, ..., Amn) , i.e., the entries of A are stacked column wise forming a vector of matrices of length mn.
Example 4.3. Let 1 0 2 0 0 1 0 2 A = , 2 0 1 0 0 2 0 1 and 1 0 2 1 0 1 −1 2 B = , 1 0 0 1 0 1 −1 0 then 1 0 2 1 2 0 4 2 0 1 −1 2 0 2 −2 4 1 0 0 1 2 0 0 2 0 1 −1 0 0 2 −2 0 A ⊗ B = . 2 0 4 2 1 0 2 1 0 2 −2 4 0 1 −1 2 2 0 0 2 1 0 0 1 0 2 −2 0 0 1 −1 0
4.1 Properties of the Kronecker Product
Theorem 4.4. Let A ∈ Mm×n(Ml(K)), B ∈ Mr×s(Ml(K)), C ∈ Mn×p(Ml(K)) and D ∈ Ms×t(Ml(K)), then (A ⊗ B)(C ⊗ D) = AC ⊗ BD.
Proof. Simply verify that A11B ... A1nB C11D ... C1pD . .. . . .. . (A ⊗ B)(C ⊗ D) = . . . . . . Am1B ... AmnB Cn1D ... CnpD n n k=1A1kCk1BD ... k=1A1kCkpBD . .. . = . . . n n k=1AmkCk1BD ... k=1AmkCkpBD = AC ⊗ BD. On Hadamard and Kronecker products over Matrix of Matrices 19
t t t Theorem 4.5. Let A ∈ Mm×n(Ml(K)) and B ∈ Mr×s(Ml(K)), then (A ⊗ B) = A ⊗B .
Proof. Suppose that A = (Aij), B = (Bkl) and simply verify using the definition kronecker product over MMs
t A11B ... A1nB t . .. . (A ⊗ B) = . . . Am1B ... AmnB t t A11B ... Am1B . .. . = . . . t t A1nB ... AnmB = At⊗Bt.
Corollary 4.6. Let A ∈ Mm×n(Ml(K)) and B ∈ Mr×s(Ml(K)) be MMs symmetric, then A ⊗ B is symmetric.
Proof. Suppose that A = (Aij) and B = (Bkl) are symmetric MMs and Aij,Bkl ∈ Ml(K), then
(A ⊗ B)t = At⊗Bt = A ⊗ B.
So A ⊗ B is symmetric.
Theorem 4.7. If A and B are nonsingular, then (A ⊗ B)−1 = A−1⊗B−1.
Proof. Using the above Theorem 9, simply note that (A ⊗ B) A−1⊗B−1 = I ⊗ I = I.
Theorem 4.8. Suppose that E ∈ Ml(K), A ∈ Mm×n(Ml(K)) and B ∈ Mr×s(Ml(K)), then
(EA) ⊗B = A⊗ (EB) =E (A ⊗ B) , where Ml(K) is commutative ring with unity.
Proof. Since EA11B ... EA1nB . .. . (EA) ⊗B = . . . EAm1B ... EAmnB A11EB ... A1nEB . .. . = . . . Am1EB ... AmnEB = A⊗ (EB) A11B ... A1nB . .. . = E . . . . Am1B ... AmnB
Remark 4.9. The above Theorem 13 is also true if we replace E by c ∈ K.
Remark 4.10. For all A and B we have; (A ⊗ B) 6= B ⊗ A which can be justified by the following example. 20 Z.Kishka et al.
Example 4.11. Let 1 0 2 0 0 1 0 2 A = , 2 0 1 0 0 2 0 1 and 1 0 2 1 0 1 −1 2 B = , 1 0 0 1 0 1 −1 0 then 1 0 2 0 B B 0 1 0 2 (A ⊗ B) = 2 0 1 0 B B 0 2 0 1 1 0 2 1 A A 0 1 −1 2 6= 1 0 0 1 A A 0 1 −1 0 = B ⊗ A.
Corollary 4.12. Let A ∈ Mm(Ml(K)) and B ∈ Mn(Ml(K)), then (1) T r (A ⊗ B) = T r (A) T r (B) = T r (B ⊗ A) (2) det (A ⊗ B) = [det (A)]m [det (B)]n = det (B ⊗ A)
Definition 4.13. Let A ∈ Mm(Ml(K)) and B ∈ Mn(Ml(K)), then the Kronecker sum(or tensor sum) of A and B, denoted A ⊕ B is the mn × mn MMs (In⊗A) + (B ⊗ Im) . Example 4.14. Let I 2I 3I 2II A = 3I 2II and B = , 2I 3I II 4I then
A ⊕ B = (I2⊗A) + (B ⊗ I3) 3I 2I 3IIOO 3I 4IIOIO II 6IOOI = . 2IOO 4I 2I 3I O 2IO 3I 5II OO 2III 7I Remark 4.15. In general, A ⊕ B= 6 B ⊕ A. Lemma 4.16. The Kronecker product over MMs is associative, i.e., (A ⊗ B) ⊗C = A⊗ (B ⊗ C) , where A ∈ Mm×n(Ml(K)), B ∈ Mr×s(Ml(K)) and C ∈ Ms×t(Ml(K)). Lemma 4.17. The Kronecker product over MMs is right–distributive, i.e., (A + B) ⊗C = A ⊗ C + B ⊗ C, where A, B ∈ Mm×n(Ml(K)) and C ∈ Ms×t(Ml(K)). Lemma 4.18. The Kronecker product over MMs is left–distributive, i.e., A⊗ (B + C) = A ⊗ B + A ⊗ C, where A ∈ Mm×n(Ml(K)), B and C ∈ Ms×t(Ml(K)). On Hadamard and Kronecker products over Matrix of Matrices 21
4.2 Application The Kronecker product can be used to present linear matrix equations in which the unknowns are MMs. Examples for such equations are:
AX = B, AX + XB= C, AX B= C.
These equations are equivalent to the following systems of equations:
(I ⊗ A) VMs (X )=VMs (B) , (I ⊗ A) + Bt⊗I VMs (X )=VMs (C) , Bt⊗A VMs (X )=VMs (C) .
4.3 Relation between Hadamard and Kronecker product It is clear that the Hadamard product of two MMs is the principal sub-MMs of the Kronecker product of the two MMs. This relation can be expressed in an equation as follows.
Lemma 4.19. For A and B ∈ Mm×n(Ml(K)), then we have
t A ◦ B = Z1 (A ⊗ B) Z2,
2 2 where Z1 is the selection MMs of order m × m and Z2 is the selection MMs of order n × n. Example 4.20. Let I 2I 2II A = , B = , OI IO where A and B ∈ M2(M2(K)), and the selection MMs IOOO Z = Z = , 1 2 OOOI then t A ◦ B = Z1 (A ⊗ B) Z2.
5 Conclusion
In this work, we have introduced Hadamard and Kronecker product over a new algebraic structure which is called MMs and some of their properties. It is worth to mention that the new concepts are applicable in many different branches of mathematics such as group theory, statistics, combinatorics including graphs, other discrete structures, and functional analysis. Further research on this topic is now under investigation and will be reported in forthcoming works.
References
[1] R. Horn, Topics in Matrix Analysis, Cambridge University Press, (1994). [2] C. Johnson, Matrix Theory and Applications, American Mathematical Society, (1990). [3] G. P. H. Styan, Hadamard products and multivariate statistical analysis, Linear Algebra Appl., 6, (1973), 217-240. [4] H. Neudecker and S. Liu, A Kronecker matrix inequality with a statistical application, Econometric Theory, 11(1995), 655. 22 Z.Kishka et al.
[5] H. Neudecker and A. Satorra, A Kronecker matrix inequality with a statistical application, Econometric Theory, 11 (1995), 654. [6] G. Trenkler, A Kronecker matrix inequality with a statistical application, Econometric Theory, 11 (1995), 654-655. [7] C. R. Rao and M. B. Rao, Matrix Algebra and Its Applications to Statistics and Econometrics, World Scientific, Singapore, (1998). [8] F. Zhang, Matrix Theory. Basic Results and Techniques, Springer, New York, (1999). [9] S. Liu, On matrix trace Kantorovich-type inequalities, In: Innovations in Multivariate Statistical Analysis- A Festschrift for Heinz Neudecker (Ed. R.D.H. Heijmans, D.S.G. Pollock and A. Satorra), Kluwer Academic Publishers, Dordrecht, (2000a), pp. 39-50. [10] S. Liu, Inequalities involving Hadamard products of positive semidefinite matrices, J. Math. Ana. Appl., 243, (2000b), 458-463. [11] H. L. V. Trees, Detection, Estimation, and Modulation Theory, Part IV, Optimum Array Processing, Wiley, (2002). [12] M.W. Browne, Generalized least squares estimators in the analysis of covariance structures, South African Statist. J., 8 (1974), 1-24. [13] M. Faliva, Identificazione Stima nel Modello Lineare ad Equazioni Simultanee, Vita e Pensiero, Milan, Italy, (1983). [14] F. Pukelsheim, On Hsu’s model in regression analysis, Statistics, 8, (1977), 323-331. [15] C.G. Khatri and C. R. Rao, Solutions to some functional equations and their applications to characterization of probability distributions, Sankhya, 30 (1968), 167-180. [16] C. R. Rao, Estimation of heteroscedastic variances in linear models, J. Amer. Statist. Assoc., 65 (1970), 161-172. [17] C. R. Rao and M. B. Rao, Matrix Algebra and Its Applications to Statistics and Econometrics, World Scientitifc, Singapore, (1998). [18] R. A. Horn and R. Mathias, Block-matrix generalizations of Schur’s basic theorems on Hadamard products, Linear Algebra Appl., 172 (1992), 337-346. [19] R. P. Singh, Some Generalizations in Matrix Differentiation with Applications in Multivariate Analysis, Ph.D. Thesis, University of Windsor, (1972). [20] D. S. Tracy and R. P. Singh, A new matrix product and its applications in matrix differentiation, Statist. Neerlandica, 26 (1972), 143-157. [21] D. S. Tracy and K. G. Jinadasa, Partitioned Kronecker products of matrices and applications, Canada J. Statist., 17 (1989), 107-120. [22] Z. Kishka, M. Saleem, M. Abul-Dahab and A. Elrawy, On the algebraic structure of certain matrices, submitted to J. of the Egyptian Math. Soc., in revised version, (2017).