arXiv:2007.14034v2 [math.NA] 15 Nov 2020 emean we eoetecnuaetasoeo arx For matrix. a of transpose conjugate the denote H oain n preliminaries and Notations Introduction 1 matrices. of combination linear pro rank semidefinite mum (SDS), similarity via diagonalization cong ous via diagonalization simultaneous similarity, matrix ones n * † eoetesto elsmerc(ep,Hriin matric Hermitian) (resp., symmetric real of set the denote ) oguneo emta arcs some matrices: Hermitian of congruence Keywords: Email: Email: C qiaetcniin n numerical a and conditions equivalent oeeuvln odtosfrta uhamti collectio matrix via a diagonalized such neously that for Theoretic conditions equivalent matrices. some Hermitian commute) pairwise need (not loih o iutnosydaoaiigvia diagonalizing simultaneously for algorithm omlmtie eie rmtepeiu tg.Illustrat stage. M previous in the hand/coding from derived matrices normal via diagonalizable initia simultaneously are the ces whether detecting comb a algorithm program is semidefinite an algorithm positive proposed propose The we problem. view, such solving of sem cally a point to practical solution definite From positive (SDP). a of existence the to leads utnosydaoaiigvia diagonalizing multaneously , eateto ahmtc n ttsis u hnUnivers Nhon Quy Statistics, and Mathematics of Department and hsppram tsligteHriinSCpolm .. t i.e., problem, SDC Hermitian the solving at aims paper This A [email protected] [email protected] iutnosdaoaiainvia diagonalization Simultaneous spstv eient rs. oiiedfiie susual as definite) positive (resp., semidefinite positive is F n × n emta arcs matrix matrices, Hermitian eoetesto all of set the denote iuL Thanh Le Hieu ATLAB oebr1,2020 17, November r lopresented. also are Let . ∗ cnrec.neetnl,oeo uhconditions such of one -congruence.Interestingly, solution F ∗ * Abstract eoetefil fra numbers real of field the denote -congruence n n gnNue Thi Nguyen Ngan and × 1 n arcswt nre in entries with matrices ∗ ∗ cnrec,ad()aJacobi-like a (2) and -congruence, cnrec,matrix -congruence, A olcino ntl many finitely of collection a ∗ cnrec h commuting the -congruence ∈ H n , une(D) simultane- (SDC), ruence by rmig(D) maxi- (SDP), gramming emta matri- Hermitian l a esimulta- be can n dfiieprogram idefinite n xmlsby examples ing A ly eprovide we ally, sin es nto f()a (1) of ination t,Vietnam ity, † o numeri- for 0 F F T a of hat . . (resp., n R Let -congruence, × n rcomplex or . S By si- n A (resp., . ≻ ∗ we 0 ) n n From now on, without stated otherwise, C1,...,Cm F × are always Her- C ∈ C m mitian matrices. By (λ) we denote the Hermitian pencil (λ) =: i=1 λiCi, i.e., the parameters λ1,...,λm ranges only over real numbers. Finally, Hermitian P matrices C1,...,Cm are said to be (i) simultaneously diagonalizable via similarity on F, abbreviated F-SDS, if there n n 1 exists a nonsingular matrix P F × such that P − CiP ’s are all diagonal n n ∈ matrices in F × . When m = 1, we will say ”C1 is similar to a diagonal matrix” or “C1 is diagonalizable (via similarity)” as usual. (ii) simultaneously diagonalizable via -congruence on F, abbreviated F-SDC, ∗ n n if there exists a nonsingular matrix P F × such that P ∗CiP is diagonal ∈ for every i = 1, . . . , m. Here P ∗ means the conjugate transpose of P. When m =1, we will say “C1 is congruent to a diagonal matrix” as usual.
It is worth mentioning that every diagonal matrix P ∗CiP is always real since Ci is Hermitian. Moreover, we will see in Theorems 4 and 7 below that P can be chosen to be real if Ci’s are all real. Additionally, one should distinguish that -congruence in this paper is different from T-one in [4]. These two types∗ of congruence will coincide only when the Hermitian matrices are real.
(iii) commuting if they pairwise commute, i.e., CiCj = CjCi for every i, j = 1, . . . , m. Depend upon particular situation in the rest of this paper, the term “SDC” will mean either “simultaneous diagnalization” or “simultaneously diagnalizing”, or “simultaneously diagnalizable” via -congruence. It is analogously to the term “SDS”. ∗ We now recall some well-know results that will be frequently used in the rest of the paper. Fn n F Lemma 1. Let A, B × be A = diag(α1In1 ,...,αkInk ), αi’s are distinct. ∈ ni ni ∋ If AB = BA then B = diag(B ,...,Bk) with Bi F × for every i =1, . . . , k. 1 ∈ Furthermore, B is Hermitian (resp., symmetric) if and only if so are Bi’s.
Proof. Partitioning B as B = [Bij]i,j=1,...,k, where Bii is a square submatrix of size ni ni, i = 1,...,k and off-diagonal blocks are of appropriate sizes. It then follows× from
α1B11 ... α1B1k α1B11 ... αkB1k ...... . . . = AB = BA = . . . αkBk1 ... αkBkk α1Bk1 ... αkBkk that αiBij = αjBij, i = j. Thus Bij =0 for every i = j. The last claim is trivial.∀ 6 6
Lemma 2. (See, eg., in [13]) (i) Every A Hn can be diagonalized via similarity ∈ by a unitary matrix. That is, it can be written as A = UΛU ∗, where U is unitary, Λ is real diagonal and is uniquely defined up to a permutation of diagonal elements. Moreover, if A Sn then U can be picked to be real. ∈ n n (ii) Let C1,...,Cm F × such that each of which is similar to a diagonal matrix. They are then F-SDS∈ if and only if they are commuting.
2 n n m m (iii) Let A F × , B F × . The matrix M = diag(A, B) is diagonalizable via similarity if∈ and only if∈ so are both A and B. (iv) A complex symmetric matrix A is diagonalizable via similarity if and only 1 if it is complex orthogonally diagonalizable, i.e., Q− AQ is diagonal for some n n T complex orthogonal matrix Q C × : Q Q = I. ∈ Proof. The first and last parts can be found in [13]. We prove (ii) and (iii) for real matrices, the complex setting was proved in [13]. n n (ii) Suppose C1,...,Cm R × are commuting. We prove by induction on n. For n =1, there are nothing∈ to prove. Suppose that n 2 and that the assertion has been proved for all collections of m commuting real≥ matrices of size k k, × 1 k n 1. ≤ ≤ − By the hypothesis, C1 is diagonalized via similarity by a nonsingular matrix n n P R × , ∈ 1 P − C P = diag(α In1 ,...,αkInk ), n + ... + nk = n, αi = αj, i = j. 1 1 1 6 ∀ 6 1 1 The commutativity of C1,Ci implies that of P − C1P and P − CiP. By Lemma 1,
1 nt nt P − CiP = diag(Ci ,...,Cik), Cit R × , t =1, . . . , k. 1 ∈ ∀ 1 1 Furthermore, P − CiP and P − CjP commute due to the commutativity of Ci,Cj. So, Cit and Cjt commute for every t = 1, . . . , t. This means the matrices C2t, ...,Cmt are commuting for every t = 1, . . . , k. By the induction hypothesis, for each t = 1, . . . , k, the matrices C2t,...,Cmt are R-SDS by a nonsingular matrix nt nt Qt R × , and so are Int ,C t,...,Cmt. Let ∈ 2 n n Q = diag(Q ,...,Qk) R × , 1 ∈ and set U = P Q. One can check C1,...,Cm are R-SDS by U. The converse is trivial. (n+m) (n+m) (iii) Suppose M is DS in R × . Then there exists a nonsingular matrix (n+m) (n+m) S R × such that ∈ 1 S− MS =Λ= diag(α ,...,αn,...αn m), αi R, i =1,...,m + n. 1 + ∈
Let sj be the j-th column of S with
ξj n m sj = , ξj R , νj R . νj ∈ ∈ n (n+m) m (n+m) Let E = [ξ1 ...ξn+m] R × , N = [ν1 ...νn+m] R × . If rank(E) < n or rank(N) < m then∈ ∈
m + n = rank(S) rank(E)+ rank(N) < m + n, ≤ which is impossible. Thus rank(E) = n and rank(N) = m. Moreover, it follows from MS = SΛ that the jth column ξj of E is the eigenvector of A w.r.t the eigenvalue λj. This means A has n linearly independent eigenvectors. By the same reasoning, B also has m linearly independent eigenvectors. Thus A and B are n n m m diagonalizable via similarity in R × and R × , respectively.
3 History of SDC problems. Matrix simultaneous diagonalizations, via similarity or congruence, appear in a number of research areas, for examples, quadratic equa- tions and optimization [12, 15], multi-linear algebra [6], signal processing, data analysis [6], quantum mechanics [22], ... The SDC problem, i.e., the one of finding a nonsingular matrix that simulta- neously diagonalizes a collection of matrices via congruence, can be dated back to 1868 by Weierstrass [25], 1930s by Finsler [8, 1, 11], and later studies devel- oped some conditions ensuring a collection of quadratic forms are SDC (see, eg., in [18, 20] and references therein). However, almost these works provide only sufficient (but not necessary) conditions, except for, eg., that in [23], [13, 14] and references therein dealing with two Hermitian/symmetric matrices; or that in [15] focusing on real symmetric matrices which have a positive definite linear combi- nation. Some other works focus on simultaneous block-diagonalizations of normal matrices, so does of Hermitian ones [13, 24]. An (equivalently) algorithmic con- dition for the real symmetric SDC problem is given in a preprint manuscript [19]. Very recently in the seminal paper [4], the authors develop an equivalent condition solving the complex symmetric SDC problem. Note that this admits T-congruence, since the matrices there are complex symmetric, instead of -one as in this paper. ∗ The simultaneous diagonalization via T-congruence for complex symmetric ma- trices does not guarantee for real setting. That is, the resulting nonsingular and diagonal matrices may not be real even thought the initial ones are real. An exam- ple given in [4] is that
0 1 1 1 C = , C = S2 1 1 1 2 1 0 ∈ are SDC via T-congruence. However, the resulting nonsingular matrix P and the T T corresponding diagonal ones P C1P,P C2P are properly complex. We will see in this paper that the simultaneous diagonalizableness via -congruence of Hermitian matrices will immediately holds true for real symmetric setting.∗ Turn- ing to the two matrices above, we will see from Theorem 4 below that they are not 1 0 1 R-SDC because C− C = is not similar to a real diagonal matrix (it has 1 2 1− 1 1 i√3 only complex eigenvalues ±2 ). From the practical point of view, there is a few works dealing with numerical methods for solving SDC problems with respect to particular type of matrices. The work in [3] deal with two commuting normal matrices where they apply Jacobi- like algorithm. This method is then extended to the case of more than commuting normal matrices, and is performed in MATLAB very recently [17]. Contribution of the paper. In this paper, we completely solve the Hermitian SDC problem. The paper contains the following contributions:
We develop some sufficient and necessary conditions, see Theorems 3, 4 and 7, • for that a collection of finitely many Hermitian matrices can be simultaneously diagonalized via -congruence. As a consequence, this solves the long-standing SDC problem for∗ real symmetric matrices mentioned as an open problem in [12].
Interestingly, one of such the conditions in Theorem 7 requires the existence of • a positive definite solution to a semidefinite program. This helps us to check
4 whether a collection of Hermitian matrices is SDC or not. In case the initial ma- trices are SDC, we apply the existing Jacobi-like method in [3, 17] (for simul- taneously diagonalizing a collection of commuting normal matrices by a unitary matrix) to simultaneously diagonalize the commuting Hermitian matrices from the previous stage. The Hermitian SDC problem is hence completely solved. In line of giving an equivalent condition that requires the maximum rank of • the Hermitian pencil (Theorem 4), we propose a Schm¨udgen-like procedure for finding such the maximum rank in Algorithm 3. This may be applied in some other simultaneous diagonalizations, for example that in [4].
The corresponding algorithms are also presented and implemented in MATLAB • among which the main is Algorithm 5. Unlike to the complex symmetric matrices as [4], what discussed for Hermitian setting in this paper thus immediately imply to real symmetric one. Construction of the paper. In Section 2 we give a comprehensive description on Hermitian SDC property. A relationship between the SDC and SDS problems is included as well. In addition, since the main result in this section, Theorem 4, asks to find a maximum rank linear combination of the initial matrices, we sug- gest a method responding to this requirement in Subsection 2.3. Section 3 presents some other SDC equivalent conditions among which leads to use semidifinite pro- grams for detecting the simultaneous diagonalizability via -congruence of Her- mitian matrices. An algorithm, that completely numerically∗ solves the Hermitian SDC problem, is then proposed. Some numerical tests are given as well in this section. The conclusion and further discussion is devoted in the last section.
2 Hermitian-SDC and SDS problems
Recall that the SDS problem is that of finding a nonsingular matrix that simultane- ous diagonalizes a collection of square complex matrices via similarity, while the Hermitian SDC one is defined as earlier.
2.1 SDC problem for commuting Hermitian matrices The following is presented in [13, Theorem 4.1.6] whose proof hides how to find a nonsingular matrix simultaneously diagonalizing the given matrices. Our proof is constructive and it may lead to a procedure of finding such a nonsingular matrix. It follows that of Theorem 9 in [15] for real symmetric matrices. n Theorem 3. The matrices I,C1,...,Cm H , m 1, are SDC if and only if they are commuting. Moreover, when this∈ is the case,≥ these are SDC by a unitary matrix, and the resulting diagonal matrices are all real. n n Proof. If I,C ,...,Cm are SDC then there is a nonsingular matrix U C × such 1 ∈ that U ∗IU, U ∗C1U,...,U ∗CmU are diagonal. Note that
U ∗IU = diag(d ,...,dn) 0. (1) 1 ≻ Let D = diag( 1 ,..., 1 ) and V = UD. Then V must be unitary and √d1 √dm
V ∗CiV = DU ∗CiUD is diagonal for every i =1, . . . , m.
5 So (V ∗CiV )(V ∗CjV )=(V ∗CjV )(V ∗CjV ), i = j, and hence CiCj = CjCi for ∀ 6 every i = j. It is worth mentioning that each V ∗CiV is real since it is Hermitian. 6 We now prove the opposite direction by induction on m. The case m =1, the proposition is clearly true since any Hermitian matrix can be diagonalized (certainly via congruence as well as similarity) by a unitary matrix. For m 2, we suppose the theorem holds true for m 1. ≥ − We now consider an arbitrary collection of matrices I,C1,...,Cm. Let P be a unitary matrix that diagonalizes C1 :
P ∗P = I, P ∗C1P = diag(α1In1 ,...,αkInk ),
where αi’s are distinctly real eigenvalues of C1. Since C1,Ci commute for every i =2, . . . , m, so do P ∗C1P and P ∗CiP. By Lemma 1, for every i =2, . . . , m, we have P ∗CiP = diag(Ci1,...,Cik),
where every Cit is Hermitian of order nt. Now, for each t = 1, . . . , k, since CitCjt = CjtCit for all i, j = 2, . . . , m, provided by CiCj = CjCi, the induction hypothesis leads to the fact that
Int ,C2t,...,Cmt (2) are SDC by a unitary matrix Qt. Set U = P diag(Q1,...,Qk). Then
U ∗C1U = diag(α1In1 ,...,αkInk ), (3)
U ∗CiU = diag(Q1∗Ci1Q1,...,Qk∗CikQk), i =2, . . . , m, are all diagonal. It is worth mentioning that the less number of multiple eigenvalues of the start- ing matrix C1 in the proof of Theorem 3, the less number of collections as in (2) must be solved. We keep into account this observation to the first step in the fol- lowing. Algorithm 1. Solving the SDC problem of commuting Hermitian matrices. n INPUT: Commuting matrices C ,...,Cm H . 1 ∈ OUTPUT: A unitary matrix U that SDC the matrices I,C1,...,Cm. Step 1: Pick a starting matrix with the least number of multiple eigenvalues.
Step 2: Find an eigenvalue decomposition of C1 : C1 = R P ∗diag(λ In1 ,...,λkInk )P, n + ... + nk = n, λi’s are 1 1 ∋ distinct and P ∗P = I.
Step 3: Computing diagonal blocks of P ∗CiP, i 2 : ≥ ni P ∗CiP = diag(Ci ,...,Cik), Cit H , t =1, . . . , k, 1 ∈ ∀
where C2t,...,Cmt pairwise commute for each t =1, . . . , k. Step 4: For each t = 1, . . . , k, simultaneously diagonalizing the collection of
matrices Int ,C2t,...,Cmt by a unitary matrix Qt.
Step 5: Define U = P diag(Q1,...,Qk) and V ∗CiV.
6 2.2 An equivalent condition via the SDS Using Theorem 3 we comprehensively describe the SDC property of a collection of Hermitian matrices as follows. It is worth mentioning that the parameter λ appearing in the following theorem is always real even if F is the field of real or complex numbers.
n n m Theorem 4. Let C ,...,Cm F × 0 be Hermitian with dimF ( ker Ct)= 1 ∈ \{ } t=1 q (always q < n). T 1) When q =0,
m (a) If det C(λ)=0 for all λ R (only real m-tuples λ) then C1,...,Cm are not SDC (on F); ∈ m (b) Otherwise, C(λ) is nonsingular for some λ R . The matrices C1,...,Cm 1 ∈ 1 are F-SDC if and only if they C(λ)− C1,..., C(λ)− Cm pairwise com- 1 mute and every C(λ)− Ci, i = 1, . . . , m, is similar to a real diagonal matrix.
2) If q > 0 then there exists a nonsingular matrix P such that
0q 0 P ∗CiP = , i =1, . . . , m, (4) 0 Cˆi ∀ n q m where 0q is the q q zero matrix and Cˆi H − with ker Cˆt =0. × ∈ t=1 F ˆ ˆ Moreover, C1,...,Cm are -SDC if and only if C1,...,TCm are SDC. m Proof. 1) Suppose dimF t=1 ker Ct =0. For the part (a), the fact that C1,...,Cm are SDC by a nonsingular matrix n n T P F × implies ∈
Ci = P ∗DiP,Di = diag(αi ,...,αin), i =1, . . . , m, 1 ∀ where we note Di is real since Ci = Ci∗. This follows that the real polynomial (with real variable λ)
n m 2 det C(λ)= det(P ) ( αijλi) (5) | | j=1 i=1 Y X is identically zero because of the hypothesis. Since R[λ1,...,λm] is an integral domain, there exists a factor identically zero, say, (α1j,...,αmj)=0 for some n j = 1, . . . , n. Pick a vector x satisfying P x = ej, the j-th unit vector in F , one obtains Cix = P ∗DiP x = P ∗Diej =0. m This means 0 = x i=1 ker Ci, contradicting to the hypothesis. This proves the part (a). 6 ∈ We now prove theT part (b). Suppose C(λ) is nonsingular for some λ Rm. n n ∈ If C1,...,Cm are SDC then there exists a nonsingular matrix P F × such that n n ∈ P ∗CiP F × are all diagonal and real, and so is P ∗C(λ)P = [P ∗C(λ)P ]∗ since λ Rm.∈Then ∈ 1 1 1 P − C(λ)− CiP = [P ∗C(λ)P ]− (P ∗CiP ) (6)
7 is diagonal for every i =1, . . . , m. n n Conversely, take a nonsingular matrix P F × such that ∈ 1 1 n n P − C(λ)− C P := D R × (7) 1 1 ∈ is diagonal. Up to a rearrangement of diagonal elements, we can assume
D = diag(α In1 ,...,αkInk ), n + ... + nk = n, αi = αj i = j. 1 1 1 6 ∀ 6 1 1 For every i =2, . . . , m, since D1 and P − C(λ)− CiP commute. Lemma 1 implies
1 1 nt nt P − C(λ)− CiP = diag(Ci ,...,Cik), Cit F × , t =1, . . . , k. 1 ∈ ∀ The proof immediately complete after two following claims are proven. Claim 1. For each t =1, . . . , k, the collection C2t,...,Cmt, is SDS by a nonsingu- 1 1 lar matrix Qt. Indeed, since P − C(λ)− CiP is diagonalizable via similarity, so is nt nt Cit F × for every t =1, . . . , k, provided by Lemma 2(iii). Moreover, the pair- ∈ 1 1 1 1 wise commutativity of C(λ)− C2,..., C(λ)− Cm implies that of P − C(λ)− C2P, 1 1 ...,P − C(λ)− CmP ; and hence that of
C2t,...,Cmt, for each t =1, . . . , k. By Lemma 2, the later matrices are thus SDS by an invertible matrix Qt, i.e.,
1 1 Qt− C2tQt =: D2t, ..., Qt− CmtQt =: Dmt are all diagonal. Claim 2. There exists a nonsingular matrix U such that U ∗C1U,...,U ∗CmU pair- wise commute. These later matrices are then F-SDC by Theorem 3. Indeed, let
U = P diag(Q1,...,Qk), we then have 1C 1 U − (λ)− C1U = diag(α1In1 ,...,αkInk )= D1 1 1 U − C(λ)− CiU = diag(Di1,...,Dik)= Di, i =2, . . . , m,
Note that Di is real for every i = 1,...,m because of the hypothesis. Since Ci is Hermitian, the equality (6) implies
[U ∗C(λ)U]Di = U ∗CiU =(U ∗CiU)∗ = Di[U ∗C(λ)U], i =1, . . . , m. ∀ Then
(U ∗CiU)(U ∗CjU) = [U ∗C(λ)U(DiDj)U ∗C(λ)U] = [U ∗C(λ)U(DjDi)U ∗C(λ)jU]
=(U ∗CjU)(U ∗CiU), for every i = j. Theorem 3 allows us to finish this part. 6 2) Pick an orthonormal basis u1,...,uq,uq+1,...,un of the F-unitary vector n n 1 m space F F × such that u ,...,uq is an orthonormal basis of ker Ct. The ≡ 1 t=1 matrix P whose the columns are uj’s will satisfy the conclusion that P ∗CiP = ˆ m ˆ T diag(0q, Ci) and t=1 ker Ct =0. Finally, we already know that C1,...,Cm are SDC if and only if so are P ∗C1P, T ...,P ∗CmP, ans so are Cˆ1,..., Cˆm (see Lemma 9 in Appendix A).
8 Algorithm 2. Detecting whether a collection of Hermitian matrices is SDC or not. n INPUT: Matrices C ,...,Cm H (not necessary pairwise commute). 1 ∈ OUTPUT: Conclude whether C1,...,Cm are SDC or not.
Compute a singular value decomposition C = UΣV ∗ of C = [C1∗ ... Cm∗ ]∗, Σ = diag(σ1,...,σn q, 0,..., 0), σ1 ... σn q > 0, 0 q n 1. Then m − ≥ ≥ − ≤ ≤ − dimF ( t=1 ker Ct)= q. IF qT= 0 :
m Step 1: If det C(λ)=0 for all λ R then C ,...,Cm are not SDC. ∈ 1 Else, go to Step 2. Step 2: Find a λ Rm such that C := C(λ) is nonsingular. ∈ 1 (a) If C− Ci is not similar to a diagonally real matrix for some i =1, . . . , m, then conclude the given matrices are not SDC. Else, go to (b). 1 1 (b) If C− C1,..., C− Cm do not pairwise commute, which is 1 equivalent to that CiC− Cj is not Hermitian for some pair i = j, then conclude the given matrices are not SDC. Else,6 conclude the given matrices are SDC.
ELSE (q > 0) :
Step 3: For C = UΣV ∗ determined at the beginning, the q last columns of V, m say ~v1,...,~vq, span ker C = t=1 ker Ci. Pick a nonsingular matrix n n P F × whose q first columns are ~v1,...,~vq. Then P satisfies (4). ∈ T n q Step 4: Apply Case 1 to the resulting matrices Cˆ ,..., Cˆm H − . 1 ∈ Note that (see also Lemma 10 in Appendix Appendix A)
det C(λ)=0 λ Rm max rankC(λ) λ Rm < n, ∀ ∈ ⇐⇒ { | ∈ } and the later condition is easier checked in practice. We hence prefer Algorithm 3 below for checking Step 1 of Algorithm 2. Since the set of n n Hermitian matrices is the Hermitian part of the -algebra of n n complex matrices.× Thanks to Schm¨udgen’s procedure for diagonalizing∗ a Hermitian× matrix over a commuta- tive -algebra [21], we provide a procedure for finding such a maximum rank of ∗ C(λ). This technique may be possible to apply to some other types of matrices, for example, complex symmetric ones [4].
2.3 Finding maximum rank of a Hermitian pencil Schm¨udgen’s procedure [21] is summarized as follows: for F Hn partitioned as ∈ α β F = , (α R), β∗ Fˆ ∈ we then have the following relations
2 4 X+X = X X+ = α I, α F = X+FX+∗ , F = X FX∗ , (8) − − − − 9 e e where
α 0 α3 0 α3 0 X = , F = := Hn. (9) ± β∗ αI 0 α(αFˆ β∗β) 0 F1 ∈ ± − e We now apply the above to the pencil F = C(λ)= λ1C1 + ... + λmCm, where n m Ci H , λ R . In our situation of Hermitian matrices, we have have the fol- lowing∈ together∈ with a constructive proof that leads to a procedure for determining a maximum rank linear combination.
n n m Lemma 5. Let C = C(λ) F[λ] × , λ R , be a pencil satisfying C∗ = C. ∈ ∈ n n Then there exist polynomial matrices X+, X F[λ] × and polynomials b, dj − F[λ], j =1, . . . , n, such that ∈ ∈
2 X+X = X X+ = b In, (10a) − − 4C X X b = +diag(d1,...,dn) +∗ , (10b)
X CX∗ = diag(d1,...,dn). (10c) − − Proof. The lemma is constructively proved as follows. The procedure in this proof will stop when Ck is diagonal. It is shown in, eg., [21] and [5], that if the (1, 1)st entry of C is zero then one can find a nonsingular matrix P for that of PCP∗ is nonzero. In logically similar way, we can assume every matrix Ck which is applied Schm¨udgen’s procedure at every step below has nonzero (1, 1)st entry. At the first step, partitioning C = C∗ as
C α β Cˆ Cˆ F(n 1) (n 1) R = , 1∗ = 1 − × − , 0 = α [λ]. (11) β∗ Cˆ ∈ 6 ∈ 1 n 1 Assigning α = α, β = β, C = α (α Cˆ β∗β ) H − and 1 1 1 1 1 1 − 1 1 ∈
α1 0 X1 := Y1 (λ)= ± ± β1∗ α1In 1 ± − as in (9), then
X X X X 2 1+ 1 = 1 1+ = α1In, − 3− C α1 0 C˜ 4C X C˜ X X1 X1∗ = := 1, α1 = 1+ 1 1+∗ . − 0 C − 1 α β If C is diagonal then one stops. Otherwise, let us partition C = 2 2 and 1 1 β Cˆ 2∗ 2 continue applying Schm¨udgen’s procedure to C1 in the second step
3 Y α2 0 Y C Y α2 0 C Hn 2 2 = , 2 1 2∗ = C , 2 − , ± β2∗ α2In 1 − − 0 2 ∈ ± − C C Cˆ where α2 = 2(1, 1), the (1, 1)st entry of 2 = α2(α2 2 β2∗β2). The updated matrices − α2 0 α2 0 X2 = X1 , X2+ = X1+ − 0 Y2 − 0 Y2+ −
10 and
3 2 α1α2 0 0 2 3 X CX 3 α2diag(α1,α2) 0 C˜ 2 2∗ = 0 α2 0 = = 2 − − 0 C2 0 0 C2 X X X X 2 2 2 then satisfy the relations (8): 2 2+ = 2+ 2 = α1α2I = b I. The second step completes. − − Suppose now we have at the (k 1)th step that − X CX diag(d1,...,dk 1) 0 C˜ (k 1) (∗k 1) = − := k 1, − − − − 0 Ck 1 − − C C F (n k+1) (n k+1) where k = k∗ [λ] − × − , and d1,...,dk 1 are all not identically ∈ − zero. If Ck 1 is not diagonal (and suppose that its (1, 1)st entry is nonzero) then − partition Ck 1 and compute as follows: −
C αk βk C C Cˆ k 1 = , αk = k 1(1, 1), k = αk 1(αk 1 k βk∗ 1βk 1), − β∗ Cˆ k − − − − − − k αkIk 1 0 αkIk 1 0 Xk+ = X(k 1)+ − , Xk = − X(k 1) , − 0 Yk+ − 0 Yk − − − C˜ diag (d1,...,dk 1,dk) 0 X CX k = − = k k∗ , 0 C − k − k
b = αt, (12) t=1 Y where k 3 3 2 dk = α , dj = α α , j =1,...,k 1. (13) k j t − t=j+1 Y The procedure will stop if Ck is diagonal, and one picks X = Xk that diagonal- izes C as in (10c). ± ± The following allows us to determine a maximum rank linear combination.
Corollary 6. With notations as in Lemma 5, and suppose the procedure stops at step k. That is, Ck in (12) is diagonal but so are not Ct for all t =1,...,k 1. −
i) Assume further that the diagonal matrix C˜ k has the form as the right-hand m side of (10c). For λ R , if dj(λ) =0 for some j =1,...,n then dt(λ) =0 ∈ 6 6 for every t =1, . . . , j.
ii) The pencil C(λ) has maximum rank r if and only if dj is identically zero for m r all j = r +1, . . . , n, and there exists λˆ R such that b(λˆ) dt(λˆ) =0. ∈ t=1 6 Q Proof. i) As shown in (12), the fact Ct = αt(αtCˆ t βtβt) and αt = Ct(1, 1) +1 − +1 for every t =1,...,k 1, implies that αt divides αt . In particular, αk is divisible − +1 by αt for every t = 1, . . . , k. Moreover, αk divides ds for every s = k +1, . . . , n, provided by the first-row of (12). The claim is hence immediately followed.
11 ii) If dj 0 for some j>r then, by the previous part, dj(λˆ) = 0, and so are 6≡ m 6 dt(λˆ), t j, for some λˆ R . This means C(λˆ) has rank j > r, a contradiction. ≤ ∈ Thus dj 0 for every j > r. ≡ Now, if k>r then dk = αk 0. This is impossible because the procedure ≡ proceeds only when Ct(1, 1) = 0 at each step t. This yields k r. This certainly 6 ≤ implies b(λˆ) =0 since b = α1 αk. The opposite6 direction is obvious.···
Remark 1. The proof of Lemma 5 provides a comprehensive update according to Schm¨udgen’s procedure. But in our situation, only the diagonal elements of m C˜ k, from which one can determine a λˆ R at the end, are needed. So, in the computations of (12), one does not need to∈ update From (12), it suffices to find Corollary 6 only (10c) is needed. However, the last update formula in (12) shows that the diagonal elements of D are not simply updated. Indeed, at step k, we can update as follows:
Ik 1 0 αk = Ck 1(1, 1), b = bαk, Xk = − X(k 1) , (14) − − 0 X˜ − − − diag α3,...,α3 ,α3 0 C˜ = 1 k 1 k . k 0 − C k+1
Moreover, if Ck+1 has order greater than 1 then αt divides αt+1 for all t =1, . . . , k. Algorithm 3. Schm¨udgen-like algorithm determining maximum rank of a pencil. n INPUT: Hermitian matrices C ,...,Cm H . 1 ∈ OUTPUT: A real m-tuple λˆ Rm that maximizes the rank of the pencil C(λ). ∈ Step 1: Partitioning C(λ) as in (11).
Step 2: At the iteration k 1 : ≥ + If Ck is not diagonal then do the computations as in (12) and go to the iteration k.
+ Else (Ck is diagonal), take into account C˜ k is diagonal whose first k elements d1,...,dk are not identically zero and go to Step 3.
m Step 3: Pick λˆ R such that dk(λˆ) =0 and return λ.ˆ ∈ 6 2.4 Examples Example 1. Let
1 3 2 00 0 1 3 2 − − − C1 = 3 16 10 ,C2 = 0 3 2 ,C3 = 3 5 4 . 2 10− 6 0− 2 1 −2− 4 3 − − − − One can check that ker C1 ker C2 C3 = 0 since rankC1 =3. We consider two matrices∩ ∩ { } 0 1 1 1 5 3 1 − 1 − − M2 = C1− C2 = 0 1 0 , M3 = C1− C3 = 0 0 0 , 0 −1 1 0 1 1 − − 2 − 2 12 which has distinct eigenvalues 1, 1 , 0 and 1, 1 , 0, respectively, and hence they − 2 − − 2 are diagonalizable via similarity. Moreover, M2 and M3 commute. Theorem 4 yields that C ,C ,C are SDC. 1 2 3 ⋄ We now consider other examples in which all given matrices are singular. Example 2. The matrices 1 3 1 0 0 0 1 3 2 − − − C = 3 6 0 , C = 0 3 2 , C = 3 5 4 . 1 2 − 3 − − 1 0 2 0 2 1 2 4 3 − − − − are all singular since rank(C1)= rank(C2)= rank(C3)=2. T On the other hand, dim(ker C1 ker C2 C3)=0 since rank[C1 C2 C3] =3. Consider the linear combination∩ ∩ x z 3x 3z 2z x − − − C = xC + yC + zC = 3x 3z 6x 3y 5z 2y +4z . 1 2 3 − − − 2z x 2y +4z 2x y 3z − − − − Applying Scm¨udgen’s procedure we have x z 0 0 (x z)3 0 − X CX∗ = − , X = 3z 3x x z 0 , − − 0 C1 − − − x 2z 0 x z − − where 3y 6x +5z 9(x z)2 (3x +2y 2z)(x z) A =(x z) − − − − − − (3x +2y 2z)(x z) (x 2z)2 (x z)(2x + y +3z) − − − − − − α β := . β γ Let 1 0 0 Xi = 0 α 0 . − 0 β α − We then have (x z)3 0 0 X X CX X − 3 i ( ∗ ) i∗ = 0 α 0 , − − − − 0 0 α(αγ β2) − where
α =(x z)[3y 6x +5z 9(x z)2], − − − − β = (3x +2y 2z)(x z)2, − − γ = (x z)(x 2z)2 (x z)2(2x + y +3z) − − − − − If we pick (x, y, z)=(2, 0, 3) then α =6, β =0,γ =3, and α(αγ β) = 108 = 0. Then − 6 1 0 0 1 3 4 − − − X = 3 1 0 , C =2C1 +3C3 = 3 3 12 , rankC =3. − 4− 0 1 −4− 12 13 − − − 13 Note that for β =0 then we do notneed to compute Xi since A is diagonal. In this 1 1 1 − 1 1 case, (C)− C1, (C)− C2, (C)− C3 all have real eigenvalue but (C− C1)(C− C2) = 1 1 6 (C− C )(C− C ), the initial matrices are not SDC by Theorem 4. 2 1 ⋄ Example 3. The matrices
1 4 4 00 0 1 3 2 − − − − C1 = 4 16 16 ,C2 = 0 1 2 ,C3 = 3 9 6 . −4− 16 16 0− 2 4 −2− 6 4 − − − are all singular and dim(ker C1 ker C2 ker C3)=1. This intersection is spanned by x =( 4, 2, 1). ∩ ∩ Consider− the linear combination x z 4x 3z 4x +2z − − − − C = xC + yC + zC = 4x 3z 16x y 9z 16x +2y +6z . 1 2 3 −4x +2− z −16x +2−y +6− z 16x 4y 4z − − − Applying Schm¨udgen’s procedure we have
x z 0 0 ( x z)3 0 − − X CX∗ = − − , X = 4x 3z x z 0 , − − 0 A − − − − − 4x +2z 0 x z − − where xy + yz + zx 2(xy + yz + zx) α β A =( x z) − := . − − 2(xy + yz + zx) 4(xy + yz + zx) β γ − Let 1 0 0 Xi = 0 α 0 . − 0 β α − We then have ( x z)3 0 0 X X CX X − − 3 i ( ∗ ) i∗ = 0 α 0 , − − − − 0 0 α(αγ β2) − where
α =( x z)(xy + yz + zx), − − β = 2( x z)(xy + yz + zx), − − γ = 4( x z)(xy + yz + zx). − − It is easy to check that αγ β2 =0 for all x, y, z. then we have α(αγ β2)=0. On the other hand, we− have − 1 0 0 2 Xi Xi+ = Xi+Xi = 0 α 0 , − − 0 0 α2 3 The procedure stops. We have r = rankC(λ)=2. Since i=1 ker Ci = x = 3 ∩ { ( 4a, 2a, a)/a R , dim( ker Ci)=1. − ∈ } ∩i=1 14 Pick 1 0 4 − Q = 0 1 2 , 4 2 1 − then 225 180 0 − 225 180 Q∗C1Q = 180 144 0 , A1 = − − 180 144 0 00 − 64 40 0 − 64 40 Q∗C2Q = 40 25 0 , A2 = − − 40 25 0 00 − 49 49 0 − 49 49 Q∗C3Q = 49 49 0 , A3 = − . − 49 49 0 00 − We have 15 9 A := A(λ)= A + A = − 2 3 9 24 − and detA(λ)= 441 =0. On the other− hand,6 8 5 60 48 1 −3 −3 1 7 7 A− A2 = 8 5 ; A− A1 = 75 60 ; −3 3 7 7 288 20 285 228 − − − − 1 1 7 3 1 1 7 7 A− A1.A− A2 = 360 25 ; A− A2.A− A1 = 35 ; − − − 4 7 7 7 By Theorem 4, A1, A2, A3 are not SDC. We conclude, therefore, that C1,C2,C3 are not SDC. ⋄ 3 Completely solving the Hermitian SDC problem
As a consequence of Theorem 3, every commuting collection of Hermitian matrices can be SDC. However, this is just a sufficient but not necessary condition. For example, it is shown in [19] that the matrices 1 2 0 12 0 2 4 0 − − C = 2 28 0 , C = 2 20 0 , C = 4 1 0 1 2 3 −0− 0 5 0 0 3 0 0 7 − are SDC by 1 0 2 − P = 00 1 01 0 but C1C2 = C2C1. The following provides some equivalent SDC conditions for Hemitian matrices.6 It turns out that the SDC property of a collection of such ma- trices is equivalent to the feasibility of a positive semidefinite program (SDP). This
15 also allows us to use SDP solvers, for example, “CVX” [10], ...to check the SDC property of Hermitian matrices. We first provide some equivalent conditions one of which, see the condition (iv), leads to our main Algorithm 5 below. Theorem 7. The following conditions are equivalent:
n (i) Matrices C ,...,Cm H are SDC. 1 ∈ n n (ii) There exists a nonsingular matrix P C × such that P ∗C1P,...,P ∗CmP are commuting. ∈
n (iii) There exists a positive definite matrix Q = Q∗ H such that QC1Q,...,QCmQ are commuting. ∈
n (iv) There exists a positive definite X = X∗ H solves the following system of m(m+1) ∈ 2 linear equations
CiXCj = CjXCi, 1 i < j n. (15) ∀ ≤ ≤
If C1,...,Cm are real then so are all other matrices in the theorem.
Proof. (i ii). If C1,...,Cm are SDC then P ∗CiP is diagonal for every i = 1, . . . , m,⇒for some nonsingular matrix P. This yields the later matrices are com- muting.. (ii iii). Applying polar decomposition to P, P = QU (Q = Q∗ positive definite, U ⇒unitary), we have
(U ∗Q∗CiQU)(U ∗Q∗CjQU)=(P ∗CiP )(P ∗CjP )=(P ∗CjP )(P ∗CiP )
=(U ∗Q∗CjQU)(U ∗Q∗CiQU),
and hence (QCiQ) and QCjQ commute. (iii i). This implication is clear by Theorem 3. (iii⇒iv). The existence of a positive definite matrix P such that ⇔
(PCiP )(PCjP )=(PCjP )(PCiP ), i = j, ∀ 6 2 2 in other words CiP Cj = CjP Ci, is equivalent to which of the positive defi- nite matrix X satisfying CiXCj = CjXCi, for all i = j. It is clear that the last equations are just linear in X. Conversely, if X is positive6 definite which satisfies CiXCj = CjXCi, for all i = j, then P is just picked as the square root of X. 6 3.1 The algorithm Based on Theorems 3 and 7, our algorithm consists of two stages: (1) detecting whether the given hermtian matrices are SDC by solving the linear system (15), and obtaining a commuting Hermitian matrices; (2) simultaneously diagonalizing via congruence the resulting matrices. It may apply Algorithm 1 to perform the second stage. However, to proceed Step 1 of Algorithm 1, one needs to compute eigenvalue decomposition of all ma- trices C1,...,Cm. This may present a high complexity. Our main algorithm (Al- gorithm 5) then prefer Algorithm 4 below to Algorithm 1 for the second stage. It
16 will exploit the works in [4, 17], where the work [3] deals with a simultaneous diagonalization of two commuting normal matrices and the very recent work [17] extends to more (commuting normal) matrices with a performance in MATLAB. (i) n This extension can be summarized as follows. Suppose Ci = [cuv] H and let ∈ m (i) 2 off = off (C ,...,Cm)= c , (16a) 2 2 1 | uv| i=1 u=v 6 T X XT T T R(u,v,c,s)= In +(c 1)eue s¯eue + seve +(¯c 1)eve , (16b) − u − v u − v where u, v = 1,...,n and c,s C with c 2 + s 2 = 1. It can be verified that for a given pair (c,s) and every pair∈(u, v) | 1| ,...,n| | 2, the following holds true: ∈{ } m (i) 2 (i) 2 off (RC R∗,...,RCmR∗)=off (C ,...,Cm) c + c 2 1 2 1 − | uv| | vu| i=1 m X 2 + c2c¯(i) + cs(¯c(i) c¯(i)) s2c¯(i) uv uu − vv − vu i=1 Xm 2 + c2c(i) + cs(c(i) c(i)) s2c(i) . (17) vu uu − vv − uv i=1 X
In their method [3, 17], at the loop w.r.t each (u, v), it tries to find c,s that makes off2(RC1R∗,...,RCmR∗) < off2(C1,...,Cm). It is shown in, eg., [9], that c,s can be looked for that minimize the last sum on the right hand-side of (17). This is equivalent to that solve that minimize the amount Muvz with k k2 (1) (1) (1) (1) c¯uv (¯cuu c¯vv ) c¯vu (1) (1) − (1) − (1) cvu (cuu cvv ) cvu c2 . −. − . Muv = . . . , z = sc . (m) (m) (m) (m) s2 c¯uv (¯cuu c¯vv ) c¯vu (m) (m) − (m) − (m) cvu (cuu cvv ) cvu − − iφuv Note that z 2 =1 and c,s can be parameterized as c = cos(θuv),s = e sin(θuv), k k π π (θuv,φuv) [ , ] [ π, π]. ∈ − 4 4 × − Algorithm 4. SDC commuting Hermitian matrices. n n INPUT: Commuting Hermitian matrices C1,...,Cm C × , a tolerance ǫ> 0. m ∈ OUTPUT: A unitary matrix U such that off ǫ Ci F := ν(ǫ). 2 ≤ i=1 k k Step 1. Accumulate Q = In. P Step 2. WHILE off2 > ν(ǫ) (i) For every pair (u, v), 1 u < v n, determine the rotation R(u,v,c,s) ≤ iφ ≤ with (c,s) = (cos θuv, e sin θuv) being the solution to the problem 2 2 T min Muvz : z = [c sc s ] above. {k k2 }
(ii) Accumulate Q = QR(u,v,c,s), Ci = R(u,v,c,s)∗CiR(u,v,c,s), i = 1, . . . , m.
What we have discussed leads to the main algorithm as follows.
17 Algorithm 5. Solving the Hermitian SDC problem. n n INPUT: Hermitian matrices C ,...,Cm C × (not necessary commuting). 1 ∈ OUTPUT: A nonsingular matrix U such that U ∗CiUP ’s are diagonal (if exists).
Step 1: If the system (15) has no a positive definite solution P, conclude the initial matrices are not SDC. Otherwise, compute the square root Q of P, Q2 = P and go to Step 2.
Step 2: Apply Algorithm 4 to find a unitary matrix U that simultaneously diago- nalizes the matrices I, Q∗C1Q,...,Q∗CmQ. Then U = QV.
Remark 2. In Algorithm 5 , the output data will be automatically real if the input i is real. In this situation, the input matrices are all real symmetric. Let cuv be the n (u, v)th entry of Ci S . A positive definite matrix X = [xuv] that solves (15) will equivalently satisfy∈
X 0, Tr(AijX)=0, 1 i < j m, (18) ≻ ≤ ≤ ij n where Aij = [a ] S with uv ∈ n i j j i ij p,q=1(cpucqu cpucqu) if u = v, a = n − uv 2 (ci cj cj ci ) if u = v. P p,q=1 pu qv − pu qv 6 It is well known, see eg., in [16,P 2], that Sn Sn X + : Tr(AijX)=0, 1 i < j m = 0 span Aij 1 i
C1XC2 = C2XC1 (= (C1XC2)∗) . This is equivalent to x> 0, xz>y2 x + y + z =0. But the last condition is impossible since there do not exist x,z > 0 such that xz >y2 =(x + z)2. Thus C and C are not SDC on R. 1 2 ⋄ 18 Example 5. Reconsider the matrices in Example 1. To apply Theorem 7 or Algo- rithm 5 , we want to find x y z X = y t u 0 x> 0,xt>y2, det(X) > 0 (19) z u v ≻ ⇔ such that
C1XC2 =(C1XC2)∗, C1XC3 =(C1XC3)∗, C2XC3 =(C2XC3)∗. By directly computing, 12u 9t 4v 3y +2z =0 − − − C XC =(C XC )∗ 7u +6t +2v +2y z =0 1 2 1 2 − − ⇔ 2u +2t 2v +z =0, − 40u 33t 12v 11y +6z =0 − − − C XC =(C XC )∗ 7u 6t 2v 2y +z =0 1 3 1 3 ⇔ 18u −14t −6v −4y +3z =0, − − − 12u +9t +4v +3y 2z =0 − − C XC =(C XC )∗ 4u 2t 2v +z =0 2 3 2 3 ⇔ 7u −6t −2v 2y +z =0. − − − Combining the linear equations above we obtain z u =2y, t = y, v =3y + . 2 We then pick y =1, z =4, x =6 and 6 1 4 X = 1 1 2 0 4 2 5 ≻ makes √XC1√X, √XC2√X, √XC3√X to be commuting by Theorem 7. Thus three initial matrices are SDC on R, and so are they on C. ⋄ We now consider other examples of which all given matrices are singular. Example 6. For the matrices in Example 2, we will check if there exists X as in (19) satisfying the following: 9u 9t 2v 3y +2z =0 − − − C XC =(C XC )∗ 5u +6t +v +2y z =0 1 2 1 2 ⇔ −12u +12t +4v +3y −z =0, − − 5u 3t +4v y z =0 − − − − C XC =(C XC )∗ 19u 12t 7v x 7y +5z =0 1 3 1 3 − − − − ⇔ 28u 24t 8v 3x 19y +11z =0, − − − − 12u +9t +4v +3y 2z =0 − − C XC =(C XC )∗ 4u 2t 2v +z =0 2 3 2 3 − − ⇔ 7u 6t 2v 2y +z =0. − − − The system of linear equations combined by three conditions above has only trivial solution (since its argument matrix is full rank). This means only zero matrix satisfies (15). The given matrices are hence not SDC on R. ⋄ 19 Example 7. Let us turn to the matrices in Example 3. We will check if there exists X as in (19) satisfying the following:
y 2z +4t 12u +8v =0 − − C XC =(C XC )∗ 2y +4z 8t +24u 16v =0 1 2 1 2 − − − ⇔ 4y 8z +16t 48u +32v =0, − − x +7y 6z +12t 20u +8v =0 − − C XC =(C XC )∗ 2x +14y 12z +24t 40u +16v =0 1 3 1 3 − − ⇔ 4x +28y 24z +48t 80u +32v =0, − − y 2z +3t 8u +4v =0 − − C XC =(C XC )∗ 2y +4z 6t +16u 8v =0 2 3 2 3 − − − ⇔ 4y 8z +12t 32u +16v =0. − − The general solutions of these linear equations are of the form
x = 4y 16u +16v − − t = 4u 4v y − z = +2u 4v. 2 − We have
xt = 16y(u v) 64(u v)2 >y2 (y +8u 8v)2 < 0 − − − − ⇔ − There do not exist x, t such that xt > y2. This means there is no positive definite matrix satisfying (15).We have C ,C and C are not R-SDC. 1 2 3 ⋄ 3.3 Numerical tests In this section we give numerical tests illustrating Algorithm 5. There are several methods for computing a square root of a positive definite ma- trix. In our MATLAB experiments, we exploit the MATLAB function “sqrtm.m” to find a square root of matrix Q in Step 1. This function executes the algorithm provided in [7] for computing a square root of an arbitrary square matrix. Table 1 shows some numerical tests with respect to several values of m and n. Each re- sult is the average of three ones. The Hermitian matrices C1,...,Cm, of which the SDC property will afterwards be estimated, are constructed as Ci = P ∗DiP, where Di is diagonal and P is invertible that are randomly taken from a uniform distribution on the interval [0, 1). The backward errors are estimated as
U ∗CiU diag(diag(U ∗CiU)) Err = max k − k2 i =1,...,m , U C U ∗ i 2 k k where diag(diag(X)) denote the diagonal matrix whose diagonal is of X. For the Stage 2 w.r.t. Algorithm 4, we fix a tolerance to be the floating-point relative accu- ATLAB 3 racy “eps”ofM , to the power of 2 .
20 m n Err CPU time (s) 3 3 3.33e-12 6.59 10 20 8.64e-13 922.81 50 100 50 200 100 100 Table 1: Numerical tests the SDC property of collections of Hermitian matrices. 4 Conclusion and discussion
We have provided some equivalent conditions detecting whether a collection of Hermitian matrices can be simultaneously diagonalized via -congruence. One of these conditions leads to solve a positive semidefinite program∗ with a positive definite solution. Combining this with an existing Jacobi-like method for the si- multaneous diagonalization of commuting normal matrices, we propose an algo- rithm for detecting/computing a simultaneous diagonalization of a collection of non-commuting Hermitian matrices. We have also present some numerical tests for this main algorithm. However, not every collection of normal matrices can be SDC. In some applica- tions, for example the quadratically constrained quadratic programming, one may seek to approximate given matrices by simultaneously diagonalizable (via congru- ence) ones. One should addressed two possibilities for approximation in the future work: the first one make the initial matrices become pairwise commute, and other is that of immediately finding SDC matrices that “colse to” the initial ones.
Acknowledgement Appendix A
0 0 0 0 Lemma 9. The matrices C = k ,...,C = k are SDC if and 1 0 Cˆ m 0 Cˆ 1 m only if so are Cˆ1,..., Cˆm.
Proof. If Cˆ1,..., Cˆm are SDC by a nonsingular matrix Pˆ then C1,...,Cm are I 0 certainly SDC by the nonsingular matrix P = k . 0 Pˆ Conversely, suppose C1,...,Cm are SDC by a nonsingular matrix U. Partition- U1 U2 k k (n k) (n k) ing U = , U3 F × , U4 F − × − . Note that rank[U3 U4] = U3 U4 ∈ ∈ n k due to the nonsingularity of U. For every i =1, . . . , m, the following matrix − ˆ ˆ 0k 0 U3∗CiU3 U3∗CiU4 U ∗ U = 0 Cˆi U Cˆ U U Cˆ U 4∗ i 3 4∗ i 4 is diagonal. Since the later block matrix is diagonal, we can assume U4 is non- singular after multiplying on the right of U by an appropriate permutation matrix. ˆ This means U4∗CiU4 is diagonal for every i =1, . . . , m. n Lemma 10. Let C1,...,Cm H and denote C(λ) = λ1C1 + ... + λmCm, m ∈ λ =(λ ,...,λm) R . Then 1 ∈ 21 m C ∗ (i) λ Rm ker (λ)= i=1 ker Ci = ker C, where C = C1∗ ... Cm∗ . ∈ m (ii) maxT rankRC(λ) λT R rankRC. { | ∈ } ≤ m m C (iii) Suppose dimR( i=1 ker Ci) = k. Then i=1 ker Ci = ker (λ) for some m λ R if andonly if rankRC(λ) = maxλ Rm rankRC(λ)= rankRC = n k. ∈ T ∈T − Proof. The part (i) is easy to check. The part (ii) is followed from the fact that
C1 C1 C . . rankR (λ)= rankR λ1I ... λmI . rankR . = rankRC, ≤ Cm Cm for all λ Rm. ∈ m For the last part, with the help of the part (i), we have ker C = ker Ci i=1 ⊆ ker C(λ). Then by the part (ii), T m m
ker Ci = ker C(λ) dimF (ker C(λ)) = dimF ker Ci = n rankFC ⇐⇒ − i=1 i=1 ! \ \ m rankFC(λ)= rankFC = n k rankFC(λ), λ F . ⇐⇒ − ≥ ∀ ∈
This is certainly equivalent to n k = rankFC(λ) = maxλ Fm rankFC(λ). − ∈ References
[1] A. A. Albert. A quadratic form problem in the calculus of variations. Bulletin of the American Mathematical Society, 44:250–253, 1938. [2] A. Barvinok. A remark on the rank of positivesemidefinite matrices subject to affine constraints. Discrete & Computational Geometry, 25(1):23–31, 2001. [3] A. Bunse-Gerstner, R. Byers, and V. Mehrmann. Numerical methods for simultaneous diagonalization. SIAM Journal on Matrix Analysis and Appli- cations, 14(4):927–949, oct 1993. [4] M. D. Bustamante, P. Mellon, and M. V. Velasco. Solving the problem of simultaneous diagonalisation via congruence. SIAM Journal on Matrix Anal- ysis and Applications, 41:1616–1629, 2020. [5] J. Cimpriˇc. Real algebraic geometry for matrices over commutative rings. Journal of Algebra, 359:89–103, jun 2012. [6] L. De Lathauwer. A link between the canonical decomposition in multilinear algebra and simultaneous matrix diagonalization. SIAM Journal on Matrix Analysis and Applications, 28(3):642–666, 2006. [7] E. Deadman, N. J. Higham, and R. Ralha. Blocked Schur algorithms for com- puting the matrix square root. In P. Manninen and P. Oster,¨ editors, Lecture Notes in Computer Science, 7782, pages 171–182. Springer-Verlag, 2013. [8] P. Finsler. Uber¨ das vorkommen de niter und semide niter formen in scharen quadratischer formen. Commentarii Mathematici Helvetici, 9:188–192, 1937.
22 [9] H. H. Goldstine and L. P. Horwitz. A procedure for the diagonalization of normal matrices. Journal of the ACM, 6(2):176–195, 1959.
[10] M. Grant and S. P. Boyd. CVX: Matlab software for disciplined convex pro- gramming, version 1.21. http://cvxr.com/cvx, apr 2011.
[11] M. R. Hestenes and E. J McShane. A Theorem on Quadratic Forms and Its Application in the Calculus of Variations. Transactions of the American Mathematical Society, 47(3):501–512, 1940.
[12] J-B. Hiriart-Urruty. Potpourri of conjectures and open questions in nonlinear analysis and optimization. SIAM Review, 49(2):255–273, 2007.
[13] R. A. Horn and C. R. Johnson. Matrix analysis. Cambridge University Press, Cambridge, 1985.
[14] R. A. Horn and C. R. Johnson. Topics in Matrix Analysis. Cambridge Uni- versity Press, Cambridge, 1991.
[15] R. Jiang and D. Li. Simultaneous Diagonalization of Matrices and Its Appli- cations in Quadratically Constrained Quadratic Programming. SIAM Journal on Optimization, 26(3):1649–1668, 2016.
[16] I. Klep and M. Schweighofer. An exact duality theory for semidefinite pro- gramming based on sums of squares. Mathematics of Operations Research, 38(3):569–590, 2013.
[17] C. Mendl. simdiag.m. MATLAB Central File Exchange, 2020. Available at https://www.mathworks.com/matlabcentral/fileexchange/46794-simdiag-m.
[18] J. J. Mor´e. Generalization of the trust region problem. Optimization Methods and Software, 2:189–209, 1993.
[19] Thi-Ngan Nguyen, Van-Bong Nguyen, Thanh-Hieu Le, and Ruey-Lin Sheu. Simultaneous Diagonalization via congruence of matrices and some applica- tions in optimization. prepreint, page 30, 2020.
[20] T. K. Pong and H. Wolkowicz. The generalized trust region subproblem. Computational Optimization and Applications, 58:273–322, 2014.
[21] K. Schm¨udgen. Noncommutative real algebraic geometry-some basic con- cepts and first ideas. In M. Putinar and S. Sullivant, editors, Emerging Ap- plications of Algebraic Geometry, The IMA Volumes in Mathematics and its Applications, volume 149, pages 325–350. Springer New York, 2009.
[22] R. Shankar. Principles of quantum mechanics. Plenum Press, New York, 1994.
[23] F. Uhlig. Simultaneous block diagonalization of two real symmetric matrices. Linear Algebra and its Applications, 7(3):281–289, 1973.
[24] J. F. Watters. Simultaneous quasi-diagonalization of normal matrices. Linear Algebra and its Applications1, 9:103–117, 1974.
23 [25] K. Weierstrass. Zur Theorie der quadratischen und bilinearen Formen. Monatsbericht der Berliner Akademie der Wissenschaften, pages 21–40, 1868.
24