Swarthmore College Works
Mathematics & Statistics Faculty Works Mathematics & Statistics
2019
Schur's Lemma For Coupled Reducibility And Coupled Normality
D. Lahat
C. Jutten
Helene Shapiro Swarthmore College, [email protected]
Follow this and additional works at: https://works.swarthmore.edu/fac-math-stat
Part of the Mathematics Commons Let us know how access to these works benefits ouy
Recommended Citation D. Lahat, C. Jutten, and Helene Shapiro. (2019). "Schur's Lemma For Coupled Reducibility And Coupled Normality". SIAM Journal On Matrix Analysis And Applications. Volume 40, Issue 3. DOI: 10.1137/ 18M1232462 https://works.swarthmore.edu/fac-math-stat/257
This work is brought to you for free by Swarthmore College Libraries' Works. It has been accepted for inclusion in Mathematics & Statistics Faculty Works by an authorized administrator of Works. For more information, please contact [email protected]. arXiv:1811.08467v2 [math.RA] 29 Nov 2018 98 ([email protected]). 19081 02 R-d-264 IS-a saprnro h aE PERS LabEx the of partner a LABX-0025). is GIPSA-lab ERC-AdG-320684. 2012- okwscridotwe .Lhtwswt IS-a,Geol,F Grenoble, GIPSA-lab, with was Lahat D. when out carried was work utni eirmme fIsiu nv eFac (christian.jut France de Univ. Institut inp.fr). of member senior a is Jutten cu’ em o ope euiiiyand Reducibility Coupled for Lemma Schur’s ∗ † ‡ § Funding RT nvri´ eTuos,CR,Tuos,Fac (Dana.Lah France Toulouse, CNRS, Toulouse, Universit´e de IRIT, nv rnbeAps NS rnbeIP IS-a,300Greno 38000 GIPSA-lab, INP, Grenoble CNRS, Alpes, Grenoble Univ. eateto ahmtc n ttsis wrhoeClee S College, Swarthmore Statistics, and Mathematics of Department ∗ omlt n ope reuiiiyconditions. irreducibility coupled and normality eas osdrarfieeto cu’ em o eso norm of sets for Lemma for Schur’s versions corresponding of prove refinement and matrices a consider also We cu’ em for Lemma Schur’s size arcs where matrices, o all for one aiyo arcs where matrices, of family an sense aaLahat Dana n Let n i i dmninlvco pc.W say We space. vector -dimensional i h oko .LhtadC utnwsspotdb h projec the by supported was Jutten C. and Lahat D. of work The : I ∈ fteeeitsubspaces, exist there if × ,j i, A m Let . = and , i Suppose . { A ij † B U B } ope Normality Coupled ij i i,j = 6= A ∈I is , A { hita Jutten Christian where , m V eebr3 2018 3, December B B ij i i ij X aifigculdirdcblt conditions. irreducibility coupled satisfying o tlatone least at for × } A j i,j m ij = Abstract ∈I j I o each For . is X U sa ne e,b obyindexed doubly a be set, index an is i lob obyidxdfml of family indexed doubly a be also i n B 1 i V ⊆ ij × o all for n i with , j A o each For . i i I ∈ uhthat such , is ‡ ,j i, euil ntecoupled the in reducible A let , epoevrin of versions prove We . U , i B eeeShapiro Helene 6= aifigcoupled satisfying X i { i [email protected] 0 I ∈ eamti of matrix a be } A VLLb(ANR11- YVAL-Lab ij o tleast at for let , rance. ( U tii.r.This [email protected]). atmr,PA warthmore, j l,Fac.C. France. ble, ) V U ⊆ i CHESS, t be al i § 1 Introduction
Let K be a positive integer and let n1,...,nK and m1,...,mK be positive K integers. Consider two doubly indexed families of matrices, = Aij i,j=1 K A { K} and = Bij i,j=1, where Aij is ni nj and Bij is mi mj. Put N = i=1 ni B { K} × × and M = mi. Arrange the Aij’s into an N N matrix, A, withPAij in i=1 × block i, j ofPA. A A A K 11 12 ··· 1 A21 A22 A2K A = . . ··· . . . . . AK AK AKK 1 2 ··· Similarly, form an M M matrix B with Bij in block i, j. Let Xi be an × ni mi matrix and form an N M matrix X with blocks X1,...,XK down the× diagonal and zero blocks elsewhere.× Thus,
X1 0 0 0 0 X 0 ··· 0 2 ··· X = 0 0 X3 0 , . . . ··· . . . . . 0 0 0 XK ··· where the 0 in position i, j represents an ni mi block of zeroes. Then × AX = XB if and only if AijXj = XiBij, (1) for all i, j = 1,...,K. We may rewrite AX = XB as AX XB = 0, a homogeneous Sylvester equation [13]. − We define several versions of coupled reducibility and prove corresponding versions of Schur’s Lemma [12] for pairs , . Imposing coupled irreducibil- A B ity constraints on and restricts the possible solutions, X1,...,XK, to the equations (1).A We alsoB discuss a refinement of Schur’s Lemma for nor- mal matrices, and prove corresponding versions for , satisfying coupled A B normality conditions. The system of coupled matrix equations (1) arises in a recent model for multiset data analysis [5, 8], called joint independent subspace analysis, or JISA. This model consists of K datasets, where each dataset is an un- known mixture of several latent stochastic multivariate signals. The blocks of A and B represent statistical links among these datasets. More specifi- cally, Aij and Bij represent statistical correlations among latent signals in the
2 ith and jth datasets. The multiset joint analysis framework, as opposed to the analysis of K distinct unrelated datasets, arises when a sufficient number of cross-correlations among datasets, i.e., Aij and Bij, i = j, are not zero. It turns out [6, 9, 10] that the identifiability and uniqueness6 of this model, in its simplest form, boil down to characterizing the set of solutions to the system of matrix equations (1), when the cross-correlations among the latent signals are subject to coupled (ir)reducibility. In other words, the coupled reducibility conditions that we introduce in this paper can be attributed with a physical meaning, and can be applied to real-world problems. We refer the reader to [6, 10] for further details on the JISA model, and on the derivation of (1). In this paper, we consider scenarios more general than those required to address the signal processing problem in [6, 10]. The analysis in Section 6, which focuses on coupled normal matrices, is inspired by the JISA model in [6, 10], in which the matrices A and B are Hermitian, and thus a spe- cial case of coupled normal. A limited version of some of the results in this manuscript was presented orally in, e.g., [7, 4, 1] and in a technical report [6]; however, they were never published. While motivated by the matrix equation, AX = XB, the main defi- nitions, theorems, and proofs do not require K to be finite. Thus, for a general index set, , we consider doubly indexed families, = Aij i,j I A { } ∈I and = Bij i,j . For the situation described above, = 1, 2,...,K . ∈I WeB use{ F }to denote the field of scalars. For some results,I { F can be} any field. For results involving unitary and normal matrices, F = C, the field of complex numbers. For each i , let ni and mi be positive integers, let i ∈ I V be an ni-dimensional vector space over F, and let i be an mi-dimensional ni W mi vector space over F. (Essentially, i = F and i = F .) For all i, j , V W ∈ I let Aij be an ni nj matrix and let Bij be mi mj. View Aij as a linear × × transformation from j to i, and Bij as a linear transformation from j V V W to i. For each i , let Xi be an ni mi matrix; view Xi as a linear W ∈ I × transformation from i to i. We are interested in families , satisfying the equations (1) forW all i, j V . A B ∈ I For some results, all of the ni’s will be equal, and all of the mi’s will be equal. In this case, we use n for the common value of the ni’s and m for the n m common value of the mi’s. We then set = F and = F . Each Xi is then n m. For = 1,...,K , we haveV N = Kn, andW M = Km. All of × I { } the blocks Aij in A are n n, while all of the blocks Bij in B are m m. × × Section 2 reviews the usual matrix version of Schur’s Lemma and its proof. Section 3 defines coupled reducibility and two restricted versions,
3 called proper and strong reducibility. Section 4 states and proves versions of Schur’s Lemma for coupled reducibility and proper reducibility, Theorem 4.2. In section 5, we define some graphs associated with the pair , and use them for versions of Schur’s Lemma corresponding to stronglyA couB pled re- ducibility, Theorems 5.1 and 5.2. Section 6 deals with a refinement of Schur’s Lemma for sets of normal matrices and corresponding versions for pairs , which are coupled normal, Theorem 6.2. The Appendix presents examplesA B to support some claims made in Section 3.
2 Reducibility and Schur’s Lemma
We begin by reviewing the ordinary notion of reducibility for a set of linear transformations and Schur’s Lemma.
Definition 2.1. A set, , of linear transformations, on an n-dimensional T vector space, , is reducible if there is a proper, non-zero subspace of such that T ( V) for all T . If is not reducible, we sayU itV is irreducible. U ⊆ U ∈ T T
The subspace is an invariant subspace for each transformation T in . We say is fullyU reducible if it is possible to decompose as a direct sumT T V = ˆ, where and ˆ are both nonzero, proper invariant subspaces ofV .U ⊕ U U U TAlternatively, one can state this in matrix terms. The linear transforma- tions in may be represented as n n matrices, relative to a choice of basis for . LetT d be the dimension of ;× choose a basis for in which the first d basisV vectors are a basis for . SinceU is an invariantV subspace of each T U U in , the matrices representing relative to this basis are block upper tri- angularT with square diagonal blocksT of sizes d d and (n d) (n d). The × − × − first diagonal block, of size d d, represents the action of the transformation on the invariant subspace .× The (n d) d block in the lower left hand corner consists of zeroes.U Since changing− basis× is equivalent to applying a matrix similarity, we have the following matrix version of Definition 2.1.
Definition 2.2. (Matrix version of reducibility.) A set of n n matrices M × is reducible if, for some d, with 0 4 When is fully reducible, we can use a basis for in which the first d basis vectorsT are a basis for , and the remaining n Vd basis vectors are a U − basis for ˆ. The corresponding matrices for are then block diagonal, with diagonalU blocks of sizes d d and (n d) T(n d). If F = C and is reducible,× the S− in Definition× − 2.2 can be chosen to be M a unitary matrix, by using an orthonormal basis for Cn in which the first d basis vectors are an orthonormal basis for . If is fully reducible, and U M the subspaces and ˆ are orthogonal, use an orthonormal basis of for the first d columnsU of S andU an orthonormal basis of ˆ for the remainingU n d 1 U − columns. Then S is unitary, and S− MS is block diagonal, with diagonal blocks of sizes d d and (n d) (n d). The following× fact is well− known;× we− include the proof because the idea is used later. For this fact, we assume we are working over C, or at least over a field that contains the eigenvalues of the transformations. Proposition 2.1. Let be an irreducible set of n n complex matrices. M × Suppose the n n matrix C commutes with every element of . Then C is a scalar matrix.× M Proof. Let λ be an eigenvalue of C, and let λ denote the corresponding U eigenspace. Let A . For any v λ, we have C(Av)= A(Cv)= λ(Av). ∈M ∈ U Hence Av is in λ, and so λ is invariant under each element of . Since U U M n an eigenspace is nonzero, and is irreducible, we must have λ = C . M U Hence C = λIn. Schur’s Lemma plays a key role in group representation theory. It is used to establish uniqueness of the decomposition of a representation of a finite group into a sum of irreducible representations. However, one need not have a matrix group; the result holds for irreducible sets of matrices. We include the usual proof [2], because the same idea is used to prove our versions for coupled reducibility. Theorem 2.1 (Schur’s Lemma). Let Ai i be an irreducible family of { } ∈I n n matrices, and let Bi i be an irreducible family of m m matrices. × { } ∈I × Suppose P is an n m matrix such that AiP = P Bi for all i . Then, × ∈ I either P = 0, or P is nonsingular; in the latter case we must have m = n. For matrices of complex numbers, if Ai = Bi for all i , then P is a scalar matrix. ∈ I 5 Proof. View the Ai’s as linear transformations on an n-dimensional vector space , and the Bi’s as linear transformations on an m-dimensional vector V space . The n m matrix P represents a linear transformation from to . SoW ker(P ) is× a subspace of and range(P ) is a subspace of . W V W V Let w ker(P ). Then P (Biw)= AiP w = 0. Hence, ker(P ) is invariant ∈ under Bi i . Since Bi i is irreducible, ker(P ) is either 0 or . In ∈I ∈I the latter{ } case, P = 0.{ If} P = 0, then ker(P ) = 0 . Now{ consider} W the 6 { } range space of P . For any w , we have Ai(P w)= P (Biw) range(P ), ∈ W ∈ so the range space of P is invariant under Ai for each i. Since Ai i ∈I is irreducible, range(P ) is either 0 or . But we are assuming {P =} 0 { } V 6 so range(P ) = . Since we also have ker(P ) = 0 , the matrix P must be nonsingular andVm = n. { } If Ai = Bi for all i , then P commutes with each Ai. If each Ai ∈ I is a complex matrix, then, since Ai i is irreducible, P must be a scalar ∈I matrix. { } 1 For nonsingular P , we have P − AiP = Bi for all i , so Ai i and Bi i ∈ I { } ∈I { } ∈I are simultaneously similar. 3 Coupled Reducibility For simultaneous similarity of Ai i and Bi i , there is a nonsingular 1 { } ∈I { } ∈I matrix, P , such that P − AiP = Bi for all i. We now define a “coupled” version of similarity for two doubly indexed families, and , with ni = mi A B for all i . In this case, Aij and Bij are matrices of the same size, and in ∈ I the equations (1), each Xi is a square matrix. Definition 3.1. Let = Aij i,j I and = Bij i,j I, where ni = mi for ∈ ∈ all i . We say Aand { are} similar inB the{ coupled} sense if there exist ∈ I A B 1 nonsingular matrices Ti i I , where Ti is ni ni, such that Ti− AijTj = Bij { } ∈ × for all i, j . ∈ I For a finite index set = 1,...,K , this can be stated in terms of the I { } matrices, A and B. Let T be the block diagonal matrix T T TK . 1 ⊕ 2 ⊕···⊕ Then AT = T B if and only if AijTj = TiBij for all i, j. Hence, and are A 1 B similar in the coupled sense if and only if T is nonsingular and T − AT = B. We define several versions of “reducible in the coupled sense” for a doubly indexed family . The basic idea is that there are subspaces, i i , where A {U } ∈I 6 i i, such that Aij( j) i for all i, j . This holds trivially when i U ⊆V U ⊆ U ∈ I U is zero for all i, and when i = i for all i, so we shall insist that at least one U V subspace is nonzero, and at least one is not i. We are also interested in two V more restrictive versions: the case where at least one i is a nonzero, proper U subspace, and the case where every i is a nonzero, proper subspace of i. U V Definition 3.2. Let = Aij i,j I where Aij is ni nj. We say is A { } ∈ × A reducible in the coupled sense if there exist subspaces i i , where i i, {U } ∈I U ⊆V such that the following hold. 1. For at least one i, we have i = 0 . U 6 { } 2. For at least one i, we have i = i. U 6 V 3. For all i, j we have Aij( j) i. ∈ I U ⊆ U We say i i is a reducing set of subspaces for , or that is reduced by {U } ∈I A A i i . If is not reducible in the coupled sense, we say it is irreducible ∈I in{U the} coupledA sense. We say is properly reducible in the coupled sense if A at least one i is a nonzero, proper subspace of i. We say is strongly U V A reducible in the coupled sense if every i is a nonzero, proper subspace of i. U V Remark 3.1. If ni = 1 for all i, the one-dimensional spaces i have no V nonzero proper invariant subspaces, so cannot be properly or strongly irreducible. If K = 1, then consists ofA a single n n matrix. A × Remark 3.2. If ni = n for all i, and the subspaces i i are all the same {U } ∈I nonzero proper subspace, i.e, for all i, we have i = where is a nonzero, proper subspace of , then is reducible inU the ordinaryU U sense, given in V A Definition 2.1. Note the following facts. 1. If j = 0 , then Aij( j) i holds for any Aij and any i. U { } U ⊆ U U 2. If i = i, then Aij( j) i holds for any Aij and any j. U V U ⊆ U U 3. If j = j and i = 0 , then Aij = 0. U V U { } 4. For i = j, we have Aii( i) i, so i is an invariant subspace of Aii. U ⊆ U U 7 An equivalent matrix version of Definition 3.2 is obtained by choosing an appropriate basis for each j. Let dj be the dimension of the subspace j. V U We have 0 dj nj. If dj is positive, let vj, ,..., vj,d be a basis for j and ≤ ≤ 1 j U let Tj be a nonsingular nj nj matrix which has vj, ,..., vj,d in the first dj × 1 j columns. If dj = 0, we may use any nonsingular nj nj matrix for Tj. Set 1 × Bij = Ti− AijTj; equivalently, AijTj = TiBij. The first dj columns of Tj are a basis for j, so Aij( j) i tells us the U U ⊆ U first dj columns of AijTj are in i. Hence, the first dj columns of TiBij U are in i, so by the definition of Ti, each of the first dj columns of TiBij U is a linear combination of the first di columns of Ti. Therefore, each of the first dj columns of Bij will have zeroes in all entries below row di, and the lower left hand corner of Bij is a block of zeroes of size (ni di) dj. When − × 0 CD Bij = , (2) 0(n di) dj E − × where C is size di dj and represents the action of Aij on the subspace j. × U The zero block in the lower left hand block has size (n di) dj, while D is − × di (nj dj) and E is (ni di) (nj dj). × − − × − Remark 3.3. The block matrix (2) has a block of zeroes in the lower left hand corner of size (n di) dj. We also use this terminology for the cases − × di = 0, di = ni, dj = 0, dj = nj, in which case we mean the following. If di = 0, the first dj columns of Bij are zero. If di = ni there is no restriction on the form of Bij. If dj = 0 there is no restriction on the form of Bij. If dj = nj the last n di rows of Bij are zero. − 1 Conversely, if each Bij = T − AijTj has the block form in (2), define i to i U be the subspace spanned by the first di columns of Ti. The subspaces i i {U } ∈I then satisfy Aij( j) i. Hence, we have the following equivalent matrix form of DefinitionU (3.2).⊆ U Definition 3.3 (Matrix version of coupled reducibility). Let = Aij i,j . ∈I We say is reducible in the coupled sense, or, reducible by coupledA similarity{ } , A if there exist integers di i , with 0 di ni, and nonsingular ni ni { } ∈I ≤ ≤ × matrices Ti, such that the following hold. 8 1. At least one di is positive. 2. At least one di is less than ni. 1 3. Each matrix Bij = Ti− AijTj has a block of zeroes in the lower left hand corner of size (ni di) dj. − × We say is properly reducible in the coupled sense if 0 < di < ni for at least oneA value of i. We say is strongly reducible in the coupled sense if A 0 C 0di (n dj ) Bij = × − . (3) 0(n di) dj E − × The di dj matrix C represents the action of Aij on j and the (ni di) × U − × (nj dj) matrix E represents the action of Aij on ˆj. For− the field of complex numbers we have unitaryU versions. Definition 3.4 (Unitary version of reducible in the coupled sense). Let be A a family of complex matrices. We say is unitarily reducible in the coupled A sense if the conditions of Definition 3.3 are satisfied with unitary matrices Ti. For complex , reducibility by coupled similarity implies reducibility by A coupled unitary similarity. Simply use an orthonormal basis for each i, and U extend it to an orthonormal basis for i to obtain a unitary matrix for Ti. V If is fully reducible, and i and ˆi are orthogonal subspaces, then, for A U U each i = i ˆi, we can form a unitary matrix Ti using an orthonormal V U ⊕ U basis for i for the first di columns and an orthonormal basis for ˆi for the U U remaining ni di columns. Each Bij then has the block form (3). Unitary reducibility− matters in the JISA model, because A and B are correlation matrices, and the appropriate linear change of variable leads to 1 a congruence, rather than a similarity. When Ti is unitary, Ti− = Ti∗. For 1 T = T T Tk we then have T − AT = T ∗AT . 1 ⊕ 2 ⊕···⊕ From the definition, it is clear that if is strongly reducible, then it is also properly reducible, and if it is properlyA reducible, it is reducible. We 9 introduce some notation. Fix an index set, . Use to denote the size of ; when is a finite set with K elements, weI assume|I| = 1, 2,...,K . FixI I I { } a family ni i of positive integers, and a field F. Consider the set of all { } ∈I = Aij i,j , where Aij is an ni nj matrix with entries from F. We use A { } ∈I × Red(F, ni i ) to denote the set of all such families which are reducible { } ∈I A in the coupled sense. We use PropRed(F, ni i ) for the set of all such { } ∈I A which are properly reducible in the coupled sense, and StrRed(F, ni i ) { } ∈I for the set of all such that are strongly reducible in the coupled sense. A When all ni’s have the same value, n, and = K, we use the notations Red(F,n,K), PropRed(F,n,K), and StrRed|I|(F,n,K). When ni = 1, the space i = F is one dimensional and has no nonzero V proper subspaces. Hence, StrRed(F, ni i ) is the empty set if ni = 1 for { } ∈I some i, and PropRed(F, ni i ) is the empty set when ni = 1 for all i. ∈I From Definition 3.2 it{ is} obvious that StrRed(F, ni i ) PropRed(F, ni i ) Red(F, ni i ). (4) { } ∈I ⊆ { } ∈I ⊆ { } ∈I Using the superscript “C” to indicate the complement of a set, we then have C C C Red (F, ni i ) PropRed (F, ni i ) StrRed (F, ni i ). (5) { } ∈I ⊆ { } ∈I ⊆ { } ∈I The symbol “ ” means “subset of or equal to.” We use “ ” to indicate ⊆ ⊂ “proper subset of.” One might expect that “ ” can generally be replaced by “ ” in (4) and (5). This is correct when ⊆ has at least four elements, ⊂ I and ni 2 for at least one value of i. Furthermore, for 2, we have ≥ |I| ≥ StrRed(F, ni i ) PropRed(F, ni i ), provided ni 2 for at least one i. { } ∈I ⊂ { } ∈I ≥ However, for = 2 and = 3, whether PropRed(F, ni i ) is equal to, or |I| |I| { } ∈I is a proper subset of, Red(F, ni i ) depends on the field F, and on the ni’s. ∈I The appendix treats this in more{ } detail. Here is a summary of what is shown there. 1. For any field F, if 4 and ni 2 for at least one i, |I| ≥ ≥ StrRed(F, ni i ) PropRed(F, ni i ) Red(F, ni i ). { } ∈I ⊂ { } ∈I ⊂ { } ∈I Consequently, C C C Red (F, ni i ) PropRed (F, ni i ) StrRed (F, ni i ). { } ∈I ⊂ { } ∈I ⊂ { } ∈I 2. For any field F, if 2 and ni 2 for at least one i, |I| ≥ ≥ StrRed(F, ni i ) PropRed(F, ni i ). { } ∈I ⊂ { } ∈I 10 3. If F is algebraically closed and n 2, then ≥ PropRed(F, n, 2) = Red(F, n, 2) and PropRed(F, n, 3) = Red(F, n, 3). 4. For the field, R, of real numbers, when n = 2, we have PropRed(R, 2, 2) Red(R, 2, 2) and PropRed(R, 2, 3) Red(R, 2, 3). ⊂ ⊂ For n 3, ≥ PropRed(R, n, 2) = Red(R, n, 2) and PropRed(R, n, 3) = Red(R, n, 3). 4 A coupled version of Schur’s Lemma The main result of this section is Theorem 4.2, a coupled version of Schur’s Lemma for reducibility and proper reducibility. Section 5 deals with the more complicated version for strong reducibility. Consider families = Aij i,j and = Bij i,j I, where Aij is ni nj A { } ∈I B { } ∈ × and Bij is mi mj, linked by equations AijXj = XiBij, where Xi is ni mi. × × Recall that Aij is a linear transformation from j to i, and Bij is a linear V V transformation from j to i The matrix Xi is a linear transformation from W W i to i. Note that ker(Xi) is a subspace of i and range(Xi) is a subspace W V W of i. VReviewing the proof of Schur’s Lemma (Theorem 2.1), the key facts are that ker(P ) is an invariant subspace of Bi i , and range(P ) is an invariant { } ∈I subspace of Ai i . For the case of complex matrices with Ai = Bi for { } ∈I all i, any eigenspace of P is an invariant subspace of Ai i . The following ∈I “coupled” versions of these facts are used to prove coupled{ } versions of Schur’s lemma for , . In the coupled versions, the Xi’s play the role of the P . A B Lemma 4.1. Let = Aij i,j and = Bij i,j I, where Aij is ni nj A { } ∈I B { } ∈ × and Bij is mi mj. Let Xi be ni mi and suppose for all i, j , we × × ∈ I have AijXj = XiBij. If mi = ni for some i, then, for any scalar α, define i(α)= v Xiv = αv . The following hold for all i, j . U { | } ∈ I 1. Bij(ker(Xj)) ker(Xi). ⊆ 2. Aij(range(Xj)) range(Xi). ⊆ 3. If = , then Aij( j(α)) i(α). A B U ⊆ U 11 Proof. For any w ker(Xj), we have Xi(Bijw) = AijXjw = 0. Hence, ∈ Bijw ker(Xi). This proves 1. ∈ For w , we have Aij(Xjw)= Xi(Bijw) range(Xi), proving 2. ∈ W ∈ Finally, suppose = . Then AijXj = XiAij for all i, j. Let v j(α). A B ∈ U Then Xi(Aijv)= AijXjv = α(Aijv), showing Aijv i(α). ∈ U If mi = ni and α is an eigenvalue of Xi, with α F, then i(α) is the ∈ U corresponding eigenspace. If α is not an eigenvalue of Xi, then i(α) is the zero subspace. U We now state a version of Schur’s Lemma for families that are irreducible in the coupled sense. The proofs simply extend the argument used to prove the usual Schur Lemma. Theorem 4.2. Let = Aij i,j and = Bij i,j I, where Aij is ni nj A { } ∈I B { } ∈ × and Bij is mi mj. Let Xi be ni mi and suppose for all i, j , we have × × ∈ I AijXj = XiBij. 1. Suppose both and are irreducible in the coupled sense. Then either A B Xi = 0 for all i, or Xi is nonsingular for all i. In the latter case, mi = ni for all i. If = , and is a family of complex matrices, then there A B A is a scalar α such that Xi = αIni for all i. 2. Suppose neither nor is properly reducible in the coupled sense. A B Then for each i, either Xi =0or Xi is nonsingular. If Xi is nonzero we must have mi = ni. If = and consists of complex matrices, then A B any nonzero Xi is a scalar multiple of Ini . Proof. For part 1, assume and are both coupled irreducible. Consider A B the subspaces ker(Xi), i . Since is irreducible in the coupled sense, statement 1 of Lemma 4.1∈ I tells us thereB are only two possibilities: either ker(Xi)= 0 for all i, or ker(Xi)= i for all i. In the latter case, Xi =0 for all i and{ so} we are done. W Suppose now that ker(Xi) = 0 for all i. We now use the subspaces { } range(Xi), i . Since is irreducible in the coupled sense, part 2 of ∈ I A Lemma 4.1 tells us the only possibilities are range(Xi) = 0 for all i or { } range(Xi) = i for all i. If range(Xi) = 0 for all i, then Xi = 0 for all i. V { } Otherwise, we have both kerXi = 0 and range(Xi) = i for all i. Hence { } V each Xi is nonsingular and mi = ni. Now suppose = and F = C. Let λ be an eigenvalue of Xp for A B some fixed p . Part 3 of Lemma 4.1 tells us Aij( j(λ)) i(λ) for ∈ I U ⊆ U 12 all i, j. Since is irreducible in the coupled sense, there are then only two A possibilities for the subspaces i(λ): either they are all zero, or i = i for U U V all i. Since λ was chosen to be an eigenvalue of Xp, we know p(λ) is not U zero. Therefore, i(λ)= i for all i and hence Xi = λIni for all i. For part 2, assumeU neitherV nor is properly reducible in the coupled A B sense. Consider ker(Xi). Since is not properly reducible in the coupled B sense, Lemma 4.1 tells us ker(Xi) cannot be a nonzero, proper subspace of i. Hence, for each particular i, either ker(Xi) = 0 or ker(Xi) = i. W { } W In the latter case, Xi = 0. Suppose ker(Xi) = 0 for some i. Since is not properly reducible { } A in the coupled sense, Lemma 4.1 tells us range(Xi) is either 0 or i. If { } V range(Xi) = 0 then Xi = 0 . Otherwise, we have both kerXi = 0 and { } { } range(Xi)= i, so Xi is nonsingular and mi = ni. V Now suppose = is a family of complex matrices. Suppose Xp = 0 A B 6 for some p. Let λp be an eigenvalue of Xp. Note λp = 0 because Xp is 6 nonsingular. By Lemma 4.1, we have Aij( j(λp)) i(λp) for all i, j. Since U ⊆ U A is not properly reducible in the coupled sense, each i(λp) is either zero or U the full vector space i. Since λp is an eigenvalue of Xp, the space p(λp) is V U not zero. Therefore, p(λj)= p and Xp = λpIn . U V p Remark 4.1. The ordinary version of Schur’s Lemma, Theorem 2.1, applies to the case where both and are irreducible in the sense of Definition 2.1, A B and Xi = P for all i. Note the different conclusions for the two parts of Theorem 4.2. For part 1, either all Xi’s are zero, or all are nonsingular. When = and A B F = C, all the Xi’s are the same scalar multiple of the identity matrix. In part 2, there are more options for the Xi’s. Each Xi is either zero or nonsingular, but some can be zero and others nonsingular. For = and F C A B = , the proof for part 2 gives Xp = λpInp for a particular value of p; it does not show every nonzero Xi equals the same scalar multiple of the identity matrix. The broader range of options for the Xi’s in part 2 makes sense when we consider that, at least for 4, we have PropRed(F, ni i ) C|I| ≥ C { } ∈I ⊂ Red(F, ni i ), and hence Red (F, ni i ) PropRed (F, ni i ). Part2 { } ∈I { } ∈I ⊂ { } ∈I applies to a broader set of pairs , than part 1. A B Consider the situation in part 2 of Theorem 4.2. Suppose Xi = 0 and Xj is nonsingular. The equation AijXj = XiBij then tells us Aij = 0, while 13 AjiXi = XjBji gives Bji = 0. Set = i Xi =0 and non = i Xi is nonsingular . I0 { ∈I | } I { ∈I | } We have Aij = 0 and Bji = 0 whenever i 0 and j non. For ex- ample, suppose = 1, 2,...,K , and, for∈ some I 0 A11 A1s 0 0 . ··· . . ··· . . . . . As1 Ass 0 0 A = ··· ··· . A(s+1)1 A(s+1)s A(s+1)(s+1) A(s+1)K . ··· . . ··· . . . . . AK AKs AK s AKK 1 ··· ( +1) ··· Returning to the case of general , one can check that is coupled reducible I A via the subspaces i = 0 for i , and i = i for i non. The family U { } ∈ I0 U V ∈ I B is coupled reducible via the subspaces i = i when i , and i = 0 U W ∈ I0 U { } when i non. ∈ I 5 Strong reducibility and Schur’s Lemma We now consider strongly coupled reducibility. Our goal is a version of Schur’s lemma for families that are not strongly reducible in the coupled sense, with a conclusion similar to that of Theorem 4.2: each Xi is either zero or nonsingular. The next example shows that for such a conclusion, we need some restrictions on Aij and Bij. Example 5.1. Let n = m = 2 (so = = F2), and K = 2. Put V W 0 1 0 0 a b A = A = A = A = 11 22 0 0 21 0 0 12 c d 0 1 a b 0 0 B = B = B = B = . 11 22 0 0 21 c d 12 0 0 14 In terms of the matrices A, B: 0 1 a b 0 1 0 0 | | 0 0 c d 0 0 0 0 A = | B = | . 0 0 0 1 a b 0 1 0 0 | 0 0 c d | 0 0 | | 1 Let be the subspace spanned by e = . One may easily check that is U 1 0 A properly reducible in the coupled sense with 1 = and 2 = 0 , while is properly reducible in the coupled sense with U = U0 andU ={ }. However,B U1 { } U2 U if c = 0, then neither nor is strongly reducible in the coupled sense. The6 reason is that isA the onlyB nonzero, proper invariant subspace for the 0U 1 diagonal blocks, , of A and B, so = = is the only possible 0 0 U1 U2 U choice for nonzero, proper subspaces and . If c = 0, then A ( ) , U1 U2 6 12 U 6⊂ U and B21( ) . So if c = 0, neither nor is strongly reducible in the U 6⊂ U 6 0 1 A 0B 0 coupled sense. Set X = , X = and X = X X . One 1 0 0 2 0 0 1 ⊕ 2 may check that AX = XB = 0 and hence AijXj = XiBij for i, j =1, 2. The point is that the matrix X1 is neither zero nor nonsingular. Our theorem for coupled pairs , that are not strongly reducible will A B be for the case when ni = n and mi = m for all i. It will have a hypothesis about graphs related to and ; roughly speaking, this hypothesis will tell A B us there are “enough” nonsingular Aij’s and Bij’s. Although our main result assumes is a family of n n matrices and is a family of m m matrices, A × B × we define the graphs for families with matrices of any size. Recall that a matrix is said to have full column rank if the columns are linearly independent; thus, the rank of the matrix equals the number of columns. If A is a p q matrix with full column rank, and is a subspace of Fq, then A( ) has× the same dimension as . U U U Consider = Aij i,j I, and subspaces i i , satisfying Aij( j) i A { } ∈ {U } ∈I U ⊆ U for all i, j. Let di be the dimension of i. If Aij has full column rank, then U Aij( j) i tells us dj di. If Aji also has full column rank, then we also U ⊆ U ≤ have di dj, and hence di = dj. When Aij and Aji both have full column ≤ rank, nj ni and ni nj, so ni = nj; hence, Aij and Aji are actually ≤ ≤ square, nonsingular matrices. If all of the Aij’s have full column rank, then all of the ni’s have the same value, n, and all of the subspaces j have the U 15 same dimension, d. However, we need not assume all of the matrices Aij are nonsingular in order to show the j’s all have the same dimension. To U explore this further, we introduce a directed graph in which directed edges correspond to the Aij’s of full column rank. Definition 5.1. Let = Aij i,j I, with Aij of size ni nj. The directed A { } ∈ × graph (digraph) of , denoted ( ), is the graph on vertices vi i , such A D A { } ∈I that there is a directed edge (vi, vj) from vi to vj if and only if Aij has full column rank. For a finite index set, = 1,...,K , there are K vertices. If ni = 1 for all i, our ( ) is just theI usual{ directed} graph associated with a K K D A × matrix. More generally, there is a vertex for each i , so there could be infinitely ∈ I many vertices. We use the same definition for directed walk as for graphs with a finite number of vertices. A directed walk is a finite sequence of vertices, vi , vi ,...,vi , such that (vi , vi ) is a directed edge for 1 j (p 1). 1 2 p j (j+1) ≤ ≤ − In this case, we write vi1 vi2 vip . Vertices v and w in a directed graph are said to be strongly→ →···→ connected if there is a directed walk from v to w andD a directed walk from w to v. We say is strongly connected if D each pair of vertices of is strongly connected. D Proposition 5.1. Let = Aij i,j and suppose the subspaces Ui i A { } ∈I { } ∈I satisfy Aij( j) i, for all i, j. Then the following hold. U ⊆ U 1. If there is a directed walk from vi to vj in ( ), then nj ni, and D A ≤ dim( j) dim( i). U ≤ U 2. If the vertices vi and vj are strongly connected in ( ), then ni = nj, D A and dim( j)= dim( i) U U 3. If ( ) is strongly connected, all of the ni’s are equal, and all of the D A subspaces i have the same dimension. U Proof. Let di = dim( i). If (vi, vj) is a directed edge of ( ), then Aij has U D A full column rank, so, as we have already observed, nj ni and dj di. ≤ ≤ More generally, suppose vi = vi vi vi = vj is a directed 1 → 2 → ···→ p walk from vi to vj in ( ). Working from right to left, we have D A nj = ni ni − ni ni = ni p ≤ p 1 ≤···≤ 2 ≤ 1 16 and dj = di di − di di = di. p ≤ p 1 ≤···≤ 2 ≤ 1 Hence, nj ni and dj di. ≤ ≤ If vi and vj are strongly connected, there is a directed walk from vi to vj and a directed walk from vj to vi. So ni = nj and di = dj. If ( ) is strongly connected, then, for all i, j, we have di = dj and D A ni = nj, so all of the subspaces j have the same dimension and all of U the ni’s have the same value. As an example, suppose = 1,...,K and A12, A23,...AK 1,K, AK1 all − have full column rank. ThenI ({ ) contains} the directed cycle D A v1 v2 vK 1 vK v1, → →···→ − → → and is strongly connected. If ( ) is not strongly connected, the strong components identify sets D A of ni’s which must be equal, and sets of subspaces i which must have the U same dimension. For each strong component, , of ( ), all ni’s corre- C D A sponding to vertices of must be equal, and all subspaces i corresponding to vertices of must haveC the same dimension. For a finite U, we can use the strong componentsC to put the N N matrix A into a blockI triangular form × in which none of the Aij’s below the diagonal blocks has full column rank. (See [3], section 3.2.) For the proofs of coupled versions of Schur’s Lemma, the subspaces i of U interest are the kernels and ranges of the matrices Xi i . { } ∈I Proposition 5.2. Let = Aij i,j and = Bij i,j . Let Xi be ni mi, A { } ∈I B { } ∈I × and suppose AijXj = XiBij for all i, j . Then the following hold. ∈ I 1. If vi and vj are strongly connected in ( ), then range(Xi) and D A range(Xj) have the same dimension, i.e., Xi and Xj have the same rank. 2. If vi and vj are strongly connected in ( ), then ker(Xi) and ker(Xj) D B have the same dimension, i.e., Xi and Xj have the same nullity. 3. If ( ) is strongly connected, all of the ni’s have the same value, n, D A and all of the Xi’s have the same rank. 17 4. If ( ) is strongly connected, all of the mi’s have the same value, m, D B and all of the Xi’s have the same nullity, d. 5. If vi and vj are strongly connected in ( ), then Xi and Xj have the D B same rank. 6. If ( ) is strongly connected, all of the Xi’s have the same rank. D B Proof. The first four parts follow from Lemma 4.1 and Proposition 5.1. For part 5, suppose vi and vj are strongly connected in ( ). Then mi = mj, so D B the matrices Xi and Xj have the same number of columns. From part 2, we know Xi and Xj have the same nullity. The rank plus nullity theorem then tells us Xi and Xj have the same rank. Part 6 is an immediate consequence of part 5. We now have a version of Schur’s lemma for families , when neither is strongly reducible in the coupled sense. A B Theorem 5.1. Assume neither = Aij i,j I nor = Bij i,j I is strongly ∈ ∈ reducible in the coupled sense. AssumeA { also} that (B) and{ }( ) are strongly D A D B connected. Let Xi be n m for all i , and suppose AijXj = XiBij for × ∈ I all i, j . Then either Xi = 0 for all i, or Xi is nonsingular for all i. In ∈ I the latter case we must have m = n. If = and is a family of complex A B matrices, then there is some scalar α such that Xi = αIn for all i. Proof. Note first that since ( ) and ( ) are both strongly connected, D A D B the Aij’s are all square matrices of the same size, n, and the Bij’s are all square matrices of the same size, m. By Proposition 5.2, the subspaces ker(Xi), for i , all have the same dimension, d. Since is not strongly reducible in the∈ coupled I sense, either B d =0 or d = m. If d = m, then Xi = 0 for all i and we are done. Assume then that d = 0. Proposition 5.2 tells us the subspaces range(Xi) all have the same dimension, r. Since is not strongly reducible in the A coupled sense either r = 0 or r = n. If r = 0, then Xi = 0 for all j. If r = n, then, since we also have d = 0, the Xi’s are nonsingular; we then have m = n. If = , we have AijXj = XiAij for all i, j. Fix p and let λ be an A B eigenvalue of Xp with corresponding eigenspace p(λ); note the subspace U p(λ) is nonzero, because λ is an eigenvalue of Xp. From part 3 of Lemma 4.1, U we have Aij( j(λ)) i(λ) for all i, j. Since ( ) is strongly connected, U ⊆ U D A 18 Proposition 5.1 tells us the spaces i(λ) all have the same dimension; call U it f. Since p(λ) is nonzero, we know f > 0. Hence, since is not strongly U A reducible in the coupled sense, we must have f = n, and Xi = λIn for all i. Remark 5.1. Earlier work [6] gives a proof, using block matrix computation, for the case where all Aij’s and Bij’s are assumed to be nonsingular. The proof of Theorem 5.1 uses the assumption that both ( ) and ( ) D A D B are strongly connected in two ways: to establish that ni = n and mi = m for all i, and to show that the relevant subspaces (kernels and ranges of the Xi’s) have the same dimension. We now develop another version of Theorem 5.1, in which we weaken the hypothesis about the graphs, but then need to explicitly assume that ni = n and mi = m for all i. The key point for this second version is that Xi and Xj have the same rank whenever vi and vj are strongly connected in either of the digraphs ( ) or ( ). D A D B We use and to define an undirected graph, ( , ), as follows. A B G A B Definition 5.2. The undirected graph, ( , ), is the graph on vertices G A B vi i , such that vi, vj is an (undirected) edge of ( , ) if and only if { } ∈I { } G A B the vertices vi and vj are either strongly connected in ( ), or in ( ) (or both). We call this the linked graph of and . D A D B A B Proposition 5.3. Let = Aij i,j and = Bij i,j . Let Xi be ni mi A { } ∈I B { } ∈I × and suppose AijXj = XiBij for all i, j . If vi and vj are connected in ∈ I ( , ) then Xi and Xj have the same rank. If ( , ) is connected, then G A B G A B all of the matrices Xi have the same rank. Proof. Suppose vi and vj are connected in ( , ). Then there is a sequence G A B of vertices, vi = vi , vi , vi ,...,vi − , vi = vj, such that vi , vi is an edge 1 2 3 p 1 p { k k+1 } of ( , ) for k =1,...,p 1. This means vi and vi are either strongly G A B − k k+1 connected in ( ) or strongly connected in ( ), or both. Therefore, D A D B rank(Xi )= rank(Xi ) for k =1,...,p 1, and rank(Xi)= rank(Xj). k k+1 − If either ( ) or ( ) is strongly connected, then ( , ) will be con- nected. However,D A ( D, B) can be a connected graph evenG A ifB neither of the G A B digraphs ( ) or ( ) is strongly connected. For example, suppose K =3 and D A D B 0 A12 0 0 0 B13 A = A21 0 0 B = 0 0 0 , 0 0 0 B 0 0 31 19 v3 v3 v3 r r r ✡ ✡ ✡ ✡✣✡ ✡ ✡✡ ✡ ✡✡✢ ✡ r ✛ ✲ r r✡ r r✡ r v1 v2 v1 v2 v1 v2 ( ) ( ) ( , ) D A D B G A B Figure 1: ( ), ( ) and ( , ) D A D B G A B where A , A , B and B are all nonsingular. Neither ( ) nor ( ) is 12 21 13 31 D A D B connected, but ( , ) is connected. (See Figure 1.) G A B As an example, suppose 1 and 2 are nonempty, disjoint subsets of such that = . PartitionI I the vertices of ( , ) into two setsI I I1 ∪ I2 G A B corresponding to and , setting I1 I2 = vi i and = vi i . S { | ∈ I1} T { | ∈ I2} Suppose rank(Aij) < nj and rank(Bij) < mj, whenever i and j . ∈ I1 ∈ I2 Then neither ( ) nor ( ) has any directed edges from vertices in to vertices in .D TheA linkedD graphB ( , ) then has no edges from verticesS in to verticesT in and hence is notG connected.A B S T Now suppose that, whenever i and j , we have rank(Aij) < nj ∈ I1 ∈ I2 and rank(Bji) < mi, (note the reversal of subscripts on Bji). In this case, ( ) has no directed edges from vertices in to vertices in , while ( ) D A S T D B has no directed edges from vertices in to vertices in . Consequently, if v and w , then the pair v,wTis not stronglyS connected in either ∈ S ∈ T ( ) or ( ). Hence, ( , ) has no edges between vertices in and verticesD A inD B; thus ( , G) isA notB connected. S The followingT variationG A B of Theorem 5.1 uses this linked graph, ( , ). G A B Theorem 5.2. Assume neither = Aij i,j I nor = Bij i,j I is strongly A { } ∈ B { } ∈ reducible in the coupled sense. Assume also that ni = n and mi = m for all i, and that ( , ) is connected. Let Xi be n m, and suppose AijXj = XiBij G A B × for all i, j . Then either Xi = 0 for all i, or Xi is nonsingular for all i. In the latter∈ case I we must have m = n. If = and is a family of complex A B matrices, then there is some scalar α such that Xi = αIn for all i. Proof. By Proposition 5.3, the subspaces range(Xi), for i in , all have the I same dimension, r. Since all of the Xi’s have the same number of columns, the rank plus nullity theorem tells us the subspaces ker(Xi), for i , must ∈ I also all have the same dimension, d. The remainder of the proof is the same as that for Theorem 5.1. 20 Comparing Theorems 4.2, 5.1, and 5.2, the simplest version is part 1 of Theorem 4.2. It is the closest to the usual Schur’s Lemma. However, the hypothesis that , be irreducible in the coupled sense is more restrictive than the hypothesisA B of part 2 of Theorem 4.2. The conclusion of part 2 has more options for the Xi’s than part 1. Theorems 5.1 and 5.2 apply to the larger class of pairs, , , which are not strongly reducible in the coupled sense, but have additionalA B restrictions about the connectivity of the graphs ( ), ( ), and ( , ) and the equality of the ni’s and mi’s. D A D B G A B 6 Normality and coupled normality We now consider a refinement of Schur’s Lemma for irreducible sets of normal matrices. This is closely related to Lemma A.4 of [11]. We obtain correspond- ing results for sets , satisfying a “coupled normality” condition. For this A B section we work over the field of complex numbers. We use * to denote the transpose conjugate of a matrix. If is a subspace of , we use ⊥ for the orthogonal complement of . We willU need the followingV facts. U U Proposition 6.1. Let A be a normal matrix; let S be nonsingular and let 1 B = S− AS. Then the following are equivalent. 1. The matrix B is normal. 1 2. S− A∗S = B∗. 3. The matrix SS∗ commutes with A. 4. The matrix SS∗ commutes with A∗. 5. The matrix S∗S commutes with B. 6. The matrix S∗S commutes with B∗. 1 Proof. The equivalence of 2, 3 and 4 is easily shown. Using B = S− AS, 1 1 S− A∗S = B∗ S− A∗S = S∗A∗S−∗ ⇐⇒ A∗SS∗ = SS∗A∗ ⇐⇒ SS∗A = ASS∗, ⇐⇒ 21 where the third line comes from taking the transpose conjugate of the equa- 1 tion in the second line. A similar calculation, starting with A = SBS− , shows 2, 5 and 6 are equivalent: 1 1 SB∗S− = A∗ SB∗S− = S−∗B∗S∗ ⇐⇒ S∗SB∗ = B∗S∗S ⇐⇒ BS∗S = S∗SB. ⇐⇒ 1 The fact that 2 implies 1 is also easy. If S− A∗S = B∗, use AA∗ = A∗A to get 1 1 1 1 BB∗ =(S− AS)(S− A∗S)=(S− A∗S)(S− AS)= B∗B. The only part needing any work at all is to show 1 implies 2. Let λ1,...,λn be the eigenvalues of A and let D be the diagonal matrix with diagonal entries λ1,...,λn. Since A is normal, A = U ∗DU for some unitary matrix U, and A∗ = U ∗DU, where the bar denotes complex conjugation. Note λ1,..., λn are the eigenvalues of A∗. Let p(x) be a polynomial such that p(λi)= λi for each eigenvalue λi. Then D = p(D), and A∗ = U ∗p(D)U = p(U ∗DU)= p(A). Since B is similar to A, the matrix B also has eigenvalues λ1,...,λn and B∗ has eigenvalues λ1,..., λn. If B is normal, B = V ∗DV for some unitary matrix V . Hence, B∗ = V ∗DV = V ∗p(D)V = p(V ∗DV )= p(B). 1 1 1 1 But p(B)= p(S− AS)= S− p(A)S = S− A∗S, so B∗ = S− A∗S. This gives an easy proof of the following. Theorem 6.1. Suppose Ai i and Bi i are irreducible families of nor- { } ∈I { } ∈I 1 mal matrices, and S is a nonsingular matrix such that S− AiS = Bi for all i . Then S is a scalar multiple of a unitary matrix. ∈ I Proof. By the preceding proposition, SS∗ commutes with each Ai. Since Ai i is an irreducible family, SS∗ must be a scalar matrix. Since S is { } ∈I nonsingular, the Hermitian matrix SS∗ is positive definite; hence SS∗ = αI 1 1 where α is a positive real number. Set U = √α S. Then UU ∗ = α SS∗ = I. So U is unitary and S = √αU. 22 Remark 6.1. This argument is essentially the proof of Lemma A.4 of [11], which says that if two irreducible representations of a -algebra of square ∗ matrices are equivalent, then they are similar via a unitary similarity. Let S be the algebra generated by Ai i and let be the algebra generated by { } ∈I T Bi i . For any normal matrix, N, the matrix N ∗ is a polynomial in N, so { } ∈I the algebras and are -algebras, (which means that whenever A is in the S T ∗ 1 algebra, so is A∗). Let S be a nonsingular matrix such that S− AiS = Bi for 1 all i. Proposition 6.1 tells us S− Ai∗S = Bi∗ for all i, so S may be extended to an isomorphism of the -algebras and in the usual way. ∗ S T We now introduce the idea of coupled normality. Definition 6.1. The family = Aij i,j is normal in the coupled sense if A { } ∈I for all i, j we have A∗ Aij = AjiA∗ . ∈ I ij ji If is normal in the coupled sense, setting i = j gives A∗ Aii = AiiA∗ , A ii ii so Aii is normal for all i. Note also that if Aji = A∗ for all i, j, then is ij A coupled normal. When = 1,...,K , the condition Aji = A∗ for all i, j I { } ij holds when A is a Hermitian matrix. In the JISA model, A is a covariance matrix, and hence is a real, symmetric matrix, so it is Hermitian. Recall that, for any matrix G, the four matrices G,G∗,GG∗, and G∗G all have the same rank. Hence, when is normal in the coupled sense, the A matrices Aij and Aji have the same rank. In particular, note that Aij is nonsingular if and only if Aji is nonsingular. Let C be a q p matrix, let D be a p q matrix, and let M be the (p + q) (p + q) matrix× × × 0 D M = , C 0 where the zero blocks are p p and q q. Then × × DD∗ 0 C∗C 0 MM ∗ = and M ∗M = . 0 CC∗ 0 D∗D Hence, M is normal if and only if C∗C = DD∗ and D∗D = CC∗. The 0 Aij connection with coupled normality is this: if we set Mij = , Aji 0 then = Aij i is normal in the coupled sense if and only if Mij is normal ∈I for allAi, j { .} ∈ I Suppose is normal in the coupled sense and the subspaces i i A {U } ∈I satisfy Aij( j) i for all i, j. Let di be the dimension of i. We use the U ⊆ U U fact that Mij is normal to show that Aij( ⊥) ⊥ for all i, j. Uj ⊆ Ui 23 Proposition 6.2. Let C,D be matrices of sizes q p and p q, respectively, × × such that C∗C = DD∗ and D∗D = CC∗. Suppose there are subspaces p q U of C , and of C , such that C( ) and D( ) . Then C( ⊥) W U ⊆ W W ⊆ U U ⊆ ⊥ and D( ⊥) ⊥. W W ⊆ U 0 D Proof. Let M = . For any x Cp and y Cq, C 0 ∈ ∈ x Dy M = . y Cx If x and y , then Dy and Cx . So , (which is a subspace∈ U of Cp C∈q), W is invariant under∈ U M. Since∈ WM is normal,U ⊕ W the orthogonal ⊕ complement of in Cp Cq must also be invariant under M. Hence, U ⊕ W ⊕ ⊥ ⊥ is invariant under M. This means that, for x ⊥ and y ⊥, U ⊕ W ∈ U ∈ W we have Dy ⊥ and Cx ⊥. So C( ⊥) ⊥ and D( ⊥) ⊥. ∈ U ∈ W U ⊆ W W ⊆ U 0 Aij Apply Proposition 6.2 to the normal matrix Mij = , to get Aji 0 Aij( j⊥) i⊥ for all i, j. Hence, if is normal in the coupled sense, and is reducibleU ⊆ in U the coupled sense, thenA it is fully reducible in the coupled sense, because we can form Tj using a basis for j for the first dj columns and a U basis for ⊥ for the remaining n dj columns. If we use orthonormal bases Uj − for j and ⊥, then Tj will be unitary. Hence is fully reducible in the U Uj A coupled sense with a coupled unitary similarity. We will give three versions of Theorem 6.1 for , which are normal in A B the coupled sense, corresponding to the three types of reducibility. The proofs depend on the following proposition. The first two statements are a “coupled” version of Proposition 6.1. Part 4 uses the digraphs ( ) and ( ). D A D B Proposition 6.3. Assume the families = Aij i,j and = Bij i,j , A { } ∈I B { } ∈I where Aij and Bij are complex matrices, are normal in the coupled sense. Suppose AijSj = SiBij for all i, j, where Si is ni mi. For any i , and × ∈ I any scalar α, define i(α)= v SiS∗v = αv and i(α)= w S∗Siw = αw . U { | i } Y { | i } Then the following hold. 1. If Si is nonsingular then SiSi∗ commutes with Aii, and Si∗Si commutes with Bii. 24 2. If Si and Sj are both nonsingular, SiSi∗Aij = AijSjSj∗ and Si∗SiBij = BijSj∗Sj. 3. If Si and Sj are both nonsingular, Aij( j(α)) i(α) and Bij( j(α)) i(α). U ⊆ U Y ⊆Y If Aij is also nonsingular, then dim( i(α)) = dim( j(α)). U U If Bij is also nonsingular, then dim( i(α)) = dim( j(α)). Y Y 4. Assume Si is nonsingular for all i . Then the following hold. ∈ I If vi and vj are strongly connected in ( ), then i(α) and j(α) have the same dimension. D A U U If vi and vj are strongly connected in ( ), then i(α) and j(α) have the same dimension. D B Y Y 5. If α = 0 then i(α) and i(α) have the same dimension. 6 U Y 6. Assume Si is nonsingular for all i . Then if vi and vj are connected ∈ I in ( , ), and α = 0, we have dim( i(α)) = dim( j(α)). G A B 6 U U Proof. Suppose Si is nonsingular. Since Aii and Bii are both normal, and 1 Si− AiiSi = Bii, Proposition 6.1 tells us that SiSi∗ commutes with Aii and Si∗Si commutes with Bii. Now suppose i = j, and Si and Sj are both nonsingular. Set Mij = 6 0 A ij . Then Aji 0 Si 0 0 Aij Si 0 Mij = 0 Sj Aji 0 0 Sj 0 A S = ij j AjiSi 0 0 S B = i ij SjBji 0 S 0 0 B = i ij . 0 Sj Bji 0 25 So, 1 Si 0 − Si 0 0 Bij Mij = . 0 Sj 0 Sj Bji 0 0 Bij Since and are both normal in the coupled sense, Mij and A B Bji 0 Si 0 are both normal. Set S = . Proposition 6.1 tells us that SS∗ 0 Sj commutes with Mij. Hence, S S 0 0 A 0 A S S 0 i i∗ ij = ij i i∗ , 0 SjSj∗ Aji 0 Aji 0 0 SjSj∗ 0 Bij and SiSi∗Aij = AijSjSj∗. Use the fact that S∗S commutes with Bji 0 to show Si∗SiBij = BijSj∗Sj for all i, j. For part 3, assume Si and Sj are nonsingular. Let v j(α). By part 2, ∈ U SiS∗(Aijv)= Aij(SjS∗v)= α(Aijv). This shows Aij( j(α)) i(α). If Aij i j U ⊆ U is nonsingular, dim(Aij( j(α))) = dim( j(α)), so dim( j(α)) dim( i(α)). U U U ≤ U Since is coupled normal, Aji is also nonsingular, giving the reverse inequal- A ity, so dim( j(α)) = dim( i(α)). The corresponding facts for come from U U B the same argument, using Si∗SiBij = BijSj∗Sj. For part 4, assume vi and vj are strongly connected in ( ). Proposi- D A tion 5.1, together with part 3, gives dim( i(α)) = dim( i(α)). The same U U argument applies when vi and vj are strongly connected in ( ) D B Part 5 comes from the fact that Sj∗Sj and SjSj∗ have the same nonzero eigenvalues with the same multiplicities. For part 6, suppose vi and vj are connected in ( , ). Then there is a G A B sequence of vertices, vi = vi , vi , vi ,...,vi − , vi = vj, such that vi , vi 1 2 3 p 1 p { k k+1 } is an edge of ( , ) for k =1,...,p 1. This means vi and vi are either G A B − k k+1 strongly connected in ( ) or strongly connected in ( ) (or both). If vi D A D B k and vi are strongly connected in ( ), then dim( i (α)) = dim( i (α)) k+1 D A U k U k+1 by part 4. If vi and vi are strongly connected in ( ), then part 4 tells k k+1 D B us dim( i (α)) = dim( i (α)). But, since α is nonzero, i(α) and i(α) Y k Y k+1 U Y have the same dimension. So, in either case, dim( i (α)) = dim( i (α)) U k U k+1 for 1 k p 1, and hence dim( i(α)) = dim( j(α)). ≤ ≤ − U U With these preliminaries completed, we state and prove a version of Schur’s Lemma for , that are normal in the coupled sense. The three cases correspond toA theB three types of coupled reducibility. 26 Theorem 6.2. Let = Aij i,j and = Bij i,j where Aij is ni nj A { } ∈I B { } ∈I × and Bij is mi mj. Assume and are normal in the coupled sense. × A B Suppose Si is ni mi and AijSj = SiBij for all i, j. × 1. If and are both irreducible in the coupled sense, then either Si =0 A B for all i, or there is a scalar α such that every Si is α times a unitary matrix; i.e., Si = αUi, where Ui is unitary. In the latter case, mi = ni for all i. Furthermore, if = , then there is a scalar β such that A B Si = βIni for all i. 2. If neither nor is properly reducible in the coupled sense, then, for A B each i, either Si =0 or Si is a scalar multiple of a unitary matrix. In the latter case, mi = ni. Furthermore, if = , then every Si is a A B scalar matrix. 3. Suppose neither nor is strongly reducible in the coupled sense. A B Assume also that ni = n and mi = m for all i , and that the graph ∈ I ( , ) is connected. Then either Si = 0 for all i, or there is a scalar α G A B such that each Si is a α times a unitary matrix; i.e., Si = αUi, where Ui is unitary. In the latter case we must have m = n. Furthermore, if = , then there is some scalar β such that Si = βIn for all i. A B Proof. The proofs are similar to those of Theorems 4.2 and 5.1. Suppose and are both irreducible in the coupled sense. Part 1 of A B Theorem 4.2 tells us that, either Si = 0 for all i, or Si is nonsingular for all i. In the latter case we must have mi = ni for all i. Suppose Si is nonsingular for all i. Fix p and let λ be an eigenvalue of SpSp∗. Proposition 6.3 gives Aij( j(λ)) i(λ) for all i, j. Since is irreducible in the coupled sense, U ⊆ U A either all of the subspaces i(λ) are zero, or i(λ)= i for all i . Since λ U U V ∈ I is an eigenvalue of SpS∗, the space p(λ) is nonzero. Therefore, i(λ) = i p U U V for all i, and SiSi∗ = λIni for all i. Since SiSi∗ is positive definite, λ is a 1 positive real number and Ui = √λ Si is a unitary matrix. For the second version, assume neither nor is properly reducible in the coupled sense. From part 2 of TheoremA 4.2,B we know that, for each i, either Si =0or Si is nonsingular. If Si is nonsingular we must have mi = ni. Suppose Sp is nonsingular for some p. Let λp be an eigenvalue of SpSp∗. Since Sp is nonsingular, λp = 0. Let denote the set of all q such that Sq is 6 N nonsingular. Consider the statement Aij( j(λp)) i(λp). (6) U ⊆ U 27 If i, j are both in , then Si,Sj are both nonsingular and Proposition 6.3 N tells us (6) holds. If j / , then Sj = 0, and hence, since λp is nonzero, ∈ N j(λp)= 0 , so (6) holds. Finally, if i / but j , then Si = 0 and Sj U { } ∈N ∈N is nonsingular. In this case, AijSj = SiBij tells us Aij = 0, and (6) holds. Hence, (6) holds for all i, j. Since is not strongly reducible in the coupled A sense, there are only two possibilities for each i(λ): it is either zero or the U whole space i. Since λp is an eigenvalue of SpS∗, we know p(λp) is nonzero; V p U therefore it must be the whole space and SpSp∗ = λpInp . Since SpSp∗ is positive 1 definite, λp is a positive real number and Up = Sp is a unitary matrix. √λp Finally, consider the third version, where we assume neither nor A B is strongly reducible in the coupled sense and ( , ) is connected. From G A B Theorem 5.2, either Si = 0 for all i, or Si is nonsingular for all i. In the latter case, m = n. Suppose Si is nonsingular for all i. Fix p and let λ be an eigenvalue of SpS∗. Since Sp is nonsingular, λ = 0. From Proposition 6.3, we have p 6 Aij( j(λ)) i(λ), for all i, j, and the subspaces i(λ) all have the same U ⊆ U U dimension. Let f be the dimension of these subspaces. Since λ is an eigen- value of SpS∗, the eigenspace p(λ) is nonzero. Hence, f > 0. Since is p U A not strongly reducible in the coupled sense we must have f = n. There- fore SiSi∗ = λIn for all i, the number λ must be a positive real number and 1 Ui = √λ Si is a unitary matrix. 7 Appendix We construct examples to establish the claims made in Section 3. Let be the index set; let ni i be a family of positive integers. If I { } ∈I ni = 1, set Ni = (0). If ni 2, let Ni be the ni ni matrix with a 1 in each superdiagonal entry and zeroes≥ elsewhere. This× is the standard nilpotent matrix used in the blocks of the Jordan canonical form. For any x Fni , ∈ 0 1 0 0 x1 x2 ··· 0 0 1 0 x2 x3 . . . ··· . . . Nix = . . . . . = . . 0 0 0 1 xni 1 xni ··· − 0 0 0 0 x 0 ··· ni Multiplying x on the left by Ni moves the coordinates up one position and i puts a 0 in the last entry. Let ej denote the vector with ni coordinates that 28 ei ei has a 1 in entry j and zeroes in all other positions. Thus, 1,..., ni are Fni i i the unit coordinate vectors for . Then Niej = ej 1. Henceforth, we omit − the superscript i on ej, as the number of coordinates will be clear from the context. For example, if we write Aijv, then it is understood that v has nj coordinates. Here is the key fact used in the examples. Proposition 7.1. For n 2, let N be the n n matrix with a 1 in each ≥ × superdiagonal entry and zeroes elsewhere. Suppose is a nonzero, proper U invariant subspace of N. Then e and en / . 1 ∈ U ∈ U Proof. Let x be a nonzero vector in , and let xk be the last nonzero coor- U k 1 dinate of x, i.e., xk+1 = = xn = 0. Then N − x = xke1, so e1 . ··· n 1 n 2 ∈ U For the second part, note that N − en, N − en,...,Nen, en are the unit coordinate vectors e ,..., en. Hence, if en , then is the whole space . 1 ∈ U U V Since is a proper subspace of , the vector en cannot be in . U V U Remark 7.1. Let j be the j-dimensional subspace spanned by e ,..., ej, Y 1 i.e., the set of all vectors with zeroes in the last n j entries. A similar argument shows that the nonzero invariant subspaces− of N are the subspaces ,..., n. Y1 Y We now construct some examples. Example 7.1. Assume 2 and that np 2 for some p . Define |I| ≥ ≥ ∈ I A as follows. 1. Aii = Ni for all i . ∈ I 2. If j = p, set Apj = 0. 6 3. If i = p let Aip be any matrix which has en in the first column. 6 i 4. If i = p, and j = p, and i = j, then Aij can be any ni nj matrix. 6 6 6 × Set i = i for i = p, and let p be the line spanned by e . Since np 2, the U V 6 U 1 ≥ subspace p is a nonzero, proper subspace of p. One can easily check that U V the subspaces i i properly reduce . {U } ∈I A We now show is not strongly reducible in the coupled sense. Suppose A there were nonzero, proper subspaces i i that reduced . (Note we must {U } ∈I A then have ni 2 for all i.) Each i is a nonzero, proper invariant subspace ≥ U 29 of Ni, so e i and en / i. Choose i = p. Then Aip has en in its first 1 ∈ U i ∈ U 6 i column, so Aipe = en . But e p and en / i, so Aip( p) i. Hence, 1 i 1 ∈ U i ∈ U U 6⊆ U we have a contradiction, and is not strongly reducible in the coupled sense. A Example 7.1 shows can be properly reducible in the coupled sense A without being strongly reducible. Thus, for any field F, when 2 and |I| ≥ ni 2 for at least one i, we have StrRed(F, ni i ) PropRed(F, ni i ). ≥ { } ∈I ⊂ { } ∈I The next example shows that if 4, and ni 2 for at least one value |I| ≥ ≥ of i, we have PropRed(F, ni i ) Red(F, ni i ). { } ∈I ⊂ { } ∈I Example 7.2. Assume 4 and that np 2 for some p . Choose any |I| ≥ ≥ ∈ I q , with q = p, and define as follows. ∈ I 6 A 1. Aii = Ni for all i . ∈ I 2. For all i with i = p and i = q, set Aip = 0 and Aiq = 0. 6 6 3. For all other choices of i, j with i = j, let Aij be any matrix with en1 in the first column. 6 We illustrate for = 1, 2,...,K , with p = 1 and q = 2. I { } N1 ∗ ∗ ∗ ··· ∗ N2 ∗ ∗ ∗ ··· ∗ 0 0 N3 ∗ ··· ∗ A = , 0 0 N4 ∗ ··· ∗ ...... ...... 0 0 NK ∗ ∗ ··· where each asterisk ( ) represents an ni nj matrix with en in the first ∗ × i column. Set p = p, and q = q. For all other values of i, set i = 0. One can U V U V U check that is coupled reducible via i i . ∈I We nowA show is not properly reducible.{U } Suppose could be properly A A reduced by subspaces i i . At least one i must be a nonzero, proper {U } ∈I U subspace; we first show this holds for at most one value of i. Suppose i U and j were both nonzero, proper subspaces, with i = j. We must then U 6 30 have ni 2 and nj 2. Since i is a nonzero, proper invariant subspace ≥ ≥ U of Ni, and j is a nonzero, proper invariant subspace of Nj, we know e is U 1 in both i and j, and en / i and en / j. If i = p and j = q, use the U U i ∈ U j ∈ U matrix Apq. Since Apqe = en , we see Apq( q) p. The same argument, 1 p U 6⊆ U using Aqp, applies when i = q and j = p. Suppose then that at least one of i, j is different from p and q. Without loss of generality, assume j p, q . 6∈ { } Then use Aij, which has en in its first column. So Aij( j) i. i U 6⊆ U So, at most one i is a proper, nonzero subspace; each of the other sub- U spaces is either the whole space i or the zero subspace. Assume i is the V U nonzero, proper subspace; note ni 2. We claim we can then choose j = i ≥ 6 so that Aij has eni in the first column and Aji has enj in the first column. If i = p, choose j = q, and if i = q, choose j = p. If i = p and i = q, choose 6 6 any j which is different from i, p and q. (This is where we use the fact that 4.) The subspace j is either the full space j, or it is the zero sub- |I| ≥ U V space. If j = j, then Aij( j) contains Aije = en , which is not in i. So U V V 1 i U Aij( j) i. If j = 0 , then, since e i, we have Ajie = en Aij( i). U 6⊆ U U { } 1 ∈ U 1 j ∈ U So Aji( i) j. Hence, is not properly reducible in the coupled sense. U 6⊆ U A In the example above, we needed 4. What can we say when K =2 |I| ≥ or K = 3? In these cases, the field F must be considered. The reason is, that for to be properly reducible in the coupled sense, at least one Aii must have a nonzero,A proper invariant subspace. If F is algebraically closed and n 2, ≥ then any n n matrix over F has an eigenvalue in F, and the line spanned by a corresponding× eigenvector is a nonzero, proper invariant subspace. But if F is not algebraically closed, there may be n n matrices over F which have no proper invariant subspaces. We shall give× an example for the real numbers later, but first we show that if F is an algebraically closed field, then PropRed(F, n, 2) = Red(F, n, 2) and PropRed(F, n, 3) = Red(F, n, 3) for all n 2. ≥We use the following lemma to deal with the cases K = 2 and K = 3. Lemma 7.1. Let = Aij i,j where Aij is ni nj. Suppose is coupled A { } ∈I × A reducible with i i satisfying one of the following. {U } ∈I 1. p = p for exactly one index value p, and i = 0 when i = p. U V U { } 6 2. p = 0 for exactly one index value p and i = i when i = p. U { } U V 6 Suppose p is a nonzero, proper invariant subspace of App. Then is prop- W A erly reducible by coupled similarity via the subspaces obtained by replacing p by p, and leaving the other i’s unchanged. U W U 31 Proof. Since p is the only subspace that is changed, we continue to have U Aij( j) i whenever i and j are both different from p. Also, p is chosen U ⊆ U W to satisfy App( p) p. It remains to consider Aip and Api for i = p. W ⊆ W 6 In case 1, we have i = 0 for i = p, so Api( i) = 0 p. We also U { } 6 U { } ⊆ W have Aip( p) i = 0 . Since p = p, we must have Aip( p)= 0 = i. U ⊆ U { } U V W { } U In case 2, we have i = i for i = p, and p = 0 . So Api( i) = 0 U V 6 U { } U { } ⊆ p. We also have Aip( p) i = i for i = p. W W ⊆V U 6 Now suppose F is algebraically closed, and n 2. Any n n matrix ≥ × over F has a nonzero, proper invariant subspace. For K = 2, Lemma 7.1 immediately tells us that is coupled reducible if and only if it is properly reducible, i.e., PropRed(FA, n, 2) = Red(F, n, 2). For the case K = 3, suppose is reduced by 1, 2, 3. If none of the i’s is a nonzero proper subspace, Athen each is eitherU UorU 0, so either two ofU them are , with the third being V V zero, or vice versa, two of them are zero, with the third being . Lemma 7.1 then tells us is properly reducible. Hence, for algebraicallyV closed F and n 2 we haveAPropRed(F, n, 3) = Red(F, n, 3). ≥ If F is not algebraically closed, then a matrix over F need not have a proper invariant subspace. Consider the case F = R, the field of real numbers. Let A be a real n n matrix, where n 2. The eigenvalues of A are in C, and the non-real eigenvalues× occur in conjugate≥ pairs. If λ is a real eigenvalue of A then there is a corresponding real eigenvector, v, and the line spanned by v is a proper, nonzero invariant subspace of A. For a pair of complex conjugate, non-real eigenvalues, λ, λ, there is a corresponding two dimensional invariant subspace. Consider the following example for K 2 and 2 2 real matrices. ≥ × Example 7.3. Choose an angle θ with 0 <θ<π. For 1 i K, set ≤ ≤ cos θ sin θ Aii = − . sin θ cos θ This is the matrix for rotation of the plane R2 by angle θ. Since no line through the origin is mapped to itself by this rotation, this map has no nonzero, proper invariant subspace. Hence, for any choice of the Aij’s when i = j, the set is not properly reducible in the coupled sense. It is, however, 6 A possible to find Aij’s such that is reducible in the coupled sense. Choose A a positive integer s with 1 s So for K 2, we have PropRed(R, 2,K) Red(R, 2,K). From Ex- ≥ ⊂ ample 7.2, we already knew this for K 4; the new information is that PropRed(R, 2, 2) Red(R, 2, 2) and PropRed≥ (R, 2, 3) Red(R, 2, 3). However, for n⊂ 3, any n n real matrix has a nonzero⊂ proper invari- ≥ × ant subspace. Lemma 7.1 then gives PropRed(R, n, 2) = Red(R, n, 2) and PropRed(R, n, 3) = Red(R, n, 3) when n 3. ≥ Acknowledgements. D. Lahat thanks Dr. Jean-Fran¸cois Cardoso for in- sightful discussions during her Ph.D., which set the foundations for and mo- tivated this work. References [1] T. Adalı, D. Lahat, and C. Jutten. Data fusion: Benefits of fully exploiting diversity. Tutorial presented at EUSIPCO, Aug. 2016. [2] Michael Artin. Algebra, 2nd edition, Pearson Prentice Hall, New Jersey, 2011. [3] Brualdi and Ryser, Combinatorial Matrix Theory, Cambridge Uni- versity Press, 1991. [4] D. Lahat. A data fusion perspective on source separation. Talk presented at the Hausdorff School: Low-rank Tensor Techniques in Numerical Analysis and Optimization, Apr. 2016. [5] D. Lahat and C. Jutten, Joint blind source separation of multidi- mensional components: Model and algorithm, in Proc. EUSIPCO, Lisbon, Portugal, Sep. 2014, pp. 1417–1421. [6] D. Lahat and C. Jutten, A generalization to Schurs lemma with an application to joint independent subspace analysis, Technical 33 Report hal-01247899, GIPSA-Lab, Grenoble, France, Dec. 2015, https://hal.archives-ouvertes.fr/hal-01247899. [7] D. Lahat and C. Jutten. On the uniqueness of coupled matrix block diagonalization in the joint analysis of multiple datasets (talk). In SIAM Conference on Applied Linear Algebra, Atlanta, GA, USA, Oct. 2015, http://meetings.siam.org/sess/dsp talk.cfm?p=72077. [8] D. Lahat and C. Jutten, Joint independent subspace analysis using second-order statistics, IEEE Trans. Signal Process., 64 (2016), pp. 4891–4904, https://doi.org/10.1109/tsp.2016. [9] D. Lahat and C. Jutten. Joint independent subspace analysis by cou- pled block decomposition: Non-identifiable cases. In Proc. ICASSP, Calgary, Canada, Apr. 2018. [10] D. Lahat and C. Jutten, Joint independent subspace analysis: Uniqueness and identifiability IEEE Trans. Signal Process., 2018, to appear, DOI 10.1109/TSP.2018.2880714 [11] K. Murota, Y. Kanno, M. Kojima, and S. Kojima, A numerical algorithm for block-diagonal decomposition of matrix *-algebras with application to semidefinite programming, Japan J. Indust. Appl. Math., 27 (2010), pp. 125–160. [12] I. Schur, Neue Begr¨undung der Theorie der Gruppencharak- tere, Sitzungsberichte der K¨oniglich-Preussischen Akademie der Wis- senschaften zu Berlin, Berlin, Germany, Jan.–Jun. 1905, pp. 406–432, https://archive.org/details/sitzungsberichte1905deutsch. [13] J. J. Sylvester. Sur l’´equation en matrices px=xq. Comptes Rendus, 99: 67–71, 1884. 34