Isotropy Groups of the Action of Orthogonal Similarity on Symmetric

Home , Toeplitz matrix

arXiv:2108.06757v1 [math.DG] 15 Aug 2021 euetenotation the use We 13 Orb( (1.3) the and nobttu ossso rhgnlysmlrmtie and isomorphic. matrices are similar matrices orthogonally these of consists thus orbit An Q h etrsaeo all of space vector the (1.4) A ersnainof representation a The (1.1) h cin(.)i h anproeo hsppr Moreover, paper. this of purpose at main representation the isotropy is (1.1) 15]) [7, action provide monographs the is (see groups action isotropy group corresponding the a concerning information important 1,1,1]o h emtyo ymti arcsasr tha assure matrices symmetric under of matrices geometry metric the on 12] 11, [10, e,uioetgop opizmatrices Toeplitz group, unipotent ces, fnonsingular of (1.2) ain bet fOrb( of objects variant agn pc fOrb( of space tangent STOYGOP FTEATO FOTOOA SIMILARITY ORTHOGONAL OF ACTION THE OF GROUPS ISOTROPY = ( = l arcscniee nti ae r ope nesoth unless complex are paper this in considered matrices All h cin(.)dsrbssmere of symmetries describes (1.1) action The eerhspotdb rn 109 rmAR,Rpbi fS of Republic ARRS, from P1-0291 grant by supported Research e od n phrases. and words Key 2000 Date stoygroup isotropy A Q T uut1,2021. 13, August : Σ T ahmtc ujc Classification. Subject Mathematics e further Let . e tpi u ro st ov eti etnua block rectangular certain a solve to is proof equation. our matrix complex Toeplitz on in similarity step of key action A the o group to the respect of with subgroups matrices isotropy the of structure the scribe orbit Abstract. A ) − × 1 h cinof action The . T A of → A n Φ efida loihi rcdr htealst opt n t and compute to enables that procedure algorithmic an find We T × is at A : n , Σ C O O A 1. stoygop,mti qain,otooa arcs s matrices, orthogonal equations, matrix groups, isotropy matrices A A m n n A ( ∈ ( T × ) n ( Q,A C nacmlxvco space vector complex a on nrdcinadtemi result main the and Introduction n se[,1].O h te ad(.)cnb ena a as seen be can (1.1) hand other the On 13]). [7, (see ) C ⊂ cnrec wihicue 11)i ut eea.An general. quite is (1.1)) includes (which -congruence S NSMERCMATRICES SYMMETRIC ON × ) Σ rhgnlsimilarity orthogonal o h e fmtie fsize of matrices of set the for n etesbru fotooa arcsi h group the in matrices orthogonal of subgroup the be ) A × S ( A n C ) n ∈ S := 7→ ( ymti matrices; symmetric ihrsett h cin(.)is (1.1) action the to respect with ) n C S A ( GL n C { at ) := ) Q Q ( C ) AE STAR TADEJ T n → ∈ 52,5H0 20 . 32M05 51H30, 15A24, stersrco f(1.1): of restricion the is ) ( Q T AQ, { A C Q O S .Amatrix A ). seas e.4.I scoeyrltdt in- to related closely is It 4). Sec. also (see T n n ( AQ ( C 1 C ) ) , | | Q CI Q ˇ A T S C ˇ on ∈ := AQ n ( ( O Q,A C S { n X Q .Hasfnaetlresults fundamental Hua’s ). n A = ( ( T C T C ) A sotooa fadol if only and if orthogonal is A A ssmerci n nyif only and if symmetric is ) lovenia. 7→ sdfie sfollows: as defined is ) } } m . ⊂ + ope orthogonal complex f ymti matrices. symmetric Q AX × S T n h stoygop of groups isotropy the n upper-triangular n ofidteefor these find to and , ( AQ. By . C | h ocle linear so-called the yisobt and orbits its by d X h td fsym- of study the t soitdt the to associated ) = riei stated. is erwise S n − ( X mercmatri- ymmetric C de- o T edenote we ) ∈ C n × n } , 2 TADEJ STARCIˇ Cˇ

C representation of On( ); note that the classiﬁcation of representations of complex classical groups along with their invariants is well understood (see e.g. [18]). Fi- nally, the isotropy groups of (1.1) are interesting from the linear algebraic point of view (check Remark 1.4). To be able to compute the isotropy groups, it is essential to have simple repre- sentatives of orbits. Thus we recall the symmetric canonical form under similarity; remember that symmetric matrices are similar if and only if they are orthogonally similar (see e.g. [6]). Given a matrix A with its Jordan canonical form:

(1.5) J(A) = J (λ ), λ C, nj j j ∈ Mj where z 1 0 .  z ..  ( ) :=   C ( -by- ) Jn z  , z n n ,  ..  ∈  . 1   0 z     the symmetric canonical form is 

(1.6) S(A) = Knj (λj), Mj in which 2z 1 0 0 1 0 − 1 ...... (1.7) ( ) := 1 . . + . . 1 C ( -by- ) Kn z  . .  i . . , z n n . 2  .. .. 1   1 .. ..  ∈  0 1 2z   −0 1 0          It is uniquely determined up to a permutation  of its direct summands. See [5] for the tridiagonal symmetric canonical form. Since the equation QT AQ = A is equivalent to (J(A))X = X(J(A)) with J(A) = 1 1 PAP − , X = PQP − , the following fact on isotropy groups follows immediately from the classical result on solutions of Sylvester’s equation (see Theorem 2.1 (1)). k Proposition 1.1. If λ1,...,λk are distinct eigenvalues of S = j=1 Sj, where each Sj is a direct sum whoose summands are of the form (1.7) and correspL ond to the eigenvalue Σ = k Σ = λj, it then follows that S j=1 Sj . Furthermore, if Sj λj Inj for some index j, then Σ = O (C). (We denoteL the n n identity-matrix by I .) Sj nj × n Therefore the isotropy groups under (1.1) of matrices with all distinct eigenvalues (hence with nonvanishing discriminants of their characteristic polynomials) are trivial. The situation in the generic case (on a complement of a complex ana- lytic subset of codimension 1) is thus quite simple. Our aim is to inspect the nongeneric matrices (especially nondiagonalizable). The principal object of the investigation will be (up to similarity) the group of all nonsingular matrices commuting with a given square matrix M, i.e. nonsingular solutions of the homogeneous linear Sylvester’s equation MX = XM; see Sec. 2 for 3 its properties. First, recall that a block upper-triangular Toeplitz matrix is:

A0 A1 A2 ...... Aβ 1 .−  0 .   A0 A1 A2 .     ......   . . A0 A1 . .  T (A0,A1,...,Aβ 1)=   (β-by-β), −  . . . .   ...... A   2   .   . .. ..   . . . A1     0 ...... 0 A0      Cm n  β where A0,A1,...,Aβ 1 × and T (A0,A1,...,Aβ 1) = [Tjk] =1 with Tjk = 0 for − ∈ − j,k j>k and T = T . Next, suppose α >α > >α and m ,...,m N. jk (j+1)(k+1) 1 2 ··· N 1 N ∈ Let be an N N block matrix such that its block rs is a rectangle αr αs block upper-triangularX × Toeplitz matrix with blocks od sizeXm m : × r × s [0 ], α <α Trs r s (1.8) = [ ]N , =  rs , α >α , b = min α ,α , rs r,s=1 rs  T0 r s rs s r X X X  " # { }  , α = α  Trs r s  in which rs is a brs brs block upper-triangular Toeplitz matrix. It turns out that orthogonalT solutions× of the equation SX = XS with S of the form (1.6) are related to matrices of the form (1.8) such that the following properties are satisﬁed: (I) The nonzero entries of for r>s can be taken as free variables. Xrs r r r C (II) If s = r, then rr = T (A ,...,A ), where A Om ( ) can be any orthogonal X 0 αr 1 0 ∈ r matrix, and for α 2, j 1,...,α − 1 we have Ar = Ar Zr + Dr for some freely r ≥ ∈{ r − } j 0 j j r r T r Cmr mr chosen skew-symmetric Zj = (Zj ) of size mr mr , and with Dj × de- − × p p ∈ pending uniquely (and polynomially) on the entries of A , Z with j 1,...,j 1 , 0 j ∈{ − } p 1,...,N and on the entries of for p,t 1,...,N with p>t. ∈{ } Xpt ∈{ } e e

(III) The entries of rs for rdiagonal matrix of the form (1.8) is N r r (1.9) = T (Imr ,W1 ,...,Wα 1), W r − Mr=1 n r 1 r r 1 r r T r W1 := Z1, Wn+1 := Zn+1 (Wj ) Wn j+1 , n 1, 2 2 − − ≥ Xj=1 r in which all Zn are skew-symmetric. Another special matrix of the form (1.8) contains the identity matrix as principal submatrix, formed by all blocks except those at the p-th and the t-th columns and rows, while blocks in the p-th and the t-th columns and rows are as follows: [ 0 rs ], α <α U r s k k N k rs (1.10) p,t(F) = [( p,t(F))rs] =1, ( p,t(F))rs =  U , αr >αs , p α >...>α , and let X be r=1 j=1 αj ∈ 1 2 N partitionedL conformallyL to blocks as J. Then X is a solution of (2.1) if and only if X = [X ]N is such that every block X is further a m m block matrix rs r,s=1 rs r × s with blocks of size α α and of the form r × s [0 T ], αr <αs T (2.2)  , α >α ,  0 r s  " #  T, α = α  r s  in which T is an b -by-b upper-triangular Toeplitz matrix (b = min α ,α ). rs rs rs { r s} For our developments it is convenient to work with matrices with smaller num- ber of blocks. This can be achieved by conjugating with a suitable permutation matrix (see e.g. [14, Sec. 3.1]). Let e1,e2,...,eαm be the standard orthonormal basis in Cαm. We set a permutation matrix formed by these vectors:

(2.3) Ωα,m := e1 eα+1 ... e(m 1)α+1 e2 eα+2 ... e(m 1)α+2 ... eα e2α ... eαm . h − − i Observe that multiplication with Ωα,m from the right puts the 1-st, the (α + 1)-th, . . . , the ((m 1)α + 1)-th column together, further the 2-nd, the (α + 2)-th, . . . , the − ΩT ((m 1)α +2)-th column together, and soforth. Similarly, multiplicating with α,m from− the left collects the 1-st, the (α + 1)-th, . . . , the ((m 1)α + 1)-th row together, further the 2-nd, the (α + 2)-th, . . . , the ((m 1)α + 2)-th− row together, and soforth. − Suppose X = [X ]N is as in Theorem 2.1 (2). Next, ﬁx r,s and let b = min α ,α . rs r,s=1 { r s} Denote the block of Xrs in the j-th row and the k-th column by

[0 Tjk], αr <αs T j 1,...mr , k 1,...ms , (X ) =  jk , α >α , ∈{ } ∈{ } rs jk  0 r s jk jk jk  " # Tjk := T (a0 ,a1 ,...,ab 1).  T , α = α −  jk r s   6 TADEJ STARCIˇ Cˇ

jk m ,m r s Cmr mr By setting An := [an ] =1 × for n 0,...,b 1 and = T (A0,...,Ab 1), we j,k ∈ ∈{ − } T − obtain a rectangular block upper-triangular Toeplitz matrix of size α α : r × s [0 ], α <α T r s ΩT X Ω =  T , α >α . αr ,mr rs αs,ms  0 r s  " #  , α = α  T r s  Thus we get a matrix of the form (1.8):  (2.4) ΩT XΩ =[ΩT X Ω ]N , (Ω := N Ω ). αr ,mr rs αs,ms r,s=1 ⊕r=1 αr ,mr Example 2.2. N = 2, α1 = 3, m1 = 2, α2 = 2, m2 = 3:

a1 b1 a2 b2 a3 b3 a1 a2 a3 b1 b2 b3  0 a1 0 a2 0 a3 a4 a5 a6 b4 b5 b6 ΩT  0 0 0 0 0 0 Ω  0 0 0 a1 a2 a3 3,2   2,3 =  . a4 b4 a5 b5 a6 b6  0 0 0 a4 a5 a6      0 a 0 a 0 a   0 0 0 0 0 0   4 5 6    0 0 0 0 0 0       0 0 0 0 0 0          Next, we observe that the set of nonsingular matrices of the form (1.8) has a special group structure, similar to the group of all nonsingular upper-triangular matrices. We use ideas from the proof of a similar (maybe somewhat stronger) result for upper-unitriangular matrices [3, Proposition 3.31], [15, Example 6.49]. Lemma 2.3. Let T be the set of all nonsingular matrices of the form (1.8). Then T is a subgroup of the group of all nonsingular matrices. Furthermore, T = D⋉U is a semidirect product of subgroups, where D contains all nonsingular block-diagonal matrices and U is a normal subgroup that consists of matrices whoose diagonal blocks are block upper-triangular Toeplitz matrices with identity as the diagonal block. Further, U is unipotent of order at most α 1 and it has has nilpotency class at most α . 1 − 1 Proof. First, we examine the set U of all nonsingular matrices of the form (1.8) such that their diagonal blocks are block upper-triangular Toeplitz matrices with identities as the diagonal blocks.

For k 1,...,α1 1 let Nk be the set of nonsingular matrices of the form (1.8) ∈{ − }rs rs rs rs rs with rs = T (0,...,0,A ,A ,...,A ) (i.e. A = ... = A = 0) for brs >k and T k k+1 brs 1 0 k 1 =0 for k b , and such that all A−rr = 0. We have − Trs ≥ rs k U =: N0 N1 Nα 1 = 0 . −I ⊃ ⊃···⊃ 1− { } Sums and products of rectangular upper-triangular Toeplitz matrices of the appropriate size are again rectangular upper-triangular Toeplitz matrices. Moreover, N + N N , N N N , N N N . k k ⊂ k 0 k ⊂ k+1 k 0 ⊂ k+1 α k 1 In particular N 1− − = 0 , thus matrices in N are nilpotent. For N we have k { } k N ∈ k 1 2 α k 1 α k 1 ( + )− = + ... + ( 1) 1− − 1− − . I N I−N N − − N Hence U := + N is a unipotent group. Taking + U and + U with k I k I N ∈ k I N ′ ∈ ( + ) 1 = + ( )2 ... we get I N ′ − I−N ′ N ′ − 1 2 ( + ′)− ( + )( + ′) = + ( ′ + ( ′) ...) ( + ′) U , I N I N I N I I−N N − N I N ∈ k 7 and the commutator is of the form 1 1 [ + , + ′] = ( + )− ( + ′)− ( + )( + ′) I N I N I N I N I N I N 2 2 = ( + ...)( ′ + ( ′) ...) ( + )( + ′) I−N N − I−N N − I N I N = ( ′ + )( + + ′ + ) I−N−N M1 I N N M2 = + U , I M3 ∈ k where , , N . Hence M1 M2 M3 ∈ k+1 U U U U (2.5) = 0 1 α 1 = ⊃ ⊃···⊃ 1− {I} U U U is a central series of normal subgroups, i.e. [ , j ] is a commutator group of j+1. Any T (nonsingular and of the form (1.8)) can be written as = , where U andX ∈ D is a nonsingular block-diagonal matrix of the formX DU (1.8). For U ∈ D D ∈ U 1 1 1 1, 2 and 1, 2 we get that ( 1 1)( 2 2)− = 1( 1 2− ) 2− is of the DformD (1.8),∈ thus TU isU a group.∈ Next, conjugatingD U D U+ D+ NU =UU byD gives I N ∈I 0 DU 1 1 1 1 − − ( + ) = + − − U, U D I N DU I U D NDU ∈ This proves normality of U and concludes the proof. Remark 2.4. It would be interesting to know whether (2.5) is a lower central sequence or not. Note that the situation seems more involved than in the case of upper-unitriangular matrices, in which the commutators of suitably chosen generators are again generators (see [3]).

~~3. Certain block matrix equation~~

In this section we consider certain block upper-triangular Toeplitz matrix equation. Its solution (Lemma 3.1) is the key ingredient in the proof of Theorem 1.3. Let α >α >...>α and m ,...,m N. Suppose 1 2 N 1 N ∈ N N N r r r r r r (3.1) = T B0,B1,...,Bα 1 , = T C0,C1,...,Cα 1 , = Eαr (Imr ), B r − C r − F Mr=1 Mr=1 Mr=1 r r C C r r r r C B0,C0 GLmr ( ) Smr ( ),B1,C1,...,Bα 1,Cα 1 Smr ( ), ∈ ∩ r − r − ∈ 0 Im where E (I ) = .. is an α α block matrix with I on the anti-diagonal and α m . × m " Im 0 # zero-matrices otherwise. We shall solve a matrix equation (3.2) = T , C FX FBX where = [ ]N is of the form as in (1.8). X Xrs r,s=1 We first observe a few simple facts. The calculation ( T )T = T T = T ( T ) = ( T ) FX FBX X B F X F FFX F FB F X F F FX FBX F T T shows that for r , s we have ( )rs = 0 if and only if ( )sr = 0. When comparing the left-handFX sideFBX with the right-hand side oFXf (3.2)FBX blockwise, it thus suffices to observe only blocks in the upper-triangular parts of XT X T F FB and . Since ( )rs and rs are rectangle block Toeplitz and of the same formC for each r,sFX, itFBX is enough toC compare the first rows of these blocks. 8 TADEJ STARCIˇ Cˇ

The following lemma explains the process of computing solutions of (3.2). In the proof of Theorem 1.3 we shall obtain (3.2) for and equal to the identity- matrix. However, due to a possible application whenB computiC ng isotropy groups of actions similar to (1.1) and since it makes no serious diﬀerence to the proof, we prove a little more general result. Lemma 3.1. Let , as in (3.1) be given. Then the dimension of the space of solutions B C of (3.2) that are of the form = [ ]N (partitioned conformally to , ) with X Xrs r,s=1 B C [0 ], α <α Trs r s α1 >α2 >...>αN , brs := min αs,αr (3.3) =  rs { } rs  T , αr >αs , rs rs rs rs Cmr ms X  0 rs = T A ,A ,...,A ,A ×  " # 0 1 brs 1 j  T − ∈  rs, αr = αs  T N  1 r 1 is α m (m 1) + − m . In particular, the general solution satisﬁes the r=1 r r 2 r − s=1 s followingP properties: P (a) The entries of Arr for r 1,...,N can be taken so that Arr is any solution of the 0 ∈{ } 0 equation Cr = (Arr )T Br Arr . If N 2 the entries of Ars for r,s 1,...,N with 0 0 0 0 ≥ j ∈{ } r>s and j 0,...,α 1 can be taken as free variables. ∈{ r − } (b) Assuming (a) and choosing the entries of matrices Zr = Zr Cmr mr for r j − j ∈ × ∈ 1,...,N , αr 1 j 1 as free variables, the remaining entries of are computed by{ the following} − ≥ algorithm:≥ X

n n j Ψ krs − kr T k ks n := j=0 l=0(Aj ) Bn j lAl − − for j =0:Pα1 P1 do − if r 1,...,N , 1 j αr 1 then ∈{ } ≤ ≤ − j 1 n j rr rr rr r 1 r − − rr T k kr Aj = A0 A0 (C0)− Zj + l=1 m=0(Al ) Bn j mAm − − − Pr 1 ΨPrr N Ψ rr + k−=1 j α +α + k=r+1 j α +α − k r − r k end if P P for p =1: N 1 do if r 1,...,N− , j α 1, r + p N then ∈{ } ≤ r+p − ≤ r(r+p) r(r+p) r 1 j n j rr T k k(r+p) r 1 Ψ r(r+p) Aj = A0 (C0)− l=1 m−=0(Al ) Bn j mAm + k−=1 j α +α − − − − k r P rP+p Ψ r(r+p) N Ψ r(rP+p) + k=r+1 j + k=r+p+1 j α +α , − r+p k end if P P end for end for n For simplicity, in this algorithm we define j=l aj = 0 if l>n, and it is understood that the inner loop (i.e. for p =1 : N-1) isP not performed for N = 1. Furthermore, assume that and are real. Then the solution is real if and only if B C X (i) Matrices Br and Cr in (3.1) have the same inertia for all r 1,...,N . 0 0 ∈{ } (ii) Matrices Ars with r>s, j 0,...,α 1 , N 2, matrices Arr , and matrices Zr j ∈{ r − } ≥ 0 j for 1 j α 1 (r,s 1,...,N ) in (a) and (b) are chosen real. ≤ ≤ r − ∈{ } For the sake of clarity we point out the importance of the correct order of calculating the entries of in Lemma 3.1. It is essential for the proof of the lemma. Re- call first that by (a) (whenX N 2) all entries of the blocks below the main diagonal N ≥ N r 1 of = [ ] can be chosen freely ( − α m m free variables). Next, we X Xrs r,s=1 r=1 s=1 r r s P P 9 compute the diagonal entries of the blocks in the upper triangular part of . We first obtain the diagonal entries of the main diagonal blocks for r 1,...,NX ; Xrr ∈{ } they add N 1 m (m 1) to the dimension of the solution space (see (a) again). r=1 2 r r − Secondly,P step j = 0, p = 1 (if N 2) of the algorithm in (b) yields the diagonal ≥ r(r+1) entries of the first upper off-diagonal blocks of (i.e. ( ) = A ). Fur- X Xr(r+1) 11 0 ther, step j = 0, p = 2 gives the diagonal entries of the second upper off-diagonal r(r+2) blocks of (i.e. ( r(r+2))11 = A0 ), step j = 0, p = 3 gives the diagonal entries of X X r(r+3) the third upper off-diagonal blocks of (i.e. ( ) = A ), and soforth. In X Xr(r+3) 11 0 the same fashion the step for fixed j 1,...,α1 1 and p 0,...,N yields the entries on the j-th upper off-diagonals of∈{ the p-th− upper} off-diagonal∈{ blocks} of (i.e. r(r+p) X ( r(r+p))1(j+1) = Aj+1 with r + p N, provided that j αr+p 1). Finally, at step X ≤ 11 ≤ − j = α1 1, p = 0 we compute ( 11)1α1 = Aα 1. Note that when calculating each en- − X 1− try Arr Cmr mr , we add 1 m (m 1) free variables. Furthermore, this algorithmic j ∈ × 2 r r − procedure allows to compute each entry from the entries that are already known.

Proof of Lemma 3.1. The idea is to write the equation (3.2) entrywise as a system of several simpler matrix equations and then consider them in an appropriate order. First, we analyze the right-hand side of the equation (3.2) for , of the form B F (3.1) and = [ ]N with blocks as in (3.3). To simplify the notation we set X Xrs r,s=1 := and := XT . The entries in the j-th column and in the first row of Y BX X F F ( ) are obtained by multiplying the first rows of the blocks ,..., with XY rs e Xr1 XrN the j-th columns of the blocks ( )1s,...,( )Ns, respectively, and then adding them: e Y Y e e N (3.4) (( ) ) = ( ) ( )(j), r,s 1,...,N , j 1,...,α . XY rs 1j Xrk (1) Yks ∈{ } ∈{ s} Xk=1 e e As mentioned in the discussion in the beginning of this section it suffices analyse the upper-triangular blocks of : XY N (( ) ) = ( ) e( )(j), 1 j α , 0 p N r. XY r(r+p) 1j Xrk (1) Yk(r+p) ≤ ≤ r+p ≤ ≤ − Xk=1 e e When N = 1 (hence r = 1, p = 0) we have (3.5) (( ) ) = ( ) (( ) )(j), XY 11 1j X11 (1) Y 11 while for N 2 we obtain e e ≥ N (3.6) (( ) ) =( ) ( )(j) + ( ) ( )(j), (r = 1) XY 1(1+p) 1j X11 (1) Y1(1+p) X1k (1) Yk(1+p) Xk=2 e e N e (3.7) (( ) ) =( ) ( )(j) + ( ) ( )(j) XY r(r+p) 1j Xrr (1) Yr(r+p) Xrk (1) Yk(r+p) kX=r+1 e er 1 e − + ( ) ( )(j), 1

~~For any r,k 1,...,N we have ∈{ } T (3.9) E (I ) T (A ,A ,...,A ) E (I ) = T (AT ,AT ,...,AT ), ark mk 0 1 brk 1 ark mr 0 1 brk 1 − − and it further implies~~

rk , α >α T0 r k T e kr T kr T rk = Eα (Im ) Eα (Im ) =  [h0 rki], α <α , Trk = T (A0 ) ,...,(A ) , X r r Xkr k k  T r k bkr  , α = α e  rke r k e  T  (Akr )T (Akr )T ... (Akr )T e0 1 ak 1 , αk αr (3.10) ( rk)(1) = − ≤ .  h 0 ... 0 (Akr )T ... (Akr )T i X 0 αr 1 , αk >αr  − e  h i Φks n k ks We define n := j=0 Bn jAj and observe that − P ks ks T0 , αk >αs S0 , αk >αs k k k (3.11) = T B ,B ,...,B 0 = 0 , ks 0 1 αk 1  h[ ksi ], αk <αs  [h ksi ], αk <αs Y −  T  S  ks, αk = αs  ks, αk = αs  T  S k k k  ks ks ks  ks ks ks ks = T B ,B ,...,B T A ,A ,...,A =T Φ ,Φ ,...,Φ . S 0 1 bks 1 0 1 bks 1 0 1 bks 1 − − − We begin with the calculation of matrices Arr for r 1,...,N . Since 0 ∈{ } k kr B0A0 kr T 0 [ (A0 ) ... ], k r (1)   . , k r ( rk)(1) = ∗ ∗ ≥ , ( kr ) =  . ≤ , X ( [ 0 ... ], kr     we get from (3.4) for r = s, j = 1 that  N (( ) ) = ( ) (( ) )(1) = ( ) ( )(1) = (Arr )T Br Arr , r 1,...,N . XY rr 11 Xrk (1) Y kr Xrr (1) Yrr 0 0 0 ∈{ } Xk=1 e e e Together with ( ) = Cr the equation (3.2) yields an equation that gives Arr : Crr 11 0 0 (3.12) Cr = (Arr )T Br Arr , r 1,...,N . 0 0 0 0 ∈{ } Next, if N 2, we fix arbitrarily the blocks below the main diagonal of [ ]N ≥ Xrs r,s=1 (hence the blocks above the main diagonal of [ ]N ). This corresponds to (a). Xrs r,s=1 Proceed with the key step in the proof: an inductive procedure that enables e to compute the remaining entries (i.e. the algorithm in (b)). We fix r 1,...,N , p 0,...,N r and j α 1, but not p = j = 0. Assuming that we have∈{ already} ∈{ − } ≤ r − determined the matrices Ar′ s′ (with 1 r ,s N) for j ≤ ′ ′ ≤

(3.13) j 1,j 0,...,je 1 ,s′ r′ or p 1,j = j,r′ s′ r′ + p 1 ≥ ∈{ − } ≥ ≥ ≤ ≤ − or s′ r′,j 0,...,b 1 ,N 2 ≤e ∈{ r′ s′ − } ≥ e r(r+p) we shall compute A e . Essentially, we shall solve the equation ( ) = j Cr(r+p) 1j (( ) ) (see (3.2)). By a careful analysis of the structures of ( ) and XY r(r+p) 1j Xrk (1) ( )(j) in formulas (3.6), (3.7), (3.8), we shall reduce this equation to a simple Yek(r+p) e r(r+p) r(r+p) T ﬃ linear matrix equation in Aj (and possibly (Aj ) ) with coe cients depend- ing only on Ar′s′ for (3.13). j e 11

For the sake of clarity we set the notation (n Z, k,r,s 1,...,N ): ∈ ∈{ } Φks n kr T kr T rr T . n (Akr )T Φks , n 0 Ψ krs (A0 ) (A1 ) ... (An )  . , n 0 j=0 j n j (3.14) n :=  ≥ = ≥ .  Φks  0P,− n< 0   0  (  0,  n< 0     Note that: 

n n j n n Ψ krs Φkr T ks kr T r T ks kr T r T ks n = ( j ) An j = (Al ) (Bj l) An j = (Al ) (Bj l ) An j − − − − − Xj=0 Xj=0 Xl=0 Xl=0 Xj=l n n l n − kr T r T ks kr T Φks Ψ ksr T (3.15) = (Al ) (Bj ) An l j = (Al ) n l = ( n ) . ′ − − ′ − Xl=0 jX′=0 Xl=0

(Arr )T ... (Arr )T Since ( rr )(1) = [ 0 αr 1 ] and X − e Φr(r+p) j Φrr . αr 1  .  (α 1) − (j+1) ( ) r = . , ( ) =  Φr(r+p) , j<α 1 or p 1, rr −  .  r(r+p)  0  s Y Φrr Y  0  − ≥  0       .     .     .   0    the ﬁrst term of (3.5), (3.6), (3.7), (3.8) is:  

(j+1) ( ) ( ) = Ψ rrr = (Arr )T Br Arr + (Arr )T Br Arr + Ξ(j,r,0), (p = 0) Xrr (1) Y rr j 0 0 j j 0 0 (j+1) rr(r+p) r(r+p) ( e ) ( ) = Ψ = (Arr )T Br A + Ξ(j,r,p), p 1, Xrr (1) Y r(r+p) j 0 0 j ≥ j 1 e − rr Φrr l=1 Al j l, j 1,p = 0 (3.16) Ξ(j,r,p) := − ≥ .  j r r(r+p)  P A Φ , j 0,p 1  l=1 l j l ≥ ≥  −  P n (For simplicity we have deﬁned l=1 al =0 for nk,   0    0  r(r+p)    Φ   .  j   .  (j+1) . (j+1)   0  ( (r+p)(r+p)) =  . , ( k(r+p)) =   Φr(r+p)  . Y Y   j αr+p α  Φr(r+p)   − − k  0   .     .      r(r+p)    Φ , k>r + p,   0    0    .    .    .    0        12 TADEJ STARCIˇ Cˇ

Hence for N r + 1 2: ≥ ≥ N (3.17) Θ(j,r,p) := ( ) ( )(j+1) Xrk (1) Yk(r+p) kX=r+1 e N Ψ krr , j 1,p = 0 k=r+1 j αr +αk = r+p kr− (r+p) kr(r+p) ≥  P Ψ + N Ψ , j 0,p 1.  k=r+1 j k=r+p+1 j αr+p+αk  − ≥ ≥  P P  N Ψ r(r+p) (For simplicity, we deﬁned k=r+p+1 j α α =0 for r + p + 1 > N.) − r+p− k Finally, the third term inP (3.7) and the second term in (3.8) (with N 2) contain of summands which are products of matrices ≥

Φk(r+p) j ... kr T kr T (j+1) k(r+p) ( ) = 0 ... 0 (A0 ) ... (A ) , ( ) =  Φ , 1 k r 1, rk (1) bkr k(r+p)  0  X Y  0  ≤ ≤ − h i  ...  e  0    hence   r 1 r 1 Λ − (j+1) − Ψ kr(r+p) (3.18) (j,r,p) := ( rk)(1)( k(r+p)) = j α +α . X Y − k r Xk=1 Xk=1 e We set the extensions by 0: Ξ(j,r,p), j 2,p 0 Θ(j,r,p), N r + 1 2 Ξ(j,r,p) = ≥ ≥ , Θ(j,r,p) = ≥ ≥ ( 0, otherwise ( 0, otherwise e Λ(j,r,p), N r 2 e Λ(j,r,p) = ≥ ≥ ( 0, otherwise ande deﬁne r(r+p) Ξ Θ Λ (3.19) Dj := (j,r,p)+ (j,r,p)+ (j,r,p).

The equation ( ) = (( e ) ) ecombinede with (3.5), (3.6), (3.7), (3.8) Cr(r+p) 1j XY r(r+p) 1j and with (3.16), (3.17), (3.18), (3.19) yields: e r(r+p) r(r+p) (3.20) (Arr )T Br A = D , p 1, 0 0 j − j ≥ (Arr )T Br Arr + (Arr )T Br Arr = Cr Drr , p = 0. 0 0 j j 0 0 j − j Ψ krs Moreover, from (3.15) it follows that n for n 0 and r = s is symmetric, thus Ξ(j,r,0), Θ(j,r,0), Λ(j,r,0) (and hence Cr Drr ) are≥ symmetric, too. j − j r(r+p) T To get Aj for p 1 we solve a simple equation of the form A X = B with ≥ rr given nonsingular A and arbitrary B, while to get Aj we solve the equation of the form AT X + XT A = B with known nonsingular A and symmetric B; the solution is 1 T 1 T 1 rr T r X = 2 (A )− B + (A )− Z with Z skew-symmetric. In particular, A = (A0 ) B0 with T 1 rr T r 1 r r 1 (A )− = ((A0 ) B0)− = A0(C0)− (see (3.12)). This proves the algorithm in (b). Ξ Θ Λ r(r+p) r(r+p) Furthermore, (j,r,p), (j,r,p), (j,r,p) (thus also Dj and Aj ) depend on the entries of Ar′ s′ with (3.13). It is straightforward to see that the algorithm in j e 13

(b) allows to compute each entry from the entries that are already known. More- over, the entries of Ars for either r = s, α 2, j 1,...,α 1 or s>r 1, N 2 j r ≥ ∈{ r − } ≥ ≥ are determined uniquely by the entries of all Ar′ s′ with j = 0, s = r or s r , j ′ ′ ′ ≤ ′ r′ j 0,...,αr 1 (chosen in (a)), by the entries ofe all Z ewith j 1,...,j 1 (if ∈ { ′ − } j ∈ { − } r′ ej 2) or j = j, r = r′, and when s>r, N 2 also by thee entriese of Z for all r′ ≥ ≥ j (chosen in (b)); r ,s 1,...,N . e ′ ′ r r ∈{ } If B0,G0 are real, then by Sylvester’s theorem the equation (3.12) has a real rr r r solution A0 precisely when B0,G0 are of the same inertia. The last statement of the lemma is then apparent.

Remark 3.2. (1) The equation in (a) is of the form C = XT BX with given nonsingular symmetric matrices B, C. By Autonne-Takagi factorization (see e.g. [8, Coro- larry 4.4.4]) B = RT IR, C = ST IS for some nonsingular R,S and the identity-matrix T 1 I. The above equation thus reduces to I = Y Y with Y = RXS− . When B and C are real with the same inertia matrix I, i.e. B = RT IR and C = ST IS for some real T 1 orthogonal R and S, we get I = Y IY with Y = RXS− (real pseudo-orthogonal). e e e (2) One could consider the equation (3.2) even when the diagonal blocks of , e e B are nonsingular. In this more general setting the equation C = AT BA is more C involved, while the solution of the equation AT X + XT A = B is known (see [2]).

Example 3.3. We solve (3.2) for = E4(I) E2(I) I, = = := I4(I) I2(I) I. Set F ⊕ ⊕ B C I ⊕ ⊕ A1 B1 C1 D1 H1 G1 J1  0 A1 B1 C1 0 H1 0   0 0 A B 0 0 0   1 1    =  0 0 0 A1 0 0 0 . Y    0 0 N1 P1 A3 B3 J3     0 0 0 N 0 A 0   1 3   0 0 0 0   R1 R3 A4   We compute:   AT BT CT DT N T P T RT 1 1 1 1 1 1 1 A1 B1 C1 D1 H1 G1 J1 0 AT BT CT 0 N T 0  1 1 1 1  0 A1 B1 C1 0 H1 0   T T   0 0 A1 B1 0 0 0  0 0 A1 B1 0 0 0   T   =  0 0 0 A 0 0 0  0 0 0 A1 0 0 0  = YY  1    0 0 T T T T T  0 0 N P A B J   H1 G1 A3 B3 R3  1 1 3 3 3  e  T  0 0 0 N 0 A 0   0 0 0 H 0 A3 0  1 3   1    0 0 0 T 0 T  0 0 0 R1 0 R3 A4  J1 J3 A4        T T T T T T T T T T T T A1 A1 A1 B1 + B1 A1 A1 C1 + C1 A1 A1 H1 + N1 A3 A1 G1 + B1 H1 + N1 B3 N1 J3 + R1 A4 T T T T T  +B1 B1 + N1 N1 ∗ +P1 A3 + R1 R3 +A1 J1   T T T T T T T   A1 A1 A1 B1 + B1 A1 A1 C1 + C1 A1 0 A1 H1 + N1 A3 0   T T   +B B + N N   1 1 1 1   T T T  =  A1 A1 A1 B1 + B1 A1 0 0 0     AT A 0 0 0   1 1   T T T T T T   A A3 A B3 + B A3 + R R3 A J3 + R A4   3 3 3 3 3 3   AT A 0   3 3   T   A A4   4    By comparing the diagonal of the diagonal blocks of the left-hand side and the right-hand side of = we deduce that A ,...,A are any orthogonal matrices. YY I 1 4 e 14 TADEJ STARCIˇ Cˇ

Next, we choose N1, P1, R1, R3 arbitrarily. The diagonal blocks on the first upper T T T T diagonal yield equations A1 H1 + N1 A3 = 0 and A3 J3 + R3 A4 = 0, which further implies H = A N T A , J = A RT A ; note that (AT ) 1 = A , (AT ) 1 = A . The 1 − 1 1 3 3 − 3 3 4 1 − 1 3 − 3 last upper diagonal gives N T J +AT J +RT A = 0, thus J = A (N T A RT A RT A ). 1 3 1 1 1 4 1 1 1 3 3 4− 1 4 By inspecting the first upper diagonal of the main diagonal blocks in = T T T T T YY I we obtain A1 B1 + B1 A1 = 0 and A3 B3 + B3 A3 + R3 R3 = 0, so we deduce B1,B3. T T T T T e Further, A1 G1 + B1 H1 + N1 B3 + P1 A3 + R1 R3 = 0 (observe the first upper diagonal of the first upper diagonal), so we get G1. The third and the fourth upper diagonal block of the first principal diagonal T T T T T T T T T block give A1 C1 +C1 A1 +B1 B1 +N1 N1 = 0, A1 D1 +B1 C1 +C1 B1 +D1 A1 +N1 P1 + P T N + RT R = 0 (see ), therefore C , D follow, respectively. 1 1 1 1 ∗ 1 1 The solutions of the equation (3.2) with a block diagonal matrix = form a group with relatively simple generators. Recall that U is the set of matricesC B of the form (1.8) with identity-matrices on the diagonals of the diagonal blocks. X N αr Lemma 3.4. The set of solutions of the equation (3.2) for = = r=1( j=1Br ) C B C X O ⋉ VC B ⊕ ⊕ with Br GLmr ( ) Smr ( ) is a semidirect product = , in which the group ∈ ∩ α B B B O N r Cmr mr consists of all matrices of the form = r=1( j=1Qr) with Qr × such that B Q ⊕ ⊕ ∈ T V U X Br = Qr Br Qr , and := (hence unipotent of order at most α1 1 and in B ∩ B V − nilpotency class at most α1). Moreover, is generated by matrices of the form B N r r (3.21) = T (Imr ,V1 ,...,Vα 1), V r − Mr=1 n r 1 1 r r 1 1 r r T r V1 := (Br )− Z1, Vn+1 := (Br )− Zn+1 (Vj ) BrVn j+1 , n 1, 2 2 − − ≥ Xj=1 r in which all Zn are skew-symmetric, and by matrices of the form [0 ], α <α Urs r s (3.22) k (F) = [( k (F)) ]N , pα , p,t p,t rs r,s=1 p,t rs  U0 r s H H H  " #  , α = α  Urs r s m m  where F C p× t and  ∈αr j=1Imr , r = s, r rr rs = ⊕ , r,s p,t , rr = T (Imr ,A1,...,Aα 1), r p,t , U ( 0, r , s { }⊂{ } U r − ∈{ } 1 T 1 n pp an 1Bp− (F BtFBp− ) Bp, j = n(2k + α β) 1 1 2n A = − − , an = j ( 0, otherwise −22n+1 n + 1 n ! 1 1 T n tt an 1Bt− (BtFBp− F ) Bt, j = n(2k + α β) A = − − , j ( 0, otherwise k k 1 T = N (F), = N ( B− F B ). Upt αt Utp αt − p t X Proof. For any 1, 2 we have: 1 TX X ∈ B1 1 T T 1 1 T 1 ( − ) ( − ) = ( − ) − = ( − ) − = F X1X2 FB X1X2 F X2 FFX1 FBX1X2 F X2 FBX2 1 T 1 T = ( − ) ( − ) = . F X2 FB B FX2 FB B 1 X X Thus 1 2− , so is a group. X X ∈ B B 15

X N αr r We describe the structure of . Lemma 3.1 for = = r=1( j=1B0) implies X B C B ⊕ ⊕ r that is of the form (1.8) such that its diagonal blocks have Q0 (satisfying r X ∈r T Br r B0 = (Q0) B0Q0) on the diagonal. Therefore can be written as = with O and U. Clearly O X (hence X ), thus X = O ⋉X V ,QY where VQ∈= XB U.Y Since ∈ V is a subgroupB ⊂ B of U, it isY a ∈ normalB subgroupB inB X B, unipotentB of orderB ∩ at most α B 1, and nilpotent of class at most α (see LemmaB 2.3). 1 − 1 Next, we ﬁnd matrices in X that are of a simple form. First, set B k ∆11 ∆12 α,β = ∆ ∆ , α>β, 0 k β 1 D " 21 22# ≤ ≤ −

(3.23) ∆11 = T (Im ,A1,...,Aα 1), ∆22 = T (Im ,D1,...,Dβ 1), 1 − 2 − k T (G0,G1,...,Gβ 1) ∆21 = 0 N (F) , ∆12 = − , β " 0 # h i in which N k(F)isa β β block matrix with F Cm1 m2 on the k-th diagonal above β × ∈ × the main diagonal and zeros othervise, A Cm1 m1 , D Cm2 m2 , and G Cm2 m1 j ∈ × j ∈ × j ∈ × for all j. Further, suppose k is a solution of the matrix equation Dα,β = ( k )T k , = I (B) I (C), = E (I ) E (I ), Bα,β Fα,β Dα,β Fα,βBα,βDα,β Bα,β α ⊕ β Fα,β α m1 ⊕ β m2 m m m m where B C 1× 1 and C C 2× 2 . Blockwise we have ∈ ∈ k T T T Nβ (F ) k (3.24) Iα(B)= T (Im ,A ,...,A )Iα(B)T (Im ,A1,...,Aα 1)+ Iβ(C) 0 Nβ (F) , 1 1 α 1 1 0 − − h i k T T T T (G0,G1,...,Gβ 1) Nβ (F ) (3.25) 0= T (Im1 ,A1 ,...,Aα 1)Iα(B) − + Iβ(C)T (Im2 ,D1,...,Dβ 1), − 0 0 − T T (3.26) Iβ(C)= T (Im2 ,D1 ,...,Dβ 1)Iβ(C)T (Im2 ,D1,...,Dβ 1) − − T T T T (G0,G1,...,Gβ 1) + 0 T (G0 ,G1 ,...,Gβ 1) Iα(B) − . − 0 h i To determine k we follow the algorithm in Lemma 3.1. Dα,β We first simplify the notation by defining A0 := Im1 and D := Im2 . By comparing the first row of the left-hand and the right-hand side of (3.24) we get

~~n 2k+α β − T , T T 0 = Aj BAn j, 1 n 2k + α β, 0 = Aj BA2k+α β j + F CF, − ≤ − − − Xj=0 Xj=0~~

If α β+k 2 we satisfy the ﬁrst equation for 1 n 2k+α β 1 by choosing A1 = − ≥ ≤ ≤ T − − T ... = A2k+α β 1 = 0. The second equation then yields F CF = A2 + B+BA2k+α β − − − k α β − 1 1 T− (in the case α β + k = 1 as well) and we take A2k+α β = 2 B− F CF. The ﬁrst − − − equation for 2k + α β + 1 n 2(2k + α β) further reduces to: − ≤ ≤ − 0 = AT B + BAT , 2k + α β + 1 j 2(2k + α β) 1 (if α β + k 2), j j − ≤ ≤ − − − ≥ T T 0 = A2(2k+α β)B + A2k+α βBA2k+α β + BA2(2k+α β). − − − − Hence we can choose Aj = 0 for 2k + α β + 1 j 2(2k + α β) 1 (if α β + k 2) 1 1 T 2 − ≤ ≤ − − − ≥ and A2(2k+α β) = 8 (B− F CF) . By continuing in this manner we obtain: − − 1 T n N an 1(B− F CF) , j = n(2k + α β), n (3.27) A = − − ∈ , j ( 0, otherwise 16 TADEJ STARCIˇ Cˇ

1 1 n 1 N where a0 = 2 and an = 2 j=0− ajan j 1 for n . The generating function asso- − − − − j ∈ 1 2 1 ciated with the sequence a Pis f (t) := ∞ a t . Observe that f (t) = t(f (t)) , n j=0 j − 2 − 2 1 1 1 1 2n thus f (t) = 1 + (1 t) 2 and we obtainP a = . For the basic theory − t − n − 22n+1 n+1 n of generating functions see e.g. [16, Chapter 2]). We now compare the entries in the first row of the left-hand and the right-hand side of (3.25) and get the following equations: n T 0 = Aj BGn j, 0 n k 1, (if k 1) − ≤ ≤ − ≥ Xj=0 k T T (3.28) 0 = Aj BGk j + F C, − Xj=0 n T T 0 = Aj BGn j + F CDn k, n k + 1, − − ≥ Xj=0 The first two equations immediately imply 1 T (3.29) G0 = ... = Gk 1 = 0 (if k 1),Gk = B− F C. − ≥ − By comparing the entries in the first row of the left-hand and the right-hand side of (3.26), we obtain: n n T T (3.30) 0 = Dj CDn j + Gj (α β)BGn 1, n 1. − − ≥ Xj=0 j=Xα β − − − Using (3.29) we deduce that the second summand on the right-hand side of (3.30) vanishes for 1 n α β + 2k 1, α β + k 2, thus ≤ ≤ − − − ≥ n T 0 = Dj CDn j, 1 n α β + 2k 1 (if α β + k 2), − ≤ ≤ − − − ≥ Xj=0 2k+α β − T T 0 = Dj CD2k+α β j + Gk BGk. − − Xj=0 Therefore we choose

(3.31) D1 = ... = D2k+α β 1 = 0 (if α β + k 2), − − − ≥ 1 1 T 1 1 T D2k+α β = C− Gk BGk = FB− F C. − −2 −2 Using (3.27) and (3.31) for α β + k 2, the last equation of (3.28) reduces to 0 = BG for α β + 2k 1 n k−+ 1, hence≥ n − − ≥ ≥ (3.32) Gk+1 = ... = G2k+α β 1 = 0 (if α β + k 2). − − − ≥ Further, we apply (3.27), (3.29), (3.31), (3.32) to the last equation of (3.28) for n = α β + 2k. If k 1 we obtain BG2k+α β = 0, while for α β 2, k = 0 we get − ≥ − − ≥ T T 1 T 1 T T 0 = BGα β + Aα βBG0 + F CDα β = BGα β 2 F CFG0 2 F G0 BG0 = BGα β. − − − − − − − Similarly for k = 0, α β = 1 we deduce BG = 0. In any case we have − 1 (3.33) G2k+α β = 0. − 17

If α β+k 2 we use (3.29), (3.31), (3.32), (3.33) to see that the second summand on the− right-hand≥ side of (3.30) for α β +2k +1 n 2(α β)+3k vanishes, while T − T ≤ ≤ − the ﬁrst summand is equal to Dn C + CDn , thus: 0 = DT C + CDT , α β + 2k + 1 n 2(α β) + 3k. n n − ≤ ≤ − We take (3.34) D = 0, α β + 2k + 1 n 2(α β) + 3k (if α β + k 2). n − ≤ ≤ − − ≥ Using (3.27), (3.31), (3.34), the third equation of (3.28) for α β + 2k + 1 n 2(α β)+3k reduces to BG = 0; it is clear for n , α β +3k, while− for n = α β≤+3k≤: − n − − T T (3.35) 0 = BGα β+3k + Aα β+2kBGk + F CDα β+2k − − − 1 T 1 T 1 T 1 T 0 = BGα β+3k + 2 F CFB− F C 2 F CFB− F C = BGα β+3k. − − − It yields: (3.36) G = 0, α β + 2k + 1 n 2(α β) + 3k. n − ≤ ≤ − Equations (3.30) for (3.29),(3.31),(3.32),(3.36),(3.34) then give 0 = CD + DT C, 2(α β) + 3k + 1 j 2(α β + 2k) 1, j j − ≤ ≤ − − T T 0 = CD2(α β+2k) + Dα β+2kCDα β+2k + D2(α β+2k)C. − − − − We take (3.37) D = 0, 2(α β) + 3k + 1 n 2(α β + 2k) 1, n − ≤ ≤ − − 1 T 1 1 T 2 D2(α β+2k) = Dα β+2kCDα β+2k = (C− Gk BGk) . − −2 − − −8 From (3.28) for (3.27),(3.31),(3.34) we further deduce (3.38) G = 0, 2(α β) + 3k + 1 n 2(α β + 2k). n − ≤ ≤ − T T If α β = 1, k = 0 then (3.30) yields 0 = CD2 + (D1) CD1 + D2 C and we choose − 1 1 T 2 D2 = 8 (C− G0 BG0) . Further, similarly as in (3.35) we apply (3.28) to get G2 = 0 − T T from 0 = BG2 + A1 BG0 + F CD1. Thus (3.37), (3.38) are valid in this case as well. By continuing this proces we eventually obtain: B 1FT C, j = k (3.39) G = − , j ( 0, otherwise 1 T n an 1(FB− F C) , j = n(2k + α β) 1 1 2n Dj = − − , an = . ( 0, otherwise −22n+1 n + 1 n ! ∆ ∆ k 1 1 k 11′ 12′ Next, we compute ( α,β(F))− = α,β− α,β( α,β(F)) α,β α,β = ∆ ∆ with D B F D F B " 21′ 22′ # 1 T 1 n ∆ an 1B− (F CFB− ) B, j = n(2k + α β) 11′ = T (Im,A1′ ,...,Aα′ 1),Aj′ = − − , − ( 0, otherwise 1 1 T n ∆ an 1C− (CFB− F ) C, j = n(2k + α β) 22′ = T (In,D1′ ,...,Dβ′ 1), Dj′ = − − , − ( 0, otherwise k 1 T k Nβ ( B− F C) 1 1 2n ∆′ = 0 N (F) , ∆′ = − , an = . 21 β 12 " 0 # −22n+1 n + 1 n ! h i 18 TADEJ STARCIˇ Cˇ

k V Set p,t(F) to be an N N block matrix such that its principal submatrix K ∈ B × formed by blocks in the p-th and the t-th columns and rows is equal to k (F), Dαp,αt while the submatrix formed by all other blocks is the identity matrix. Clearly k (F) := ( k (F)) 1 is of the same form as k (F), only with ( k (F)) 1 as a Hp,t Kp,t − Kp,t Dαp,αt − principal submatrix formed by blocks in the p-th and the t-th columns and rows. We use the inductive procedure of multiplying V by matrices of the form Y ∈ B k (F) for the appropriate p,t,k,F. To describe the inductive step, suppose that Kp,t during the process we have a matrix that by a slight abuse of notation is still called , and such that the blocks under the main diagonal in the ﬁrst p 1 columns Y − vanish (i.e. rs vanishes for p,r >s), and the ﬁrst αp αp+1 + k columns of rp for Y − , Y r>p vanish. Let t be the largest index such that ( tp)1(αp αp+1+k+1) 0, i.e ( tp)(1) = tp tp tp Y tp − Y 0 ... 0 R ... R Cmt mp , k αp+1+αt αt 1 with all Rj × and Rk α +α 0. We multiply − − ∈ − p+1 t Y h k tp i (k+αt αp+1+1) with p,t( Rk α +α ) to get ′ of the same form as , and with ( tp′ ) − = K − − p+1 t Y Y Y 0. It is apparent for with r,p>s or t>r>s = p, while for r t, s = p we have Yrs′ ≥ tp tp pp pp pp ( ) = 0 ... 0 R ...R T (I ,A ,A ,...,A ) tp′ (1) k α +1+α αt 1 mp 1 1 α 1 Y − p t − p− k+α α +1 tp + T (I ,Att,...,Att ) 0 N t − p ( R ) mt 1 αt 1 αp αt k (α α ) − − − − p+1− t tp tp = 0 ... 0 S ...R k α +α +1 αt 1 , − p+1 t − rp rp 0 ... 0 R ...R pp ( rp′ )(1) = k α +α αt 1 T (Im ,...,Aα 1) Y − p+1 t − p p h i − k+α α +1 tp + 0 t − p ... 0 Nαp αt ( Rk (α α )) ∗ ∗ − − − p+1− t h rpi rp = 0 ... 0 S ...R k αp+1+αt+1 αt 1 , r>t, h − − i sp m m for some S C s× p with s r,t . This process (i.e. choosing the appropriate j ∈ ∈{ } p ,t ,k ,F n ) eventually yields a block upper-triangular matrix and it is of the { j j j j }j=1 form (1.8) such that the blocks on the main diagonal are block upper-triangular Toeplitz with identities on the diagonals; we denote it by : V n n kj 1 kj = = ( (F ))− = (F ). X QY QV Kpj ,tj j QV Hpj ,tj j Yj=1 Yj=1

The inverse of a nonsingular block upper-triangular Toeplitz matrix is again a 1 block upper-triangular Toeplitz, hence − is block upper-triangular. On the other hand is a solution of the equation (3.2),V so 1 = 1 is also a block lower- V V − B− FVFB triangular matrix. Hence = N T (I ,V r ,...,V r ); the algorithm that provides V ⊕r=1 mr 1 αr 1 the solution of (3.2) (see Lemma 3.1) yields equations− that give (3.21):

~~r T r r T r (B0) V1 + (V1 ) B0 = 0, n r T r r T r r T r r (B0) Vn+1 + (Vn+1) B0 = (Vj ) B0Vn+1 j , n 1. − − ≥ Xj=1~~

~~This concludes the proof of the lemma. 19~~

~~4. Proof of Theorem 1.3~~

We begin with a direct simple proof of Corollary 1.5, since the tangent space of Orb(A) at A (see TA in (1.4)) is easily computed. Indeed, if Q(t) is a complex- diﬀerentiable path of orthogonal matrices with Q(0) = I, then

d T T (Q(t)) AQ(t) = (Q′(0)) A + AQ′(0), dt t=0 ﬀ T T and di erentiation of ( Q(t)) Q(t) = I at t = 0 yields (Q′(0)) +Q′(0) = 0; conversely, for any X = XT we have e0 X = I and d (etX) = X. − · dt t=0 Observe that the dimension of T in (1.4) is precisely the codimension of the A solution space of XT A + AX = 0 with X = XT (with respect to the space of all − 1 skew-symmetric matrices). If J is the Jordan form of A with A = P − JP , we get 1 T JY = YJ, Y = PXP − ,X = X . − N Thus Y = [Yjk]j,k=1 has rectangle upper-triangular Toeplitz blocks Yjk (see Theo- rem 2.1), and Y = P 2Y T P 2. Note that (see e.g. [8, Theorem 4.4.24]): − − 0 1 1 1 (4.1) K (λ) = P J (λ)P − , P := (I + iE ),E = .. (α-by-α), α α α α α √2 α α α . " 1 0 # in which Eα is the backward identity-matrix (with ones on the anti-diagonal); and 2 T Pα = Eα. If A is of the form (1.6), then Y = EY E, in which E is a direct sum of backward identity matrices and it is partitioned− conformally to Y . In view of (3.9), Tjk (3.10) we further obtain that all Y =0and Y = , Y = [ 0 Tkj ] are related with jj jk 0 kj T = T (both T , T are upper-triangular Toeplitz).h i Corollary 1.5 now follows. jk − kj jk kj We now prove Theorem 1.3.

~~Proof of Theorem 1.3. Given a symmetric matrix S we need to solve the equation provided as part of the proof of (4.2) SQ = QS, where Q is an orthogonal matrix and provided as part of the proof of~~

N mr S = K (λ) , λ C. αr ∈ Mr=1 Mj=1 We shall first use Theorem 2.1 to solve (4.2) on Q. Taking into account that Q satisfies QT Q = I (I is the identity matrix), it will yield a certain matrix equation and further restricting the form of Q; at this point Lemma 3.1 will be applied. We have N mr N mr 1 S = P − JP,J = Jαr (λ) , P = Pαj , Mr=1 Mj=1 Mr=1 Mj=1 where Pαj is defined in (4.1). The equation (4.2) thus transforms to 1 JX = XJ,X = PQP − . From Theorem 2.1 (2) we obtain where X = [X ]N with further X is a m m rs r,s=1 rs r × s block matrix whoose blocks of dimension α α are of the form r × s 20 TADEJ STARCIˇ Cˇ

[0 T ], αr <αs T (4.3)  , α >α ,  0 r s  " #  T, α = α  r s Cm m  where T × , m = min αr ,αs is a complex upper-triangular Toeplitz matrix. ∈ { } T 1 2 2 2 2 2 2 Since Pα = Pα , Pα− = P α, Pα = P ′α = Pα− = iEα, we deduce P = P = P − = m − − − iE, where E := N r (E ) . Therefore I = QT Q if and only if ⊕r=1 ⊕j=1 αr T T 1 T 1 I =(P X (P − ) )(P − XP ) T T 1 T 1 1 I =P (P X (P − ) )(P − XP )P − , 2 T 2 (4.4) I =P X P − X I =iEXT ( iE)X − I =EXT EX. Proceed by conjugating with the permutation matrix Ω = N Ω as in (2.4): ⊕r=1 αr ,mr (4.5) I = (ΩT EΩ)(ΩT XT Ω)(ΩT EΩ)(ΩT XΩ) I = T , FX FX where = ΩT EΩ = N E (E ) and = ΩT XΩ (see (2.4)) are of the form (1.8) F ⊕j=1 αr mr X with block rectangular upper-triangular Toeplitz blocks. By applying Lemma 3.1 for = = I and Lemma 3.4 with = I to (4.5) we conclude the proof. B C B Remark 4.1. (1) The equation (4.4) is very similar to the equation that we obtained in [17, Proof of Theorem 1.1] when examining orthogonal *congruence of certain Hermitian matrices. However, to compute the isotropy groups under orthogonal *congruence, a more detailed analysis of a few more cases would need to be done (due to the existence of three different types of normal forms). (2) Canonical forms under orthogonal similarity are known for skew-symmetric and orthogonal matrices, too. Using the same general approach as in the case of symmetric matrices, isotropy groups are described by matrix equations which involve an important difference in comparison to equations that we deal in this paper (Lemma 3.1 and Lemma 3.4). We expect that by developing some special techniques, similar results can be obtained. Acknowledgement. This research was supported by Slovenian Research Agency (grant no. P1-0291). References [1] V. I. Arnold, Geometric Methods in the Theory of ordinary Differential Equations, Springer-Verlag New York Inc, 1988. [2] H. W. Braden, The equations AT X XT A = B, SIAM J. Matrix Anal. Appl. 20 (1998), 295-302. ± [3] V. Climenhaga, A. Katok, From Groups to Geometry and Back, American Mathematical Society, 2017. [4] F. de Teran, F. M. Dopico, The solution of the equation XA + AXT = 0 and its application to the theory of orbits, Linear Algebra Appl. 434 (1), 44-67. [5] D. Z.ˇ ¯Dokovic,´ K. Zhao, Tridiagonal normal forms for orthogonal similarity classes of symmetric matrices, Linear Algebra Appl. 384 (2004), 77-84. 21

[6] F. R. Gantmacher, The theory of matrices, Chelsea Publishing Company, New York, 1959. [7] V. V. Gorbatsevich, A. L. Onischuk, E. B. Vinberg, Lie Groups and Lie Alge- bras I, Encyclopaedia of Mathematical Sciences, Vol. 20, Springer-Verlag Berlin Heidelberg, 1993. [8] R. A. Horn, C. R. Johnson, Matrix analysis, Cambridge University Press, Cam- bridge, 1990. [9] R. A. Horn, Topics in matrix analysis, Cambridge University Press, Cam- bridge, 1991. [10] L. K. Hua, Geometries of matrices I. Generalizations of van Staudt’s theorem, Trans. Amer. Math. Soc., 57 (1945), 441-481. [11] L. K. Hua, Geometries of matrices III. Fundamental theorems in the geometries of symmetric matrices, Trans. Amer. Math. Soc., 61 (1947), 229-255. [12] L. K. Hua, Geometries of symmetric matrices over any ﬁeld with characteristic other than two, Ann. of Math., 50 (1949), 8-31. [13] S. Kobayashi, K. Nomizu. Foundations of Diﬀerential Geometry. Volume II, Interscience Publishers, 1969. [14] W.-W. Lin, V. Mehrmann, H. Xu, Canonical Forms for Hamiltonian and Sym- plectic Matrices and Pencils, Linear Algebra Appl. 302-303 (1999), 469-533. [15] J. S. Milne, Algebraic Groups: The Theory of Group Schemes of Finite Type over a Field Cambridge University Press, 2017 [16] J. Riordan, An Introduction to Combinatorial Analysis, Princeton University Press, 2014. [17] T. Starciˇ c,ˇ Hong’s canonical form of a Hermitian matrix with respect to orthogonal *congruence, to appear in Linear Algebra Appl. [18] H. Weyl, The classical groups, their invariants and representations, Princeton Univ. Press (1946)

~~Faculty of Education, University of Ljubljana, Kardeljeva Plosˇcadˇ 16, 1000 Ljubljana, Slove- nia~~

~~Institute of Mathematics, Physics and Mechanics, Jadranska 19, 1000 Ljubljana, Slovenia Email address: [email protected]~~