A Sort-Jacobi Algorithm on Semisimple Lie Algebras∗ 1

A SORT-JACOBI ALGORITHM ON SEMISIMPLE LIE ALGEBRAS∗

M. KLEINSTEUBER†

Abstract. A structure preserving Sort-Jacobi algorithm for computing eigenvalues or singular values is presented. The proposed method applies to an arbitrary semisimple Lie algebra on its ( 1)-eigenspace of the Cartan involution. Local quadratic convergence for arbitrary cyclic schemes is− shown for the regular case. The proposed method is independent of the representation of the underlying Lie algebra and generalizes well-known normal form problems such as the symmetric, Hermitian, skew-symmetric, symmetric and skew-symmetric R-Hamiltonian eigenvalue problem and the singular value decomposition. Due to the well-known classiﬁcation of semisimple Lie algebras, the approach yields structure-preserving Jacobi eigenvalue methods for so far unstudied problems as Takagi’s and a Takagi-like factorization, the Hermitian quaternion, the Hermitian C-Hamiltonian eigenvalue decomposition and a symplectic singular value decomposition. The algorithm is exempli- ﬁed for an exceptional Lie algebra.

Key words. Jacobi algorithm, Cartan decomposition, semisimple Lie algebra, restricted-root space, structured eigenvalue decomposition, structured singular value decomposition, Takagi’s factorization, local quadratic convergence

AMS subject classiﬁcations. 16D70, 45C05, 65F15, 15A18, 17B20, 74P20, 41A25

1. Introduction. Structured eigenvalue and singular value problems frequently arise in areas such as control theory and signal processing, where e.g. optimal control and signal estimation tasks require the fast computation of eigenvalues for Hamil- tonian and symmetric matrices, respectively. We refer to [4] for an extensive list of structured eigenvalue problems and relevant literature. Such problems cannot be solved easily by using standard software packages, as these eigenvalue algorithms do not necessarily preserve the underlying matrix structures and therefore may suffer by accumulation of rounding errors or numerical instabilities. In contrast, structure preserving eigenvalue methods have the potential of combining memory savings with high accuracy requirements and therefore are the methods of choice. Most of the previous approaches to structured problems have been on a case-to-case basis, by try- ing to adapt either known solution methods or software packages to each individual problem. However, such an ad–hoc approach is bound to fail if the problem of interest cannot be easily reformulated as a standard problem from numerical linear algebra. Thus a more flexible kind of algorithm is needed, that enables one to tackle whole classes of interesting eigenvalue problems. Clearly, one cannot expect to find one method that works for all structured problems. Nevertheless, it seems reasonable to develop methods that allow a unified approach to sufficiently rich classes of interesting eigenvalue problems. The structure preserving Jacobi-type methods proposed here are of this kind. Their inherent parallelizability, cf. [2], and high computational accuracy, cf. [11, 39] make them also useful for large scale matrix computations. They have moreover the potential to compete in convergence speed with state-of-the-art eigenvalue tools. Re- cently, a modified version of the one-sided Jacobi method outperforms the established QR-algorithm for computing the singular value decomposition of a dense matrix, cf. [9, 10].

∗This paper contains results from the author’s PhD thesis, [34]. †National ICT Australia, Canberra Research Laboratory, SEACS Program, Locked Bag 8001, Canberra ACT 2601, Australia and Department of Information Engineering, RSISE, The Australian National University, Canberra, ACT 0200, Australia ([email protected]). 1 Variants of the Jacobi algorithm have been applied to many structured eigenvalue problems (EVP), including e.g. the real skew-symmetric eigenvalue decomposition (EVD), [20, 31, 40], computations of the singular value decomposition (SVD) [36], non-symmetric EVPs [6, 14, 43, 46], complex symmetric EVPs [15], and the computation of eigenvalues of normal matrices [18]. For extensions to different types of generalized EVPs, we refer to [7, 47]. For applications of Jacobi–type methods to problems in systems theory, see [25, 26, 27]. In contrast to earlier work, we extend the classical concept of a Jacobi-algorithm towards a new, unified Lie algebraic approach to structured EVDs, where structure of a matrix is defined by being an element of a Lie algebra (or of a suitably defined sub-structure). To the best of the author’s knowledge, Wildberger [48] has been the first who proposed a generalization of the non-cyclic classical Jacobi algorithm on arbitrary compact Lie algebras and succeeded in proving global convergence of the algorithm. The well–known classification of compact Lie algebras shows that this approach essentially includes structure preserving Jacobi-type methods for the real skew-symmetric, the complex skew-Hermitian, the real skew-symmetric Hamiltonian, the complex skew- Hermitian Hamiltonian eigenvalue problem, and some exceptional cases. Wildberger’s work has been subsequently extended by Kleinsteuber et al. [33], where local quadratic convergence for general cyclic Jacobi schemes is shown to hold for arbitrary regular elements in a compact Lie algebra. In this paper we propose and analyze the Sort-Jacobi algorithm for a large class of structured matrices that, besides the compact Lie algebra cases, essentially includes the normal form problems quoted in Table 5. These cases arise from the well-known classification of simple Lie algebras. Cyclic Jacobi-type methods for some of these cases have been discussed earlier, e.g. for the symmetric/Hermitian EVD see [17, 31], for the skew-symmetric EVD see [20, 31, 40], for the real and complex SVD see [31, 36], for the real symmetric and skew-symmetric EVD see [16], for the Hermitian R-Hamiltonian EVD see [6], and for the perplectic EVD see [38]. Thus we achieve a simultaneous generalization of all this prior work. Note that the methods proposed in this paper exclusively use one-parameter transformations and therefore slightly differ from the algorithms in [6, 16, 20, 31, 40], where block Jacobi methods are used, i.e. multiparameter transformations that annihilate more than one pair of off-diagonal elements at the same time. Explicitly, our setting is the following. Let G be a semisimple Lie group and g = k p be the Cartan decomposition of its Lie algebra. We propose a Jacobi–type method⊕ that ”diagonalizes” an element S p by conjugation with some k K, where K G is the Lie subgroup corresponding∈ to k. Of course, if A is a Hermitian∈ matrix, then⊂ iA is skew-Hermitian and therefore the Hermitian and the skew-Hermitian EVD are equivalent. A similar trick extends to arbitrary compact Lie algebras. In this sense it becomes clear that the present work immediately includes the corresponding results for compact Lie algebras in [33]. In fact, if k is a compact semisimple Lie algebra, then g0 := k ik is the Cartan decomposition of g0. Thus the analysis of Jacobi-type methods on⊕ compact Lie algebras appears as a special case of our result. A characteristic feature of all known Jacobi-type methods is that they act to minimize the distance to diagonality while preserving the eigenvalues. Thus different measures to diagonality can be used to design different types of Jacobi algorithms. Conventional Jacobi algorithms, like classical cyclic Jacobi algorithms for symmetric matrices, Kogbetliantz’s method for the SVD, cf. [36], methods for the skew- symmetric EVD, cf. [20], and for the Hamiltonian EVD, cf. [37], as well as recent 2 methods for the perplectic EVD, cf. [38], are all based on reducing the sum of squares of off-diagonal entries (the so-called off-norm). Although local quadratic convergence to the diagonalization has been shown in some of these cases at least for generic situ- ations, the analysis of such conventional Jacobi methods becomes considerably more complicate for clustered eigenvalues. This difficulty is unavoidable for conventional Jacobi methods and is due to the fact, that the off-norm function that is to be min- imized has a complicated critical point structure and several global minima. Thus this difficulty might be remedied by a better choice of cost function that measures the distance to diagonality. In fact, Brockett’s trace function turns out to be a more ap- propriate distance measure than the off-norm function. In [3], R.W. Brockett showed that the gradient flow of the trace function can be used to diagonalize a symmetric matrix and simultaneously sort the eigenvalues. This trace function has also been considered by e.g. M.T. Chu, [8] associated with gradient methods for matrix factorizations, and subsequently by many others. For a systematic critical point analysis of the trace function in a Lie group setting, we refer to [13]. See also [44] for more recent results on this topic in the framework of reductive Lie groups. In his PhD thesis [31], K. Hüper has been the first who realized, that Brockett’s trace function can be effectively used to design a new kind of Jacobi algorithm for symmetric matrix diagonalization, that automatically sorts the eigenvalues in any prescribed order. This Sort-Jacobi algorithm uses Givens rotations that do not only annihilate the off-diagonal element, but also sort the two corresponding elements on the diagonal. The idea of combining sorting with eigenvalue computations can be carried over to the SVD as well, cf. [31]. It is this sorting property that distinguishes the Sort-Jacobi algorithm from the known conventional schemes and leads both to improved convergence properties as well as to a simplified theory. Lie theory provides us both with a unified treatment and a coordinate free approach of Jacobi-type methods. In particular, it allows a formulation of Jacobi methods that is independent of the underlying matrix representation. Together with the above specific matrix cases, all isomorphic types can be simultaneously treated. Thus we can completely avoid the often tiring case-by-case analysis. We illustrate this process in Subsection 5.2. In this paper we examine the local convergence behavior of the Sort-Jacobi algorithm for the above mentioned isomorphic classes and prove local quadratic convergence for the regular case, independent of any cyclic scheme. A local convergence analysis for the irregular case, i.e. where eigenvalues/singular values occur in clusters is more subtle and will be subject matter in a subsequent publication. This paper is organized as follows. The first section summarizes basic definitions and facts on Lie algebras and Lie groups. We focus on reviewing the structure of semisimple Lie algebras as they play a major role for our purposes. Since the so- called restricted-root space decomposition is of special importance in determining the sweep directions, it is explained in full detail. In Section 3 we discuss a Lie algebraic version of the aforementioned trace function and propose the Sort-Jacobi algorithm in full generality. Local quadratic convergence is proven in Section 4 for the regular case. Section 5 discusses applications to structured SVD and EVD, where the Sort- Jacobi is compared to the classical method, the situation of equivalent structured EVPs is discussed and two examples concretize the foregoing theory. The Sort-Jacobi algorithm is exemplified for the case of the exceptional Lie algebra g2. 2. Preliminaries on Lie Algebras. We recall facts on Lie algebras and espe- cially semisimple Lie algebras in this section, cf. [24] or [35] for further details. In the 3 sequel, let K denote the fields R, C of real or complex numbers, respectively. Definition 2.1. A K-vector space g with a bilinear product

[ , ] : g g g, · · × −→ is called a Lie algebra over K if (i) [X,Y ] = [Y, X] for all X,Y g (ii) [[X,Y ],Z−] + [[Y,Z], X] + [[Z, X∈],Y ] = 0 (Jacobi identity). Example 2.2. Let K = R or C. Classical Lie algebras are given for example by

sl(n, K) := X Kn×n trX = 0 { ∈ | } so(n, K) := X Kn×n X⊤ + X = 0 { ∈ | } sp(n, K) := X K2n×2n X⊤J + JX = 0 , { ∈ | } where 0 I J = n In 0 − and In denotes the (n n)-identity matrix. In any of these cases, the above bilinear product is given by the× usual matrix commutator [X,Y ] := XY Y X. A Lie algebra g over R (C) is called real (complex). A Lie− subalgebra h is a subspace of g for which [h, h] h holds. A Lie algebra is called Abelian, if [g, g] = 0. In the sequel, g is always assumed⊂ to be a ﬁnite dimensional Lie algebra. For any X g, the adjoint transformation is the linear map ∈

adX : g g, Y [X,Y ] (2.1) −→ 7−→ and

ad : g End(g), Y adY (2.2) −→ 7−→ is called the adjoint representation of g. By means of (2.1) and (2.2) properties (i) and (ii) of Deﬁnition 2.1 are equivalent to adX Y = adY X and ad[X,Y ] = adX adY adY adX , respectively. Definition− 2.3. Let g be a ﬁnite dimensional− Lie algebra over K. The symmetric bilinear form

κ : g g K, κ(X,Y ) tr(adX adY ) (2.3) × −→ 7−→ ◦ is called the Killing form of g. A Lie group is deﬁned as a group carrying a manifold structure such that the group operations are smooth functions. For an arbitrary Lie group G, the tangent space T1G at the unit element 1 G possesses a Lie algebraic structure. This tangent space is called the Lie algebra of∈ the Lie group G, denoted by g. The tangent mapping of the conjugation mapping in G at 1,

conj (y): = xyx−1, x,y G, x ∈ is given by

Adx : = T (conj ): g g 1 x −→ 4 and leads to the so-called adjoint representation of G in g, given by

Ad: x Adx. 7−→ Considering now the tangent mapping of Ad at 1 leads to the adjoint transformation (2.1). If G is a matrix group, i.e. G consists of invertible real or complex matrices, then the elements of the corresponding Lie algebra can also be regarded as matrices, cf. [35], Section I.10. In this case the adjoint representation of g G applied to Y g is given by ∈ ∈

−1 AdgY = gY g , (2.4) i.e., by the usual similarity transformation of matrices, and the adjoint transformation is given by

adX Y = [X,Y ] = XY Y X. − Example 2.4. Some classical Lie groups are

SL(n, K) := g Kn×n det g = 1 corresponding Lie algebra: sl(n, K), { ∈ | } SO(n, K) := g Kn×n g⊤g = 1, det g = 1 corresponding Lie algebra: so(n, K), { ∈ | } Sp(n, K) := g K2n×2n g⊤Jg = J corresponding Lie algebra: sp(n, K), { ∈ | } where J is deﬁned as in Example 2.2. A basic property of the Killing form is its Ad-invariance, i.e.

κ(AdgX, AdgY ) = κ(X,Y ) for all X,Y g and g G. (2.5a) ∈ ∈ Diﬀerentiating this equation with respect to g immediately yields

κ(adX Y,Z) = κ(Y, adX Z) for all X,Y,Z g. (2.5b) − ∈ Property (2.5a) is just the special case of the following more general result. A (Lie algebra-) automorphism of g is an invertible linear map ϕ: g g with ϕ([X,Y ]) = [ϕ(X), ϕ(Y )]. We denote the set of automorphisms of g by Aut(→g). Proposition 2.5. The Killing form is invariant under automorphisms of g, i.e

κ(X,Y ) = κ(ϕ(X), ϕ(Y )) for all ϕ Aut(g). ∈

Proof. Cf. [35], Prop. 1.119. An analytic subgroup H of a Lie group G is a connected subgroup where the inclusion mapping is smooth. The group of inner automorphisms Int(g) of g is deﬁned as the analytic subgroup of Aut(g) with Lie algebra ad(g). Hence if for X g we deﬁne the exponential map ∈

∞ 1 k exp: ad(g) GL(g), adX ad , −→ 7−→ k! X k X=0 then

Int(g) = the group generated by exp(adX ) X g . (2.6) { | ∈ } 5 Theorem 2.6. If g is the Lie algebra of a Lie group G and if G0 denotes the identity component of G, then

Int(g) = ϕ Aut(g) ϕ = Adg for some g G . (2.7) { ∈ | ∈ 0}

Proof. Cf. [35], Prop. 1.91, Ch. I, Sec. 10. In the case where g is a Lie algebra of complex or real (n n)-matrices, then × ∞ Xk multiplication is deﬁned on g and for X,Y g we have exp(X) = k=0 k! GL(n) and ∈ ∈ P

Ad X Y = exp(adX )Y = exp(X)Y exp( X). exp −

Let k g be a Lie subalgebra. We denote by Intg(k) the analytic subgroup of Int(g) with⊂ Lie algebra ad(k) End(g). Note that ⊂

Intg(k) = the group generated by exp(adX ) X k , { | ∈ } or, equivalently, if g is the Lie algebra of a Lie group G and K is the analytic subgroup of G with Lie algebra k, then

Intg(k) = ϕ Int(g) ϕ = Adk, k K . (2.8) { ∈ | ∈ } A real ﬁnite dimensional Lie algebra g is called compact if Int(g) is compact. A subalgebra k g is called a compactly embedded subalgebra if Intg(k) is compact. ⊂ Example 2.7. (a) If g is an Abelian Lie algebra, then adX 0 for all X g. Therefore Int(g) = 1 and hence g is a compact Lie algebra. ≡ ∈ (b) Although{ } the Lie subalgebra

a 0 a := a R sl(2, R) 0 a | ∈ ⊂ − is Abelian, it is not compactly embedded in sl(2, R). Note that 1 = Int(a) = { } 6 Intsl(2,R)(a). Since the map Intg(k) Int(k), ϕ ϕ k is smooth, every compactly embedded Lie subalgebra is itself compact.→ 7→ | Although in general differently but equivalently defined in the literature, it proves to be convenient for our purposes to use Cartan’s Criterion, cf. [35], Thm. 1.45, to define semisimple Lie algebras. Definition 2.8. A finite dimensional Lie algebra is called semisimple, if its Killing form is nondegenerate. Example 2.9. (a) The Lie algebras sl(n, K), so(n, K) and sp(n, K) are semisimple. Furthermore, let

Ip 0 Ip,q := . 0 Iq − 6 Then the Lie algebras

(p+q)×(p+q) ⊤ so(p, q) := X R X Ip,q + Ip,qX = 0 , p + q 3, { ∈ | } ≥ (p+q)×(p+q) ∗ su(p, q) := X C X Ip,q + Ip,qX = 0, trX = 0 , p + q 2, { ∈ | } ≥ sp(n, R) := X R2n×2n X⊤J + JX = 0 , n 1, { ∈ | } ≥ 0 I 0 I so∗(2n) := X su(n, n) X⊤ n + n X = 0 , n 2, { ∈ | In 0 In 0 } ≥ are also semisimple. (b) The Lie algebra u(n) is not semisimple, because adΛ 0 for all Λ = λIn, hence κ(Λ, X) = 0 for all X u(n). ≡ Let g be a complex semisimple∈ Lie algebra. We write gR for g if it is regarded as a real Lie algebra, i.e. by restricting the scalars and call gR the realiﬁcation of g. R Note, that dimR g = 2 dimC g. Proposition 2.10. Let g be a complex Lie algebra with Killing form κ: g g R R R × −→ C. Denote by κgR : g g R the Killing form on g , i.e. g regarded as a real vector space. Then × −→

κgR = 2Reκ. In particular, g is semisimple if and only if gR is semisimple. Proof. Cf. [35], Ch. I, Sec. 8. Example 2.11. For the classical complex semisimple Lie algebras, the Killing form alters in the following way by realiﬁcation.

Lie algebra g (complex) Killing form (real) Killing form of gR

sl(n, C) 2ntr(XY ) 4nRetr(XY ) so(n, C) (n 2)tr(XY ) 2(n 2)Retr(XY ) − − sp(n, C) 2(n + 1)tr(XY ) 8(n + 1)Retr(XY ), cf. [23], p.221. Note that in [23], the notation sp(2n, K) is used instead of sp(n, K). The following deﬁnition applies only to real semisimple Lie algebras g. In the case where g is complex we consider therefore its realiﬁcation gR. Definition 2.12. Let g be a real semisimple Lie algebra. An involution θ Aut(g), i.e. θ2 = 1, such that the symmetric bilinear form ∈

Bθ(X,Y ) := κ(X,θ(Y )) (2.9) − is positive deﬁnite is called a Cartan involution. Example 2.13. (a) The Killing form on sl(n, C) is given by 2ntr(XY ). According to Proposition 2.10, it translates into

κ(X,Y ) = 4nRetr(XY ) when regarding sl(n, C) as a real vector space. It is easily seen, that the map θ(X) := X∗ is an involution that respects brackets: − θ([X,Y ]) = [X,Y ]∗ = [Y ∗, X∗] = [ X∗, Y ∗] = [θ(X),θ(Y )]. − − − − 7 ∗ 1 C R Now since Retr(XY ) = 4n κ(X,θ(Y )) is an inner product on sl(n, ) , the map θ is indeed a Cartan involution.− (b) Similar, for n 3, the Killing form on so(n, R) is, up to a factor dependent on n, equal to tr(XY ). Since≥ tr(XY ⊤) = tr(XY ) yields an inner product on so(n, R), it follows that a Cartan involution is given− by the identity mapping θ(X) = X. Cartan involutions are unique up to conjugation with an inner automorphism. Theorem 2.14. If g is a real semisimple Lie algebra, then it has a Cartan involution. Any two Cartan involutions of g are conjugate via Int(g). Proof. [35], Ch. VI, Cor. 6.18 & 6.19. Example 2.15. By Example 2.13 (a) θ(X) := X∗ is a Cartan involution on the C − C −1 Lie algebra sl(n, ). Now let Adg Int(g) with g SL(n, ). Define θ = AdgθAdg . Then ∈ ∈ e θ(X) = g( (g−1Xg)∗)g−1 = gg∗X∗(gg∗)−1. − − On the other hand, noting that d := gg∗ is positive definite, we obtain e κ(X, θ(Y )) = 4nRetr(Xθ(Y )) = 4nRetr(XdY ∗d−1), − − which indeed is an inner product. This shows in fact, that all Cartan involutions on e e sl(n, C) are given by X dX∗d−1 with positive definite d. Definition 2.16 (7→Cartan − Decomposition). Let θ be a Cartan involution of a real semisimple Lie algebra g. Let k denote the +1-eigenspace of θ and p denote the 1-eigenspace of θ. − Then g = k p (2.10) ⊕ is called a Cartan decomposition of g. Theorem 2.17. Let g be a real semisimple Lie algebra and g = k p a Cartan decomposition of g. Then ⊕ (a) [k, k] k, [k, p] p and [p, p] k. (b) The Cartan⊆ decomposition⊆ is⊆ orthogonal with respect to the Killing form and the positive definite form Bθ. (c) The Killing form is negative definite on k and positive definite on p. (d) The Lie subalgebra k is maximal compactly imbedded in g. Proof. Cf. [24], Ch. III, Prop. 7.4. Example 2.18. (a) Since on compact, semisimple Lie algebras, the Killing form is negative definite, cf. [35], Ch. IV, Cor. 4.25 and 4.26, the Cartan decomposition and hence the Cartan involution are trivial. (b) Corresponding to the Cartan involution θ(X) = X⊤, the Cartan decomposition of sl(n, R) is given by − sl(n, R) = so(n, R) P Rn×n P ⊤ = P, trP = 0 . ⊕ { ∈ | } (c) Corresponding to the Cartan involution θ(X) = X∗, the Cartan decomposition of su(p, q) is given by −

S1 0 0 B su(p, q) = ∗ , 0 S2 ⊕ B 0 8 ∗ Cp×p ∗ Cq×q Cp×q where S1 = S1 ,S2 = S2 , tr(S1) + tr(S2) = 0 and B . Understanding− ∈ of the structure− ∈ of semisimple Lie algebras is the crucial∈ point to develop structure preserving Jacobi-type methods for normal form problems. The so- called restricted-root space decomposition of a semisimple Lie algebra that is explained below yields the correspondence between the desired normal form, the off-diagonal entries and finally the elementary Givens rotations. Let g be a semisimple Lie algebra, θ a Cartan involution and g = k p the corresponding Cartan decomposition. By finite dimensionality of p, there⊕ exists a maximal Abelian subspace a p. Let H a. Then θ(H) = H, since H p. ⊂ ∈ − ∈ Moreover adH is self-adjoint with respect to Bθ, cf. [35], Ch. VI, Lemma 6.27, and if H , H a then 1 2 ∈

adH adH X = [H , [H , X]] = [X, [H , H ]] [H , [X, H ]] = adH adH X 1 2 1 2 − 1 2 − 2 1 2 1 for all X g. Hence the set adH H a is a commuting family of self-adjoint transformations∈ of g. Therefore{ g decomposes| ∈ } orthogonally into the simultaneous eigenspaces of these commuting operators. Let be a (simultaneous) eigenspace and X denote by λ(H) the corresponding eigenvalue of adH . Then λ(H) R since adH is self-adjoint. For s,t R and X it follows by the identity ∈ ∈ ∈X

λ(tH1 + sH2)X = adtH1+sH2 X = tadH1 X + sadH2 X = (tλ(H1) + sλ(H2))X that λ: a R is linear. This motivates the following deﬁnition. We denote by a∗ := ν : a−→ R ν is linear the dual space of a. Definition{ → | 2.19. Let g}= k p be a Cartan decomposition of a semisimple Lie algebra g and let a p be a maximal⊕ Abelian subalgebra. For λ a∗, let ⊂ ∈

gλ := X g adH X = λ(H)X for all H a . (2.11) { ∈ | ∈ }

If λ = 0 and gλ = 0, the vector space gλ is called restricted-root space and λ is called 6 6 restricted root. The set of restricted roots is denoted by Σ. A vector X gλ is called a restricted-root vector. ∈ We summarize the above results. Theorem 2.20 (Restricted-root space decomposition). Let g be a semisimple Lie algebra with Cartan involution θ and corresponding Cartan decomposition g = k p. Let a p be maximal Abelian, denote by Σ the set of restricted roots and for λ ⊕ Σ ⊂ ∈ let gλ denote the corresponding restricted-root space. Then g decomposes orthogonally with respect to Bθ into

g = g gλ (2.12) 0 ⊕ λ X∈Σ and g0 = zk(a) a, where zk(a) := X k [X, H] = 0 for all H a denotes the centralizer of a ⊕in k. Furthermore, for{ λ,∈ µ |Σ 0 , we have ∈ } ∈ ∪ { } gλ+µ if λ + µ Σ 0 (a) [gλ, gµ] ∈ ∪ { } ⊆ ( 0 else. (b) θ(gλ) = g−λ, and hence λ Σ λ Σ. Proof. Cf. [35], Ch. VI, Prop.∈ 6.40.⇐⇒ − ∈ Remark 2.21. Similar to the above, it is possible to decompose complex semisimple Lie algebras into a maximal Abelian subalgebra and the so called root spaces (in contrast to restricted-root spaces), cf. [35], Sec. 1, Ch. II. In this context, the term 9 Cartan subalgebra arises. Note, that although a Cartan subalgebra is related to the maximal Abelian subspace a p, they do not coincide. A further investigation is not relevant for our purposes and⊂ we refer to the literature. The word restricted roots is due to the fact, that they are the nonzero restrictions to a of the (ordinary) roots of the complexification gC. Cf. [35], Ch. VI, Prop. 6.47 and the subsequent remark. Remark 2.22. Note that the restricted-root space decomposition can be equivalently computed via the eigenspaces of a single operator adH for a generic element H a with pairwise distinct roots. Such elements are dense in a since they are ob- ∈ tained by omitting from a the finitely many hyperplanes H a λi(H) λj (H) = 0 , { ∈ | − } λi = λj. Let µ denote an eigenspace of adH with eigenvalue µ. By definition of 6 E the restricted-root spaces µ = gλ must hold for some λi Σ 0 . Assume E i ∈ ∪ { } now that µ contains at least two restricted-root spaces, say gλ and gλ . Then E L 1 2 µ = λ1(H) = λ2(H) which is a contradiction to the choice of H. Restricted-root spaces are orthogonal with respect to the Killing form as long as the corresponding restricted roots do not add up to zero. Corollary 2.23. Let λ, µ Σ 0 and λ = µ. Then κ(gλ, gµ) = 0. ∈ ∪ { } 6 − Proof. Let X gλ,Y gµ and H a, then ∈ ∈ ∈ µ(H)κ(X,Y ) = κ(adH X,Y ) = κ(X, adH Y ) = λ(H)κ(X,Y ), − − for all H a, implying that κ(X,Y ) = 0. Example∈ 2.24. By Example 2.18 (b), the Cartan decomposition of sl(3, R) = k p is given by ⊕

k = Ω R3×3 Ω⊤ = Ω , p = X R3×3 trX = 0, X⊤ = X . { ∈ | − } { ∈ | } A maximal Abelian subalgebra in p is, for example, the Lie algebra of diagonal matrices

a = H p H is diagonal . { ∈ | } We have zk(a) = 0 and therefore by Theorem 2.20, the only subspace where adH H a acts trivial is g = a. It is easily seen, that { | ∈ } 0 0 x 0 0 0 x 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 , 0 0 0 , 0 0 x , x 0 0 , 0 0 0 , 0 0 0 , 0 0 0 0 0 0 0 0 0 0 0 0 x 0 0 0 x 0             x R, are (one-dimensional) invariant subspaces of adH H a . Parameterizing ∈ { | ∈ } a1 0 0 3 a = 0 a 0 a , a , a R, ai = 0  2  1 2 3 ∈   0 0 a3 i=1  X   yields the corresponding roots  λ (H) = a a , λ (H) = a a , λ (H) = a a , 12 1 − 2 13 1 − 3 23 2 − 3 λ (H) = a a , λ (H) = a a , λ (H) = a a , 21 2 − 1 31 3 − 1 32 3 − 2 respectively. The Lie algebraic pendant to symmetric matrices with pairwise distinct eigenvalues are the regular elements, i.e. elements in p that are Intg(k)-conjugate to an element in a with no root equal to zero. 10 Lemma 2.25. The set

a := H a λ(H) = 0 for all λ Σ (2.13) reg { ∈ | 6 ∈ } is open and dense in a. Moreover, zg(H) = g0 if H areg. Proof. Cf. [35], Ch. VI., Lemma 6.50. ∈ The following theorem generalizes the well-known fact that every symmetric (Her- mitian) matrix is SO(n)- (SU(n)-)conjugate to a real diagonal matrix. ′ Theorem 2.26. If a, a p are maximal Abelian, then there exists a ϕ0 Intg(k) ′ ⊂ ∈ such that ϕ (a ) = a. Consequently, for every X p there exists a ϕ Intg(k) such 0 ∈ 0 ∈ that ϕ0(X) a and p = a. ∈ ϕ∈Intg(k) Proof. Cf. [35], Ch. VI, Thm. 6.51. S 3. The Linear Trace Function and the Cyclic Sort-Jacobi Algorithm. Following the idea in [30] and [33], the Jacobi algorithm is formulated as an optimization task on the Intg(k)-adjoint orbit of an element S p, i.e. ∈

(S) = ϕ(S) ϕ Intg(k) , (3.1) O { | ∈ } for a smooth cost function f. For a more general approach cf. [30] and [34], where a coordinate descent method on manifolds is considered. In a next step, the cost function f is concretized to a Lie algebraic version of the linear trace function considered by Brockett [3]. The resulting algorithm is the Sort-Jacobi Algorithm, that, in contrast to common Jacobi-Eigenvalue methods orders the entries in the diagonalized matrix. Let λ be a restricted root and denote by Hλ a its dual, i.e. ∈

λ(H) = κ(Hλ, H) for all H a. (3.2) ∈

By [35], Ch. VI, Prop. 6.52, there exists normalized Eλ gλ such that ∈ 2 Tλ := [Eλ,θ(Eλ)] = Hλ. (3.3) − λ 2 | | Furthermore, let

Ωλ := Eλ + θ(Eλ) and Ωλ := Eλ θ(Eλ). (3.4) − 2 2 Note that Ωλ k, Ωλ p, κ(Ωλ, Ωλ) = 2 ,κ(Ωλ, Ωλ) = 2 and λ(Tλ) = 2. ∈ ∈ |λ| − |λ| − Example 3.1. Consider the case where g := sl(n, R) and the Cartan involution yields the decomposition into skew symmetric and symmetric matrices with the diagonal matrices as the maximal Abelian subalgebra. Denote by Xij the (i, j)-entry of the matrix X. Then the roots are given by

λij(H) = Hii Hjj, i = j. − 6 Recall that the Killing form κ is given by κ(X,Y ) = 2ntr(XY ). Therefore,

1 ⊤ ⊤ Hλ = (eie ej e ) ij 2n i − j where ei denotes the i-th standard basis vector and

2 1 λij(Hλ ) = λij = . ij | | n 11 ⊤ ⊤ ⊤ Hence Tλij = eiei + ej ej and Eλij = eiej . Depending on the choice of Eλij , −⊤ ⊤ ⊤ ± ⊤ either Ωλij = eiej ej ei or Ωλij = ej ei eiej . We introduce a− notion of positivity on−a∗ 0 . A subset of a∗ 0 consists of positive elements if for any l a∗ 0 exactly\ { one} of l and Pl is in\ { and} the sum and any positive multiple of elements∈ \ { in} is again in . We denote− theP set of positive restricted roots by Σ+. Theorem 2.20 assuresP that λP Σ if and only if λ Σ and, moreover, that Σ is ﬁnite. Thus a set of positive roots∈ is obtained by a− hyper∈ plane through the origin in a∗ that does not contain any root and deﬁning all roots on one side to be positive. Hence partitioning Σ into Σ+ Σ−, where Σ− := Σ Σ+ is the set of negative roots, is not unique. Now let ∪ \

kλ := X + θ(X) X gλ (3.5) { | ∈ } denote the projection of gλ onto k. Starting from a normalized basis for the gλ, + λ Σ , cf. Eq. (3.3), one obtains an orthogonal basis of kλ, normalized in terms of Eq.∈ (3.4). The union of these basis yields an orthogonal basis of

⊥ kλ = g0 k (3.6) + ∩ λX∈Σ and will further be denoted by

:= Ω , ..., Ωm . (3.7) B { 1 } We are now ready to explain a Jacobi Sweep for maximizing a function f. Note, that a minimization task is analogously deﬁnied. Cyclic Jacobi Sweep. Deﬁne for Ω the search directions ∈ B

r : R (S) (S), (t, X) Ad t X, (3.8) Ω × O −→ O 7−→ exp Ω (i) and let the step-size t∗ (X) be defined as the local maximum of f rΩi (X,t) with (i) ◦ smallest absolute value. To achieve uniqueness, we choose t∗ (X) to be nonnegative if t(i)(X) as well as t(i)(X) fulfill this condition. Note, that t(i)(X) is well defined ∗ − ∗ ∗ since f rΩ(X,t) is periodic for any function f. A rigorous proof uses advanced Lie algebraic◦ techniques and is beyond the scope of this paper. It can be found in [34]. A sweep on Intg(k) is the map s: (S) (S), (3.9) O −→ O (0) explicitly given as follows. Set Xk := X.

(1) (1) (0) (0) Xk := rΩ1 t∗ Xk , Xk (2) (2) (1) (1) Xk := rΩ2 t∗ Xk , Xk (3) (3) (2) (2) Xk := rΩ3 t∗ Xk , Xk . .

(m) (m) (m−1) (m−1) Xk := rΩm t∗ Xk , Xk , 12 (m) and set s(X) := Xk . The Jacobi algorithm consists of iterating sweeps.

1. Assume that we already have X , X , ..., Xk (S) for some k N. 0 1 ∈ O ∈ 2. Put Xk+1 := s(Xk) and continue with the next sweep.

Remark 3.2. Note, that by construction, a Jacobi sweep does not work in directions Ω zk(a). ∈ Let N a with λ(N) < 0 for all λ Σ+. Our goal now is to minimize the distance function∈ ∈

(S) R, X Bθ(X N, X N). (3.10) O −→ −→ − − This simpliﬁes to

Bθ(X N, X N) = κ(X N, X N) = κ(X, X) + κ(N, N) 2κ(X, N) − − − − − = κ(S,S) + κ(N, N) 2κ(X, N) − because of Bθ p = κ p and the Ad-invariance of κ. Minimizing the function deﬁned in Eq. (3.10) is| therefore| equivalent to maximizing the following function. Definition 3.3. Let S p and let κ denote the Killing form. The trace function is given by ∈

f : (S) R, X κ(X, N). (3.11) O −→ 7−→

Proposition 3.4. (a) X is a critical point of the trace function (3.11) if and only if X a. In particular, there are only ﬁnitely many critical points. ∈ (b) The trace function (3.11) has exactly one maximum, say Z, and one minimum, say Z, and λ(Z) 0, λ(Z) 0 for all λ Σ+. ≤ ≥ ∈ Proof. (a) To compute the critical points, let Ω k and denote by ξ = ad X an ∈ Ω arbitrary tangente vector in TX e(S). The ad-invariance of κ yields O

Df(X)ξ = κ(ξ, N) = κ(adΩX, N) = κ(Ω, adX N).

The Killing form is negative deﬁnite on k and hence

Df(X) = 0 [X, N] = 0. ⇐⇒ Since N is a regular element, it follows X a, cf. Lemma 2.25. Now (S) a is ﬁnite, cf. [24], Ch. VII, Thm. 2.12 and [34],∈ and hence f has only ﬁnitely|O ∩ many| critical points. (b) We compute the Hessian Hf at the critical points X. Again, let ξ = adΩX be tangent to X. Decompose Ω k according to Eq. (3.7) into ∈ m

Ω = Ω0 + riΩi, i=1 X where Ωi , Ω zk(a) and denote by λi the positive restricted root with Ωi kλ . ∈ B 0 ∈ ∈ i 13 Then

2 Hf (X)(ξ, ξ) = κ(ad X, N) = κ(ad X, ad N) Ω − Ω Ω m

= λi(X) λi(N) κ(riΩi, riΩi) − i=1 (3.12) X m 2 2ri = λi(X) λi(N) 2 . − λi i=1 X | | By assumption, λ(N) < 0 for all λ Σ+, so a necessary condition for a local maximum Z is that λ(Z) 0 for all λ Σ+.∈ The orbit (S) intersects the closure of the Weyl chamber ≤ ∈ O

C− = H a λ(H) < 0 for all λ Σ+ { ∈ | ∈ } exactly once, cf. [24], Ch. VII, Thm. 2.12 and [34]. Hence Z is the only local maximum of the function and by compactness of (S) it is the unique global maximum. A similar argument proves the existence of a uniqueO minimum, having all positive roots greater or equal to zero. We restrict the trace function (3.11) to the orbits of one-parameter subgroups in order to explicitly compute the step size t∗. Let Ωλ, Ωλ and Tλ as in Eq. (3.4), X p. Let ∈

p: g g , p(X) = X (3.13) −→ 0 0 denote the orthogonal projection with respect to Bθ. Note, that Bθ p = κ p and a p p(X) if X . Similar to Eq. (3.5), deﬁne ∈ ∈ pλ := X θ(X) X gλ (3.14) { − | ∈ } as the projection of gλ onto p. Theorem 3.5. Let

κ(X, Ωλ) cλ := (3.15) κ(Ωλ, Ωλ) be the Ωλ-coeﬃcient of X. Then

p (Adexp tΩλ X) = X0 + g(t)Tλ, where X0 := p(X) and g is given by

1 g : R R, t λ(X ) 1 cos(2t) cλ sin(2t). (3.16) −→ 7−→ 2 0 − − The proof of Theorem 3.5 is based on the following two lemmas. Lemma 3.6. The following identities hold for n N0. 2n+1 n 2n+1 ∈ (a) adΩλ Ωλ = ( 1) 2 ( Tλ), 2n −n 2n − (b) ad Tλ = ( 1) 2 Tλ. Ωλ − 14 Proof. (a) The ﬁrst formula is shown by induction. For n = 0, we compute

[Ωλ, Ωλ] = [Eλ,θ(Eλ)] [θ(Eλ), Eλ] = 2[Eλ,θ(Eλ)] = 2Tλ. − Assume now that it is true for n 0. Then ≥ 2n+3 2n+2 2n+1 n+1 2n+3 ad Ωλ = 2ad Tλ = 4ad Ωλ = ( 1) 2 ( Tλ), Ωλ − Ωλ − Ωλ − − and the formula is shown for n + 1. (b) The second identity follows from (a) by a straightforward calculation. It is clearly true for n = 0. Now let n 1. Then ≥ 2n 2n−1 n−1 2n−1 n 2n ad Tλ = 2ad Ωλ = 2( 1) 2 ( Tλ) = ( 1) 2 Tλ Ωλ Ωλ − − − k Lemma 3.7. Let λ, µ be positive restricted roots with λ = µ. Then p(adΩµ Ωλ) = 0 for all k N. 6 Proof∈. The proof is done by induction, separately for the even and the odd case. The assumption is clearly true for n = 0 and n = 1 by Theorem 2.20. Now let H a be arbitrary. Then, by the induction hypothesis, ∈

k k−1 k−2 κ(adΩµ Ωλ, H) = µ(H)κ(adΩµ Ωλ, Ωµ) = µ(H)κ(adΩµ Ωλ, [Ωµ, Ωµ])

k−2 = µ(H)κ(adΩµ Ωλ, 2Tµ) = 0, since Tµ a. This completes the proof. Proof∈ of Theorem 3.5. For all t R we have the identity ∈ ∞ k t k Ad t X = exp(adt )X = ad X. (3.17) exp Ωλ Ωλ k! Ωλ k X=0 It is shown that, if we decompose X p in its pλ-components then the only summands ∈ in Eq. (3.17) that aﬀect the projection onto a are X0 and cλΩλ. First, assume that ′ ′ Ω , Ωλ pλ and κ(Ωλ, Ω ) = 0. Then we have for all H a that λ ∈ λ ∈ ′ ′ ′ 0 = λ(H)κ(Ωλ, Ωλ) = κ(Ωλ, [H, Ωλ]) = κ([Ωλ, Ωλ], H)

′ ⊥ ′ and hence [Ωλ, Ωλ] a . Therefore, Theorem 2.20 implies that [Ωλ, Ωλ] p2λ if 2λ Σ and is zero∈ otherwise. For µ = λ we can apply Lemmas 3.6 and∈ 3.7 and compute∈ 6

∞ k ∞ k t k t k p (Ad t X) = p ad X = p ad (X + cλΩλ) = exp Ωλ k! Ωλ k! Ωλ 0 k ! k ! X=0 X=0 ∞ k ∞ k t k−1 t k = X0 + p ad [Ωλ, X0] + cλp ad Ωλ = k! Ωλ k! Ωλ k ! k ! X=1 X=1 ∞ k ∞ k t k−1 t k = X0 λ(X0)p ad Ωλ + cλp ad Ωλ = − k! Ωλ k! Ωλ k ! k ! X=1 X=1 ∞ 2k+2 ∞ 2k+1 t 2k+1 t 2k+1 = X0 λ(X0) ad Ωλ + cλ ad Ωλ. − (2k + 2)! Ωλ (2k + 1)! Ωλ k k X=0 X=0 15 Again by Lemma 3.6, the last sum simpliﬁes to

∞ 2k+1 ∞ 2k+1 t 2k+1 t k 2k+1 ad Ωλ = ( 1) 2 Tλ (2k + 1)! Ωλ − (2k + 1)! − k k X=0 X=0 ∞ 2k+1 k (2t) = ( 1) Tλ = sin(2t)Tλ. − − (2k + 1)! − k X=0 Furthermore,

∞ 2k+2 ∞ 2k+2 t 2k+1 t k 2k+1 ad Ωλ = ( 1) 2 Tλ. (2k + 2)! Ωλ − (2k + 2)! − k k X=0 X=0 Now we have

d ∞ t2k+2 ∞ (2t)2k+1 ( 1)k22k+1 = ( 1)k = sin(2t), dt (2k + 2)! − − (2k + 1)! k k X=0 X=0 and

∞ t2k+2 ( 1)k22k+1 = 1 (1 cos(2t)), (2k + 2)! − 2 − k X=0 and therefore

1 p (Ad t X) = X + λ(X )(1 cos(2t))Tλ cλ sin(2t)Tλ. exp Ωλ 0 2 0 − − As it can be seen from Theorem 3.5, the torus algebra component varies along the orbit of the one–parameter group Adexp tΩλ . In the next lemma we analyze this variation in more precise terms by discussing the function (3.16). 1 Lemma 3.8. The function g(t) = 2 λ(X0)(1 cos(2t)) cλ sin(2t) is π-periodic and is either constant or possesses on ( π , π ] exactly− one− minimum t and one − 2 2 min maximum tmax. In this case

2cλ λ(X0) sin 2tmin = , cos 2tmin = , 2 2 2 2 4cλ + λ(X0) 4cλ + λ(X0) (3.18) sin 2t = sin 2t , cos 2t = cos 2t , max p− min max p− min and g(t ) = 1 λ(X ) 1 4c2 + λ(X )2 and g(t ) = 1 λ(X ) + 1 4c2 + λ(X )2. min 2 0 − 2 λ 0 max 2 0 2 λ 0 Proof. The ﬁrst assertion is trivial and we only need to prove Eqs. (3.18). Sub- p p stituting v := sin 2t and u := cos 2t into the function g(t) = 1 λ(X ) 1 cos(2t) 2 0 − − cλ sin(2t), leads to the following optimization task.

1 Minimize/Maximize λ(X0)(1 u) cλv 2 − − (3.19) subject to u2 + v2 = 1.

We use the Lagrangian multiplier method to ﬁnd the solutions. Let

1 2 2 Lm(u,v) := λ(X )(1 u) cλv + m(u + v 1) 2 0 − − − 16 be the Lagrangian function with multiplier m. By assumption, g(t) is not constant and therefore the system of equations

1 DuLm(u,v) = λ(X ) + 2mu = 0 − 2 0 DvLm(u,v) = cλ + 2mv = 0 − u2 + v2 = 1 has exactly the two solutions

λ(X0) 2cλ 1 2 2 (u1,v1,m1) = , , 4c + λ(X0) and 2 2 2 2 2 λ 4cλ + λ(X0) 4cλ + λ(X0) ! p (3.20) p p λ(X0) 2cλ 1 2 2 (u2,v2,m2) = , , 4c + λ(X0) . 2 2 2 2 2 λ − 4cλ + λ(X0) − 4cλ + λ(X0) − ! p p p An inspection of the Hessian of Lmi (ui,vi) for i = 1, 2 and noting that (u1,v1) = (u2,v2) completes the proof of the ﬁrst assumption. The last assertion is proven by a− straightforward computation. Theorem 3.9. Let f be the trace function (3.11) and let g(t) as in Eq. (3.16). Then the following holds. 2λ(N) (a) f (Ad t X) = κ(X , N) g(t). exp Ωλ 0 − λ 2 π π | | (b) Let t ( , ] be the (local) maximum of f (Ad t X). Then ∗ ∈ − 2 2 exp Ωλ

λ(X0) 2cλ cos 2t∗ = , sin 2t∗ = 2 2 2 2 − 4cλ + λ(X0) − 4cλ + λ(X0) and hence p p

1−cos 2t∗ if sin 2t 0 1 + cos 2t∗ 2 ∗ ≥ cos t∗ = , sin t∗ =  2 q 1−cos 2t∗ r  if sin 2t∗ < 0.  − 2 q 1 2  2 (c) λ (p (Adexp t∗Ωλ X)) = 2 4c + λ(X0) 0. Proof. (a) The orthogonality− of p yields ≤ p f(X) = κ(X, N) = κ(p(X), N).

Let Tλ = [Eλ,θ(Eλ)] be deﬁned as in Eq. (3.3). By Theorem 3.5,

f (Adexp tΩλ X) = κ(X0 + g(t)Tλ, N) 2λ(N) (3.21) = κ(X , N) g(t), 0 − λ 2 | | 1 where g(t) = 2 λ(X0)(1 cos(2t)) cλ sin(2t) and the last identity holds since by 2 − − deﬁnition Tλ = 2 Hλ. − |λ| (b) Since by assumption λ(N) < 0, the second statement now follows immediately by Lemma 3.8 and a standard trigonometric argument. (c) is an immediate consequence of Lemma 3.8. 17 We present a Matlab-like pseudo code for the algorithm on semisimple Lie algebras. Note that in Section 5, the algorithm is exempliﬁed for the exceptional case g2. Let = Ω , ..., Ωm be as in Eq. (3.7) and let λi denote the restricted root with B { 1 } Ωi kλi . ∈Partial Step of Sort-Jacobi Sweep. For a given X p, the following algorithm ∈ computes (sin t∗, cos t∗, sin 2t∗, cos 2t∗), such that

X := Adexp t∗Ωi X has no Ωi-component and such that λi(X ) 0. Note that in the case where g is a e 0 ≤ Lie algebra of matrices X = exp(t Ωi)X exp( t Ωi). ∗ − ∗ e e function: (cos t∗, sin t∗, cos 2t∗, sin 2t∗) = elementary.rotation(X, Ωi) Set cλ := Ωi-coeﬃcient of X. Set X0 := p(X). 2 2 Set dis := λi(X0) + 4cλ. if dis = 0 6 p 1 Set (cos 2t , sin 2t ) := (λi(X ), 2cλ). ∗ ∗ −dis 0 else Set (cos 2t∗, sin 2t∗) := (1, 0). endif 1+cos 2t∗ Set cos t∗ := 2 . if sin 2t∗ 0q ≥ 1−cos 2t∗ Set sin t∗ = 2 . else q Set sin t = 1−cos 2t∗ . ∗ − 2 endif q end elementary.rotation

Remark 3.10. The algorithm is designed to compute (cos t∗, sin t∗, cos 2t∗, sin 2t∗) of the step size t∗ since this is natural by the chosen normalization of the sweep directions Ωi, cf. Eq. (3.4). Nevertheless, depending on the underlying matrix representation, we can not exclude that Adexp tΩi involves entries of the type (cos rt, sin rt) with r = 1, 2. In this case it is advisable to use standard trigonometric arguments to 6 compute cos rt∗, sin rt∗ respectively, by means of cos t∗, sin t∗, cos 2t∗, sin 2t∗. 2 Sort-Jacobi Algorithm. Denote by d(X) := X X0 the squared distance from X to the maximal Abelian subalgebra a. This|| coincides− || up to a constant with the oﬀ-norm. Let = Ω1, ..., Ωm be as in Eq. (3.7). Given a Lie algebra element S p and a toleranceB tol{ > 0, the} following algorithm overwrites S by ϕ(S) where ∈ ϕ Intg(k) and d(ϕ(S)) tol. ∈ ≤

Set ϕ := identity. while d(S) > tol for i = 1 : m (cos t∗, sin t∗, cos 2t∗, sin 2t∗) := elementary.rotation(S, Ωi). 18 ϕ := Adexp t∗Ωi ϕ. S := ϕ(S). ◦ endfor endwhile 4. Local Quadratic Convergence for the Regular Case. The local convergence analysis for the regular case is based on the investigation of a more general setting for Jacobi-type methods on manifolds, cf. [30, 34]. We particularize these results to the situation at hand and prove in a ﬁrst step that, for regular elements, (i) the step-size selections t∗ (X) are smooth in a neighborhood of a maximum of the trace function. Lemma 4.1. Let Z (S) a be a critical point of the trace function with ∈ O ∩ (i) λ(Z) < 0. Then the step-size selection t∗ (X) is smooth in a neighborhood of Z if Ωi kλ. In this case, the derivative is given by ∈ (i) 1 2 Dt (Z)(ξ) = ri λ ∗ 2 | |

κ(Ω,Ωi) where ξ = [Z, Ω] and ri = . κ(Ωi,Ωi) Proof. The main argument is the Implicit Function Theorem, cf. [1], Theorem 2.5.7. Deﬁne the C∞-function d ψ : R (S) R, ψ(t, X) := (f r (t, X)) . × O −→ dt ◦ Ωi By the chain rule we have

′ ψ(t, X) = Df (rΩi (t, X)) rΩi (t, X). (4.1)

Since Z is a local maximum of f (rΩi (t, X)) it follows that

ψ(0,Z) = 0.

Diﬀerentiating ψ with respect to the ﬁrst variable yields

2 d d H ψ(t, X) = f rΩi (t, X) = f (Z)(ξi, ξi), (4.2) dt (0,Z) dt2 ◦ (0,Z)

′ where ξi := [Z, Ωi] = r (0,Z) TZ (S). By Eq. (3.12), Ωi ∈ O 2 Hf (Z)(ξi, ξi) = λ(X) λ(N) < 0. − λ 2 | | Now the Implicit Function Theorem yields that there exists a neighborhood U ′ of Z and a unique smooth function ϕ: U ′ R such that ψ(ϕ(X), X) = 0 for all ′ (i) −→ X U . Since ψ(t∗ (Z),Z) = 0, it follows from the uniqueness of ϕ that there ∈ ′ (i) exists a suitable neighborhood U U of X such that ϕ(X) = t∗ (X) for all X U. Diﬀerentiating ψ with respect to the⊂ second variable yields together with Eq. (4.1)∈ d DX ψ(t, X) ξ = DX ( f ri(t, X)) ξ t=0,X=Z dt ◦ t=0,X=Z

′ = DX (Df(X)ri(0, X)) ξ X=Z H = f (Z)(ξi, ξ).

19 By symmetrizing Eq. (3.12) it is easy to check that

Hf (Z)(ξi, ξ) = λ(X)λ(N)ri. − Since ψ(t(i)(X), X) = 0 for all X U, ∗ ∈

(i) d (i) (i) (i) 0 = Dψ(t∗ (X), X) ξ = ψ(t∗ (Z),Z) Dt∗ (Z)ξ + DX ψ(t∗ (Z),Z)ξ X=Z dt ·

and the assertion follows. The Sort-Jacobi algorithm converges locally quadratically fast to the maximum Z in the regular case. Theorem 4.2. Denote by f : (S) R the trace function and let Z be a maximum of f. If S is a regular element,O −→ then the Sort-Jacobi Algorithm is locally quadratic convergent to Z. Proof. The proof consists essentially of two parts. In the ﬁrst part we show that the Hessian of f in Z is nondegenerate and that for the sweep directions Ωi, the set [Z, Ωi] forms a basis of TZ (S) that is orthogonal with respect to the Hessian of f. {In the second} part it is shownO that this orthogonality is suﬃcient for local quadratic convergence. We only sketch how a Taylor series argument applies and refer to [34] for details. The Hessian is given by

2 Hf (Z)(ξ, ξ) = κ(ad Z, N) = κ(adZ Ω, adN Ω), Ω − cf. Eq. (3.12). We have [Z, Ωi] = λ(Z)Ωi and since by assumption Z is regular, [Z, Ωi] = 0 for all Ωi. Orthogonality with respect to the Hessian is shown straightfor- 6 wardly, since for Ωi = Eλ + θ(Eλ) and Ωj = Eµ + θ(Eµ) the orthogonality of the Ωi implies for i = j 6 H 1 f (Z)(adZ Ωi, adZ Ωj ) = 2 λ(Z) µ(N) + λ(N)µ(Z) κ(Ωi, Ωj) = 0. Now let ξ TZ (S) denote an arbitrary tangent space element. The derivative ∈ O of rΩi in Z is given by

(i) DrΩi (Z)ξ = D rΩi (t∗ (X), X) ξ X=Z (i) = DrΩ (t, X) ( i) D(t∗ (X), id) X=Z ξ i |(t,X)=(t∗ (Z),Z) ◦ | (i) = Dt∗ (Z)ξi + ξ

(i) since t∗ (Z) = 0. By Lemma 4.1 therefore

Hf (Z)(ξi, ξ) DrΩi (Z)ξ = ξ ξi. − Hf (Z)(ξi, ξi)

Thus DrΩi (Z) is a projection operator that – by orthogonality of the ξi’s with ⊥ respect to Hf – maps ξ into (Rξi) . The composition of these projection operators is the zero map. Since Z is a ﬁxed point, i.e. ri(Z) = Z for all i = 1, ..., N, we conclude

Ds(Z) = D(rm ... r )(Z) = 0. ◦ ◦ 1 20 Consequently, a Sort-Algorithm sweep deﬁnes a smooth map on a neighborhood of Z with vanishing derivative. Now reformulating everything in local coordinates, Taylor’s Theorem yields

s(X) = s(Z) + Ds(Z)(X Z) + 1 D2f(ξ)(X Z, X Z), − 2 − − where ξ U, a suitable compact neighborhood of Z. Using that s(Z) = Z, it follows ∈ s(X) Z sup D2s(ξ) X Z 2. || − || ≤ ξ∈U || || · || − ||

Thus the algorithm induced by s converges locally quadratically fast to Z. Remark 4.3. Although the order in which the different elementary rotations Ωi are worked off is irrelevant for the proof as long as regular elements are considered, and although regular elements form a dense subset in p, it is worth to point out that in practice, the ordering does matter. In fact, the relevance of the ordering is the bigger, the more the eigenvalues/singular values of S are clustered. A theoretical treatment of the irregular case will be published elsewhere. 5. Applications to Structured Singular Value and Eigenvalue Prob- lems. We start with a comparison of the classical and the Sort-Jacobi algorithm and discuss subsequently how some of the well known normal form problems from numerical linear algebra fit into the developed Lie algebraic setting. Table 5 provides an overview of the Cartan decompositions of simple Lie algebras and the corresponding matrix factorizations. The Sort-Jacobi algorithm from the previous section is spec- ified for the real symmetric Hamiltonian EVD and for one exceptional case. Each of the examples is of course only one representative of the corresponding isomorphism class. By knowledge of the isomorphism, it is then straightforward to adapt the presented algorithm in order to obtain structure preserving Jacobi-type methods for the isomorphic classes. To the author’s knowledge, most of the corresponding Sort-Jacobi-algorithms are new. 5.1. Classical Jacobi vs Sort-Jacobi. The Lie algebraic generalization of the classical Jacobi method for the symmetric EVP in order to tackle normal form problems that are isomorphic to the ones listed in Table 5 is completely analogous to the proposed approach here. A detailed exposition can be found in [34], cf. also [33] for the case of compact Lie algebras. The intention of the classical method is to reduce the off-norm by annihilating in each step the corresponding off-diagonal entry. The rotation angle thereby is chosen to having minimal absolute value. The underlying optimization task consists of minimizing the off-norm function, or, in our general setting its Lie algebraic version

f : (S) R, X κ(X X , X X ), (5.1) O −→ 7−→ − 0 − 0 where, as in the previous sections, κ denotes the Killing form, (S) the Intg(k)-adjoint O orbit of S p and X0 the orthogonal projection of X onto the maximal Abelian subalgebra ∈a p. The Sort-Jacobi Algorithm, however, is conceived to choose the angle that annihilates⊂ the oﬀ-diagonal entry and sorts the resulting diagonal entries. It has several advantages over the classical method. From a theoretical point of view, the underlying cost function (trace function) is much simpler than the oﬀ-norm. The latter one for example, generally possesses continua of critical points that are not relevant for the optimization task, cf. [31, 34], whereas all critical points of the trace 21 g k p Matrix Factorization

sl(n, R) so(n, R) S Rn n, S = S , trS = 0 symmetric EVD ∈ × ⊤ sl(n, C) su(n) S Cn n, S = S , trS = 0 Hermitian EVD ∈ × ∗ so(n, C) so(n, R) Ψ iRn n,Ψ= Ψ skew-symmetric EVD ∈ × − ⊤

S Ψ su (2n) sp(n) P := ∗ , S, Ψ Cn n, Hermitian Quaternion EVD, i.e. ∗ Ψ S ∈ ×

Λ trS = 0, S = S , Ψ= Ψ uPu = , trΛ = 0, Λ real diagonal, u Sp(n) ∗ − ⊤ ∗ Λ ∈

0 B p q so(p, q) so(p, R) so(q, R) ,B R × real SVD, skew-symm. persymmetric EVD, etc. ⊕ B⊤ 0 ∈ 22

0 B p q su(p, q) s(u(p) u(q)) ,B C × complex SVD, Hermitian R-Hamiltonian EVD ⊕ B∗ 0 ∈

0 B n n so∗(2n) u(n) ,B C × ,B = B⊤ Takagi-like factorization, i.e. B∗ 0 ∈ −

¾ ¿ 0 x1

6 x1 0 7 6 − 7 6 . 7 R uBu⊤ = 6 .. 7 ,xi ,u U(n) 6 7 ∈ ∈ 4 0 xn 5 xn 0 − Table 5.1 Classical Cartan-decompositions and corresponding matrix factorizations, Part I. g k p Matrix Factorization

S C sp(n, R) u(n) ,S,C Rn n, S = S ,C = C symmetric Hamiltonian EVD C S ∈ × ⊤ ⊤ − Takagi’s factorization

S C sp(n, C) sp(n) P := ,S,C Cn n, S = S ,C = C Hermitian C-Hamiltonian EVD C S ∈ × ∗ ⊤ − ¾ B F ¿ − 6 F B 7

6 7 p q sp(p, q), p q sp(p) sp(q) ,B,F C × symplectic SVD, i.e. ≥ ⊕ 4 B∗ F ∗ 5 ∈ F ⊤ B⊤ − B F Σ 0 u − v∗ = ,

23 F B 0 Σ ¾ 0 ¿

6 a1 0 7 6 7 Rp q Σ= 6 . 7 × , u Sp(p), v Sp(q) 4 .. 5 ∈ ∈ ∈ 0 an

¾ ¿ ¾ ¿ 0 √2b √2b b1 ⊤ − ⊤ g2 su(2) su(2) 4 √2b S B 5 ,b = 4 b2 5 , cf. Subsection 5.4 ⊕ √2b B S b3 −¾ − −¿ 0 b3 b2 − 3 3 B = 4 b3 0 b1 5 ,bi R, S R , − ∈ ∈ × b2 b1 0 − trS = 0, S = S⊤ Table 5.2 Classical Cartan-decompositions and corresponding matrix factorizations, Part II. function are in fact diagonalizations. A by far more important fact is that the Sort- Jacobi algorithm has a unique fixed point on the orbit, namely the diagonalization with ordered eigenvalues. This ordering will be of crucial interest when discussing the rate of convergence for irregular elements, i.e. in the case of multiple eigenvalues, a fact that van Kempen already noticed when investigating the convergence behavior of the special cyclic Jacobi method for symmetric matrices, cf. [45]. One cannot assume a priori that the classical algorithm converges to such ordered diagonal form, since all diagonalizations are fixed points. We therefore assume that particularly in applications where clustered or multiple eigenvalues occur, the Sort-Jacobi method outperforms the classical method. Simulations support this hypothesis, cf. also [31] for the real symmetric, the real skew-symmetric EVD and the real SVD. Both, the classical and the Sort-Jacobi algorithm have been implemented in Mathematica 5.0 for the symmetric eigenvalue problem. Each use three randomly generated symmetric (50 50)-matrices with prescribed eigenvalues. In Figure 5.1 the eigenvalues are 1, ...,×50. Figure 5.2 illustrates both methods in the case of clustered eigenvalues. Explicitly, the eigenvalues 20 as well as 20 occur each with multiplicity 15 and 0 is an eigenvalue with multiplicity 20. The off-norm− is labelled versus the vertical axes.

105 104 103 102 101 1 10−1 10−2 10−3 10−4 10−5 10−6 10−7 10−8 10−9 10−10 12345678 Sweeps

Fig. 5.1. Regular case: Diagonalizing real 50 50-matrices with eigenvalues 1, ..., 50; solid line × 10 = Sort-Jacobi algorithm; dashed line = Classical method; stopping criterion: oﬀ-norm < 10− .

5.2. Equivalent EVD/SVD-Problems. We now illustrate the advantage of the coordinate free approach of the Sort-Jacobi algorithm. If a class of matrices is known to be isomorphic to one of the classes listed in Table 5, then by a straightforward re-definition of terms, a Jacobi algorithm is obtained together with the ap- propriate convergence results. Stronger than isomorphic is the following definition of equivalence. Definition 5.1. Two Lie algebras g, g′ Cn×n are equivalent, if there exists an invertible g Cn×n such that ggg−1 = g′. ⊂ ∈ If g is semisimple, then ggg−1 =: g′ is also semisimple and their Cartan decompositions transform accordingly, i.e. if g = k p is the Cartan decomposition of g, then gkg−1 gpg−1 yields the Cartan decomposition⊕ of g′. This means, that to every row in Table⊕ 5, there correspond infinitely many structured eigenvalue problems, and as is often not recognized in the literature, seemingly ”different” structured eigenvalue problems can be equivalent. We mention three examples. 24 105 104 103 102 101 1 10−1 10−2 10−3 10−4 10−5 10−6 10−7 10−8 10−9 10−10 123456789 Sweeps

Fig. 5.2. Irregular case: Diagonalizing real 50 50-matrices with eigenvalues clustered at 20 and 0; solid line = Sort-Jacobi algorithm; dashed× line = Classical method; stopping criterion: ± 10 oﬀ-norm < 10− .

The Lie algebra of the perskew-symmetric matrices • . 1 A Rn×n A⊤R + RA = 0 , where R := .. (5.2) { ∈ | } 1    is equivalent to so(k, k) if n = 2k and to so(k + 1, k) if n = 2k + 1. More pre- cisely, if X R(2k+1)×(2k+1) is perskew-symmetric, then gXg−1 so(k, k+1) with orthogonal∈ ∈ 1 0 0 1 · · · .. ..  0 . . 0 1 0 1 1  . . g =  . 0 √2 0 .. (5.3) √2   1 0 1   −   .. ..   0 . . 0  1 0 0 1 − · · ·    The symmetric perskew-symmetric EVD, cf. [38], is therefore equivalent to the SVD of a real (k k)-matrix, (k + 1 k) respectively. Takagi’s factorization× is equivalent to the× symmetric Hamiltonian EVD, cf. • Section 5.3. In systems theory, the real Lie algebra • A G g := , A, G, Q Cn×n, G∗ = G, H∗ = H (5.4) Q A∗ ∈ − plays an important role in linear optimal control and associated algebraic Riccati equations. We refer to g as the set of R-Hamiltonian matrices in order to avoid confusion with the complex Lie algebra sp(n, C), whose elements are called, following the established convention in mathematics, (complex or C)-Hamiltonian. The Lie algebra g is equivalent to su(n, n) and hence the diagonalization of a Hermitian R-Hamiltonian matrix is equivalent to the SVD of a complex (n n)-matrix, cf. [5, 34]. × 25 Note, that different choices for the maximal Abelian subalgebra a also lead to equivalent problems, cf. Theorem 2.26. 5.3. A Standard Example - The Real Symmetric Hamiltonian EVD & Takagi’s Factorization. We demonstrate how the real Symmetric Hamiltonian EVD can be derived as a special case of the ideas developed in the previous sections. In [16], the authors use a Jacobi-type method that is based on the direct solution of a 4 4 subproblem. We will not follow this approach and restrict ourselves to optimization× along one-parameter subgroups. Moreover, we prove that the real symmetric Hamiltonian EVD is isomorphic to Takagi’s Factorization of a complex symmetric matrix. That is, given a complex symmetric matrix B Cn×n, find a unitary matrix u U(n) such that uBu⊤ is real and diagonal. ∈ ∈ The set of real Hamiltonian matrices forms the real and semisimple Lie algebra A B sp(n, R) = , B⊤ = B, C⊤ = C, A Rn×n . (5.5) C A⊤ ∈ − The Killing form on sp(n, R) is κ(X,Y ) = 2(n+1)tr(XY ). Therefore, θ(X) := (X)⊤ yields a Cartan involution since − ⊤ Bθ(X,Y ) = κ(X,θ(Y )) = 2(n + 1)tr(XY ) − is an inner product. The corresponding Cartan decomposition is given by sp(n, R) = k p with ⊕ Ψ B k = , B⊤ = B, Ψ⊤ = Ψ Rn×n , B Ψ − ∈ − (5.6) S C p = , C⊤ = C,S⊤ = S Rn×n . C S ∈ − Hence p consists of all symmetric Hamiltonian matrices of size 2n 2n. Note, that k is isomorphic to u(n) via the Lie algebra isomorphism × X Y ι: X + iY , X,Y Rn×n. 7−→ Y X ∈ − and the same mapping also yields a Lie group isomorphism Reu Imu ι: U(n) R2n×2n, u . −→ 7−→ Imu Reu − As a maximal Abelian subalgebra in p we choose the diagonal matrices, i.e. Λ a = , Λ = diag(a , ..., a ) . (5.7) Λ 1 n − The equivalence of the real Symmetric Hamiltonian EVD and Takagi’s Factorization is stated in the following proposition. The two normal form problems can be carried over to one another via conjugation by a fixed matrix. Proposition 5.2. Let p be defined as in Eq. (5.6), let a be as in Eq. (5.7) and S C let p. Then C S ∈ − S C ι(u) ι(u∗) a C S ∈ − 26 if and only if u(S iC)u⊤ is real and diagonal. − Proof. Let g U(2n) be defined by 0 ∈

1 iIn In g0 := √2 In iIn and let u U(n). Then we have ∈ Reu + iImu 0 u 0 g ι(u)g−1 = = . 0 0 0 Reu iImu 0 u − Therefore for symmetric S, C Rn×n and real diagonal Λ we have ∈ S C Λ 0 ι(u) ι(u∗) = C S 0 Λ ⇐⇒ − − S C Λ 0 g ι(u)g−1g g−1g ι(u∗)g−1 = g g−1 0 0 0 C S 0 0 0 0 0 Λ 0 ⇐⇒ − − u 0 0 C + iS u∗ 0 0 iΛ = 0 u C iS 0 0 u⊤ iΛ 0 ⇐⇒ − − u(C + iS)u⊤ = iΛ u(S iC)u⊤ =Λ. ⇐⇒ − Proposition 5.2 justiﬁes that we restrict our discussion to the real symmetric Hamiltonian case. With a parameterized as in Eq. (5.7), a set of positive restricted roots is given by

ai aj , 1 i

For g2ai the only nonzero entry is at (n + i, i).

0 1 0 1 0 0 1 0 ga −a = R   , ga +a = R   , i j · 0 0 i j · 0 0  1 0   0 0   −        (5.9) 1 0 0 0 g2a = R   . i · 0 0  0 0      The above basis vectors are all normalized in the sense of Eq. (3.3). An implementa- tion of the Sort-Jacobi Algorithm is now straightforward.

5.4. The Exceptional Case g2. We examine now a rather exotic looking case that leads to a new structured eigenvalue problem and its corresponding Sort-Jacobi 27 algorithm. It is not isomorphic to any of the other cases in Table 5. Consider the 14-dimensional real Lie algebra

0 √2b⊤ √2c⊤ b 0 b b 1 − 3 2 g2 := √2cM B b = b2 ,B = b3 0 b1 , − ⊤    −   √2b C M b3 b2 b1 0 − − −        c1 0 c3 c2 c = c , C = c −0 c , b , c R,M R3×3, trM = 0 . 2 3 1 i i  c   c c −0  ∈ ∈ 3 − 2 1      (5.10)  We work with the following basis of g2. Let Eij denote the (7 7)-matrix with (i, j)-entry 1 and 0 elsewhere. ×

X := √2(E E ) + E E ; X := E E ; 1 16 − 31 54 − 72 2 23 − 65 X := √2(E E ) + E E ; X := √2(E E ) + E E ; 3 15 − 21 73 − 64 4 14 − 71 35 − 26 X := E E ; X := E E ; (5.11) 5 34 − 76 6 24 − 75 ⊤ Yi := X , i = 1, ..., 6; − i H := E E E + E ; H := E E E + E . 1 22 − 44 − 55 77 2 33 − 44 − 66 77 By help of the Killing form we compute the Cartan involution and the corresponding Cartan decomposition of g2. Proposition 5.3. The Killing form on g2 is given by

κg2 (X,Y ) = 4tr(XY ). (5.12)

Proof. Using the commutator relations of the basis (5.11) one can easily con- 14×14 struct matrix representations of the adjoint operators adX , adY , adH C . Itis i i j ∈ straightforward to check that for all Z, Z X , ..., X ,Y , ..., Y , H , H the relation ∈ { 1 6 1 6 1 2}

κ(Z, Z) = tr(adZ ad e ) = 4tr(ZZ) e ◦ Z 2 holds. Hence for arbitrary X eg2 we have κ(X, X) = 4tr(Xe ). The claim now follows by symmetrizing. ∈ ⊤ Corollary 5.4. A Cartan involution on g2 is given by θ(X) = X . Corre- spondingly, the Cartan decomposition is g = k p with − 2 ⊕ k = g so(7, R), p = X g X⊤ = X . (5.13) 2 ∩ { ∈ 2 | }

Proof. For θ(X) = (X)⊤, the bilinear form − ⊤ Bθ(X,Y ) = κ(X,θ(Y )) = 4tr(XY ) − is an inner product of g2. Therefore θ is a Cartan involution. Obviously, for Ω k and Ω p, one has θ(Ω) = Ω and θ(Ω) = Ω. ∈ With∈ respect to the maximal Abelian− subspace

a := a H + a H ai R p, { 1 1 2 2 | ∈ } ⊂ 28 we can choose the set of positive restricted roots by

λ1 := a2, λ2 := a1 a2, λ3 := a1, − (5.14) λ4 := a1 + a2, λ5 := a1 + 2a2, λ6 := 2a1 + a2.

The corresponding restricted-root spaces are given by

R R gλi = Xi, g−λi = Yi, i = 1, ..., 6. (5.15)

We now present a Sort-Jacobi algorithm that diagonalizes an element S p, preserv- ∈ ing the special structure of p. Note, that for i = 1, ..., 6 we have θ(Xi) = Yi and the Xi gλ are normalized such that ∈ i

λi([Xi,θ(Xi)]) = λi([Xi,Yi]) = 2, for all i = 1, ..., 6. −

Let Ωi := Xi + θ(Xi) = Xi + Yi k. Then the elementary structure preserving rotations are given by ∈

t sin 2t sin 2t cos 2 0 √2 0 0 √2 0 0 cos t 0 0 0 0 sin t sin 2 1 1 1 1  t 0 + cos 2t 0 0 + cos 2t 0  − √2 2 2 − 2 2 exp(tΩ1) = 0 0 0 cos t sin t 0 0 ,  0 0 0 sin t −cos t 0 0   sin 2 1 1 1 1   t 0 + cos 2t 0 0 + cos 2t 0   √2 2 2 2 2  − −   0 sin t 0 0 0 0 cos t  −  10 000 00 0 cos t sin t 0 0 00 0 sin t cos t 0 0 00  −  exp(tΩ2) = 00 010 00 , 0 0 0 0 cos t sin t 0   0 0 00 sin t cos t 0 00 000− 01     t sin 2t sin 2t cos 2 √2 0 0 √2 0 0 sin 2 1 1 1 1 t + cos 2t 0 0 + cos 2t 0 0 − √2 2 2 − 2 2  0 0 cos t 0 0 0 sin t − exp(tΩ3) = 0 0 0 cos t 0 sin t 0 ,  sin 2t 1 1 1 1   2 + 2 cos 2t 0 0 2 + 2 cos 2t 0 0  − √2 −   0 0 0 sin t 0 cos t 0   −   0 0 sin t 0 0 0 cos t    t sin 2t sin 2t cos 2 0 0 √2 0 0 √2 0 cos t 0 0 0 sin t 0  0 0 cos t 0 sin t − 0 0  sin 2t 1 1 1 1 exp(tΩ ) = 0 0 2 + 2 cos 2t 0 0 2 + 2 cos 2t , 4 − √2 −   0 0 sin t 0 cos t 0 0   0 sin t − 0 0 0 cos t 0   sin 2 1 1 1 1   t 0 0 + cos 2t 0 0 + cos 2t  − √2 − 2 2 2 2   29  100 000 0 010 000 0 0 0 cos t sin t 0 0 0  exp(tΩ5) = 0 0 sin t cos t 0 0 0 , 000− 010 0    0 0 0 0 0 cos t sin t 00 0 00 sin t cos t  −    1000 000 0 cos t 0 sin t 0 0 0 0010 000  exp(tΩ6) = 0 sin t 0 cos t 0 0 0 . 0− 0 0 0 cos t 0 sin t   0000 010  0 0 00 sin t 0 cos t  −    The Sort-Jacobi algorithm explicitly reads as follows. Denote by Xij the (i, j)- entry of the matrix X.

Algorithm G2(I). Partial Step of Jacobi Sweep. function: (cost∗, sint∗, cos2t∗, sin2t∗) = elementary.rotation(X, Ωi) Case i=1: Set c := X45, λ := X33. Case i=2: Set c := X , λ := X X . 32 22 − 33 Case i=3: Set c := X73, λ := X22. Case i=4 : Set c := X35, λ := X77. Case i=5: Set c := X34, λ := X22 + 2X33. Case i=6: Set c := X24, λ := 2X22 + X33. Set dis := λ2 + 4c2. if dis = 0 6 1 Set (cos2t , sin2t ) := (λ, 2c). ∗ ∗ −dis else Set (cos2t∗, sin2t∗) := (1, 0). endif 1+cos2t∗ Set cost∗ := 2 . if sin2t∗ 0q ≥ 1−cos2t∗ Set sint∗ = 2 . else q Set sint = 1−cos2t∗ . ∗ − 2 endif q end elementary.rotation

As usual we denote by subscribing 0 the projection of X onto a. Let d: p + 2 −→ R , X tr(X X0) be the squared distance of an element in p to a. Given a 7−→ − −1 matrix S p0 and a tolerance tol > 0, this algorithm overwrites S by kSk where k exp(k ∈) and d(kSk−1) tol. ∈ 0 ≤

Algorithm G2(II). Jacobi Algorithm. Set k := identity matrix. while d(S) > tol 30 for i =1:6 (cost∗, sint∗, cos2t∗, sin2t∗) := elementary.rotation(S, Ωi). u := exp(t∗Ωi). S := uSu−1. k := uk. endfor endwhile As an example, the regular element

0 3.17415 3.90421 4.63169 3.17415 3.90421 4.63169 3.17415− 0.993208− 3.14172− 2.55770 0 3.27510 2.76069 −3.90421 3.14172 3.224433 1.97516 3.27510 0− 2.24446 − − −  Sreg = 4.63169 2.55770 1.97516 4.23754 2.76069 2.24446 0 −3.17415 0 −3.27510− 2.76069 0.993208 −3.14172 2.5577   − − − −   3.90421 3.27510 0 2.24446 3.14172 3.24433 1.97516   4.63169 2.76069 2.24446− 0 − 2.5577− 1.97516 4.23754   − −    is almost diagonalized after 3 sweeps (oﬀ-norm < 10−10). It converges to the diagonal matrix

Zreg = diag 0, 9.12818, 1.97129, 11.0995, 9.12818, 1.97129, 11.0995 . − − − h i Irregular elements show the same convergence behavior. In all simulations, at most 3 sweeps were required for quasi diagonalization (off-norm < 10−10). Acknowledgments. This work presents partial results from my PhD thesis, [34]. I am very grateful to my supervisor U. Helmke, Würzburg, for his support and guidance. Many thanks also to G. Dirr, Würzburg, for a lot of fruitful discussions. National ICT Australia is funded by the Australian Government’s Department of Communications, Information Technology, and the Arts and the Australian Re- search Council through Backing Australia’s Ability and the ICT Research Centre of Excellence programs.

REFERENCES

[1] R. Abraham, J.E. Marsden, and T. Ratiu. Manifolds, Tensor Analysis, and Applications. 2nd Ed. Applied Mathematical Sciences 75. Springer, New York, 1988. [2] R.P. Brent and F.T. Luk. The solution of singular value and symmetric eigenvalue problems on multiprocessor arrays. SIAM J. Sci. Stat. Comput., 6(1):69–84, 1985. [3] R.W. Brockett. Dynamical systems that sort lists, diagonalize matrices, and solve linear pro- gramming problems. Lin. Algebra & Appl., 146:79–91, 1991. [4] A. Bunse-Gerstner, R. Byers, and V. Mehrmann. A chart of numerical methods for structured eigenvalue problems. SIAM J. Matrix Anal. Appl., 13(2):419–453, 1992. [5] A. Bunse-Gerstner and H. Fassbender. A Jacobi-like Method for Solving Algebraic Riccati Equations on Parallel Computers. IEEE Transactions on Automatic Control, 42(8):1071- 1084, 1997. [6] R. Byers. A Hamiltonian-Jacobi algorithm. IEEE Transactions on Automatic Control, 35(5):566-570, 1990. [7] J.-P. Charlier and P. Van Dooren. A Jacobi-like algorithm for computing the generalized Schur form of a regular pencil. J. Comput. Appl. Math., 27(1/2):17–36, 1989. [8] M.T. Chu and K. Driessel. The projected gradient method for least squares matrix approxi- mations with spectral constraints. SIAM J. Numer. Anal., 27:1050–1060, 1990. [9] Z. Drmaˇc and K. Veselić. New Fast and Accurate Jacobi SVD Algorithm I. Preprint. http://www.fernuni-hagen.de/MATHPHYS/veselic/pslist.html. [10] Z. Drmaˇcand K. Veselić. New Fast and Accurate Jacobi SVD Algorithm II. Preprint. http://www.fernuni-hagen.de/MATHPHYS/veselic/pslist.html. 31 [11] J. Demmel and K. Veselić. Jacobi’s method is more accurate than QR. SIAM J. Matrix Anal. Appl., 13(4):1204–1245. [12] J.J. Duistermaat and J.A.C. Kolk. Lie Groups. Springer, Berlin, 2000. [13] J.J. Duistermaat, J.A.C. Kolk, and V.S. Varadarajan. Functions, Flows and Oscillatory Inte- grals on Flag Manifolds and Conjugacy Classes in Real Semisimple Lie Groups. Compositio Mathematica, 49:309–398, 1983. [14] P.J. Eberlein. A Jacobi-like method for the automatic computation of eigenvalues and eigen- vectors of an arbitrary matrix. J. Soc. Indust. Appl. Math., 10(1):74–88, 1962. [15] P.J. Eberlein. On the diagonalization of complex symmetric matrices. J. Inst. Maths. Appl., 7:377–383, 1971. [16] H. Faßbender, D.S. Mackey, and N. Mackey. Hamilton and Jacobi come full circle: Jacobi algorithms for structured Hamiltonian eigenproblems. Lin. Algebra & Appl., 332-334:37– 80, 2001. [17] G.E. Forsythe and P. Henrici. The cyclic Jacobi method for computing the principal values of a complex matrix. Transactions of the American Math. Soc., 94:1–23, 1960. [18] H.H. Goldstine and L.P. Horwitz. A procedure for the diagonalization of normal matrices. J. of the ACM, 6:176–195, 1959. [19] G. Golub and C. F. van Loan. Matrix Computations. The Johns Hopkins University Press, Baltimore, 1996. [20] D. Hacon. Jacobi’s method for skew-symmetric matrices. SIAM J. Matrix Anal. Appl., 14(3):619–628, 1993. [21] V. Hari. On sharp quadratic convergence bounds for the serial Jacobi methods. Numer. Math., 60:375–406, 1991. [22] V. Hari and K. Veselić. On Jacobi methods for singular value decomposition. SIAM J. Sci. Stat. Comput., 8:741–754, 1987. [23] W. Hein. Struktur- und Darstellungstheorie der klassischen Gruppen. Springer, Berlin, 1990. [24] S. Helgason. Differential Geometry, Lie Groups, and Symmetric Spaces. Academic Press, San Diego, 1978. [25] U. Helmke and K. Hüper. The Jacobi Method: A Tool for Computation and Control. In: Systems and Control in the Twenty-First Century. Eds: C.I. Byrnes, and B.N. Datta, and D.S. Gilliam, and C.F. Martin. pp. 205–228, Birkhäuser Boston, 1997. [26] U. Helmke and K. Hüper. A Jacobi-type method for computing balanced realizations. Systems & Control Letters, 39:19–30, 2000. [27] U. Helmke, K. Hüper, and J.B. Moore. Computation of Signature Symmetric Balanced Real- izations. J. Global Optimization, 27:135–148, 2003. [28] P. Henrici. On the speed of convergence of cyclic and quasicyclic Jacobi methods for computing eigenvalues of Hermitian matrices. J. Soc. Indust. Appl. Math, 6(2):144–162, 1958. [29] K. Hüper. A Calculus Approach to Matrix Eigenvalue Algorithms. Habilitation thesis, Dep. of Mathematics, Würzburg University, Germany, July 2002. [30] K. Hüper. A Dynamical System Approach to Matrix Eigenvalue Algorithms. In: Mathemat- ical Systems Theory in Biology, Communications, Computation, and Finance. Eds: J. Rosenthal and D. S. Gilliam. IMA Vol 134, 257–274, Springer 2003. [31] K. Hüper. Structure and convergence of Jacobi-type methods for matrix computations. Ph.D. thesis, Technical University of Munich, June 1996. [32] C.G.J. Jacobi. Uber¨ ein leichtes Verfahren, die in der Theorie der Säcularstörungen vor- kommenden Gleichungen numerisch aufzulösen. Crelle’s J. für die reine und angewandte Mathematik, 30:51–94, 1846. [33] M. Kleinsteuber, U. Helmke, and K. Hüper. Jacobi’s Algorithm on Compact Lie Algebras. SIAM J. Matrix Anal. Appl. 26(1):42–69, 2004. [34] M. Kleinsteuber Jacobi-Type Methods on Semisimple Lie Algebras - A Lie Algebraic Approach to the Symmetric Eigenvalue Problem. PhD thesis, Dep. of Mathematics, University of Würzburg, January 2006. http://www.mathematik.uni-wuerzburg.de/ kleinsteuber/ [35] A.W. Knapp. Lie Groups Beyond an Introduction, 2nd Ed. Birkhäuser, Boston,∼ 2002. [36] E.G. Kogbetliantz. Solution of linear equations by diagonalization of coefficient matrix. Quart. Appl. Math., 13:1231–132, 1955. [37] N. Mackey. Hamilton and Jacobi meet again: Quaternions and the eigenvalue problem. SIAM J. Scientific and Statistical Computing, 5:470–475, 1984. [38] D.S. Mackey, N. Mackey, and D.M. Dunlavy. Structure preserving algorithms for perplectic Eigenproblems. Electronic J. Linear Algebra, 13:10–39, 2005. [39] W.F. Mascarenhas. A note on Jacobi being more accurate than QR. SIAM J. Matrix Anal. Appl., 15(1):215–218, 1994. [40] M.H.C. Paardekooper. An eigenvalue algorithm for skew-symmetric matrices. Num. Math.,

32 17:189–202, 1971. [41] C.C. Paige and P. Van Dooren. On the quadratic convergence of Kogbetliantz’s algorithm for computing the singular value decomposition. Lin. Algebra & Appl., 77:301–313, 1986. [42] N.H. Rhee and V. Hari. On the cubic convergence of the Paardekooper method. BIT, 35:116– 132, 1995. [43] G.W. Stewart. A Jacobi-like algorithm for computing the Schur decomposition of a nonhermi- tian matrix. SIAM J. Sci. Stat. Comput., 6(4):853–864, 1985. [44] T.Y. Tam. Gradient flows and double bracket equations. Differ. Geom. Appl., 20(2):209–224, 2004. [45] H.P.M. van Kempen. On the quadratic convergence of the special cyclic Jacobi method. Num. Math., 9:19–22, 1966. [46] K. Veselić. On a class of Jacobi-like procedures for diagonalizing arbitrary real matrices. Num. Math., 33:157–172, 1979. [47] K. Veselić. A Jacobi eigenreduction algorithm for definite matrix pairs. Num. Math., 64:241– 269, 1993. [48] N.J. Wildberger. Diagonalization in compact Lie algebras and a new proof of a theorem of Kostant. Proc. of the AMS, 119(2):649–655, 1993.