An Asymptotic Behavior of QR Decomposition and Its Extension In

An asymptotic behavior of QR decomposition and its extension in

semisimple Lie group. Tin-Yau Tam http://www.auburn.edu/∼tamtiny Auburn University

(Joint work with Huajun Huang)

1 I. QR decomposition A = n × n nonsingular matrix: A = QR where ∗ Q Q = In,R upper ∆, +ve diag entries

Remark: The factorization is unique.

QR = Gram-Schmidt orthogonalization on the columns of A

In matrix form, the algorithm goes: −1 AR1R2 ··· Rn = Q, R = (R1R2 ··· Rn) where R1,...,Rn are upper triangular Gram-Schmidt as Triangular Orthogonalization

2 • Some variants:

A = R1Q1,R1 upper ∆

A = L2Q2,L2 lower ∆ Gram-Schmidt on the rows of A

• Disadvantage of Gram-Schmidt Sensitive to rounding error (orthogonality of the computed vectors can be lost quickly or may even be completely lost) → modiﬁed Gram-Schmidt.

3 Example:   1 + ² 1 1 A =  1 1 + ² 1  1 1 1 + ² with very small ² such that 3 + 2² will be computed accurately but 3 + 2² + ²2 will be computed as 3 + 2². Then   √1+² √−1 √−1  3+2² 2 2 √ 1 √1 0 Q ≈  3+2² 2  √ 1 0 √1 3+2² 2 and cos θ12 = cos θ13 ≈ π/2 but cos θ23 ≈ 2π/3. Remark: Gram-Schmidt resurfaces in some recent arti- cles, especially regarding its usefulness because it takes advantage of BLAS2

4 Computing QR by Householder reﬂections

A Householder reﬂection is a reﬂection about some hyperplane: T Qv = I − 2vv , kvk = 1.

Qv sends v to −v and ﬁxes pointwise the hyperplane that ⊥ to v. Write

A = [a1| · · · |an] Set u T u = a1 − ka1ke1, v = ,Q1 = I − 2vv . kuk

5 Then   ka1k ∗ ... ∗  0  Q1A =  .   . A1  0 After t iterations of this process, t ≤ n,

R = Qt ··· Q2Q1A is upper ∆. So, with

Q = Q1Q2 ··· Qt A = QR is a QR decomposition of A.

This Householder approach is known as orthogonal tri- angularization

Remark: This method has greater numerical stability than Gram-Schmidt. Backward stable.

6 Computing QR by Givens rotations

• A Givens rotation is simply a rotation · ¸ cos θ − sin θ R(θ) = sin θ cos θ rotates x ∈ R2 by θ.

• Choose θ ∈ R so that · ¸ · ¸ " q # cos θ − sin θ x x2 + x2 i = i j , sin θ cos θ xj 0 x −x cos θ = q i , sin θ = q j . 2 2 2 2 xi + xj xi + xj

7 • Zero things bottom-up and left-right.     × × × × × × (2, 3) (1, 2) A =  × × ×   x x x  → → × × × 0 x x     x x x × × × (2, 3) (2, 3)  0 x x   x x  R → → × × 0 x

8 II. QR iterations Computing the eigenvalues of nonsingular A:

Deﬁne a sequence {Ak}k∈N of matrices with

A1 := A and

Aj+1 := RjQj if Aj = QjRj, j = 1, 2,...

Notice that −1 Aj+1 = Qj AjQj. (1)

Similarity → the eigenvalues of A are ﬁxed in the process, counting multiplicities.

One hopes to have some sort of convergence on the sequence {Ak}k∈N so that the “limit” would provide the eigenvalues of A.

9 Theorem 1. (Classical) Suppose that the moduli of the eigenvalues λ1, . . . , λn of A ∈ GLn(C) are distinct:

|λ1| > |λ2| > ··· > |λn| (> 0). (2) Let −1 A = Y diag (λ1, . . . , λn)Y. Assume Y = LU, where L is lower ∆ and U is unit upper ∆. Then the strictly lower triangular part of Ak converges to zero and

diag Ak → diag (λ1, . . . , λn)

Remark: In the proof the QR decomposition of Am plays a role.

10 Basic QR algorithm

1. Turn A into Hessenberg form H via orthogonal similarity by Householder reﬂections.

(Hp ··· H2H1)A(H1H2 ··· Hp) = H where   ∗ ∗ · · · ∗ ∗ ∗ ∗ · · · ∗    ∗ ∗ ∗ · · · ∗ H =      ...... ∗ ∗ ∗

2. Perform QR iterations of H.

Hi+1 inherits the Hessenberg shape of Hi.

11 Theorem 2. Same assumption as in the previous theorem. Let ω be the permutation matrix uniquely determined by Y = LωU, where L is unit lower triangular and U is upper triangular.

1. The strictly lower triangular part of Ak converges to zero.

diag Ak → diag (λω(1), ··· , λω(n))

Remark: The decomposition A = LωU for nonsingular A is due to Gelfand and Naimark (1950)

Remark: QR iteration to compute eigenvalues is due to Francis (1961)

12 III. a-component and numerical ex- periment Recall QR decomposition A = QR Set

a(A) = diag (a1(A), . . . , an(A)) =: diag (r11, . . . , rnn) where written in column form

A = [a1| · · · |an]

Geometric interpretation of a(A): rii is the distance between ai and the span of a1, . . . , ai−1, i = 2, . . . , n.

13 Computing the discrepancy between a(Am)1/m and |λ| of randomly generated A.

Example:   −13 + 52i −40 − 64i 4 − 50i −82 − 59i  36 + 7i −55 − 42i −94 + 5i −51 + 16i  A = .  −30 − 18i −73 + 57i −64 − 97i −14 + 91i  −48 − 88i 80 − 99i −87 − 45i 76 + 43i

|λ(A)| = (194.6, 158.3, 144.0, 24.64). Use MATLAB to plot the graph (Figure 1) of

m 1/m ka(A ) − diag (|λ1|,..., |λn|)k2 versus m (m = 1,..., 100):

14 100 70

80 50

40 60

40 20

10 20 0 20 40 60 80 100 0 20 40 60 80 100

Figure 1 Figure 2

If we consider m 1/m |a1(X ) − |λ1|| for the above example, in contrast to Figure 1, convergence occurs (Figure 2) using ﬂoating point:

15 More numerical experiments Let us do some computer generated pictures

m 1/m Examine [a1(A )] − |λ1(A)| qr large rand(6,10,100))

m 1/m and [an(A )] − |λn(A)| qr small rand(6,10,100)

What is your conjecture?

16 IV. An asymptotic result

Theorem 3. A, X, B = nonsingular matrices. Let X = Y −1DY be the Jordan decomposition of X, where D is the Jordan form of X,

diag D = diag (λ1, . . . , λn) satisfying |λ1| ≥ · · · ≥ |λn|. Then m 1/m lim a(AX B) = diag (|λω(1)|,..., |λω(n)|), m→∞ where the permutation ω is uniquely determined by YB = LωU: rank ω(i|j) = rank YB(i|j), 1 ≤ i, j ≤ n.

∞ Remark: There is no convergence of the sequence {Qm}m=1, for example, if h i 0 1 A = B := I2,X := 1 0 , ∞ then {Qm}m=1 = {X,I2,X,I2,... } does not converge. 17 (1/m) ∞ Example: {|Rm| }m=1 may not converge. Let · ¸ 1 1 A := I ,X := ,B := I . 2 0 −1 2 Then ½ X, m odd AXmB = I2, m even. and · ¸ 1 1 |R |(1/(2m+1)) = , 2m+1 0 1 · ¸ 1 0 |R |(1/(2m)) = . 2m 0 1

18 Example: (Even it converges...) Let 1 > a > b > 0 and       1 0 0 1 0 0 0 0 1 A := I3,X :=  0 a 0  ,B :=  1 1 0   1 0 0  . 0 0 b 2 1 1 0 1 0 By direct computation  p  a2m + b2m √ b2m √a2m+2b2m  a2m+b2m a2m+b2m  ambm ambm |Rm| =  0 √ √  .  a2m+b2m a2m+b2m  0 0 1

So   a b2/a a (1/m) lim |Rm| =  0 b b  , m→∞ 0 0 1

(m) 1/m 2 and limm→∞ |r12 | = b /a, which is not an eigenvalue modulus of X.

19 VI Extension to Iwasawa component G = connected semisimple Lie group.

Iwasawa decomposition of G: G = KAN

For g ∈ G, write g = kan where k ∈ K, a ∈ A, n ∈ N are uniquely deﬁned.

Example: For G = SL(n, C), or SL(n, R),

QR → Iwasawa decomposition k = Q, a = diag R, n = a−1R

20 Example: The symplectic group µ ¶ T 0 In Spn(R) = {g ∈ GL2n(R): g Jng = Jn},J = . −In 0

The Iwasawa decomposition G1 = K1A1N1: µ ¶ AB K = { : A + iB ∈ U(n)}, 1 −BA −1 −1 A1 = {µdiag (a1, . . . ,¶ an, a1 , . . . , an ): a1, . . . , an > 0}, AB N = { : A unit upper∆, ABT = BT A}. 1 0 (A−1)T

It is not the Iwasawa decomposition of

g = kan ∈ SL2n(R), k ∈ SO(2n), a +ve diagonal, and n unit upper triangular.

21 Iwasawa decomposition

• g = Lie algebra of connected semisimple G.

• g = k + p a ﬁxed Cartan decomposition of g

• K ⊂ G the connected subgroup with Lie algebra k.

• a ⊂ p a maximal abelian subspace.

• Fix a closed Weyl chamber a+ in a

Set A := exp a,A := exp a + P + N := exp n, n := α>0 gα, n is the sum of all positive root spaces. Iwasawa decomposition: G = KAN

22 Example: A ∈ sl(n, C) has Hermitian decomposition: A = (A − A∗)/2 + (A + A∗)/2. It is a Cartan decomposition for sl(n, C) sl(n, C) = k + p = su(n) + isu(n) where k = su(n) is the algebra of all trace 0 skew Her- mitian matrices and p = isu(n) is the space of all trace 0 Hermitian matrices. K = SU(n) the special unitary group. Pick

a = {diag (a1, . . . , an): a1 + ··· + an = 0}, a maximal abelian subspace in p. a+ = {diag (a1, . . . , an): a1 ≥ · · · ≥ an, a1 + ··· + an = 0} a closed Weyl chamber of a.

A+ = {diag (a1, . . . , an): a1 ··· an = 1} ⊂ SLn(C). 23 CMJD for real semisimple G • h ∈ G is hyperbolic if h = exp(X) where X ∈ g is real semisimple, that is, ad X ∈ End (g) is diagonalizable over R. • u ∈ G is unipotent if u = exp(N) where N ∈ g is nilpotent, that is, ad N ∈ End (g) is nilpotent. • e ∈ G is elliptic if Ad (e) ∈ Aut (g) is diagonalizable over C with eigenvalues of modulus 1. Each g ∈ G can be uniquely written as g = ehu, where e is elliptic, h is hyperbolic and u is unipotent and the three elements e, h, u commute. Write g = e(g)h(g)u(g)

Remark: h ∈ G is hyperbolic if and only if it is conjugate to a unique element b(h) ∈ A+. Denote b(g) := b(h(g)).

24 Bruhat decomposition The Weyl group W = M 0/M where M 0 is the normalizer of a in K M is the centralizer of a in K W acts on A and a: for each s ∈ W , s · a = σ a σ−1, a ∈ A, σ ∈ s, s · H = Ad (σ)H,H ∈ a, σ ∈ s. Set X − − − n = gα,N = exp n α<0 Bruhat decomposition: − G = ∪s∈W N msMAN − is a disjoint union, where ms ∈ W . Moreover N MAN is an open submanifold of G and other terms are sub- manifolds of lower dimensions. 25 Theorem 4. Let u, v, g ∈ G. Let g = ehu be the CMJD of g. Let h = y−1b(g)y for some y ∈ G, and − yv ∈ N msMAN in the Bruhat decomposition, where s ∈ W is the Weyl 0 group element determined by ms ∈ M . Then m 1/m −1 −1 lim [a(ug v)] = s · b(g) = ms b(g)ms, (3) m→∞ 0 where the limit is independent of the choice of ms ∈ M such that s = msM.

Idea: Use the result for GLn(C) and adjoint representa- tion Ad : G → Aut (g).

26 Remark: How about G = NAK? Write g = n0a0k0 = kan, n, n0 ∈ N, a, a0 ∈ A, k, k0 ∈ K. Since g−1 = n−1a−1k−1, by the uniqueness of Iwasawa decomposition a0(g) = [a(g−1)]−1 so that lim [a0(ugmv)]1/m = lim [a(v−1(g−1)mu−1)]−1/m m→∞ m→∞ = ( lim [a(v−1(g−1)mu−1)]1/m)−1. m→∞ Now the CMJD g = ehu ⇒ g−1 = e−1h−1u−1, yhy−1 = b(g) ⇒ yh−1y−1 = [b(g)]−1 So b(g−1) = `[b(g)]−1`−1 where ` is the longest element of the Weyl group. So by Theorem 4 lim [a0(ugmv)]1/m = [s−1 · ` · (b(g))−1]−1 = (s−1`) · b(g), m→∞ −1 − where m`yu ∈ N msMAN. 27 The map L : G → A where L(g) := lim [a(gm)]1/m m→∞ is well deﬁned but not continuous: for example −1 gk := ykdyk , where d := diag (d1, d2) with d1 > d2 > 0,

yk := I2/k + ω ∈ GL2(R) where ω is the transposition (1, 2). So L(gk) = d but

L( lim gk) = diag (d2, d1). k→∞ Corollary:

L(OK(g)) = L(OG(g)) = W b(g), where −1 OG(g) := {vgv : v ∈ G} −1 OK(g) := {vgv : v ∈ K}

28 The End! Have a nice weekend.