An asymptotic behavior of QR decomposition and its extension in semisimple Lie group. Tin-Yau Tam http://www.auburn.edu/»tamtiny Auburn University (Joint work with Huajun Huang) 1 I. QR decomposition A = n £ n nonsingular matrix: A = QR where ¤ Q Q = In;R upper ¢; +ve diag entries Remark: The factorization is unique. QR = Gram-Schmidt orthogonalization on the columns of A In matrix form, the algorithm goes: ¡1 AR1R2 ¢ ¢ ¢ Rn = Q; R = (R1R2 ¢ ¢ ¢ Rn) where R1;:::;Rn are upper triangular Gram-Schmidt as Triangular Orthogonalization 2 ² Some variants: A = R1Q1;R1 upper ¢ A = L2Q2;L2 lower ¢ Gram-Schmidt on the rows of A ² Disadvantage of Gram-Schmidt Sensitive to rounding error (orthogonality of the computed vectors can be lost quickly or may even be completely lost) ! modi¯ed Gram-Schmidt. 3 Example: 0 1 1 + ² 1 1 A = @ 1 1 + ² 1 A 1 1 1 + ² with very small ² such that 3 + 2² will be computed accurately but 3 + 2² + ²2 will be computed as 3 + 2². Then 0 1 p1+² p¡1 p¡1 B 3+2² 2 2C p 1 p1 0 Q ¼ @ 3+2² 2 A p 1 0 p1 3+2² 2 and cos θ12 = cos θ13 ¼ ¼=2 but cos θ23 ¼ 2¼=3. Remark: Gram-Schmidt resurfaces in some recent arti- cles, especially regarding its usefulness because it takes advantage of BLAS2 4 Computing QR by Householder reflections A Householder reflection is a reflection about some hy- perplane: T Qv = I ¡ 2vv ; kvk = 1: Qv sends v to ¡v and ¯xes pointwise the hyperplane that ? to v. Write A = [a1j ¢ ¢ ¢ jan] Set u T u = a1 ¡ ka1ke1; v = ;Q1 = I ¡ 2vv : kuk 5 Then 2 3 ka1k ¤ ::: ¤ 6 0 7 Q1A = 6 . 7 4 . A1 5 0 After t iterations of this process, t · n, R = Qt ¢ ¢ ¢ Q2Q1A is upper ¢. So, with Q = Q1Q2 ¢ ¢ ¢ Qt A = QR is a QR decomposition of A. This Householder approach is known as orthogonal tri- angularization Remark: This method has greater numerical stability than Gram-Schmidt. Backward stable. 6 Computing QR by Givens rotations ² A Givens rotation is simply a rotation · ¸ cos θ ¡ sin θ R(θ) = sin θ cos θ rotates x 2 R2 by θ. ² Choose θ 2 R so that · ¸ · ¸ " q # cos θ ¡ sin θ x x2 + x2 i = i j ; sin θ cos θ xj 0 x ¡x cos θ = q i ; sin θ = q j : 2 2 2 2 xi + xj xi + xj 7 ² Zero things bottom-up and left-right. 2 3 2 3 £ £ £ £ £ £ (2; 3) (1; 2) A = 4 £ £ £ 5 4 x x x 5 ! ! £ £ £ 0 x x 2 3 2 3 x x x £ £ £ (2; 3) (2; 3) 4 0 x x 5 4 x x 5 R ! ! £ £ 0 x 8 II. QR iterations Computing the eigenvalues of nonsingular A: De¯ne a sequence fAkgk2N of matrices with A1 := A and Aj+1 := RjQj if Aj = QjRj; j = 1; 2;::: Notice that ¡1 Aj+1 = Qj AjQj: (1) Similarity ! the eigenvalues of A are ¯xed in the process, counting multiplicities. One hopes to have some sort of convergence on the sequence fAkgk2N so that the \limit" would provide the eigenvalues of A. 9 Theorem 1. (Classical) Suppose that the moduli of the eigenvalues ¸1; : : : ; ¸n of A 2 GLn(C) are distinct: j¸1j > j¸2j > ¢ ¢ ¢ > j¸nj (> 0): (2) Let ¡1 A = Y diag (¸1; : : : ; ¸n)Y: Assume Y = LU; where L is lower ¢ and U is unit upper ¢. Then the strictly lower triangular part of Ak converges to zero and diag Ak ! diag (¸1; : : : ; ¸n) Remark: In the proof the QR decomposition of Am plays a role. 10 Basic QR algorithm 1. Turn A into Hessenberg form H via orthogonal sim- ilarity by Householder reflections. (Hp ¢ ¢ ¢ H2H1)A(H1H2 ¢ ¢ ¢ Hp) = H where 0 1 ¤ ¤ ¢ ¢ ¢ ¤ B¤ ¤ ¤ ¢ ¢ ¢ ¤C B C B ¤ ¤ ¤ ¢ ¢ ¢ ¤C H = B C B C @ ... ... ¤A ¤ ¤ 2. Perform QR iterations of H. Hi+1 inherits the Hessenberg shape of Hi. 11 Theorem 2. Same assumption as in the previous theo- rem. Let ! be the permutation matrix uniquely deter- mined by Y = L!U; where L is unit lower triangular and U is upper triangular. 1. The strictly lower triangular part of Ak converges to zero. 2. diag Ak ! diag (¸!(1); ¢ ¢ ¢ ; ¸!(n)) Remark: The decomposition A = L!U for nonsingular A is due to Gelfand and Naimark (1950) Remark: QR iteration to compute eigenvalues is due to Francis (1961) 12 III. a-component and numerical ex- periment Recall QR decomposition A = QR Set a(A) = diag (a1(A); : : : ; an(A)) =: diag (r11; : : : ; rnn) where written in column form A = [a1j ¢ ¢ ¢ jan] Geometric interpretation of a(A): rii is the distance between ai and the span of a1; : : : ; ai¡1, i = 2; : : : ; n. 13 Computing the discrepancy between a(Am)1=m and j¸j of randomly generated A. Example: 2 3 ¡13 + 52i ¡40 ¡ 64i 4 ¡ 50i ¡82 ¡ 59i 6 36 + 7i ¡55 ¡ 42i ¡94 + 5i ¡51 + 16i 7 A = : 4 ¡30 ¡ 18i ¡73 + 57i ¡64 ¡ 97i ¡14 + 91i 5 ¡48 ¡ 88i 80 ¡ 99i ¡87 ¡ 45i 76 + 43i j¸(A)j = (194:6; 158:3; 144:0; 24:64): Use MATLAB to plot the graph (Figure 1) of m 1=m ka(A ) ¡ diag (j¸1j;:::; j¸nj)k2 versus m (m = 1;:::; 100): 14 100 70 60 80 50 40 60 30 40 20 10 20 0 20 40 60 80 100 0 20 40 60 80 100 Figure 1 Figure 2 If we consider m 1=m ja1(X ) ¡ j¸1jj for the above example, in contrast to Figure 1, conver- gence occurs (Figure 2) using floating point: 15 More numerical experiments Let us do some computer generated pictures m 1=m Examine [a1(A )] ¡ j¸1(A)j qr large rand(6,10,100)) m 1=m and [an(A )] ¡ j¸n(A)j qr small rand(6,10,100) What is your conjecture? 16 IV. An asymptotic result Theorem 3. A; X; B = nonsingular matrices. Let X = Y ¡1DY be the Jordan decomposition of X, where D is the Jordan form of X, diag D = diag (¸1; : : : ; ¸n) satisfying j¸1j ¸ ¢ ¢ ¢ ¸ j¸nj: Then m 1=m lim a(AX B) = diag (j¸!(1)j;:::; j¸!(n)j); m!1 where the permutation ! is uniquely determined by YB = L!U: rank !(ijj) = rank YB(ijj); 1 · i; j · n: 1 Remark: There is no convergence of the sequence fQmgm=1, for example, if h i 0 1 A = B := I2;X := 1 0 ; 1 then fQmgm=1 = fX; I2; X; I2;::: g does not converge. 17 (1=m) 1 Example: fjRmj gm=1 may not converge. Let · ¸ 1 1 A := I ;X := ;B := I : 2 0 ¡1 2 Then ½ X; m odd AXmB = I2; m even. and · ¸ 1 1 jR j(1=(2m+1)) = ; 2m+1 0 1 · ¸ 1 0 jR j(1=(2m)) = : 2m 0 1 18 Example: (Even it converges...) Let 1 > a > b > 0 and 2 3 2 3 2 3 1 0 0 1 0 0 0 0 1 A := I3;X := 4 0 a 0 5 ;B := 4 1 1 0 5 4 1 0 0 5 : 0 0 b 2 1 1 0 1 0 By direct computation 2 p 3 a2m + b2m p b2m pa2m+2b2m 6 a2m+b2m a2m+b2m 7 ambm ambm jRmj = 6 0 p p 7 : 4 a2m+b2m a2m+b2m 5 0 0 1 So 2 3 a b2=a a (1=m) lim jRmj = 4 0 b b 5 ; m!1 0 0 1 (m) 1=m 2 and limm!1 jr12 j = b =a, which is not an eigenvalue modulus of X. 19 VI Extension to Iwasawa compo- nent G = connected semisimple Lie group. Iwasawa decomposition of G: G = KAN For g 2 G, write g = kan where k 2 K, a 2 A, n 2 N are uniquely de¯ned. Example: For G = SL(n; C), or SL(n; R), QR ! Iwasawa decomposition k = Q; a = diag R; n = a¡1R 20 Example: The symplectic group µ ¶ T 0 In Spn(R) = fg 2 GL2n(R): g Jng = Jng;J = : ¡In 0 The Iwasawa decomposition G1 = K1A1N1: µ ¶ AB K = f : A + iB 2 U(n)g; 1 ¡BA ¡1 ¡1 A1 = fµdiag (a1; : : : ;¶ an; a1 ; : : : ; an ): a1; : : : ; an > 0g; AB N = f : A unit upper¢; ABT = BT Ag: 1 0 (A¡1)T It is not the Iwasawa decomposition of g = kan 2 SL2n(R); k 2 SO(2n), a +ve diagonal, and n unit upper triangular. 21 Iwasawa decomposition ² g = Lie algebra of connected semisimple G. ² g = k + p a ¯xed Cartan decomposition of g ² K ½ G the connected subgroup with Lie algebra k. ² a ½ p a maximal abelian subspace. ² Fix a closed Weyl chamber a+ in a Set A := exp a;A := exp a + P + N := exp n; n := ®>0 g®; n is the sum of all positive root spaces. Iwasawa decomposition: G = KAN 22 Example: A 2 sl(n; C) has Hermitian decomposition: A = (A ¡ A¤)=2 + (A + A¤)=2: It is a Cartan decomposition for sl(n; C) sl(n; C) = k + p = su(n) + isu(n) where k = su(n) is the algebra of all trace 0 skew Her- mitian matrices and p = isu(n) is the space of all trace 0 Hermitian matrices. K = SU(n) the special unitary group. Pick a = fdiag (a1; : : : ; an): a1 + ¢ ¢ ¢ + an = 0g; a maximal abelian subspace in p. a+ = fdiag (a1; : : : ; an): a1 ¸ ¢ ¢ ¢ ¸ an; a1 + ¢ ¢ ¢ + an = 0g a closed Weyl chamber of a. A+ = fdiag (a1; : : : ; an): a1 ¢ ¢ ¢ an = 1g ½ SLn(C).