A New Eigenvalue Algorithm for Unitary Hessenberg Matrices Via Quasiseparable Representations

A new eigenvalue algorithm for unitary Hessenberg matrices via quasiseparable representations

Yuli Eidelman Tel-Aviv University Joint work with I. Haimovici

Kalamata, July, 2018

1 Unitary upper Hessenberg matrices.

The N ×N matrix U is called upper Hessenberg if its entries below the ﬁrst subdiagonal are zeros, i.e. Uij = 0 for i > j +1. We consider here upper Hessenberg matrices which are also unitary. Also, we suppose that U is unreduced, which means that the subdiagonal entries could not be 0.

If the subdiagonal entries of the matrix U are nonnegative the matrix U has the representation  ∗ ∗ ∗ ∗ ∗  −ρ ρ −ρ µ ρ −ρ µ µ ρ · · · −ρ − µ − ...µ ρ −ρ µ − ...µ ρ  1 0 2 1 0 3 2 1 0 N 1 N 2 1 0 N N 1 1 0   − ∗ − ∗ · · · − ∗ − ∗   µ1 ρ2ρ1 ρ3µ2ρ1 ρN−1µN−2...µ2ρ1 ρN µN−1...µ2ρ1   − ∗ · · · − ∗ − ∗   0 µ2 ρ3ρ2 ρN−1µN−2...µ3ρ2 ρN µN−1...µ3ρ2   . . . .  ,  . .. µ3 . .   ...... − ∗ − ∗   . ρN−1ρN−2 ρN µN−1ρN−2  ··· ··· − ∗ 0 0 µN−1 ρN ρN−1 | |2 2 − − | | where µk > 0, ρk + µk = 1 (k = 1,...,N 1), ρ0 = 1, ρN = 1. 2 The quasiseparable representation.

For an N × N matrix:

Quasiseparable representation:    piai−1 ··· aj+1qj, 1 ≤ j < i ≤ N, A = d , 1 ≤ i = j ≤ N, . ij  i gibi+1 ··· bj−1hj, 1 ≤ i < j ≤ N. Here

× L L × L × L pi : 1 ri−1, qj : rj 1, ak : rk rk−1 di : 1 × 1

× U U × U × U gi : 1 ri , hj : rj−1 1, bk : rk−1 rk

L U are called quasiseparable generators of the matrix A with orders rk , rk . 3 The Hermitian matrices.

For a Hermitian matrix the diagonal entries d(k) are real and the upper quasiseparable generators could be obtained from the lower ones by taking ∗ ∗ ∗ − gk = qk, hk = pk, bk = ak k = 2,...,N 1,

∗ ∗ g1 = q1, hN = pN .

The eigenvalue computation methods for Hermitian matrices with quasiseparable representations were developed. We use essentially the results by

E., I. Haimovici, In Operator Theory: Advances and Applications (OTAA) volume in honour of Rien Kaashoek, accepted 4 The basic approach.

For a unitary matrix U of size N × N, all its eigenvalues are on the unit circle, i.e. they are of the form

λk = cos θk + i · sin θk, θk ∈ [0, 2π), k = 1,...,N, The cosine part and respectively the sine part are the eigenvalues of the Hermitian matrices 1 ∗ i ∗ A = (U + U ),B = (U − U). 2 2 So that if we ﬁnd the eigenvalues of A, we readily know the eigenvalues of U up to the sign of the imaginary part: √ 2 λk = cos θk i · 1 − cos θk, θk ∈ [0, 2π), k = 1,...,N.

5 The quasiseparable generators for unitary Hessenbeg matrices.

For a unitary upper Hessenberg matrix U we use the generators

pk = µk−1, hk = −ρk, k = 2,...,N,

gk = µkρk−1, qk = 1, k = 1,...,N − 1,

ak = 0, bk = µk, k = 2,...,N − 1,

d1 = ρ1, dk = −ρkρk−1, k = 2,...,N. Similarly for U∗.

6 The quasiseparable generators for Hermitian matrices.

1 ∗ For the Hermitian matrix A = 2(U + U ) we can use the lower quasiseparable generators [ ] = 1 −ρk = 2 pk 2 2 , k ,...,N, [ ] T qk = µk µkρk−1 , k = 1,...,N − 1, [ ] 0 0 ak = , k = 2,...,N − 1, 0 µk

d1 = ℜ(ρ1), dk = −ℜ(ρk−1ρk), k = 2,...,N. Similarly for the Hermitian matrix B.

7 The bisection method.

Let γk(λ) = det(A(1 : k, 1 : k) − λI) be characteristic polynomials of the principal submatrices of the quasiseparable matrix A.

The number of sign changes for the sequence of polynomials γk(λ) equals the number of negative elements in the sequence Dk(λ) for the ratio of characteristic polynomials

γk(λ) Dk(λ) = . γk−1(λ)

8 The quasiseparable recursive relations for Hermitian matrices.

∗ q1q1 1. D1(λ) = d1 − λ, f1(λ) = D1(λ)

2. For k = 2,...,N − 1: − − ∗ Dk(λ) = dk λ pkfk−1(λ)pk, 1 ∗ − ∗ zk(λ) = [qk pkfk−1(λ)ak], Dk(λ) ∗ − ∗ fk(λ) = akfk−1(λ)ak + [qk akfk−1(λ)pk]zk(λ). − − ∗ 3. DN (λ) = dN λ pN fN−1(λ)pN . Here fk(λ) are auxiliary matrix valued rational functions.

9 The bisection step for the cosines.

The present algorithm receives the Schur parameters ρ(k), µk, k = 0,...,N of the N × N unitary upper Hessenberg matrix U and a real number λ and it computes the number ν of the eigenvalues of the 1 ∗ matrix A = 2(U + U ) which are less than λ. ( ) 1 1. Compute D = ℜ(ρ ) − λ, z = , f = z z∗µ2/D . Set 1 1 1 −1 1 1 1 1 1 ν = 1 if D1 < 0 and ν = 0 otherwise. [ ] 2. For = 2 − 1 set = 1 −ρk and compute k ,...,N pk 2 2 ∗ D = −ℜ(ρ ρ ) − λ − p f p , ( k k−1 k k k−1 k ) 1 zk = − 1 − ρk−1 2(fk−1(2, 1) fk−1(2, 2)ρk) and 10 ∗ ϕ = z z /D , ( k k k k ) 2 ϕk(1, 1) ϕk(1, 2) fk = µk . ϕk(2, 1) ϕk(2, 2) + fk−1(2, 2)

Set ν = ν + 1 if Dk < 0. [ ] 3. = 1 −ρN and compute pN 2 2 −ℜ − − ∗ DN = (ρN−1ρN ) λ pN fN−1pN .

Set ν = ν + 1 if DN < 0.

11 The complexity.

When completely optimized, without 2×2 and 2×1 small matrices or vectors and when all what is possible is pre-computed, the complexity is cν = 16.5N arithmetic operations on scalars. The 0.5 comes from the fact that only if Dk < 0, i.e. in half of the cases, one computes ν = ν + 1.

The complexity is only 3.3 times the one for symmetric tridiagonal matrices (Wilkinson).

12 The bisection procedure.

1. For k = 1,...,N set L0(k) = −1,R0(k) = 1. Set U = 1.

2. For k = N,..., 2, 1 perform the following steps.

2.1 Set L = max1≤j≤k L0(j) and U = min{U, R0(k)}.

2.2. While U − L is larger than the machine precision, perform the following. 2.2.1. Set λ = (L + U)/2.

2.2.2. Use the algorithm to obtain the number ν of eigenvalues of the matrix A which are less than λ.

2.2.3. If ν ≥ k set U = λ, else set L = λ, L0(ν + 1) = L and R0(ν) = L if ν ≠ 0 and R0(ν) > L.

th 2.3. Set R0(k) = (U + L)/2 to contain the k eigenvalue. 13 Finding the sign of sines. √ Assume that α is an eigenvalue of A and β = 1 − α2.

If the multiplicity of α equals 2 then the numbers αiβ are eigenvalues of the unitary matrix U. This happens always for real orthogonal matrices.

If the multiplicity of α equals 1 then one of the numbers α + iβ and α − iβ is an eigenvalue of U.

If β is an eigenvalue of the matrix B then α + iβ is an eigenvalue of U. If −β is an eigenvalue of the matrix B then α − iβ is an eigenvalue of U.

It is sufficient to find the difference of the numbers ν just before and just after non-negative values of β to find out which of β gives an eigenvalue. 14 The cases.

However, we do not want to call ﬁnd ν for the matrix B 2N times and we want to determine precisely what means ”just before”, so that we perform ﬁrst some un-expensive linear time steps and then we call it only 0.72N times at most.

In the rare case when both α are simple eigenvalues of A and both β are simple eigenvalues of B, then one of the pairs α + iβ, −α − iβ or α − iβ, −α + iβ are the eigenvalues of U. To recognize which is indeed, some eigenvector computation is used.

We ﬁnd the eigenvector v such that Bv = βv, we ﬁnd out for which index kM we have |v(kM )| = max1≤k≤N |v(k)| and we compute λ = U(kM , 1 : N)v/v(kM ).

Finding the sign of all sines takes only 1.7% of overall time. 15 The unitary Hessenberg eigenvector algorithm.

Fast but nonstable.

16 Numerical tests.

All the numerical experiments have been performed on a computer with an i7-5820 microprocessor, 31.9 gigabytes installed memory (RAM) at 3.30GHz and another 4GB in the video card GTX, which is exploited by Matlab as well. The operating system is Windows 10, 64 bits and the machine precision is 2.2204e-16, as given by the Matlab command eps. We wrote the program using Matlab version R2016B.

17 The errors estimate.

The formula used for plotting the errors is the average error over the k = 1,...,N eigenvalues of each of the n = 20 considered matrices of each size N × N, N being a power of 2 starting with 22 and ﬁnishing with 213 = 8, 192.

j If we denote by λT (s) the true known eigenvalue s of one of the j = 1,..., 20 considered matrices and by λj(k) the eigenvalue obtained by bisection, then the formula is

1 ∑20 1 ∑N ( min |λj(k) − λj (s)|), 20 s∈S T j=1 N k=1 where the set of indexes S contains those s ∈ {1,...,N} that have not been previously used by another λj(k). 18 Random data.

We build generators for random unitary Hessenberg matrices, which in principle rarely have multiple eigenvalues, since this is not even possible if the matrix is unreduced and the latter takes place whenever none of the Schur parameters ρk, k = 1,...,N −1 is on the unit circle. We build the (sub) unit complex numbers ρk, k = 0,...,N, such that ρ0 = −1 and |ρN | = 1, and

ρk = exp(2π · i · rand) · rand, k = 1,...,N − 1, where rand gives a random number between 0 and 1. The subdiagonal entries of U are 2 − | |2 − µk = 1 ρk , k = 1,...,N 1. In this way any possible unitary upper Hessenberg unreduced matrix could be obtained. 19 Comparison with implicit QR.

The implicit QR developed in Gragg, 1986 and Wang, Gragg, 2002 and presented in Vandebril, Van Barel, Mastronardi, Matrix compu- tations and semiseparable matrices, Vol. II, Eigenvalue and singular value methods, 2008.

After we ﬁnd the quasiseparable generators of the matrix U, we build the whole unitary upper Hessenberg matrix U out of its generators and we use the Matlab function eig(U) in order to ﬁnd the Matlab eigenvalues, which we consider to be the golden, true ones.

20 Comparison with implicit QR, the ﬁgure

105 10-2 QR Bisection 4 10 10-4

3 10 10-6

102 10-8

101 10-10

100 10-12 Time QR versus the new algorithm 10-1 10-14 Average and max error versus Matlab eigenvalues 10-2 4 8 16 64 256 1024 8192 4 8 16 64 256 1024 8192 For QR a few times faster For bisection much better

21 Comparison with implicit QR.

Optimized implicit QR works much faster than the bisection algorithm presented here and it would be for sure faster, even without any optimization. However, surprisingly again, the bisection is much more exact. Among all the matrices that we checked, the largest error of the bisection, for the worst ever eigenvalue has been 4e − 13, while for implicit QR 5e − 07 and errors of 5e − 08 are very common. In general, for sizes 16 to 32 bisection is only 5 times more exact, while for 64 up to 256 it gives 3 correct decimal digits more than QR for the average error, and for larger sizes it is 104 times more correct and in some experiments with matrices up to the size 2, 048, 105 times. For one matrix of size 8, 192 QR gave an error of 3e − 03 as its worst error, while the worst error ever of bisection has been better by 8 decimal digits. Moreover, the error of QR does not increase smoothly in proportion with the size of the matrix. 22 Eigenvalues known in advance.

In Ammar, Gragg, Reichel, 1991 the Schur parameters of a unitary upper Hessenberg unreduced matrix are built from eigenvalues which are given a priori. We implemented their method and we checked our algorithm and Matlab eig() function results when compared to the true eigenvalues, which are known in advance. Since the Schur parameters are built in O(N2) operations and for checking Matlab one must also build the whole matrix U the results contained larger errors than when comparing our results to those of Matlab. What is important is that there exists a strong correlation between us and Matlab in all the eigenvalues.

23 Eigenvalues known in advance.

10-10 Matlab Bisection Our implementation of the O(N2) 10-11 function that builds a unitary Hessenberg matrix out of 10-12 eigenvalues known in advance as in the paper of Ammar, Gragg and Reichel is not good enough 10-13 and building the whole matrix for the application of Matlab function eig() also introduces 10-14 errors. But we can see that bisection and Matlab find most of the time the same eigen- -15 10 values. Error Matlab versus the new algorithm

10-16 4 8 16 32 64 128 256 5121024 The errors are correlated