Analysis of a Fast Hankel Eigenvalue Algorithm
a b
Franklin T Luk and Sanzheng Qiao
a
Department of Computer Science
Rensselaer Polytechnic Institute
Troy New York USA
b
Department of Computing and Software
McMaster University
Hamilton Ontario L S L Canada
ABSTRACT
This pap er analyzes the imp ortant steps of an O n log n algorithm for nding the eigenvalues of a complex Hankel
matrix The three key steps are a Lanczos type tridiagonalization algorithm a fast FFT based Hankel matrix vector
pro duct pro cedure and a QR eigenvalue metho d based on complex orthogonal transformations In this pap er we
present an error analysis of the three steps as well as results from numerical exp eriments
Keywords Hankel matrix eigenvalue decomp osition Lanczos tridiagonalization Hankel matrix vector multipli
cation complex orthogonal transformations error analysis
INTRODUCTION
The eigenvalue decomp osition of a structured matrix has imp ortant applications in signal pro cessing In this pap er
nn
we consider a complex Hankel matrix H C
h h h h
n n
h h h h
B C
n n
B C
B C
H
B C
A
h h h h
n n n n
h h h h
n n n n
The authors prop osed a fast algorithm for nding the eigenvalues of H The key step is a fast Lanczos tridiago
nalization algorithm which employs a fast Hankel matrix vector multiplication based on the Fast Fourier transform
FFT Then the algorithm p erforms a QR like pro cedure using the complex orthogonal transformations in the di
agonalization to nd the eigenvalues In this pap er we present an error analysis and discuss the stability prop erties
of the key steps of the algorithm
This pap er is organized as follows In Section we discuss eigenvalue computation using complex orthogonal
transformations and describ e the overall eigenvalue pro cedure and three key steps In Section we introduce
notations and error analysis of some basic complex arithmetic op erations We analyze the orthogonality of the
Lanczos vectors in Section and the accuracy of the Hankel matrix vector multiplication in Section In Section
we derive the condition number of a complex orthogonal plane rotation and we provide a backward error analysis of
the application of this transformation to a vector Numerical results are given in Section
Supp orted by the Natural Sciences and Engineering Research Council of Canada under grant OGP
COMPLEX ORTHOGONAL TRANSFORMATIONS
An eigenvalue decomp osition of a nondefective matrix H is given by
H XDX
T T
We will pick the matrix X to b e complex orthogonal that is XX I So H XDX We apply a sp ecial Lanczos
tridiagonalization to the Hankel matrix assuming that the Lanczos pro cess do es not prematurely terminate
T
H QJ Q
where Q is complex orthogonal and J is complex symmetric tridiagonal
B C
B C
B C
J
B C
B C
A
n
n n
T
Let Q q q q Since Q Q I we see that
n
T
q q
j ij
i
We call the prop erty de ned by as complex orthogonality c orthogonality for short
The dominant cost of the Lanczos metho d is matrix vector multiplication We prop ose an O n log n FFT based
n
multiplication scheme so that we can tridiagonalize a Hankel matrix in O n log n op erations Given w C we
n
b
want to compute the pro duct p H w De ne c C by
T
b
c h h h h h h
n n n n
T
n
b
Let w w w w De ne w C by
n
T
b
w w w w
n n
Let t v denote one dimensional FFT of a vector v i t v one dimensional inverse FFT of v and comp onent
b b
wise multiplication of two vectors The desired pro duct is given by the rst n elements of the vector i t t c t w
The authors showed that Algorithm requires n log n O n ops real oating p oint op erations
To maintain symmetry and tridiagonality of J we apply complex orthogonal rotations in the QR iterations
T
J WDW
where W is complex orthogonal and D is diagonal Thus we get with X QW The workhorse of our QR like
nn
iteration is a complex orthogonal plane rotation P C That is P is an identity matrix except for four
strategic p ositions formed by the intersection of the ith and j th rows and columns
c s
p p
ii ij
G
s c
p p
j i j j
T
where c s The matrix G is constructed to annihilate the second comp onent of a vector x x x
We present our computational schemes in the following
Overall Procedure Hankel Eigenvalue Computation Given H compute al l its eigenvalues
Apply a Lanczos pro cedure Algorithm to reduce H to a tridiagonal form J Use Algorithm to compute
the Hankel matrix vector pro ducts
Apply a QR type pro cedure Algorithm to nd the eigenvalues of J
Algorithm Lanczos Tridiagonalization Given H this algorithm computes a tridiagonal matrix J
T
Initialize q such that q q
v H q
for j n
T
q v
j j
j
r v q
j j j j
if j n
q
T
r r
j j
j
if break end
j
q r
j j j
v H q q
j j j j
end
end
n
Algorithm Fast Hankel Matrix Vector Multiplication Given w C compute p H w
n
b b
Construct the vectors c and w as in and Calculate y C by
b b
y i t t c t w
T
n
The desired p C is given by the rst n elements of y p y y y y
n n
mm
Algorithm Complex Symmetric QR Step Given J C this algorithm implements one step of
the QR method with the Wilkinson shift
Initialize Q I
Find the eigenvalue of J that is closer to J
m m m m mm
Set x J x J
for k m
T
Find a complex orthogonal matrix P to annihilate x using x
k
T
J P JP
k
k
Q QP
k
if k m
x J x J
k k k k
end if
end for
ERROR ANALYSIS FUNDAMENTALS
Let u denote the unit roundo Since the ob jective of this pap er is to analyze the stability of our algorithm and
not to derive precise error b ounds we carry out rst order error analysis Sometimes we ignore mo derate constant
co e cients The following results for the basic complex arithmetic op erations are from Higham
f l x y x y j j u
p
f l xy xy j j
p
f l x y x y j j
Using and similar to the real case rep eatedly applying we can show that for two n vectors x and y
T T T
f l x y x y y x x y
where
n u
j y j jy j j xj jxj
n n n
n u
We assume that nu So we can think of as approximately nu n
LANCZOS TRIDIAGONALIZATION
Analogous to the analysis of the conventional Lanczos tridiagonalization we shall show that column vectors q can
j
lose c orthogonality Let
T T
b b b b
b f l q v q v v
j j j
j j
b
for some v It follows immediately from that k v k kv k Thus
n j
T
b b b b
b q v j j kq k kv k
j j n j j
j
and
b b
j b j kq k kv k
j n j j
If
b b b b b
q v v b q q r f l v b
j j j j j j j
for some v and q then and show that
p
b b b
q k kq k k v k ukv b k qk
j j j j
b
Denoting r v b q as the rst order error in r and using we get
j j
p p
b b b b b
q k j b j k rk ukv b kq k u u kq k kv k
j j j j j j j
Now from we have
p
b b
b b b b
q f l r r r k rk kr k
j j j j j j
that is
p
b
b b b
q r r k rk kr k
j j j j
T
b
To check c orthogonality we multiply q on the b oth sides of the ab ove equation
j
T T T T T
b
b b b b b b b b b
q r r q q q r r q q q v b
j j j j j j
j j j j j
T
b b b
v b from we get Since q is normalized and q
j j j
j
T T
b
b b b
q q q r r
j j
j j
Next we derive the b ound for k rk by using and
p p
b b b b
q k kv b kq k kv k k rk
j j j j j
It follows from and that
k r r k k rk k rk
p p
b b b b
u kq k u kv k kq k kv k
j j j j
p p p
b b
u kq k u kv k
j j
Finally and imply that
T
b
b b b
q q j j j k r r k kq k j
j j j
j
p p p
b b b
kv k kq k u u kq k
j j j n
Using the approximation nu when nu and ignoring the mo derate constant co e cients we obtain
n
b b b
n kq k kq k kv k
j j j
T
b b
u jq q j
j
j
b
j j
j
b
b b b
The ab ove inequality says that q can lose c orthogonality when either j j is small or kq k is large recall that q
j j j j
b
and denote computed results Hence the loss is due to either the cancellation in computing or the growth
j j
b b
in kq k and not the accumulation of roundo errors In the real case kq k and is consistent with the
j j
b
results in Paige whereas in the complex case the norm kq k can b e arbitrarily large This is an additional factor
j
that causes the loss of c orthogonality
HANKEL MATRIX VECTOR MULTIPLICATION
The only computation in the fast Hankel matrix vector multiplication is
b b
y i t t c t w
Denoting the computed y as
b b b
y f l i t t c t w
j
b
we consider the computation of t w Assume that the weights in the FFT are precalculated and that the
k
j
computed weights b satisfy
k
j j
b
k j
k k
where
j j
k j
for all j and k Dep ending on the metho d that computes the weights we can pick
cu cu log j or cuj
where c denotes a constant dep endent on the metho d Following Higham we let
b b b
x t w and x f l t w
We have
p
log n
b
n kxk kx xk
log n
where
Approximately
p
n log n
b
kxk kx xk
log n
Similarly let
b b b
v t c and v f l t c
Then
p
n log n
b
kv k kv v k
log n
b b
Next we consider the comp onentwise multiplication z v x Denoting the computed z as
b b b
z f l v x
we have approximately
b
kz z k u kzk
Finally we consider
b b b
y i t z and y f l i t z
Since F is the FFT matrix we have
n
F n F
n
n
Thus similar to and we get
p
n n log
b
ky k ky y k
log n
As n F is unitary it follows that
n
p
p
b b b b
zk n kzk ky k ki t z k kF n kzk n
n
Consequently using and we get
log n log n
b b
ky y k kzk u kzk
log n log n
Ignoring the second and higer order terms of u we have
log n
b b b
ky y k kv xk
log n
b b b b
Now it remains to express kv xk in terms of kck and kw k Using this inequality
ka bk max ja j kbk kak kbk
i
i
and ignoring the second and higher order terms of u we obtain
b b b b b
kv x k kv xk k v v xk kv x x k
b b b
kv k kxk kv v k kxk kv k kx x k
b b
kv k kxk kv v k kxk kv k kx x k
It follows from and that
p
log n
b b
kv k kxk kv xk n
log n
Note that
p
b b
ck kv k k t c k n k
and
p
b
kxk n kw k
Using and and ignoring the second and higher order terms we obtain
n log n
b b b
ky y k kck kw k
log n
What do es the ab ove error b ound tell us ab out the accuracy of the fast multiplication For simplicity we assume
b
that the entries in the Hankel matrix H are ab out the same size Recall that c consists of the rst column and the
last row of H So
p
b
n kH k kck
F
and we conclude that
p
log n n
p
b b
ky y k kH k kw k
F
log n
n
For conventional matrix vector multiplication H w we have
kH w f l H w k kH k kw k
n
Since nu and is of size u our fast multiplication is more accurate than conventional multiplication by roughly
n
p
a factor of log n n
In summary the fast Hankel matrix vector multiplication scheme is stable This is consistent with the error
b ound for the FFT which says that the FFT has an error b ound smaller than that for conventional multiplication
by the same factor as the reduction in complexity of the metho d
CONDITION OF COMPLEX ORTHOGONAL TRANSFORMATION
In this section we derive the condition number of G of and present a backward error analysis of an application
of G First consider cond G We have
t iv
H
G G
iv t
where
t jcj jsj and v Im cs
H
It can b e veri ed that the two eigenvalues of G G are t v and t v and that
s
t jv j
cond G
t jv j
T
Supp ose that G is constructed to annihilate the second entry of x x x Then in
c x and s x
where
T
jx xj
Consequently in
i
H H
x t x x and v x
i
The two eigenvalues t v are therefore
i i
H H
x x and x x
i i
T
The condition number of G can b e arbitrarily large for some complex x For example when x i the two
eigenvalues are
j j j j
and
j j j j
This also shows that kGk can b e arbitrarily large For any real x however t v and cond G
kGk
T
Now we consider the application of G to y y y Let
cy sy
b
z f l Gy f l
sy cy
From and we get
cy sy
b
z
sy cy
p
for i and j j u for i Using rst order error analysis we derive where j j
i i
c s
b
z G G y where G
s s
and
p
u jGj j Gj
This shows that the application of G is p erfectly stable in the real case but can b e unstable in the complex case
b ecause jGj can b e large for some complex x −9 10
−10 10
−11 10 ERRORS −12 10
−13 10
−14 10 0 10 20 30 40 50 60 70 80 90 100
TESTS
Figure Errors in the eigenvalues of one hundred random Hankel matrices
NUMERICAL EXPERIMENTS
We implemented the fast eigenvalue algorithm for Hankel matrices in MATLAB and tested the following examples
on a SUN Ultra station
Example We generated one hundred random complex Hankel matrices Each matrix was constructed
by choosing two random complex vectors as its rst column and as its last row Both the real and imaginary parts
of each comp onent of the random vectors were uniformally distributed over For each matrix we measured
b
errors by assuming that the eigenvalues computed by the MATLAB function eig are exact Let and b e the
eigenvalues computed by our program and by MATLAB resp ectively Figure plots the square ro ots of the sum of
squares of the relative errors
X
b
j j
i i
E
eig
j j
i
i
We observe that the relative accuracy is generally b etter than
Example A rank de cient complex Hankel matrix H was generated using the Vandermonde decomp osition
n
z z
a
n
B B C C C B
z z z
a
z z
k
B B C C C B
H
B B C C C B
A A A
n n n
n
a
z z z
z z
k
k
k
k
The n n matrix H has rank k In this example we chose n k z as random complex numbers with
i
mo dulus one and a as random real numbers uniformly distributed on
i
i
B C B C
i
B C B C
B C B C
i
B C B C
z a
B C B C
i
B C B C
A A
i
i
Using we generated the rst column and the last row of a complex Hankel matrix Then we slightly p erturb ed
these two vectors by adding two small random noise vectors of size with comp onents normally distributed with
zero mean We obtained the following two vectors as the rst column and the last row of our test Hankel matrix
T
i i
C C B B
i i
C C B B
C C B B
i i
C C B B
C C B B
i i
C C B B
C C B B
i i
C C B B
H H
n
C C B B
i i
C C B B
C C B B
i i
C C B B
C C B B
i i
C C B B
A A
i i
i i
In the sixth and seventh Lanczos iterations we got small sub diagonal elements
i and i
The complex orthogonality of the matrix Q computed by the Lanczos metho d was lost
T
kQ Q I k
F
Our fast algorithm computed the eigenvalues
i
i
i
i
i
i
i
i
i
i
The last two eigenvalues are spurious For comparison MATLAB eig gave
i
i
i
i
i
i
i
i
i
i
The errors in the rst six eigenvalues were in the magnitude of So if we know the rank or an estimate k we
can stop the Lanczos pro cess after k iterations This leads to an algorithm for computing the k dominant eigenvalues
of a complex Hankel matrix In this example we quit the Lanczos pro cedure after six iterations and computed the
eigenvalues of the by triadiagonal matrix All six computed eigenvalues were correct to at least four digits The
three largest eigenvalues were correct to at least nine digits −11 10
−12 10 ERRORS
−13 10
−14 10 0 10 20 30 40 50 60 70 80 90 100
TESTS
Figure Errors in the eigenvalues of one hundred random Hermitian Toeplitz matrices
Example Our fast algorithm for complex Hankel matrices can b e adapted for Hermitian Toeplitz matrices by
incorp orating the following simple changes
Replace the fast matrix vector multiplication algorithm for Hankel matrices with one for Toeplitz matrices
Replace the mo di ed Lanczos metho d with a conventional one for Hermitian matrices
Replace the mo di ed QR metho d with a conventional one for Hermitian matrices
We generated one hundred element random complex vectors as the rst columns of the Hermitian Toeplitz
matrices Figure shows the error parameter E as de ned in Not surprisingly the errors here are smaller
eig
than those in Example sp eci cally the accuracy is b etter than
REFERENCES
G H Golub and C F Van Loan Matrix Computations rd Ed The Johns Hopkins University Press Baltimore
MD USA
N J Higham Accuracy and Stability of Numerical Algorithms So ciety for Industrial and Applied Mathematics
Philadelphia PA USA
F T Luk and S Qiao Using Complex Orthogonal Transformations to Diagonalize a Complex Symmetric Matrix
In Advanced Signal Processing Algorithms Architectures and Implementations VII Franklin T Luk Editor
Pro ceedings of SPIE Vol pp
F T Luk and S Qiao A Fast Eigenvalue Algorithm for Hankel Matrices In Advanced Signal Processing Al
gorithms Architectures and Implementations VIII Franklin T Luk Editor Pro ceedings of SPIE Vol
pp
C C Paige Error Analysis of the Lanczos Algorithm for Tridiagonalizing a Symmetric Matrix J Inst Maths
Applics Vol pp
C F Van Loan Computational Frameworks for the Fast Fourier Transform So ciety for Industrial and Applied
Mathematics Philadelphia PA USA