?

Vandermonde Factorization of a Hankel

1 2 2

Daniel L Boley Franklin T Luk David Vandevo orde

University of Minnesota Minneap olis MN USA

Rensselaer Polytechnic Institute Troy NY USA

Abstract We show that an arbitrary of a nite

T

admits a Vandermonde decomp osition H V DV where V is a con

uent and D is a blo ck This result

was rst derived by Vandevo orde our contribution here is a presentation

that uses only sp ecically the Jordan canonical form We

discuss the choices for computing this decomp osition in only O n op

erations and we illustrate how to employ the decomp osition as a fast

way to analyze a noisy signal

Intro duction

1

denote a complexvalued signal and let H represent the asso ciated Let fh g

k

k =1

innite Hankel matrix whose i j element is dened by H h

ij i+j 1

h h h h

1 2 3 4

h h h h

C B

2 3 4 5

C B

h h h h H

C B

3 4 5 6

A

h h h h

4 5 6 7

T

This matrix is symmetric not Hermitian if complex H H Throughout

T

this pap er the notation M denotes the transp ose of M and not the conjugate

transp ose Supp ose that the underlying signal is a sum of r exp onentials ie

for k

r

X

k

h d

k i

i

i=1

where the s are distinct complex numb ers Then the Hankel matrix H will

i

have rank r In this case the Hankel matrix admits the factorization

T

H V D V

where D is diagonal and V is Vandermonde

4

D diagd d d

1 2 r

This research was partially supp orted by NSF Grant CCR

and

2 3

1

1 1

2 3

B C

2

2 2

B C

2 3

4

B C

3

3 3

V

B C

A

2 3

r

r r

We stress that a diagonal decomp osition is p ossible only if the s are distinct

j

In this pap er we consider the general case where the s are multiple The

j

factorization must b e generalized so that the matrix D b ecomes blo ck diagonal

and the matrix V takes on a conuent Vandermonde structure The theory was

rst develop ed in Vandevo ordes PhD thesis The next section is devoted

to a derivation of this generalized Vandermonde decomp osition based entirely on

concepts from linear algebra In Section we sketch how the decomp osition can

2

b e computed quickly viz using O n op erations and O n space We conclude

in Section with an example illustrating to use the metho d to analyze a noisy

signal

Derivation via Jordan Canonical Form

Assume that the matrix H of has rank r By a theorem of Gantmacher

vol p the signal satises a recurrence relation of length r

h a h a h a h

k r 1 k 1 r 2 k 2 0 k r

which generates the entire signal once the r initial values fh h h g are

1 2 r

xed The recurrence is a dierence equation which can b e used to solve for

the a s after the next r values fh h h g b ecome known

i r +1 r +2 2r

Let C denote the corresp onding to the p olynomial

4

r r 1

p a a a

r 1 1 0

that is

B C

B C

B C

4

B C

C

B C

B C

B C

A

a a a a a

0 1 2 r 2 r 1

We show that the rst r rows of H can b e regarded as a Krylov sequence gen

erated by C Let

h

k

h

C B

k +1

4

C B

h

k

A

h

k +r 1

denote the rst r entries in the k th column of H The rst r rows of H can b e

written as

2

H h h h h C h C h

1:r1:1 1 2 3 1 1 1

Supp ose denote the ro ots of the p olynomial p with resp ective

1 2 s

multiplicities m m m so that

1 2 s

m m m r

1 2 s

Denote a Jordan canonical decomp osition of C by

1

C PJP

with J in the canonical form

J

1 m

1

J

C B

2 m

2

C B

J

A

J

s m

s

r r

where

i

B C

i

B C

B C

J

m

i

B C

A

i

i

m m

i i

for i s The companion matrix C is guaranteed to b e nonderogatory

ie it has one Jordan blo ck p er distinct eigenvalue The transformation P is not

unique but having xed the order for the eigenvalues any alternative transfor

e

mation P also yielding the same Jordan canonical form must b e related to P

by

e

PQ P

where

Q

m

1

Q

C B

m

2

C B

Q

A

Q

m

s

r r

as an m m nonsingular and upp er whose with each Q

i i m

i

e

diagonal entries are all the same This is b ecause the columns of any P are

Jordan chains and there are only limited kinds of transformations to the Jordan

chains that will yield the same Jordan form vol p Note that Q is

blo ck diagonal with blo cks conforming to the Jordan blo cks

For the purp oses of computation or for xing ideas it is often convenient to

cho ose a sp ecic P One such choice is that of a conuent Vandermonde matrix

p

T

p

T

p J

B C

4

B C

P

A

T r 1

p J

r r

where

[m ]T [m ]T [m ]T

T

1 2 s

p

e e e

1 1 1

1r

so p is partitioned conformally with the Jordan canonical form J and each

[m ]T

i

partition e represents a rst unitco ordinate vector

1

[m ]T

i

e

1

1m

i

It is easy to verify for any choice of a starting vector p that

T T

p J p J

T 2 T 2

p J p J

B C B C

B C B C

B C B C

CP PJ

B C B C

A A

T r 1 T r 1

p J p J

T r 1 T r

p a J a J a I p J

r 1 1 0

where we have used the the characteristic equation for J

r r 1

pJ J a J a J a I

r 1 1 0

1

Hence C PJP supp orting our choice of the sp ecial form of P in

Now dene a generalized Vandermonde matrix V by

4

2

V v J v J v

where

4

1

v P h

1

The relation can b e written as

H P V

1:r1:1

Let V denote the rst r columns of V We will express the Jordan decomp osition

r

T

of C in terms of V Forming the pro duct V C columnwise we get

r r

T r 2 r 1 T

V C v J v J v J v C

r

2 r 1 r 1

J v J v J v a a J a J v

0 1 r 1

2 r 1 r

J v J v J v J v

JV

r

Since the rst r columns of are indep endent by assumption the matrix V

r

must b e nonsingular So we obtain the decomp osition

T 1

JV C V

r r

or equivalently

T T T

C V J V

r r

We will use this result to express the transformation P in terms of V Dene

r

the blo ck diagonal ip matrix as follows

F

m

1

F

C B

m

2

4

C B

F

A

F

m

s

r r

where

B C

4

B C

F

B C

m

i

A

m m

i i

This for i s partitioned conformally with the Jordan blo ck J

i m

i

2 T

matrix is involutory F I and symmetric F F When applied to the

Jordan matrix J the matrix F has the eect of transp osing it

T

FJF J

We can thereby write the Jordan decomp osition of C as

T T T T T

C V J V V FJFV

r r r r

Setting

T

P V FQ

r

for some matrix Q of the form we get the decomp osition of the leading r r

part of the Hankel matrix as

T

H PV V F QV

r r r

r

Given that H is symmetric and V is nonsingular we obtain

r

T T T T

V F QV H H V FQ V

r r

r r

Hence the matrix D dened by

4

D F Q

must b e symmetric furthermore it is blo ck diagonal with blo cks conforming to

the Jordan blo cks Since each diagonal blo ck of Q is upp er triangular we derive

the following form for D

D

1

D

C B

2

C B

D

A

D

s

r r

where

d

i

d

B C

i

B C

D

B C

i

A

d

i

d

i

m m

i i

for i s each blo ck D is symmetric and upp er antitriangular with

i

a constant value along the main antidiagonal Combining these formulas we

obtain b oth a relation b etween P and V

r

T

P V D

r

and a symmetric decomp osition for the leading r r Hankel matrix

T 1 T

H PV V DV PD P

r r r

r

From we get

r 2r T r T 2r

V V J V J V V V C V C

r r r r r r

T

Since CH H C we obtain

r r

I

r

r

C B

C

T

T r T 2r

C B

H I C C V D V H

2r

r r

A C

ie a factorization for the entire innite Hankel matrix

A further analysis of the structure of the matrices in yields the fact that

1 1

the diagonal blo cks of D and D have Hankel structure For D we start

with the identity

1 1 T T 1 T T T T

PJP PD P CH HC PD P P J P

1 1 T 1

which simplies to JD D J The matrix D is blo ck diagonal with

blo cks conforming to those of J So the ith blo ck of this last relation is

1 1

T

D D J J

i i m

i

m

i i

i

for i s Subtracting I from b oth sides yields

i

1 1

T

Z D D Z

i

i

i i

where Z has the form of an upshift matrix of appropriate size

B C

B C

B C

I Z J

i i i m

i

B C

A

m m

i i

1

must have a Hankel structure For D we carry out a similar argument Hence D

i

T 1

using V F instead of P and D instead of D and apply the factorization

r

to get a Hankel structure for the diagonal blo cks D

i

We now show how we may mo dify the Vandermonde matrix V so that its

r

rst column consists of all ones Supp ose that no comp onent of v is zero Dene

a new diagonal matrix D by

v

4

D diag v v v

v 1 2 r

and a new generalized Vandermonde matrix W by

r

4

1

W D V

r r

v

Then

4

T

1

D v e

v

and we can rewrite as

T

H W D DD W

r v v r

r

where the pro duct D DD is a blo ck diagonal matrix with blo cks conforming

v v

to J The matrix W has the following structure

r

r 1 2 1 1 1

J D e J D e D JD e D W e D

v v v r

v v v

2 r 1

b b b

e J e J e J e

with

4

1

b

J D JD

v

v

b

We observe that the matrix J p ossesses a sort of generalized Jordan structure

We conclude this section with three notes

First consider the sp ecial case that all the eigenvalues of C are simple In

b

this case the Jordan matrix J J is diagonal F is the identity D Q is

scalar diagonal and

T

P V D

r

The decomp osition simplies to

T T 2

H V D V W DD W

r r r

r r v

where the parts enclosed in parentheses are diagonal matrices The last expres

sion on the right applies if v has no zero comp onent which is guaranteed in this

case by the nonsingularity of V In fact the matrix W of for this case has

r r

the usual Vandermonde structure The ab ove holds for any choice of P but if

T

we x P as given in we see that P W

r

Second the factorization can b e used to factor any nonsingular r r

Hankel matrix H This matrix is lled by the entries h h h hence

r 1 2 2r 1

to x the p olynomial and carry out the rest of the development ab ove we

must cho ose some value for h With such a choice the rest of the development

2r

ab ove go es through unchanged

Third we address the issue of factoring a singular n n Hankel matrix H

n

One way to do this is to emb ed this n n matrix inside an innite Hankel matrix

H of a nite rank r by extending it with innitely many zeros We could use

1

a dierent choice for the extension to avoid a nilp otent C and one op en issue is

how to cho ose the extension to minimize the resulting rank Factor the innite

Hankel matrix as

T

H V D V

1

where D is r r and V is r and extract the rst n rows and columns of

this decomp osition to get

T

H V DV

n n

n

where V is the r n matrix consisting of the rst n columns of V Note that it

n

could b e that r n or r n The former case o ccurs for example if the rank

of H is r n and the leading r r part of H happ ens to b e nonsingular

n n

Algorithms Generic case

We briey discuss choices of algorithms that can b e used to compute the Van

dermonde decomp osition Many of the individual pieces to the algorithms are

otheshelf metho ds some are quite exp erimental and some have received very

little attention in the literature The metho ds we present are based on the use

of the Lanczos algorithm or its derivatives Most details can b e found in

We b egin with an outline of the basic steps

Compute the mo des generating the Hankel matrix viz the ro ots of the

p olynomial p of

Compute the diagonal matrix D

T 1

D V HV

where V is the Vandermonde matrix generated by the eigenvalues in step

The diagonal structure of D follows from the theory develop ed in the previ

ous section

Optionally scale the columns of V to unit norm scaling the entries in D

appropriately

For each of these steps there are choices for the algorithm to use For step we

could solve for the co ecients in the recurrence by simply plugging the values

for h i n yielding a sp ecial set of n equations in n unknowns

i

originally prop osed by Prony and p opularized by Yule and Walker

Then we must nd all the ro ots of the p olynomial p

Vandevo orde prop osed an alternative for step We use a variant of a

nonsymmetric Lanczos pro cess to generate a T whose eigen

values match those of C Then we compute the eigenvalues of T It turns out that

b oth these steps generating T and computing its eigenvalues can b e p erformed

very eciently Space do es not p ermit a full description of the pro cess but we

can give a hint on the basic ideas used We have already seen that the n n

Hankel matrix H can b e thought of as a Krylov sequence generated by C and

n

h cf If we let H b e the n n Hankel matrix obtained by extending

1 2n

the signal fh g with all zeros then H can similarly b e thought of as the

k 2n

Krylov sequence generated by the n n upshift matrix Z of the form

The nonsymmetric Lanczos pro cess in is equivalent to a pro cedure that

biorthogonalizes the Krylov sequence we just mentioned against another se

quence called the left Krylov sequence If the co ecients computed by the

enforcement of the biorthogonalization conditions involve only the rst n en

tries of each vector the Lanczos co ecients generated by C and by H will

2n

b e identical This can happ en for example by making the left Krylov sequence

upp er triangular so that the right Krylov sequence will b e biorthogonalized

to lower triangular This can b e arranged by starting the left Krylov sequence

with the co ordinate unit vector e As a result each new vector is generated

1

by shifting up the previous vector and then orthogonalizing it against the two

previous vectors This takes linear time and linear space and there are at most

2

O n steps so that the total time is O n The resulting algorithm is describ ed

in detail in There is also a symmetric variant originally prop osed in that

can generate a symmetrized tridiagonal matrix directly from this nonsymmetric

recursion This symmetric variant has similar costs

Once the tridiagonal matrix has b een generated the task is to nd its eigen

values There are two variants of the QRtyp e algorithm that can b e applied

here One is the complex symmetric QR algorithm prop osed in for which

the matrix T must b e symmetrized Even when T is real if the signs of the

corresp onding sup erdiagonal and sub diagonal entries of T are opp osite then the

symmetrized matrix will b e complex The resulting QR algorithm is a direct

analog of the ordinary Hermitian QR metho d but it uses complex orthogonal

rotations and complex symmetric matrices instead of unitary rotations and Her

mitian matrices resp ectively Another option is to use the LR algorithm

which is based on the LU factorization without pivoting to preserve the tridi

agonal structure The LR algorithm can break down but if a random shift is

applied when zero pivot o ccurs during the LU factorization the pro cess can still

exhibit very rapid convergence If T is real an implicit doubleshift LR algorithm

can in principle b e carried out in real arithmetic Both algorithms require

linear time for each iteration in a manner very similar to the Hermitian analog

and the numb er of iterations is generally O n in a manner very similar to the

QR algorithm usually employed The relative merits b etween these alternative

algorithms have not b een studied in detail

The other ma jor task is nding the diagonal matrix D in step Because of

the structure of V the diagonal entries of D app ear in the rst column of the

T

pro duct DV But DV V H Hence this rst column is the solution d to the

Vandermonde system

T

V d h

1

where h is the rst column of H This can b e solved with a fast O n Van

1

dermonde solver where to maintain stability Higham p recommends

arranging the eigenvalues with a socalled Leja ordering

Analysis of a Signal

Consider a signal fh g which suers from the presence of noise How can we

k

recover the principal mo des that generate the signal A p opular metho d by

Kung based on the singular value decomp osition SVD is known to b e an

eective metho d for this purp ose but it suers from the need to carry out b oth

3

an SVD and a matrix eigensolution each costing O n op erations A second

p opular approach is to form the Hankel matrix generated by the signal and then

pro ceed to nd a nearby Hankel matrix of a lower rank The Vandermonde

decomp osition of this nearby lowrank Hankel matrix yields the parameters in

The metho d of iterates until it converges to a nearby Hankel matrix

Unfortunately this metho d requires the rep eated use of the SVD and hence

3

costs up to O n op erations p er iteration

We indicated in Section how the Vandermonde decomp osition can b e com

puted quickly An obvious way to obtain a nearby Hankel matrix of a lower rank

is to set to zero all the diagonal entries in D that are smaller than a certain

tolerance Although this crude metho d do es not always yield the b est approxi

mation a judicious combination of this approach with other criteria can yield a

go o d result We conclude this pap er with an illustration of one such approach

in the next paragraph

Start with a signal generated by ve mo des shown by circles on the complex

plane in Figure to which has b een added white noise with a signaltonoise

ratio of dB Form the Hankel matrix H and compute its Vander

T

monde decomp osition H V DV Figure shows the absolute values of the

diagonal entries of D in descending order It turns out that selecting the mo des

corresp onding to the ve largest values of D do es not yield satisfactory results

but we can almost recover the correct mo des by the following simple pro cedure

Cho ose the mo des corresp onding to the largest entries in D also called weights

sp ecically those that are within of the largest entry in absolute value in

this case fourteen mo des remained Then cho ose a subset of these fourteen using

a second criterion based on the Discrete Fourier Transform DFT of the signal

The DFT of the original signal is shown by the dotted line in Figure As most

of the mo des lie relatively close to the unit circle their argument angle on the

complex plane maps to the horizontal axis of Figure In fact we have marked

the angles corresp onding to the ve original unknown mo des by means of cir

cles along the xaxis This leads to our second criterion viz select those mo des

for which the DFT is larger than a certain threshold in this case of the

largest value in the DFT in absolute value This selection criterion is applied

only to those mo des that survived the rst selection pro cess In this example out

of the fourteen mo des only seven survived the second selection pro cess These

nal seven mo des are marked by *s in Figure and the resulting DFT using

these seven mo des is shown by the solid line in Figure We remark that one can

still distinguish the two close p eaks in this DFT corresp onding to the two very

close original mo des We should emphasize that the choice of criteria requires

further study Indeed a more sophisticated selection criterion is presented in

Computed Vandermonde Modes (eigenvalues) 1.5

1

0.5

0 RS0P02

−0.5

−1

−1.5 −1.5 −1 −0.5 0 0.5 1 1.5

o=true modes; *=selected modes; x=unselected modes (7 selected)

Fig Original mo des o and those computed from the tridiagonal matrix discussed

in in Section x

References

A Bjorck and V Pereyra Solution of Vandermonde system of equations Math

Comp

D L Boley T J Lee and F T Luk The Lanczos algorithm and Hankel matrix

factorization Lin Alg Appl

J Cadzow Signal enhancement a comp osite prop erty mapping algorithm IEEE

Trans Acoust Speech Sig Proc

J K Cullum and R A Willoughby A QL pro cedure for computing the eigenvalues

of complex symmetric tridiagonal matrices SIAM J Matrix Anal Appl

F R Gantmacher Theory of Matrices Chelsea New York

G H Golub and C F Van Loan Matrix Computations Johns Hopkins Univ

Press rd edition

N J Higham Accuracy and Stability of Numerical Algorithms SIAM Philadel

phia

S Y Kung A new identication and mo del reduction algorithm via singular value

decomp ositions In Proc th Asilomar Conf Circ Syst Comp pages

Novemb er diagonal weights in Vandermonde decomposition 2 10

1 10

0 10

−1 10

−2 10

−3 10 RS0P01

−4 10

−5 10

−6 10

−7 10

−8 10 0 20 40 60 80 100 120 140

14 wgts over 10% of max, 7 of those over 30% of max FFT

Fig Diagonal entries from Vandermonde decomp osition

J L Phillips The triangular decomp osition of Hankel matrices Math Comp

R Prony Essai experimental et analytique sur les lois de la dilatabilite et sur celles

de la force expansive de la vap eur de leau et de la vap eur de lalko ol a dierentes

temperatures J de lEcole Polytechnique

D Vandevo orde A Fast Exponential Decomposition Algorithm and its Applica

tions to Structured Matrices PhD thesis Computer Science Department Rensse

laer Polytechnic Institute

G Walker On p erio dicity in series of related terms Proc Roy Soc London Ser

A A

D Watkins and L Elsner Chasing algorithms for the eigenvalue problem SIAM

J Matr Anal

G Yule On a metho d of investigating p erio dicities in disturb ed series with sp ecial

reference to Wolfers sunsp ot numb ers Trans Roy Soc London Ser A A

A

This article was pro cessed using the L T X macro package with LLNCS style E FFTs: .. = original; − = Vandermonde: 7 modes selected 450

400

350

300

250 RS0P05 200

150

100

50

0 0 50 100 150 200 250 300 350

degrees around unit circle (o’s mark angles of original modes)

Fig Discrete Fourier Transform DFT of the original signal dotted and the re

constructed reducedorder signal solid Small circles mark the angles corresp onding

the original mo des