Mathematical Background for Wavelet Estimators of Cross-Covariance and Cross-Correlation

Brandon Whitcher Peter Guttorp Donald B. Percival

NRCSE T e c h n i c a l R e p o r t S e r i e s

NRCSE-TRS No. 038

The NRCSE was established in 1996 through a cooperative agreement with the United States Environmental Protection Agency which provides the Center's primary funding.

Mathematical Background for Wavelet Estimators of

Cross-Covariance and Cross-Correlation

Brandon Whitcher

EURANDOM, P.O. Box 513, 5600 MB Eindhoven, The Netherlands

Peter Guttorp

University of Washington, Department of Statistics, Box 354322, Seattle, WA 98195-2933

Donald B. Percival

University of Washington, Applied Physics Lab oratory,Box 355640, Seattle, WA 98195-5640

MathSoft, Inc., 1700 Westlake Avenue North, Seattle, Washington 98109-9891

June 8, 1999

Abstract

In this technical rep ort we provide mathematical supp ort to the claims in Whitcher, Guttorp,

and Percival 1999. First, the decomp osition of covariance by the wavelet covariance is rmly

established. Central limit theorems for MODWT estimators of the wavelet covariance and

correlation are then provided along with the de nition of the variance for estimators of wavelet

covariance under the assumption of Gaussianity. 1

1 Wavelet-Based Estimators of Covariance and Correlation

Here we de ne the basic quantities of interest for estimating asso ciation between two time series

using the MODWT. The decomp osition of covariance on a scale by scale basis of the wavelet covari-

ance is shown, and central limit theorems are provided for the wavelet covariance and correlation.

1.1 De nition and Prop erties of the Wavelet Cross-Covariance

Let fU g f::: ;U ;U ;U ;::: g be a sto chastic pro cess whose dth order backward di erence

t 1 0 1

1 B U = Z is a stationary Gaussian pro cess with zero mean and sp ectral density function

t t

S , where d is a non-negativeinteger. Let

L 1

~ ~

W = h U h U ; t = ::: ;1; 0; 1;::: ;

j;l j;l tl

j;t

l =0

b e the sto chastic pro cess obtained by ltering fU g with the MODWT wavelet lter fh g. Percival

j;l

W g is a stationary pro cess with zero and Walden 2000, Sec. 8.2 showed that if L 2d, then f

j;t

mean and sp ectrum given by S .

j;U

Let fX g f::: ;X ;X ;X ;::: g and fY g f::: ;Y ;Y ;Y ;::: g be sto chastic pro cesses

t 1 0 1 t 1 0 1

whose d th and d th order backward di erences are stationary Gaussian pro cesses as de ned

X Y

ab ove, and de ne d maxfd ;d g. Let S denote their cross sp ectrum and, S and S

X Y XY X Y

j 1

denote their autosp ectra, resp ectively. The wavelet cross-covariance of fX ;Y g for scale =2

t t j

and lag is de ned to b e

n o

X Y

Cov W W ; 1 ;

;XY j

j;t j;t+

X Y

where f W g and fW g are the scale MODWT co ecients for fX g and fY g, resp ec-

j t t

j;t j;t

tively. The MODWT co ecients have mean zero, when L 2d , and therefore =

;XY j

X Y

W W g. When = 0 we obtain the wavelet covariance between fX ;Y g, which we E f

t t

j;t j;t+

denote as = to simplify notation.

XY j 0;X Y j

Setting = 0 and Y to X or X to Y , Equation 1 reduces to the wavelet variance for X

t t t t t

2 2

or Y denoted as, resp ectively, or Percival 1995. The wavelet variance decomp oses

t j j

X Y

the pro cess variance on a scale by scale basis, and the wavelet cross-covariance give a similar

decomp osition for the pro cess cross-covariance.

Theorem 1 Let fX g and fY g be two weakly stationary processes with autospectra given by S f

t t X

and S f ,respectively. If we require L>2d, then for any integer J 1 we have

n o

X Y

C Cov fX ;Y g =Cov V ; V + ;

;XY t t+ ;XY j

J;t J;t+

j =1 1

X Y

where V g~ X and V g~ Y are obtained by ltering fX g and fY g using the

t t t t

J;l J;l

J;t J;t

MODWT scaling lter fg~ g, respectively. As J !1,we have

J;l

C = ;

;XY ;XY j

j =1

which gives the required decomposition.

Pro of of Theorem 1 Before proving Theorem 1, we require the following lemma.

o n

X Y

< for J >J . V ; V Cov Lemma 1 For al l >0, there exists a J such that

J;t J;t

2 J=2

Pro of of Lemma 1 For the orthonormal DWT g =1 and by de nition g~ = g =2 .

J;l J;l

J;l

2 J

Therefore wehave g~ =1=2 . Parseval's relation tells us that

J;l

Z Z

L 1

1=2 1=2

e e

G f df = G f df = g~ = :

J J

J;l

1=2 1=2

l =0

We know the amplitude sp ectrum A f jS f j is a non-negative real valued function.

XY XY

Hence, if A is b ounded by some nite number C , then for J >J ,

Z Z

o n

1=2 1=2

X Y

e e

V ; V G f jS f j df = C G f df = <: Cov

J XY J

J;t J;t

1=2 1=2

If A cannot be b ounded by any nite number C , there at least exists a constant C such

that A f df <=2, using a Leb esgue integral. A rough b ound on the squared gain

A f C

function of the scaling lter for Daub echies wavelets is G f 1, so for all J >J ,

Z Z

1=2

e e

G f jS f j df G f S f df

J XY J XY

A f C 1=2

+ G f jS f j df

J XY

A f

Z Z

A f df + C G f df

XY J

A f C A f

XY XY

1=2

+ C G f df + <:

2 2 2

1=2

Y X

W g with resp ect to fW g to get Without loss of generality, we set = 0 and simply shift f

j;t j;t

X Y

6=0. Because fW g and fW g are obtained by ltering the pro cesses fX g and fY g with a

t t

j;t j;t

Daub echies compactly supp orted wavelet lter of even length L>2d, resp ectively, we know that

X Y

f W g and fW g are stationary pro cesses with autosp ectra de ned by S f H f S f

j;X j X

j;t j;t

j 2

j 1 l

e e e e

and S f H f S f where H f H 2 f G 2 f is the squared gain function for

j;Y j Y j

l =0 2

fh g. Note, the squared gain functions asso ciated with unit scale for the wavelet and scaling lters

2 2

e e e e

are given by H f jH f j and G f jGf j .

X Y

The covariance between f W g and fW g is given by

j;t j;t

1=2

H f S f df: =

j XY XY j

1=2

This is a straightforward generalization of the univariate case; see Whitcher 1998 for more details.

X Y

The covariance between f V g and fV g is given by

j;t j;t

o n

1=2

X Y

V ; V = G f S f df; Cov

J XY

J;t J;t

1=2

J 1

e e

where G f G 2 f is the squared gain function for fg~ g. Because of the following identity

J J

l =0

e e

for squared gain functions H f +G f = 1 for all f Percival and Walden 2000, Sec. 4.3, wehave

h i n o

1=2

X Y

e e

V ; V Cov fX ;Y g = G f +H f S f df =Cov + ;

t t XY XY 1

1;t 1;t

1=2

and the case when J = 1 holds. Wenow pro ceed to prove the main assertion by induction. Assume

the prop erty holds for J 1; i.e.,

J 1

n o

X Y

Cov fX ;Y g =Cov ; V V + :

t t XY j

J 1;t J 1;t

j =1

So wehave

J 2

o n

1=2

Y X

G 2 f S f df = ;V Cov V

J 1;t J 1;t

1=2

l =0

J 2

i h

1=2

l J 1 J 1

e e e

G 2 f S f df G 2 f +H 2 f =

1=2

l =0

h i

1=2

e e

= G f +H f S f df

J J XY

1=2

n o

X Y

= Cov V ; V + :

XY J

J;t J;t

The decomp osition of covariance b etween fX ;Y g has now b een established for a nite number of

t t

scales. From Lemma 1, as J !1 the remaining covariance between the scaling co ecients go es

to zero. Hence, the theorem is established.

J +1 J +1

If we think of fg~ g asalow-pass lter covering the nominal frequency band [2 ; 2 ],

this statementisintuitively plausible since the scaling lter fg~ g is capturing smaller and smaller

p ortions of the cross sp ectrum as J !1. 3

1.2 Estimating the Wavelet Cross-Covariance

Supp ose X ;::: ;X and Y ;::: ;Y can be regarded as realizations of p ortions of the pro-

0 N 1 0 N 1

cesses fX g and fY g, whose d th and d th order backward di erences form stationary Gaussian

t t X Y

pro cesses. As b efore, let d = maxfd ;d g.

X Y

f f

Let W = W for those indices t where W is una ected by the b oundary { this is true as

j;t j;t j;t

long as t L 1. Thus, if N L , we can de ne a biased estimator ~ of the wavelet

j j ;XY j

cross-covariance based up on the MODWT via

X Y

1 N 1

e f f e

N W W ; =0;::: ;N 1;

l =L 1 j;l j;l +

X Y

1 N 1

e f f e

;XY j

N W W ; = 1;::: ;N 1;

l =L 1 j;l j;l +

0; otherwise;

e e

where N N L +1. The bias is due to the denominator 1=N remaining constant for all lags,

j j j

though it disapp ears at lag = 0. Let us now state the large sample prop erties of the wavelet

covariance estimator; i.e., ~ ~ . Wemay generalize to the wavelet cross-covariance

XY j 0;X Y j

Y X

by simply replacing shifting f W g with resp ect to fW g and app ealing to the same result.

j;t j;t

X Y

W ; W g is a bivariate Gaussian stationary process with Theorem 2 If L>2d, and suppose f

j;t j;t

R R

1=2 1=2

2 2

f < 1, then f < 1 and S autospectra satisfying S

j;Y j;X

1=2 1=2

~ N ; N S 0 ;

XY j XY j

j;XY

i.e., the MODWT-based estimator of wavelet covariance is normal ly distributed with mean

XY j

and large sample variance N S 0. The quantity S 0 is the spectral density function

j;XY j;XY

X Y

for f g the product of the scale MODWT coecients at zero frequency. W W

j;t j;t

X Y

W W g and f g Pro of of Theorem 2 With L>2d, b oth series of MODWT co ecients f

j;t j;t

have zero mean. Square integrability of the autosp ectra implies that fs g ! S and

j; ;X j;X

fs g ! S ; i.e., the auto covariance sequences and autosp ectra are Fourier transform

j; ;Y j;Y

pairs. Because L>2d, the squared gain function for Daub echies wavelet lters guarantees wehave

s . A similar statement holds for fW g and, therefore, fs g and S 0 = 0 =

j; ;X j; ;X j;X

j;t

fs g are absolutely summable.

j; ;Y

Let S f H f S f denote the MODWT ltered cross sp ectrum. From the magnitude

j;X Y j XY

squared coherence b eing b ounded by unity, and using the Cauchy{Schwarz inequality,we know that

Z Z

1=2 1=2

S f S f df jS f j df

j;X j;Y j;X Y

1=2 1=2

1=2

Z Z

1=2 1=2

2 2

< 1: f df S f df S

j;Y j;X

1=2 1=2 4

So the cross-covariance sequence and cross sp ectrum asso ciated with scale are also a Fourier pair

and, again, by using a Daub echies wavelet lters with L>2d, we have S 0=0. Therefore,

j;X Y

X Y

the cross-covariance sequence for f W ; W g is absolutely summable.

j;t j;t

We rst note that the MODWT estimate of the wavelet covariance ~ is essentially a

XY j

XY X Y

sample mean for the time series W W W cf. Equation 1. This pro cess also has

j;t j;t j;t

an absolutely summable cumulant sequence by Theorem 2.9.1 of Brillinger 1981, p. 38. Then

Theorem 4.4.1 of Brillinger 1981, p. 94 tells us that ~ is asymptotically normal with mean

XY j

and large sample variance given by N S 0, where S 0 is the sp ectral density

XY j

j;XY j;XY

for W evaluated at f =0.

j;t

Since we are exclusively interested in Gaussian pro cesses, S 0 may b e re-expressed as a

j;XY

X Y

function of the auto and cross sp ectra of the wavelet co ecients fW g and fW g. The variance

j;l j;l

of the estimated MODWT wavelet covariance at scale can b e computed directly via

N 1 N 1

n o

X X

X Y X Y

f f f f

Cov W W ; W W Var f ~ g =

XY j

j;m j;m

j;l j;l

l =L 1 m=L 1

j j

N 1

n o

j j 1

X Y X Y

f f f f

Cov W W 1 ; W W =

j;l j;l j;l + j;l +

e e

N N

j j

=N 1

N 1

j j 1

s ; 2

j; ;X Y

e e

N N

j j

=N 1

where s is the auto covariance sequence for the pro duct of the scale MODWT co ecients

j; ;X Y j

with resp ect to fX g and fY g.

t t

Using the Isserlis theorem and prop erties of the Fourier transform, the sp ectrum of fZ g at

R R

1=2 1=2

S S f S f df + f df Whitcher 1998. Since we have the f = 0 is S 0 =

U V Z

1=2 1=2

Fourier relationship fs g ! S , we necessarily have S 0 = s , when f = 0.

;Z Z Z ;Z

Re-examining Equation 2 and utilizing Ces aro summability Titchmarsh 1939, p. 411, we can

say

N 1

j j

N Var f ~ g = lim lim 1 s

j XY j j; ;X Y

e e

N !1 N !1

j j

=N 1

= s = S 0;

j; ;X Y

j;XY

where

Z Z

1=2 1=2

S 0 = S f S f df + S f df

j;X j;Y

j;XY

j;X Y

1=2 1=2 5

1.3 Wavelet Cross-Correlation

We can de ne the wavelet cross-correlation for scale and lag as

n o

X Y

Cov W W ;

j;t j;t+

;XY j

: =

;XY j

n o n o

1=2

X Y

X j Y j

Var W Var W

j;t j;t+

Since this is just a correlation co ecientbetween two random variables, 1 1 for all

;XY j

; j. The wavelet cross-correlation is roughly analogous to its Fourier counterpart { the magnitude

squared coherence { but it is related to bands of frequencies scales. Just as the cross-correlation

is used to determine lead/lag relationships b etween two pro cesses, the wavelet cross-correlation will

provide a lead/lag relationship on a scale by scale basis.

1.4 Estimating the Wavelet Cross-Correlation

Since the wavelet cross-correlation is simply made up of the wavelet cross-covariance for fX ;Y g

t t

and wavelet variances for fX g and fY g, the MODWT estimator of the wavelet cross-correlation

t t

is simply

;XY j

; 3 ~

;XY j

~ ~

X j Y j

2 2

where ~ is the wavelet covariance, and ~ and ~ are the wavelet variances. When

;XY j j j

X Y

= 0 we obtain the MODWT estimator of the wavelet correlation between fX ;Y g, denoted as

t t

~ for simplicity.

XY j

Large sample theory for the cross-correlation is more dicult to come by than for the cross-

covariance. Brillinger 1979 constructed approximate con dence intervals for the auto and cross-

correlation sequences of bivariate stationary time series. We use his technique to establish a central

limit theorem for the MODWT estimated wavelet cross-correlation. To simplify notation the fol-

lowing gives a central limit theorem for the wavelet correlation = 0 but easily generalizes to

arbitrary lags.

X Y

Theorem 3 Let L > 2d, and suppose f W W ; g is a bivariate Gaussian stationary process

j;t j;t

with square integrable autospectra, then

~ N ; N R ;

XY j XY j j

i.e., the MODWT-based estimator ~ of the wavelet correlation is asymptotical ly normal ly

XY j 6

distributed with mean and large sample variance given by

XY j

N 1

R Var f~ g f +

j XY j ;X j ;Y j ;XY j ;Y X j

=N 1

2 [ + ]

0;X Y j ;X j ;Y X j ;Y j ;Y X j

2 2 2 2

1 1

+ [ + + ] g;

j j j j

0;X Y ;X ;XY ;Y

2 2

X X

where E f g=[2 ] is the scale wavelet autocorrelation for the W W

;X j j j j

j;t

j;t+j j

process fX g.

g and Pro of of Theorem 2 Since L>2d,wehave that b oth sets of wavelet co ecients fW

j;t

Y X Y X Y

2 2

f f f f f

fW g have mean zero. Let us de ne A [W ] , B [W ] , and C W W , and

j;t j;t j;t

j;t j;t j;t j;t j;t

subsequently de ne their sample means

N 1

A =~ ; A

j;t j j

t=L 1

N 1

B B =~ ; and

j j;t j

t=L 1

N 1

C C =~ :

j j;t XY j

t=L 1

The vector-valued pro cess fA ;B ;C g has an absolutely summable joint cumulant sequence by

j;t j;t j;t

Theorem 2.9.1 of Brillinger 1981, p. 38. Hence, from Theorem 4.4.1 of Brillinger 1981, p. 94

the vector of sample means f A ; B ; C g are asymptotically normally distributed with mean vector

j j j

2 2

f ; ; g, and large sample variance given by N S 0, where S is

j j XY j j;AB C j;AB C

X Y j

the 3 3 sp ectral matrix for fA ;B ;C g.

j;t j;t j;t

The MODWT estimator of the wavelet correlation ~ is essentially a function of these

XY j

sample means g A ; B ; C , where g x; y ; z z= xy . App ealing to Mann and Wald 1943, we

j j j

have that ~ is asymptotically normally distributed with mean and large sample

XY j XY j

variance

2 2 2 2

N g_ ; ; S 0g _ ; ; 4

j j XY j j;AB C j j XY j

X Y X Y

where g_ ; ; is the gradientofg ; ; . Now let us re-express Equation 4 into the desired result

using the fact that we are only interested in Gaussian pro cesses. Because we are evaluating S

j;AB C

at f = 0, it is in fact a symmetric matrix of the form

2 3

S 0 S 0 S 0

j;AA j;AB j;AC

6 7

S 0 S 0 S 0 S 0 = ;

4 5

j;AB j;B B j;B C j;AB C

S 0 S 0 S 0

j;AC j;B C j;C C 7

where the elements of the matrix are

Z Z

1=2 1=2

2 2

S 0 = 2 S f df; S 0 = 2 S f df;

j;AA j;B B

j;X j;Y

1=2 1=2

Z Z

1=2 1=2

S f df; S f S f df + S 0 =

j;X j;Y j;C C

j;X Y

1=2 1=2

1=2

S 0 = 2 S f S f df;

j;AB j;X Y j;Y X

1=2

S 0 = 2 S f S f df; and

j;AC j;X j;Y X

1=2

S 0 = 2 S f S f df:

j;B C j;Y j;Y X

1=2

The gradient is explicitly given by

2 2

g_ ; ; =

j j XY j

X Y

2 3

XY j XY j

4 5

q q q

;

2 2 2 2 2 2 2 2

2 2

j j j j j j j j

X X Y Y X Y X Y

and, through matrix multiplication and application of Parseval's relation to each auto and cross

sp ectra in S 0, wemay express Equation 4 as

j;AB C

N 1

2 2

j j

XY XY

2s 2C C +

j; ;X Y j; ;Y X

j; ;X

2 4 6 4

4 2

j j j j

Y Y X X

=N 1

2 2

+ 2s s s + C +

j; ;X j; ;Y

j; ;Y j; ;X Y

2 2 6 2

j j j j

X X Y Y

XY j XY j

: 2s C 2s C

j; ;X j; ;Y X j; ;Y j; ;Y X

4 2 2 4

j j j j

X Y X Y

Each of the auto covariance terms are equivalent to the wavelet auto covariance for scale de ned

by letting X = Y in Equation 1 and each cross-covariance term is equivalent to the wavelet

t t

cross-covariance for scale . This yields the desired result.

References

Brillinger, D. R. 1979. Con dence intervals for the crosscovariance function. In Mathematical

Statistics,Volume 5 of Selecta Statistica Canadiana, pp. 1{16. Hamilton, Ontario: McMaster

University Printing Services.

Brillinger, D. R. 1981. Time Series: Data Analysis and Theory. Holden-Day Series in Time

Series Analysis. San Francisco: Holden-Day. Expanded edition. 8

Mann, H. B. and A. Wald 1943. On sto chastic limit and order relationships. The Annals of

Mathematical Statistics 14, 217{226.

Percival, D. B. 1995. On estimation of the wavelet variance. Biometrika 82 3, 619{631.

Percival, D. B. and A. T. Walden 2000. Wavelet Methods for Time Series Analysis. Cambridge:

Cambridge University Press. Forthcoming.

Titchmarsh, E. C. 1939. The Theory of Functions 2 ed.. Oxford: Oxford University Press.

Whitcher, B. 1998. Assessing Nonstationary Time Series Using Wavelets. Ph. D. thesis, Uni-

versityofWashington.

Whitcher, B., P. Guttorp, and D. B. Percival 1999. Wavelet analysis of covariance with ap-

plication to atmospheric time series. Submitted to the Journal of Geophysical Research {

Atmospheres. 9