Introduction to Time Series Analysis. Lecture 15. Spectral Analysis 1. Spectral density: Facts and examples. 2. Spectral distribution function. 3. Wold’s decomposition. 1 Spectral Analysis Idea: decompose a stationary time series {Xt} into a combination of sinusoids, with random (and uncorrelated) coefficients. Just as in Fourier analysis, where we decompose (deterministic) functions into combinations of sinusoids. This is referred to as ‘spectral analysis’ or analysis in the ‘frequency domain,’ in contrast to the time domain approach we have considered so far. The frequency domain approach considers regression on sinusoids; the time domain approach considers regression on past values of the time series. 2 A periodic time series Consider Xt = A sin(2πνt)+ B cos(2πνt) = C sin(2πνt + φ), where A, B are uncorrelated, mean zero, variance σ2 = 1, and C2 = A2 + B2, tan φ = B/A. Then µt = E[Xt] = 0 γ(t, t + h) = cos(2πνh). So {Xt} is stationary. 3 An aside: Some trigonometric identities sin θ tan θ = , cos θ sin2 θ + cos2 θ = 1, sin(a + b) = sin a cos b + cos a sin b, cos(a + b) = cos a cos b − sin a sin b. 4 A periodic time series For Xt = A sin(2πνt)+ B cos(2πνt), with uncorrelated A, B (mean 0, variance σ2), γ(h)= σ2 cos(2πνh). The autocovariance of the sum of two uncorrelated time series is the sum of their autocovariances. Thus, the autocovariance of a sum of random sinusoids is a sum of sinusoids with the corresponding frequencies: k Xt = (Aj sin(2πνj t)+ Bj cos(2πνj t)) , j=1 X k 2 γ(h)= σj cos(2πνj h), j=1 X 2 where Aj,Bj are uncorrelated, mean zero, and Var(Aj )= Var(Bj)= σj . 5 A periodic time series k k 2 Xt = (Aj sin(2πνj t)+ Bj cos(2πνj t)) , γ(h)= σj cos(2πνj h). j=1 j=1 X X Thus, we can represent γ(h) using a Fourier series. The coefficients are the variances of the sinusoidal components. The spectral density is the continuous analog: the Fourier transform of γ. (The analogous spectral representation of a stationary process Xt involves a stochastic integral—a sum of discrete components at a finite number of frequencies is a special case. We won’t consider this representation in this course.) 6 Spectral density If a time series {Xt} has autocovariance γ satisfying ∞ h=−∞ |γ(h)| < ∞, then we define its spectral density as P ∞ f(ν)= γ(h)e−2πiνh h=X−∞ for −∞ <ν< ∞. 7 Spectral density: Some facts ∞ −2πiνh 1. We have h=−∞ γ(h)e < ∞. This is because |eiθ| = | cos θ + i sin θ| = (cos2 θ + sin2 θ)1/2 = 1, P and because of the absolute summability of γ. 2. f is periodic, with period 1. This is true since e−2πiνh is a periodic function of ν with period 1. Thus, we can restrict the domain of f to −1/2 ≤ ν ≤ 1/2. (The text does this.) 8 Spectral density: Some facts 3. f is even (that is, f(ν)= f(−ν)). To see this, write −1 ∞ f(ν)= γ(h)e−2πiνh + γ(0) + γ(h)e−2πiνh, h=X−∞ hX=1 −1 ∞ f(−ν)= γ(h)e−2πiν(−h) + γ(0) + γ(h)e−2πiν(−h), h=X−∞ hX=1 ∞ −1 = γ(−h)e−2πiνh + γ(0) + γ(−h)e−2πiνh hX=1 h=X−∞ = f(ν). 4. f(ν) ≥ 0. 9 Spectral density: Some facts 1/2 5. γ(h)= e2πiνhf(ν) dν. Z−1/2 1/2 1/2 ∞ e2πiνhf(ν) dν = e−2πiν(j−h)γ(j) dν −1/2 −1/2 j=−∞ Z Z X ∞ 1/2 = γ(j) e−2πiν(j−h) dν j=−∞ −1/2 X Z γ(j) = γ(h)+ eπi(j−h) − e−πi(j−h) 2πi(j − h) jX6=h γ(j) sin(π(j − h)) = γ(h)+ = γ(h). π(j − h) jX6=h 10 Example: White noise 2 For white noise {Wt}, we have seen that γ(0) = σw and γ(h) = 0 for h 6= 0. Thus, ∞ f(ν)= γ(h)e−2πiνh h=X−∞ 2 = γ(0) = σw. That is, the spectral density is constant across all frequencies: each frequency in the spectrum contributes equally to the variance. This is the origin of the name white noise: it is like white light, which is a uniform mixture of all frequencies in the visible spectrum. 11 Example: AR(1) 2 |h| 2 For Xt = φ1Xt−1 + Wt, we have seen that γ(h)= σwφ1 /(1 − φ1). Thus, ∞ 2 ∞ −2πiνh σw |h| −2πiνh f(ν)= γ(h)e = 2 φ1 e 1 − φ1 h=X−∞ h=X−∞ σ2 ∞ = w 1+ φh e−2πiνh + e2πiνh 1 − φ2 1 1 h=1 ! X σ2 φ e−2πiν φ e2πiν = w 1+ 1 + 1 1 − φ2 1 − φ e−2πiν 1 − φ e2πiν 1 1 1 2 −2πiν 2πiν σw 1 − φ1e φ1e = 2 −2πiν 2πiν (1 − φ1) (1 − φ1e )(1 − φ1e ) 2 σw = 2 . 1 − 2φ1 cos(2πν)+ φ1 12 Examples 2 White noise: {Wt}, γ(0) = σw and γ(h) = 0 for h 6= 0. 2 f(ν)= γ(0) = σw. 2 |h| 2 AR(1): Xt = φ1Xt−1 + Wt, γ(h)= σwφ1 /(1 − φ1). 2 σw f(ν)= 2 . 1−2φ1 cos(2πν)+φ1 If φ1 > 0 (positive autocorrelation), spectrum is dominated by low frequency components—smooth in the time domain. If φ1 < 0 (negative autocorrelation), spectrum is dominated by high frequency components—rough in the time domain. 13 Example: AR(1) Spectral density of AR(1): X = +0.9 X + W t t−1 t 100 90 80 70 60 ) ν 50 f( 40 30 20 10 0 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 ν 14 Example: AR(1) Spectral density of AR(1): X = −0.9 X + W t t−1 t 100 90 80 70 60 ) ν 50 f( 40 30 20 10 0 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 ν 15.