Spectral Density Periodogram Spectral Density (Chapter 12) Outline 1 Spectral Density 2 Periodogram Arthur Berg Spectral Density (Chapter 12) 2/ 19 Spectral Density Periodogram Spectral Density Periodogram Spectral Approximation Motivating Example Any stationary time series has the approximation Let’s consider the stationary time series q X xt = Uk cos(2π!k t) + Vk sin(2π!k t) xt = U cos(2π!0t) + V sin(2π!0t) k=1 where Uk and Vk are independent zero-mean random variables with where U; V are independent zero-mean random variables with equal 2 2 variances σk at distinct frequencies !k . variances σ . We shall assume !0 > 0, and from the Nyquist rate, we The autocovariance function of xt is (HW 4c Exercise) can further assume !0 < 1=2. q The “period” of xt is 1=!0, i.e. the process makes !0 cycles going from xt X 2 γ(h) = σk cos(2π!k h) to xt+1. k=1 From the formula q e−iα + e−iα In particular X 2 cos(α) = γ(0) = σk 2 |{z} k=1 we have var(xt ) | {z } sum of variances σ2 γ( ) 2 −2π!0h 2πi!0h Note: h is not absolutely summable,1 i.e. γ(h) = σ cos(2π!0h) = e + 2 X 2 jγ(h)j = 1 h=−∞ Arthur Berg Spectral Density (Chapter 12) 4/ 19 Arthur Berg Spectral Density (Chapter 12) 5/ 19 Spectral Density Periodogram Spectral Density Periodogram Riemann-Stieltjes Integration Integration w.r.t. a Step Function Riemann-Stieltjes integration enables one to integrate with respect to a general nondecreasing function. Usual Integration General Situation R g(x) dx R g(x) dF (x) P Given F(x) = αi 1(xi ≤ x), then Case of interest: F(x) is a step function, i.e. a function with a finite Z X number of jump discontinuities. Such an F(x) has the representation g(x) dF(x) = αi g(xi ) n X F(x) = αi 1(xi ≤ x) i=1 So F(x) has jumps at xi with heights αi . (Simplest step function is the Heaviside step function given by H(x) = 1(x > 0).) Arthur Berg Spectral Density (Chapter 12) 6/ 19 Arthur Berg Spectral Density (Chapter 12) 7/ 19 Spectral Density Periodogram Spectral Density Periodogram Back to our Motivating Example General Theorem We can now write the autocovariance function of xt = U cos(2π! t) + V sin(2π! t) 0 0 The autocovariance function for any stationary time series has the as representation Z 1=2 2πi!h σ2 γ(h) = e dF (!) γ(h) = e−2π!0h + 22πi!0h −1=2 2 Z 1=2 for a unique monotonically increasing function F(!) such that = e2πi!h dF (!) −1=2 F(−∞) = F(−1=2) = 0 and F(1) = F(1=2) = σ2: where 8 The function F(!) behaves like a CDF for a discrete random variable, 0; ! < −! > 0 except that F(1) = σ2 = γ(0). More on this later. < σ2 F(!) = 2 ; −!0 ≤ ! < !0 > 2 :σ ;! ≥ !0 2 F(!) jumps by σ =2 at −!0 and !0. Arthur Berg Spectral Density (Chapter 12) 8/ 19 Arthur Berg Spectral Density (Chapter 12) 9/ 19 Spectral Density Periodogram Spectral Density Periodogram Approximating Step Functions Spectral Density Any step function can easily be approximated by an absolutely Suppose the autocovariance function of a stationary process satisfies continuous function. 1 X jγ(h)j < 1 (?) h=−∞ then autocovariance function has the representation Z 1=2 γ(h) = ei!hf (!); h = 0; ±1; ±2;::: −1=2 where f (!) is the spectral density. Similarly, 1 X f (!) = γ(h)e−i!h; −π ≤ ! ≤ π h=−∞ For an absolutely continuous spectral distribution function, a spectral Note: (?) is sufficient, but not necessary, to guarantee a spectral density. density exists. The functions γ(h) and f (!) form a Fourier transform pair. Arthur Berg Spectral Density (Chapter 12) 10/ 19 Arthur Berg Spectral Density (Chapter 12) 11/ 19 Spectral Density Periodogram Spectral Density Periodogram Spectral Density of an ARMA(p; q) Process Example — Spectral Density of an MA(2) and an AR(1) Spectral density of xt = (wt+1 + wt + wt−1)=3. Theorem Let xt by an ARMA(p; q) process (not necessarily causal or invertible) 2 satisfying φ(B)xt = θ(B)wt where wt ∼ WN(0; σ ), θ and φ have no common zeros, and φ has no zeros on the unit circle. Then xt has the spectral density jθ(e−i!)j2 f (!) = σ2 ; −π ≤ ! ≤ π jφ(e−i!)j2 Spectral density of xt = xt−1 − :9xt−2. Many people define the spectral density on −π to π, but this is just a matter of scaling. The spectral density of white noise is a constant (equal to σ2). Arthur Berg Spectral Density (Chapter 12) 12/ 19 Arthur Berg Spectral Density (Chapter 12) 13/ 19 Spectral Density Periodogram Spectral Density Periodogram Discrete Fourier Transform The Periodogram Definition Given data x ;:::; x , the discrete Fourier transform is defined to be 1 n Definition n 1 The periodogram is defined at Fourier frequencies to be I(!j ) where X −2πi!j t d(!j ) = p xt e n 2 t=1 I(!j ) = jd(!j )j for j = 0; 1;:::; n − 1 where wj = j=n are the “Fourier frequencies”. Amazingly, via some mathematical calculations, we have the identity The discrete Fourier transform is a one-to-one transform of the data. The n−1 X −2πi!j h original data can be produced from I(!j ) = γb(h)e h=−(n−1) − 1 n 1 X 2πi!j t xt = p d(! )e n j j=0 Arthur Berg Spectral Density (Chapter 12) 15/ 19 Arthur Berg Spectral Density (Chapter 12) 16/ 19 Spectral Density Periodogram Spectral Density Periodogram Periodogram as an Analysis of Variance Calculation in R The cosine and sine transforms are defined as n 1 X > x = c(1,2,3,2,1) dc(! ) = p xt cos(2π! t) j n j > c1 = cos(2*pi*1:5*1/5) t=1 > s1 = sin(2*pi*1:5*1/5) n 1 X > c2 = cos(2*pi*1:5*2/5) ds(! ) = p xt sin(2π! t) > s2 = sin(2*pi*1:5*2/5) j n j t=1 > omega1 = cbind(c1, s1) > omega2 = cbind(c2, s2) Then the total variation can be written as (for n odd) > anova(lm(x~omega1+omega2)) # ANOVA Table (n−1)=2 (n−1)=2 n h i X ¯ 2 X 2 2 X Analysis of Variance Table (xt − x) = 2 dc(!j ) + ds(!j ) = 2 I(!j ) Response: x t=1 j=1 j=1 Df Sum Sq Mean Sq F value Pr(>F) Source df SS MS omega1 2 2.74164 1.37082 omega2 2 0.05836 0.02918 !1 2 2I(!1) I(!1) Residuals 0 0.00000 ! 2 2I(! ) I(! ) 2 2 2 > abs(fft(x))^2/5 # the periodogram (as a check) . [1] 16.20000000 1.37082039 0.02917961 0.02917961 1.37082039 !(n−1)=2 2 2I(!(n−1)=2) I(!(n−1)=2) Total n − 1 Pn (x − x¯)2 Arthur Berg Spectralt Density=1 (Chaptert 12) 17/ 19 Arthur Berg Spectral Density (Chapter 12) 18/ 19 Spectral Density Periodogram Limitations of the Periodogram From its structure: n−1 X −2πi!j h I(!j ) = γb(h)e h=−(n−1) it looks like a potential estimate of the spectral density given by 1 X f (!) = γ(h)e−2πi!h h=−∞ But there’s a problem: 2I(!j ) d 2 −! χ2 f (!j ) What we would like is a limit to a constant in the right hand side of the above equation. Pros, Cons and fixes to the periodogram will be discussed next time. Arthur Berg Spectral Density (Chapter 12) 19/ 19.