Introduction to Time Series Analysis. Lecture 18
Total Page:16
File Type:pdf, Size:1020Kb
Introduction to Time Series Analysis. Lecture 18. 1. Review: Spectral density, rational spectra, linear filters. 2. Frequency response of linear filters. 3. Spectral estimation 4. Sample autocovariance 5. Discrete Fourier transform and the periodogram 1 Review: Spectral density If a time series X has autocovariance γ satisfying { t} ∞ γ(h) < , then we define its spectral density as h=−∞ | | ∞ ∞ f(ν)= γ(h)e−2πiνh h=−∞ for <ν< . We have −∞ ∞ 1/2 γ(h)= e2πiνhf(ν) dν. −1/2 2 Review: Rational spectra For a linear time series with MA( ) polynomial ψ, ∞ 2 2πiν 2 f(ν)= σw ψ e . If it is an ARMA(p,q), we have 2 θ(e−2πiν ) f(ν)= σ2 w φ (e−2πiν) 2 q −2πiν 2 θ e zj = σ2 q j=1 − , w 2 p −2πiν 2 φp e pj j=1 | − | where z1,...,zq are the zeros (roots of θ(z)) and p1,...,pp are the poles (roots of φ(z)). 3 Review: Time-invariant linear filters A filter is an operator; given a time series X , it maps to a time series { t} Y . A linear filter satisfies { t} ∞ Yt = at,jXj. j=−∞ time-invariant: at,t−j = ψj : ∞ Yt = ψjXt−j. j=−∞ causal: j < 0 implies ψj = 0. ∞ Yt = ψj Xt−j. j=0 4 Time-invariant linear filters The operation ∞ ψjXt−j j=−∞ is called the convolution of X with ψ. 5 Time-invariant linear filters The sequence ψ is also called the impulse response, since the output Y of { t} the linear filter in response to a unit impulse, 1 if t = 0, Xt = 0 otherwise, is ∞ Yt = ψ(B)Xt = ψj Xt−j = ψt. j=−∞ 6 Introduction to Time Series Analysis. Lecture 18. 1. Review: Spectral density, rational spectra, linear filters. 2. Frequency response of linear filters. 3. Spectral estimation 4. Sample autocovariance 5. Discrete Fourier transform and the periodogram 7 Frequency response of a time-invariant linear filter Suppose that X has spectral density f (ν) and ψ is stable, that is, { t} x ∞ ψ < . Then Y = ψ(B)X has spectral density j=−∞ | j | ∞ t t 2πiν 2 fy(ν)= ψ e fx(ν). The function ν ψ(e2πiν ) (the polynomial ψ(z) evaluated on the unit → circle) is known as the frequency response or transfer function of the linear filter. The squared modulus, ν ψ(e2πiν ) 2 is known as the power transfer → | | function of the filter. 8 Frequency response of a time-invariant linear filter For stable ψ, Yt = ψ(B)Xt has spectral density 2πiν 2 fy(ν)= ψ e fx(ν). We have seen that a linear process, Yt = ψ(B)Wt, is a special case, since f (ν)= ψ(e2πiν ) 2σ2 = ψ(e2πiν ) 2f (ν). y | | w | | w When we pass a time series X through a linear filter, the spectral density { t} is multiplied, frequency-by-frequency, by the squared modulus of the frequency response ν ψ(e2πiν ) 2. → | | This is a version of the equality Var(aX)= a2Var(X), but the equality is true for the component of the variance at every frequency. This is also the origin of the name ‘filter.’ 9 Frequency response of a filter: Details 2πiν 2 Why is fy(ν)= ψ e fx(ν)? First, ∞ ∞ γy(h)= E ψj Xt−j ψkXt+h−k j=−∞ k=−∞ ∞ ∞ = ψj ψkE [Xt+h−kXt−j] j=−∞ k=−∞ ∞ ∞ ∞ ∞ = ψ ψ γ (h + j k)= ψ ψ γ (l). j k x − j h+j−l x j=−∞ k=−∞ j=−∞ l=−∞ It is easy to check that ∞ ψ < and ∞ γ (h) < imply j=−∞ | j | ∞ h=−∞ | x | ∞ that ∞ γ (h) < . Thus, the spectral density of y is defined. h=−∞ | y | ∞ 10 Frequency response of a filter: Details ∞ −2πiνh fy(ν)= γ(h)e h=−∞ ∞ ∞ ∞ −2πiνh = ψj ψh+j−lγx(l)e h=−∞ j=−∞ l=−∞ ∞ ∞ ∞ 2πiνj −2πiνl −2πiν(h+j−l) = ψje γx(l)e ψh+j−le j=−∞ l=−∞ h=−∞ ∞ 2πiνj −2πiνh = ψ(e )fx(ν) ψhe h=−∞ 2πiνj 2 = ψ(e ) fx(ν). 11 Frequency response: Examples 2πiν 2 2 For a linear process Yt = ψ(B)Wt, fy(ν)= ψ e σw. For an ARMA model, ψ(B)= θ(B)/φ(B), so Y has the rational { t} spectrum 2 θ(e−2πiν ) f (ν)= σ2 y w φ (e−2πiν) 2 q −2πiν 2 θ q e zj = σ2 j=1 − , w 2 p −2πiν 2 φp e pj j=1 | − | where pj and zj are the poles and zeros of the rational function z θ(z)/φ(z). → 12 Frequency response: Examples Consider the moving average 1 k Y = X . t 2k + 1 t−j j=−k This is a time invariant linear filter (but it is not causal). Its transfer function is the Dirichlet kernel 1 k ψ(e−2πiν )= D (2πν)= e−2πijν k 2k + 1 j=−k 1 if ν = 0, = sin(2π(k+1/2)ν) (2k+1) sin(πν) otherwise. 13 Example: Moving average Transfer function of moving average (k=5) 1 0.8 0.6 0.4 0.2 0 −0.2 −0.4 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 ν 14 Example: Moving average Squared modulus of transfer function of moving average (k=5) 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 ν This is a low-pass filter: It preserves low frequencies and diminishes high frequencies. It is often used to estimate a monotonic trend component of a series. 15 Example: Differencing Consider the first difference Y = (1 B)X . t − t This is a time invariant, causal, linear filter. Its transfer function is ψ(e−2πiν ) = 1 e−2πiν, − so ψ(e−2πiν) 2 = 2(1 cos(2πν)). | | − 16 Example: Differencing Transfer function of first difference 4 3.5 3 2.5 2 1.5 1 0.5 0 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 ν This is a high-pass filter: It preserves high frequencies and diminishes low frequencies. It is often used to eliminate a trend component of a series. 17 Introduction to Time Series Analysis. Lecture 18. 1. Review: Spectral density, rational spectra, linear filters. 2. Frequency response of linear filters. 3. Spectral estimation 4. Sample autocovariance 5. Discrete Fourier transform and the periodogram 18 Estimating the Spectrum: Outline We have seen that the spectral density gives an alternative view of • stationary time series. Given a realization x ,...,x of a time series, how can we estimate • 1 n the spectral density? One approach: replace γ( ) in the definition • · ∞ f(ν)= γ(h)e−2πiνh, h=−∞ with the sample autocovariance γˆ( ). · Another approach, called the periodogram: compute I(ν), the squared • modulus of the discrete Fourier transform (at frequencies ν = k/n). 19 Estimating the spectrum: Outline These two approaches are identical at the Fourier frequencies ν = k/n. • The asymptotic expectation of the periodogram I(ν) is f(ν). We can • derive some asymptotic properties, and hence do hypothesis testing. Unfortunately, the asymptotic variance of I(ν) is constant. • It is not a consistent estimator of f(ν). We can reduce the variance by smoothing the periodogram—averaging • over adjacent frequencies. If we average over a narrower range as n , we can obtain a consistent estimator of the spectral density. → ∞ 20 Estimating the spectrum: Sample autocovariance Idea: use the sample autocovariance γˆ( ), defined by · n−|h| 1 γˆ(h)= (x x¯)(x x¯), for n<h<n, n t+|h| − t − − t=1 as an estimate of the autocovariance γ( ), and then use a sample version of · ∞ f(ν)= γ(h)e−2πiνh, h=−∞ That is, for 1/2 ν 1/2, estimate f(ν) with − ≤ ≤ n−1 fˆ(ν)= γˆ(h)e−2πiνh. h=−n+1 21 Estimating the spectrum: Periodogram Another approach to estimating the spectrum is called the periodogram. It was proposed in 1897 by Arthur Schuster (at Owens College, which later became part of the University of Manchester), who used it to investigate periodicity in the occurrence of earthquakes, and in sunspot activity. Arthur Schuster, “On Lunar and Solar Periodicities of Earthquakes,” Proceedings of the Royal Society of London, Vol. 61 (1897), pp. 455–465. To define the periodogram, we need to introduce the discrete Fourier transform of a finite sequence x1,...,xn. 22 Introduction to Time Series Analysis. Lecture 18. 1. Review: Spectral density, rational spectra, linear filters. 2. Frequency response of linear filters. 3. Spectral estimation 4. Sample autocovariance 5. Discrete Fourier transform and the periodogram 23 Discrete Fourier transform For a sequence (x1,...,xn), define the discrete Fourier transform (DFT) as (X(ν0),X(ν1),...,X(νn−1)), where 1 n X(ν )= x e−2πiνkt, k √n t t=1 and ν = k/n (for k = 0, 1,...,n 1) are called the Fourier frequencies. k − (Think of ν : k = 0,...,n 1 as the discrete version of the frequency { k − } range ν [0, 1].) ∈ First, let’s show that we can view the DFT as a representation of x in a different basis, the Fourier basis. 24 Discrete Fourier transform Consider the space Cn of vectors of n complex numbers, with inner product a,b = a∗b, where a∗ is the complex conjugate transpose of the vector a Cn. ∈ Suppose that a set φ : j = 0, 1,...,n 1 of n vectors in Cn are { j − } orthonormal: 1 if j = k, φj,φk = 0 otherwise.