EDUCATIONE DUCATION Editor: Denis Donnelly, [email protected] THE FAST FOURIER TRANSFORM FOR EXPERIMENTALISTS PART III: CLASSICAL SPECTRAL ANALYSIS By Bert Rust and Denis Donnelly ACH ARTICLE IN THIS CONTINUING SERIES ON THE FAST Pf()= 1 T 2 FOURIER TRANSFORM (FFT) IS DESIGNED TO ILLUMINATE lim∫ xt ( ) exp(−2πi ftdt ) , E T →∞T −T NEW FEATURES OF THE WIDE-RANGING APPLICABILITY OF THIS –¥£f £¥.(3) TRANSFORM. THIS SEGMENT DEALS WITH SOME ASPECTS OF THE But we have only a discrete, real time series spectrum estimation problem. Before Spectrum Estimation’s ⌬ we begin, here’s a short refresher Central Problem xj = x(tj), with tj = j t, about two elements we introduced The periodogram, invented by j = 0, 1, …, N – 1, (4) previously, windowing1 and convolu- Arthur Schuster in 1898,3 was the tion.2 As we noted in those install- first formal estimator for a time se- defined on a finite time interval of ments, a convolution is an integral ries’s frequency spectrum, but many length N⌬t. We saw in Part I1 that that expresses the amount of overlap others have emerged in the ensuing sampling x(t) with sample spacing ⌬t of one function as it is shifted over an- century. Almost all use the FFT in confined our spectral estimates to the other. The result is a blending of the their calculations, but they differ in Nyquist band 0 £ f £ 1/2⌬t. We used two functions. Closely related to the their assumptions about the missing the FFT algorithm to compute the dis- convolution process are the processes data; that is, the data outside the ob- crete Fourier transform (DFT) of cross-correlation and autocorrela- servation window. These assumptions tion. Computing the cross-correlation have profound effects on the spectral N −1 =−π j differs only slightly from the convolu- estimates. Let t be time, f be fre- Xxkj∑ exp 2 i k = N tion; it’s useful for finding the degree quency, and x(t) a real function on the j 0 of similarity in signal patterns from interval – < t < ¥. The continuous k = 0, 1, …, N/2, (5) two different data streams and in de- Fourier transform (CFT) of x(t) is de- termining the lead or lag between fined by which approximates the CFT X(f ) at such similar signals. Autocorrelation the Fourier frequencies ∞ is also related to the convolution; it’s X ()fxtftdt=−∫ ()exp(2πi ) , k described later. Windowing, used in −∞ f = , k = 0, 1, ..., N/2. (6) k Nt∆ extracting or smoothing data, is typi- cally executed by multiplying time- –¥£f £¥,(1)We then computed periodogram esti- domain data or its autocorrelation mates of both the PSD and the ampli- function by the window function. A where i ≡−1. If we knew x(t) per- tude spectrum by disadvantage of windowing is that it fectly and could compute Equation 1, alters or restricts the data, which, of then we could compute an energy = 1 2 Pf()kk||X , k = 0, 1, …, N/2, course, has consequences for the spec- spectral density function N tral estimate. In this installment, we continue our discussion, building on E(f ) = |X(f )|2, –¥£f £¥,(2) = 2 Af()kk||X , k = 0, 1, …, N/2. (7) these concepts with a more general N approach to computing spectrum es- and a power spectral density function timates via the FFT. (PSD) by We also saw that we could approximate 74 Copublished by the IEEE CS and the AIP 1521-9615/05/$20.00 © 2005 IEEE COMPUTING IN SCIENCE & ENGINEERING x(t) = sin [2π(0.50)(t + 0.25)] + noise 2.5 2.0 Signal and noise 1.5 Signal without noise ) i t ( 1.0 x 0.5 = i x 0.0 –0.5 –1.0 –1.5 the CFT and the frequency spectrum 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 on a denser frequency mesh simply by (a) t (time units) appending zeroes to the time series. Periodogram This practice, called zero padding, is 9 just an explicit assertion of an implicit 8 Signal and noise 7 Signal without noise assumption of the periodogram 6 method—namely, that the time series 5 4 is zero outside the observation window. 3 density (PSD) 2 Frequency spectrum estimation is a Power spectral 1 classic underdetermined problem be- 0 cause we need to estimate the spectrum 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4 at an infinite number of frequencies us- (b) Frequency ing only a finite amount of data. This problem has many solutions, differing Figure 1. Original and new time series as defined by Equation 8. (a) The noise- mainly in what they assume about the corrupted time series and the uncorrupted series originally used in Part I’s Figure 1b. missing data. The noise is independently, identically distributed n(0, 0.25). (b) Periodograms of Before considering other solutions the two times series plotted in (a). For the noise-corrupted series, the peak is ^ to this problem, let’s reconsider one of centered on frequency f 0 = 0.493. the examples from Part I1 (specifically, Figure 1b), but make it more realistic 4 ␶ ⌬ by simulating some random measure- Tukey’s correlogram estimators. m = m t, m = 0, 1, …, N – 1. (11) ment errors. More precisely, we take They’re based on the autocorrelation N = 32, ⌬t = 0.22, and consider the theorem (sometimes called Wiener’s Because we’re working with a real time ␳ ␶ ␳ ␶ time series theorem), which states that if X(f ) is series, and ( –m) = ( m), we don’t need the CFT of x(t), then |X(f )|2 is the to worry about evaluating ␳(␶) at neg- ⌬ tj = j t, j = 0, 1, 2, …, N – 1, CFT of the autocorrelation function ative lags. (ACF) of x(t). Norbert Wiener defined Because ␳(␶) is a limit of the average ␲ Î 5 ␶ xj = x(tj) = sin[2 f0(tj + 0.25)] + j, (8) the latter function as value of x*(t)x(t + ) on the interval [–T, T ], the obvious estimator is the 1 T with f = 0.5, and each Î a random ρτ( )=+ lim∫ xtxtdt * ( ) ( τ ) , sequence of average values 0 j T →∞ T −T number drawn independently from a 2 ρρˆˆ=∆()mt normal distribution with mean zero –¥ < ␶ < ¥, (9) m −− and standard deviation ␴ = 0.25. This 1 Nm1 = ∑ xx new time series is plotted together in which the variable ␶ is called the lag − nnm+ , Nmn=0 with the original uncorrupted series in (the time interval for the correlation of Figure 1a. Both series were zero x(t) with itself), and x*(t) is the com- m = 0, 1, …, N – 1. (12) padded to length 1,024 (992 zeroes plex conjugate of x(t). Thus, if we appended) to obtain the periodogram could access x(t), we could compute This sequence is sometimes called the estimates given in Figure 1b. It’s re- the PSD in two ways: either by Equa- unbiased estimator of ␳(␶) because its markable how well the two spectra tion 3 or by expected value is the true value—that ^ agree, even though the noise’s stan- ∞ is, ⑀{␳(m⌬t)} = ␳(m⌬t). But the data =−ρτ π τ τ. (10) dard deviation was 25 percent of the P()ffd∫−∞ ()exp(2 i ) are noisy, and for successively larger ␳^ signal’s amplitude. values of m, the average m is based on But again, we have access to only a fewer and fewer terms, so the variance The Autocorrelation Function noisy time series x0, x1, …, xN–1, so to grows and, for large m, the estimator After the periodogram, the next fre- use the second method, we need esti- becomes unstable. Therefore, it’s quency spectrum estimators to emerge mates for ␳(␶) evaluated at the discrete common practice to use the biased were Richard Blackman and John lag values estimator SEPTEMBER/OCTOBER 2005 75 E DUCATION Autocorrelation estimates 1.0 0.8 Biased estimate 0.6 Unbiased estimate 0.4 t) ∆ 0.2 m ( 0.0 ␳ –0.2 –0.4 –0.6 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 corresponding to the biased and un- (a) Lag = m∆t biased ACF estimates, shown in Fig- Correlogram estimates of power spectral density (PSD) ure 2a. The negative sidelobes for the unbiased correlogram show dramati- 14 Using biased ACF cally why most analysts choose the bi- 10 Using unbiased ACF ased estimate even though its central 6 ) f peak is broader. The reason for this 2 P( broadening, and for the damped side- –2 lobes, is that the biased ACF, Equa- –6 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4 tion 13, can also be computed by multiplying the unbiased ACF, Equa- (b) f tion 12, by the triangular (Bartlett) ta- pering window Figure 2. Autocorrelation and correlogram estimates for the noisy time series k defined by Equation 8. (a) Biased and unbiased estimates of the autocorrelation w =−1 , k N function (ACF); (b) correlogram estimates obtained from the ACF estimates in (a). k = 0, 1, 2, …, N – 1. (17) Nm−−1 Recall that we observed the same sort ρρ=∆=1 £ ^ £ ˆˆmnnm()mt ∑ xx+ , –1 rm 1, m = 0, 1, …, N – 1. (15) of peak broadening and sidelobe sup- N n=0 pression in Part I’s Figure 10 when we m = 0, 1, …, N – 1, (13) Correlogram PSD Estimators multiplied the observed data by a Once we’ve established the ACF esti- Blackman window before computing which damps those instabilities and has mate, we can use the FFT to calculate the periodogram.