Chapter
Fourier metho ds
Intro duction
Of the many b o oks on Fourier metho ds those by Chat eld Proakis and Manolakis
and Blo om eld are particularly go o d
Sinewaves and Samples
Sines and cosines can b e understo o d in terms of the vertical and horizontal displace
ment of a xed p oint on a rotating wheel the wheel has unit length and rotates
anti clockwise The angle round the wheel is measured in degrees or radians
for unit radius circles the circumference is radians tell us how much of the circum
ference we ve got If we go round the wheel a whole numb er of times we end up in
the same place eg cos cos cos Frequency f is the numb er of times
round the wheel p er second Therefore given x cos f t x at t f f
etc For x cos f t we get a head start lead of radians Negative
frequencies may b e viewed as a wheel rotating clockwise instead of anti clo ckwise
If we assume we have samples of the signal every T seconds and in total we have
s
N such samples then T is known as the sampling p erio d and F T is the
s s s
sampling frequency in Hertz Hz samples p er second The nth sample o ccurs at
time t n nT n F The cosine of sampled data can b e written
s s
x n cos f t n
When dealing with sampled signals it is imp ortant to note that some frequencies
b ecome indistinguishable from others at a sampling frequency F the only unique
s
frequencies are in the range to F Hz Any frequencies outside this range b ecome
s
aliases of one of the unique frequencies
For example if we sample at Hz then a Hz signal b ecomes indistinguishable from
Signal Pro cessing Course W D Penny April
1
0.5
0
−0.5
−1
0 0.2 0.4 0.6 0.8 1
Figure Aliases The gure shows a Hz cosine wave and a Hz cosine wave
as solid curves At sampling times given by the dotted lines which correspond to a
sampling frequency of Hz the H z signal is an alias of the Hz signal Other
aliases are given by equation
a Hz signal This is shown in gure More generally if f is a unique frequency
0
then its aliases have frequencies given by
f f k F
0 s
where k is any p ositive or negative integer eg for f and F the two lowest
0 s
frequency aliases given by k and k are Hz and Hz
Because of aliasing we must b e careful when we interpret the results of sp ectral
analysis This is discussed more at the end of the lecture
Sinusoidal mo dels
If our time series has a p erio dic comp onent in it we might think ab out mo delling it
with the equation
x n R R cos f t n e n
0
where R is the o set eg mean value of x n R is the amplitude of the sine wave f
0
is the frequency and is the phase What our mo del do es nt explain will b e soaked
up in the error term e n Because of the trig identity
cos A B cos A cos B sin A sin B
the mo del can b e written in an alternative form
x n R a cos f t n b sin f t n e n 0
Signal Pro cessing Course W D Penny April
where a R cos and b R sin This is the form we consider for subsequent
analysis
This typ e of mo del is similar to a class of mo dels in statistics called Generalised Linear
Models GLIMS They p erform nonlinear regression by rst taking xed nonlinear
functions of the inputs these functions b eing called basis functions and second form
an output by taking a linear combination of the basis function outputs In sinusoidal
mo dels the basis functions are sines and cosines In statistics a much broader class
of functions is considered However sinewaves have some nice prop erties as we shall
see
Fitting the mo del
T T T
If we let x x x x N w R a b e e e e and
0 1 2 N
cos f t sin f t
cos f t sin f t
cos f t sin f t
A
cos f t N sin f t N
then the mo del can b e written in the matrix form
x Aw e
which is in the standard form of a multivariate linear regression problem The solution
is therefore
T 1 T
w A A A x
But sinewaves are orthogonal
Because we are dealing with sinewaves it turns out that the ab ove solution simpli
es We restrict ourselves to a frequency f which is an integer multiple of the base
p
frequency
f pF
p b
where p N and
F
s
f
b
N
eg for F and N seconds worth of data f H z and we can have
s b
1
f from Hz up to Hz The orthogonality of sinewaves is expressed in the following
p
equations
N N
X X
sin f t n cos f t n
k k
n=1 n=1
1
To keep things simple we don t allow f where p N if we did allow it we d get N and in
p
equations and for the case k l Also we must have N even
Signal Pro cessing Course W D Penny April
1 1
0.5 0.5
0 0
−0.5 −0.5
−1 −1
0 0.2 0.4 0.6 0.8 1 0 20 40 60 80 100
a b
Figure Orthogonality of sinewaves Figure a shows cos f t n and
b
cos f t n cosines which are and times the base frequency f H z For
b b
P
N
cos f t n cos f t n This can be any two integer multiples k l we get
k l
n=1
seen from Figure b which shows the product cos f t n cos f t n Because of
b b
the trig identity cosAcosB cos A B cos A B this looks like a Hz
signal superimposed on a Hz signal The sum of this signal over a whole number of
cycles can be seen to be zero because each cos term sums to zero If however k or l
are not integers the product does not sum to zero and the orthogonality breaks down
N
X
cos f t n sin f t n
k l
n=1
N
X
cos f t n sin f t n
k l
n=1
N
X
k l
cos f t n cos f t n
k l
N k l
n=1
N
X
k l
sin f t n sin f t n
k l
N k l
n=1
These results can b e proved by various trig identities or more simply by converting
to complex exp onentials see or later in this chapter The results dep end on the
fact that all frequencies that app ear in the ab ove sums are integer multiples of the
base frequency see gure
This prop erty of sinewaves leads to the result
T
A A D
where D is a diagonal matrix The rst entry is N from the inner pro duct of two
columns of s of length N the s are the co e cients of the constant term R and
0
all the other entries are N A matrix Q for which
T
Q Q D
is said to b e orthogonal Therefore our A matrix is orthogonal This greatly simpli es
the tting of the mo del which now reduces to
1 T
w D A x
Signal Pro cessing Course W D Penny April
160
140
120
100
80
60
40
20
0
1730 1740 1750 1760 1770 1780 1790
Figure Sunspot index solid line and prediction of it from a simple sinusoidal
model dotted line
which is simply a projection of the signal onto the basis matrix with some pre factor
1
D rememb er the inverse of a diagonal matrix is simply the inverse of each of
T
the diagonal terms so this is easy to compute Given that w a b R we can
0
see that for example a is computed by simply pro jecting the data onto the second
column of the matrix A eg
N
X
cos f t x a
t
N
n=1
Similarly
N
X
b sin f t x
t
N
n=1
N
X
x R
t 0
N
n=1
We applied the simple sinusoidal mo del to a sunsp ot data set as follows We chose
samples b etween the years and b ecause there was a fairly steady mean
level in this p erio d The sampling rate F Year This gives a base frequency of
s
f We chose our frequency f pf with p giving a complete cycle once
b b
every ten years This gave rise to the following estimates R a and
0
b The data and predictions are shown in Figure
Signal Pro cessing Course W D Penny April
Fourier Series
We might consider that our signal consists of lots of p erio dic comp onents in which
case the multiple sinusoidal model would b e more appropriate
p
X
x t R R cos f t e
0 k k k t
k =1
where there are p sinusoids with di erent frequencies and phases In a discrete Fourier
series there are p N such sinusoids having frequencies
k F
s
f
k
N
where k N and F is the sampling frequency Thus the frequencies range from
s
F N up to F The Fourier series expansion of the signal x t is
s s
N 2
X
x t R R cos f t
0 k k k
k =1
Notice that there is no noise term Because of the trig identity
cos A B cos A cos B sin A sin B
this can b e written in the form
N 2
X
x t a a cos f t b sin f t
0 k k k k
k =1
2 2 2
where a R cos and b R sin Alternatively we have R a b
k k k k k k
k k k
1
and tan b a The signal at frequency f is known as the k th harmonic
k k k
Equivalently we can write the nth sample as
N 2
X
x n a a cos f t n b sin f t n
0 k k k k
k =1
where t n nT
s
The imp ortant things to note ab out the sinusoids in a Fourier series are i the
frequencies are equal ly spread out ii there are N of them where N is the numb er
of samples iii Given F and N the frequencies are xed Also note that in the
s
Fourier series mo del there is no noise The Fourier series aims to represent the data
2
perfectly which it can do due to the excessive numb er of basis functions
The Fourier co e cients can b e computed by a generalisation of the pro cess used to
compute the co e cients in the simple sinusoidal mo del
N
X
cos f t n x n a
k k
N
n=1
2
Statisticians would frown on tting a mo del with N co e cients to N data p oints as the estimates
will b e very noisy the Fourier series is a low bias actually zero high variance mo del This underlines
the fact that the Fourier metho ds are transforms rather than statistical mo dels
Signal Pro cessing Course W D Penny April
Similarly
N
X
b sin f t n x n
k k
N
n=1
N
X
a x n
0
N
n=1
These equations can b e derived as follows To nd for example a multiply b oth
k
sides of equation by cos f t n and sum over n Due to the orthogonality
k
prop erty of sinusoids which still holds as all frequencies are integer multiples of a
base frequency all terms on the right go to zero except for the one involving a This
k
just leaves a N on the right giving rise to the ab ove formula
k
Example
The plots on the right of Figure show four comp onents in a Fourier series expan
sion The comp onents have b een ordered by amplitude The plots on the left of the
Figure show the corresp onding Fourier approximation
Fourier Transforms
Fourier series are representations of a signal by combinations of sinewaves of di erent
magnitudes frequencies and o sets or phases The magnitudes are given by the
Fourier coe cients These sinewaves can also b e represented in terms of complex
exponentials see the app endix for a quick review of complex numb ers a representa
tion which ultimately leads to algorithms for computing the Fourier co e cients the
Discrete Fourier Transform DFT and the Fast Fourier Transform FFT
Discrete Fourier Transform
Fourier series can b e expressed in terms of complex exp onentials This representation
leads to an e cient metho d for computing the co e cients We can write the cosine
terms as complex exp onentials
exp i f t n exp i f t n
k k
a cos f t n a
k k k
2
where i Picture this as the addition of two vectors one ab ove the real axis
and one b elow Together they make a vector on the real axis which is then halved
We can also write the sine terms as
exp i f t n exp i f t n
k k
b sin f t n b
k k k
i
Signal Pro cessing Course W D Penny April
1.5 1.5
1 1
0.5 0.5
0 0
−0.5 −0.5
−1 −1
−1.5 −1.5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
a e
1.5 1.5
1 1
0.5 0.5
0 0
−0.5 −0.5
−1 −1
−1.5 −1.5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
b f
1.5 1.5
1 1
0.5 0.5
0 0
−0.5 −0.5
−1 −1
−1.5 −1.5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
c g
1.5 1.5
1 1
0.5 0.5
0 0
−0.5 −0.5
−1 −1
−1.5 −1.5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
d h
Figure Signal solid line and components of the Fourier series approximation
P
p
R cos f dotted lines with a p b p c p and d p
k k k
k =1
where we have ordered the components according to amplitude The corresponding
2 2
individual terms are e R f and f R f
2 2
and g R f and and h R
f and
Signal Pro cessing Course W D Penny April
Picture this as one vector ab ove the real axis minus another vector b elow the real
axis This results in a purely imaginary and p ositive vector The result is halved
and then multiplied by the vector exp i from multplying top and b ottom
by i which provides a rotation to the real axis
Adding them and moving i to the numerator by multiplying b top and b ottom by
k
i gives
a b i exp i f t n a b i exp i f t n
k k k k k k
Note that a single term at frequency k has split into a complex combination the co
e cients are complex numb ers of a p ositive frequency term and a negative frequency
term Substituting the ab ove result into equation and noting that f t n k n N
k
we get
N 2 N 2
X X
x n a a b i exp i k n N a b i exp i k n N
0 k k k k
k =1 k =1
If we now let
N
a b i X k
k k
and note that for real signals X k X k negative frequencies are re ections
across the real plane ie conjugates then the a b i terms are equivalent to
k k
X k Hence
N 2 N 2
X X
x n a X k exp i k n N X k exp i k n N
0
N N
k =1 k =1
Now b ecause X N k X k this can b e shown by considering the Fourier
transform of a signal x n and using the decomp osition exp i N k n N
exp i N N exp i k n N where the rst term on the right is unity we can write
the second summation as
N 2
N 1
X X
x n a X k exp i k n N X k exp i N k n N
0
N N
k =1
k =N 2
Using the same exp onential decomp osition allows us to write
N 1
X
x n a X k exp i k n N
0
N
k =1
If we now let X k X k then we can absorb the constant a into the sum
0
giving
N
X
x n X k exp i k n N
N
k =1
which is known as the Inverse Discrete Fourier Transform IDFT The terms X k
are the complex valued Fourier coe cients We have the relations
a R efX g 0
Signal Pro cessing Course W D Penny April
a R efX k g
k
N
I mfX k g b
k
N
The complex valued Fourier co e cients can b e computed by rst noting the orthog
onality relations
N
X
N k N N
exp i k n N
other w ise
n=1
If we now multiply equation by exp i l n N sum from to N and re arrange
we get
N
X
x n exp i k n N X k
n=1
which is the Discrete Fourier Transform DFT
The Fourier Matrix
T
If we write X k as a vector X X X X N and the input signal as
T
a vector x x x x N then the ab ove equations can b e written in
matrix form as follows The Inverse Discrete Fourier Transform is
x FX
where F is the Fourier Matrix and the Discrete Fourier Transform is
1
X F x
If we let
w exp i N
N
we can write the Fourier matrix as
(N 1)
2
w w w
N
N N
2(N 1)
1
2 4
F
w w w
N
N N N
N
2
(N 1) 2(N 1) (N 1)
w w w
N N N
3
which has elements
(k 1)(n 1)
F w
N k n
N
3
We have re indexed such that we now have x to x N Hence we have n instead of
n
Signal Pro cessing Course W D Penny April
Now the inverse Fourier matrix is
(N 1)
2 1
w w w
N N N
2(N 1)
1
4 2
F
w w w
N
N N N
2
(N 1) 2(N 1) (N 1)
w w w
N N N
where the elements are
(k 1)(n 1)
1
w F
k n
N N
In the Fast Fourier Transform FFT an N dimensional matrix multiplication can b e
replaced by M dimensional multiplications where M N This is b ecause the
exp onential elements in the Fourier matrix have the key prop erty
2
w w
M
N
2
eg exp i exp i Co oley and Tukey realised you could use this
prop erty as follows If you split the IDFT
N 1
X
j k
X x w
k j
N
k =0
into a summation of even parts and a summation of o dd parts
M 1 M 1
X X
(2k +1)j
2j k
x w X w X
j 2k 2k +1
N N
k =0 k =0
2
then we can use the identity w w to give
M
N
M 1 M 1
X X
j k j k j
x w X w w X
j 2k 2k +1
M N M
k =0 k =0
which is the summation of two IDFTs of dimension M a similar argument applies
for the DFT
This reduces the amount of computation by approximately a factor of We can then
replace each M dimensional multiplication by an M dimensional one etc FFTs
require N to b e a p ower of b ecause at the lowest level we have lots of dimensional
op erations For N we get an overall sp eed up by a factor of For larger
N
2
log N multiplications by N the sp eed ups are even greater we are replacing N
2
2
Time Frequency relations
Signals can b e op erated on in the time domain or in the frequency domain We now
explore the relations b etween them
Signal Pro cessing Course W D Penny April
Power Sp ectral Density
The p ower in a signal is given by
N
X
2
jx n j P
x
n=1
We now derive an expression for P in terms of the Fourier co e cients If we note
x
that jx n j can also b e written in its conjugate form the conjugate form has the same
magnitude the phase is di erent but this do es nt matter as we re only interested in
magnitude
N
X
jx n j X k exp i k n N
N
k =1
then we can write the p ower as
N N
X X
X k exp i k n N j jx n P
x
N
n=1
k =1
If we now change the order of the summations we get
N N
X X
P x n exp i k n N j jX k
x
N
n=1
k =1
where the sum on the right is now equivalent to X k Hence
N
X
2
jX k j P
x
N
k =1
We therefore have an equivalence b etween the p ower in the time domain and the
p ower in the frequency domain which is known as Parseval s relation The quantity
2
P k jX k j
x
is known as the Power Spectral Density PSD
Filtering
The ltering pro cess
1
X
x n x l x n l
1 2
l = 1
is also known as convolution
x n x n x n
1 2
We will now see how it is related to frequency domain op erations If we let w
k N multiply b oth sides of the ab ove equation by exp iw n and sum over
n the left hand side b ecomes the Fourier transform
1
X
X w x n exp iw n
n= 1
Signal Pro cessing Course W D Penny April
and the right hand side RHS is
1 1
X X
x l x n l exp iw n
1 2
n= 1
l = 1
Now we can re write the exp onential term as follows
exp iw n exp iw n l exp iw l
0
Letting n n l we can write the RHS as
1 1
X X
0 0
x l exp iw l x n exp iw n X w X w
1 2 1 2
0
l = 1
n = 1
Hence the ltering op eration is equivalent to
X w X w X w
1 2
which means that convolution in the time domain is equivalent to multiplication in
the frequency domain This is known as the convolution theorem
Auto covariance and Power Sp ectral Density
The auto covariance of a signal is given by
1
X
n x l x l n
xx
l = 1
Using the same metho d that we used to prove the convolution theorem but noting
that the term on the right is x l n not x n l we can show that the RHS is
equivalent to
2
X w X w jX w j
which is the Power Sp ectral Density P w Combining this with what we get for the
x
left hand side gives
1
X
P w n exp iw n
x xx
n= 1
which means that the PSD is the Fourier Transform of the auto covariance This is
known as the Wiener Khintchine Theorem This is an imp ortant result It means
that the PSD can b e estimated from the auto covariance and vice versa It also means
that the PSD and the auto covariance contain the same information ab out the signal
It is also worth noting that since b oth contain no information ab out the phase of a
signal then the signal cannot b e uniquely constructed from either To do this we need
to know the PSD and the Phase spectrum which is given by
b
k
1
k tan
a k
Signal Pro cessing Course W D Penny April
where b and a are the real Fourier co e cients
k k
We also note that the Fourier transform of a symmetric function is real This is
b ecause symmetric functions can b e represented entirely by cosines which are them
selves symmetric the sinewaves which constitute the complex comp onent of a Fourier
series are no longer necessary Therefore b ecause the auto covariance is symmetric
the PSD is real
Sp ectral Estimation
The Perio dogram
The p erio dogram of a signal x is a plot of the normalised p ower in the k th harmonic
t
versus the frequency f of the k th harmonic It is calculated as
k
N
2 2
I f a b
k
k k
where a and b are the Fourier co e cients
k k
4
The p erio dogram is a low bias actually unbiased but high variance estimate of the
p ower at a given frequency This is therefore a problem if the numb er of data p oints
is small the estimated sp ectrum will b e very spiky
To overcome this a numb er of algorithms exist to smo oth the p erio dogram ie to
reduce the variance The Bartlett metho d for example takes an N p oint sequence
and sub divides it into K nonoverlapping segments and calculates I f for each The
k
nal p erio dogram is just the average over the K estimates This results in a reduction
in variance by a factor K at the cost of reduced sp ectral resolution by a factor K
The Welch metho d is similar but averages mo di ed p erio dograms the mo di cation
b eing a windowing of each segment of data Also the segments are allowed to overlap
For further details of this and other smo othing metho ds see Chapter in Proakis
and Manolakis This smo othing is necessary b ecause at larger lags there are fewer
data p oints so the estimates of covariance are commensurately more unreliable
Auto covariance metho ds
The PSD can b e calculated from the auto covariance However as the sample auto
covariance on short segments of data has a high variance then so will the resulting
sp ectral estimates
To overcome this a numb er of prop osals have b een made The auto covariance func
tion can rst b e smo othed and truncated by applying various smo othing windows
4
It is an inconsistent estimator b ecause the variance do es nt reduce to zero as the numb er of
samples tends to in nity
Signal Pro cessing Course W D Penny April
for example Tukey Parzen Hanning or Hamming windows For further details see
Chat eld p or Chapter in Proakis and Manolakis
Aliasing
Because of aliasing if we wish to uniquely identify frequencies up to B Hz then we
must sample the data at a frequency f B Hz
s
Alternatively given a particular sample rate f in order to uniquely identify frequen
s
cies up to f Hz and not confuse them with higher frequencies we must ensure that
s
there is no signal at frequencies higher than f This can b e achieved by applying
s
a Low Pass Filter LPF
Filtering
There are two main classes of lters I IR lters and FIR lters Their names derive
from how the lters resp ond to a single pulse of input their so called impulse resp onse
The output of an In nite Impulse Resp onse I IR lter is fed back to its input The
resp onse to a single impulse is therefore a gradual decay which though it may drop
rapidly towards zero no output will never technically reach zero hence the name
I IR
In Finite Impulse Resp onse FIR lters the output is not fed back to the input so
if there is no subsequent input there will b e no output The output of an FIR lter
page is given by
p 1
X
y n b x n k
k
k =0
where x n is the original signal and b are the lter co e cients
k
The simplest FIR lter is the normalised rectangular window which takes a moving
average of a signal This smo oths the signal and therefore acts a low pass lter
Longer windows cut down the range of frequencies that are passed
Other examples of FIR lters are the Bartlett Blackman Hamming and Hanning
windows shown in Figure The curvier shap e of the windows means their frequency
characteristics are more sharply de ned See Chapter in for more details FIR
lters are also known as Moving Average MA mo dels which we will encounter in
the next lecture
The output of an I IR lter is given by
p 1 p 1
a
b
X X
y n a y n k b x n k
k k
k =0 k =0
where the rst term includes the feedback co e cients and the second term is an FIR
Signal Pro cessing Course W D Penny April
1 1
0.8 0.8
0.6 0.6
0.4 0.4
0.2 0.2
0 0
0 5 10 15 20 25 30 0 5 10 15 20 25 30
a b 1 1
0.8 0.8
0.6 0.6
0.4 0.4
0.2 0.2
0 0
0 5 10 15 20 25 30 0 5 10 15 20 25 30
c d
Figure Filter coe cients of a Bartlett triangular b Blackman c Hamming
and d Hanning windows for p
20
15
10
5
0
0 10 20 30 40 50
Figure Frequency response of a Hamming window solid line and a rectangular
window dotted line The Hamming window cuts of the higher frequencies more sharply
Signal Pro cessing Course W D Penny April
mo del This typ e of lter is also known as a Autoregressive Moving Average ARMA
mo del the rst term b eing the Autoregressive AR part
I IR lters can b e designed by converting analog lters into the ab ove I IR digital form
See section for details Examples of resulting I IR implementations are the
Butterworth Chebyshev Elliptic and Bessel lters