1 the Fourier Transform of Discrete Time Sequences

Home , Bandlimiting

Sampling and Reconstruction Supplement

1 The Fourier Transform of Discrete Time Sequences

Wehave found that x t, the analog signal sampled by a train of Dirac delta functions with

spacing T =1=f , had aFourier transform related to Ffxtg = X f by

X f mf 1 X f = f

s s s

m=1

This Fourier transform is p erio dic in f , with p erio d f , as is easily shown:

X f kf = f X f kf mf

s s s s s

m=1

= f X f m + k f

s s

m=1

X f lf ; l = m + k = f

s s

l =1

= X f : 2

We de ned X f as b eing identical to X f . Since this function is p erio dic, it can be

d s

represented by aFourier series, which led us to the inverse DTFT

f =2

j 2nT f

x n = X f e df

T d

f =2

1=2T

j 2nT f

= T X f e df 3

1=2T

This view tends to leave the reader with the impression that without the ideal impulsive

sampling representation, we would be at a loss to deal with the DTFT. Actually, we can

do nicely without resorting to impulsive sampling, if we like, for generating the DTFT.

Consider the sequence x n, and supp ose that there is some continuous time signal xt,

whose instantaneous values at multiples of time T constitute the values of x n, as in Fig.

1. The CT signal is equal to

j 2f t

xt= X f e df 4

if X f = Ffxtg. But also,

j 2f nT

X f e df; 5 x n=xnT =

1 1 x(t)

t -T 0 T 8T

x T (n)

-1 0 1 8

Figure 1: The samples xnT =x n are shown here simply as the function xtevaluated

at the particular instants fnT g.

X(f) m=0 m=1 m=-1 m=2 m=-2 f

-3/(2T) -1/(2T) 1/(2T) 3/(2T)

Figure 2: The inverse Fourier transform of X f can b e evaluated as the sum of integrals over

these nite sub-intervals. The subsequent simpli cations are p ossible b ecause the complex

exp onentials by which X f is multiplied in the inversion have exp onents j 2nT f , and are

all therefore p erio dic on these intervals.

where we've just chosen to evaluate the inverse FT only at the sample lo cations. Now, just

for fun, we'll express the integral in 5 as the summation of integrals over nite intervals,

each with length 1=T , as illustrated in Figure 2.

Z m 1

T 2T

j 2f nT

X f e df x n =

1 m

m=1

T 2T

Z 1

m m

j 2nT f m=T

~ ~ ~

= e df; for f = f + X f

T T

m=1

Z 1

j 2nT f j 2nm

e e df 6 = X f

m=1

But the second exp onential in 6 is equal to 1 for all integers n and m, and can b e discarded.

This is what we get for restricting our consideration of the inverse FT to the discrete

lo cations fnT g. Thus we have

j 2nT f

x n = X f e df

m=1

2T 2

1 m

j 2nT f

= T e df X f

T T

m=1

j 2nT f

f X f mf e df =

s s

m=1

j 2nT f

= X f e df 7

for any continuous time signal xt , X f which matches the samples x n. This is

exactly the inverse DTFT of 3 we found earlier, when the expression within the brackets

was taken from the impulsively sampled signal. So while the impulse train sampling is a very

useful to ol, it is not necessary to create x n and its DTFT. The impulsive representation

is p erhaps more imp ortant in analyzing the reconstruction of CT signals from samples.

2 Filters for Reconstructing Continuous Time Signals

from Samples

Often our system will, at some p oint, need to reconstruct from x n a CT signal x^t to

approximate xt. Our analytical description could b e a mo del for digital/analog conversion

at the output of many digital systems such as CD players, digital telephones, etc. This

transition from a DT signal to aCT signal will b e made by assigning to each sample x n

a CT delta function t nT , after which we can use familiar ltering analysis on the CT

signal x t.

2.1 Bandlimited Interp olation for Perfect Reconstruction of Nyquist

Sampled Signals

Xs (f) m=0 Ideal HR (f)

m=-1fs X(f) m=1

-fs 0 fs

Figure 3: The sp ectrum X f of the sampled signal, x t, with xt bandlimited, and the

s s

Nyquist sampling rate satis ed. Simple extraction of the m = 0 sp ectral replication yields

exactly x^t=xt. 3

For the moment, we'll assume that xt is bandlimited,

X f = 0; jf j f < 1 8

and T satis es the Nyquist sampling criterion of f > 2f , or T < 1=2f . Clearly, as

s h h

illustrated in Fig. 3, a CT signal with Fourier transform X f , or the m =0 replication of

the CT sp ectrum in the DT transform, could b e reconstructed by p erfect lowpass ltering.

This simple graphic argument suggests that a lowpass lter, with cuto frequency of

f =2, should recover the bandlimited signal perfectly. This must corresp ond to convolution

of the impulsive samples in x t with some lter h t in the time domain. To see what

s R

sort of h t this pro duces, we lo ok at the inverse Fourier transform:

j 2f t

h t = H f e df

R R

f =2

j 2f t

e df =

f =2

jf t jf t

s s

e e

j 2f t

sinf t t

= = sinc : 9

f t T

A plot of this lter is given in Fig. 4, with T=0.1, and zero crossings marked.

Bandlimited Sinc Interpolation Filter FT Magnitude of Bandlimited Interpolation Filter

0.1 1

0.08 0.8

0.6 0.06

0.4 0.04

0.2 0.02

0 -0.2 0 5 10 15 20 25 30

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 Frequency (Hz)

Figure 4: The bandlimited interp olation lter corresp onding to sample spacing T of 0.1.

Note the extent of the \tails" of this highly non-causal lter.

To use CT system and transform theory,wehave ascrib ed a Dirac delta function to each

sample, and can apply the p erfect H f lter in terms of the time-domain op eration with

R 4

F fH f g = h t.

R R

xt = x n t nT h t

T R

n=1

x n nT h t d =

T R

n=1

= x nh t nT : 10

T R

n=1

In other words, this linear interp olation pro cess, whichisintended to give x de ned values

between samples, amounts to adding a bunch of copies of h t, each scaled by the value of

a particular sample, and shifted by nT .

Knowing that the H f in 9 repro duces exactly X f , and substituting the sinc for

h tinto 10, we have

t nT

xt= x nsinc 11

n=1

This is very useful, esp ecially for intuitiveevaluations; it should o ccupy a place in your heart

alongside the sampling theorem. In fact, 11 is typically included as part of the sampling

theorem. In other words, 11 falls into the \don't you ever dare forget this" category. It

gives us an interp olation lter for exact repro duction of xt from samples taken at at least

Nyquist rate. In Fig. 5, an example of the approximation of a bandlimited signal by a

truncated sinc expansion is illustrated. This example features a cosine wave of frequency

0.5, which is sampled well ab ove the Nyquist rate. If we were to provide the contributions

of samples outside the window also, a p erfect reconstruction would b e p ossible.

2.2 Non-Ideal Non-Bandlimited Interp olation

Lo oking at 11 as the means of computing the value of xt at, for example, t = t , we

see that to nd the interp olated value, we need the contributions of an in nite number of

samples, including the very distant ones. This is b ecause the sinc function is noncausal,

and in fact has very heavy \tails," falling o in magnitude only as 1=t. It is easy to see

the e ects in Fig. 5. Imagine that you are resp onsible for pro ducing the value x0:25, and

T =0:1. The formula states x0:25 = x nsinc[0:25 0:1n]=0:1, and we need to

n=1

lo ok in nitely far into the past and into the future to reconstruct. Because the sinc decays,

one can obtain a good approximation of x0.25 by using, for example, the weighted sinc

values corresp onding to 30 samples on either side of t = 0:25, and ignoring others. With

this approximation, the lack of causalitymay b e of little concern, esp ecially if wework with

discretely stored data, whichwe can access in any order we like. In digitally recorded sp eech, 5 Bandlimited (Sinc) Interpolation with Finite No. Samples

0.8

0.6

0.4

0.2

-0.2

-0.4

-0.6

-0.8

-1

-0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5

Figure 5: Partial sum of sinc kernels to approximate an adequately sampled cosine function.

The dashed line is the approximation from only those samples within this window. Clearly,

we need the contribution of more neighb oring samples to get a go o d estimate of the cosine.

for example, wemayhave the luxury of scanning to and fro throughout the record to p erform

reconstruction. Some noncausality can be also accommo dated by using delay; that is, we

wait to compute x0:25 until we have at least up to the sample x 30, and can lo ok to

b oth sides of the lo cation 0.25.

But real time applications often demand more. Delay can be costly in b oth the con-

sequences of real time delay, and the cost of computation. Both e ects are features of

attempting to realize a p erfect lowpass lter. Wemay also need to use the interp olated sig-

nal, for example, in a controller system to make immediate adjustments. If your auto-piloted

aircraft is ying toward an immovable ob ject at 1500 miles per hour, you may not want to

wait for 30T seconds to let your controller interp olate x^t and decide to change course.

Finally, esp ecially if we should fail to meet the Nyquist criterion when we sample, X f

contains aliasing, and it is not always clear that a p erfect lowpass lter is the b est option.

In Fig. 6, for example, we have used a nite sampling rate on a square pulse, which is not

bandlimited, pro ducing aliasing in X f . When these samples are interp olated by the ideal

lowpass lter, the sinc function, the result is the highly oscillatory reconstruction shown.

Such oscillation near sharp transitions in the signal is known as \ringing." The error in this

reconstruction includes the e ects of aliasing into the reconstruction lter's passband, and

the loss of high frequencies which are necessary to construct such a sharp pulse remember

partial sums of Fourier series basis functions?. For all the ab ove reasons, it is imp ortant that

we consider interp olation metho ds other than the p erfect lowpass lter for reconstructing

the approximation x^ t. 6 Sampling of Square Pulse Bandlimited Interpolation from Samples

1 1

0.8 0.8

0.6 0.6

0.4 0.4

0.2 0.2

0 0

-0.2 -0.2

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

Figure 6: Square pulse sampled at T =0:1, and reconstructed with a p erfect bandlimiting

lter. The oscillatory b ehavior at the edges is known as \ringing."

The formula derived ab ove in 10 works for any linear, shift-invariant reconstruction

lter, so we can generalize for estimates x^ t with

x^t= x nh t nT : 12

T R

n=1

The interp olation pro cess for reconstruction may use any of a wide variety of lters. See

Fig. 7 for a simple illustration, obviously not a great interp olation lter.

A particularly simple, quick, and therefore p opular way to reconstruct is by simply holding

the value x n until the next sample comes along. This lter is usually called \zero-order

hold," and may be part of a \sample-and-hold" system. It is equivalent to CT convolution

of the impulsive train of samples with the interp olation lter of Fig. 8. Recall that, taking

a single impulsive sample, x n t nT h t = x nh t nT . In the frequency

T R T R

domain, where we can deal with the simpler multiplication of signals, we nd from Fourier

transform tables and theorems that

sinT f 1 f

j 2 f jf =f

H f =Te 13 = sinc e

T f f f

s s

The magnitude of this lter's frequency resp onse is also given in Fig. 8.

If you were only six years old, you would probably come up with a b etter technique,

which is just a little more complicated. Why not just \connect the dots" to get a CT signal

from the samples? This metho d is often called simply \linear" interp olation. Note that

all the metho ds we are discussing are linear in the sense of their b eing accomplished by

linear systems. But this one is linear in another more graphic sense, in that it consists of

lines b etween sample p oints. One way of lo oking at this metho d is that as the interp olated 7 x(t)

t 0 T 2T 4T

hR (t)

tt -T/20 T/2

^x(t)

Figure 7: An example of \interp olation" by the lter h t. Note the obvious failure of this

kernel to accurately repro duce a go o d approximation of xt.

function lo cation moves closer to a given sample, that sample's in uence increases. Only

the two samples surrounding the p oint in uence its value. The reconstruction lter which

accomplishes this is shown in Fig. 9, and is another familiar entry in transform tables,

t=T . Note that it is non-causal and that in real time, a delayofT linear phase would

be required between input and output. So here,

jtj

jtjjT j 1

14 h t =

0 jtj > jT j;

1 f

H f = sinc 15

f f

s s

3 Analysis of Error in Reconstructed Signals

The measure of success of a system consisting of only sampling and reconstruction is in the

di erence b etween the interp olated signal and the one from which the samples were originally

taken. The examples ab ove should give some qualitative idea of the nature of the error for

the typ es of reconstruction lters we've considered. We could make this comparison in the

time domain, but it's generally far simpler to do so with Fourier transforms. Generally, the

error can b e divided into two categories: 1 the distortion of the m = 0 sp ectral replication,

from the original transform of the signal, by the reconstruction lter. These are \desired"

frequency comp onents, but they are generally at least slightly attenuated relative to the zero

frequency, and p erhaps phase-shifted; 2 higher frequency energy not found in the original 8 Zero-Order Hold Interpolation Filter FT Magnitude of Zero-Order Hold Interpolation Filter 0.1

0.09 1 0.08

0.8 0.07

0.06 0.6 0.05

0.4 0.04

0.03 0.2 0.02

0 0.01

0 -0.2 0 5 10 15 20 25 30 -0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 Frequency (Hz) Interpolation by Zero-Order Hold

0.8

0.6

0.4

0.2

-0.2

-0.4

-0.6

-0.8

-1

-0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5

Figure 8: The zero-order hold lter used in sample-and-hold systems, and its Fourier

transform for p ositive frequencies, with an interp olation of a cosinusoid. The cosine has

frequency of only 0.5 Hz.

signal. This is the result of remnants of sp ectral replications for jmj > 0, which would be

removed by the p erfect lowpass lter in the ideal case, but are not removed by practical

lters.

When f is smaller than the Nyquist rate, aliasing o ccurs, and the distinction between

the two typ es of error ab ove may not be clear, since the aliased comp onents will add to

the m = 0 replication. These aliased comp onents may be either in or out of phase with

the desired frequency comp onents. In some of our examples, however, we've seen unwanted

aliased sinusoidal comp onents at frequencies b elow f =2 which are clearly identi able, and

distinct from any part of xt. Typ es of error are illustrated in Fig. 10. 9 Linear (First-Order Hold) Interpolation Filter FT Magnitude of First-Order Hold (Linear Interp) Filter 0.1

0.09 1 0.08

0.8 0.07

0.06 0.6 0.05

0.4 0.04

0.03 0.2 0.02

0 0.01

0 -0.2 0 5 10 15 20 25 30 -0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 Frequency (Hz) Interpolation by First-Order Hold (Linear)

0.8

0.6

0.4

0.2

-0.2

-0.4

-0.6

-0.8

-1

-0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5

Figure 9: "Linear" reconstruction lter, and the cosine samples interp olated.

3.1 Example of Reconstruction Error Analysis

Let xt = A cos6 10 t + , with a phase shift. We'll sample xt at 10 Khz, which

gives us the sp ectrum in Fig. 11. Note that this is b oth the CT Fourier transform of x t,

and the DT Fourier transform of the sequence x n. An imp ortant di erence is that when

we compute energy or power, or p erform the inverse transform, we must use the entire set

of frequencies 1 < f < 1 for CT op erations, while the DT equivalents involve only

the interval f =2 f f =2. Supp ose we want to compute the power in the CT and

s s

DT representations after sampling. The p ower density sp ectrum PDS in either case is the

same pattern of impulses, each with magnitude A =4. The power in the DT signal x n,

which is intended to approximate the power in the CT signal, is

2 2 2

f =2

A A A

+ = ; 16 P = S f df =

dx dx

4 4 2

f =2

s 10

which is exactly the p ower in CT measure in xt. But the p ower in the signal x tis1,

since we have to sum an in nite number of impulses in the power density sp ectrum. This

makes sense, since each nonzero magnitude impulse in x t has in nite energy, giving it

in nite average energy. Thus this x t is neither a power nor an energy signal.

Consider now the reconstruction of the estimate x^t from these samples. Let's do the

zero-order hold one more time. Rememb er that the f factors from the sampling and ltering

cancel. We'll write frequencies in terms of Khz to save space for now.

jf =f j j

X f = e Asincf=f 0:5[e f mf +3+e f mf 3]

s s s

m=1

j 3=10 j 3=10

= 0:5A sinc3=10 e f +3+e f 3+ + 17

j m3=10 j m3=10

+ sinc m 3=10[e f 10m +3+e f +10m 3]

m=1

j m+3=10 j m+3=10

+sinc m +3=10 [e f +10m +3+e f 10m 3]

This sp ectrum is represented in Fig. 12. If we cho ose to lo ok at this reconstruction in the

time domain, we can \pair o " the impulses in 17 and invert the transform. Notice that

except for m = 0, all the pairs are combinations from di erent replications.

x^t = A[sinc3=10 cos6 10 t + 3=10

+ [sincm 3=10 cos2 10 10m 3t + m 3=10

m=1

+sincm +3=10 cos2 10 10m +3t + m +3=10 18

3.2 Error Signal

The di erence et=xt x^ t represents error in the reconstructed estimate. If we want

the total p ower in the error signal, we might compute the two comp onents separately. First

consider the undesired comp onents, those in x^t having frequencies ab ove 3 Khz. Each

cosine of magnitude A, regardless of phase, has power A =2. So we could write the error

power for spurious frequencies as

2 2

A A

2 2

sincm 3=10 + sinc m +3=10 19

2 2

m=1

We could also have gotten this almost directly from 17, by simply ignoring the phase no

e ect on power, and observing that each impulse of magnitude 0:5A in X f gives us an

impulse of magnitude 0:25A in the PDS. 11

The desired frequency comp onent at 3Khz is attenuated, and undergo es a phase shift of

3=10. The error signal at this frequency in the time domain is just the original signal

minus the rst term in 18. The error p ower is again probably more easily computed in the

transform domain. The error signal at the desiredfrequencyofxt x^ t has the transform

by linearity:

j 3 j 3

Ffxt x^tg = 0:5A e f 3 10 +e f +3 10

j 3=10 3 j 3=10 3

e sinc3=10 f 3 10 e sinc3=10 f +3 10

j j 3=10 3

= 0:5A e 1 sinc 3=10e f 3 10

j j 3=10 3

+ e 1 sinc3=10e f +3 10

The phase which multiplies each pulse in the error sp ectrum is cancelled in the PDS, so we

drop the terms, and expand the complex exp onentials to nd:

h i

2 2 3 2 3

S f 0:25A j1 0:860:59 + j 0:81j f 3 10 +j1 0:860:59 j 0:81j f +3 10

h i

2 2 3 2 3

0:25A j0:51 j 0:70j f 3 10 +j0:51 + j 0:71j f +3 10

h i

2 3 3

0:25A 0:74 f 3 10 +0:74 f 3 10 21

These are approximate equivalences due to roundo .

The total p ower in this error at the desired frequency is just the sum of these magnitudes,

or 0:37A . For total error p ower, we add the result in 19. Compare this with results you've

seen in other examples, and you'll notice the phase shift causes a large error power if we

include time shift as an error. In this case, we have an error power approximately as large

as the power of the original signal, xt. It could be worse, of course. If we simply give a

sinusoid a phase shift of , the power in the signal consisting of the di erence between it

and the original is 2A . Error analysis for the other lters we've intro duced may be easier,

since they're zero-phase. 12 X(f)

(a) f 0

Xs (f)

m=-1 m=0 m=1 (b)

f 0 -f s fs

|HR (f)|

Unwanted Frequency X(f)^ Components (d)

-a 0 a

Figure 10: The estimated X f consists of the sum of the three \lumps" in the b ottom

sketch. We have not added them in this gure, b ecause the way in which they add will

dep end on phase. If everything is zero phase, the pieces will add directly. If elements at the

same frequency are out of phase, they may at least partially cancel. 13 |X s (f)| = |Xd (f)| (A/2)x104 (A/2)x104

m=-2 m=-1 m=0 m=1 m=2 f(Khz) -20 -10 -30 3 10 20

Phase[Xs (f)]

θ θ

f(Khz) -20 -10 0 3 10 20

−θ −θ

Figure 11: Fourier transform magnitude of sampled signal x t. s

|X(f)|^ A/2

m=-2 m=-1 m=0 m=1 m=2 f(Khz)

-20 -10 -30 3 10 20

Figure 12: Fourier transform magnitude of reconstructed CT signal x^ t. 14