Sampling Signals

Home , Aliasing, Upsampling

Sampling Signals

Overview:

¯ We use the Fourier transform to understand the discrete sampling and re-sampling of signals.

¯ One key question is when does sampling or re-sampling provide an adequate representation of the original signal?

Terminology:

¯ sampling – creating a discrete signal from a continuous process.

¯ downsampling (decimation) – subsampling a discrete signal

¯ upsampling – introducing zeros between samples to create a longer signal

¯ aliasing – when sampling or downsampling, two signals have same sampled representation but differ between sample locations.

Matlab Tutorials: samplingTutorial.m, upSample.m

320: Sampling Signals c A.D. Jepson and D.J. Fleet, 2005 Page: 1 Sampling Signals

I(x) continuous signal

I[n] ● ● ● ● ● sampled ● ● ● ● ● ● ● ● ● signal ● ● ● ● n

g[m] downsampling ● ● ● ● ● ● ● I[n] 2 g[m] ● ● m

f[n] ● ● upsampling ● ● ● ● ● ● ● g[m] 2 f[n] ● ● ● ● ● ● ● ● ● n

320: Sampling Signals Page: 2

Up-Sampling

[Ò] Æ

Consider up-sampling a signal Á of length :

¯ Æ Ò

Increase number of samples by a factor of × .

¯ Ò ½ Á [Ò] f [Ò] ¼

Step 1. Place × zeros after every sample of , to form

Ò Æ

of length × , namely

Á [Ò=Ò ] Ò ÑÓ d Ò = ¼; ×

× for

f [Ò] =

¼ (1)

¼ otherwise.

Signal Raw Upsampled Signal 1.5 2

1.5 1

1 0.5 0.5 0 0 f 0 I[n] −0.5 −0.5 −1

−1.5 −1

−2 −1.5 0 5 10 15 0 5 10 15 20 25 30 n n

320: Sampling Signals Page: 3

Up-Sampling (Cont.)

¯ f

Step 2. Interpolate signal ¼ :

f = Ë £ f ; Ë [Ò]:

¼ for some smoothing kernel (2)

f [Ò] f [Ò] Ë [¼] = ½

Here, in order for to interpolate ¼ , we require that

Ë [jÒ ] = ¼ j

and × for nonzero integers .

Sinc Filtered Upsampled Signal (b) 1.5

0.5

0 f[n]

−0.5

−1

−1.5 0 5 10 15 20 25 30 n

¯ Many smoothing kernels can be used (see upSample.m).

320: Sampling Signals Page: 4

Frequency Analysis of Up-Sampling

f [Ò]

Step 1. Fourier transform of the raw up-sampled signal ¼ :

ÆÒ ½

Æ ½

X X

× ×

Ò i! jÒ i!

k k

F ´f [Ò]µ[k ] = f [Ò]e f [jÒ ]e

¼ ¼ ¼ ×

Ò=¼

j =¼

Æ ½

i! jÒ

= Á [j ]e

j =¼

k =

Here ! , so the last line above becomes

ÆÒ

Æ ½

ÆÒ ÆÒ

× ×

kj i

F ´Á µ[k ]; F ´f µ[k ] = Á [j ]e k < :

¼ for (3)

¾ ¾

j =¼

F ´Á µ Æ F ´f µ

Since is -periodic, equation (3) implies that ¼ consists of

Ò F ´Á µ

× copies of concatenated together.

Amplitude Spectrum of I[n] Amplitude Spectrum of Raw Upsampled Signal 6 6

5 5

4 4

3 3 |FT| |FT|

2 2

1 1

0 0 −5 0 5 −15 −10 −5 0 5 10 15 k k

320: Sampling Signals Page: 5

Fourier Analysis of Up-Sampling Step 2

f [Ò] = Ë £ f Ë

Recall Step 2 is to form ¼ , for some interpolation ﬁlter .

However, notice from the inverse Fourier transform that (for Æ even)

Æ=¾ ½

kj i

Á [j ] = Á [k ]e

k = Æ=¾

Æ=¾ ½

i kj Ò

ÆÒ

f [k ]e = f [jÒ ]: = ¼

× (4)

ÆÒ

k = Æ=¾

f ´jÒ µ = Á ´j µ

Here we used × in the last line. Notice the left term in the

Ò B [k ]f [k ] ¼

last line above is × times the inverse Fourier transform of

B [k ] = ½ Æ=¾ k < Æ=¾

where B is the box function, for and

[k ] = ¼

B otherwise. We can therefore evaluate this inverse Fourier

Ò jÒ

transform at every pixel , and not just at the interpolation values × ,

[Ò]

to construct a possible interpolating function f

f [Ò] = Ò F ´B ´k µf [k ]µ: ¼ × (5)

Sinc Filtered Upsampled Signal (b) Amp. Spec. (Upsampled Sig.(r), Sinc Filter(g), Filtered Sig.(b) 1.5 12

1 10

0.5 8

0 6 f[n] |FT|

4 −0.5

2 −1

0 −1.5 −15 −10 −5 0 5 10 15 0 5 10 15 20 25 30 k n 320: Sampling Signals Page: 6

Down-Sampling

[Ò] Æ

Consider down-sampling a signal Á of length :

¯ Æ Ò Ò × Reduce number of samples by a factor of × , where is a divi-

sor of Æ .

¯ Deﬁne the comb function:

´Æ=Ò µ ½

Æ C ´Ò; Ò µ =

Ò;ÑÒ ×

Ñ=¼

I [n] ns ns = 4 ●●●●●●●●●●●●●●●●●

Á [Ò]

¯ Step 1. Introduce zeros in at unwanted samples.

g [Ò] = C [Ò; Ò ]Á [Ò]: ×

¼ (6)

¯ g

Step 2. Downsample signal ¼ :

g [Ñ] = g [ÑÒ ]; ¼ Ñ < Æ=Ò :

× × ¼ for (7)

320: Sampling Signals Page: 7 Frequency Domain Analysis of Down-Sampling

Proposition 1. The Fourier transform of the comb function is another

comb function:

F ´C [Ò; Ò ]µ = C [k ; Æ=Ò ]: ×

× (8)

Ò ×

∧ ω C( ) 2π n = 4 ns s

−π 0 π ω

k ! = k

Recall the frequency ! and wave number are related by .So

¾ Æ ¾

! = =

the spacing in the plot above is × .

Æ Ò Ò

× ×

Proposition 2. Pointwise product and convolution of Fourier trans-

[Ò] g [Ò] Æ forms. Suppose f and are two signals of length (extended to

be Æ -periodic). Then

F ´f µ £F´g µ F ´f [Ò]g [Ò]µ =

Æ=¾ ½

f [j ] g^[k j ]:

(9)

j = Æ=¾

g^ f g where f and denote the Fourier transforms of and , respectively.

The proofs of these two propositions are straight forward applications of the deﬁnition of the Fourier transform given in the preceeding notes, and are left as exercises.

320: Sampling Signals Page: 8

Fourier Analysis of Down-Sampling Step 1

g [Ò] = C [Ò; Ò ]Á [Ò] × Recall Step 1 is to form ¼ . By Prop. 1 and 2 above,

we have

F ´g µ = F ´C [Ò; Ò ]Á [Ò]µ

¼ ×

½ Æ

C [¡; Æ=Ò ] £ Á [¡] =

Ò Æ

Æ=¾

C [j ; Æ=Ò ]Á [k j ]

j = Æ=¾·½

Ò Ö

½ Æ

= Á [k Ö ;

] (10)

Ò Ò

× ×

Ö = Ö ·½

Á [k ] Ö

where, due to the periodicity of , we can use any integer ¼ (eg.

Ö = Ò =¾ Ò

× × ¼ for even ).

∧ ∧ I(ω) g (ω) 2π 0 n s ns = 4

−π 0 π ω −π 0 π ω

g^ [k ] Á [k ÖÆ=Ò ] ¼

Therefore consists of the sum of replicas × of the

Á Æ=Ò

Fourier transform of the original signal , spaced by wavenumber ×

! =

or, equivalently, by frequency × .

g^ ¾

Note the Fourier transform ¼ has period , so the contribution sketched

¾ above for ! can be shifted to the left.

320: Sampling Signals Page: 9

Fourier Analysis of Down-Sampling Step 2

g [Ò]

In Step 2 we simply drop the samples from ¼ which were set to zero

C [Ò; Ò ]

by the comb function × . That is

g [Ñ] = g [ÑÒ ]; ¼ Ñ < Æ=Ò :

× × ¼ for

In terms of the Fourier transform, it is easy to show

F ´g µ[k ] = g^[k ] = g^ [k ]; Æ=´¾Ò µ k < Æ=´¾Ò µ:

× ×

¼ for (11)

! = k g [Ñ]

Rewriting this in terms of the frequency ×;k (note is

´Æ=Ò µ

Æ=Ò ! = k k

a signal of length × ), and the corresponding frequency

g [Ò] = ! Ò

¼ ×

×;k of the longer signal ,wehave

g^[! ] = g^ [! ] = g^ [! =Ò ]; ! < :

¼ k ¼ ×;k × ×;k ×;k for (12)

∧ ω f( ) ns = 4 original signal −π ω π ω 0 s

∧ g(ω) downsampled signal

−π 0 π 4π ω

∧ ω f( ) upsampled and filtered signal −π 0 π ω

320: Sampling Signals Page: 10

Nyquist Sampling Theorem

[Ò]

Sampling Theorem: Let f be a band-limited signal such that

f [! ] = ¼ j! j > !

for all ¼

! f [Ò] g [Ñ] =

for some ¼ . Then is uniquely determined by its samples

f [ÑÒ ]

× when

> ! ! =¾ = Ò <

¼ ×

× or equivalently

Ò ¾

= ¾=! ¼ where ¼ . In words, the distance between samples must be

smaller than half a wavelength of the highest frequency in the signal. !

In terms of the previous ﬁgure, note that the maximum frequency ¼ ! must be smaller than one-half of the spacing, × , between the replicas

introduced by the sampling. This ensures the replicas do not overlap.

[Ñ]

When the replicas do not overlap, we can up-sample the signal g

[Ò]

and interpolate it to recover the signal f , as discussed above.

g [k ]

Otherwise, when the replicas overlap, the Fourier transform ^ con-

[k ]

tains contributions from more than one replica of f . Due to these

[Ò] aliased contributions, we cannot then recover the original signal f .

320: Sampling Signals Page: 11

Sampling Continuous Signals

´Üµ Ü ¾ [¼;Äµ A similar theorem holds for sampling signals f for .We

can represent f as the Fourier series

i! Ü

f [k ]e ; f ´Üµ =

a:e:

k = ½

! = k = a:e:

where k and denotes equals almost everywhere. Suppose

f ´Üµ ! > ¼

is band-limited so that for some ¼

j! j > ! : f [k ] = ¼ ¼

for all k

´Üµ Á [Ñ] = f ´Ñµ

Then f is uniquely determined by its samples when

¼ <

(13)

= ¾=! ¼ where ¼ . In words, the distance between samples must be

smaller than half the wavelength of the highest frequency in the signal.

´Üµ The link with the preceeding analysis is that sampling f withasam-

ple spacing of causes replicas in the Fourier transform to appear with

! = ¾=

spacing × . As before, the condition that these replicas do not

! < ! =¾ × overlap is ¼ , which is equivalent to condition (13).

320: Sampling Signals Page: 12 Aliasing

Aliasing occurs when replicas overlap:

Consider a perspective image of an inﬁnite checkerboard. The signal is dominated by high frequencies in the image near the horizon. Properly designed cameras blur the signal before sampling, using

¯ the point spread function due to diffraction,

¯ imperfect focus,

¯ averaging the signal over each CCD element. These operations attenuate high frequency components in the signal. Without this (physical) preprocessing, the sampled image can be severely aliased (corrupted):

320: Sampling Signals Page: 13 Dimensionality A guiding principal throughout signal transforms, sampling, and aliasing is the underlying dimension of the signal, that is, the number of linearly independent degress of freedom (dof). This helps clarify many

issues that might otherwise appear mysterious.

Æ Æ ¯ Real-valued signals with samples have dof. We need a basis

of dimension Æ to represent them uniquely.

Æ Æ ¯ Why did the DFT of a signal of length use sinusoids? Be-

cause Æ sinusoids are linearly independent, providing a minimal Æ spanning set for signals of length Æ . We need no more than .

¯ But wait: Fourier coefﬁcients are complex-valued, and therefore Æ have ¾ dofs. This matches the dof needed for complex signals of

length Æ but not real-valued signals. For real signals the Fourier spectra are symmetric, so we keep half of the coefﬁcients.

¯ When we down-sample a signal by a factor of two we are moving ¾

to a basis with Æ= dimensions. The Nyquist theorem says that ¾ the original signal should lie in an Æ= dimensional space before you down-sample. Otherwise information is corrupted (i.e. signal

structure in multiple dimensions of the original Æ -D space appear ¾ thesameintheÆ= -D space).

¯ The Nyquist theorem is not primarily about highest frequencies and bandwidth. The issue is really one of having a model for the signal; that is, how many non-zero frequency components are in the signal (i.e., the dofs), and which frequencies are they.

320: Sampling Signals Page: 14