6.003: Signal Processing

Sampling and Reconstruction

September 13, 2018 Importance of Discrete Representations

Our goal is to develop signal processing tools that help us under- stand and manipulate the world.

analyze Model Result (math, computation)

make model interpret results

World New Understanding

analysis

System Behavior

design

The increasing power and decreasing cost of computation makes the use of computation increasingly attractive. However, many impor- tant signals are naturally described with continuous domain.

→ understand the relation between CT and sampled signals Sampling

Converting continuous-time (CT) signals to discrete time (DT).

x(t) x[n] = x(nT )

t n 0T 2T 4T 6T 8T 10T 0 2 4 6 8 10

T = sampling interval

How does sampling affect the information contained in a signal? Sampling

We would like to sample in a way that preserves information. However, information is clearly lost in the sampling process.

x(t)

t

From this example, it’s clear that uniformly spaced samples tell us nothing about the values of x(t) between the samples. Sampling

Even worse, information can be distorted, becoming misinformation. 7π Samples of an original signal x(t) = cos 3 n

7π π cos 3 n? cos 3 n?

t

π are just as easily interpreted as coming from cos 3 n. Effects of Undersampling Are Easily Heard

Sampling Music

1 f = s T

• fs = 44.1 kHz

• fs = 22 kHz

• fs = 11 kHz

• fs = 5.5 kHz

• fs = 2.8 kHz

J.S. Bach, Sonata No. 1 in G minor Mvmt. IV. Presto Nathan Milstein, violin Effects of Undersampling are Easily Seen

Sampling Images

original: 2048 × 1536 Effects of Undersampling are Easily Seen

Sampling Images

downsampled: 1024 × 768 Effects of Undersampling are Easily Seen

Sampling Images

downsampled: 512x384 Effects of Undersampling are Easily Seen

Sampling Images

downsampled: 256 × 192 Effects of Undersampling are Easily Seen

Sampling Images

downsampled: 128 × 96 Effects of Undersampling are Easily Seen

Sampling Images

downsampled: 64 × 48 Effects of Undersampling are Easily Seen

Sampling Images

downsampled: 32 × 24 Effects of Undersampling

If we hope to use sampled signals to understand behaviors of con- tinuous signals, we must understand effects of sampling. Effects of Undersampling

If we hope to use sampled signals to understand behaviors of con- tinuous signals, we must understand effects of sampling.

Some of the effects of undersampling arise from fundamental prop- erties of discrete time.

CT: x(t) = cos ωt

DT: x[n] = cos Ωn

These signals differ in important ways. Check Yourself Compare two signals: 3πn x [n] = cos 1 4 5πn x [n] = cos 2 4

How many of the following statements are true?

x1[n] has period N=8.

x2[n] has period N=8.

x1[n] = x2[n]. Check Yourself

3πt cos 4 = cos ω1t

t

5πt cos 4 = cos ω2t

t

ω1 6= ω2 but x1[n] = x2[n]!

ω1 + ω2 = 2π

cos ω1n = cos(2π − ω2)n = cos(2πn − ω2n) = cos(−ω2n) = cos ω2n Check Yourself Compare two signals: 3πn x [n] = cos 1 4 5πn x [n] = cos 2 4

How many of the following statements are true? 3 √ x1[n] has period N=8. √ x2[n] has period N=8. √ x1[n] = x2[n]. Check Yourself

Consider 3 CT signals:

x1(t) = cos(4000t) ; x2(t) = cos(5000t) ; x3(t) = cos(6000t) Each of these is sampled so that

x1[n] = x1(nT ) ; x2[n] = x2(nT ) ; x3[n] = x3(nT ) where T = 0.001.

Which list goes from lowest to highest DT frequency?

0. x1[n] x2[n] x3[n] 1. x1[n] x3[n] x2[n]

2. x2[n] x1[n] x3[n] 3. x2[n] x3[n] x1[n]

4. x3[n] x1[n] x2[n] 5. x3[n] x2[n] x1[n] Check Yourself

The CT signals are simple sinusoids:

x1(t) = cos(4000t) ; x2(t) = cos(5000t) ; x3(t) = cos(6000t)

The DT signals are sampled versions (T = 0.001):

x1[n] = cos(4n) ; x2[n] = cos(5n) ; x3[n] = cos(6n)

How does the shape of cos(Ωn) depend on Ω? Check Yourself

As the frequency Ω increases, the shapes of the sampled signals deviate from those of the underlying CT signals.

Ω = 1 : x[n] = cos(n)

n

Ω = 2 : x[n] = cos(2n)

n

Ω = 3 : x[n] = cos(3n)

n Check Yourself

Worse and worse representation.ΩL Ω = 4 : x[n] = cos(4n) = cos(2π − 4n) ≈ cos(2.283n)

n

Ω = 5 : x[n] = cos(5n) = cos(2π − 5n) ≈ cos(1.283n)

n

Ω = 6 : x[n] = cos(6n) = cos(2π − 6n) ≈ cos(0.283n)

n

The same DT sequence represents many different values of Ω. Check Yourself

For Ω > π, a lower frequency ΩL has the same sample values as Ω. Ω = 4 : x[n] = cos(4n) = cos(2π − 4n) ≈ cos(2.283n)

n

Ω = 5 : x[n] = cos(5n) = cos(2π − 5n) ≈ cos(1.283n)

n

Ω = 6 : x[n] = cos(6n) = cos(2π − 6n) ≈ cos(0.283n)

n

The same DT sequence represents many different values of Ω. Check Yourself

Although there are multiple frequencies Ω with the same samples, only one of these frequencies is in the range 0 ≤ Ω ≤ π. Thus we can remove the ambiguity of which frequency is represented by a set of samples by choosing the one in the range 0 ≤ Ω ≤ π. We call that range of frequencies the base band of frequencies.

If we limit our attention to frequencies in the base band, then there is a maximum possible discrete frequency Ωmax = π. Note this difference from CT, where there is no maximum frequency. Check Yourself

Consider 3 CT signals:

x1(t) = cos(4000t) ; x2(t) = cos(5000t) ; x3(t) = cos(6000t) Each of these is sampled so that

x1[n] = x1(nT ) ; x2[n] = x2(nT ) ; x3[n] = x3(nT ) where T = 0.001.

Which list goes from lowest to highest DT frequency? 5

0. x1[n] x2[n] x3[n] 1. x1[n] x3[n] x2[n]

2. x2[n] x1[n] x3[n] 3. x2[n] x3[n] x1[n]

4. x3[n] x1[n] x2[n] 5. x3[n] x2[n] x1[n] Relation Between CT and DT Frequencies

If a CT signal x(t) = cos ωt is sampled at times t = nT , the resulting DT signal is x[n] = cos Ωn where Ω = ωT .

If we restrict DT frequencies to the range 0 ≤ Ω ≤ π, then the corresponding CT frequencies are in the range 0 ≤ ω ≤ ωN where π ω = N T and ω 1 1 f = N = = f N 2π 2T 2 s where fN is the Nyquist frequency, which is half the sample rate fs. Base Band Reconstruction

If all frequencies in a CT signal are less than the Nyquist frequency, then we can reconstruct the original CT signal perfectly by discarding frequencies in the reconstruction that exceed the Nyquist frequency.

This is called base-band reconstruction. Anti-Aliasing

If there are frequencies in the CT signal that are greater than the Nyquist frequency, then these will alias to frequencies in the base band that have nothing to do with the original frequency.

These should be removed before sampling. Anti-Aliasing Demonstration

Sampling Music

2π ω = = 2πf s T s

• fs = 11 kHz without anti-aliasing

• fs = 11 kHz with anti-aliasing

• fs = 5.5 kHz without anti-aliasing

• fs = 5.5 kHz with anti-aliasing

• fs = 2.8 kHz without anti-aliasing

• fs = 2.8 kHz with anti-aliasing

J.S. Bach, Sonata No. 1 in G minor Mvmt. IV. Presto Nathan Milstein, violin Quantization

The information content of a signal scales not only with sample rate but also with the number of bits used to represent each sample.

1

e 2 bits10 3 bits 4 bits g a t l

o 01 v 0 t u p t u

O ­1 00 ­1 0 1 ­1 0 1 ­1 0 1 Input voltage Input voltage Input voltage

1

0

­1 0 0.5 1 0 0.5 1 0 0.5 1 Time (second) Time (second) Time (second)

Bit rate = (# bits/sample)×(# samples/sec) Check Yourself

We hear sounds that range in amplitude from 1,000,000 to 1.

How many bits are needed to represent this range?

1. 5 bits 2. 10 bits 3. 20 bits 4. 30 bits 5. 40 bits Check Yourself

How many bits are needed to represent 1,000,000:1? bits range 1 2 2 4 3 8 4 16 5 32 6 64 7 128 8 256 9 512 10 1, 024 11 2, 048 12 4, 096 13 8, 192 14 16, 384 15 32, 768 16 65, 536 17 131, 072 18 262, 144 19 524, 288 20 1, 048, 576 Check Yourself

We hear sounds that range in amplitude from 1,000,000 to 1.

How many bits are needed to represent this range? 3

1. 5 bits 2. 10 bits 3. 20 bits 4. 30 bits 5. 40 bits Quantization Demonstration

Quantizing Music

• 16 bits/sample

• 8 bits/sample

• 6 bits/sample

• 4 bits/sample

• 3 bits/sample

• 2 bit/sample

J.S. Bach, Sonata No. 1 in G minor Mvmt. IV. Presto Nathan Milstein, violin Quantization Demonstration

Quantizing Music

• 16 bits/sample

• 8 bits/sample

• 6 bits/sample

• 4 bits/sample

• 3 bits/sample

• 2 bit/sample

J.S. Bach, Sonata No. 1 in G minor Mvmt. IV. Presto Nathan Milstein, violin Quantization Demonstration

Quantizing Music

• 16 bits/sample

• 8 bits/sample

• 6 bits/sample

• 4 bits/sample

• 3 bits/sample

• 2 bit/sample

J.S. Bach, Sonata No. 1 in G minor Mvmt. IV. Presto Nathan Milstein, violin Quantization Demonstration

Quantizing Music

• 16 bits/sample

• 8 bits/sample

• 6 bits/sample

• 4 bits/sample

• 3 bits/sample

• 2 bit/sample

J.S. Bach, Sonata No. 1 in G minor Mvmt. IV. Presto Nathan Milstein, violin Quantization Demonstration

Quantizing Music

• 16 bits/sample

• 8 bits/sample

• 6 bits/sample

• 4 bits/sample

• 3 bits/sample

• 2 bit/sample

J.S. Bach, Sonata No. 1 in G minor Mvmt. IV. Presto Nathan Milstein, violin Quantization Demonstration

Quantizing Music

• 16 bits/sample

• 8 bits/sample

• 6 bits/sample

• 4 bits/sample

• 3 bits/sample

• 2 bit/sample

J.S. Bach, Sonata No. 1 in G minor Mvmt. IV. Presto Nathan Milstein, violin Quantization Demonstration

Quantizing Music

• 16 bits/sample

• 8 bits/sample

• 6 bits/sample

• 4 bits/sample

• 3 bits/sample

• 2 bit/sample

J.S. Bach, Sonata No. 1 in G minor Mvmt. IV. Presto Nathan Milstein, violin Quantization Demonstration

Quantizing Music

• 16 bits/sample

• 8 bits/sample

• 6 bits/sample

• 4 bits/sample

• 3 bits/sample

• 2 bit/sample

J.S. Bach, Sonata No. 1 in G minor Mvmt. IV. Presto Nathan Milstein, violin Quantizing Images

Converting an image from a continuous representation to a discrete representation involves the same sort of issues.

This image has 280 × 280 pixels, with brightness quantized to 8 bits. Quantizing Images

8 bit image 7 bit image Quantizing Images

8 bit image 6 bit image Quantizing Images

8 bit image 5 bit image Quantizing Images

8 bit image 4 bit image Quantizing Images

8 bit image 3 bit image Quantizing Images

8 bit image 2 bit image Quantizing Images

8 bit image 1 bit image Check Yourself

What is the most objectionable artifact of coarse quantization?

8 bit image 4 bit image Dithering

One very annoying artifact is banding caused by clustering of pixels that quantize to the same level.

Banding can be reduced by dithering. 1 Dithering: adding a small amount (± 2 quantum) of random noise to the image before quantizing.

Since the noise is different for each pixel in the band, the noise causes some of the pixels to quantize to a higher value and some to a lower. But the average value of the brightness is preserved. Quantizing Images with Dither

7 bit image 7 bits with dither Quantizing Images with Dither

6 bit image 6 bits with dither Quantizing Images with Dither

5 bit image 5 bits with dither Quantizing Images with Dither

4 bit image 4 bits with dither Quantizing Images with Dither

3 bit image 3 bits with dither Quantizing Images with Dither

2 bit image 2 bits with dither Quantizing Images with Dither

1 bit image 1 bit with dither Check Yourself

What is the most objectionable artifact of dithering?

3 bit image 3 bit dithered image Check Yourself

What is the most objectionable artifact of dithering?

One annoying feature of dithering is that it adds noise. Quantization Schemes

Example: slowly changing backgrounds.

Quantization: y = Q(x)

Quantization with dither: y = Q(x + n) Check Yourself

What is the most objectionable artifact of dithering?

One annoying feature of dithering is that it adds noise. 1 Robert’s technique: add a small amount (± 2 quantum) of random noise before quantizing, then subtract that same amount of random noise. Quantization Schemes

Example: slowly changing backgrounds.

Quantization: y = Q(x)

Quantization with dither: y = Q(x + n)

Quantization with Robert’s technique: y = Q(x + n) − n Quantizing Images with Robert’s Method

7 bits with dither 7 bits with Robert’s method Quantizing Images with Robert’s Method

6 bits with dither 6 bits with Robert’s method Quantizing Images with Robert’s Method

5 bits with dither 5 bits with Robert’s method Quantizing Images with Robert’s Method

4 bits with dither 4 bits with Robert’s method Quantizing Images with Robert’s Method

3 bits with dither 3 bits with Robert’s method Quantizing Images with Robert’s Method

2 bits with dither 2 bits with Robert’s method Quantizing Images with Robert’s Method

1 bits with dither 1 bit with Robert’s method Quantizing Images: 3 bits

8 bits 3 bits

dither Robert’s Quantizing Images: 2 bits

8 bits 2 bits

dither Robert’s Quantizing Images: 1 bit

8 bits 1 bit

dither Robert’s