Lecture 5 — January 26, 2021 1 Outline 2 Motivation for Sampling

Home , Dirac comb

MATH 262/CME 372: Applied Fourier Analysis and Winter 2021 Elements of Modern Signal Processing Lecture 5 — January 26, 2021 Prof. Emmanuel Candes Scribe: Carlos A. Sing-Long, Edited by E. Bates

1 Outline

Agenda: Sampling of analog signals

1. Poisson summation formula

2. Aliasing formula

3. Shannon’s sampling theorem

Last Time: We proved Weyl-Heisenberg’s uncertainty principle, which establishes that a signal cannot be concentrated simultaneously in both the time domain and the frequency domain. Although this is a mathematical statement, it has profound implications in quantum mechanics. In order to understand this connection, we introduced the formalism of quantum mechanics. We deﬁned observables as hermitian operators acting on a Hilbert space, and their complete set of eigenvectors allows us to represent the state of a quantum system in diﬀerent bases. Two observables of fundamental interest are the position and momentum operators, and the Fourier transform serves as a change of variables between the two associated bases. In particular, a computation showed that when expressed in the position basis, each eigenvector of the momentum operator is a complex exponential. Consequently, the uncertainty principle shows that one cannot measure simultaneously the position and momentum of a quantum state with arbitrarily high precision.

2 Motivation for sampling

In many real world applications one is interested in measuring physical magnitudes that vary continuously in time. These are called analog signals or continuous-time signals and are naturally modeled as a continuous function f(t). However, due to physical limitations of the measurement process, often one cannot acquire the complete set of values {f(t)}t∈R for the function of interest. Therefore, one needs to sample the signal, that is, acquire only a discrete set of values. A classical approach to model this process is to assume one measures the values of f at time intervals of length T . We call 1/T the sampling rate or sampling frequency. In this case, the discrete sequence of values {f(nT )}n∈Z represents the acquired information. In practice, the values are measured (and discretized) by analog-to-digital converters.

A natural question then follows: can we recover f from the collection of samples {f(nT )}n∈Z? The answer is no, of course, in general. After all, given a sample sequence f(0), f(1), f(2),... , there

1 f f

t t Analog signal Sampling grid

t Discretized signal

Figure 1: Discretization of analog signals and representation of the discretized signal are inﬁnitely many functions f from which such a sequence could have come. A trivial example is sampling f(t) = sin(πt) on the integers. Every sample returns the value 0. But then the signal could just as well have been tan(πt), or 2 sin(πt), or just the zero function! So a priori the task of recovery may be hopeless. Remarkably, though, exact recovery from discrete samples is possible upon adding one condition, namely that f is bandlimited. This is the property of fˆ vanishing outside a ﬁnite interval [−ωb, ωb], which means f contains no content of frequency greater than ωb. In this case, fˆ is said to have compact support. Central to understanding why a bandlimited signal can be reconstructed from discrete samples is the phenomenon of aliasing. And to determine how and when aliasing arises, we shall employ the beautiful Poisson summation formula.

3 Poisson summation formula

The Poisson summation formula is no more than a computation of the Fourier transform of a special type of discrete (generalized) function. While the calculation it makes is very speciﬁc, we shall see just how important that calculation is. Consider the distribution X f(t) = δ(t − nT ), n∈Z which we call the Dirac comb. It is a superposition of Dirac delta functions placed on an equi- spaced grid with spacing T > 0 and containing the origin. By construction, it is T -periodic. We can formally compute the spectrum (synonym for Fourier transform) of f to obtain X fˆ(ω) = e−inT ω, n∈Z which is 2π/T -periodic. The Poisson summation formula states that fˆ is also a Dirac comb.

2 Theorem 1 (Poisson summation formula). For T > 0 we have

X 2π X 2π e−inT ω = δ ω − n , T T n∈Z n∈Z where equality holds in the distributional sense. That is, ! Z X 2π X 2πn e−inT ω ϕ(ω) dω = ϕ , T T n∈Z n∈Z for any smooth and rapidly decaying test function ϕ.

T 2⇡/T 2π Time domain: Dirac comb of period T Frequency domain: Dirac comb of period T Figure 2: The Poisson summation formula states that the Fourier transform of a Dirac comb of period T is a Dirac comb of period 2π/T (modulo a factor of 2π/T ).

Proof. We prove the result for T = 1. The general case follows by a suitable rescaling, due to the identity 1 ω F (t) = f(T t) ⇒ Fˆ(ω) = fˆ . T T As noted before, fˆ is 2π-periodic. Therefore, we need to prove that the restriction of fˆ to [−π, π] is proportional to a Dirac delta at the origin. In other words, that

N ! Z π X lim e−inω ϕ(ω) dω = 2πϕ(0), N→∞ −π n=−N for ϕ smooth and rapidly decaying. Summing the geometric series, we have

(2N+1)ω N i(2N+1)ω sin X −inω −iNω 1 − e 2 e = e = = DN (ω), 1 − eiω sin ω n=−N 2 where (2N+1)ω sin 2 DN (ω) = ω sin 2 is known as the Dirichlet kernel. Using this notation, we need to show that Z π lim DN (ω)ϕ(ω) dω = 2πϕ(0). N→∞ −π

3 We rewrite the integral above as

 (2N+1)ω  Z π Z π sin " ω # 2N + 1 2 2 DN (ω)ϕ(ω) dω =  (2N+1)ω  ω ϕ(ω) dω. −π −π 1 sin 2 2

1 Now, if b(t) = I{|t| ≤ 2 } is the boxcar function, then

(2N+1)ω 2N + 1 sin 2 = (2N + 1)ˆb((2N + 1)ω), 1 (2N+1)ω 2

ω/2 which is the Fourier transform of b(t/(2N + 1)). Next note that sin(ω/2) is bounded for ω ∈ [−π, π], and so we can deﬁne ω/2 θˆ(ω) := I{|ω| ≤ π} ϕ(ω). sin(ω/2) We may now extend the integral to the whole real line and use the Parseval-Plancherel theorem to conclude Z π Z Z 1 N→∞ ˆ DN (ω)ϕ(ω) dω = 2π θ(t)I |t| ≤ N + dt −−−−→ 2π θ(t) dt = 2πθ(0). −π 2

Since by construction θˆ(0) = ϕ(0), this proves the claim.

4 Aliasing formula

Aliasing arises when discretization of a signal f causes additive distortion in the spectrum. The aliasing formula shall make this precise. Represent the discretized signal coming from the set of samples {f(nT )}n∈Z by the distribution X fd(t) = f(nT )δ(t − nT ). n∈Z (Other representations are brieﬂy discussed in the appendix.) Formally, we know the Fourier transform of fd: X −inT ω fˆd(ω) = f(nT )e . n∈Z

Perhaps surprisingly, fˆd can be expressed in terms of fˆ, the true spectrum of f.

Proposition 2 (aliasing formula). The spectrum of the discrete signal fd is

1 X 2π fˆ (ω) = fˆ ω − k . d T T k∈Z

4 Proof. We prove the case T = 1. The general case follows by scaling. We have X X fd(t) = f(n)δ(t − n) = f(t) δ(t − n), n∈Z n∈Z where we used the fact that f(n)δ(t − n) = f(t)δ(t − n) where the equality is in the sense of distributions. From this it follows that !! 1 X X fˆ (ω) = fˆ∗ 2π δ (ω) = fˆ(ω − 2πk), d 2π 2πk k∈Z k∈Z where we have used the Poisson summation formula.

The aliasing formula shows that sampling discretely in time corresponds to replicating the frequency content of the signal at intervals of size 2π/T and adding the contributions of all replicas (modulo a factor of 1/T ). This is known as the 2π/T -periodization of fˆ.

sampling a signal at time intervals of size T m periodizing the Fourier transform by summing ˆ the translates {f(ω − 2πk/T )}k∈Z and multipliying by 1/T Intuitively, low sampling rates (i.e. large values of T ) will cause substantial overlap between the replica copies of fˆ, making the actual spectrum diﬃcult to detect. On the other hand, high sampling rates reduce this overlap (since fˆ should decay at high frequencies), better revealing a single copy of fˆ within the spectrum of the discretized signal. In the optimal situation in which f is bandlimited (fˆ actually vanishes at high frequencies), the overlap can be avoided entirely, allowing for the exact recovery of fˆ and thus of f. In this case, the Shannon interpolation theorem will tell us how to perform the reconstruction.

5 Shannon sampling theorem

The aliasing formula has shown us where the spectrum of f can be found in the spectrum of its discretization fd. We have seen that in the case that f is bandlimited, the true spectrum fˆ can be seen in fˆd, undisturbed by aliasing, provided samples have been acquired at a suﬃciently high frequency. The Shannon sampling (or interpolation) theorem is simply the calculation that extracts fˆ and then recovers f by inversion. Theorem 3 (Shannon sampling). Suppose f is bandlimited in [−π/T, π/T ] for some T > 0. Then X t f(t) = f(nT )hT (t − nT ) where hT (t) = sinc T n∈Z

Proof. Once again we assume T = 1, as the general case follows by rescaling. The support of fˆ is contained in [−π, π]. So for any integer k =6 0, the support of fˆ(ω − 2πk) does not overlap with the support of fˆ(ω). Consequently, the aliasing formula reveals ˆ ˆ f(ω) = fd(ω)I{|ω| ≤ π},

5 so that ! ! X X X f(t) = f(n)δn ∗ sinc (t) = f(n) sinc(t − n) = f(n)h1(t − n). n∈Z n∈Z n∈Z

f fˆ

t ! (a) Time domain: analog signal f(t) (b) Frequency domain: signal is not bandlimited

f fˆ

t !

Figure 3: The aliasing formula states that the Fourier transform of the discretized signal is equal to the periodization of the spectrum of the original signal. The dashed lines in (d) show the replicas of the original spectrum, shown in (b). The overlapping causes artifacts in the spectrum which will be signiﬁcant if, roughly speaking, the original signal is not close to bandlimited.

f fˆ t ! (a) Time domain: analog signal f(t) (b) Frequency domain: signal is bandlimited

f d fˆ t !

Figure 4: Shannon’s sampling theorem is deduced from the aliasing formula. If the original signal is bandlimited, the periodization in the Fourier domain of its spectrum does not cause any artifacts, and we can recover the original signal from its discretization by ﬁltering.

Remark 1: In order to recover a function bandlimited in [−π/T, π/T ] (i.e. bandwidth equal to 2π/T ), sampling acquisition must occur at rate 1/T (or greater), precisely the bandwidth (the factor of 2π appears because of our convention of the Fourier transform). This is called the Nyquist

6 rate. This is not to be confused with the Nyquist frequency, which is half the sampling rate (multiplied by 2π because of our convention). This is the maximum frequency that a bandlimited signal can contain in order to be recoverable from the given sampling rate. Remark 2: While we have proved that a bandlimited signal is exactly recoverable from a discrete set of samples, the collection of the samples required is still inﬁnite. This may be seen as a consequence of the fact that a function cannot be compactly supported in both the time and frequency domains. In turn, this fact can be understood as a consequence of the uncertainty principle.

A Appendix: Representation of discretized signals

Representing the discretized signal as the distribution fd allows us to deduce important results. However, why do we use a distribution instead of a function? Given the set of samples {f(nT )}n∈Z, we could choose to represent the discrete signal as X f˜d(t) = f(nT )ψ(t − nT ), (1) n∈Z T T ˜ where ψ is a smooth function, compactly supported on [− 2 , 2 ] with ψ(0) = 1. Sampling fd clearly produces the same set of samples as sampling f. And ! ! X X f˜d(t) = f(nT )ψ(t − nT ) = f(nT )δnT ∗ ψ (t) = (fd ∗ ψ)(t). n∈Z n∈Z

That is, representations of the form (1) are obtained by convolving fd with a suitable function ψ, and we can deduce results for f˜d similar to those for fd. For example, using the convolution theorem one can show the aliasing formula also applies to f˜d with an additional factor of ψˆ(ω).