MATH 262/CME 372: Applied Fourier Analysis and Winter 2021 Elements of Modern Signal Processing Lecture 5 | January 26, 2021 Prof. Emmanuel Candes Scribe: Carlos A. Sing-Long, Edited by E. Bates 1 Outline Agenda: Sampling of analog signals 1. Poisson summation formula 2. Aliasing formula 3. Shannon's sampling theorem Last Time: We proved Weyl-Heisenberg's uncertainty principle, which establishes that a sig- nal cannot be concentrated simultaneously in both the time domain and the frequency domain. Although this is a mathematical statement, it has profound implications in quantum mechanics. In order to understand this connection, we introduced the formalism of quantum mechanics. We defined observables as hermitian operators acting on a Hilbert space, and their complete set of eigenvectors allows us to represent the state of a quantum system in different bases. Two observ- ables of fundamental interest are the position and momentum operators, and the Fourier transform serves as a change of variables between the two associated bases. In particular, a computation showed that when expressed in the position basis, each eigenvector of the momentum operator is a complex exponential. Consequently, the uncertainty principle shows that one cannot measure simultaneously the position and momentum of a quantum state with arbitrarily high precision. 2 Motivation for sampling In many real world applications one is interested in measuring physical magnitudes that vary continuously in time. These are called analog signals or continuous-time signals and are naturally modeled as a continuous function f(t). However, due to physical limitations of the measurement process, often one cannot acquire the complete set of values ff(t)gt2R for the function of interest. Therefore, one needs to sample the signal, that is, acquire only a discrete set of values. A classical approach to model this process is to assume one measures the values of f at time intervals of length T . We call 1=T the sampling rate or sampling frequency. In this case, the discrete sequence of values ff(nT )gn2Z represents the acquired information. In practice, the values are measured (and discretized) by analog-to-digital converters. A natural question then follows: can we recover f from the collection of samples ff(nT )gn2Z? The answer is no, of course, in general. After all, given a sample sequence f(0); f(1); f(2);::: , there 1 f f t t Analog signal Sampling grid fd t Discretized signal Figure 1: Discretization of analog signals and representation of the discretized signal are infinitely many functions f from which such a sequence could have come. A trivial example is sampling f(t) = sin(πt) on the integers. Every sample returns the value 0. But then the signal could just as well have been tan(πt), or 2 sin(πt), or just the zero function! So a priori the task of recovery may be hopeless. Remarkably, though, exact recovery from discrete samples is possible upon adding one condition, namely that f is bandlimited. This is the property of f^ vanishing outside a finite interval [−!b;!b], which means f contains no content of frequency greater than !b. In this case, f^ is said to have compact support. Central to understanding why a bandlimited signal can be reconstructed from discrete samples is the phenomenon of aliasing. And to determine how and when aliasing arises, we shall employ the beautiful Poisson summation formula. 3 Poisson summation formula The Poisson summation formula is no more than a computation of the Fourier transform of a special type of discrete (generalized) function. While the calculation it makes is very specific, we shall see just how important that calculation is. Consider the distribution X f(t) = δ(t − nT ); n2Z which we call the Dirac comb. It is a superposition of Dirac delta functions placed on an equi- spaced grid with spacing T > 0 and containing the origin. By construction, it is T -periodic. We can formally compute the spectrum (synonym for Fourier transform) of f to obtain X f^(!) = e−inT !; n2Z which is 2π=T -periodic. The Poisson summation formula states that f^ is also a Dirac comb. 2 Theorem 1 (Poisson summation formula). For T > 0 we have X 2π X 2π e−inT ! = δ ! − n ; T T n2Z n2Z where equality holds in the distributional sense. That is, ! Z X 2π X 2πn e−inT ! '(!) d! = ' ; T T n2Z n2Z for any smooth and rapidly decaying test function '. T 2⇡/T 2π Time domain: Dirac comb of period T Frequency domain: Dirac comb of period T Figure 2: The Poisson summation formula states that the Fourier transform of a Dirac comb of period T is a Dirac comb of period 2π=T (modulo a factor of 2π=T ). Proof. We prove the result for T = 1. The general case follows by a suitable rescaling, due to the identity 1 ! F (t) = f(T t) ) F^(!) = f^ : T T As noted before, f^ is 2π-periodic. Therefore, we need to prove that the restriction of f^ to [−π; π] is proportional to a Dirac delta at the origin. In other words, that N ! Z π X lim e−in! '(!) d! = 2π'(0); N!1 −π n=−N for ' smooth and rapidly decaying. Summing the geometric series, we have (2N+1)! N i(2N+1)! sin X −in! −iN! 1 − e 2 e = e = = DN (!); 1 − ei! sin ! n=−N 2 where (2N+1)! sin 2 DN (!) = ! sin 2 is known as the Dirichlet kernel. Using this notation, we need to show that Z π lim DN (!)'(!) d! = 2π'(0): N!1 −π 3 We rewrite the integral above as 2 (2N+1)! 3 Z π Z π sin " ! # 2N + 1 2 2 DN (!)'(!) d! = 4 (2N+1)! 5 ! '(!) d!: −π −π 1 sin 2 2 1 Now, if b(t) = Ifjtj ≤ 2 g is the boxcar function, then (2N+1)! 2N + 1 sin 2 = (2N + 1)^b((2N + 1)!); 1 (2N+1)! 2 !=2 which is the Fourier transform of b(t=(2N + 1)). Next note that sin(!=2) is bounded for ! 2 [−π; π], and so we can define !=2 θ^(!) := Ifj!j ≤ πg '(!): sin(!=2) We may now extend the integral to the whole real line and use the Parseval-Plancherel theorem to conclude Z π Z Z 1 N!1 ^ DN (!)'(!) d! = 2π θ(t)I jtj ≤ N + dt −−−−! 2π θ(t) dt = 2πθ(0): −π 2 Since by construction θ^(0) = '(0), this proves the claim. 4 Aliasing formula Aliasing arises when discretization of a signal f causes additive distortion in the spectrum. The aliasing formula shall make this precise. Represent the discretized signal coming from the set of samples ff(nT )gn2Z by the distribution X fd(t) = f(nT )δ(t − nT ): n2Z (Other representations are briefly discussed in the appendix.) Formally, we know the Fourier transform of fd: X −inT ! f^d(!) = f(nT )e : n2Z Perhaps surprisingly, f^d can be expressed in terms of f^, the true spectrum of f. Proposition 2 (aliasing formula). The spectrum of the discrete signal fd is 1 X 2π f^ (!) = f^ ! − k : d T T k2Z 4 Proof. We prove the case T = 1. The general case follows by scaling. We have X X fd(t) = f(n)δ(t − n) = f(t) δ(t − n); n2Z n2Z where we used the fact that f(n)δ(t − n) = f(t)δ(t − n) where the equality is in the sense of distributions. From this it follows that !! 1 X X f^ (!) = f^∗ 2π δ (!) = f^(! − 2πk); d 2π 2πk k2Z k2Z where we have used the Poisson summation formula. The aliasing formula shows that sampling discretely in time corresponds to replicating the frequency content of the signal at intervals of size 2π=T and adding the contributions of all replicas (modulo a factor of 1=T ). This is known as the 2π=T -periodization of f^. sampling a signal at time intervals of size T m periodizing the Fourier transform by summing ^ the translates ff(! − 2πk=T )gk2Z and multipliying by 1=T Intuitively, low sampling rates (i.e. large values of T ) will cause substantial overlap between the replica copies of f^, making the actual spectrum difficult to detect. On the other hand, high sampling rates reduce this overlap (since f^ should decay at high frequencies), better revealing a single copy of f^ within the spectrum of the discretized signal. In the optimal situation in which f is bandlimited (f^ actually vanishes at high frequencies), the overlap can be avoided entirely, allowing for the exact recovery of f^ and thus of f. In this case, the Shannon interpolation theorem will tell us how to perform the reconstruction. 5 Shannon sampling theorem The aliasing formula has shown us where the spectrum of f can be found in the spectrum of its discretization fd. We have seen that in the case that f is bandlimited, the true spectrum f^ can be seen in f^d, undisturbed by aliasing, provided samples have been acquired at a sufficiently high frequency. The Shannon sampling (or interpolation) theorem is simply the calculation that extracts f^ and then recovers f by inversion. Theorem 3 (Shannon sampling). Suppose f is bandlimited in [−π=T; π=T ] for some T > 0. Then X t f(t) = f(nT )hT (t − nT ) where hT (t) = sinc T n2Z Proof. Once again we assume T = 1, as the general case follows by rescaling. The support of f^ is contained in [−π; π]. So for any integer k =6 0, the support of f^(! − 2πk) does not overlap with the support of f^(!).