Going off the grid Benjamin Recht University of California, Berkeley
Joint work with Badri Bhaskar Parikshit Shah Gonnguo Tang We live in a continuous world...
But we work with digital computers...
What is the price of living on the grid? imaging astronomy seismology
520 IEEE JOURNAL ON EMERGING AND SELECTED TOPICS IN CIRCUITS AND SYSTEMS, VOL. 2, NO. 3, SEPTEMBER 2012
spectroscopy
Fig. 6. NUS IC sampling timing and waveforms. Left panel: Interface timing between NUS die and ADC. Right panel: Simulated waveforms before and after being sampled by NIN. Horizontal scale is in nanoseconds.
spacings vary between 12 and 27 clock cycles, the instanta- neous sampling rate of the NUS receiver varies between 163 and 367 MHz, which is within the range of the 400 MHz ADC. We designed the NUS pattern to repeat every 8192 Nyquist samples, during which time there are 440 pulses which set the k sampling locations. This corresponds to an average sample rate of 236 MHz. We evaluate the quality of our pattern using a third i2⇡fj t criterion: the Fourier transform of favorable patterns will tend to have a flat, noise-like spectrum. Fig. 10 compares two NUS x(t)= cje patterns with different inter-sample spacings. The pattern shown DOA Estimation in the top plots has strong resonances across the Nyquist band. Fig. 7. NUS IC die photo. Die size is mm. In contrast, the pattern shown in the bottom plots, which has j=1 undergone a randomization of its sample locations, has a much X whiter spectrum. The flat spectrum is preferred since then all signals have equal gain.
III. DATA PROCESSING In this section, we describe our computational techniques for efficiently recovering a Nyquist-rate signal from the NUS data by filling in the missing samples. Sections III-A–III-D describe our procedures for windowing the NUS data and recovering the missing samples. Section III-E then brieflydiscussesadditional practical concerns and the computational complexity of imple- menting this reconstruction algorithm.
A. Windowing While the NUS produces an arbitrarily long sequence of samples, the recovery algorithm can only deal with a finite Fig. 8. NUS test fixture. The NUS IC is mounted on a custom pallet. Also number of them at any given time.Itis,therefore,necessary shown is the 14-bit ADC as well as various test equipment connector interfaces. to segment the data stream, and we achieve this by windowing the signal. An effective windowing process must guard against Sampling edge effects as well as the well-known spectral spreading C. Nonuniform Sample Pattern effect, which would destroy theveryFouriersparsityweseek The NUS sampling pattern is a pulse train with nonuniform to exploit. Fortunately, the concept of a perfect reconstruction spacing between pulses. It is selected to meet a list of certain cri- filter bank (PRFB) [24] can be readily adapted to our pur- teria. First, the pattern is clocked at 4.4 GHz, and reconstruction poses. A windowing procedure breaks the infinite signal into produces the equivalent of Nyquist samples taken at this rate. aseriesof(possiblyoverlapping)vectorsbyusingananalysis Second, the pulse widths and spacings must satisfy the clocking window. After signal processing, the infinite length signal can GPS Radar Ultrasoundrequirements of the ADS5474. Specifically, the minimum pulse be recovered by stitching together the finite series using the width is 6 clock cycles, the minimum spacing ( )between analysis window. Using windows from a PRFB ensures that pulses is 12 clock cycles, and the maximum spacing ( )be- the windowing process itself does not introduce any errors. tween pulses is 27 clock cycles. An example pattern illustrating An example of a PRFB is a rectangular analysis and synthesis these specifications is shown in Fig. 9. In effect, as the sample window with no overlap, but of course this causes spectral imaging astronomy seismology
520 IEEE JOURNAL ON EMERGING AND SELECTED TOPICS IN CIRCUITS AND SYSTEMS, VOL. 2, NO. 3, SEPTEMBER 2012
spectroscopy
Fig. 6. NUS IC sampling timing and waveforms. Left panel: Interface timing between NUS die and ADC. Right panel: Simulated waveforms before and after being sampled by NIN. Horizontal scale is in nanoseconds.
spacings vary between 12 and 27 clock cycles, the instanta- neous sampling rate of the NUS receiver varies between 163 and 367 MHz, which is within the range of the 400 MHz ADC. We designed the NUS pattern to repeat every 8192 Nyquist samples, during which time there are 440 pulses which set the k sampling locations. This corresponds to an average sample rate of 236 MHz. We evaluate the quality of our pattern using a third criterion: the Fourier transform of favorable patterns will tend to have a flat, noise-like spectrum. Fig. 10 compares two NUS x(t)= cjg(t ⌧j) patterns with different inter-sample spacings. The pattern shown DOA Estimation in the top plots has strong resonances across the Nyquist band. Fig. 7. NUS IC die photo. Die size is mm. In contrast, the pattern shown in the bottom plots, which has j=1 undergone a randomization of its sample locations, has a much X whiter spectrum. The flat spectrum is preferred since then all signals have equal gain.
III. DATA PROCESSING In this section, we describe our computational techniques for efficiently recovering a Nyquist-rate signal from the NUS data by filling in the missing samples. Sections III-A–III-D describe our procedures for windowing the NUS data and recovering the missing samples. Section III-E then brieflydiscussesadditional practical concerns and the computational complexity of imple- menting this reconstruction algorithm.
A. Windowing While the NUS produces an arbitrarily long sequence of samples, the recovery algorithm can only deal with a finite Fig. 8. NUS test fixture. The NUS IC is mounted on a custom pallet. Also number of them at any given time.Itis,therefore,necessary shown is the 14-bit ADC as well as various test equipment connector interfaces. to segment the data stream, and we achieve this by windowing the signal. An effective windowing process must guard against Sampling edge effects as well as the well-known spectral spreading C. Nonuniform Sample Pattern effect, which would destroy theveryFouriersparsityweseek The NUS sampling pattern is a pulse train with nonuniform to exploit. Fortunately, the concept of a perfect reconstruction spacing between pulses. It is selected to meet a list of certain cri- filter bank (PRFB) [24] can be readily adapted to our pur- teria. First, the pattern is clocked at 4.4 GHz, and reconstruction poses. A windowing procedure breaks the infinite signal into produces the equivalent of Nyquist samples taken at this rate. aseriesof(possiblyoverlapping)vectorsbyusingananalysis Second, the pulse widths and spacings must satisfy the clocking window. After signal processing, the infinite length signal can GPS Radar Ultrasoundrequirements of the ADS5474. Specifically, the minimum pulse be recovered by stitching together the finite series using the width is 6 clock cycles, the minimum spacing ( )between analysis window. Using windows from a PRFB ensures that pulses is 12 clock cycles, and the maximum spacing ( )be- the windowing process itself does not introduce any errors. tween pulses is 27 clock cycles. An example pattern illustrating An example of a PRFB is a rectangular analysis and synthesis these specifications is shown in Fig. 9. In effect, as the sample window with no overlap, but of course this causes spectral imaging astronomy seismology
520 IEEE JOURNAL ON EMERGING AND SELECTED TOPICS IN CIRCUITS AND SYSTEMS, VOL. 2, NO. 3, SEPTEMBER 2012
spectroscopy
Fig. 6. NUS IC sampling timing and waveforms. Left panel: Interface timing between NUS die and ADC. Right panel: Simulated waveforms before and after being sampled by NIN. Horizontal scale is in nanoseconds.
spacings vary between 12 and 27 clock cycles, the instanta- neous sampling rate of the NUS receiver varies between 163 and 367 MHz, which is within the range of the 400 MHz ADC. We designed the NUS pattern to repeat every 8192 Nyquist samples, during which time there are 440 pulses which set the k sampling locations. This corresponds to an average sample rate of 236 MHz. We evaluate the quality of our pattern using a third i2⇡f t criterion: the Fourier transform of favorable patterns will tend j to have a flat, noise-like spectrum. Fig. 10 compares two NUS x(t)= cjg(t ⌧j)e patterns with different inter-sample spacings. The pattern shown DOA Estimation in the top plots has strong resonances across the Nyquist band. Fig. 7. NUS IC die photo. Die size is mm. In contrast, the pattern shown in the bottom plots, which has j=1 undergone a randomization of its sample locations, has a much X whiter spectrum. The flat spectrum is preferred since then all signals have equal gain.
III. DATA PROCESSING In this section, we describe our computational techniques for efficiently recovering a Nyquist-rate signal from the NUS data by filling in the missing samples. Sections III-A–III-D describe our procedures for windowing the NUS data and recovering the missing samples. Section III-E then brieflydiscussesadditional practical concerns and the computational complexity of imple- menting this reconstruction algorithm.
A. Windowing While the NUS produces an arbitrarily long sequence of samples, the recovery algorithm can only deal with a finite Fig. 8. NUS test fixture. The NUS IC is mounted on a custom pallet. Also number of them at any given time.Itis,therefore,necessary shown is the 14-bit ADC as well as various test equipment connector interfaces. to segment the data stream, and we achieve this by windowing the signal. An effective windowing process must guard against Sampling edge effects as well as the well-known spectral spreading C. Nonuniform Sample Pattern effect, which would destroy theveryFouriersparsityweseek The NUS sampling pattern is a pulse train with nonuniform to exploit. Fortunately, the concept of a perfect reconstruction spacing between pulses. It is selected to meet a list of certain cri- filter bank (PRFB) [24] can be readily adapted to our pur- teria. First, the pattern is clocked at 4.4 GHz, and reconstruction poses. A windowing procedure breaks the infinite signal into produces the equivalent of Nyquist samples taken at this rate. aseriesof(possiblyoverlapping)vectorsbyusingananalysis Second, the pulse widths and spacings must satisfy the clocking window. After signal processing, the infinite length signal can GPS Radar Ultrasoundrequirements of the ADS5474. Specifically, the minimum pulse be recovered by stitching together the finite series using the width is 6 clock cycles, the minimum spacing ( )between analysis window. Using windows from a PRFB ensures that pulses is 12 clock cycles, and the maximum spacing ( )be- the windowing process itself does not introduce any errors. tween pulses is 27 clock cycles. An example pattern illustrating An example of a PRFB is a rectangular analysis and synthesis these specifications is shown in Fig. 9. In effect, as the sample window with no overlap, but of course this causes spectral imaging astronomy seismology
520 IEEE JOURNAL ON EMERGING AND SELECTED TOPICS IN CIRCUITS AND SYSTEMS, VOL. 2, NO. 3, SEPTEMBER 2012
spectroscopy
Fig. 6. NUS IC sampling timing and waveforms. Left panel: Interface timing between NUS die and ADC. Right panel: Simulated waveforms before and after being sampled by NIN. Horizontal scale is in nanoseconds.
spacings vary between 12 and 27 clock cycles, the instanta- neous sampling rate of the NUS receiver varies between 163 and 367 MHz, which is within the range of the 400 MHz ADC. We designed the NUS pattern to repeat every 8192 Nyquist samples, during which time there are 440 pulses which set the k sampling locations. This corresponds to an average sample rate of 236 MHz. We evaluate the quality of our pattern using a third criterion: the Fourier transform of favorable patterns will tend to have a flat, noise-like spectrum. Fig. 10 compares two NUS x(s)=PSF~ cj (s sj) patterns with different inter-sample spacings. The pattern shown DOA Estimation in the top plots has strong resonances across the Nyquist band. 8 9 Fig. 7. NUS IC die photo. Die size is mm. In contrast, the pattern shown in the bottom plots, which has j=1 undergone a randomization of its sample locations, has a much III. DATA PROCESSING : ; In this section, we describe our computational techniques for efficiently recovering a Nyquist-rate signal from the NUS data by filling in the missing samples. Sections III-A–III-D describe our procedures for windowing the NUS data and recovering the missing samples. Section III-E then brieflydiscussesadditional practical concerns and the computational complexity of imple- menting this reconstruction algorithm. A. Windowing While the NUS produces an arbitrarily long sequence of samples, the recovery algorithm can only deal with a finite Fig. 8. NUS test fixture. The NUS IC is mounted on a custom pallet. Also number of them at any given time.Itis,therefore,necessary shown is the 14-bit ADC as well as various test equipment connector interfaces. to segment the data stream, and we achieve this by windowing the signal. An effective windowing process must guard against Sampling edge effects as well as the well-known spectral spreading C. Nonuniform Sample Pattern effect, which would destroy theveryFouriersparsityweseek The NUS sampling pattern is a pulse train with nonuniform to exploit. Fortunately, the concept of a perfect reconstruction spacing between pulses. It is selected to meet a list of certain cri- filter bank (PRFB) [24] can be readily adapted to our pur- teria. First, the pattern is clocked at 4.4 GHz, and reconstruction poses. A windowing procedure breaks the infinite signal into produces the equivalent of Nyquist samples taken at this rate. aseriesof(possiblyoverlapping)vectorsbyusingananalysis Second, the pulse widths and spacings must satisfy the clocking window. After signal processing, the infinite length signal can GPS Radar Ultrasoundrequirements of the ADS5474. Specifically, the minimum pulse be recovered by stitching together the finite series using the width is 6 clock cycles, the minimum spacing ( )between analysis window. Using windows from a PRFB ensures that pulses is 12 clock cycles, and the maximum spacing ( )be- the windowing process itself does not introduce any errors. tween pulses is 27 clock cycles. An example pattern illustrating An example of a PRFB is a rectangular analysis and synthesis these specifications is shown in Fig. 9. In effect, as the sample window with no overlap, but of course this causes spectral imaging astronomy seismology 520 IEEE JOURNAL ON EMERGING AND SELECTED TOPICS IN CIRCUITS AND SYSTEMS, VOL. 2, NO. 3, SEPTEMBER 2012 spectroscopy Fig. 6. NUS IC sampling timing and waveforms. Left panel: Interface timing between NUS die and ADC. Right panel: Simulated waveforms before and after being sampled by NIN. Horizontal scale is in nanoseconds. spacings vary between 12 and 27 clock cycles, the instanta- neous sampling rate of the NUS receiver varies between 163 and 367 MHz, which is within the range of the 400 MHz ADC. We designed the NUS pattern to repeat every 8192 Nyquist samples, during which time there are 440 pulses which set the k sampling locations. This corresponds to an average sample rate of 236 MHz. We evaluate the quality of our pattern using a third i2⇡!sj criterion: the Fourier transform of favorable patterns will tend xˆ(!)=PSF(!) c e to have a flat, noise-like spectrum. Fig. 10 compares two NUS j patterns with different inter-sample spacings. The pattern shown DOA Estimation in the top plots has strong resonances across the Nyquist band. Fig. 7. NUS IC die photo. Die size is mm. j=1 In contrast, the pattern shown in the bottom plots, which has undergone a randomization of its sample locations, has a much X whiter spectrum. The flat spectrum is preferred since then all d signals have equal gain. III. DATA PROCESSING In this section, we describe our computational techniques for efficiently recovering a Nyquist-rate signal from the NUS data by filling in the missing samples. Sections III-A–III-D describe our procedures for windowing the NUS data and recovering the missing samples. Section III-E then brieflydiscussesadditional practical concerns and the computational complexity of imple- menting this reconstruction algorithm. A. Windowing While the NUS produces an arbitrarily long sequence of samples, the recovery algorithm can only deal with a finite Fig. 8. NUS test fixture. The NUS IC is mounted on a custom pallet. Also number of them at any given time.Itis,therefore,necessary shown is the 14-bit ADC as well as various test equipment connector interfaces. to segment the data stream, and we achieve this by windowing the signal. An effective windowing process must guard against Sampling edge effects as well as the well-known spectral spreading C. Nonuniform Sample Pattern effect, which would destroy theveryFouriersparsityweseek The NUS sampling pattern is a pulse train with nonuniform to exploit. Fortunately, the concept of a perfect reconstruction spacing between pulses. It is selected to meet a list of certain cri- filter bank (PRFB) [24] can be readily adapted to our pur- teria. First, the pattern is clocked at 4.4 GHz, and reconstruction poses. A windowing procedure breaks the infinite signal into produces the equivalent of Nyquist samples taken at this rate. aseriesof(possiblyoverlapping)vectorsbyusingananalysis Second, the pulse widths and spacings must satisfy the clocking window. After signal processing, the infinite length signal can GPS Radar Ultrasoundrequirements of the ADS5474. Specifically, the minimum pulse be recovered by stitching together the finite series using the width is 6 clock cycles, the minimum spacing ( )between analysis window. Using windows from a PRFB ensures that pulses is 12 clock cycles, and the maximum spacing ( )be- the windowing process itself does not introduce any errors. tween pulses is 27 clock cycles. An example pattern illustrating An example of a PRFB is a rectangular analysis and synthesis these specifications is shown in Fig. 9. In effect, as the sample window with no overlap, but of course this causes spectral Observe a sparse combination of sinusoids s i2⇡muk xm = cke for some uk [0, 1) 2 kX=1 Spectrum Estimation: find a combination of sinusoids agreeing with time series data Classic (1790...): Prony’s method Assume coefficients are positive for simplicity 1 1 x0 x¯1 x¯2 x¯3 r i2 u i2 u e k e k x1 x0 x¯1 x¯2 ck i4 u i4 u = =: toep(x) e k e k x2 x1 x0 x¯1 k=1 i6 u i6 u e k e k x3 x2 x1 x0 • toep(x) is positive semidefinite, and any null vector corresponds to a polynomial that vanishes at ei2⇡uk • MUSIC, ESPRIT, Cadzow, etc. Observe a sparse combination of sinusoids s i2⇡muk xm = cke for some uk [0, 1) 2 kX=1 Spectrum Estimation: find a combination of sinusoids agreeing with time series data sparse Contemporary: x Fc n N Fab =exp(i2 ab/N) Solve with LASSO: minimize x Fc 2 + µ c 2 1 Observe a sparse combination of sinusoids s i2⇡muk xm = cke for some uk [0, 1) 2 kX=1 Spectrum Estimation: find a combination of sinusoids agreeing with time series data Classic Contemporary SVD gridding+L1 minimization robust grid free model selection quantitative theory need to know model order discretization error lack of quantitative theory basis mismatch unstable in practice numerical instability Can we bridge the gap? Linear Inverse Problems • Find me a solution of y = x • Φ n x p, n • Of the infinite collection of solutions, which one should we pick? • Leverage structure: Sparsity Rank Smoothness Symmetry • How do we design algorithms to solve underdetermined systems problems with priors? Sparsity 1 • 1-sparse vectors of Euclidean norm 1 -1 1 • Convex hull is the unit ball of the l1 norm -1 p x = x k k1 | i| i=1 X x2 Φx=y minimize x 1 subject to k xk= y x1 Compressed Sensing: Candes, Romberg, Tao, Donoho, Tanner, Etc... Rank • 2x2 matrices • plotted in 3d rank 1 x2 + z2 + 2y2 = 1 Convex hull: X = i(X) k k⇤ i X • 2x2 matrices • plotted in 3d X = i(X) k k⇤ i Nuclear NormX Heuristic Fazel 2002. R, Fazel, and Parillo 2007 Rank Minimization/Matrix Completion Integer Programming • Integer solutions: all components of x (-1,1) (1,1) are ±1 • Convex hull is the unit ball of the l1 norm (-1,-1) (1,-1) x2 Φx=y minimize x subject to k xk1= y x1 Donoho and Tanner 2008 Mangasarian and R 2009. Parsimonious Models rank model weights atoms • Search for best linear combination of fewest atoms • “rank” = fewest atoms needed to describe the model Atomic Norms • Given a basic set of atoms, , define the function A x =inf t>0:x tconv( ) k kA { 2 A } • When is centrosymmetric, we get a norm A x =inf ca : x = caa k kA { | | } a a X2A X2A minimize z IDEA: subject to k zkA= y • When does this work? Atomic Norm Minimization minimize z IDEA: subject to k zkA= y • Generalizes existing, powerful methods • Rigorous formula for developing new analysis algorithms • Precise, tight bounds on number of measurements needed for model recovery • One algorithm prototype for a myriad of data- analysis applications Chandrasekaran, R, Parrilo, and Willsky Union of SubspacesHierarchical dictionary for image patches • X has structured sparsity: linear combination of elements 26/42 from a set of subspaces {Ug}. • Atomic set: unit norm vectors living in one of the Ug Permutations and Rankings • X a sum of a few permutation matrices • Examples: Multiobject Tracking, Ranked elections, BCS • Convex hull of permutation matrices: doubly stochastic matrices. • Moments: convex hull of of [1,t,t2,t3,t4,...], t∈T, some basic set. • System Identification, Image Processing, Numerical Integration, Statistical Inference • Solve with semidefinite programming • Cut-matrices: sums of rank-one sign matrices. • Collaborative Filtering, Clustering in Genetic Networks, Combinatorial Approximation Algorithms • Approximate with semidefinite programming • Low-rank Tensors: sums of rank-one tensors • Computer Vision, Image Processing, Hyperspectral Imaging, Neuroscience • Approximate with alternating least- squares Tangent Cones • Set of directions that decrease the norm from x form a cone: (x)= d : x + ↵d x for some ↵ > 0 TA { k kA k kA } y = z x minimize z subject to k zkA= y (x) TA z : z x { k kA k kA} • x is the unique minimizer if the intersection of this cone with the null space of Φ equals {0} Rates • Hypercube: n p/2 • Sparse Vectors, p vector, sparsity s n 2s log p + 5s s 4 • Block sparse, M groups (possibly overlapping), maximum group size B, k active groups 2 n k 2 log (M k)+pB + kB ⇣p ⌘ • Low-rank matrices: p1 x p2, (p1 • Suppose we observe y = x + w w k k2 minimize z subject to k kzA y k k 2 • If xˆ is an optimal solution, then x xˆ provided that k k ✏ p 1 2 pw( (x) S ) n TA \ (1 ✏)2 z y k k z : z x { k kA k kA} Robust Recovery (statistical) • Suppose we observe y = x + w minimize z y 2 + µ z k k k kA • If xˆ is an optimal solution, then x x ˆ 2 µ x k k k kA provided that µ Ew[ ⇤w ⇤ ] k kA p xˆ And under an additional “cone condition” x xˆ ⌘(x, , , )µ k k2 A cone u : x + u x + u { k kA k kA k k} Bhaskar, Tang, and R 2011 Denoising Rates (re-derivations) • Sparse Vectors, p vector, sparsity s 1 2s log(p) xˆ x? 2 = O pk k2 p ✓ ◆ • Low-rank matrices: p1 x p2, (p1 Atomic Set ei✓ ei2⇡ +i✓ 82 3 ✓ [0, 2⇡) , 9 = ei4⇡ +i✓ : 2 A > [0, 1) > >6 7 2 > <6 ··· 7 = 6 ei2⇡ n+i✓ 7 >6 7 > >4 5 > :> ;> Classical techniques (Prony, Matrix Pencil, MUSIC, ESPRIT, Cadzow), use the fact that noiseless moment matrices are low-rank: 1 1 ⇤ µ0 µ1 µ2 µ3 r ki ki e e µ¯1 µ0 µ1 µ2 k 2 2 ki 3 2 2 ki 3 = 2 3 0 e e µ¯2 µ¯1 µ0 µ1 ⌫ k=1 X 6 e3 ki 7 6 e3 ki 7 6 µ¯ µ¯ µ¯ µ 7 6 7 6 7 6 3 2 1 0 7 4 5 4 5 4 5 Atomic Norm for Spectrum Estimation minimize z IDEA: subject to k kzA y k k Atomic Set: ei✓ ei2⇡ +i✓ 8 9 2 ei4⇡ +i✓ 3 ✓ [0, 2⇡) , = > : 2 > A >6 . 7 [0, 1) > <>6 . 7 2 => 6 7 6 ei2⇡ n+i✓ 7 >6 7 > >4 5 > :> ;> • How do we solve the optimization problem? • Can we approximate the true signal from partial and noisy measurements? • Can we estimate the frequencies from partial and noisy measurements? Which atomic norm for sinusoids? s i2⇡muk xm = cke for some uk [0, 1) 2 kX=1 Assume c k are positive for simplicity 1 1 ⇤ x0 x¯1 x¯2 x¯3 r i2⇡uk i2⇡uk e e x1 x0 x¯1 x¯2 ck 2 i4⇡uk 3 2 i4⇡uk 3 = 2 3 0 e e x2 x1 x0 x¯1 ⌫ k=1 X 6 ei6⇡uk 7 6 ei6⇡uk 7 6 x x x x 7 6 7 6 7 6 3 2 1 0 7 4 5 4 5 4 5 • Convex hull is characterized by linear matrix inequalities (Toeplitz positive semidefinite) 1 1 tx⇤ x =inf t + w0 : 0 k kA 2 2 x toep(w) ⌫ ⇢ 1 • Moment Curve of [t,t2,t3,t4,...], t S 2 A A [ { A} conv( ) conv( ) A A [ { A} cos(2⇡u k)/2 + cos(2⇡u k)/2 cos(2⇡u k)/2 cos(2⇡u k)/2 1 2 1 2 u1 =0.1413 u2 =0.1411 Nearly optimal rates Observe a sparse combination of sinusoids s ? ? i2⇡mu? x = c e k ? m k for some uk [0, 1) k=1 2 X ? Observe: y = x + ! (signal plus noise) 4 Assume frequencies are far apart: min d(up,uq) p=q n 6 1 2 Solve: xˆ = arg min x y 2 + µ x x 2 k k k kA 1 C 2s log(n) Error Rate: xˆ x? 2 nk k2 n No algorithm can do better than No algorithm can do better than 2 2 1 ? 2 C0 s log(n/s) 1 C0 s E xˆ x xˆ x? 2 nk k2 n 2 nk k n even if the frequencies are well- even if we knew all of the * separated frequencies (uk ) Mean Square Error Performance Profile ) ( P • Frequencies generated at • Performance profile over all random with 1/n separation. parameter values and settings Random phases, fading • For algorithm s: amplitudes. # p :MSE(p) min MSE (p) P ( )= { 2 P s s s } s #( ) • Cadzow and MUSIC provided P model order. AST and LASSO estimate noise power. Lower is better Higher is better Localization Guarantees 1 Estimated Frequency True Frequency 0.16/n Spurious Frequency Dual Polynomial Magnitude 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Frequency k2 (n) Spurious Amplitudes cˆ C | l| 1 n l:fˆ F l 2 2 ˆ k (n) Weighted Frequency Deviation cˆl d(fj, fl) C2 | | fj T n l:fˆ N l j k2 (n) Near region approximation cj cˆl C3 n l:fˆ N l j Frequency Localization Spurious Amplitudes Weighted Frequency Deviation Near region approximation 2 2 k2 (n) ˆ k (n) k2 (n) cˆ C cˆl d(fj, fl) C2 cj cˆl C3 l 1 | | fj T n n | | n ˆ ˆ ˆ l:fl Nj l:fl Nj l:fl F 1 1 1 0.8 0.8 0.8 ) 0.6 ) 0.6 ) 0.6 β β β ( ( ( s s s P P 0.4 0.4 P 0.4 AST AST 0.2 0.2 0.2 AST MUSIC MUSIC MUSIC Cadzow Cadzow Cadzow 0 0 0 20 40 60 80 100 100 200 300 400 500 5 10 15 20 β β β 100 0.04 5 AST AST AST MUSIC MUSIC MUSIC 4 80 Cadzow Cadzow 0.03 Cadzow 60 k = 32 k = 32 3 k = 32 3 2 1 0.02 m m m 40 2 0.01 20 1 0 0 10 5 0 5 10 15 20 0 −10 −5 0 5 10 15 20 − − −10 −5 0 5 10 15 20 SNR (dB) SNR (dB) SNR (dB) Incomplete Data/Random Sampling • Observe a random subset of samples T 0, 1,...,n 1 ⇢ { } • On the grid, Candes, Romberg & Tao (2004) • Off the grid, new, Compressed Full observation Sensing extended to the wide continuous domain • Recover the missing part by solving minimize z z k kA subject to zT = xT . Random sampling • Extract frequencies from the dual optimal solution Exact Recovery (off the grid) Theorem: (Candès and Fernandez- Granda 2012) A line spectrum with minimum frequency separation Δ > 4/s { can be recovered from the first 2s Fourier Δ coefficients via atomic norm minimization. s Theorem: (Tang, Bhaskar, Shah, and R. x = c exp(2⇡imu ) m k k 2012) A line spectrum with minimum k=1 X frequency separation Δ > 4/n can be recovered from most subsets of the first n WANT uk,ck { } Fourier coefficients of size at least O(s log(s) log(n)). • s random samples are better than s equispaced samples. • On a grid, this is just compressed sensing • Off the grid, this is new • No balancing of coherence as n grows. imaging astronomy seismology k i2⇡!sj xˆ(!)=PSF(!) cje j=1 X d k x(t)= c g(t ⌧ )ei2⇡fj t j j j=1 X GPS Radar Ultrasound Summary and Future` Work • Unified approach for continuous problems in estimation. • Obviates incoherence and basis mismatch • New insight into bounds on estimation error and exact recovery through convex duality and algebraic geometry • State-of-the-art results in classic fields of spectrum estimation and system identification (ask me later!) • Recovery of general sparse signal trains • Scaling atomic norm algorithms • More atomization in signal processing... (moments, PDEs, deep atoms) Acknowledgements Work developed with Venkat Chandrasekaran, Babak Hassibi (Caltech), Weiyu Xu (Iowa), Pablo A. Parrilo, Alan Willsky (MIT), Maryam Fazel (Washington) Badri Bhaskar, Rob Nowak, Nikhil Rao, Gongguo Tang (Wisconsin). For all references, see: http://pages.cs.wisc.edu/~brecht/publications.html References • Atomic norm denoising with applications to line spectral estimation. Badri Narayan Bhaskar, Gongguo Tang, and Benjamin Recht. Submitted to IEEE Transactions on Signal Processing. 2012. • Compressed sensing off the grid. Gongguo Tang, Badri Bhaskar, Parikshit Shah, and Benjamin Recht. Submitted to IEEE Transactions on Information Theory. 2012. • Near Minimax Line Spectral Estimation. Gongguo Tang, Badri Narayan Bhaskar, and Benjamin Recht. Submitted to IEEE Transactions on Information Theory, 2013. • The convex geometry of inverse problems. Venkat Chandrasekaran, Benjamin Recht, Pablo Parrilo, and Alan Willsky. To Appear in Foundations on Computational Mathematics. 2012. • All references can be found at http://pages.cs.wisc.edu/~brecht/publications.html