arXiv:1903.12289v1 [cs.IT] 28 Mar 2019 EEI osrimvateIre iityo cnm n In and 1523/16. Economy grant of ISF Ministry from Israel support the acknowledges via consortium GENESIS cec n niern,Hbe nvriyo euae,I Jerusalem, of University Hebrew {elad.romanov,or.ordentlich}@mail.huji.ac.il). Engineering, and Science es infiatbt LB) foehst hoebetween choose to has one If (LSBs). reconstructing bits significant least the inl oeta h signal the that Note signal. fqatzto,a ela n te nt i-aefr of form bit-rate finite other any as well quantization, as quantization, of R ersne by represented adn l u the but all carding prto fan of operation ecie sta fwiigtebnr xaso feach of expansion binary the writing as of sample that as described edgtlysoe/rcse,tesignal the stored/processed, digitally be iiie,wihi civdb uniigi oasto finit of set a to it quantizing by cardinality. achieved is which digitized, a epretyrcntutdfo h iceetm sign discrete-time the from reconstructed { perfectly be can oevaasaa nfr unie.Frasga ihbounde with say signal a range, For quantizer. dynamic uniform scalar a via done hslte eipoeuo hsrsl n hwta o finit any for for that possible show is and recovery result this perfect upon signals, improve energy we letter this apigrt xed h yus aeb atrof factor a by rate Nyquist shown the has exceeds al. et rate Bhandari sampling by any work for Recent that rate. Nyquist called signal h etsml fadsrt-iesga rmisps,and past, predic its the small. signal from arbitrarily bandlimited signal made a be discrete-time for can that a predictin error fact of of the problem sample the leveraging and next version rate reduced the establishi sampling modulo by discrete-time its proved the a of from is problem that claim recovery the provided This between connection rate. signal, foldi Nyquist modulo the the Thus, exceeds degrade rate. Nyquist not the does above rate sampling any osbepoie httesmln rate recovery sampling the perfect that that provided guarantees possible theorem sampling Nyquist ∆ iebnlmtdsga rmtedsrt-iesga obta signal discrete-time the every it from sampling from signal bandlimited time bv h yus ae ouoFligDe Not Does Folding Modulo Rate, Nyquist the Above x btuiomqatzr and quantizer, uniform -bit o some for , hswr a upre,i at yIFudrGat1791/17, Grant under ISF by part, in supported, was work This ldRmnvadO retihaewt h colo Compute of School the with are Ordentlich Or and Romanov Elad Let npatc,qatzto fteprocess the of quantization practice, In h hno-yus apigtermgaate htany that guarantees theorem sampling Shannon-Nyquist The Abstract n R = { x Sso ahsml n epn l h remaining the all keeping and sample each of MSBs x x ( ( t n Q nT ) W osdrtepolmo eoeigacontinuous- a recovering of problem the consider —We } x hs ore rnfr sspotdon supported is transform Fourier whose s etesga bandb quantizing by obtained signal the be ∆ n ) x ∆ } { ( = > a R t x sln as long as n, ) > n 0 btsaa nfr unie a eroughly be can quantizer uniform scalar -bit − 0 a olne epretyreconstructed. perfectly be longer no can } R a , efc eosrcini tl osbei the if possible still is reconstruction perfect 0 a rmeither from For . .I I. n, x T otsgicn is(Ss,i.e., (MSBs), bits significant most n, n s 0 1 2 eod n euigtersl modulo result the reducing and seconds a , . . . , NTRODUCTION m ∈ T = ∆ + { s ( { − x x < P n res n,R n res 2 m { 1 i ∞ } ∞ =1 x / = − , n Q 2 orsod odiscarding to corresponds 2 1 W a x m } h eertdShannon- celebrated the lal,atrti form this after Clearly, . n,i n ) or oee,i re to order in However, . − { 2 o all for 1 x m x { /T n ldRmnvadO Ordentlich Or and Romanov Elad { x n Q − } x s n res } i n n hndis- then and , utb further be must xed h so- the exceeds } } eteresidual the be h intuitive the , { n sinvariably is x re (emails: srael ∆ n ∈ [ } − uty ER dustry. n ythe by and > sn a using ,W W, Z πe 0 signal x the , ga ng ined Hurt n In . tion and ng al is is d e g e r ] nqenme in number unique elnumber real a apigamdl-eue inl o given a For signal. modulo-reduced a sampling pntercn eut fBadr ta.[]adshow and [1] al. Shannon- any et the for exceeds Bhandari frequency, frequency of Nyquist sampling the results whenever recent that the upon efcl eosrce from reconstructed perfectly oehtsrrsnl,ti etrsosta h opposite the that that provided shows true, letter this surprisingly, somewhat whereas hiewl be will choice fulmtdsmln.Nml,w r assuming are we Namely, sampling. unlimited of ore rnfr sspotdon supported is transform Fourier osbefrany for possible ltd aihsfor vanishes plitude recover hra h atrepiil tde h rdofbetween tradeoff reduction. the modulo signal, after studies reduced effect distortion and explicitly modulo the rate the consider latter quantization to not the added does whereas noise former quantization the of so-called that the is between framework difference sampling main unlimited The quantization. to signal T { ne htcniin on conditions what under i-aeAC htrdc hi nu inlmodulo signal input finite their recent are which reduce related [9]–[12], that closely ADCs modulo ADCs A of bit-rate [2]–[8]. that is e.g. work see of line work, up follow of c considered the signals. within of recovery perfect guarantee to order ymdl reducing modulo by eut mrv h apigrt bandi 1 yafactor a by [1] in obtained πe rate sampling the improve results in saoeo h ls fsgas efc reconstructi perfect signals, any of for class possible the is on above as tions Work Related A. statement. formal the for on eei h etpsil,sneee ihu modulo without even since possible, best (alternatively, the reduction is here found orsod ordcn ohtera n h mgnr part imaginary the and real the modulo both reducing to corresponds ope number complex a x s ob oecnrt,testpw osdri htof that is consider we setup the concrete, more be To u anrsl sta o l nt nrysgaswhose signals energy finite all for that is result main Our h eu osdrdi hslte al ihnteframewo the within falls letter this in considered setup The h eut fBadr ta.[]hdaraysakdalot a sparked already had [1] al. et Bhandari of results The n[] twssonta ne setal h aeassump- same the essentially under that shown was it [1], In n ape e eod.Cery h apigrate sampling the Clearly, seconds. per samples eod.Nt that Note seconds. = { x x ∆ x | n ∗ x ( ( i.e., , t n nT ) = − from s x x ) { ∗ } x n res x ( x h usinw drs nti etris letter this in address we question The . ∆ nT ∗ | n Q ∈ { a ea ag as large as be can [ x } > = ∆ T x | − rmwr f[]adthe and [1] of framework s t fe all, After . n ∗ R x s ) | } 0 } ∆ a = 2 ( > edefine we , ag nuh efc eosrcinis reconstruction perfect enough, large ? < t ∗ = ∆ x , n any and sotie ysampling by obtained is ) n ∗ + a ∆ 0 2 taltms n h discrete-time the and times, all at T 1 + ) ib n any and s / [ = R { 2 ∞ uhthat such ∗ and x ib W h signal The . ∈ n res hscniini eurdin required is condition this ) x | ∈ npriua,w improve we particular, In . T x n } N ∆ s . n ] [ − ∗ h signal the , C x < − T ∗ ,W W, 2 si osbet exactly to possible it is h ouooperation modulo the , o all for s m x 2 [ = x W 1 < (1 n Q | − e eto II-D Section See . − ] x πe x n hs am- whose and , < 1 o ∆ mod ] 2 x ∗ n 2 ( − ∗ 2 W t 1 ouoADC modulo R { m ) ∈ ∆ ∈ x ) x hs our Thus, . sobtained is However, . n · ∗ Z T > } ( 2 ∆ { t s where , − ∆ ) x Z a be can 0 < ( sthe as n ∗ R every For . and , prior } − lass 2 1) W on 1 of rk is is s 1 , 2 Sampler Decoder available at perfect precision, and we are only interested in recovered xn mod∆ Σ mod∆ Σ x characterizing the optimal tradeoff between Ts and ∆ for n − which xn (and consequently also x(t)) can be perfectly reconstructed{ } despite the modulo folding. Our results indicate Filter that modulo reduction per se, incurs no loss on our ability to reconstruct x(t), provided that the Shannon-Nyquist condition is satisfied. Therefore, the main question of interest is what can be gained from modulo folding in terms of quantization Fig. 1. Schematic architecture for the proposed recovery algorithm. rate. The results in [9] show that for stationary Gaussian processes, oversampled modulo ADCs achieve rate-distortion 1 tradeoff which is close to the information theoretic limits. [ γ,γ], where γ = W Ts < . To see this, note that − · 2 Similar results were also obtained in [9] for spatially correlated ∞ processes. e = x h x = x ⋆c , n n − i n−i n n Despite the differences between the unlimited sampling and i=1 X modulo ADCs frameworks, the insights gained from develop- where cn = δn hn is a monic filter, i.e., a causal filter with ing recovery algorithms for oversampled modulo ADCs [9] − first tap equal to 1. Since the coefficients h1,h2,... are not turn out to also be suitable for recovery under the unlimited constrained, we are free to take cn as any monic filter. In sampling framework. In particular, the recovery algorithm { } particular, for any ǫ> 0, we may choose cn to satisfy outlined below is inspired by that of [9, Section 3]. { } ∞ 2 2 −i2πfk ǫ f γ C(f) = cke = 2γ | |≤ . | | ( 1 1−2γ f (γ, 1 ] B. Recovery Through Prediction k=1 ǫ | | ∈ 2 X The existence of a monic filter with this
frequency response Our main result may look somewhat absurd at first, as it is guaranteed by Paley-Wiener’s spectral factorization theo- seems that as we drive ∆ to 0, we should end up with no rem [13], and the fact that 1/2 log C(f) 2df = 0. Now, information left. However, the result becomes very intuitive −1/2 | | since x has no spectral energy in f >γ, we have that once a connection is established between the problem of { n} R | | modulo unwrapping of a discrete-time signal and the problem e 2 = x ⋆c 2 = ǫ x 2. (1) k{ n}k2 k{ n n}k2 k{ n}k of predicting the next sample of a discrete-time signal from 2 ∆ ∆ Thus, if we take ǫ < 2 we get that en 2 < , its past. 4k{xn}k 2 ∆ k{ }k ∗ and in particular en < for all n Z. The argument is In particular, our algorithm for recovering xn from xn | | 2 ∈ is of a sequential nature. Since we assume x{(t) }is small{ for} made precise in Section II, where the infinite length filter used all large enough t , there exists some negative| integer| N Z above is replaced with a finite length one, whose coefficients such that x < |∆| for all nFourier transform: for a signal x L ( ) L ( ), its Fourier transform is given by ∈ ∩ that en can be made arbitrarily small at all times. This follows| since| the discrete-time process x is bandlimited, i.e., n (x)(f)= x(t)e−2πiftdt , its discrete-time Fourier transform (DTFT) is supported on F R Z 3 with the inverse transform given by B. The general recipe Suppose, for a moment, that x ,...,x were given to −1(ˆx)(t)= xˆ(f)e2πiftdf , 0 L−1 F R us, and we would now like to recover the proceeding samples Z x , x ,... from their modulo-reduced version. If we were so that the extension : L2(R) L2(R) is an isometry in L L+1 able to find L numbers h ,...,h such that the sequence the sense of Hilbert spaces.F → 1 L Definition 1: For W,E,T ,ρ R , the class of signals en := xn (h1xn−1 + ... + hLxn−L) (3) 0 ∈ + − W,E,T0,ρ consists of all square-integrable, bandlimited signals ∆ C is guaranteed to satisfy that for all n, en < , then we could x(t) of the form | | 2 recover xL, xL+1,..., in a manner which we now describe. W ∆ ∗ Since en < , clearly, e = en. Moreover, since the x(t)= X(f)e2πiftdf | | 2 n coefficient of xn is an integer, we have Z−W ∗ ∗ ∗ 2 2 e = [x (h x + ... + h x )] , for some X L (R), such that x 2 R E, and such that n n 1 n−1 L n−L ∈ || ||L ( ) ≤ − moreover the tails of x(t) can be explicitly controlled, in the ∗ hence given xn−1,...,xn−L and xn, we can compute en. But sense that then we can also compute xn as x(t) t −ρ when t T . | | ≤ | | | |≥ 0 xn = en + (h1xn−1 + ...hLxn−L) ,

The last condition in the definition above might seem some- and therefore iteratively recover xL, xL+1,.... what non-standard, but it is in fact not particularly strong: if, What remains to be done, then, is: for example, the Fourier transform X is absolutely continuous, 1) Find a set of coefficients h1,...,hL, where L may de- hence, weakly differentiable, integration by parts readily gives ∆ pend on ∆, that guarantees that en < 2 universally for us that 1 | | any x W,E,T0,ρ. Below, we specify a method based 1 on Chebyshev∈ C polynomials for choosing . This x(t) = X(W )e2πiW X( W )e−2πiW h1,...,hL | | 2πit − − method guarantees that e < ∆ whenever T < 1 ,  | n| 2 s 2W W whereas the required filter length L grows as Ts gets ′ 2πift X (f)e df closer to 1 . − 2W −W ∗ Z  ′ 2) Find a negative integer N such that x = xn for all X(W ) + X( W ) + X 1 n | | | − | k kL (−W,W ) . n < N. Indeed, by the Riemann-Lebesgue Lemma, ≤ 2π t | | lim|t|→∞ x(t) = 0, so in particular we know that ∆ This is the case, for instance, when x is obtained by band- xn < 2 for all small enough n. To give a quantitative limiting a time-limited signal, that is, bound| | for how far we need to go, we use the assumption

T on the tail behavior of x W,E,T0,ρ: choosing N < −2πift −1 −1/ρ ∈ C X(f)= x0(t)e dt Ts max(T0, (∆/2) ) would guarantee us that for −T − ∆ ∗ Z all n 0 is the sampling period. The celebrated Shannon- [ 1, 1] s on approximation theory,− e.g [14]). That maximum is given Nyquist sampling theorem guarantees, then, that x could be by perfectly reconstructed from the sequence x , provided n n∈Z max 2−K+1T (y) =2−K+1 . (5) that 1 is greater than the so-called Nyquist{ rate} 2W . K Ts y∈[−1,1] We consider the case where instead of observing x , we n On any other interval [a,b], we can easily construct a family observe a modulo-reduced version of the samples x∗ , n n∈Z of Chebyshev polynomials that take the least maximal value and we would now like to reconstruct x( ). Assuming{ } that 1 · on that interval among all monic degree-K polynomials: the Ts < 2W , it would clearly suffice to simply recover the unfolded measurements x Z from their modulo-reduced 1 { n}n∈ In fact, the choice of coefficients h1,...,hL depends only on W and E, versions. whereas the tail behavior dictated by T0 and ρ is of no importance here. 4

2 mapping y b−a (y a) 1 maps [a,b] bijectively into and , and7→ so − − [ 1, 1] −2πifTs − max pK e K f∈[−W,W ] [a,b] b a 2 TK (y)=2 − TK (y a) 1 (6) [2 cos(2πW
T s),2] −2πifTs 2πifTs 4 · b a − − = max TK (e + e )    −  f∈[−W,W ] is clearly the polynomial we need. Its maximal value on [a,b] [2 cos(2πW Ts),2] = max TK (2 cos(2πfTs)) is given by f∈[−W,W ]

K [2 cos(2πW Ts),2] [a,b] b a = max TK (y) y∈[2 cos(2πW T ),2] max TK (y) =2 − . s y∈[a,b] 4 K   2 2 cos(2πWT s) =2 − . Note that max T [a,b](y ) 0 as K if and only 4 y∈[a,b] K → → ∞   if b a < 4. In the construction that follows, this condition 1 Note that whenever Ts < , we have that 2 cos(2πWTs) > will− correspond exactly to the condition that T < 1 , that 2W s 2W 2, and so in that case the right-hand-side above tends to 0 is, we’re sampling at a rate strictly above the Nyquist rate. as− K . Thus, we readily obtain → ∞ Proposition 1: Choose any

log √32WE/∆ K > (8) log  2  D. Choosing the Coefficients 1−cos(2πW Ts)   and let h1,...,h2K be given by the coefficients of the poly- nomial Denote by S : RZ RZ the backward-shift operator, that → p (z)= zK T [2 cos(2πW Ts),2] z + z−1 is, K · K (9) 2K 2K−1 (Sx)n = xn−1 . = h z h z ... h z +1 . − 2K − 2K−1 −
− 1 Writing x in terms of the Fourier inversion formula, we have Then for every x W,E,T0,ρ, the sequence ∈C W 2πif·(nTs) −2πifTs en = xn (h1xn−1 + ...h2K xn−2K ) (Sx)n = X(f)e e df , − −W · ∆ Z satisfies that en < for all n. | | 2 hence for any Laurent polynomial p(z),2 we have We summarize by stating our main result: Theorem 1: Fix W,E,T ,ρ R and any ∆ > 0. W 0 ∈ + 2πif·(nTs) −2πifTs Then, any can be perfectly recovered from (p(S)x)n = X(f)e p e df . x W,E,T0,ρ · ∗ ∈ C 1 Z−W x (nTs) n∈Z provided that Ts < .
{ } 2W Now, choose h1,...,h2K to be given by the coefficients of Proof: By Proposition 1, the recovery procedure we the polynomial described in Section II-B with h1,...,h2K chosen as in (9)

K [2 cos(2πW Ts),2] −1 correctly recovers the unfolded samples xn n∈Z from the pK (z)= z T z + z ∗ { } · K (7) folded samples x (nTs) n∈Z. Now, using the Shannon- 2K 2K−1 { } = h2Kz h2K−1z ... h1z +1 . Whittaker interpolation formula, see e.g. [15], we recover x − − −
− Well, at every point.

en = xn (h1xn−1 + ... + h2K xn−2K ) | | | − | III. DISCUSSION ON QUANTIZATION NOISE = (p (S)x) | K n| W The recovery algorithm described above, relies on filtering 2πif·(nTs) −2πifTs = X(f)e pK e df xn using a monic filter cn with vanishing magnitude for −W · { } { } Z the in-band frequencies. By Paley-Wiener’s theorem, com- −2πifT
s X L1(−W,W ) max pK e . bined with Jensen’s inequality, it is easy to see that any ≤ || || · f∈[−W,W ] such filter must have unbounded energy in the out-of-band
We can bound frequencies (i.e., the high frequencies where x has no { n} √ energy). X L1(−W,W ) 2W X L2(−W,W ) ∗ || || ≤ || || If the input to the recovery algorithm were [xn + un] ∗{ } = √2W x 2 R for some white process u , instead of x , we would || ||L ( ) { n} { n} √2WE, therefore have that the result of convolving with cn would ≤ no longer be restricted to [ ∆ , ∆ ), due to the contribution{ } of − 2 2 un ⋆cn in the out-of-band frequencies. Modeling the effect {of quantization} as an additive white noise, we see that the 2 N1 N1−1 developed algorithm collapses in the presence of quantization That is, an expression of the form p(z) = aN1 z + aN1−1z + −N2 ... + a−N2 z . noise. This is also the case for the reconstruction algorithm 5 of [1], if the order of discrete derivative used there is large enough. The remedy to this phenomenon is to judiciously balance the in-band energy of cn and its out-of-band energy, taking into account the effect of{ quantization} noise, as done in [9] (see also [16]). However, taking the quantization noise into account yields a lower bound on the in-band energy of cn , which in turn dictates that ∆ can no longer be arbitrarily{ small.} We conclude that it is essential to take into account quantization noise in oversampled systems

REFERENCES [1] A. Bhandari, F. Krahmer, and R. Raskar, “On unlimited sampling,” in 2017 International Conference on Sampling Theory and Applications (SampTA). IEEE, 2017, pp. 31–35. [2] A. Bhandari, F. Krahmer, and R. Raskar, “Unlimited sampling of sparse sinusoidal mixtures,” in 2018 IEEE International Symposium on Information Theory (ISIT), June 2018. [3] M. Cucuringu and H. Tyagi, “On denoising modulo 1 samples of a function,” arXiv preprint arXiv:1710.10210, 2017. [4] O. Musa, P. Jung, and N. Goertz, “Generalized approximate mes- sage passing for unlimited sampling of sparse signals,” arXiv preprint arXiv:1807.03182, 2018. [5] S. Rudresh, A. Adiga, B. A. Shenoy, and C. S. Seelamantula, “Wavelet-based reconstruction for unlimited sampling,” in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), April 2018. [6] L. Rencker, F. Bach, W. Wang, and M. D. Plumbley, “Sparse recovery and dictionary learning from nonlinear compressive measurements,” arXiv preprint arXiv:1809.09639, 2018. [7] V. Shah and C. Hegde, “Signal reconstruction from modulo observa- tions,” arXiv preprint arXiv:1812.00557, 2018. [8] F. Ji, W. P. Tay et al., “Recovering graph signals from folded samples,” arXiv preprint arXiv:1903.03741, 2019. [9] O. Ordentlich, G. Tabak, P. K. Hanumolu, A. C. Singer, and G. W. Wornell, “A modulo-based architecture for analog-to-digital conversion,” IEEE Journal of Selected Topics in Signal Processing, vol. 12, no. 5, pp. 825–840, Oct 2018. [10] P. T. Boufounos, “Universal rate-efficient scalar quantization,” IEEE Transactions on Information Theory, vol. 58, no. 3, pp. 1861–1872, March 2012. [11] O. Ordentlich and U. Erez, “Integer-forcing source coding,” IEEE Transactions on Information Theory, vol. 63, no. 2, pp. 1253–1269, Feb 2017. [12] E. Romanov and O. Ordentlich, “Blind unwrapping of modulo re- duced Gaussian vectors: Recovering MSBs from LSBs,” arXiv preprint arXiv:1901.10396, 2019. [13] A. Gersho and R. M. Gray, Vector quantization and signal compression. Springer Science & Business Media, 2012, vol. 159. [14] R. A. DeVore and G. G. Lorentz, Constructive approximation. Springer Science & Business Media, 1993, vol. 303. [15] S. Mallat et al., “A wavelet tour of signal processing: The sparse way,” [3rd ed.] ed. Amsterdam: Elsevier Academic Press, 2009. [16] R. Zamir, Y. Kochman, and U. Erez, “Achieving the Gaussian rate- distortion function by prediction,” IEEE Transactions on Information Theory, vol. 54, no. 7, pp. 3354–3364, July 2008.