Optimal Noise-Shaping DPCM

Milan S. Derpich, Eduardo I. Silva, Daniel E. Quevedo and Graham C. Goodwin School of Electrical Engineering and Computer Science, The University of Newcastle, NSW 2308, Australia {milan.derpich, Eduardo.silva}@studentmail.newcastle.edu.au, [email protected], [email protected].

Abstract— This paper presents novel results on the optimal relied upon heuristic design methods [2], [9]. Since optimal design of Noise-Shaping Differential Pulse-Coded Modulation performance can, in general, only be attained by arbitrary order coders. The main contribution resides in the derivation of explicit filters designed accounting for fed back quantization noise, an analytic formulas for the optimal filters and the minimum achievable frequency weighted reconstruction error. A novel exact characterization of the optimal performance (and filters) aspect in the analysis is the fact that we account for fed-back for NS-DPCM converters has remained an open problem. quantization noise and that we make no restrictions on the order In this paper we derive an explicit analytic expression for of the filters deployed. the optimal performance (and filter frequency responses) for I.INTRODUCTION NS-DPCM converters. We characterize the scalar quantizer via its signal-to-noise ratio, and adopt a white quantization noise Analog-to-Digital converters which utilize a scalar quantizer model [12]. The performance bound obtained corresponds to and linear, time invariant filters in a feedback loop have been the minimum FWMSE that can be achieved by an NS-DPCM extensively employed as a source coding method since the encoder-decoder with any linear, time-invariant filters. A key concept was first introduced in the 1960’s. The generalized departure from [3] (which, to the best of our knowledge, gives form of this architecture, which we denote Noise Shaping the only currently available explicit analytic solutions to the Differential Pulse Code Modulation1 (NS-DPCM), can be problem), is that we account for fed back quantization noise. represented as in Fig. 1. The filters in a NS-DPCM system This allows us to derive exact expressions. allow one to account for the correlation between consecutive Our results show that an optimal NS-DPCM converter ex- input samples, and to spectrally shape the quantization noise hibits several interesting properties. These include a spectrally in the output, so as to minimize the frequency weighted flat frequency weighted error spectrum, and a white signal mean square reconstruction error (FWMSE). Special cases at the input of the scalar quantizer. We also show that, for of the NS-DPCM architecture include ∆-Modulators, DPCM AR Gaussian sources, the rate-distortion efficiency with the converters [4], and noise-shaping converters, such as one and optimal filters depends only on how efficient the embedded multi-bit Sigma-Delta modulators [5]. NS-DPCM converters scalar quantizer is at quantizing nearly Gaussian samples. are extensively used in the context of audio compression [6], digital image half-toning [7] and oversampled A/D conver- Notation and Preliminaries sion [8]. We use standard vector space notation for signals. For example, x is used to denote {x(k)} . We also use z as v w k∈Z x A(z) Q B(z) x˜ the argument of the z-transform. Given two square integrable complex valued functions f(ω) and g(ω) definedR over [−π, π], 1 π ∗ we adopt the inner product h f, g i , 2π −π f(ω) g(ω)dω, n where ()∗ denotes complex conjugation. We denote the usual Encoder F (z) Decoder q R 1 π 2 2-norm as kfk , 2π −π |f(ω)| dω . If F (z) is a transfer Fig. 1: Noise Shaping-DPCM Encoder and Decoder function, then we use the short hand notation F to refer to the associated frequency response F (ejω), ω ∈ [−π, π]. If I is a Provided that the input power spectral density (PSD), fre- set, then we will write “a.e. on I” (almost everywhere on I) quency weighting error criterion, and scalar quantizer char- as a short hand notation for “everywhere on I except at most acteristics are known, the design of an NS-DPCM converter on a zero Lebesgue measure set of points”. 2 that achieves minimum FWMSE amounts to finding the cor- We use σx to denote the variance of a given wide sense sta- jω responding optimal filters. This has been an intense area of tionary (w.s.s.)© randomª processR x, having PSD Sx(e ). Note that σ2 , E x(k)2 = 1 π S (ejω)dω = kΩ k2, where research for at least 40 years. However, available to date results x 2π −π x √x on optimal filter design for NS-DPCM encoders have been Ωx is a frequency response satisfying |Ωx| , Sx , ∀ω ∈ [−π, π]. For a given function f :[−π, π] → C, we define η , obtained assuming either fixed, finite order filters [1], [2], [8]– ³ R ´ f 1 π [10], negligible fed back quantization noise [3], [11], or have exp 2π −π |f(x)|dx (provided this integral converges). This allows one to describe the Kolmogorov’s minimal prediction 1The same configuration can be found under different names in the error variance [13] of a w.s.s. process x via η2 , η = η2 . literature, e.g.: error feedback systems [1], direct feedback coders [2] and x Sx Ωx DPCM with noise feedback [3]. The spectral flatness measure of a w.s.s. process x is denoted 2 ηx by ζx , 2 . It is easy to show that 0 ≤ ζx ≤ 1, and that where γ is the signal-to-noise ratio of the scalar quantizer (not σx jω ζx = 1 if and only if Sx(e ) is constant a.e. on [−π, π]. to be confused with that of the NS-DPCM encoder-decoder system). γ depends on the number of quantization levels, II.NS-DPCMMODEL the PDF of the signal being quantized and the companding As foreshadowed in the introduction, we consider the gen- characteristics of the scalar quantizer itself4. eral form of an NS-DPCM architecture shown in Figure 1. In our model, the input sequence x is assumed to be a zero III.FORMULATION OF THE OPTIMIZATION PROBLEM mean, w.s.s. random process, with known PSD S = |Ω |2 x x Our ultimate goal is to find the frequency responses of the satisfying S (ejω) > 0, a.e. on [−π, π]. The element denoted x filters A, B, and F that minimize the variance of ² under by Q describes a scalar quantizer, with given and known Assumptions 1 and 2, and for given and known Ω , P and γ. characteristics2. For each input v(k), k ∈ Z, it outputs w(k) x The quantity σ2 so obtained will constitute the (achievable) and generates the quantization error n(k) , w(k)−v(k). The ² lower bound on the FWMSE for the NS-DPCM converter. three discrete-time filters A(z), B(z) and F (z) in Fig. 1 are design choices. Towards the above goal, we first derive an expression that To asses performance, we introduce the delay-compensated relates the decision variables to the error measure that we wish frequency weighted error to minimize. From Fig.√ 1, equation (1), Assumption 1, and recalling that |Ωx| = Sx , ∀ω ∈ [−π, π], we have ² , P (z)(˜x − z−τ x), (1) 2 2 2 2 σ² = σnk(1 − F )BP k + k (W − 1) ΩxP k , (3) where τ ≥ 0. The error weighting filter P (z) models the 2 impact that reconstruction errors have on each frequency. where σn is the variance of the quantization error, and Thus, it is application dependent. W (ejω) , ejωτ A(ejω)B(ejω), ∀ω ∈ R, (4) ¯ In this¯ paper, we restrict attention to the cases in which ¯P (ejω)¯ > 0, ∀ω ∈ [−π, π], i.e., P (z) has no zeros on the is a delay compensated version of AB, the frequency response unit circle. Additionally, we require: from x to x˜. The first term on the right hand side of (3) Constraint 1: A(z),B(z), F (z) and P (z) are stable. In corresponds to the variance of the frequency weighted quan- addition, F (z) is strictly causal (i.e., limz→∞ F (z) = 0). 4 The first part in the above constraint is required in order tization error in ². The second term in (3) accounts for the to avoid unbounded signals in the NS-DPCM converter. The frequency weighted linear distortion introduced by the filters in the encoder-decoder pair5. additional requirement on F (z) is needed for the feedback 2 2 The variance σn is related to σv via (2). From Assumption 1, loop in Fig. 1 to be well defined (see, e.g., [5, Chap. 4]). 2 2 2 2 Since the NS-DPCM architecture embeds a nonlinear ele- the latter is given by σv = kAΩxk + σnkF k . Combining ment (a scalar quantizer) within a feedback loop, exact analysis this result with (2) gives of quantization errors is, in general, a formidable task [15]. kAΩ k2 σ2 = x . (5) This has motivated the widespread use of an additive noise n γ − kF k2 model for quantization errors [1]–[3], [8]–[12]. This model allows one to study the converter via linear analysis tools. It When substituted into (3), this yields is usually formulated as follows: kAΩ k2k(1 − F )BP k2 Assumption 1: The quantization errors are i.i.d. random 2 x 2 σ² = 2 + k (W − 1) ΩxP k . (6) variables, uncorrelated with the input signal. 4 γ − kF k In order not to limit our subsequent analysis to a specific The above expression relates the filters A(z),B(z),F (z), type of scalar quantizer, the following is also assumed: and the quantizer signal-to-noise ratio γ, to the FWMSE. Assumption 2: The probability density function (PDF) of Minimization of this cost functional will yield expressions for v is not affected by the filters in the converter other than via the optimal filters and performance. 3 its second moment . 4 For comparison, we note that the cost functional (6), to- Under Assumption 2, any given type of scalar quantizer with gether with Assumptions 1 and 2, is also part of the analysis a fixed number of quantization levels leads to quantization in [1], [3] and [10], wherein equivalent optimization problems errors whose variance is proportional to the variance of its are addressed 6. input. This can be stated as 2 Finally, we note that, since σ² must be positive, (6) implies 2 2 σv Constraint 2: kF k < γ. 4 γ , 2 , (2) σn 4The actual bit-rate associated with γ will also depend on whether or not 2This may include, for example, any of the scalar quantizers described entropy coding is utilized to encode w, as discussed in Section V. in [14]. 5Note that perfect reconstruction (in the absence of quantization errors) is 3This can be expected to be a realistic approximation especially when x is achieved if and only if there is no linear distortion, i.e., when W = 1. a first-order Gaussian AR source. Indeed, it has been shown in [16] that the 6We note that quantization noise is not assumed white in [1], and that [10] prediction error in a DPCM converter with a first-order Gaussian AR input is only considers P = 1, restricts to first order AR Gaussian inputs, and 2 close to Gaussian, even for as few as two quantization levels. minimizes γ for a given σ² . IV. OPTIMAL NS-DPCM These two, in turn, determine the optimal A and B via (8) In this section we derive explicit analytic expressions for the and (4), respectively. optimal filters and the associated optimal performance for the We can now state the main result of this paper: NS-DPCM scheme, subject to a mild restriction. The analysis Theorem 1: If is based on a set of equations that the optimal filters must η2 γ + 1 > ΩxP a.e. on [−π, π], (13) necessarily satisfy. To facilitate the flow of ideas, all proofs 2 |ΩxP | are given in the Appendix. then the minimum achievable frequency weighted reconstruc- Minimization of (6) is simplified by noting that, for stable tion MSE of an NS-DPCM converter is and strictly causal F (z), it holds that kF k2 = k1 − F k2 − 1. η2 Substitution of this equality into (6) yields 2 2 ΩxP σ˘² , min σ² = . (14) kAΩ k2k(1 − F )BP k2 γ + 1 σ2 = x + k (W − 1) Ω P k2. (7) ² γ + 1 − k1 − F k2 x This minimum is achieved when the filters F , A and B satisfy: We then have the following result: s 2 Lemma 1: For given frequency responses F and W , the ηxP κ σ˘² |1 − F | = , |A| = 1 − 2 , (15a) optimal A(z) satisfies |ΩxP | |Ωx| |ΩxP | q s σ˘2 |Ω | σ˘2 |A| = κ |P | |Ω |−1 |1 − F | |W | , a.e. on [−π, π], (8) ² x ² x W = 1 − 2 , |B| = 1 − 2 , (15b) |ΩxP | κ |ΩxP | where κ > 0 is an arbitrary constant. This choice yields a.e. on [−π, π], where κ > 0 is an arbitrary constant. 4 2 2 h|1 − F | , |ΩxP | |W |i 2 σ = + k (W − 1) Ω P k . (9) V. DISCUSSION ² γ + 1 − k1 − F k2 x The results stated in Theorem 1 have very interesting con- 4 sequences. Some of these consequences are discussed below. Notice that the cost functional in (9) involves only two a) Optimality of Scalar Quantization Without Feedback: unknown functions, namely W and F . This makes it simpler It is easy to verify from the results in Theorem 1 that to work with than the functional in (7). scalar quantization without feedback is optimal if and only The optimization problem can be further simplified by if |ΩxP | is constant. In particular, it follows from (15) that writing the optimal W in terms of |1 − F |. Unfortunately, the if |ΩxP | = 1, a.e. on [−π, π], then the optimal NS-DPCM relationship between F and the optimal W , for the general converter reduces to a PCM converter with a fully whitening case, can only be stated implicitly, as shown next. −1 pre-filter and a post-filter satisfying |A| = κ |Ωx| and Lemma 2: For a given frequency response F , the optimal |B| = |A|−1 γ/(γ + 1). W satisfies b) Comparison with [3]: The minimum FWMSE for an ½ ¾ h|1 − F | , |ΩxP | |W |i |1 − F | NS-DPCM system derived by Noll in [3], neglecting fed back W = max 0 , 1 − . (10) η2 2 ΩxP γ + 1 − k1 − F k |ΩxP | quantization noise, is γ . Perhaps surprisingly, Theorem 1 shows that the optimal performance is slightly better (com- a.e. on [−π, π]. pare with (14)). Moreover, the corresponding optimal filters Remark 1: Notice that, from (10), W is a positive, sym- A , B and F derived in [3] satisfy |A | , κ |Ω |−1, metric and real valued function of ω. It then follows from (4) N N N N x |1 − F | , η |Ω P |−1 and B = A−1, respectively. Sub- that the product of the optimal filters A(z), B(z) must exhibit N ΩxP x N N stituting these expressions into (9) actually yields an FWMSE linearly decreasing phase. ζ 2 2 (xP )−1 σ²N , σ˘² · 1 , where ζ(xP )−1 is the spectral flatness In general, the presence of |W | in the inner product on the ζ(xP )−1 − γ+1 −1 2 2 right hand side of (10) makes it difficult, if not impossible, measure of (ΩxP ) . It then follows that σ²N > σ˘² for any 2 2 to express the optimal W explicitly in terms of F . However, finite γ, and that σ²N → σ˘² as γ → ∞. under specific conditions on γ, ΩxP and |1 − F |, an analytical c) Total Frequency Weighted Distortion is White: It explicit solution to (10) can be obtained, as follows: follows from (14) and (15) that, in an optimized NS-DPCM Lemma 3: Provided system, the PSDs of frequency weighted quantization noise |1 − F | and linear distortion are, respectively γ + 1 > h|1 − F | , Ω P i , a.e. on [−π, π], (11) h i x |Ω P | 2 2 2 2 2 2 2 x Sn0 , σn |1 − F | |B| P =σ ˘² 1 − σ˘² / |ΩxP | then, for a given frequency response F , the optimal W satisfies 2 2 2 2 2 SL , (W − 1) |ΩxP | = (˘σ² ) / |ΩxP | . h|1 − F | , |Ω P |i |1 − F | W = 1 − x , a.e. on [−π, π]. (12) From the above, one can see that when the condition of γ + 1 |ΩxP | Theorem 1 holds, the noise shaping effected by an optimal 4 NS-DPCM system is not “complete”, i.e., frequency weighted To summarize our results so far, we have shown that, pro- quantization noise is not white. However, the PSD of the total 2 vided (11) holds, F determines the optimal W through (12). frequency weighted error is white, since S² = Sn0 + SL =σ ˘² . d) Relation with the Reverse Water-Filling Paradigm: quantizer types (see, e.g. [14] and the references therein). The parametric Rate-Distortion formula for a Gaussian w.s.s. Assuming v to be Gaussian in the optimized NS-DPCM process and FWMSE as the distortion measure is given by the converter, ∆SNR can be approximated by −1.5, −2.45, −4.35 well known reverse water-filling paradigm (see, e.g., [17]). and −7.3 for a uniform quantizer with entropy coding (E.C.), 2 2 For σ² ≤ minω(Sx |P | ), it predicts total frequency weighted non-uniform quantizer with E.C., non-uniform quantizer op- distortion to be equally distributed over frequency. It also timized for MSE without E.C., and uniform quantization predicts the input signal to appear at the output with PSD without E.C. and a loading factor of 4, respectively9 (see [14]). 2 2 Sx |P | − σ² , i.e., less significant spectral components of x suffer higher attenuation. Interestingly, (13) is equivalent VI.CONCLUSIONS 2 2 to Sx |P | ≥ σ˘² , a.e. on [−π, π]. Furthermore, S² is flat, This paper has derived explicit analytic expressions for the 2 2 as discussed in c) above, and (15) yields Sx |W | |P | = best achievable performance (and optimal filters) for noise- 2 2 Sx |P | − σ˘² , in full agreement with the above prediction. shaping DPCM encoders. These expressions, which we be- e) Output of the Scalar Quantizer is White: It can be lieve to be novel, were found by accounting for fed back seen from (15a) that, unless |ΩxP | is constant, the optimal A quantization noise in the optimization. The results presented is not a full whitening filter for Ωx. Interestingly, however, it is in this paper simplify the analysis and design of NS-DPCM straightforward to verify that the optimal filters in (15) render converters, and provide valuable insight into the trade-offs a sequence w (see Fig. 1) with flat PSD. More precisely, Sw , inherent in linear feedback quantizers. 2 2 2 2 2 |Ωx| |A| + σ |1 − F | = κ , where κ is the same arbitrary n VII.APPENDIX constant that appears in (15a). A remarkable implication is that the quantized output of the optimized NS-DPCM converter A. Preliminary Result 2 can be efficiently translated into bits by means of a first-order Lemma 4: Let g ∈ L Rbe a given function such that g(ω) > 1 π ln(g(ω))dω entropy coder. 0, ∀ω ∈ [−π, π] and e 2π −π is finite. Then f) Rate-Distortion Analysis: The rate-distortion effi- R 1 π ln(g(ω))dω −1 ciency of any source encoding scheme (with quadratic error arg min h f, g i = e 2π −π g , f∈B+ as distortion measure) can be established by comparing its 2 where σx ½ Z π ¾ SNR , 2 + σ² B+ , f : R → R : 0 ≤ ln (f(ω)) dω < ∞ (18) against the upper bound derived by O’Neal in [18]. For the −π 2 jω jω is the set of non-negative log-integral functions. 4 case σ² ≤ minω Sx(e )P (e ), and restricting to Gaussian inputs, this bound [18, eq. (6)] can be written as7 Proof: SinceRln(·) is a monotonically increasing function,¡R ¢ minimization of fg is equivalent to minimizing ln fg . SNR , 6R − 10 log ζ2 − 10 log η2 , (16) maxdB x P From Jensen’s inequality and the constraint f ∈ B+, we obtain 2 µZ ¶ Z Z Z Z where R denotes the bit-rate (in bits per sample), ζx is the (a) (b) 2 ln fg ≥ ln fg = ln f + ln g ≥ ln g. spectral flatness measure of x and ηP is the minimum variance associated with P (see Section I). Notice that 6R is Shannon’s upper bound [19] for the SNR (in decibels) of encoding a Equality is obtained in (a) if and only if f = −1 Gaussian memoryless source. ηg , a.e. on [−π, π], for some ηR > 0. Inequality (b) becomes 1 π ln(g(ω))dω On the other hand, under the conditions of Theorem 1 and equality if and only if η = e 2π −π g−1. This com- using (14), the best achievable SNR of an NS-DPCM system pletes the proof. is given by B. Proofs of Lemmas 1-3 and Theorem 1 σ2(γ + 1) γ + 1 SNR = x = . Proof: [Lemma 1] The numerator of the first term on the η2 ζ2η2 ΩxP x P right side of (7), denoted here by N, is given by

In decibels, this ratio is 2 2 N , kAΩxk k(1 − F )BP k . (19) SNR = 10 log(γ + 1) − 10 log ζ2 − 10 log η2 . (17) dB x P We can use the Cauchy-Schwartz inequality to obtain 8 By comparing (17) and (16), we see that the SNR of the 2 2 NS-DPCM converter optimized via Theorem 1 departs from N ≥ h |AΩx| , |(1−F )BP | i =h |ΩxP | , |1−F | |W | i . (20) the information-theoretic upper bound (16) as follows: Substituting (20) into (7) yields (9), which is obtained with equality in (20). The latter is achieved if and only if |AΩx| = SNRdB −SNRmaxdB = 10 log(γ +1)−6R ≈ 10 log γ −6R. κ2 |(1 − F )BP | , a.e. on [−π, π], for arbitrary κ2 ∈ R+, or, 2 2 The difference ∆SNR , 10 log γ−6R for of Gaussian sources equivalently, |A| |Ωx| = κ |(1 − F )P | |W | , a.e. on [−π, π]. has long been known for a variety of scalar (memoryless) 9These figures are good approximations only for many quantization lev- 7 Hereafter, log denotes the base 10 logarithm. els. Better estimates for ∆SNR with few quantization levels can obtained 8Notice that the observations made in d) above validate this comparison. from [20]. This last equation leads directly to (8), completing the proof. to belong to the set of non-negative log-integral functions defined in (18), see, e.g. [21, Theorem 3.4.4] and [22]. Then, it Proof: [Lemma 2] Expanding the squared norm of the follows from Lemma 4 that the optimal |1 − F | is as in (15a). last term in (3) we obtain Substitution of the latter into (28) yields (14). It also follows from (15a) that the inequality in Lemma 3 is equivalent 2 2 2 2 © 2 2 ª σ² = σnk(1−F )PBk +kΩxP ABk −2Re hΩxP ,W i to the condition required by the theorem. This validates our 2 + kΩxP k . (21) initial supposition. Notice also that the latter inequality also guarantees that k1−F k2 < γ +1, as required by Condition 2. Substituting (4) into the above equation yields Finally, substituting (15a) into (12), (8) and (4) yields the 2 2 © 2 2 −jωτ ∗ ª 2 σ² =kTPBk −2Re hΩxP e A ,Bi +kΩxP k , (22) remaining equalities of (15). This completes the proof. 2 2 2 2 1/2 where T , (σn |1 − F | + Ωx |A| ) . Rearranging terms, ° ° ° ° REFERENCES °¡ ¢ °2 ° 2 °2 2 ° 2 2 −jωτ ∗ P ° °σnΩxP |1 − F |° σ² = ° T B − Ωx e A ° + ° ° , [1] E. Kimme and F. Kuo, “Synthesis of optimal filters for a feedback T T quantization system.” IEEE Trans. Circuit Theory, vol. CT-10, pp. 405– 2 −jωτ A∗ 413, September 1963. which is clearly minimized if and only if B = Ωx e T 2 [2] R. Brainard and J. Candy, “Direct-feedback coders: design and perfor- a.e. on [−π, π]. Multiplying the latter by A yields10 mance with television signals.” Proc. IEEE, vol. 57, no. 7, pp. 776–786, July 1969. [3] P. Noll, “On predictive quantizing schemes,” Bell. Syst. Tech. J., vol. 57, 2 2 |Ωx| |A| no. 5, pp. 1499–1532, May-June 1978. W = , a.e. on [−π, π], (23) [4] N. Jayant, “Digital coding of speech waveforms: PCM, DPCM, and DM 2 2 2 2 σn |1 − F | + |Ωx| |A| quantizers,” Proc. IEEE, vol. 62, no. 5, pp. 611–633, May 1974. 2 [5] S. R. Norsworthy, R. Schreier, and G. C. Temes, Eds., Delta–Sigma where σn is given by (5). By direct substitution of (8) into (23), Data Converters: Theory, Design and Simulation. Piscataway, N.J.: and after some algebra, the optimal W is found to satisfy (10). IEEE Press, 1997. This completes the proof. [6] N. Jayant and P. Noll, Digital coding of waveforms. Principles and approaches to speech and video. Englewood Cliffs, NJ: Prentice Hall, Proof: [Lemma 3] Define 1984. h|1 − F | , |Ω P | W i [7] F. Baqai, J.-H. Lee, A. Agar, and J. Allebach, “Digital color halftoning,” α , x , (24) Signal Processing Magazine, IEEE, vol. 22, no. 1, pp. 87–96, Jan 2005. γ + 1 − k1 − F k2 [8] H. Spang, III and P. Schultheiss, “Reduction of quantizing noise by use of feedback,” IRE Trans. Comm. Syst., vol. CS-10, no. 4, pp. 373–380, and suppose that Dec. 1962. −1 [9] B. S. Atal and M. Schroeder, “Predictive coding of speech signals and α |1 − F | |ΩxP | ≤ 1, a.e. on [−π, π]. (25) subjective error criteria,” IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-27, no. 3, pp. 247–254, June 1979. Then, from (10), [10] O. Guleryuz and M. Orchard, “On the DPCM conpression of Gaussian −1 autoregressive sequences,” IEEE Trans. Inf. Theory, vol. 47, no. 3, pp. W = 1 − α |1 − F | |ΩxP | , a.e. on [−π, π]. (26) 945–956, March 2001. 2 [11] S. K. Tewksbury and R. W. Hallock, “Oversampled, linear predictive h|1−F |,|ΩxP |i−αk1−F k Substituting (26) into (24) yields α = γ+1−k1−F k2 . and noise-shaping coders of order N > 1,” IEEE Trans. Circuits Syst., Thus, vol. 25, no. 7, pp. 436–447, July 1978. h|1 − F | , |ΩxP |i [12] D. Marco and D. L. Neuhoff, “The validity of the additive noise model α = . (27) for uniform scalar quantizers,” IEEE Trans. Inf. Theory, vol. 51, no. 5, γ + 1 pp. 1739–1755, May 2005. Substituting this into (26) yields (12). Notice that (11) guar- [13] U. Grenander and G. Szegö, Toeplitz forms and their applications. Berkeley, Calif.: University of California Press, 1958. antees that the denominator on the right hand side of (24) [14] A. Gersho, “Principles of quantization,” IEEE Trans. Circuits Syst., is strictly positive. The proof is completed by noting that vol. 25, no. 7, pp. 427– 436, Ju. 1978. substitution of (27) into (11) gives the inequality (25), thus [15] R. M. Gray, “Spectral analysis of quantization noise in a single-loop Sigma–Delta modulator with dc input,” IEEE Trans. Commun., vol. 37, validating our initial supposition. no. 6, pp. 588–599, June 1989. Proof: [Theorem 1] Suppose the optimal F is such [16] D. Arnstein, “Quantization error in predictive coders,” IEEE Trans. that (11) in Lemma 3 holds. Then, one can substitute (12) Commun., vol. COM-23, no. 4, pp. 423–429, April 1975. h|1−F |,|ΩxP |i 2 [17] T. Berger, Rate distortion theory: a mathematical basis for data com- h|1−F | , |ΩxP |− |1−F |i 2 γ+1 pression. Englewood Cliffs, N.J.: Prentice-Hall, 1971. into (9) to obtain σ² = γ+1−k1−F k2 + 2 [18] J. O’Neal Jr., “Bounds on subjective performance measures for source h|1−F |,|ΩxP |i 2 (γ+1)2 k1 − F k . After some algebra, this becomes encoding systems,” IEEE Trans. Inf. Theory, vol. IT-17, no. 3, pp. 224– 231, May 1971. 2 2 h|1 − F | , |ΩxP |i [19] C. E. Shannon and W. Weaver, The Mathematical Theory of Communi- σ² = . (28) cation. Urbana: Univ. of Illinois Press, 1949. γ + 1 [20] P. Noll and R. Zelinski, “Bounds on quantizer performance in the low bit-rate region,” vol. COM-26, no. 2, pp. 300–304, February 1978. Requiring F (z) to be stable and strictly causal¯ (from Con-¯ ¯ jω ¯ [21] M. M. Serón, J. H. Braslavsky, and G. C. Goodwin, Fundamental straint 1) is equivalent to requiring the function 1 − F (e ) Limitations in Filtering and Control. London: Springer Verlag, 1997. [22] M. R. Aaron, R. A. McDonald, and E. Protonotarios, “Entropy power 10Notice that W in (23) describes the anti-causal form of the Wiener filter loss in linear sampled data filters,” Proc. IEEE (letters), vol. 55, no. 6, 2 for a w.s.s. signal with PSD Sx |A| corrupted by uncorrelated additive noise pp. 1093–1094, June 1967. 2 2 with PSD σn |1 − F | . This filter is known to minimize error variance.