Optimal Noise-Shaping DPCM Milan S. Derpich, Eduardo I. Silva, Daniel E. Quevedo and Graham C. Goodwin School of Electrical Engineering and Computer Science, The University of Newcastle, NSW 2308, Australia {milan.derpich, Eduardo.silva}@studentmail.newcastle.edu.au, [email protected], [email protected]. Abstract— This paper presents novel results on the optimal relied upon heuristic design methods [2], [9]. Since optimal design of Noise-Shaping Differential Pulse-Coded Modulation performance can, in general, only be attained by arbitrary order coders. The main contribution resides in the derivation of explicit filters designed accounting for fed back quantization noise, an analytic formulas for the optimal filters and the minimum achievable frequency weighted reconstruction error. A novel exact characterization of the optimal performance (and filters) aspect in the analysis is the fact that we account for fed-back for NS-DPCM converters has remained an open problem. quantization noise and that we make no restrictions on the order In this paper we derive an explicit analytic expression for of the filters deployed. the optimal performance (and filter frequency responses) for I. INTRODUCTION NS-DPCM converters. We characterize the scalar quantizer via its signal-to-noise ratio, and adopt a white quantization noise Analog-to-Digital converters which utilize a scalar quantizer model [12]. The performance bound obtained corresponds to and linear, time invariant filters in a feedback loop have been the minimum FWMSE that can be achieved by an NS-DPCM extensively employed as a source coding method since the encoder-decoder with any linear, time-invariant filters. A key concept was first introduced in the 1960’s. The generalized departure from [3] (which, to the best of our knowledge, gives form of this architecture, which we denote Noise Shaping the only currently available explicit analytic solutions to the Differential Pulse Code Modulation1 (NS-DPCM), can be problem), is that we account for fed back quantization noise. represented as in Fig. 1. The filters in a NS-DPCM system This allows us to derive exact expressions. allow one to account for the correlation between consecutive Our results show that an optimal NS-DPCM converter ex- input samples, and to spectrally shape the quantization noise hibits several interesting properties. These include a spectrally in the output, so as to minimize the frequency weighted flat frequency weighted error spectrum, and a white signal mean square reconstruction error (FWMSE). Special cases at the input of the scalar quantizer. We also show that, for of the NS-DPCM architecture include ∆-Modulators, DPCM AR Gaussian sources, the rate-distortion efficiency with the converters [4], and noise-shaping converters, such as one and optimal filters depends only on how efficient the embedded multi-bit Sigma-Delta modulators [5]. NS-DPCM converters scalar quantizer is at quantizing nearly Gaussian samples. are extensively used in the context of audio compression [6], digital image half-toning [7] and oversampled A/D conver- Notation and Preliminaries sion [8]. We use standard vector space notation for signals. For example, x is used to denote {x(k)} . We also use z as v w k∈Z x A(z) Q B(z) x˜ the argument of the z-transform. Given two square integrable complex valued functions f(ω) and g(ω) definedR over [−π, π], 1 π ∗ we adopt the inner product h f, g i , 2π −π f(ω) g(ω)dω, n where ()∗ denotes complex conjugation. We denote the usual Encoder F (z) Decoder q R 1 π 2 2-norm as kfk , 2π −π |f(ω)| dω . If F (z) is a transfer Fig. 1: Noise Shaping-DPCM Encoder and Decoder function, then we use the short hand notation F to refer to the associated frequency response F (ejω), ω ∈ [−π, π]. If I is a Provided that the input power spectral density (PSD), fre- set, then we will write “a.e. on I” (almost everywhere on I) quency weighting error criterion, and scalar quantizer char- as a short hand notation for “everywhere on I except at most acteristics are known, the design of an NS-DPCM converter on a zero Lebesgue measure set of points”. 2 that achieves minimum FWMSE amounts to finding the cor- We use σx to denote the variance of a given wide sense sta- jω responding optimal filters. This has been an intense area of tionary (w.s.s.)© randomª processR x, having PSD Sx(e ). Note that σ2 , E x(k)2 = 1 π S (ejω)dω = kΩ k2, where research for at least 40 years. However, available to date results x 2π −π x √x on optimal filter design for NS-DPCM encoders have been Ωx is a frequency response satisfying |Ωx| , Sx , ∀ω ∈ [−π, π]. For a given function f :[−π, π] → C, we define η , obtained assuming either fixed, finite order filters [1], [2], [8]– ³ R ´ f 1 π [10], negligible fed back quantization noise [3], [11], or have exp 2π −π |f(x)|dx (provided this integral converges). This allows one to describe the Kolmogorov’s minimal prediction 1The same configuration can be found under different names in the error variance [13] of a w.s.s. process x via η2 , η = η2 . literature, e.g.: error feedback systems [1], direct feedback coders [2] and x Sx Ωx DPCM with noise feedback [3]. The spectral flatness measure of a w.s.s. process x is denoted 2 ηx by ζx , 2 . It is easy to show that 0 ≤ ζx ≤ 1, and that where γ is the signal-to-noise ratio of the scalar quantizer (not σx jω ζx = 1 if and only if Sx(e ) is constant a.e. on [−π, π]. to be confused with that of the NS-DPCM encoder-decoder system). γ depends on the number of quantization levels, II. NS-DPCM MODEL the PDF of the signal being quantized and the companding As foreshadowed in the introduction, we consider the gen- characteristics of the scalar quantizer itself4. eral form of an NS-DPCM architecture shown in Figure 1. In our model, the input sequence x is assumed to be a zero III. FORMULATION OF THE OPTIMIZATION PROBLEM mean, w.s.s. random process, with known PSD S = |Ω |2 x x Our ultimate goal is to find the frequency responses of the satisfying S (ejω) > 0, a.e. on [−π, π]. The element denoted x filters A, B, and F that minimize the variance of ² under by Q describes a scalar quantizer, with given and known Assumptions 1 and 2, and for given and known Ω , P and γ. characteristics2. For each input v(k), k ∈ Z, it outputs w(k) x The quantity σ2 so obtained will constitute the (achievable) and generates the quantization error n(k) , w(k)−v(k). The ² lower bound on the FWMSE for the NS-DPCM converter. three discrete-time filters A(z), B(z) and F (z) in Fig. 1 are design choices. Towards the above goal, we first derive an expression that To asses performance, we introduce the delay-compensated relates the decision variables to the error measure that we wish frequency weighted error to minimize. From Fig.√ 1, equation (1), Assumption 1, and recalling that |Ωx| = Sx , ∀ω ∈ [−π, π], we have ² , P (z)(˜x − z−τ x), (1) 2 2 2 2 σ² = σnk(1 − F )BP k + k (W − 1) ΩxP k , (3) where τ ≥ 0. The error weighting filter P (z) models the 2 impact that reconstruction errors have on each frequency. where σn is the variance of the quantization error, and Thus, it is application dependent. W (ejω) , ejωτ A(ejω)B(ejω), ∀ω ∈ R, (4) ¯ In this¯ paper, we restrict attention to the cases in which ¯P (ejω)¯ > 0, ∀ω ∈ [−π, π], i.e., P (z) has no zeros on the is a delay compensated version of AB, the frequency response unit circle. Additionally, we require: from x to x˜. The first term on the right hand side of (3) Constraint 1: A(z),B(z), F (z) and P (z) are stable. In corresponds to the variance of the frequency weighted quan- addition, F (z) is strictly causal (i.e., limz→∞ F (z) = 0). 4 The first part in the above constraint is required in order tization error in ². The second term in (3) accounts for the to avoid unbounded signals in the NS-DPCM converter. The frequency weighted linear distortion introduced by the filters in the encoder-decoder pair5. additional requirement on F (z) is needed for the feedback 2 2 The variance σn is related to σv via (2). From Assumption 1, loop in Fig. 1 to be well defined (see, e.g., [5, Chap. 4]). 2 2 2 2 Since the NS-DPCM architecture embeds a nonlinear ele- the latter is given by σv = kAΩxk + σnkF k . Combining ment (a scalar quantizer) within a feedback loop, exact analysis this result with (2) gives of quantization errors is, in general, a formidable task [15]. kAΩ k2 σ2 = x . (5) This has motivated the widespread use of an additive noise n γ − kF k2 model for quantization errors [1]–[3], [8]–[12]. This model allows one to study the converter via linear analysis tools. It When substituted into (3), this yields is usually formulated as follows: kAΩ k2k(1 − F )BP k2 Assumption 1: The quantization errors are i.i.d. random 2 x 2 σ² = 2 + k (W − 1) ΩxP k . (6) variables, uncorrelated with the input signal. 4 γ − kF k In order not to limit our subsequent analysis to a specific The above expression relates the filters A(z),B(z),F (z), type of scalar quantizer, the following is also assumed: and the quantizer signal-to-noise ratio γ, to the FWMSE.