Estimating Frequency Characteristics of Quantization Noise for Performance Evaluation of Fixed Point Systems

18th European Signal Processing Conference (EUSIPCO-2010) Aalborg, Denmark, August 23-27, 2010

ESTIMATING FREQUENCY CHARACTERISTICS OF QUANTIZATION NOISE FOR PERFORMANCE EVALUATION OF FIXED POINT SYSTEMS

Karthick Parashar, Daniel Menard, Romuald Rocher and Olivier Sentieys INRIA/ IRISA, University of Rennes-1 6 rue de Kerampont, Lannion, France phone: + (33) 296469078, email: {ﬁrstname}.{lastname}@irisa.fr

ABSTRACT total power consumed and the delay in execution by the system. Cost evaluation is typically the sum of individual com- Word-length optimization of signal processing algorithms is ponent costs in the system. The performance of any signal a necessary and crucial step for implementation. System processing system is evaluated in terms of a system specific. level performance evaluation happens to be the most time Typically, parameters such as output noise power, signal to consuming step during word-length optimization. Analytical noise ratio (SNR), bit error rate (BER) are used for this pur- techniques have been proposed as an alternative to simulation pose. Evaluating performance requires the knowledge of the based approachto accelerate this step. The inability to handle system functionality and is hence not trivial. all types of operators analytically and the increasing diver- A hierarchical methodology has been developed [2] for sity and complexity of signal processing algorithms demand efficient optimization and performance evaluation of large a mixed evaluation approach where both simulation and an- systems. At the heart of this methodology is the single-noise alytical techniques are used for performance evaluation of source model, which is used to sub-divide the optimization the whole system. The interoperability between simulation problem into smaller optimization problems. In this paper, and analytical techniques requires study of noise sources and we further develop the single-noise source model to incorpo- noise propagation characteristics. While the noise power and rate the noise frequency response estimation. noise PDF have been studied, the output noise power distri- The paper is organised as follows, the next section pro- bution has not been studied. This paper addresses the prob- vides motivation and previous related work. Section 3 de- lem of power spectral density estimation of the noise analyt- scribes the hierarchical framework, the role of single-noise ically. This paper also proposes to use the spectral density source model and the importance of estimation of the fre- estimate for noise power calculation by having an approxi- quency response. Section 4 develops the necessary mathe- mate filter thereby accelerating the process of performance matical framework to deal with non-linear time-variant but evaluation. stationary sub-systems. Section 5 applies the proposed framework for output noise spectrum analysis of a non-linear 1. INTRODUCTION second order Volterra filter and presents the results obtained Signal processing algorithms are increasingly finding their thereof. In Section 6, the paper concludes with a summary of way into modern electronic gadgets. While developing the contributions and discuss future work. algorithm to fulfil objectives is quite a challenge, optimal implementation of these algorithms such that the implementa- 2. MOTIVATION tion cost is kept minimal consists of many tough challenges. Simulation based techniques and analytical models have One such challenge is to choose optimal computational ker- been proposed to measure performance of fixed-point sys- nels having the right word-lengths to perform the signal pro- tems. While the simulation based techniques [3, 4] are appli- cessing computations. It is an obvious choice to use fixed cable universally, long execution times is a deterrent to use point kernels over their floating point counter parts to save them always. On the other hand, the analytical models pro- area, power and time. vide closed form expressions for evaluating noise power for Finite precision word-lengths introduce quantization er- different choice of word-lengths. The noise power can then rors into the system. These errors have been studied with be used to evaluate the performance metric of the system. the help of Widrow quantization [1] model which treats such However, analytical models have not been developed thus noise as small signal perturbations more popularly referred far for all types of systems (e.g. un-smooth operators such to as quantization noise. The amount of quantization noise as decision operators) and are applicable only on a subset of introduced into the system brings to the forefront an opti- systems. mization problem which makes a choice between the cost Complex signal processing systems are often described and accuracy of the system. with well known signal processing sub-systems. The exist- The process of optimizing word-lengths is typically an ing techniques for performanceevaluation such as [5] require iterative process where an optimization heuristic is used to the system to be flattened all the way to the operator level determine optimality by comparing the system cost and the in order to consolidate the noise contributions to the output. corresponding system performance. While good optimiza- This is not only cumbersome but it is also sometimes impos- tion heuristics can reduce the number of iterations, efficient sible as there can be sub-systems made of operators which techniques for cost and performance evaluation reduce the do not have an analytical model. Hence a hybrid approach to time taken for each optimization iteration. The cost of the measure performance that can exploit the good of both sim- system is generally defined as a function of area occupied, ulation and analytical approaches is formulated. Also, the

© EURASIP, 2010 ISSN 2076-1465 552 existing optimization techniques concentrate mostly on opti- system. It is easy to see from comparison between Equa- mizing word-lengths at the operator level and treat the word- tions 1 and 2 that the sub-system noise power replaces the length of each operator as an optimization variable. The hier- noise due to word-length quantization in the global optimiza- archical methodology proposed in [2] uses a divide and con- tion problem. quer approach which makes it easier to scale the optimization To evaluate the performance at the system level, the entire algorithm on bigger systems. The single-noise source model system is simulated at the system level with double precision which is at the heart of this approach captures the effect of floating point accuracy only once to collect the data required quantization noise at the sub-system level for use in the sys- for use with the noise model. The fixed-point system is then tem level optimization. replaced by the single-noise source and the double precision In a hierarchical approach, it is important that the noise floating point system. Any greedy optimization technique is generated in the sub-systems is propagated across hierar- used to optimize the sub-system optimization problem stated chies and eventually to the system output. The sub-systems in Equation 2. The same could be applied recursively up to through which the noise is propagated could be sensitive to the top-level design abstraction to solve the problem defined noise distribution and frequency spectrum. Hence, instead of by Equation 1. a simplistic metric like noise power, the quantization noise While the proposed divide-and-conquer approach re- spectral density and distribution needs to be studied at the duces the complexity of the problem by using just the noise output of every sub-system. In the case of sub-systems made power, performance evaluation at the system level is not a of arithmetic operators, it has already been shown experi- simple function of the noise power. It is clear from first prin- mentally that the noise distribution is a parametric summa- ciples of signal processing that the performance of the total tion of uniform and Gaussian distributions [6]. In this paper, system performance would depend on the sub-system noise we focus on characterizing the spectral characteristics of the spectral and distribution characteristics. Hence, from the per- sub-system generated quantization noise. spective of performance evaluation, study of noise spectral A technique to shape the noise spectral characteristics for density and distribution is important. While the word-length LTI systems has been proposed in [7]. In this approach, the quantization effects have been extensively studied in terms fixed point design optimization constraints are specified by of quantization noise power, little has been done to under- means of spectral characteristics of noise. The proposed al- stand and analyse other properties of quantization noise at gorithm determines the word-lengths such that the spectral the sub-system level. constraints are met while minimizing area. The proposed technique is interesting as it relates word-length choice to the 3.1 Single Noise Source Model spectral density function. However, the problem consideration in this approach is converse of the problem considered in The single-noise source model allows the use of output noise this paper. Also, the proposed technique has not been applied power as given by Equation 1. To explain this central theme, on non-linear or time-variant systems. Thus, the problem of consider a sub-system B in a hierarchically defined system B spectral estimation of the quantization noise spectral density with input x and output s as shown in Figure 1. The noises bx remains largely unexplored. and bs are associated with signals x and s respectively. The This paper demonstrates a simple technique to estimate power Pi in Equation 1 corresponds to the noise power of the the power spectral density of the output quantization noise of signal s. The total quantization noise bs at the output of the the sub-system and proposes a linear filter model to model system consists of two components bsg and bst for the noise the effect of quantization at various sub-system hierarchical generated within the system and noise transmitted through levels. The proposed model is shown to be applicable to non- the system respectively. linear, time variant systems. bx T˜ 3. THE HIERARCHICAL APPROACH bst The hierarchical approach in [2], a divide-and-conquer strat- Sub-System s + b egy is used to solve the multi-variable optimization problem. x s + s The system is hierarchically divided at the boundaries of pre- B defined sub-systems recursively. The problem of system op- bsg timization in this hierarchical approach is formulated as ˜ bg G min(C (P)) such as λ (P) ≥ λob j (1) P where, = [P0,P1,...,PNb−1] is a vector of sub-system noise Figure 1: Single Noise Source Model powers at any given hierarchy. Each of Pi represents the total th noise power at the output of the i sub-system. This tech- The noise bx(n) associated with the input signal x(n) is nique uses the sub-system noise power as the optimization independent of the signal. The effect of this noise at the out- variable. put bst is calculated by passing it through the noise propaga- Each sub-system is optimized by solving the original op- tion filter T˜ which modifies the power spectrum of bx like the timization problem which is defined at the word-length level sub-system B. The noise bsg, generated in the sub-system B stated as is modeled by passing the single-noise source bg through the noise generation filter G˜. The noise generation filter shapes min(Ci(WDi )) such as fP (WDi ) < Pi (2) i the spectral characteristics of the noise to represent the effect where WDi is the vector of word-lengths and Ci, fPi are re- of quantization noise generated within the sub-system B. It spectively the cost and performance functions of the ith sub- has been shown in [6] that the output PDF is not uniform and

553 is in fact closer to being a Gaussian (due to central limit the- The noise propagation model for all arithmetic operator- orem). Hence, noise source bg is modeled as white Gaussian. based functions (including non-linear) can be linearized [5]. The noise model used for propagation of quantiza- Hence without any loss of generality, the propagation and tion noise through arithmetic operators [5] essentially lin- generation equivalent system functions is considered to be earizes the noise propagation for all types of arithmetic non- linear and time-varying system. linearities under the conditions described by the Widrow model [1]. Following the Widrow model for quantization 4.1 Windowing for Stationarity noise, all the operator noise sources are known to be uniform Frequency spectrum of the noise is meaningful only when and white. The noise propagation models however tend to be the noise is stationary. However, not all signals are station- time varying in nature. ary. One way of handling this problem is by windowing the Taking both noise propagated and generated into consid- signals such that they are locally stationary [8]. A patholog- eration and given that the noise models are linear, the total ical scenario is where there are too many such windows and noise at the output of sub-system Bi is given by the windowing overhead out weighs the benefits. In such extreme cases, one has to revert back to simulation based bs(n)= g˜(n) bg(n)+t˜(n) bx(n). (3) techniques. The windowing is performed such that all relevant sig- Whereg ˜(n) and t˜(n) are the impulse response of the filters G˜ nals in the given window may be considered stationary. This and T˜ from Figure 1 respectively. As long as the output noise sense of pseudo-stationarity or temporally stationarity en- is stationary, it is possible to compute its noise power density ables estimation of the noise power spectrum. It is not always jω spectrum Sbsbs (e ). Deriving an expression for analytical easy to find windows where the so called pseudo-stationarity evaluation of the noise power spectrum at the output is the would exist. In such cases, the definition of pseudo needs primary issue addressed in this paper. to be approximated such that the performance degradation is still conservative. 3.2 Accelerating Performance Evaluation While evaluating the sub-system noise, the noise power, its 4.2 Noise Power Spectrum spectral density and distribution are estimated. The noise The noise power spectral distribution at the sub-system out- spectrum is represented by an equivalent linear filter whose put is the magnitude response obtained by calculating the magnituderesponse is a close approximationof the estimated Fourier transform of the autocorrelation function. spectrum. The linear filters replace the original sub-system The output of any time varying system H is obtained by while preserving the topology of the signal network. The the convolution operation of the system function with the in- idea here is to provide a framework which can be used for put signal. Consider a time window W wherein all the coef- CAD-based automation for mixed performance evaluation ficients of the varying system H are stationary. The output technique. noise by(n) contributed by a noise bx(n) at the input given by The noise-equivalent is used in order to eliminate the ∞ necessity of a full simulation to estimate the performance by(n)= ∑ h(k,n)bx(n − k) (4) degradation with every combination of word-lengths. How- ∞ ever, the original signal is required for evaluating systems k=− which cannot be handled analytically (such as decision op- Here, h(k,n) can be thought of as the pseudo impulse re- erators). For handling such cases, the signal at the output of sponse of the time varying system H at any discrete point in th the sub-system which is the sub-system response to the in- time n at the k tap. The autocorrelation function Rbyby of the put signal is obtained by means of a floating point simulation noise by(n) at the output of such a system can be expanded and stored in memory. The stored signal along with the filters and simplified as successfully mimic the behavior of the original sub-system, thus providing an analytical means to perform the perfor- Rbyby (m)= E{by(n)by(n + m)} mance estimation of the fixed point system without having ∞ ∞ to simulate the sub-system. This method provides a seamless = ∑ ∑ E{h(k,n)h(r,n + m)... framework for interoperability between usage of simulation k=−∞ r=−∞ and analytical modeling for performance evaluation of the bx(n − k)bx(n + m − r)} (5) entire system. This technique also enables traversing hierar- chies and utilizes the sub-system filter model to construct its Applying Fourier transform on both sides of Equation 5, the equivalent counterparts at any higher hierarchical level. power spectral density of the noise at output can be written and expanded as 4. ESTIMATING FREQUENCY RESPONSE ∞ ∞ ∞ jw Sbyby (e )= ∑ E{ ∑ ∑ h(k,n)h(r,n + m)... Consider a sub-system B which is a non-linear time variant m=−∞ k=−∞ r=−∞ system as shown in Figure 1 with noise generation and noise − jωm propagation system functions T˜ and G˜ respectively. Both of bx(n − k)bx(n + m − r)}e (6) these filters are similar for analysis in the context of this pa- So, per except that, the noise power spectrum of the input to the ∞ ∞ ∞ generation filter bg(n) is always known to be white while the jw , , ... input noise b (n) does not need to be white. Thus the results Sbyby (e )= E{ ∑ ∑ ∑ h(k n)h(r n + m) x ∞ ∞ obtained in the following Equations are equally applicable to m=− k=−∞ r=− − jωm both system functions. bx(n − k)bx(n + m − r)e } (7)

554 By taking m = p + r − k in Equation 6, the expression response depends on the number of coefficients whose auto power spectral density can be written as correlation and cross correlations are to be found. When there are Nd number of delays in the noise expression, the ∞ ∞ ∞ total number of correlation terms Nc that need to be calcu- ω 2 jw j k lated is Nd . Sbyby (e )= E{ ∑ ∑ ∑ h(k,n)e ... p=−∞k=−∞ r=−∞ Each of the delay terms in the noise expression is associ- − jω p ated with a coefficient. These coefficient terms are typically bx(n − k)bx(n + p − k)e ... dependenton the input signal and hence need to be calculated − jωr h(r,n + p − k)e } (8) for every input sample Ns. The effort involved in computing ∞ each correlation at 0 is O(Ns). Hence the total time for eval- jw jωk Sbyby (e )= E{ ∑ h(k,n)e ... uating the filter expression TExpr is given by k=−∞ ∞ 2 ω TExpr = Nd .tCorr + Nd.tCoef f . (14) ∑ h(r,n + p − k)e− j r}... r=−∞ ∞ Where tCorr is the time for computing one correlation − jω p term and TCoef f is the average time to evaluate each coeffi- E{ ∑ bx(n − k)bx(n + p − k)e } (9) p=−∞ cient expression. The multiplication and addition operations | {z } required during correlation and the time for coefficient ex- jω Sbxbx (e ) pression are proportional to the number of samples. Hence, it is possible to estimate the time required for computing cor- Thus the noise spectrum at the output can be written as relation terms and coefficient terms can be written as jw ˜ jω jω α Sbyby (e )= Hk,r(e )Sbxbx (e ) (10) tCorr = Corr.Ns (15) α . jω tCoef f = Coef f Ns (16) where H˜k,r(e ) is defined as ∞ ∞ Where αCorr and αCoef f are proportional constants. jω jωk − jωr Though there is a contribution of the N term in the com- H˜k,r(e )= E{ ∑ h(k,n)e ∑ h(r,n + p − k)e } d k=−∞ r=−∞ plexity estimation, it is of the order of few tens of elements. | {z }| {z } Whereas the order of Ns is in millions of samples. Thus, it ˜ − jω ˜ jω Hk(e ) Hr(e ) can be concluded that the computational complexity to deter- (11) mine the analytical expression for the system is O(Ns). The h(k,n) and h(r,n) terms in Equation 11 are the time- In a simulation based approach the performance evalu- varying coefficients of the system under consideration. By ation takes O(Ns) time and it has to be repeated for every expanding the Equation 11 with (k,r) ∈ (−∞,∞), and uti- iteration. In the proposed analytical method, though the time lizing the available symmetry, the expression for the power to arrive at the analytical expression is O(Ns), it is performed spectrum of the transfer function can be written as follows only once which can be regardedas a pre-processingstep and ∞ ∞ hence better than any simulation based approach. jω H˜ (e )= ∑ ∑ (E{hkhk+∆} + E{hk+∆hk})cos(∆ω) k=−∞ ∆=0 5. EXPERIMENT AND RESULTS (12) Clearly, the computation of H˜ requires the correlation 5.1 Volterra Filter evaluated at 0 ∀(k,∆) in Equation 12. This is calculated only While it is trivial to obtain the transfer function of an LTI sys- once from one single floating point simulation of the system tem, an equivalent propagation and generation filter is con- and is independent of the fixed point format. Hence, it is sidered for a non-linear second-order Volterra filter. Given used in every iteration of the optimization process thereby input x(n), the output y(n) for the Volterra filter is given by delivering the acceleration during optimization over simula- 2 2 tion based approaches. y(n)= a11x (n)+ a22x (n − 1)+ ... In case of LTI systems, the filter coefficients are constant. a x(n)x(n − 1)+ a x(n)+ a x(n − 1). (17) Hence it is possible to write the spectrum of the filter as 21 1 2 ∞ ∞ By replacing the operators with their equivalent noise model, jω H˜ (e )= ∑ ∑ (hihi+∆ + hi+∆hi)cos(∆ω) (13) the noise output expression for the Volterra filter is i=−∞ ∆=0 by = {2a11x(n)+ a1 + a12x(n − 1)}bx(n)+ From Equation 13, it can be seen that the expression for | {z } jω jω 2 α H˜ (e ) degenerates to |H˜i(e )| which is a classical result 1 obtained in the case of LTI systems, thus also verifying the {2a22x(n − 1)+ a2 + a12x(n)}bx(n − 1)+ bg(n) (18) expression for the spectrum in the LTI case. | {z } α2 4.3 Complexity Analysis where bg groups together all the noise generated inside the The proposed technique attempts to provide a closed form system. In this example, all the noise generated within the expression as given in Equation 13 to represent the noise system happens to be white and is not very interesting to coloring filter. The effort for computation of the frequency study. Also, in the course of study, no quantization effects

555 are introduced within the filter. Therefore, only the propaga- original sub-system is replaced by the double precision float- tion filter is presented. The propagation filter is given as ing point system and linear time-invariant filters whose magnitude spectrum is a close approximation of the noise power h˜i(n)= α1(n)bx(n)+ α2(n)bx(n − 1) (19) spectrum. The floating point data and the filters are used for all noise power calculation during every iteration during The terms α1(n) and α2(n) are time varying. Thus, by ex- optimization. The idea is to be able to seamlessly use this panding the transfer function Equation 12, the spectrum of technique in a complex and hierarchically defined signal pro- the filter becomes cessing system to find equivalent approximations at all hierarchical levels. Using this technique accelerates the perfor- jω 2 |H(e )| = (Rα1,α1 (0)+ Rα2,α2 (0)) + mance evaluation process in situations where both simulation (Rα ,α (0)+ Rα ,α (0))cos(ω) (20) and analytical techniques are used. This technique also helps 1 2 2 1 in noise power budgeting in situations where a divide-and- conquer algorithm is used to for optimization. where Rαi,α j (0) are the cross correlation and auto correlation functions. This paper contributes by providing the necessary framework and a generic filter coefficient formula for computing the spectrum of any arithmetic based signal processing sys- 0 tem. The acceleration obtained is due to the fact that, a fixed −20 point evaluation of the system is not required during every iteration of the optimization algorithm. Though the proposed −40 method requires windowing in case of non-stationary signals, −60 it provides an alternative to complete simulation. As long as it is possible to define pseudo-stationary windows over the −80 Simulation (Color Noise) signal, this alternative stands to gains over a pure simulation Analytical (Color Noise) Noise Power (in dB) −100 approach. Simulation (White Noise) Analytical (White Noise) −120 0 0.5 1 1.5 2 2.5 3 3.5 Frequency REFERENCES [1] B. Widrow, I. Kollár, and M.-C. Liu, “Statistical Theory Figure 2: Output noise power spectrum estimation for of Quantization,” IEEE Trans. on Instrumentation and Volterra Measurement, vol. 45, no. 2, pp. 353–61, Apr. 1996. As input to the Volterra filter, a sine wave perturbed by [2] K. Parashar, D. Menard, R. Rocher, and O. Sentieys, “A Gaussian noise is used as the input to the system. The corre- hierarchical Methodology for Word-Length Optimiza- lation coefficients and the impulse response of the equivalent tion of Signal Processing Systems,” in 23rd International filter is calculated. The magnitude response of the filter com- Conference on VLSI Design, Bangalore, India, Jan. 2010. puted analytically is compared with the noise spectrum at the [3] H. Keding, M. Willems, M. Coors, and H. Meyr, output of Volterra filter which is obtained by fixed-point sim- “FRIDGE: A Fixed-Point Design And Simulation Envi- ulation. ronment,” in IEEE/ ACM conference on Design, Automa- When the perturbation noise input into the Volterra filter tion and Test in Europe 1998 (DATE 98), Paris, France, is white, the output noise spectrum traces the frequency re- march 1998, pp. 429 – 435. sponse. In the Figure 2, the output noise spectrum calculated [4] W. Sung and K. Kum, “Simulation-Based Word-Length analytically and the spectrum obtained by simulation match Optimization Method for Fixed-Point Digital Signal Pro- very closely. In the Volterra filter example considered, the cessing Systems,” IEEE Transactions on Signal Process- presence of cos(ω) term in the magnitude response as shown ing, vol. 43, no. 12, pp. 3087–3090, Dec. 1995. in Equation 20 is clearly seen as the spectrum tapers towards [5] D. Menard, R. Rocher, and O. Sentieys, “Analyti- the x-axis at higher frequencies. In the case of a colored cal Fixed-Point Accuracy Evaluation in Linear Time- noise, the noise used to perturb the sinusoid is a coloured Invariant Systems,” IEEE Transactions on Circuits and band-pass noise. The output noise spectrum obtained analyt- Systems I: Regular Papers,, vol. 55, no. 1, November ically matches the spectrum obtained by simulation. It has to 2008. be noted that the effect of the cos(ω) term is visible even in the coloured noise case. [6] D. Menard, R. Serizel, R. Rocher, and O. Sentieys, “Ac- The outcome of the experiments suggests that, the fre- curacy Constraint Determination in Fixed-Point System quency response of the filter derived analytically by adopting Design,” EURASIP Journal on Embedded Systems, vol. the proceduredescribed in this paper can faithfully represents 2008, p. 12, 2008. the frequency characteristics of the system. [7] G. Constantinides, P. Cheung, and W. Luk, “Roundoff- noise shaping in filter design,” in IEEE Symposium on 6. CONCLUSION Circuits and Systems, ISCAS. Geneva, Switzerland: In- stitute of Electrical and Electronics Engineers, May 28- A technique to estimate the noise power spectrum of arith- 31 2000, pp. IV57–IV60. metic operator based systems and thereby accelerating the performance evaluation of fixed point systems is presented. [8] C. Fancourt and J.C.Principe, “Competitive principal Computation of the power spectrum in this technique re- component analysis for locally stationary time series,” quires the auto-correlation functions of the filter coefficients Signal Processing, IEEE Transactions on, vol. 46, evaluated at 0 and is hence not computation intensive. The no. 11, pp. 3068 –3081, Nov 1998.

556