Uniform Quantization of Signals with Clipped and Truncated Laplace Distributions

Jerald L. Bauck* January 9, 2021

Abstract

Many natural and man-made signals including much of speech and music are well-modeled by the Laplace distribution. Methods of synthesizing such signals are available and sometimes preferred over traditional test signals. However, some- times those Laplace-like signals are clipped or do not exhibit infinitely-long tails. These situations are analyzed to determine their with an application of estimating signal-to-noise ratio as they are quantized by an analog-to-digital converter.

1 Background there are two practical signal distributions that are derived from the Laplace distribution: a long-tailed distribution ANY natural and man-made signals including much that has been clipped by the ADC or associated electron- M of speech and music are well-modeled in amplitude ics and a truncated, finite-tailed distribution that has not by the Laplace distribution [1], [2], [3], [4]. (Reference been clipped. In [6], clipped audio signals are examined to [5] includes a brief survey of Laplace and similar distribu- find their SNR with restoration attempted by estimating a tion models in various applications, many outside of au- parameter of a . dio.) Such signals, before processing, are frequently con- A naive use of the Laplace for the signal verted to digital form by a uniform analog-to-digital con- power in an estimate of the signal-to-noise ratio (signal-to- verter (ADC). Laplace signal models can also be used in distortion ratio) of the quantizer can yield a number that testing and simulations thus requiring the synthesis of suit- is too high in cases where a clipped or truncated model able signals [5]; in some usage cases the Laplace signal can is more appropriate. The error in SNR estimate depends be a substitute for or used in conjunction with the more upon the range and resolution of the quantizer and the common sine and uniformly distributed noise signals, pro- width of the Laplace distribution that is assumed as a sig- viding for example more realistic real-world estimates of nal model. As it happens, the naive application of the usual signal-to-noise ratio (SNR) of various equipment including Laplace variance might be suitable in some situations but ADCs and communication channels than provided by the the informed engineer knows when that approximation is more traditional signals. Other applications include heat- appropriate. ing and excursion studies of loudspeakers and other elec- The general equation for SNR in quantizer applications tromechanical equipment. is 2 Laplace signal models are characterized by a high sharp σ SNR X (1) peak near the origin and long tails that are fatter than gaus- = 2 σE sian tails. However, those tails are infinite which is not al- 2 2 ways a good modeling feature and which do not fit into the where variance σX is the power of the signal and σE is the finite range of a normal ADC. In addition, actual signals do power of the error signal due to quantization. The ratio is commonly expressed in decibels, SNR 10log (SNR). not have infinitely long tails and in that respect the Laplace dB = 10 model fails, if modestly. Actual signals can have a Laplace- X shall be replaced herein by L for Laplace, C for clipped, like distribution except for truncated tails, and synthesized and T for truncated, both for variances and probability Laplace-like test signals can also have finite tails by design density functions (PDF). or default, thus making them perhaps a better model than To be specific, when considering the tails of Laplace- the unmodified Laplace shape. With these considerations like signals relative to the quantizer range there are sev- eral possibilities. One possibility is that the tails are small *Originally published on July 2, 2020. Available at engrxiv.org. This pa- 2 enough that ignoring the problem and using σL causes lit- per has not been peer reviewed. Jerry Bauck welcomes comments and can tle error in the resulting SNR estimate. Another possibility be reached by e-mail at this address after left-shifting each alpha charac- ter by one position, e.g., f becomes e: [email protected]. © 2020 Jerald L. is that the signal is clipped either before or by the quan- Bauck tizer so that the signal power in the tails is captured in two

1 2 CLIPPED LAPLACE DISTRIBUTION 2

5 1.0

4 0.8

) 3

x 0.6 ( t L f P 2 0.4

1 0.2

0 G< x 0.0 -1.0 -0.5 0.0 0.5 M 1.0 x 0 2 4 6 8 Figure 1: Laplace probability density function for β 0.1 with hy- = pothetical lower and upper clipping or truncation levels indicated Figure 2: Probability of Laplace overload (clipping) as a function by xm and x . of β when and xm xM 1. − M = =

particular quantizer regions, usually the lowest and high- to xm and similarly the probability of overload on the est such regions but sometimes other regions depending −∞ − xM /β high side is PM e− /2 with a total overload probability on how the signal was clipped. A third possibility is that = 1/β P P P . With x 1 and x 1, P e− which is t = m + M m = M = t = the quantizer is designed in such a way that its lowest and plotted as a function of β in Figure 2. With a selector func- highest regions are open, capturing at least in effect any tion defined as signal values that extend respectively to and ; as −∞ +∞ considered here, this is effectively the same as clipping the ( 1 xm x xM signal before encountering a quantizer with closed outer c (x) − < < = 0 otherwise regions. Another possibility is that the signal does not have long tails but instead has distinctly finite tails that are well- the clipped PDF is modeled by the truncated distribution. And finally, there is the possibility that the signal has finite tails and is never- 1 x /β theless clipped; this situation can be analyzed by the tools fC (x) Pmδ(x xm) c (x) e−| | PM δ(x xM ) = + + 2β + − provided but the details are omitted. where the weighted Dirac impulses δ( ) represent the prob- · 2 Clipped Laplace Distribution ability masses due to clipping. Consider the The Laplace probability density function (PDF) is Z 1 2 x /β 1 x µ /β I (x) x e−| | dx (3) fL (x) e− − (2) = 2β = 2β | |

where β 0 and for present purposes the mean µ 0 which the integrand of which is plotted in Figure 3 for β 0.1. The > = = is usually not restrictive for real-world signals. The vari- indefinite integral is 2 2 ance, discussed below, is σL 2β . A plot for the central =  ³ 2 ´ part of this PDF with β 0.1 is shown in Figure 1. x/β 2 x e β βx 2 x 0 = I (x) −x + < (4) Consider asymmetric clipping of signal x (t) with the ³ x2 ´ = 2β2 e− β β2 βx x 0 lower clipping level at x and the upper clipping level at − + + 2 ≥ − m xM with xm 0 and xM 0. The clipped signal is defined > > which is plotted in Figure 4 again for β 0.1. Definite inte- as =  grals will be represented as I (x) b . xM x (t) xM |a  ≥ The variance of the clipped PDF is xc (t) x (t) x x (t) x . = − m < < M  xm x (t) xm Z − ≤ − 2 ∞ 2 σ x fC (x)dx The probability of overload, that is, clipping, on the neg- C = −∞ xm /β 2 x 2 ative side is Pm e− /2 found by integrating (2) from M Pm xm I (x) xm PM xM . = = + |− + 3 TRUNCATED LAPLACE DISTRIBUTION 3

0.030 0.8

0.025 0.6 0.020 2 C / | x | 0.015 e

2 0.4 x 0.010

0.005 0.2

0.000 -1.0 -0.5 0.0 0.5 1.0 x 0.0

Figure 3: Integrand for computing Laplace variance σ2 with β 0.0 0.5 1.0 1.5 2.0 2.5 3.0 L = 0.1.

2 Figure 5: Clipped Laplace variance σ versus β with xm x 1. 0.020 C = M =

variance is plotted against β in Figure 5 when x x 1 m = M = 0.015 and against x x when β 0.1 in Figure 6 M = m = I 3 Truncated Laplace Distribution 0.010 The truncated Laplace PDF is derived from the Laplace PDF generally by asymmetrically truncating the tails to the left and right, then adjusting the area to restore unity area 0.005 which is required of a PDF. The area lost by truncating the tails is P P so the area of the truncated function is m + M 1 Pm PM implying that the truncated PDF is 0.000 − −  0 x x -1.0 -0.5 0.0 0.5 1.0  m x  x /β < − e−| | x x x fT (x) ¡ xm /β x /β¢ m M = β 2 e− e− M − ≤ ≤ Figure 4: Indefinite integral I (x) of (3) related to the Laplace vari-  − − 0 x xM ance σ2 with β 0.1. The asymptotic value to the right is 2β2. > L = The variance of the truncated PDF is Z The integral is σ2 ∞ x2 f (x)dx T = T −∞ xm µ 2 ¶ Z xM x 2 2 xm 1 M − β 2 x /β I (x) xm 2β e β βxm x e−| | dx ¡ xm /β xM /β¢ |− = − + + 2 = β 2 e e xm − − − − − x à 2 ! M x 1 x β 2 M M e− β βxM . I (x) xm . − + + 2 = 1 P P |− − m − M Defining Identifying Pm and PM within the expression, the clipped xm /β ¡ 2 2 ¢ B e− 2β 2βx x variance becomes m = + m + m xM /β ¡ 2 2 ¢ BM e− 2β 2βxM x 2 2 xm /β ¡ 2 ¢ xM /β ¡ 2 ¢ = + + M σC 2β e− β βxm e− β βxM . (5) = − + − + the variance becomes The variance of the clipped signal is reduced from that of 2 4β Bm BM an unclipped signal with the same β because the impulsive σ2 − − (6) T xm /β xM /β = 2 e− e− masses in fC (x) are concentrated nearer to the origin than − − 2 the extended regions of the tails from which they derive. As which is also reduced from σL. As with the clipped case, x /β and x /β , σ2 2β2 σ2 . The clipped as x /β and x /β , σ2 σ2 . The truncated m → −∞ M →∞ C → = L m → ∞ M → ∞ T → L 4 SIGNAL-TO-NOISE RATIO 4

0.020 0.30

0.25 0.015 0.20 2 C 2 T 0.010 0.15

0.10

0.005 0.05

0.00 0.000 0.0 0.5 1.0 1.5 2.0 2.5 3.0 0.0 0.2 0.4 0.6 0.8 1.0 xM 2 Figure 7: Truncated Laplace variance σ versus β with xm x 2 T M Figure 6: Clipped Laplace variance σ versus x xm with β = = C M = = 1. 0.1.

bit word is 2 variance is plotted against β in Figure 7 when xm xM 1 2 1 xmax = = σE . and against xM xm when β 0.1 in Figure 8. = 3 22w = = If the entire Laplace signal range is assumed to fit in the 0.020 ADC without significant error then

2w 2 3 2 SNRL 2β · = x2 0.015 max and 2 T SNR 10log 6 20log β 20w log 2 20log x LdB = 10 + 10 + 10 − 10 max 0.010 7.7815 20log β 6.02w 20log x . ≈ + 10 + − 10 max Many times it is convenient to assume x 1 which max = drops the rightmost term. The pieces required to complete 0.005 the SNR formulas for the situations studied are available above and will be indicated here for clarity. For the clipped Laplace case, (5) applies so that

0.000 2 σC SNRC (7) 0.0 0.2 0.4 0.6 0.8 1.0 = σ2 xM E

2 and for the truncated Laplace case (6) applies and Figure 8: Truncated Laplace variance σ versus x xm with T M = β 0.1. = σ2 SNR T . (8) T = 2 σE

4 Signal-to-Noise Ratio The forms of these variances do not lend themselves to convenient logarithmic expressions. The SNR (1) requires the noise (distortion) power of the An application of (8) appears in [5] wherein methods of error of a uniform quantizer; the well-known result with synthesizing classes of continuous deterministic bandlim- identical-magnitude lower and upper quantization ranges ited test signals with nearly Laplace distributions are de- x x x and quantized output represented by a w- scribed. m = M = max REFERENCES 5

5 Version History [3] S. Kotz, T. Kozubowski, and K. Podgorski, The Laplace Distribution and Generalizations: A Revisit With Ap- • July 2, 2020. First published. plications to Communications, Economics, Engineer- ing, and Finance. Springer Science & Business Media, • July 3, 2020. Corrected sign for x in expression for m 2001. xc (t) and in text immediately following, and in ex- pression for fT (x). Corrected signs on xm and xM in (5). [4] W. B. Davenport, “An experimental study of speech- wave probability distributions,” The Journal of the • July 4, 2020. Re-rendered with properly sized Figure 5. Acoustical Society of America, vol. 24, no. 4, pp. 390– 399, July 1952. [Online]. Available: https://doi.org/10. • July 7, 2020. Changed [5] to “Accepted.” 1121%2F1.1906909 • January 5, 2021. Updated [5] as a published paper. Corrected signs on four instances of x . Changed m [5] J. L. Bauck, “Generating continuous deterministic “truncated” to “clipped” near the end of Section 3. band-limited test signals with nearly laplace distribu- Added references. tion,” Journal of the Audio Engineering Society, vol. 68, no. 9, pp. 664–679, September 2020. [Online]. Available: References http://www.aes.org/e-lib/browse.cfm?elib=20897

[1] S. Gazor and Wei Zhang, “Speech probability distribu- [6] X. Liu, J. Jia, and L. Cai, “SNR estimation for clipped tion,” IEEE Signal Processing Letters, vol. 10, no. 7, pp. audio based on amplitude distribution,” in 2013 204–207, July 2003. Ninth International Conference on Natural Computa- [2] Free sound clips, sound bites, and sound effects. tion (ICNC). Shenyang, China: Institute of Electrical [Online]. Available: www.soundbible.com and Electronics Engineers, July 2013, pp. 1434–1438.