Signal Processing Manuscript Draft Manuscript Number: SIGPRO-D-14-00375 Title: Optimum Linear Regression in Additive Cauchy-Gaussian Noise Article Type: Fast Communication Keywords: Impulsive noise, Cauchy distribution, Gaussian distribution, mixture noise, Voigt profile, maximum likelihood estimator, pseudo-Voigt function, M-estimator Abstract: In this paper, we study the estimation problem of linear regression in the presence of a new impulsive noise model, which is a sum of Cauchy and Gaussian random variables in time domain. The probability density function (PDF) of this mixture noise, referred to as the Voigt profile, is derived from the convolution of the Cauchy and Gaussian PDFs. To determine the linear regression parameters, the maximum likelihood estimator is first developed. Since the Voigt profile suffers from a complicated analytical form, an M-estimator with the pseudo-Voigt function is also derived. In our algorithm development, both scenarios of known and unknown density parameters are considered. In the unknown scenario, density parameters need to be estimated prior to proposals, by utilizing the empirical characteristic function and characteristic function. Simulation results show that the performance of both proposed methods can attain the Cram\'{e}r-Rao lower bound. Highlights (for review) Highlights: An additive mixture noise is studied in this paper and the corresponding noise PDF, i.e., the Voigt function is derived. To determine the parameters of a linear regression model, the maximum likelihood estimator (MLE) is developed, where both the scenarios of known and unknown density parameters, are considered. To reduce the computational complexity of the MLE, an M-estimator with pseudo-Voigt function is presented. Both presented estimators approach the CRLB well. *Manuscript Click here to view linked References 1 2 3 4 5 6 7 8 9 10 Optimum Linear Regression in Additive Cauchy-Gaussian Noise 11 12 ∗ ,1 2 1 13 Yuan Chen , Ercan Engin Kuruoglu , Hing Cheung So 14 1 15 Department of Electronic Engineering, City University of Hong Kong, Hong Kong SAR, China 16 2 17 ISTI-CNR (Italian National Council of Research), Pisa, Italy 18 19 20 Abstract: In this paper, we study the estimation problem of linear regression in the presence of a new impulsive 21 22 noise model, which is a sum of Cauchy and Gaussian random variables in time domain. The probability density 23 function (PDF) of this mixture noise, referred to as the Voigt profile, is derived from the convolution of the 24 25 Cauchy and Gaussian PDFs. To determine the linear regression parameters, the maximum likelihood estimator 26 is first developed. Since the Voigt profile suffers from a complicated analytical form, an M-estimator with the 27 28 pseudo-Voigt function is also derived. In our algorithm development, both scenarios of known and unknown 29 density parameters are considered. In the unknown scenario, density parameters need to be estimated prior to 30 31 proposals, by utilizing the empirical characteristic function and characteristic function. Simulation results show 32 33 that the performance of both proposed methods can attain the Cram´er-Rao lower bound. 34 Indexing terms: Impulsive noise, Cauchy distribution, Gaussian distribution, mixture noise, Voigt profile, max- 35 36 imum likelihood estimator, pseudo-Voigt function, M-estimator 37 38 39 1 Introduction 40 41 42 Impulsive noise is encountered in a variety of applications such as wireless communications, radar, sonar and 43 image processing [1]. Unlike Gaussian noise, impulsive noise belongs to a family of heavy-tailed noise distri- 44 45 butions. Popular models in the literature for impulsive noise are divided into two categories, namely, single 46 47 process and hybrid process mixed in the probability density function (PDF) domain. Typical single distribu- 48 tions are Student’s t-distribution [2], α-stable distribution [3] and generalized Gaussian (GG) process [4], while 49 50 the mixture models include Gaussian mixture (GM) [5] and Cauchy Gaussian mixture (CGM) [6]. Neverthe- 51 less, these models alone may not be able to represent all varieties of impulsive noises in the real world such 52 53 as the case that the noise measured is the sum of two separate time series: one is an intrinsic Gaussian noise 54 55 due to the electronic devices in receiver and the other is environmental noise which can be non-Gaussian, in 56 particular impulsive. For example, considering some schemes in frequency-hopping spread spectrum (FH SS) 57 58 radio communication networks [7], binary transmission systems [8] and multiple-input multiple-output (MIMO) 59 systems [9], we model the multiple access interference as the α-stable distribution and regard the environmental 60 61 noise as the Gaussian distribution. Similarly, in astrophysical imaging [10], the cosmic microwave background 62 radiation is contaminated with the Gaussian noise from the satellite beam and α-stable distributed radiation 63 64 ∗Corresponding Author (Email: [email protected]; Fax: (852) 2788 7791) 65 1 1 2 3 4 5 from galaxies and stars. In these potential applications, the disturbance components can be combined into a 6 7 new mixture model which is a sum of two different random processes in the time domain. 8 To demonstrate the applicability of this model, we consider the linear regression problem and take the sum of 9 2 10 a symmetric Cauchy distribution with dispersion γ and zero-mean Gaussian distribution with variance σ as an 11 illustrative example. This mixture model belongs to the Middletons Class B [11] which is a classical impulsive 12 13 noise model that has been employed for decades. The PDF of the mixture has an analytical form, known as the 14 Voigt function [12], which is obtained via the convolution of the PDFs of these two processes. When the density 15 16 parameters, namely, γ and σ2 are known, the PDF of the mixture is readily determined, and the maximum 17 18 likelihood estimator (MLE) which is a special case of M-estimator, can be directly applied to find the parameters 19 of interest. The class of M-estimators introduced by Huber [13] generalizes the MLE by replacing the logarithm 20 21 of the likelihood function by an arbitrary ρ-function. Note that the MLE is in the class of M-estimators by 22 letting ρ = log (f(y)) with f(y) denoting the likelihood function. However, when γ and σ2 are unknown, 23 − 24 they should be estimated through the relationship between the empirical characteristic function (ECF) and the 25 26 characteristic function (CF) prior to employing the MLE. Although the MLE has the best performance in the 27 sense of attaining Cram´er-Rao lower bound (CRLB), it suffers from having a highly complex analytical form 28 29 because of the Faddeeva function that appears in the PDF of the mixture noise. Therefore, in order to keep 30 the high accuracy of the MLE and to reduce the computational complexity, a new M-estimator with the loss 31 32 function chosen as the logarithm of pseudo-Voigt function is employed, which is referred to as the MEPV. 33 The rest of this paper is organized as follows. The proposed methods, namely, the MLE and MEPV are 34 35 presented in Section 2. Both cases of known and unknown density parameters are investigated. Computer 36 37 simulations are provided in Section 3 to evaluate the accuracy and complexity of the MLE and MEPV. Finally, 38 conclusions are drawn in Section 4. 39 40 41 42 2 Proposed Algorithms 43 44 Without loss of generality, the observed data vector y = [y y ]T is modeled as: 45 1 ··· N 46 47 yn = sn(θ)+ en, n =1, 2, . , N, (1) 48 49 where sn(θ) denotes the noise-free signal with θ being the parameter vector of interest, en = pn + qn is the 50 mixture noise which is a sum of two independent and identically distributed (i.i.d.) processes p and q , whose 51 n n 52 PDFs are fP and fQ, respectively. 53 54 The PDF of en can be obtained from the convolution of fP and fQ: 55 56 fE = fP fQ, (2) 57 ∗ 58 where stands for the convolution operator. 59 ∗ T 60 Considering the simplest case of the linear regression model, i.e., sn(θ) = sn([A B] ) = An + B, where A 61 and B are the unknown parameters, the data model can be rewritten in vector form as: 62 63 64 y = Hθ + e n =1, 2, . , N, (3) 65 2 1 2 3 4 5 where 6 7 1 1 8 2 1 A 9 H = , θ = (4) . 10 . B 11 12 N 1 13 and e = [e e ]T with e = c + g denoting the additive Cauchy Gaussian (ACG) noise which is the 14 1 ··· N n n n 15 sum of i.i.d. Cauchy noise c with dispersion γ and the i.i.d. zero-mean Gaussian noise g with variance σ2. 16 n n 17 Although we only study this simple model, our analysis can be extended to the general linear data model [14], 18 N M M that is, H R × where N M is known and θ R is unknown. It is noteworthy that (3)-(4) are also 19 ∈ ≥ ∈ 20 a common signal model for kick detection in oil drilling [15]. The PDFs of Cauchy and Gaussian distributions 21 22 are: 23 γ 24 f (c ; γ)= , (5) C n π(c2 + γ2) 25 n 2 26 2 1 gn fG(gn; σ )= exp . (6) 27 √2πσ −2σ2 28 29 Then the PDF of en is calculated based on (2): 30 2 2 ∞ γ 1 τ − 2σ2 31 fE(en; γ, σ )= 2 2 e dτ. (7) π((en τ) + γ ) √2πσ 32 Z−∞ − 33 The result of (7) is called the Voigt function which can be represented as [12] 34 35 2 Re w fE(en; γ, σ )= { }, (8) 36 σ√2π 37 38 where 2 en+iγ 39 √ en + iγ 2i σ 2 2 40 w = exp 1+ exp t dt (9) − σ√2 √π 41 ! Z0 ! 42 and w is called the Faddeeva function with Re denoting the real part.