Performance of the maximum likelihood estimators for the parameters of multivariate generalized Gaussian distributions Lionel Bombrun, Frédéric Pascal, Jean-Yves Tourneret, Yannick Berthoumieu To cite this version: Lionel Bombrun, Frédéric Pascal, Jean-Yves Tourneret, Yannick Berthoumieu. Performance of the maximum likelihood estimators for the parameters of multivariate generalized Gaussian distributions. IEEE International Conference on Acoustics, Speech, and Signal Processing, Mar 2012, Kyoto, Japan. pp.3525 - 3528, 10.1109/ICASSP.2012.6288677. hal-00744600 HAL Id: hal-00744600 https://hal.archives-ouvertes.fr/hal-00744600 Submitted on 23 Oct 2012 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. PERFORMANCE OF THE MAXIMUM LIKELIHOOD ESTIMATORS FOR THE PARAMETERS OF MULTIVARIATE GENERALIZED GAUSSIAN DISTRIBUTIONS Lionel Bombrun1, Fred´ eric´ Pascal2 ,Jean-Yves Tourneret3 and Yannick Berthoumieu1 1 : Universite´ de Bordeaux, ENSEIRB-Matmeca, Laboratoire IMS, Groupe Signal et Image flionel.bombrun, yannick.berthoumieu [email protected] 2 : SONDRA, Supelec,´ Plateau du Moulon, [email protected] 3 : Universite´ de Toulouse, IRIT/INP-ENSEEIHT, [email protected] ABSTRACT (which corresponds to most of the real-life problems), the maximum likelihood estimator (MLE) of the MGGD scatter matrix exists and This paper studies the performance of the maximum likelihood esti- is unique up to a scalar factor. An iterative algorithm based on a mators (MLE) for the parameters of multivariate generalized Gaus- Newton-Raphson recursion is then proposed to compute the MLE of sian distributions. When the shape parameter belongs to ]0; 1[, we the normalized MGGD scatter matrix. Some experiments are then have proved that the scatter matrix MLE exists and is unique up to conducted to evaluate the convergence speed of the algorithm as well a scalar factor. After providing some elements about this proof, an as the bias and the consistency of the scatter matrix estimator. Some estimation algorithm based on a Newton-Raphson recursion is inves- results regarding the estimation of the MGGD shape parameter are tigated. Some experiments illustrate the convergence speed of this finally presented. The paper is structured as follows. Section 2 intro- algorithm. The bias and consistency of the scatter matrix estimator duces the MGGD and the MLEs of its parameters. Section 3 derives are then studied for different values of the shape parameter. The per- properties of the scatter matrix MLE. Some simulations results are formance of the shape parameter estimator is finally addressed by presented in Section 4 to evaluate the performance of the MLEs of comparing its variance to the Cramer-Rao´ bound. the MGGD parameters. Conclusions and future works are finally Index Terms— Multivariate generalized Gaussian distribution, reported in Section 5. Newton-Raphson recursion, M-estimators. 2. THE MULTIVARIATE GENERALIZED GAUSSIAN 1. INTRODUCTION DISTRIBUTION (MGGD) In various signal and image processing applications, the univariate 2.1. Definition and stochastic representation generalized Gaussian (UGG) distribution has been introduced due to its more peaky and heavy-tailed shape compared to the univari- The probability density function of an MGGD is [7] ate Gaussian distribution [1]. In order to capture inter-band depen- 1 T −1 dencies in a wide-sense (multiscale, multichannel, spatial dependen- px(xjM; m; β) = 1 h x M x (1) cies), different multivariate distributions which generalize the UGG jMj 2 distribution have been proposed in the literature. A copula-based model with UGG distributed marginals has notably been proposed where h (·) is a so-called density generator defined by in [2] [3] to model the multichannel dependencies of wavelet co- Γ p β efficients. An extension of the UGG distribution to the multivari- 2 β v h (v) = p exp − (2) p p p β ate case, referred to as anisotropic multivariate generalized Gaussian π 2 m 2 2 2β 2m Γ 2β distribution, has also been proposed in [4]. In order to satisfy ellip- tically contoured distribution properties [5] [6], a natural extension for any v 2 + and M = Σ=m is a normalized matrix such that of the UGG distribution is to consider the multivariate generalized R tr (M) = p (where tr (M) is the trace of the matrix M and p is Gaussian distribution (MGGD) introduced in [7], also known as the the dimension of the vector x). Note that β = 0:5 corresponds to multivariate power exponential distribution [8]. The MGGD is com- the multivariate Laplace distribution, while β = 1 corresponds to pletely characterized by its scatter matrix Σ and its shape parameter the multivariate Gaussian distribution. When β tends toward infin- β. This model has recently shown good properties for several image ity, the MGGD is also known to reduce to the multivariate uniform processing applications such as multispectral image indexing [9], distribution. image denoising [10] and texture image retrieval [11] [12]. In these Let x be a random vector distributed according to an MGGD of scat- applications, the unknown parameters Σ and β have to be estimated ter matrix Σ = mM and shape parameter β.Gomez´ et al. have from the observed images. These parameters can be estimated by shown in [8] that x admits the following stochastic representation minimizing a χ2 distance as in [9], or by minimizing an L2-norm as in [10]. Estimators based on the method of moments and on the d 1 maximum likelihood method have also been proposed in [11] [12]. x = τ Σ 2 u (3) However, the performance of these estimators has not been inves- tigated, which is the main objective of this work. More precisely, where =d means equality in distribution, u is a random vector uni- p the main contribution of the paper is to show that for β 2]0; 1[ formly distributed on the unit sphere R , and τ is a scalar positive random variable such that 1: Initialisation of β and M. 2: for k = 1 : N iter max do 2β p τ ∼ G ; 2 (4) 3: Estimation of M using one iteration of (7) and normalization. 2β 4: Estimation of β by a Newton-Raphson iteration combin- where G(a; b) is the univariate Gamma distribution with parameters ing (8) and (10). a and b (see [13] for definition). 5: end for 6: Estimation of m using (9). 2.2. MGGD parameter estimation Note that we have observed that the algorithm convergence is sig- nificantly faster when the normalization constraint tr(M) = p is Let (x1;:::; xN ) be N vectors independent and identically dis- imposed at each iteration. tributed according to an MGGD. Since an MGGD is a particular real elliptical distribution, the MLE of the matrix M satisfies the 3. PROPERTIES OF THE M-ESTIMATOR following fixed point equation [14] N T −1 MGGDs belong to the general class of elliptical distributions (ED) 2 X −g(x M xi) T M = i x x denoted E . Maronna derived in [16] very useful results for estimat- T −1 i i (5) p N h(x M xi) i=1 i ing the parameters of EDs. Let (x1;:::; xN ) be a N-sample (N in- dependent and identically distributed vectors) of p-dimensional real @h(v) where g(v) = . By replacing h (·) by its definition (2), the vectors, with zero mean and distributed according to an ED, i.e., @v following result can be obtained xi ∼ Ep(0; Λ) for i = 1;:::;N. The M-estimator of Λ is defined as the solution of the following equation N T β X xix M = i with tr(M) = p: (6) N β T −1 1−β 1 X T −1 T Nm (x M xi) V = u x V x x x (11) i=1 i N N n N n n n n=1 Note that the first term outside the summation is a scalar that can be encompassed in the constraint. Thus (6) reduces to the following where u(·) is a function satisfying a set of general assumptions pro- M-estimator fixed point equation vided in [16] and recalled below N T (i) u is non-negative, non increasing, and continuous in [0; 1). 1 X xix M = i with tr(M) = p: (7) T −1 1−β (ii) If K = sup (s) with p < K < 1 and (s) = s u(s), is N (x M xi) i=1 i s≥0 Note that (7) reduces to the sample covariance matrix (SCM) estima- non decreasing and strictly increasing in the interval where tor when β = 1 and to the fixed point covariance matrix estimator < K. derived in [14] [15] for β = 0. (iii) Let PN (·) the empirical distribution of x1;:::; xN , there Moreover, by differentiating the joint distribution of (x1;:::; xN ) exists a > 0 such that for every hyperplane H such that with respect to m and β, the following results can be obtained dim(H) ≤ p − 1 we have N p pN X β pN p PN (H) ≤ 1 − − a: (12) f(β) = ln uiu − Ψ + ln 2 N i 2β 2β K P β i=1 2 ui i=1 Theorem 1. Under the assumptions (i), (ii) and (iii), a solution of N ! (11) exists and is unique. Moreover, this solution is consistent up pN β X β − N − ln u = 0 (8) to a scale factor and a simple iterative procedure can be used to 2β pN i i=1 determine VN .