Necessary Conditions for a Maximum Likelihood Estimate to Become Asymptotically Unbiased and Attain the Cramer–Rao Lower Bound
Total Page:16
File Type:pdf, Size:1020Kb
Necessary conditions for a maximum likelihood estimate to become asymptotically unbiased and attain the Cramer–Rao Lower Bound. Part I. General approach with an application to time-delay and Doppler shift estimation Eran Naftali and Nicholas C. Makris Massachusetts Institute of Technology, Room 5-222, 77 Massachusetts Avenue, Cambridge, Massachusetts 02139 ͑Received 17 October 2000; revised 20 April 2001; accepted 29 May 2001͒ Analytic expressions for the first order bias and second order covariance of a general maximum likelihood estimate ͑MLE͒ are presented. These expressions are used to determine general analytic conditions on sample size, or signal-to-noise ratio ͑SNR͒, that are necessary for a MLE to become asymptotically unbiased and attain minimum variance as expressed by the Cramer–Rao lower bound ͑CRLB͒. The expressions are then evaluated for multivariate Gaussian data. The results can be used to determine asymptotic biases, variances, and conditions for estimator optimality in a wide range of inverse problems encountered in ocean acoustics and many other disciplines. The results are then applied to rigorously determine conditions on SNR necessary for the MLE to become unbiased and attain minimum variance in the classical active sonar and radar time-delay and Doppler-shift estimation problems. The time-delay MLE is the time lag at the peak value of a matched filter output. It is shown that the matched filter estimate attains the CRLB for the signal’s position when the SNR is much larger than the kurtosis of the expected signal’s energy spectrum. The Doppler-shift MLE exhibits dual behavior for narrow band analytic signals. In a companion paper, the general theory presented here is applied to the problem of estimating the range and depth of an acoustic source submerged in an ocean waveguide. © 2001 Acoustical Society of America. ͓DOI: 10.1121/1.1387091͔ PACS numbers: 43.30.Pc, 43.60.Gk, 43.60.Pt ͓DLB͔ I. INTRODUCTION The purpose of the present paper is not to derive a new parameter resolution bound, but rather to determine, within In many practical problems in ocean acoustics, geophys- the framework of classical estimation theory1,6,7 the condi- ics, statistical signal processing, and other disciplines, non- tions on sample size, or SNR, necessary for the MLE to linear inversions are required to estimate parameters from measured data that undergo random fluctuations. The nonlin- become asymptotically unbiased and attain minimum vari- ear inversion of random data often leads to estimates that are ance. The approach is to apply the tools of higher order biased and do not attain minimum variance, namely the asymptotic inference, which rely heavily on tensor analysis, Cramer–Rao lower bound ͑CRLB͒, for small sample sizes or to expand the MLE as a series in inverse orders of sample equivalently low signal-to-noise ratio ͑SNR͒. The maximum size or equivalently inverse orders of SNR.7 From this series likelihood estimator ͑MLE͒ is widely used because if an as- analytic expressions for the first order bias, second order ymptotically unbiased and minimum variance estimator ex- covariance and second order error correlation of a general ists for large sample sizes, it is guaranteed to be the MLE.1 MLE are presented in terms of joint moments of the log- Since exact expressions for the bias, variance, and error cor- likelihood function and its derivatives with respect to the relation of the MLE are often difficult or impractical to de- parameters to be estimated. Since the first order error corre- rive analytically, it has become popular in ocean acoustics lation is shown to be the CRLB, which is only valid for and many other areas to simply neglect potential biases and unbiased estimates, the second order error correlation can to compute limiting bounds on the mean square error, such as provide a tighter error approximation to the MLE than the the CRLB, since these bounds are usually much easier to CRLB that is applicable in relatively low SNR even when obtain. The CRLB, however, typically provides an unrealis- the MLE is biased to first order. These expressions are then tically optimistic approximation to the MLE error correlation in many nonlinear inverse problems when the sample size is used to determine general analytic requirements on sample small, or equivalently the SNR is low. A number of bounds size, or SNR, that are necessary for an MLE to become as- on the error correlation exist that are tighter than the ymptotically unbiased and attain minimum variance. This is CRLB.1–5 Some of these bounds are based on Bayesian done by showing when the first order bias becomes negli- assumptions4,5 and so require the a priori probability density gible compared to the true value of the parameter and when of the parameters to be estimated, which can be problematic the second order covariance term becomes negligible com- when the a priori probability density is not known.6 pared to the CRLB. J. Acoust. Soc. Am. 110 (4), October 2001 0001-4966/2001/110(4)/1917/14/$18.00 © 2001 Acoustical Society of America 1917 The first order bias is evaluated for general multivariate dard linear frequency modulated ͑LFM͒, hyperbolic fre- Gaussian data. The second order covariance and error corre- quency modulated ͑HFM͒, and canonical waveforms are pro- lation terms are evaluated for two special cases of Gaussian vided for typical low-frequency active-sonar scenarios in data that are of great practical value in ocean acoustics, geo- ocean acoustics. physics, and statistical signal processing. The first is for a deterministic signal vector embedded in additive noise and II. GENERAL ASYMPTOTIC EXPANSIONS OF THE the second is for a fully randomized signal vector with zero MLE AND ITS MOMENTS mean in additive noise. These cases have been widely used Suppose the random data vector X, given m-dimensional in ocean acoustic inversions, spectral estimation, beamform- parameter vector , obeys the conditional probability density ing, sonar and radar detection, and localization problems, as function ͑pdf͒ p(X;). The log-likelihood function l()is well as statistical optics.8–10 In a companion paper, each of defined as l()ϭln(p(X;)) when evaluated at measured these cases is applied to determine the asymptotic bias and values of X. Let the rth component of be denoted by r. covariance of maximum likelihood range and depth esti- The first log-likelihood derivative with respect to r is then mates of a sound source submerged in an ocean waveguide r.IfR ϭr ...r ,.....,Rץ/(l(ץdefined as l ϭ from measured hydrophone array data as well as necessary r 1 11 1n1 m 11 ϭr are sets of coordinate indices, joint moments of conditions for the estimates to attain the CRLB. m1...mnm the log-likelihood derivatives can be defined by v In the present paper, these expressions are applied to the R1 ,...,Rm active sonar and radar time-delay and Doppler-shift estima- ϭEbl ...l c, where, for example, v ϭE͓l l ͔ and Rl Rm s,tu s tu ϭ tion problems, where time delay is used for target range es- va,b,c,de E͓lalblclde͔. timation and Doppler shift is used for target velocity estima- The expected information, known as the Fisher informa- ϭ tion. Attention is focused on the commonly encountered tion, is defined by irs E͓lrls͔ where the indices r, s are scenario of a deterministic signal with unknown spatial or arbitrary. Lifting the indices produces quantities that are de- R ,...,R r s r s temporal delay received together with additive white noise. noted by v 1 mϭi 11 11...i mnm mnmv , s11 ...s1n ,...,sm1 ...smn The time-delay MLE is then the time lag at the peak value of Ϫ 1 Ϫ m where irsϭ͓i 1͔ is the r, s component of the inverse i 1 of a matched filter output. The matched filter estimate for a rs the expected information matrix i. The inverse of the Fisher signal’s time delay or position is widely used in many appli- Ϫ information matrix i 1 is also known as the Cramer–Rao cations of statistical pattern recognition in sonar, radar, and lower bound ͑CRLB͒. Here, as elsewhere, the Einstein sum- optical image processing. This is because it has long been mation convention is used. That is, if an index occurs twice known that the matched filter estimate attains the CRLB in in a term, once in the subscript and once in the superscript, high SNR. Necessary analytic conditions on how high the summation over the index is implied. SNR must be for the matched filter estimate to attain the The MLE ˆ, the value of that maximizes l() for the CRLB have not been previously obtained but are derived 1,6,7 here using the general asymptotic approach developed in given data X, can now be expressed as an asymptotic Secs. II–IV. expansion around in increasing orders of inverse sample size nϪ1 or equivalently SNR. Following the derivation of A number of authors have derived tighter bounds than 7 the CRLB for the time-delay estimation problem to help Barndorff-Nielsen and Cox, the component lr is first ex- evaluate performance at low SNR where the CRLB is not panded around as attained by the MLE, as, for example, in Refs. 5, 12, 13. The present paper follows a different approach by providing ex- ˆ ϭ ϩ ͑ˆ Ϫ͒sϩ 1 ͑ˆ Ϫ͒s͑ˆ Ϫ͒t l r lr lrs 2lrst plicit expressions for the second order variance of the time- delay and Doppler-shift MLEs that are attained in lower ϩ 1 ͑ˆ Ϫ͒s͑ˆ Ϫ͒t͑ˆ Ϫ͒uϩ ͑ ͒ 6lrstu , 1 SNR than the CRLB.