JANUARY 1999 DABAS 19 Semiempirical Model for the Reliability of a Matched Filter Frequency Estimator for Doppler Lidar ALAIN DABAS MeÂteÂo-France, Centre National de Recherches Meteorologiques, Toulouse, France (Manuscript received 10 February 1997, in ®nal form 22 December 1997) ABSTRACT The author proposes a heuristic semiempirical model for predicting the reliability of a matched-®lter frequency estimator applied to Doppler lidar signals. The model is tuned by a single coef®cient b empirically related to the ratio of the number of signal samples per estimate over the number of speckles. It can deal with any signal characteristics (spectrum width, number of samples, etc.) as well as any factor of accumulation. 1. Introduction a signi®cant number of bad measurements. Either way, the controlled dataset is useless. It follows that a high Due to the limits imposed by the available technology reliability is required for measured data. For a space- and the weak backscatter coef®cient of the atmopshere, borne application, for instance, the data speci®cation coherent atmospheric Doppler lidars often operate at a typically ranges from 0.5 to 0.95, or even better (Kavaya low signal-to-noise ratio (SNR). Under such conditions, 1995; Marini and Culoma 1995; Baker et al. 1995; ESA Doppler lidars may occasionally produce ``bad'' mea- 1996). This is a major constraint for the design of the surements, which are characterized by a uniform dis- Doppler lidar. To treat it properly, a precise knowledge tribution over the search band. Such bad measurements of how frequency estimators behave at low SNR regimes originate from the noise generated by random peaks in is required. To answer this need, several studies have the signal periodogram that are mistakenly interpreted been conducted. In Frehlich and Yadlowsky (1994), ge- by the frequency estimator as atmospheric echoes. Not neric formulas are provided for predicting the proba- only are they deprived of any useful information on the bility of bad measurements as a function of SNR. The dynamics of sounded atmospheric volumes but they also formulas contain three parameters that depend on the can be very dangerous for the meteorological analysis frequency estimator and vary as a function of the signal in charge of exploiting the measurements for scienti®c characteristics. The parameters are determined empiri- purposes (for instance, weather prediction). Even in cally and tabulated. Though the model is accurate and small numbers, the bad measurements can result in large useful, it has several limitations. First, it cannot work errors in retrieved meterological ®elds. Dedicated pro- when the signal characteristics are outside the range cedures of quality control (QC) are indeed implemented resolved by Frehlich and Yadlowsky since the tuning in analysis systems to avoid this. Based on a contextual parameters cannot be extrapolated from the table. Also, approach (measurements that are too different from all though accumulation is considered by Frehlich and Yad- others in the neighborhood are removed) or related to lowsky (1994), no model parameters are given for it. the frequency estimator itself (Rye and Hardesty 1997), Yet, accumulation is often considered in lidar projects the QC procedures can detect a signi®cant fraction of because it is one of the most ef®cient ways to improve the bad measurements. But their performances are nev- accuracy (Rye and Hardesty 1993a,b); moreover, high ertheless limited: on the one hand, the QC procedures repetition rates are now available with solid-state lasers, leave some of the bad measurements undetected and on thus permitting one to accumulate a large number of the other hand, remove some ``good'' measurements. If signals in short periods of time (Frehlich et al. 1997). the original dataset contains too much bad data, the QC The impact of accumulation on reliability is studied spe- procedure either removes almost all of the data or leaves ci®cally in Frehlich (1996). Scaling (power) laws are provided, giving the dependence on the accumulation factor of the SNR threshold that is required to achieve a given reliability (0.9 and 0.5 in the paper). However, Corresponding author address: Dr. Alain Dabas, MeÂteÂo-France, CNRM/GMEI/LISA, 42, av. G. G. Coriolis, F-31057 Toulouse, Ced- the parameters necessary for the computation of the ex, France. reliability as a function of SNR are not given, preventing E-mail: [email protected] extrapolation of other reliability thresholds. q 1999 American Meteorological Society Unauthenticated | Downloaded 09/25/21 05:54 PM UTC 20 JOURNAL OF ATMOSPHERIC AND OCEANIC TECHNOLOGY VOLUME 16 This paper proposes a heuristic model predicting the Under the previous assumptions, the signal power p probability of Levin's (1965) matched ®lter to make bad averaged over a range gate follows approximately a chi- measurements. The input parameters are the SNR, the square distribution (Goodman 1985): main signal characteristics (spectrum width and length), mpmm21 p and the accumulation factor. The model is derived from prob(p) 5 exp 2m , (1) a heuristic analysis of the mechanism responsible for G(m)(S 1 N)m 12S 1 N the bad measurements. It is tuned by a single coef®cient where S and N are the mean powers of the atmospheric related to the input parameters. It thus can adapt to any return and noise, respectively, and m is given by set of signal characteristrics. In principle, it is restricted to Levin's frequency estimator, but since all ``best-per- 11 122 1S/N forming'' estimators are nearly equivalent with regard 51. (2) mM1 1 S/N m 1 1 S/N to reliability (see Frehlich 1996), it should be possible 1212 to use it for other estimators. Nevertheless, it gives use- Here, M is the number of signal samples contained in ful information on the reliability one must expect for a the processing gate. An interesting mathematical prop- given lidar system under speci®ed conditions. erty of the chi-square distribution (1) is The paper begins in section 2a with a presentation of (S 1 N)2 the assumptions underlying the model and a simpli®- m 5 , (3) s 2 cation of the signal periodogram. A formula giving the p probability of a bad estimate follows in section 2b. It which provides a useful experimental way for measuring contains an unknown parameter denoted b, whose value m consisting of the estimation of the ®rst- and second- is determined empirically in section 3c for various signal order moments p (Ancellet et al. 1989). characteristics. An empirical formula is derived after- With no noise (N 5 0), m reduces to m, so m is the ward to relate b to the only one-signal parameter iden- fractional variance of speckle-induced power ¯uctua- ti®ed as relevant: the ratio of the number of samples tions. It is therefore often called the number of speckles per estimate M to the number of speckles m (section and can be interpreted as the number of independent 3d). The model capacity to predict reliability outside realizations of the speckle effect inside the range gate. the range of processing conditions examined by Frehlich It can be related (Rye 1995) to the normalized auto- and Yadlowsky is investigated in sections 3e and 3f. correlation (AC) function of the atmospheric echo g(t) 5 E[s*(x)s(x 1 t)]/S by M 2. Semiempirical model m 5 . (4) M21 k 1 1 212 |g(k)|2 a. Simpli®ed model for the periodogram kO51 12M The basic assumptions made throughout the article An important consequence for lidar applications is that are 1) lidar signals form a Gaussian process, 2) they are m can be considered a system parameter because the AC function is primarily related to the transmitted laser stationary, and 3) they are contaminated by white noise. pulse (Churnside and Yura 1983), with wind and at- These are usual assumptions in the radar or lidar signal mospheric inhomogeneities generally playing a minor processing studies. The Gaussian nature of the process and negligible role. is well veri®ed in reality because the signal results from In the frequency domain, the frequency components the addition of a great number of backscattered ``wave- of the periodogram are exponentially distributed. Due lets.'' Stationarity, however, is never perfectly met be- to the ®nite length of the range gate, they are correlated. cause the signal power decreases with range and the According to Zrnic (1980), the correlation can be rather optical and dynamic properties of the atmosphere are high (50% or more)Ðespecially when working with never purely homogeneous. But, nevertheless, station- short range gatesÐbut it decreases with noise. Since arity can be approximated when the measurements are bad measurements arise at low SNRs, in the following, made at long ranges (then the range dependence of the we make the basic assumption that the spectral corre- signal power can be neglected) in homogeneous at- lation can be neglected. As we see and discuss in section mospheric volumes. And the white color of the noise 3c, this is the probable cause for the major limitation can be obtained in reality, provided the analog chain of of the model. We nevertheless use it since it offers the the lidar is designed carefully. useful possibility of deriving closed-form mathematical In the frequency domain, the basic assumptions men- expression and, most importantly, since it leads to an tioned above result in a signal periodogram that is com- ef®cient model. posed of a peak that rests on top of a bottom ¯oor of To facilitate mathematical derivations, we now sim- noise. The peak represents the atmospheric signatureÐ plify the shape of the periodogram. We transform the that is, the useful part of the signal.