chirp, 7/8/1996 y BAYESIAN SPECTRUM AND CHIRP ANALYSIS E. T. Jaynes Wayman Crow Professor of Physics Washington University, St. Louis MO 63130 Abstract : We seek optimal metho ds of estimating p ower sp ectrum and chirp frequency change rate for the case that one has incomplete noisy data on values y t of a time series. The Schuster p erio dogram turns out to b e a \sucient statistic" for the sp ectrum, a generalization playing the same role for chirp ed signals. However, the optimal pro cessing is not a linear ltering op eration like the Blackman{Tukey smo othing of the p erio dogram, but a nonlinear op eration. While suppressing noise/side lob e artifacts it achieves the same kind of improved resolution that the Burg metho d did for noiseless data. CONTENTS 1. INTRODUCTION 2 2. CHIRP ANALYSIS 4 3. SPECTRAL SNAPSHOTS 5 4. THE BASIC REASONING FORMAT 5 5. A SIMPLE BAYESIAN MODEL 6 6. THE PHASELESS LIKELIHOOD FUNCTION 9 7. DISCUSSION { MEANING OF THE CHIRPOGRAM 10 8. POWER SPECTRUM ESTIMATES 13 9. EXTENSION TO CHIRP 16 10. MANY SIGNALS 17 11. CONCLUSION 23 APPENDIX A. OCEANOGRAPHIC CHIRP 23 APPENDIX B. WHY GAUSSIAN NOISE? 25 APPENDIX C. CALCULATION DETAILS 27 REFERENCES 28 y A revised and up dated version of a pap er presented at the Third Workshop on Maximum{Entropy and Bayesian Metho ds, Laramie Wyoming, August 1{4, 1983. Published in Maximum{Entropy and Bayesian Spectral Analysis and Estimation Problems, C. R. Smith and G. J. Erickson, Editors, D. Reidel Publishing Co., 1987. This is the original investigation that evolved into the Bayesian Sp ectrum Analysis of Bretthorst 1988. The \plain vanilla" Bayesian analysis intro duced here proved to b e unexp ectedly p owerful in applications b ecause of its exibility, which allows it to accommo date itself to all kinds of complicating circumstances in a way determined uniquely by the rules of probability theory, with no need for any ad hoc devices or app eal to unreliable intuition. 2 1. INTRODUCTION 2 1. INTRODUCTION The Maximum Entropy solution found by Burg 1967, 1975 has b een shown to give the optimal sp ectrum estimate { by a rather basic, inescapable criterion of optimality { in one well{de ned problem Jaynes, 1982. In that problem we estimate the sp ectrum of a time series y y , 1 N from incomplete data consisting of a few auto covariances R R ; m<N, measured from the 0 m entire time series, and there is no noise. This is the rst example in sp ectrum analysis of an exact solution, which follows directly from rst principles without ad hoc intuitive assumptions and devices. In particular, we found Jaynes, 1982 that there was no need to assume that the time series was a realization of a \stationary Gaussian pro cess". The Maximum Entropy principle automatically created the Gaussian form for us, out of the data. This indicated something that could not have b een learned by assuming a distribution; namely that the Gaussian distribution is the one that can b e realized by Nature in more ways than can any other that agrees with the given auto covariance data. In this sense it is the `safest' probability assignment one could make from the given information. This classic solution will go down in history as the \hydrogen atom" of sp ectrum analysis theory. Butamuch more common problem, also considered by Burg, is the one where our data consist, not of auto covariances, but the actual values of y y , a subset of a presumably longer full 1 N time series, contaminated with noise. Exp erience has shown Burg's metho d to b e very successful here also, if we rst estimate m auto covariances from the data and then use them in the MAXENT calculation. The choice of m represents our judgment ab out the noise magnitude, values to o large intro ducing noise artifacts, values to o small losing resolution. For any m, the estimate we get would b e the optimal one if a the estimated auto covariances were known to b e the exact values; and b we had no other information b eyond those m auto covariances. Although the success of the metho d just describ ed indicates that it is probably not far from optimal when used with go o d judgment ab out m,wehaveasyet no analytical theory proving this or indicating any preferred di erent pro cedure. One would think that a true optimal solution should 1 use all the information the data can give; i.e. estimate not just m<N auto covariances from the data, but nd our \b est" estimate of all N of them and their probable errors; 2 then make allowance for the uncertainty of these estimates by progressively de{emphasizing the unreliable ones. There should not b e any sharp break as in the pro cedure used now, which amounts to giving full credence to all auto covariance estimates up to lag m, zero credence to all b eyond m. In Jaynes 1982 we surveyed these matters very generally and concluded that much more analytical work needs to b e done b efore we can knowhow close the present partly ad hoc metho ds are to optimal in problems with noisy data. The following is a sequel, rep orting the rst stage of an attempt to understand the theoretical situation b etter, by a direct Bayesian analysis of the noisy data problem. In e ect, we are trying to advance from the \hydrogen atom" to the \helium atom" of sp ectrum analysis theory. One might think that this had b een done already, in the many pap ers that study autoregressive AR mo dels for this problem. However, as wehave noted b efore Jaynes, 1982, intro ducing an AR mo del is not a step toward solving a sp ectrum analysis problem, only a detour through an alternativeway of formulating the problem. An AR connection can always b e made if one wishes to do so; for anypower sp ectrum determines a covariance function, which in turn determines a Wiener prediction lter, whose co ecients can always b e interpreted as the co ecients of an AR mo del. Conversely, given a set of AR co ecients, we can follow this sequence backwards and construct a unique p ower sectrum. Therefore, to ask, \What is the p ower sp ectrum?" is entirely equivalent to asking, \What are the AR co ecients?" A reversible mathematical transformation converts one formulation into the other. But while use of an AR representation is always p ossible, 2 it may not b e appropriate just as representing the function f x = exp x by an in nite series 3 3 of Bessel functions is always p ossible but not always appropriate. Indeed, learning that sp ectrum analysis problems can b e formulated in AR terms amounts to little more than discovering the Mittag{Leer theorem of complex variable theory under rather general conditions an analytic function is determined by its p oles and residues. In this eld there has b een some contention over the relative merits of AR and other mo d- els such as the MA moving average one. Mathematicians never had theological disputes over the relative merits of the Mittag{Leer expansion and the Taylor series expansion. We exp ect that the AR representation will b e appropriate i.e., conveniently parsimonious when a certain resp onse function has only a few p oles, which happ en to b e close to the unit circle; it maybevery inappropriate otherwise. Better understanding should come from an approach that emphasizes logical economyby going directly to the question of interest. Instead of invoking an AR mo del at the b eginning, which might bring in a lot of inappropriate and unnecessary detail, and also limits the scop e of what can b e done thereafter, let us start with a simpler, more exible mo del that contains only the facts of data and noise, the sp eci c quantities wewant to estimate; and no other formal apparatus. If AR relations { or any other kind { are appropriate, then they ought to app ear automatically,asa consequence of our analysis, rather than as arbitrary initial assumptions. This is what did happ en in Burg's problem; MAXENT based on auto covariance data led automatically to a sp ectrum estimator that could b e expressed most concisely and b eautifully in P a a . AR form, the Lagrange multipliers b eing convolutions of the AR co ecients: = k nk n k The rst reaction of some was to dismiss the whole MAXENT principle as \nothing but AR", thereby missing the p oint of Burg's result. What was imp ortantwas not the particular analytical form of the solution; but rather the logic and generality of his metho d of nding it. The reasoning will apply equally well, generating solutions of di erent analytical form, in other problems far b eyond what any AR mo del could cop e with. Indeed, Burg's metho d of extrap olating the auto covariance b eyond the data was identical in rationale, formal relations, and technique, with the means by which mo dern statistical mechanics predicts the course of an irreversible pro cess from incomplete macroscopic data. This demonstration of the p ower and logical unityofaway of thinking, across the gulf of what app eared to b e entirely di erent elds, was of vastly greater, and more p ermanent, scienti c value than merely nding the solution to one particular technical problem.