A Historical Perspective of Spectrum Estimation

PROCEEDINGSIEEE, OF THE VOL. 70, NO. 9, SEPTEMBER885 1982 A Historical Perspective of Spectrum Estimation ENDERS A. ROBINSON Invited Paper Alwhrct-The prehistory of spectral estimation has its mots in an- times, credit for the empirical discovery of spectra goes to the cient times with the development of the calendar and the clock The diversified genius of Sir Isaac Newton [ 11. But the great in- work of F’ythagom in 600 B.C. on the laws of musical harmony found mathematical expression in the eighteenthcentury in terms of the wave terest in spectral analysis made its appearanceonly a little equation. The strueto understand the solution of the wave equation more than a century ago. The prominent German chemist was fhlly resolved by Jean Baptiste Joseph de Fourier in 1807 with Robert Wilhelm Bunsen (18 1 1-1899) repeated Newton’s his introduction of the Fourier series TheFourier theory was ex- experiment of the glass prism. Only Bunsen did not use the tended to the case of arbitrary orthogollpl functions by Stmn and sun’s rays Newton did. Newtonhad found that aray of Liowillein 1836. The Stum+Liouville theory led to the greatest as empirical sum of spectral analysis yet obbhed, namely the formulo sunlight is expanded into a band of many colors, the spectrum tion of quantum mechnnics as given by Heisenberg and SchrMngm in of the rainbow. In Bunsen’s experiment, the role of pure sun- 1925 and 1926. In 1929 John von Neumann put the spectral theory of light was replaced by the burning of an old rag that had been the atom on a Turn mathematical foundation in his spectral represent, soaked in a salt solution (sodium chloride). The beautiful rain- tion theorm in Hilbert space. Meanwhile, Wiener developed the mathematical theory of Brownian mavement in 1923, and in 1930 he in- bow of Newton did not appear. The spectrum, which Bunsen duced genernlized harmonic analysis, that is, the spectral repreaentation saw, only exhibited a few narrow lines, nothing more. One of of a stationuy random process The cOmmon ground of the spectral the lines was a bright yellow. repreaentations of von Neumm and Wiener is the Hilbert space; the Bunsen conveyed this result to Gustav Robert Kirchhoff von Neumm result is for a Hermitian operator, whereas the Wiener (1824-1887),another well-known German scientist.They result is for a unitary operator. Thus these two spectral representations are dated by the Cayley-MBbius transformation. In 1942 Wiener ap knew that the role of the glass prism consisted only in sorting plied his methods to problems of prediction and filtering, and his work the incident rays of light into their respective wavelengths (the wasinterpreted and extended by Norman Levinson. Wienerin his process known as dispenion). The Newton rainbow was the empiricalwork put mm emphasison the autocorrelationfunction extended continuous band of the solar spectrum; it indicates than on the power spectrum that all wavelengths of visible light are present in pure sunlight. The modern history of spectral eaimation begins with the breakthrough of J. W. Tukey in 1949, which is the statistical counterpart of The yellow line, which appeared when the light source was a the breakthrough of Fourier 142 years earlier. This result made pos burning rag, indicated thatthe spectrum of tablesalt con- sible an active development of empirical spectral analysis by research taineda single specific wavelength. Further experiments workers in dl scientifiidisciplines However, spectral analysiswas showed that this yellow line belonged to the element sodium. computationally expensive. A major computational breakthrough oc- No matter what the substance in which sodium appeared, that curred with the publication in 1965 of the fast Fourier transform dge rithm by J. S. Cooley and J. W. Tukey. The Cooley-Tukey method element made its whereabouts known by its bright yellow made it practical to do signal pmcessiy on waveforms in either the spectral line. As time went on, it was found that every chemi- time or thefrequency domain, sometlung never practical with con- cal element has its own characteristic spectrum, and that the tinuous systems. The Fourier trpnaOrm became not just a theoretical spectrum of a given element is always the same, no matter in description, but a tod With the development of the fast Fouriertram form the feld of empirical .spectral analysis grew from obscurity to what compound or substance the element is found. Thus the importance, and is now a major discipline. Further important contribu- spectrum identifies the element, and in this way we can tell tions were the introduction of maximum entropy spectral analysis by what elements are in substances fromthe distant stars to John Burg in 1967, the development ofspectral windows by Emmanuel microscopic objects. Parzen and others starting in the 195O’s, the statistical work of Maurice Priestley and his shod, hypothesis testing in time series analysis by The successes of spectral analysis were colossal. However, Peter Whiffle startiug in 1951, the Box-Jenkins approach by George the spectral theory of the elements could not be explained by Box and G. M. Jenkins in 1970, and autorepshe spectral estimation classical physics. As we know, quantum physics was born and and order-determining criteria by E. Parzen and H. Akaike starting in spectral theory was explained in 1925 and 1926 by the work the 1960’9 To these statistical contributions mustbe added the equdly importantengineering contributions to empirid spectrum analysis, of Werner Heisenberg (1901-1976) and Erwin Schrijdinger which are not treated at dl in this paper, but form the subject matter (1887-1961). In this paper, we will show how spectral theory of the other papen in this special issue. developed in the path to thisgreat achievement. Although most of the glamour of spectral theory has been I. INTRODUCTION associated with quantum physics, we will not neglect the paral- PECTRAL estimation has its roots in ancient times, with lel pathtaken in classical physics. Although thetwo paths the determination of the length of the day, the phases of began diverging with the work of Charles Sturm (1803-1855) the moon, and the length of the year. The calendar and and Joseph Liouville (1809-1882) on the spectral theory of s differential equations, we wiU see that thefinal results, namely, the clock resulted from empirical spectral analysis. In modem the spectral representation of John von Neumann(1903-1957) Manuscript received January 22, 1982; revised April 28, 1982. Thls for quantumphysics, and that of Norbert Wiener (1894-1964) paper is dedicated to Norman Levinson, August 11, 1912-October 10, for classical physics, are intimately related, 1975, my teacher in differential equations. Because light has high frequencies, our instruments cannot Theauthor is with the Departmentof Theoretical and Applied Mechanics and theDepartment of Geological Sciences, Cornell Uni- respondfast enough to directly measure the waveforms. In- versity, Ithaca, NY 14853. stead, the instruments measure the amount of energy in the 0018-9219/82/0900-0885$00.750 1982 IEEE 886 PROCEEDINGSVOL. IEEE, OF THE 70, NO. 9, SEPTEMBER 1982 frequencybands of interest. The measurementand analysis borhood of a certain point x = a, into an infinite series whose of the spectra of other typesof signals, however, take different coefficients are the successive derivatives of the function at the forms. With lower frequency signals, such as mechanical vibra- given point tions, speech, sonar signals, seismic traces, cardiograms, stock market data, and so on, we can measure the signals as functions of time (that is, as time series) and then find the spectra by computation. With the advent of the digital computer, Thus analytic functions are functions which can be differenti- numerical spectrum estimation has become an important field ated to any degree. We know that the definition of the deriva- of study. tive of any order at the pointx = a does not require more than Let us saya few words aboutthe terms“spectrum” and knowledge of the function in anarbitrarily small neighbor- “spectral.” Sir Isaac Newton introducedthe scientific term hood of thepoint x = u. The astonishing property of the “spectrum” using the Latin word for an image. Today, in Taylor series is that the shape of the function at a finite dis- English, we have the word specter meaning ghost or appari- tance h from the point x = u is uniquely determined by the tion, and the corresponding adjective spectral. We also have behavior of the function in the infinitesimal vicinity of the the scientific word spectrum and the dictionary lists the word point x = u. Thus the Taylor series implies that an analytic spectrul as the corresponding adjective. Thus“spectral” has function has a very strong interconnected structure; by study- two meanings. Many feel that we should be careful to use ing the function in a small vicinity of the point x = a, we can “spectrum” in place of “spectral” a) whenever the reference precisely predict what happens at the point x =(I + h, which is to data or physical phenomena, and b) whenever the word is at a finite distance from the point of study. This property, modifies “estimation.”They feel thatthe word “spectral,” however, is restricted to the classof analytic functions. The withits unnecessary ghostly interpretations, should be con- best known analytic functions are, of course, the sine and fined to those usages in a mathematical discipline where the cosine functions, the polynomials, and the rational functions term is deeply embedded. (away from their poles). The material in this paper through Section XI1 surveying the period from antiquity through Levinson and Wiener can be 111. THE DANIELBERNOULLI SOLUTION OF THE described as “The Prehistory of Spectrum Estimation” to em- WAVE EQUATION phasize that spectrum estimation is interpreted estimation as Thegreat Greek mathematicianPythagoras (ca 600 B.C.) from data.The remaining sections may be described as was the fmt to consider a purely physical problem in which “Some Pioneering Contributions to the Development of Meth- spectrum analysis made its appearance.

Load more