Mathematical Work of Norbert Wiener V

Mathematical Work of Norbert Wiener V. Mandrekar

he Norbert Wiener Centenary Con- Norbert Wiener began his mathematical life in gress was held at Michigan State Uni- logic and foundations. He went to England to versity, November 27–December 3, work with B. Russell. His contact with G. H. 1994. The Congress was cosponsored Hardy and others expanded the spectrum of his by the American Mathematical Society, work to include analysis, engineering, statistics, Tthe International Association of Cybernetics, and and physics. He is known to the general public the World Organization of Systems and Cyber- as a founder of cybernetics, but to mathemati- netics. cians he is known for his fundamental contri- The aim of the Congress was to reveal the butions in analysis and for his use of random- depth and strong coherence of thought that runs ness to expand the vision and applications of through Wiener’s legacy and to exhibit its influ- mathematics. ence on current research. We shall explore this aspect of his work. His The Congress drew fifty- interest in randomness begins with his work on eight participants, includ- realizing Brownian motion in a function space. ing thirteen from Europe, The insight gained raised further questions and two from Asia, one from issues which naturally led him to generalized Mexico, and two from harmonic analysis, Tauberian theorems, and Canada. There were later Paley-Wiener theory, which was then used twenty-nine addresses, six- to study problems involving randomness in sig- teen contributed papers, nal analysis. In addition, Wiener introduced fun- and one round table dis- damental ideas and techniques in potential the- cussion. The Congress ory and homogeneous chaos, which to this day ended with the award of are finding applications to emerging fields in the Norbert Wiener Centenary Medal to Claude mathematics at the hands of other researchers. E. Shannon (Shannon’s wife accepted the medal In order to refer systematically to Wiener’s on his behalf. See photo above.). works, we shall refer to his Collected works [NW1] The following article, written by one of the co- with double brackets. For example, [[34a]] shall organizers of the Congress, describes some of mean reference 34a in [NW1]. We also bring at- Wiener’s work explored at the Congress and dis- cusses its impact on contemporary research. V. Mandrekar is professor of statistics and probability at Michigan State University in East Lansing, MI. His e- —Allyn Jackson mail address is [email protected].

664 NOTICES OF THE AMS VOLUME 42, NUMBER 6 tention to the biographical article by P. Masani [NW2], written at the time of Wiener’s death. We shall begin our look at Wiener’s work first by examining his construction of the measure W on the space C0[0, 1] of real-valued continuous functions x on [0,1] with x(0) = 0 as a model for paths traversed by a particle following Brown- ian motion. This is contained in five papers from 1920–1924. First, we describe the measure space for W (Wiener measure). Let tn,i be a doubly { } indexed set of points in [0,1] with these properties: a. for each n, 0 < tn,1 < tn,2 < < t < 1; · · · n,kn b. for each n the set tn,i is a subset of tn+1,i ; { } { } c. the set of all tn,i is dense in [0,1]. { } For each n, introduce the set πn of subsets of C0[0, 1]: an element of πn is of the form

I = x C0[0, 1] : { ∈ ai < x(tn,i) < bi, i = 1, 2, . . . , kn . } Notice that, because of b) πn πn+1. The ⊂ Wiener measure of I, W (I) is b1 bn p(t1, y1)p(t2 t1, y2 y1) Za1 · · · Zan − − · · · p(tn tn 1, yn yn 1)dy1 dyn where − − − − · · · 2 1/2 y p(t, y) = (2πt)− exp( ), y R. − 2t ∈ πn is the basis for a topology on C0[0, 1], and Before we go to Wiener’s contribution to po- ∪ W defines a Borel measure on this space. To tential theory in the twenties, we want to examine show this, Wiener follows the Daniell approach a trail of ideas starting from the above con- to measure theory: starting with a positive func- struction to his major work of the thirties on gen- tional on a (sufficiently large) linear space of eralized harmonic analysis. Baire functions, satisfying the monotone con- Immediately after his paper [[24d]], Wiener vergence theorem of Lebesgue, Daniell extends gave a Fourier series representation for Brown- this function as an integral on all Baire functions. ian motion ([[24e]], p. 570), in terms of inde- Here we take the functions space to be , the pendent identically distributed (i. i. d.) standard L set of simple functions based on πn; the normal random variables an (see also [[34a]], ∪ p. 21). In [[27a]] Wiener gives{ } the definition of functional is fdW on : for f = viχIi , L the spectrum of a sequence of complex numbers fdW = viW(IRi). The crucial point is toP prove continuity of W from above at 0: For fn a non- fn satisfying R P { } { } increasing sequence in converging to 0, Wiener N L ¯ succeeds in showing that fndW 0. For this lim fn+kfn. → N →∞ N he constructs (for each integerR m) a compact set −X Km C0[0, 1] of Hölder continuous functions of The above-mentioned sequence an satisfies ⊂ order < 1/2, with Wiener measure greater than Wiener’s condition a. e., and from the represen- 1 1/m. From this it follows that W is a Borel tation in [[34a]], one gets − measure on C0[0, 1] and that Hölder continuous 2π inu paths with finite quadratic variation exist al- an = e dB(u) most everywhere in Wiener space. These paths Z0 in particular are nowhere differentiable. Thus where B is the Brownian motion and the integral Wiener uses the topological structure of C[0, 1] is in the sense of Wiener as given in [[23a]] using to construct Brownian motion. In the work of A. integration by parts. Thus Wiener seems to have N. Kolmogorov (see [10]) in 1933 for probabil- been aware of the harmonic analysis of a se- ity measures on R T (T arbitrary), regularity in the quence where the “spectral measure” B may not finite-dimensional case is used to produce the be a measure. In recent years one defines the compact sets, and the topological structure of spectrum in the generalized sense [14] using R T is exploited to produce a function of Baire the theory of generalized functions. The latter Class two. theory was unavailable to Wiener in a systematic

JUNE 1995 NOTICES OF THE AMS 665 way, although he was conceptually aware of it inal theorem. The work of Hardy and (see [20], p. 427). In the continuous parameter Littlewood, unlike that of Tauber, case, the relation to generalized function was no- makes very appreciable demands on ticed in [5] (see also [3]). Wiener instead chose analytical technique and is capable, the techniques of the Plancherel Theorem. This among other things, of supplying the had two advantages: first, it led to Tauberian the- gaps in Poisson’s imperfect discus- orems, and second, it found applications to elec- sion of the convergence of the Fourier trical engineering, as Fourier theory was an es- series. For these reasons, I feel that tablished method for engineers. it would be far more appropriate to In the mind of an analyst of the 1920s, a term these theorems Hardy-Little- Tauberian theorem is one which deduces the wood theorems, were it not that convergence of an infinite series on the basis of usage has sanctioned the other ap- the properties of the function it defines (via pellation. power series, Fourier series, or as coefficients in any standard expansion of a function) and any Wiener’s central Tauberian theorem is an kind of auxiliary hypothesis which prevents the analogous theorem about indefinite integrals. It general term of the series from converging to goes like this. Letf be a bounded measurable function on the real line (f L ), and let K be zero too slowly. Here is Wiener’s discussion in ∈ ∞ an L1 function (that is, ∞ K(ξ) dξ < ). What his 1932 paper, “Tauberian Theorems”, in the An- | | ∞ is at issue is the limit of− ∞f K(x) as x : nals of Mathematics: R ∗ → ∞ Tauberian theorems gain their name ∞ (1) limx ) f (ξ)K(ξ x)dξ. from a theorem published by →∞ − Z−∞ A. Tauber in 1897, to the effect that The hope is that this limit is A ∞ K(ξ)dξ for if −∞ some number A independent of KR . The theorem of Wiener is that if this is true for some K L1 n ∈ (0.07) limx 1− 0∞anx = A → whose Fourier transform never vanishes, then it is true for all K L1. ∈ and Σ Using the Tauberian theorems, Wiener gave 1 a proof in this paper of the Prime Number The- (0.08) an = o( ), orem. His proof reduces it to the convergence n of a certain definite integral (derived from the then Riemann zeta function) based on function-the- oretic information about the function the in- (0.09) 0∞an = A. definite integral defines. Wiener’s Tauberian theorems followed from This is a conditionedΣ converse of two results which themselves have had great Abel’s theorem, which stated that impact on the development of modern analysis. (0.07) follows from (0.09) without the I. Suppose that f is a continuous function on mediation of any hypothesis such as the circle which has an absolutely convergent (0.08). Such conditioned inverses of Fourier series. If f never vanishes on the unit cir- Abel’s theorem, and of other analo- cle, then 1/f also has an absolutely convergent gous theorems which assert that the Fourier series. convergence of a series implies its Wiener’s argument proceeds from a local summability by a certain method to statement to the global one, using some very the same sum, have been especially technical convergence arguments. This result is studied by G. H. Hardy and J. E. Lit- a lemma on the way to proving: tlewood and have been termed by II. Let f be an L1 function on the real line. The them Tauberian. linear span of the set of translates of f (functions of x of the form f (x + y)) is dense in L1 if and It is the service of Hardy and Little- only if the Fourier transform of f vanishes on a wood to have replaced hypothesis set of measure zero. (0.08) by hypotheses of the form It is quite easy to establish the necessary condition, for if F represents the Fourier transform 1 (0.10) an = O( ), of f, then the Fourier transform S of the span n of all translates of f is the linear span of the set iyξ or even of the form nan > K. The of all functions of the form e F(ξ). Since the importance of these generalizations− closure of the linear span of the exponentials is scarcely to be exaggerated. They far eiyξ in the uniform norm on any compact set co- exceed in significance Tauber’s orig- incides with all continuous functions on that

666 NOTICES OF THE AMS VOLUME 42, NUMBER 6 2 compact set, all functions in S will vanish on the L1( , ) is of the form f = ϕ a. e. where ϕ −∞ ∞ 2| | intersection of that compact set with the zero is as above is log f (λ)/(1 + λ ) L1( , ). set of F. If that set has positive measure, then These also turned out to be important∈ −∞ ∞results there certainly are functions in L1 whose Fourier in the study of one variable prediction theory and transform is identically 1 there; such a function filtering problems. In his work on multivariate cannot be in the closure of the linear span of the prediction theory with P. Masani and E. J. translates of f. The real issue is the sufficiency. Aukowitz, Wiener took the approach of Kol- The property of the Fourier transform, that mogorov [11]. Let us illustrate this in the one- it changes translation operators into multipli- variable case. cation operators, is fundamental to all the uses Let Xn, n Z be a sequence in the complex of the Fourier transform. What was apparently Hilbert{ space∈ H}= L2( , , P) with ( , , P) a first understood by Wiener is that it changes geo- probability space. Let L(XF: n) be the pastF of the metric questions about function spaces on the process, namely, sp XΩk, k n whereΩ sp de- circle or in Rn into algebraic questions about the notes the closure of{ the linear≤ } span. Under ring of multiplication operators on the trans- Wiener’s condition on ergodicity formed space. This ultimately led to considera- 2π inx tions of abstract normed rings and great gener- EXnX¯0 = (Xn, X0) = e dS(λ) alizations and simplifications of these works of Z Wiener (first by the Uppsala school of analysts where S is the spectral measure. Thus and completed by Gelfand). 2 sp Xk,k Z is isometric to L ([0,2π],S), Thus, starting from the construction of Brown- { ∈ } inx sending Xn e . Following Wold, one says the ian motion, Wiener was led to several significant → process is nondeterministic if nL(X : n) = 0 contributions to analysis, harmonic analysis, al- (there is no remote past) and ∩deterministic{ if} most periodic functions, and number theory. L(X : n) = L(X : n + 1) for all n (the remote past His generalized harmonic analysis has had major contains all the information of the present and impact in signal analysis and time series ([NW2] the future). By projecting Xn onto the remote and [[49g]]), since in practical situations one can past L(X : ) = nL(X : n) and its orthogonal only observe precisely the correlation functions −∞ ∩ complement in L(X) = n L(X : n), one gets an associated with a signal. Here one needs to an- orthogonal decompositionW Xn = Yn + Zn where alyze the spectrum. In all of Wiener’s work, the Yn is deterministic and Zn is purely non- { } { } concept of ergodicity, to which we shall come in deterministic. Yn is uninteresting for prediction a while, was implicit. theory. Let ξn be the new part (innovation) of Zn, Wiener’s work with Paley started with [[33a]], the part of Zn which is orthogonal to where they considered problems of random L(Z : n 1). Writing Zn in terms of ξn , one Fourier series with i. i. d. Gaussian random vari- sees that− the spectral{ measure} of the process{ } is ables replacing Rademacher functions. In his absolutely continuous with respect to Lebesgue work on differential space [[23d]], Wiener had measure and its spectral density factors. Being given a definition of a stochastic integral. He con- geometric, this technique goes through in the sidered the process multivariate case with minor changes. In terms + Y (t, ) = ∞ ϕ(u t)B(du, ) of correlation, a process with factorable density · − · is purely nondeterministic. The important result Z−∞ for ϕ L2( , + ). Using the Paley-Wiener re- used here, (the Szegö Theorem) that ∈ −∞ ∞ 2π sult that there is a measure-preserving ergodic 0 `nf(x)dx > is equivalent to the factor- flow on [0, 1] given by translations of Brownian −∞ izationR of f, requires ideas from Paley-Wiener the- motion, one can see that this is a stationary ory. For a vector process, the spectral measure process. In fact, considering this representation is matrix (or, in general, operator) valued. This of the signal, one can show that a correlation method for factoring density survives in the full function exists a. e. and is constant. It has ab- rank case. However, the Szegö Theorem gets solutely continuous spectrum with density delicate but can also be reduced to univariate ϕˆ (λ) 2. To study causality, one needs to as- case using a technique of Lowdenslager [13] and | | sume that ϕ(u) = 0, u 0. In this context the some geometric arguments. Because of the gen- ≥ following two basic results of Paley and Wiener erality of these arguments, this has implications are indispensable [[34d]]. to the invariant subspace problem, scattering the- 1. ϕ L2( , ) is the boundary value of a ory, and harmonic analysis [1]. ∈ −∞ ∞ function ϕ+ analytic in the upper half plane Extension of the idea of expressing a (strictly) which is uniformly L2 on every horizontal line stationary process in terms of orthogonal (in- (the Hardy class H2) if the Fourier transform ϕˆ dependent) innovations for nonlinear predic- vanishes on ( , 0). tion problem was initiated in the work of Wiener −∞ 2. The necessary and sufficient condition that with Kallianpur and was successfully carried a nonnegative function f on ( , ) and in out in special cases by M. Rosenblatt and D. Han- −∞ ∞

JUNE 1995 NOTICES OF THE AMS 667 son. Attempts to obtain computable expressions condition is strikingly geometric: Let x be a for the optimal nonlinear predictor were made point of and κn the capacity of the intersec- n in Wiener-Masani [[59b]]. Under rather restrictive tion of D with the disk of radius 2− centered 2 assumptions, the best (nonlinear) predictor of Xn at x. ThenΓ x is regular if and only if κn di- with lead h > 0 is given by verges. X 2 Thus, Wiener introduced the central conceptsΣ E(Xn 0 ) = Ln lim Qn(X0, X1, ..., X mn ) |F − − of the modern potential theory: harmonic mea- where (i) X is the α-algebra generated by sure, generalized solutions, capacity of sets, and F0 Xm, M 0 , (ii) mn 0 integer based on n, regular points. At just about the same time (al- { ≤ } ≥ (iii) Qn is a polynomial in mn + 1 variables. Here though the paper was published a year earlier Xn, n Z is a strictly stationary process than Wiener’s), O. Perron found another method { ∈ } L ( , ,P). The most general result in this di- for solving the Dirichlet problem on an arbi- ⊆ ∞ F rection is due to Ito and Stein and is stated in trary domain. His condition for regularity at a KallianpurΩ [9]. point x was stated in terms of the local existence However, synthesis of the black box is a del- of a barrier function: a function subharmonic icate problem [19] and Wiener gave his thoughts near x which, on the closure of D, has a strict on it in his monograph [[58a]] using the homo- maximum at x. In his paper [[25a]], Wiener geneous chaos paper. Wiener also used this to showed that the two constructions and condi- study the continuous analogue of the nonlinear tions for regularity are equivalent. prediction problem. This is one of many of Let us now turn to Wiener’s work on homo- Wiener’s papers in which his ideas were shown geneous chaos, motivated by his interest in tur- to be basic to the development of the founda- bulence and statistical mechanics. Wiener defines tions of the area. His first work of this type was a Gaussian function for α on [0,1] whose values in potential theory. are set functions ξ(A, α) on Borel sets in Rn { } Almost simultaneously with his work on of finite Lebesgue measure with this property: 1 Brownian motion, Wiener undertook his funda- 0 ξ(A, α)ξ(B, α)dα is the Lebesgue measure of mental work on potential theory. Interestingly, A B. This is called chaos and he then defines R ∩ he again used the Daniell integral to attack the what can be called the multiple Wiener integral. It Dirichlet problem. Let D be a domain in the should be noted that if the chaos is constructed from plane with boundary . For what continuous Brownian motion, i.e., B(A) = A dB(t) with n = ϕ k functions ϕ on is there a function u con- 1, then 1A A AdBdB dB = B(A) . × ×···× R · · · tinuous on the closure ofΓ D, equal to ϕon , and This is differentR R R R from the definition of K. Ito and satisfying Laplace’sΓ equation in D? It had been I. Segal, where the above integral is Hk(B(A)), Hk known for some time that this question (DirichΓ - being the Hermite polynomial of order k. If one let’s problem) can be solved for all continuous examines Wiener’s later work [[58a]], one can see functions on when D is a smoothly bounded that he orthogonalizes his integral, relating it to domain. Now, for a general domain D, Wiener the Ito multiple integral. What Wiener defines is selected an increasingΓ sequence of smoothly called in current literature the Stratonowich bounded domains Dn filling out D from the in- multiple integral. The relation between the terior. Now fix a point y in D. For ϕ defined on Wiener and Ito integrals is precisely the Hu- ϕ ϕ , let Iy(ϕ) = limn un , where un solves Meyer formula [6], a special case of which for →∞ Dirichlet’s problem on Dn for some continuous k = 2 occurs in [[58a]]. It should be noted that extensionΓ of the given f on . Wiener showed that coefficients in the Hu-Meyer formula are pre- Iy is well defined, is independent of the choices cisely those expressing polynomials through of the sequence Dn and theΓ continuous exten- Hermite polynomials. Wiener’s aim was to prove sion of ϕ, and meets all the conditions of the a multidimensional generalization of the Birk- Daniell integral on C( ). This gives for each y a hoff Ergodic Theorem and apply it to larger probability measure µy such that classes of functions on chaos using approxi- Γ mations through his integral. Although Wiener ϕ u (y) = ϕ(x)µy (dx) was not able to attack the problem of statistical Z∂D turbulence, the ideas of Wiener have influenced is harmonic in D. The probability measure µy physics problems. This can be seen in the work is called harmonic measure. Kakutani related it of Segal [22] on quantum field theory and Hu- to the exit time of Brownian motion in 1944. A Meyer on the computation of Feynman integrals, point x on the boundary is called regular if justifying the work of Hida-Streit. An interest- uϕ(y) ϕ(x), as y x for all continuous func- ing consequence of this work on multiple inte- → → tions ϕ on ∂D. Wiener gave a necessary and suf- grals is the work of A. V. Skorokhod [23], which ficient condition for a boundary point to be reg- extends Ito’s generalization of the Wiener inte- ular in terms of capacity, a notion he adapted gral. The latter integral can be used to study non- to this purpose based on physical intuition.The linear filter theory. It should also be observed

668 NOTICES OF THE AMS VOLUME 42, NUMBER 6 that Segal introduced a finitely additive (cylin- [9] G. Kallianpur, Comments on [59b], Collected drical) measure in his work on quantum field the- Works, Vol. III, pp. 402–424. ory. Countably additive extension of this mea- [10] A. N. Kolmogorov, Foundations of the theory of sure led L. Gross to the study of abstract Wiener probability, Chelsea, New York, 1950. [11] , Stationary sequences in Hilbert space, Bull. spaces—an algebraic generalization of Wiener’s ——— Math. Univ. Moscow 2 (1941). original idea on Brownian motion! Expansion of [12] N. Levinson, The Wiener RMS (root mean square) nonlinear functions of chaos has made it possi- error criterion in filter design and prediction and ble to create a calculus on the abstract Wiener heuristic exposition of Wiener’s mathematical the- space called Malliavin calculus (see [24]), which ory of prediction and filtering, Appendix B and C, has proven useful in the study of statistical me- Time Series, N. Wiener, M.I.T. Press, Cambridge, chanics. The just-mentioned work studies “gen- MA, 1949. [13] D. B. Lowdenslager, On factoring matrix valued eralized functions” on abstract Wiener space, the functions, Ann. of Math. 78 (1963), 450–454. analogue of which for Lebesgue measure was also [14] A. Makagon and V. Mandrekar, The spectral in Wiener’s work (see [20], p. 427). Although representation of SαS processes: Harmonizability Wiener proved a generalization of Birkhoff’s Er- and regularity, Probab. Theory Related Fields 85 godic Theorem for multiparameter flows, he (1990), 1–11. ended up proving a very general Ergodic Theo- [15] ———, Remarks on AC continuity and the spectral rem [[39a]]. representation of stationary SαS sequences, Ulam We are unable to cover the work of Wiener on Quart. 2 (1994). [16] P. R. Masani, Norbert Wiener: A survey of a frag- foundations, cybernetics, prosthesis, and eco- ment of his life and work, A Century of Math- nomic philosophy; the bases of these also have ematics in America, part III, Amer. Math. Soc., deep mathematical ideas. Excellent references for Providence, RI, 1989, pp. 299–341. Wiener’s complete work, of course, are [17] and [17] ———, Norbert Wiener 1894–1964, Birkhauser, [21]. Basel and Boston, 1990. I would like to thank Professor P. R. Masani [18] ———, The scientific methodology in the light of cy- for guiding me through the work of Wiener and bernetics, Kybernetes 23 (1994), no. 4. [19] H. P. McKean, Wiener’s theory of non-linear noise, Professor R. V. Ramamoorthi for the discussions Stochastic Differential Equations (Proc. SIAM-AMS about the style of presentation. In addition, I Sympos. Appl. Math., New York, 1972), SIAM-AMS want to thank the editor for his untiring effort Proc., vol. 6, Amer. Math. Soc., Providence, RI, for the improvement of this work. 1973, pp. 191–209. [20] L. Schwartz, Comments on [26c]. Collected Works, Bibliography Vol. II, p. 425–427. [NW1] Norbert Wiener: Collected works I, 1976; II, 1979; [21] I. E. Segal, Norbert Wiener’s biographical memoirs, III, 1981; IV, 1984 (P. Masani, ed.), M.I.T Press, Vol. 61, Natl. Acad. Press, 1992, pp. 388–436. [22] , Tensor algebra over Hilbert space, Trans. Cambridge, MA. ——— Amer. Math. Soc. 81 (1956), 106–134. [NW2] Norbert Wiener 1894–1964, Bull. Amer. Math. [23] A. V. Skorokhod, On generalization of stochastic Soc. 72 (1966), no. 1, part II. integral, Theory Probab. Appl. 20 (1975), 219–233. [1] E. J. Akutowitz, Comments on [59a], Collected [24] S. Watanabe, Stochastic differential equations and Works, Vol. III, p. 274. Malliavin calculus, Tata Inst. Fund. Res. Lectures [2] E. Berkson, T. A. Gillespie, and P. S. Muhly, Ab- on Math. and Phys., Bombay, 1984. stract spectral decompositions guaranteed by Hilbert transform, Proc. London Math. Soc. 53 (1986), 489–517. [3] S. D. Chatterji, The mathematical work of Nor- bert Wiener (1894–1964), Kybernetes 23 (1994), 34–45. [4] ——— , Life and works of Norbert Wiener (1894–1964), Jahrbuch Vberblicke Mathematik, Vieweg Verlag, Wiesbaden, 1993, pp. 153–184. [5] Y. Z. Cheng and K. S. Lau, Wiener transformations of functionals with bounded averages, Proc. Amer. Math. Soc. 108 (1990), 441–421. [6] Y. Z. Hu and P. A. Meyer, Chaos de Wiener et in- tegrale de Feynman, Seminaire de Probabilites XXII, Lecture Notes in Math., vol. 1321, Springer, New York, 1988, pp. 51–71. [7] K. Ito, Multiple Wiener integral, J. Math. Soc. Japan 3 (1951), 157–169. [8] S. Kakutani, Two-dimensional Brownian motion and harmonic functions, Proc. Imp. Acad. Tokyo 20 (1944), 706–714.

JULY 1995 NOTICES OF THE AMS 669