Roots of Ensemble Forecasting
Total Page:16
File Type:pdf, Size:1020Kb
JULY 2005 L E W I S 1865 Roots of Ensemble Forecasting JOHN M. LEWIS National Severe Storms Laboratory, Norman, Oklahoma, and Desert Research Institute, Reno, Nevada (Manuscript received 19 August 2004, in final form 10 December 2004) ABSTRACT The generation of a probabilistic view of dynamical weather prediction is traced back to the early 1950s, to that point in time when deterministic short-range numerical weather prediction (NWP) achieved its earliest success. Eric Eady was the first meteorologist to voice concern over strict determinism—that is, a future determined by the initial state without account for uncertainties in that state. By the end of the decade, Philip Thompson and Edward Lorenz explored the predictability limits of deterministic forecasting and set the stage for an alternate view—a stochastic–dynamic view that was enunciated by Edward Epstein. The steps in both operational short-range NWP and extended-range forecasting that justified a coupling between probability and dynamical law are followed. A discussion of the bridge from theory to practice follows, and the study ends with a genealogy of ensemble forecasting as an outgrowth of traditions in the history of science. 1. Introduction assumption). And with guidance and institutional sup- port from John von Neumann at Princeton’s Institute Determinism was the basic tenet of physics from the for Advanced Study, Charney and his team of research- time of Newton (late 1600s) until the late 1800s. Simply ers used this principle to make two successful 24-h fore- stated, the future state of a system is completely deter- casts of the transient features of the large-scale flow mined by the present state of the system. The evolution (initialized on 30 January and 13 February 1949) (Char- of the state is governed by causal relationships such as ney et al. 1950; Platzman 1979). It should be mentioned, the Newtonian equations of motion. Orbital mechanics, however, that other 24-h forecasts made by the team (5 especially its application to the prediction of motion of January 1949, e.g.) were not particularly good. It is in- the heavenly bodies, was the test bed for determinism. structive to read the even-handed accounts of these Pierre-Simon Laplace became champion of determin- events by two of the participants, George Platzman and ism, or the mechanistic view of the universe, and Carl Joseph Smagorinsky (Platzman 1979; Smagorinsky Gauss exhibited its power when he predicted the reap- 1983]. pearance of the planetoid Ceres after its conjunction The success at Princeton set the meteorological with the sun (in 1801–02) (Gauss 1963). world abuzz, and interest in NWP quickly spread to Numerical weather prediction (NWP) initially fol- institutional efforts worldwide [notably at the Univer- lowed the path of determinism. It was Jule Charney’s sity of Stockholm’s International Institute of Meteorol- theoretical treatment of the scales of motion in the at- ogy (“Rossby’s Institute”; Wiin-Nielsen 1991) and the mosphere that laid the foundation for the first success- Air Force Cambridge Research Laboratories (AFCRL; ful NWP (Charney 1948). For the larger-scale motion Thompson 1983)]. By mid-decade, operational short- of the atmosphere, he convincingly demonstrated that range NWP based on these quasigeostrophic principles it was appropriate to predict changes of the hemi- was taking place in the United States and in Sweden. spheric flow pattern by advecting the geostrophic vor- The meteorological community was filled with an opti- ticity with the geostrophic wind (the quasigeostrophic mistic fervor (Smagorinsky 1983, p. 25 ), and by the late 1950s there was hope that extended-range prediction would be possible (beyond several days). In this milieu, Corresponding author address: John M. Lewis, Desert Research Institute, Division of Atmospheric Sciences, 2215 Raggio Pkwy., international politico–scientific efforts were underway Reno, NV 89512. to promote a coordinated collection of weather data E-mail: [email protected] over the globe—in essence, an observational program © 2005 American Meteorological Society Unauthenticated | Downloaded 09/23/21 08:31 PM UTC MWR2949 1866 MONTHLY WEATHER REVIEW VOLUME 133 that would increase the chances of successful long- (MIT). Bureau Chief Francis Reichelderfer’s attitude range weather prediction (the order of weeks). toward NWP is clearly presented in a letter he wrote to Despite promising results from the NWP community Shuman. This letter is reproduced in Fig. 1. Shuman (in both research and operations), questions began to believed that “. Reich had always been searching for arise regarding the limits of deterministic prediction—a an objective way to forecast weather” (F. Shuman 2004, limit that is governed by growth of error. The initial personal communication), and his interest in the new state is generally erroneous and the models are imper- numerical method is obvious from the letter. Yet, it is fect. Furthermore, as would be made known in the clear that he expected Shuman to rigorously evaluate early 1960s, the essential character of the causal laws— the prospects for NWP as an alternative to existing unstable systems characterized by nonperiodicity— methods at the bureau. Shuman’s evaluation was a key placed limits on the predictability of the system. factor in the bureau’s decision to join the U.S. Air Whereas Gauss and contemporaries found that the Force and U.S. Navy in establishing the Joint Numeri- two-body problem of celestial mechanics tolerated cal Weather Prediction (JNWP) unit in 1954. This co- small error in the initial state, meteorological predic- ordinated project led to the first operational NWP in tion under nonperiodic constraints would be found to the United States in 1955. Shuman remembers the at- be less forgiving of these uncertainties. titude of the forecasters toward this new approach to The line of research that explored the limits of de- prediction: terministic prediction is the subject of this historical There was some reluctance on the part of the field study. The thrust of this research led to an alternative forecasters to accept the NWP products. But at the approach to NWP—a stochastic–dynamic approach center [National Meteorological Center (NMC)]1 that coupled probability with determinism. Its practical there was enthusiasm among those in the Analysis implementation in meteorology has come to be called Section, especially Harlan Saylor, Bill Burnett, and ensemble prediction. Ed Fawcett. It was evident even when things didn’t To set the stage for discussions of the historical work, look that great, when they looked “very chancy,” sort the zeitgeist associated with the early decades of NWP of like Lincoln’s charge to Grant. (F. Shuman 2004, is reconstructed. The contributions of the scientists who personal communication) laid the foundation for stochastic–dynamic prediction Shuman’s assessment is echoed in the following remi- in meteorology are examined. This is followed by a niscence of Aksel Wiin-Nielsen, an NWP pioneer from discussion of events that built on this framework and Denmark who later worked at NMC and became the led to the practice of ensemble forecasting in meteorol- first director of the European Centre for Medium- ogy. Finally, a schematic diagram is presented in the Range Weather Forecasts (ECMWF) in the 1970s: appendix that links stochastic–dynamic NWP to the There was a degree of animosity against NWP by fundamental research traditions in the history of sci- the forecasters and operations people in meteorology. ence. They said, “We really can’t use this stuff you’re put- ting out. Why don’t you get some humidity and clouds 2. Zeitgeist into it, real weather!” (A. Wiin-Nielsen 1992, personal communication) Following the successful demonstration of NWP at Princeton in 1950, and the operational production of b. Frederick Sanders NWP forecasts in the mid-1950s, a wide range of reac- The following recollection of Sanders, then a newly tions to these developments took place in the meteo- appointed professor at MIT with a wealth of experience rological community. In an effort to reconstruct the as a forecaster during and after World War II, exhibits spirit of this time, vignettes are offered from meteo- the divergence of opinion in academia: rologists who were actively involved in administration, Well, I remember this seminar in the mid-1950s by research, or operations during this period—the 1950s a theoretician who shall remain unnamed. At one and 1960s. point he was prompted to say: “A hierarchy of ever more powerful and sophisticated models will be a. Frederick Shuman/Francis Reichelderfer/Aksel brought to bear on meteorology until our problems Wiin-Nielsen will crumble before the onslaught of mathematical Frederick Shuman joined the U.S. Weather Bureau (USWB) in the early 1950s after completion of his doc- 1 Information inserted into the quotations by the author is torate at the Massachusetts Institute of Technology bracketed. Unauthenticated | Downloaded 09/23/21 08:31 PM UTC JULY 2005 L E W I S 1867 FIG. 1. Letter from Francis Reichelderfer to Fred Shuman. (Courtesy of F. Shuman.) physics.” I didn’t share in this unbridled enthusiasm 1947; Eady 19492). Their separate but interlocking con- nor did I anticipate better forecasts because of the tributions are masterfully reviewed by Gill (1982, sec- computer. (F. Sanders 1994, personal communication) c. Eric Eady 2 Eady was awarded the Ph.D. in 1948, Department of Math- ematics, Imperial College, London, United Kingdom. The title of Eady and Charney are credited with independently his unpublished dissertation, “the theory of development in dy- developing the theory of baroclinic instability (Charney namical meteorology,” is essentially the same as Eady (1949). Unauthenticated | Downloaded 09/23/21 08:31 PM UTC 1868 MONTHLY WEATHER REVIEW VOLUME 133 tions 13.3 and 13.4). The basic condition of the atmo- • rapid retrogression of the large-scale components of spheric state under which infinitesimal perturbations motion; grow is the fundamental underpinning of their theoret- • only 65% of the variance in the day-to-day changes in ical development.