Numerical Recip es: Do es This Paradigm Havea Future? William H. Press Harvard-Smithsonian Center for Astrophysics and Saul A. Teukolsky DepartmentofPhysics, Cornell University June 6, 1997 Long-time readers of Computers in Physics may rememb er us as the ed- itors/authors of the Numerical Recip es column that ran from 1988 through 1992. At that time, with the publication of the Second Edition of our Nu- merical Recipes (NR) b o oks in C and Fortran[1, 2 ], we to ok a sabbatical leave from column writing, a leave that b ecame inadvertently p ermanent. Now, as a part of CIP's Tenth Anniversary celebration, wehave b een invited backto o er some observations ab out the past of scienti c computing, including the educational niche o ccupied by our b o oks, and to make some prognostications ab out where the eld is going. CIP's rst decade of publication closely overlaps Numerical Recipes' rst decade in print, NR's original Fortran edition having b een published in early 1986. This is not entirely a coincidence. In the preceding 15 years (the 1970s through mid 80s) the constituency of scienti c computing had undergone an extraordinary broadening, from a few sp ecialists in only certain sub elds, to virtually every researcher in broad areas of science. Available technology had moved from the single campus computer, jobs submitted via punch-card decks handed to a functionary b ehind a little window, to widely available \time sharing systems", minicomputers, and the nascent PC, whose revolu- tion has b een the story of the last decade. Nevertheless, certain scienti c 1 institutions are notoriously slow to react to structural revolutions in science, with journals and textb o oks among the slowest and sto dgiest of all. By the 80s, there were obvious needs to which b oth CIP (as a journal) and NR (as a textb o ok) were inevitable reactions. Inside the BlackBoxes Through the 1970s, at least, almost all physicists { and we include ourselves { enjoyed a blissful, almost touching, ignorance of numerical metho ds. To give a p ersonal example, we recall an early collab oration, when wewere b oth graduate students at Caltech. Wewere having dicultyinintegrating a second-order ODE numerically,andourmutual thesis advisor suggested that we consult Herb Keller, the distinguished professor of applied mathematics. \What metho d are you using?" Keller asked us. Welooked at each other blankly { you mean there's more than one numerical metho d? \Runge-Kutta," of course weanswered. \Runge-Kutta! Runge-Kutta!" he exclaimed, banging his head with his st. \That's all you physicists know!" He suggested that welookinto Bulirsch-Sto er, whichwe did, to our eventual b ene t. Eventual, but not immediate, b ecause there is a twist to this story: Bulirsch-Sto er turned out to b e a poor choice of metho d for our problem, while Runge-Kutta, when weeventually learned to integrate awayfrom{ not into { singularities, and to make a pre-integration changeofvariables in the equations, worked splendidly. So this story illustrates not only our lack of knowledge ab out numerical metho ds, but also that when physicists con- sult professional numerical analysts, either in p erson or through a b o ok or journal article, they not infrequently will b e disapp ointed. This and similar early exp eriences rmly convinced us of the necessity for an algorithm's user (the physicist) to understand \what is inside the blackbox." This ideal b ecame, later, one of the de ning features of our Numerical Recip es pro ject. It was then, and remains now, exceedingly controversial. The physicist-reader may b e astonished, b ecause physicists are all tinker- ers and black-b ox disassemblers at heart. However, there is an opp osite, and much b etter established, dogma from the communityofnumerical an- alysts, roughly: \Go o d numerical metho ds are sophisticated, highly tuned, and based on theorems, not tinkering. Users should interact with such algo- 2 rithms through de ned interfaces, and should b e prevented from mo difying their internals { for the users' own go o d." This \central dogma of the mathematical software community" informed the large scienti c libraries that dominated the 1970s and 1980s, the NAG library (develop ed in the U.K.) and its American cousin, IMSL. It continues to b e a dominant in uence to dayinsuch useful computational environments as MATLAB and Mathematica. We don't subscrib e to this religion, but it is worth p ointing out that it is the dominant religion (except p erhaps among physicists and astronomers) in mathematical software. This controversy will continue, and NR's viewp ointisby no means assured of survival! An early b o ok review of NR, by Iserles[3], gives an interesting and balanced p ersp ective on the issue. Origins of Numerical Recip es Not infrequently we are asked questions like, \How did NR come ab out?" and \How long did it take to write?" A brief history: Although Brian Flannery and Bill Vetterling have pursued highly success- ful careers in industry (at Exxon Research Labs, and Polaroid, resp ectively), we four NR authors were all in academia at the pro ject's inception. Three of us (Flannery, Press, and Teukolsky) had taught courses on scienti c com- puting in our resp ective departments, and we had all noticed { one could hardly not notice { the almost complete lackofany usable text for a physics- oriented course. Flannery, who taught a course at Harvard to astronomers in 1979, was the rst to suggest that a b o ok should b e written, and his course notes were the original sca olding for the pro ject. Press to ok over the course, and leadership of the pro ject, when Flannery left Harvard for Exxon at the b eginning of 1981. Just as there was a \ fth Beatle," there was also a fth NR author! Paul Horowitz, co-author of the well known b o ok on electronics[4], was an NR author from Septemb er, 1982, through the rst part of 1983; so he was in fact the third, not fth, Beatle. When he left the collab oration (parting on friendly terms, but having written nothing), he contributed his no-doubt fa- vorite phrase, \the art of " to the b o ok's subtitle. Press sp ent the month of April, 1983, at the Institute for Advanced Study in Princeton, writing as fast as he could. The sheer volume of the output (or rather, lack thereof ) con- 3 vinced Press and Flannery that more recruitmentwas in order. Saul Teukol- sky joined the collab oration in August; and Bill Vetterling, in Septemb er. Vetterling, an exp erimental physicist, replaced Horowitz's useful lab oratory and instrumentation p ersp ective in the pro ject. With the increased numb er of collab orators, the b o ok was more than half done by early 1984, at whichtimewe b egan to talk to publishers. Several had expressed early interest, including Addison-Wesley's Carl Harris (who was probably the rst to suggest the idea of a b o ok based on Flannery's course), and Ed Tenner of Princeton University Press. Tenner senta big piece of our manuscript out for review, and agreed to share the reviews with us. The reviewers were Forman Acton (himself the author of a wonderful b o ok on computing[5]) and Stephen Wolfram (long b efore the rst release of Mathematica). Acton's review was laudatory.Wolfram wrote, \The b o ok's style is chatty, and in parts entertaining to read. But in achieving its practical ends, much of it is quite sterile." One would never saysuch a thing ab out Wolfram's new b o dice-ripp er[6], of course! David Tranah ultimately convinced us to sign with Cambridge University Press, a choice that we don't regret. C.U.P. is big enough to do a go o d job on worldwide distribution, but small enough to b e resp onsive to our needs as authors. As a not-for-pro t organization, C.U.P. also has the luxury of taking a longer view, where a purely commercial publisher might optimize only for the short run. For manyyears now, Lauren Cowles has b een our capable and indulgent editor at C.U.P.'s North American branchinNew York. The b eginning was somewhat ro ckier, however. Wedelivered the com- pleted manuscript, in the original version of TeX (now called TeX78 [7]), on Novemb er 24, 1984, in the form of a VAX/VMS half-inchbackup tap e, as cre- ated on a MicroVAX I I. (We include these details for their quaint ring.) Ours was the rst TeX manuscript ever received by C.U.P.Wewanted C.U.P.to have our crude formatting macros rede ned by a professional b o ok designer, and wewanted to have the b o ok copy-edited by a professional copy editor. In the event, we got the copy editing, but not the b o ok design. Atthat time, a mere decade ago, there was simply no interface b etween \book de- signers" and \p eople who can use TeX". Indeed, it to ok C.U.P.morethan 6months to nd a rm that could even read our tap e and pro duce camera- ready output in the same format that we originally delivered. (C.U.P. sug- gested seriously that the b o ok could b e published faster { and cheap er { 4 if they reset it all in typ e, by hand, in the U.K.