The Definition of Numerical Analysis

The following essay app eared in the Novemb er issue of SIAM News and the March issue of the Bul letin of the Institute for Mathematics and Applications THE DEFINITION OF NUMERICAL ANALYSIS Lloyd N Trefethen Dept of Computer Science Cornell Unviersity LNTcscornelledu What is numerical analysis I b elieve that this is more than a philosophical question A certain wrong answer has taken hold among b oth outsiders to the eld and insiders distorting the image of a sub ject at the heart of the mathematical sciences Here is the wrong answer Numerical analysis is the study of rounding errors D The reader will agree that it would b e hard to devise a more uninviting description of a eld Rounding errors are inevitable yes but they are complicated and tedious and not fundamental If D is a common p erception it is hardly surprising that numerical analysis is widely regarded as an unglamorous sub ject In fact mathematicians physicists and computer scientists have all tended to hold numerical analysis in low esteem for manyyearsa most unusual consensus Of course nob o dy b elieves or asserts D quite as baldly as written But consider the following op ening chapter headings from some standard numerical analysis texts Isaacson Keller Norms arithmetic and wellp osed computations Hamming Roundo and function evaluation Dahlquist Bjorck Some general principles of numerical calculation How to obtain and estimate accuracy Sto er Bulirsch Error analysis Conte de Bo or Numb er systems and errors Atkinson Error its sources propagation and analysis Kahaner Moler Nash Intro duction Computer arithmetic and computational errors Error roundo computer arithmetic these are the words that keep reapp earing What impression do es an inquisitive college student get up on op ening such b o oks Or consider the denitions of numerical analysis in some dictionaries Websters New Collegiate Dictionary The study of quantitative approxi mations to the solutions of mathematical problems including consideration of the errors and b ounds to the errors involved Chamb ers th Century Dictionary The study of metho ds of approximation and their accuracy etc The American Heritage Dictionary The study of approximate solutions to mathematical problems taking into account the extent of p ossible errors Approximations accuracy errors again It seems to me that these denitions would serve most eectively to deter the curious from investigating further The singular value decomp osition SVD aords another example of the p erception of nu merical analysis as the science of rounding errors Although the ro ots of the SVD go back more than years it is mainly since the s through the work of Gene Golub and other numerical analysts that it has achieved its present degree of prominence The SVD is as fundamental an idea as the eigenvalue decomp osition it is the natural language for discussing all kinds of questions of norms and extrema involving nonsymmetric matrices or op erators Yet to daythirtyyears later most mathematical scientists and even many applied mathe maticians do not havea working knowledge of the SVD Most of them have heard of it but the impression seems to b e widespread that the SVD is just a to ol for combating rounding errors A glance at a few numerical analysis textb o oks suggests why In one case after an other the SVD is buried deep in the b o ok typically in an advanced section on rankdecient leastsquares problems and recommended mainly for its stability prop erties I am convinced that consciously or unconsciouslymany p eople think that D is at least half true In actualityitisavery small part of the truth And although there are historical explanations for the inuence of D in the past it is a less appropriate denition to day and is destined to b ecome still less appropriate in the future I prop ose the following alternative denition with whichtoenter the new century Numerical analysis is the study of algorithms D for the problems of continuous mathematics Boundaries b etween elds are always fuzzy no denition can b e p erfect But it seems to me that D is as sharp a characterization as you could come up with for most disciplines The pivotal word is algorithms Where was this wordinthosechapter headings and dictionary denitions Hidden b etween the lines at b est and yet surely this is the center of numerical analysis devising and analyzing algorithms to solve a certain class of problems These are the problems of continuous mathematics Continuous means that real or complex variables are involved its opp osite is discrete A dozen qualications aside numerical ana lysts are broadly concerned with continuous problems while algorithms for discrete problems are the concern of other computer scientists Let us consider the implications of D First of all it is clear that since real and complex numb ers cannot b e represented exactly on computers D implies that part of the business of numerical analysis must b e to approximate them This is where the rounding errors come in Now for a certain set of problems namely the ones that are solved by algorithms that take anitenumb er of steps that is all there is to it The premier example is Gaussian elimination for solving a linear system of equations Ax bTo understand Gaussian elimination you have to understand computer science issues such as op eration counts and machine architectures and you have to understand the propagation of rounding errorsstability Thats all you have to understand and if someb o dy claims that D is just a more p olite restatementof D you cant prove him or her wrong with the example of Gaussian elimination But most problems of continuous mathematics cannot be solved by nite algorithms Unlike Ax b and unlike the discrete problems of computer science most of the problems of numer ical analysis could not b e solved exactly even if we could work in exact arithmetic Numerical analysts know this and mention it along with a few words ab out Ab el and Galois when they teach algorithms for computing matrix eigenvalues To o often they forget to mention that the same conclusion extends to virtually any problem with a nonlinear term or a derivative in itzeronding quadrature dierential equations integral equations optimization you name it Even if rounding errors vanished numerical analysis would remain Approximating mere numb ers the task of oatingp oint arithmetic is indeed a rather small topic and mayb e even a tedious one The deep er business of numerical analysis is approximating unknowns not knowns Rapid convergence of approximations is the aim and the pride of our eld is that for many problems wehaveinvented algorithms that converge exceedingly fast These p oints are sometimes overlo oked byenthusiasts of symb olic computing esp ecially recent converts who are apt to think that the existence of Maple or Mathematica renders Matlab and Fortran obsolete It is true that rounding errors can b e made to vanish in the sense that in principle any nite sequence of algebraic op erations can b e represented exactly on a computer by means of appropriate symb olic op erations Unless the problem b eing solved is a nite one however this only defers the inevitable approximations to the end of the calculation by whichpoint the quantities one is working with mayhave b ecome extraordinarily cumb ersome Floatingp oint arithmetic is a name for numerical analysts habit of doing their pruning at every step along the way of a calculation rather than in a single act at the end Whichever way one pro ceeds in oatingp ointorsymb olically the main problem of nding a rapidly convergent algorithm is the same In summary it is a corollary of D that numerical analysis is concerned with rounding errors and also with the deep er kinds of errors asso ciated with convergence of approximations which go byvarious names truncation discretization iteration Of course one could cho ose to make D more explicit by adding words to describ e these approximations and errors But once words b egin to b e added it is hard to know where to stop for D also fails to mention some other imp ortant matters that these algorithms are implemented on computers whose architecture may b e an imp ortant part of the problem that reliability and eciency are paramount goals that some numerical analysts write programs and others prove theorems and most imp ortant that all of this work is applied applied daily and successfully to thousands of applications on millions of computers around the world The problems of continuous mathematics are the problems that science and engineering are built up on without numerical metho ds science and engineering as practiced to daywould come quickly to a halt They are also the problems that preo ccupied most mathematicians from the time of Newton to the twentieth century As muchasany pure mathematicians numerical analysts are the heirs to the great tradition of Euler Lagrange Gauss and the rest If Euler were alivetodayhe wouldnt b e proving existence theorems Ten years ago I would have stopp ed at this p oint But the evolution of computing in the past decade has given the dierence b etween D and D a new topicality Let us return to Ax bMuchofnumerical computation dep ends on linear algebra and this highly develop ed sub ject has b een the core of numerical analysis since the b eginning Nu merical linear algebra served as the sub ject with resp ect to which the now standard concepts of stability conditioning and backward error analysis were dened and sharp ened and the central gure in these developments from the s to his

Load more