sign in sign up
<<
Home , Carl Friedrich Gauss, Gaussian elimination, Isaac Newton

Mathematicians of Gaussian Elimination Joseph F. Grcar

aussian elimination is universally known as “the” method for solving simultaneous linear . As remarked, it is the most natural way of proceeding (“der Gnatürlichste Weg” [Euler, 1771, part 2, sec. 1, chap. 4, art. 45]). Because Gaussian elimination solves linear problems directly, it is an important tech- nique in computational and , through which it makes continuing, albeit indi- rect, contributions to advancing knowledge and to human welfare. What is natural depends on the context, so the has changed many with the problems to be solved and with computing technology. Gaussian elimination illustrates a not often explained in histories of . are usually portrayed as “discover- ers”, as though is a curtain that gradually Bildarchiv Preussischer Kulturbresitz/Art Resource/NY. Photo by Olaf M. Tebmer. rises to reveal a static edifice which has been there all along awaiting discovery. Rather, Gaussian elim- Figure 1. It began in cuneiform tablets like ination is living mathematics. It has mutated suc- VAT 8389. Vorderasiatisches Museum in cessfully for the last two hundred years to meet , 12.1 by 7.9 cm. changing social needs. Many people have contributed to Gaussian elim- ination, including Carl Friedrich . His method von Neumann. There may be no other for calculating a special case was adopted by pro- that has been molded by so many renowned fessional hand computers in the nineteenth cen- mathematicians. tury. Confusion about the history eventually made Gauss not only the namesake but also the origina- Ancient Mathematics tor of the subject. We may write Gaussian elimina- Problems that can be interpreted as simultaneous tion to honor him without intending an attribution. linear equations are present, but not prevalent, in This article summarizes the evolution of Gauss- the earliest written mathematics. About a thou- ian elimination through the middle of the twentieth sand cuneiform tablets have been published that century [Grcar, 2011a,b]. The sole development record mathematics taught in the Tigris and Eu- in ancient times was in China. An independent phrates valley of Iran and Iraq in 2000 BC [Robson, origin in modern has had three phases. 2008, table B.22]. Most tablets contain tables First came the schoolbook lesson, beginning with of various kinds, but some tablets are didactic Isaac . Next were methods for profes- (Figure 1). The first problem on VAT 8389 asks for sional hand computers, which began with Gauss, the of two fields whose total is 1800 sar, who apparently was inspired by of Joseph- when the rent for one field is 2 silà of grain per 3 Louis Lagrange. Last was the interpretation in sar, the rent for the other is 1 silà per 2 sar, and the algebra by several authors, including John total rent on the first exceeds the other by 500 silà. If you do not remember these , then you Joseph F. Grcar’s email address is [email protected]. are not alone. The author of the tablet frequently

782 Notices of the AMS Volume 58, 6 reminds readers to “keep it in your head” in the literal translation by Høyrup [2002, pp. 77–82].1 For perspective, when problems on these tablets can be written as simultaneous equations, usu- ally one is nonlinear [Bashmakova and Smirnova, 2000, p. 3], which suggests that linear were not a large part of the Babylonian curriculum. Simultaneous linear equations are prominent in just one ancient source. The Jiuzhang Suanshu,or Nine Chapters of the Mathematical Art,isacollec- tion of problems that were composed in China over 2000 years ago [Shen et al., 1999] (Figure 2). The first of eighteen similar problems in chapter 8 asks for the grain yielded by sheaves of rice stalks from three grades of rice paddies. The combined yield is 39 dou of grain for 3 sheaves from top-grade pad- dies, 2 from medium-grade, and 1 from low-grade; and similarly for two more collections of sheaves. Figure 2. A depiction of Liu Hui, who said the Mathematicians in ancient China made elaborate Nine Chapters were already old when he wrote calculations using counting rods to represent num- explanations of them in the third century. bers, which they placed inside squares arranged in a rectangle called a counting table (an ancient spreadsheet essentially). The right column in the sum of all the numbers, from which the other un- following rectangle represents the first collection knowns can be found. The earliest surviving work of paddies in the problem. of ancient Hindu mathematics is the Aryabhat¯ .¯ı y a of Aryabhat¯ .a from the end of the fifth century. His 1 2 3 top linear problems are reminiscent of Diophantus but 2 3 2 medium more general, for any of unknowns. 3 1 1 low Problem 29 in chapter 2 is to find several num- 26 34 39 yield bers given their total less each number [Plofker, 2009, p. 134]. The immediate sources for European The solution was by Gaussian elimination. The algebra were Arabic texts. Al-Khwarizmi of Bagh- right column was paired with each of the other dad and his successors could solve the equivalent columns to remove their top numbers, etc. The of quadratic equations, but they do not appear to retained by multiplying each have considered simultaneous linear equations be- column in a pair by the number atop the other and yond the special cases of Diophantus. For example, then subtracting right from left. R¯ashid [1984, p. 39] cites a of linear equa- 3 tions that Woepcke [1853, p. 94, prob. 20] traces to a problem in the Arithmetica. 4 5 2 8 1 1 Schoolbook Elimination 39 24 39 Diophantus and Aryabhat¯ .asolvedwhatcanbein- Hart [2011, pp. 70–81] explains the entire calcula- terpreted as simultaneous linear equations with- tion. out the generality of the Nine Chapters. An obvious The Nine Chapters and similar, apparently de- prerequisite is technology able to express the prob- rivative, work in Asia in later centuries [Hart, 2011; lems of interest. The Nine Chapters had counting Libbrecht, 1973; Martzloff, 1997] are the only treat- tables, and Europe had symbolic algebra. Possess- ments of general linear problems until early mod- ing the expressive capability does not mean it is ern Europe. The main surviving work of Greek and used immediately. Some was needed to de- Roman algebra is the Arithmetica problem book velop the of equations [Heeffer, 2011], and by Diophantus, which is believed to be from the even then, of 107 algebras printed between 1550 third century. Problem 19 in chapter (or book) 1 and 1660 in the late Renaissance, only four books is to find four numbers such that the sum of any had simultaneous linear equations [Kloyda, 1938]. three exceeds the other by a given amount [Heath, The earliest example found by Kloyda was from 1910, p. 136]. Diophantus reduced the repetitive Jacques Peletier du Mans [1554]. He solved a prob- conditions to one with a new unknown, such as the lem of Girolamo Cardano to find the money held by three men when each man’s amount plus a 1Exercise: Solve the problem in your head by any means fraction of the others’ is given. Peletier first took and then consult Høyrup [2002, pp. 81–82] for the the approach of Cardano. This solution was by scribe’s method. Discuss. verbal reasoning in which the symbolic algebra is a

June/July 2011 Notices of the AMS 783 convenient shorthand. The discourse has variables for just two amounts, and it represents the third by the given formula of the other two. Peletier [p. 111] then re-solved the problem almost as we do, starting from three variables and equations and using just symbolic manipulation (restated here with modern symbols): 2R + A + B = 64 R + 3A + B = 84 R + A + 4B = 124 2R + 4A + 5B = 208 3A + 4B = 144 3R + 4A + 2B = 148 3R + 2A + 5B = 188 6R + 6A + 7B = 336 6R + 6A + 24B = 744 17B = 408 Figure 3. (1643–1727) closed a Peletier’s overlong calculation suggests that gap in algebra textbooks. 1689 portrait by removing unknowns systematically was a further . advance. That step was soon made by Jean Bor- rel, who wrote in as Johannes Buteo [1560, While Newton’s notes awaited publication, p. 190]. Borrel and the Nine Chapters both used Michel Rolle [1690, pp. 42ff.] also explained how the same double-multiply elimination process to solve simultaneous, specifically linear, equa- (restated with modern symbols): tions. He arranged the work in two columns with 3A + B + C = 42 strict of substitutions. We may speculate A + 4B + C = 32 that Rolle’s emphasis survived in the “method of A + B + 5C = 40 substitution” and that his “columne du retour” is remembered as “backward” substitution. Nev- 11B + 2C = 54 ertheless, Newton influenced later authors more 2B + 14C = 78 strongly than Rolle. 150C = 750 In the eighteenth century many textbooks ap- Lecturing on the algebra in Renaissance texts peared “all more closely resembling the algebra of became the job of Isaac Newton (Figure 3) upon Newton than those of earlier writers” [Macomber, his promotion to the Lucasian professorship. In 1923, p. 132]. Newton’s direct influence is marked 1669–1670 Newton wrote a note saying that he by his of words. He wrote “extermino” in intended to close a gap in the algebra textbooks: his Latin notes [Whiteside, 1968–1982, v. II, p. 401, “This bee omitted by all that have writ introduc- no. 63] that became “exterminate” in the English tions to this Art, yet I judge it very propper & edition and the texts. A prominent ex- necessary to make an introduction compleate” ample is the algebra of Thomas Simpson [1755, pp. [Whiteside, 1968–1982, v. II, p. 400, n. 62]. New- 63ff.]. He augmented Newton’s lessons for “the Ex- ton’s contribution lay unnoticed for many years termination of unknown ” with the rule until his notes were published in Latin in 1707 and of addition and/or subtraction ( then in English in 1720. Newton stated the recur- of equations). sive strategy for solving simultaneous equations Among many similar algebras, Sylvestre Lacroix whereby one equation is used to remove a (Figure 4) made an important contribution to the from the others. nomenclature. His polished textbooks presented the best material in a consistent style [Domingues, And you are to know, that by each Æquation 2008], which included a piquant name for each one unknown Quantity may be taken away, concept. Accordingly, Lacroix [1804, p. 114] wrote, and consequently, when there are as many “This operation, by which one of the unknowns Æquations and unknown Quantities, all at is removed, is called elimination” (Cette opéra- length may be reduc’d into one, in which tion, par laquelle on chasse une des inconnues, se there shall be only one Quantity unknown. nomme élimination). The first algebra printed in — Newton [1720, pp. 60–61] the United States was a translation by John Farrar Newton meant to solve any simultaneous algebraic of Harvard College [Lacroix, 1818]. As derivative equations. He included rules to remove one vari- texts were written, “this is called elimination” able from two equations which need not be lin- became a fixture of American algebras. ear: substitution (solve an equation for a variable Gaussian elimination for the purpose of school- and place the formula in the other) and equality-of- books was thus complete by the turn of the nine- values (solve in both and set the formulas equal). teenth century. It was truly schoolbook elimination,

784 Notices of the AMS Volume 58, Number 6 A is vectors column sized matrix a for first-order statistics, the to given minimum. the still for is conditions the erential ff di equations Gauss, Following “Normal- normal 84]. and name p. 214], 1822, p. [Gauss, 1809, gleichungen” “elimina- [Gauss, 73], vulgarem” p. tionem 1805, [Legendre, ordinares” “méthodes 2 of technology the computation. Gaussian hand with soon professional evolve that would method elimination least-squares the of “ordi- elimination. by schoolbook elimination”, said, “common or respectively methods” they nary which solved, equations”, be “normal the could sense Gauss’s of or “equations least-squares Legendre’s minimum” from the obtained Gauss ap- in were The once solutions 1986]. led dispute [Stigler, proximate which calculations priority his model, unfortunate disclosed is linear dwarf an general what to “lost” the stated called succinctly the to now Legendre of methods be- Then unstated . using the discrepancies. by the calculate the celebrity minimizing of a squares came by the equations of simultane- linear overdetermined, in ous unknowns squares inferences the statistical draw “car- for to least method modern a was of quarrés,” It rés”). moindres method des what the (“méthode invented named 5) Legendre (Figure Legendre Adrien-Marie [1809] Gauss when and [1805] arose equa- linear finally simultaneous solve tions to need societal A Elimination Professional readily algebra. provide symbolic to in developed exercises undertaken been had it because d’Angers. David called (1765–1843) it Lacroix Sylvestre 4. Figure June/July h utdprssae déutosd iiu”and minimum” du “d’équations are: phrases quoted The t min b élimination nmdr oaino ueia nlssrte than rather numerical of notation modern In ,t ow h i c he l i m i n a t i o nw a sa p p l i e d . J. G. Reinis, The Portrait Medallions of David D’Angers. x ! b − Ax 2011 ! .I m a g eo fa n1 8 4 1b a s - r e l i e fb y 2 ,a n dt h en o r m a le q u a t i o n sa r e b A and ffl ounrn n suitably and column full of 2 x uhwsteimportance the was Such ,t h el e a s t - s q u a r e sp r o b l e m A t Ax oie fteAMS the of Notices = ySefidDte edxnfrte1828 of the frontispiece for Bendixen Detlev Siegfried by Lithograph elimination”. “common replacing method, professional first (1777–1855) the Gauss devised Friedrich Carl 5. Figure n oo,i em fwihh osrce linear parameters: constructed fewer he successively which of of combinations terms in on, so and in to equivalently, notation coeffi- bracket form equations, numeric quadratic normal the the the represent in to cients used he which notation, unnamed an squares, chose Gauss of the sum of of quantities whose growing instance the The observed by primes). indicated each (as with model er diff while that model, linear the where as equations [1810, overdetermined Gauss the wrote [1759]. 22] Lagrange p. the of in squares form of canonical sum reformulated the reexpress He to problem calculations. the own his he passage described one in but 2004, justifications, statistical dealt with mostly [Dunnington, publications least-squares calculations numbers His 138]. prodigious p. million his a that involved estimated who A C B as isl a nicriil computer, incorrigible an was himself Gauss = = = n n n [an] [cn, [bn, ,q ,... . . r, q, p, [xy] ## # [xy, [xy, + + + 2 1 ] ] a a ap = 2 1 ## # + w,w ] ] p p [aa]p xy = = srnmsh Nachrichten Astronomische + + + r h nnw aaeesof parameters unknown the are [xy, [xy] # + ,w b b bq Ω ## # x + 1 ## as hnetne his extended then Gauss . q q − etc., etc. # ] ,... y + [bb, ,a ,c . . . c, b, a, n, + + + − [ax][ay] # Ω [ab]q c c cr [bx, minimized. be to is , + [aa] r h discrepancies the are ## # 1 x r r ]q [bb, ## 1 + + + y ][by, + + ## , ...... 1 + [bc, [cc, ] + r numbers are [ac]r = = = 1 ..., ] 1 2 w w w , ]r ]r ## # + + + ...... 785 These formulas complete the squares of successive parameters in the , leaving A2 B2 C2 = + + +···+[nn, µ], Ω [aa] [bb, 1] [cc, 2] where µ is the quantity of variables. Thus A = 0, B = 0, C = 0, … can be solved in reverse order to obtain values for p, q, r, . . . ,alsoinreverseorder, at which attains its minimum, [nn, µ]. SolvingΩ least-squares problems by Gauss’s method required the numbers [xy, k] for in- creasing indices k. Gauss halved the work of schoolbook elimination, because he needed x, y only in alphabetic order. More importantly, his notation discarded the equations of symbolic al-

gebra, thereby freeing computers to organize the Provided by the family ofCourtesy Adele also Chauvenet of Hanifin. the United States Naval . work more efficiently. When Gauss calculated, he Figure 6. William Chauvenet (1820–1870) and simply wrote down lists of numbers using his others taught Gauss’s computing methods. bracket notation to identify the values [Gauss, Portrait circa 1839 by Eunice Makepeace 1810, p. 25]. Towle. The subsequent evolution of Gaussian elimi- nation illustrates the meta-disciplinary of mathematics [Grcar, 2011c]. People such as Gauss with significant knowledge of fields besides math- feat was astounding, because in it re- ≈ 3 ematics were responsible for these developments. quires about 23000 n /3 arithmetic operations The advances do not superficially resemble either for n = 41. Doolittle dispensed with Gauss’s brack- schoolbook elimination or what is taught at uni- ets and instead identified the numbers by their versities today, but they are no less important, positions in tables. He replaced the divisions in because through them social needs were met. the bracket formulas by reciprocal multiplications. Professional elimination began to develop due Doolittle’s tables colocated values to help him use to the economic and military of cartogra- the product tables of August Leopold Crelle [1864]. phy. Gauss took government funds to map the An unrecognized but prescient feature of Doolit- German principality where he lived. Maps were tle’s work combined graphs with algebra to reduce drawn from networks of triangles using . He used the carto- that were measured by surveyors. The angles had graphic triangulation to suggest an ordering of to be adjusted to make the triangles consistent. To the overdetermined equations (equivalently, an that end, Gauss [1826] devised a method to find arrangement of coefficients in the normal equa- least-squares solutions of underdetermined equa- tions) to preserve zeroes in the work. The zeroes tions by a quadratic-form calculation similar obviated many of the 23000 operations. 3 The next innovations were made by the French to the overdetermined case. Once Friedrich military geodesist André-Louis Cholesky [Benoit, Bessel [1838] popularized Gauss’s approach, 1924] (Figure 8). He, too, addressed the nor- cartographic bureaus adopted the bracket no- mal equations of the underdetermined, - tation. Gauss’s calculations became part of the adjustment problem. Although Cholesky is re- mathematics curriculum for geodesists. membered for the square roots in his formulas,4 By whatever method of elimination is per- his innovation was to order the arithmetic steps formed, we shall necessarily arrive at the to exploit a feature of multiplying calculators. The same final values …; but when the number machines were -produced starting in 1890 of equations is considerable, the method of [Apokin, 2001], and Cholesky personally used a substitution, with Gauss’s convenient nota- Dactyle (Figure 9). A side effect of long multi- tion, is universally followed. plication is that the machines could internally —Chauvenet[1868,p.514](Figure6) 5 accumulate sums of products. Calculations thus The first innovations after Gauss came from arranged were quicker to perform because fewer Myrick Doolittle [1881] (Figure 7). He was a com- puter at the United States Coast Survey who could 4Cholesky originated a construction that is now inter- solve forty-one normal equations in a week. This preted as decomposing a symmetric and positive definite matrix into a product LLt ,whereL is a lower triangular 3For A of full row rank, these problems were matrix and t is transposition. 5 minAx=b !x!2,wherex are the angle adjustments The sums of products became known as dot-, inner-, or and Ax = b are the consistency conditions. The solu- -products of vectors after the adoption of matrix tion was given by correlate equations x = At u,where notation. Calculations employing sums were described as AAt u = b are again normal equations to be solved. “abbreviated” or “compact”.

786 Notices of the AMS Volume 58, Number 6 June/July Doolittle’s to compared numbers paper. on for recorded tables be Cholesky’s to had results intermediate 1895. (1875–1918), circa Cholesky André-Louis 8. Figure (1830–1911), 1862. Doolittle circa Hascall Myrick 7. Figure aetetasto osaitc.Frexample, on For insisted successfully statistics. employees government it to States that transition United Doolittle’s prominent the War. so World made was First the method least-squares after for use problems primary the became analysis Ldra,14]oc twsrivne ythe by reinvented [1945]. used Dwyer was Paul widely statistician it American become The once methods. did 1948] own [Laderman, method” its root develop to “square statis- begun [Brezinski, after had notice in tics appeared little it received because 2006] it was but method superior, Cholesky’s in 1989]. analyses [Fox, used regression econometrics department for The methods 1927]. least-squares Ezekiel, Agriculture and of from Department [Tolley the moved to Survey they Coast when the method Doolittle’s using ttsia nlssi h omo regression of form the in analysis Statistical

© École Polytechnique. Courtesy of Antiochiana, Antioch College. 2011 n qain edonly held equations O (n 3 ) . O (n 2 oie fteAMS the of Notices ) i n oiiedfiie h edt routinely to need The definite. modern positive symmet- In be and must equations. coefficients ric of normal matrix the for terms, only sufficed in Bros. Chateau Foncine-le-Haut. by 1905 from in and Brunsviga by made machines resold Dactyle Odhner. Willgodt entrepreneur Russian-Swedish the the of on design based pinwheel calculator Dactyle 9. Figure ounpout hc stentrlwyt cal- hand. to by way figures of natural columns column-by- the with is culate a which have product, They column Cracovians. the called in matrices form invent independently to 12) (Figure [1938a,b] Banachiewicz moti- Tadeusz computing vated Manual bookstore. in the campus presently are your to that University textbooks analysis Jagiellonian numerical the of observatory computers. electronic the programming for of calculations purpose organize help Eventually would decompositions. matrices matrix re- through trivially lated were pro- the elimination all liferating rep- that new showed the technology hand, by resentational compute to needed not were century matrices Although nineteenth 2008]. the 1977a,b, of 1975, [Hawkins, half the ma- in developed trices of modify had adoption authors to Several the algebra. algebra with matrix ended symbolic elimination using Gaussian of milieu The Interpretation Matrix machines 11). calculating (Figure of manufacturer publicized leading was a method by reorganized accumulate Crout’s products. to 10), of Technology, sums elimination (Figure of Gaussian [1941] schoolbook Institute Crout a Massachusetts Cholesky, Prescott the by of at work Unaware earlier engineering. the in developed equations linear gradually simultaneous of kinds more solve h ehd fGus oltl,adCholesky and Doolittle, Gauss, of methods The yCaoin n h hoyb matrices. by theory the made and are Cracovians computations in by the may, that It said …. be column , by row column by multiply prac- column to multiply than in to easier that is conceded it tice be however, must, It astronomical the from leads development This —Jensen[1944]

Photo by Freddy Haeghens, [email protected]. 787 788 in a fbidn p oe“so-called some up” building was University of method the way Doolittle’s “efficient of that an 14) showed Michigan (Figure of year [1944] same Dwyer to the In Paul [1938]: matrices”. al. “elementary by multipli- et cation through Frazer operations arithmetic by represent presen- suggested Jensen’s was of tation aspect Cholesky’s noteworthy and A method, the method. Cracovian product: the triangular Gauss’s brackets), a with calculation as (the express- algorithm” matrix “Gaussian to square a amounted ing equations normal of pictograms, solving 13) used Institut (Figure Geodætisk [1944] Danish Jensen by the algebra matrix people. Cayley’s The two Arthur 1933. in as realized early was as calculations represent Oakland, in manufactured was It 10ACT. model iue1.Cotsmto a efre with performed was method Crout’s 11. Figure iue1.Pect uadCot(1907–1984), Crout Durand Prescott 10. Figure = aaheizavctduigCaoin to Cracovians using advocated Banachiewicz Courtesy of John Wolff, ahnssc sti acatcalculator, Marchant this as such machines www.vicnet.net.au/~wolff. Courtesy of MIT Museum. oepaieta he loihsfor algorithms three that emphasize to , aiona nte13sad1940s. and 1930s the in California, oie fteAMS the of Notices ic 1936. circa opoewa o emn i not. did Neumann von what prove to through paper 6 Jensen’s found 16) ure by aided be to case. general yet the has in bound be established error to achieved. comparable observed been a is accurate, has elimination that Gaussian best Although the their remains definite, positive bound and When symmetric is available. matrix became the computers al- once mechanized of the gorithm errors be rounding would anticipated the they on what bounds establish to bra decompo- matrix the sition. introduced analysis part initial their The of elimination. Gaussian Golds- study and to Neumann tine von motivated machines of the efficacy programmable, the first over Concerns the computers. building electronic of col- process his the were and others in and 15) They Goldstine. (Figure Herman Neumann laborator von Cayleyan John of from terms in is, that the matrices. form, roughly are in modern Dwyer elimination the Gaussian and depict Jensen to earliest of The inter- Banachiewicz. papers of similar work coincident the no in found except pretation He matrices. triangular” sdCaoinagbat eciecomputing describe to algebra Cracovian used oeecs:Teedrsl fmc td s onttry not do is, study much of result end The exercise: No iue1.TduzBnciwc (1882–1954) Banachiewicz Tadeusz 12. Figure h etse otecmu oktr was bookstore campus the to step next The alge- matrix used Goldstine and Neumann Von inductive explicit, simple, process. a by forms inverses it their second] [and … matrices tricks: [triangular] two decomposes of it combination First, elimination the … the as interpret method therefore may We came interpretation matrix the for use deep A o emn n odtn [1947] Goldstine and Neumann von — ahmtclReviews Mathematical ehd.Poofo 1946. from Photo methods. A noapouto two of product a into Volume 6 onTd (Fig- Todd John . 58 Number ,

MR Courtesy of Adam Strzałkowski. and 6 icpieta a tfis ouae ythose June/July by populated first at was that discipline a clear made wrote them. that overworking expression Turing without of of matrices. brevity manner a elementary with the with in case and general Jensen elimination the Neu- treating schoolbook von described by of of Goldstine He manner and the cited. mann in [1948] elimination Gold- and Turing Gaussian Neumann Todd von whom to through visit stine, a matrix Jensen during the also Turing from and of Alan learned both evidently Phys- was interpretation who National them 17), the Among (Figure at Laboratory. the staff communicated ical to auditor [Taussky formalism An London matrix 2006]. College Todd, King’s at and it on lectured 1960s. Photo circa algebra. matrix used (1901–1982) Dwyer independently Sumner Paul 14. Figure Cayleyan use algebra. to Banachiewicz by was (1915–1974) inspired Jensen Henry 13. Figure h neto feetoi optr created computers electronic of invention The

Courtesy of Prof. Ingram Olkin, Dept. of Statistics, Stanford University. Courtesy of the University of Copenhagen (Andersen 1975). 2011 oie fteAMS the of Notices tecmiaino w rcs.Poofrom 1947. Photo March tricks”. two of saw combination (1903–1957) “the Neumann von John 15. Figure uaigmcie,teaihei tp a be can steps arithmetic cal- the for products machines, of culating sums Cholesky emphasized While Crout 1983]. and Reid, parallel and with ff [Du 1970] computers of [Irons, were equations analysis the use methods solve finite-element to for other recently more method tables, developed a developed calculations with Doolittle least-squares then pro- for first the and the and method developed problem Gauss fessional the as to Just cal- adapted the computer. the well whether because is on way, culation depends al- computing straightforward of an a becomes in seldom gorithm elimination Gaussian interpretation matrix of The problem. mathematical An Coda they textbooks then presentation. Turing’s interpretation, reprised analysis matrix [1967] the featured Moler numerical that and influential 1960s. Forsythe the and wrote until [1964] taught Fox algebra commonly When Linear not was slowly. itself more descriptions matrix decade. a within used widely was tab. name 2011a, The first [Grcar, 1]. the elimination” “Gaussian been it have call to to appears and Society, Gauss elimi- to Mathematical nation school” “high American misattributed [1953] an the Forsythe In “elimination”. to for simply address named origin was an then suggested The what Gauss describe discipline. of to calculations new used the the had geodesists of science” that terminology “computer subject credence the lent named to involvement who have Gauss’s mathematician 1972]. to [Knuth, visionary reputed a is Forsythe was 1972; George 18) [Traub, them, (Figure Among calculations 1970]. scientific Wilkinson, made who h nvriymteaiscriuu adopted curriculum mathematics university The algorithm sasre fsesfrsliga solving for steps of a is

Courtesy of the Los Alamos National Laboratory Archives. 789 790 oerdcltasomtosaepsil that possible solve to are work the transformations reduce com- 1998]. radical erent di ff Dongarra, More and suit [Whaley architectures to puter automatically reordered ae ahrta en ltncarchetype, Platonic technique. a their evolving an being is is elimination than algorithms Gaussian the future Rather Perhaps about name. 2007]. al., certainty et only and Demmel Cohn 2003; 1969; Umans, [Strassen, operations arithmetic aobr[93,adWieie[1968–1982] Whiteside and [1938], for pleasure [1923], Kloyda a referees of is Macomber scholarship the It the and article. acknowledge editor this to improved the that to advice grateful am I Acknowledgments iue1.Jh od(9120)lcue on lectured (1911–2007) Todd John 16. Figure iue1.Aa uig(9215)described (1912–1954) Turing Alan 17. Figure

Courtesy of the Royal Society Archives, Courtesy of the Archives, California Institute of Technology.

ti h a nvriisnwtahit. teach now universities way the in it © Godfrey Argent Studio. n htgahcra1951. circa Photograph esnswr n1946. in work Jensen’s qain below equations oie fteAMS the of Notices O (n 3 ) .D Crout D. P. Crelle L. A. Cohn H. Chauvenet W. Buteo J. Brezinski C. Bessel W. F. Benoit Bashmakova G. I. Banachiewicz T. Apokin A. I. Andersen E. to possible References it made which contribution. 21), Newton’s identify 20, 19, (Figures iue1.Gog oste(9717)called (1917–1972) Forsythe George 18. Figure n ovn ytm flna qain …, equations Engrs. linear Elect. Inst. of Amer. systems solving and 1864. edition, 2nd ersparen Berlin, ganz Reimer, Tausend unter Zahlen mit Dividiren Sci. 2003. Comput. IEEE, of Foundations in on Symp. multiplication, matrix fast to 1868. Philadelphia, Co., & Lippincott B. J. Algorithms Numerical … Ostpreussen Commandant as only identified is Benoit. author The 1924, Cholesky), dant 2000. DC, Washington, Algebra Amer., Assoc. of Evolution and Math. Sc. …, linéaires Math. Sc. A. Série Polonaise, carrés, moindres des méthode la and Nitussov, Y. 2001. A. Braunschweig/Wiesbaden, Vieweg, Trogemann, editors, G. Ernst, in W. ifmometr”, 1975. Universitet, in 1974, éhd erslto uéiu e équations des numérique résolution de Méthode , oesruemtoe…(rcd uComman- du (Procédé … méthode une sur Note , , Logistica and ,pages393–404,1938b. ro 1973–1974 Ârbog hr ehdfreautn evaluating for method short A , , er esn coe 951,August 1915–10, October 7 Jensen, Henry , ilotTepi dnradhs“Ar- his and Odhner Theophil Willgodt , h ieadwr fAdéCholesky, André of work and life The , ehnaenwlh le utpiie und Multipliciren alles welche Rechentafeln .Umans C. ul nen elAa.Plnie éi A. Série Polonaise, l’Acad. de Intern. Bull. , and raieo h ehdo es Squares Least of Method the on öi.Aa.Ws. eln 1838. Berlin, Wiss., Akad. König. , rnie ’n ovletcnqede technique nouvelle d’une Principes , ,Paris,1560. and ultngéodésique Bulletin optn nRussia in Computing .J Baeyer J. J. .S Smirnova S. G. 33:7–8,2006. 43(3):279–288, , ru-hoei approach -theoretic A , ,60:1235–1241,1941. t“asinelimination”. “Gaussian it ae 24.Københavns 42–45. pages , Volume T.A hnte) Math. Shenitzer), A. (Tr. ,pages134–135,1938a. , ul nen el’Acad. de Intern. Bull. rc 4hAn IEEE Ann. 44th Proc. rdesn in Gradmessung April (2):67–77, , , ,pages438–449, 58 h Beginnings The ,pages39–46. Number , Trans. ,G. Josè Mercado/Stanford News Service. 6 , June/July 1920. yearbook, California, School Redding, High County primary Shasta the sources. examine to neglected who academicians later prominent more and bested textbooks American to historical antecedents of She importance record”. the on anticipated method this of earliest appearance “the was equations solving simultaneous for lesson Newton’s [1923] that Macomber observed Louise Gertrude 20. Figure . electronic the of before age assembled being more for the impressive all is that of sources collection primary unrivaled an is by dissertation supported Her 2009]. LaDuke, and mathematics [Green American of a woman was pioneering She texts. Renaissance in few examples the Kempis discovered à (1896–1977) Thomas Kloyda Mary Sister 19. Figure

Courtesy of the Shasta County Historical Society. Courtesy of Sisters of Saint Francis, Rochester, Minnesota. 2011 oie fteAMS the of Notices olcino i papers. his monumental of a collection in wrote Newton what remembered 2008] [Guicciardini, Whiteside (1932–2008) Thomas Derek 21. Figure .A rzr .J Duncan J. W. Frazer, A. R. Fox L. Fox A. K. Forsythe E. G. Forsythe E. G. Euler L. Dwyer S. P. Dunnington W. G. Duff S. I. Doolittle H. M. Domingues C. J. Dumitriu I. Demmel, J. abig,1938. , and Equations to erential Diff Applications Some and Matrices tary 1964. Oxford, Press, Clarendon 1989. (1):53–70, Revolution, 1967. Jersey, New Cliffs, Systems Algebraic Linear of 1953. interesting, be 1945. regression, and 1944. 82–89, im- solution, of of methods justification proved matrix with theory correlation and 2004. ence Software Math. systems, linear symmetric sparse definite 1881. DC, 1878 Washington, June, ffice, O Printing with Government Ending 115–120. … Survey Geodetic triangu- and a of adjustment in the larization, in and equations normal 2008. Basel, stable, is algebra ah so.Ae. ahntn C n edition, 2nd DC, Washington, Amer., Assoc. Math. , , h qaero ehdadisuei correlation in use its and method root square The , , nItouto oNmrclLna Algebra Linear Numerical to Introduction An giutrlEooit nteEconometric the in Economists Agricultural , and netn u Algebra zur Anleitung arxpeetto flatsquares least of presentation matrix A , .K Reid K. J. xodEooi aes e Series New Papers, Economic Oxford eoto h ueitneto h Coast the of Superintendent the of Report ovn ieragbaceutoscan equations algebraic linear Solving , , ()3235 1983. 9(3):302–325, , and ehdepoe nteslto of solution the in employed Method , ul mr ah Soc. Math. Amer. Bull. ari n h the and Lacroix .Ae.Sait Assoc. Statist. Amer. J. ue.Math. Numer. , alFidihGus ia fSci- of Gauss: Friedrich Carl .B Moler B. C. h utfotlslto fin- of solution multifrontal The , ,C a m b r i d g eU n i v e r s i t yP r e s s , ,and ,and rnieHl,Englewood Prentice-Hall, , n.Mt.Statist. Math. Ann. .Holtz O. ud 1771. Lund, , 0()5–1 2007. 108(1):59–91, , .R Collar R. A. , optrSolution Computer 59(4):299–329, , ,Fastlinear ,40:493–503, ,Birkhäuser, C Trans. ACM , Courtesy of Dr. Piers Bursill-Hall, Cambridge University. Elemen- ,pages 15(1): , ,41 , 791 C. F. Gauss, Theoria Motus Corporum Coelestium in J. Laderman, The square root method for solving simul- Sectionibus Conicis Solum Ambientium,Perthesand taneous linear equations, Math. Tables Aids Comp.,3 Besser, Hamburg, 1809. (21):13–16, 1948. , Disquisitio de elementis ellipticis Palladis, Com- J. L. Lagrange, Recherches sur la méthode de maximis et mentationes Societatis Regiae Scientiarum Gottingensis minimis, Miscellanea Taurinensia,1,1759.Reprinted recentiores: Commentationes classis mathematicae,1 in Œuvres de Lagrange, (J.-A. Serret, editor), vol. 1, (1808–1811):1–26, 1810. pages 1–16, Gauthier–Villars, Paris, 1867. ,AnwendungderWahrscheinlichkeitsrech-A. M. Legendre, Nouvelle méthodes pour la détermi- nung auf eine Aufgabe der practischen Geometrie, nation des orbites des comètes, Chez Didot, Paris, Astronomische Nachrichten, 1(6):81–86, January 1822. 1805. , Supplementum theoriae combinationis observa- U. Libbrecht, in the Thir- tionum erroribus minimis obnoxiae, Commentationes teenth Century,MITPress,Cambridge,1973. Societatis Regiae Scientiarum Gottingensis recentiores: G. L. Macomber, The influence of the English and French Commentationes classis mathematicae, 6 (1823–1827): writers of the sixteenth, seventeenth, and eighteenth 57–98, 1826. centuries on the teaching of algebra, M.A. thesis, J. F. Grcar, How ordinary elimination became Gaussian University of California, Berkeley, 1923. elimination, Historia Math. 38(2):163Ð218, 201a. doi: J.-C. Martzloff, AHistoryofChineseMathematics(Tr. 10.1016/j.hm.2010.06.003. S. S. Wilson), Springer, Berlin, 1997. , John von Neumann’s Analysis of Gaussian elim- I. Newton, Arithmetick, Senex, Taylor et al., ination and the origins of modern , London, 1720. SIAM Rev.,toappear,2011b. J. Peletier du Mans, L’Algebre, Lyon, 1554. , Mathematics turned inside out: The intensive K. Plofker, Mathematics in India, Princeton Univ. Press, faculty versus the extensive faculty, Higher Education, 2009. 61(6), 2011c. doi: 10.1007/s10734-010-9358-y. R. R¯ashid, Entre arithmétique et algèbre: Recherches sur J. Green and J. LaDuke, Pioneering Women in American l’histoire des mathématiques arabes, Société d’Édition Mathematics: The Pre-1940 Ph.D.’s, Amer. Math. Soc., , Paris, 1984. Providence, 2009. J. G. Reinis, The Portrait Medallions of David d’Angers, N. Guicciardini, Derek Thomas Whiteside (1932–2008), Polymath Press, New York, 1999. Historia Math., 36(1):4–9, 2008. E. Robson, Mathematics in Ancient Iraq: A Social History, R. Hart, The Chinese Roots of , Johns Princeton Univ. Press, 2008. Hopkins University Press, Baltimore, 2011. M. Rolle, Traité d’algèbre,E.Michallet,Paris,1690. T. Hawkins, Cauchy and the spectral theory of matrices, K. Shen, J. N. Crossley,andA. W.-C. Lun, The Nine Historia Math.,2:1–29,1975. Chapters of the Mathematical Art Companion and , Another look at Cayley and the theory of matri- Commentary, Oxford Univ. Press, New York, 1999. ces, Archives Internationales d’Histoire des ,27 T. Simpson, A Treatise of Algebra, John Nourse, London, (100):82–112, 1977a. 2nd edition, 1755. , Weierstrass and the theory of matrices, Arch. S. M. Stigler, The : The Hist. Exact Sci., 17(2):119–164, 1977b. of Uncertainty before 1900,HarvardUniversityPress, , Frobenius and the symbolical algebra of Cambridge, 1986. matrices, Arch. Hist. Exact Sci., 62(1):23–57, 2008. V. Strassen, Gaussian elimination is not optimal, T. L. Heath, Diophantus of Alexandria,Cambridge Numer. Math., 13(4):354–356, 1969. University Press, Cambridge, 2nd edition, 1910. O. Taussky and J. Todd, Cholesky, Toeplitz and A. Heeffer, From the second unknown to the sym- the triangular of symmetric matrices, bolic equation, in A. Heeffer and M. Van Dyck, Numerical Algorithms,41:197–202,2006. editors, Philosophical Aspects of Symbolic Reasoning H. R. Tolley and M. Ezekiel, The Doolittle Method …, J. in Early Modern Mathematics, pages 57–101, College Amer. Statist. Assoc., 22(160):497–500, 1927. Publications, London, 2011. J. F. Traub, Numerical mathematics and computer J. Høyrup, Lengths, Widths, Surfaces: A Portrait of Old science, Comm. ACM, 15(7):537–541, 1972. Babylonian Algebra and Its Kin, Springer-Verlag, New A. M. Turing, Rounding-offerrors in matrix processes, York, 2002. Quart. J. Mech. Appl. Math., 1(3):287–308, 1948. B. M. Irons, A frontal solution program for fi- J. von Neumann and H. H. Goldstine,Numericalin- nite element analysis, Int. J. Numer. Methods Eng., verting of matrices of high order, Bull. Amer. Math. 2(1):5–32, 1970. Soc., 53(11):1021–1099, 1947. H. Jensen, An attempt at a systematic classification of C. Whaley and J. Dongarra, Automatically tuned lin- some methods for the solution of normal equations, ear algebra software, in Proc. 1998 ACM/IEEE SC98 Meddelelse 18, Geodætisk Institut, Copenhagen, 1944. Conference,pages1–27,IEEE,1998. M. Thomas à Kempis Kloyda, Linear and Quadratic D. T. Whiteside, editor, The Mathematical Papers of Equations 1550–1660, Edwards Brothers, Ann Arbor, Isaac Newton, 8 vols., Cambridge University Press, 1938, University of Michigan dissertation. 1968–1982. D. E. Knuth, George Forsythe and the Development of J. H. Wilkinson, Some comments from a numerical Computer Science, Comm. ACM, 15(8):721–726, 1972. analyst, J. ACM, 18(2):137–147, 1970. S. F. Lacroix, Elements of Algebra (Tr. J. Farrar), F. Woepcke, Extrait du Fakhrî,Paris,1853. University Press, Cambridge, Massachusetts, 1818. , Elemens d’algèbre, Chez Courcier, Paris, 5th edition, 1804.

792 Notices of the AMS Volume 58, Number 6