Mathematicians of Gaussian Elimination Joseph F

Mathematicians of Gaussian Elimination Joseph F. Grcar

aussian elimination is universally known as “the” method for solving simultaneous linear equations. As Leonhard Euler remarked, it is the most natural way of proceeding (“der Gnatürlichste Weg” [Euler, 1771, part 2, sec. 1, chap. 4, art. 45]). Because Gaussian elimination solves linear problems directly, it is an important technique in computational science and engineering, through which it makes continuing, albeit indi- rect, contributions to advancing knowledge and to human welfare. What is natural depends on the context, so the algorithm has changed many times with the problems to be solved and with computing technology. Gaussian elimination illustrates a phenomenon not often explained in histories of mathematics. Mathematicians are usually portrayed as “discover- ers”, as though progress is a curtain that gradually Bildarchiv Preussischer Kulturbresitz/Art Resource/NY. Photo by Olaf M. Tebmer. rises to reveal a static edifice which has been there all along awaiting discovery. Rather, Gaussian elim- Figure 1. It began in cuneiform tablets like ination is living mathematics. It has mutated suc- VAT 8389. Vorderasiatisches Museum in cessfully for the last two hundred years to meet Berlin, 12.1 by 7.9 cm. changing social needs. Many people have contributed to Gaussian elimination, including Carl Friedrich Gauss. His method von Neumann. There may be no other subject for calculating a special case was adopted by pro- that has been molded by so many renowned fessional hand computers in the nineteenth cen- mathematicians. tury. Confusion about the history eventually made Gauss not only the namesake but also the origina- Ancient Mathematics tor of the subject. We may write Gaussian elimina- Problems that can be interpreted as simultaneous tion to honor him without intending an attribution. linear equations are present, but not prevalent, in This article summarizes the evolution of Gauss- the earliest written mathematics. About a thou- ian elimination through the middle of the twentieth sand cuneiform tablets have been published that century [Grcar, 2011a,b]. The sole development record mathematics taught in the Tigris and Eu- in ancient times was in China. An independent phrates valley of Iran and Iraq in 2000 BC [Robson, origin in modern Europe has had three phases. 2008, table B.22]. Most tablets contain tables First came the schoolbook lesson, beginning with of various kinds, but some tablets are didactic Isaac Newton. Next were methods for profes- (Figure 1). The first problem on VAT 8389 asks for sional hand computers, which began with Gauss, the areas of two fields whose total area is 1800 sar, who apparently was inspired by work of Joseph- when the rent for one field is 2 silà of grain per 3 Louis Lagrange. Last was the interpretation in sar, the rent for the other is 1 silà per 2 sar, and the matrix algebra by several authors, including John total rent on the first exceeds the other by 500 silà. If you do not remember these numbers, then you Joseph F. Grcar’s email address is [email protected]. are not alone. The author of the tablet frequently

782 Notices of the AMS Volume 58, Number 6 reminds readers to “keep it in your head” in the literal translation by Høyrup [2002, pp. 77–82].1 For perspective, when problems on these tablets can be written as simultaneous equations, usually one equation is nonlinear [Bashmakova and Smirnova, 2000, p. 3], which suggests that linear systems were not a large part of the Babylonian curriculum. Simultaneous linear equations are prominent in just one ancient source. The Jiuzhang Suanshu, or Nine Chapters of the Mathematical Art, is a collection of problems that were composed in China over 2000 years ago [Shen et al., 1999] (Figure 2). The first of eighteen similar problems in chapter 8 asks for the grain yielded by sheaves of rice stalks from three grades of rice paddies. The combined yield is 39 dou of grain for 3 sheaves from top-grade paddies, 2 from medium-grade, and 1 from low-grade; and similarly for two more collections of sheaves. Figure 2. A depiction of Liu Hui, who said the Mathematicians in ancient China made elaborate Nine Chapters were already old when he wrote calculations using counting rods to represent num- explanations of them in the third century. bers, which they placed inside squares arranged in a rectangle called a counting table (an ancient spreadsheet essentially). The right column in the sum of all the numbers, from which the other un- following rectangle represents the first collection knowns can be found. The earliest surviving work of paddies in the problem. of ancient Hindu mathematics is the Aryabhat¯ .¯ıya of Aryabhat¯ .a from the end of the fifth century. His 1 2 3 top linear problems are reminiscent of Diophantus but 2 3 2 medium more general, being for any quantity of unknowns. 3 1 1 low Problem 29 in chapter 2 is to find several num- 26 34 39 yield bers given their total less each number [Plofker, 2009, p. 134]. The immediate sources for European The solution was by Gaussian elimination. The algebra were Arabic texts. Al-Khwarizmi of Bagh- right column was paired with each of the other dad and his successors could solve the equivalent columns to remove their top numbers, etc. The of quadratic equations, but they do not appear to arithmetic retained integers by multiplying each have considered simultaneous linear equations be- column in a pair by the number atop the other and yond the special cases of Diophantus. For example, then subtracting right from left. R¯ashid [1984, p. 39] cites a system of linear equa- 3 tions that Woepcke [1853, p. 94, prob. 20] traces to a problem in the Arithmetica. 4 5 2 8 1 1 Schoolbook Elimination 39 24 39 Diophantus and Aryabhat¯ .a solved what can be in- Hart [2011, pp. 70–81] explains the entire calcula- terpreted as simultaneous linear equations with- tion. out the generality of the Nine Chapters. An obvious The Nine Chapters and similar, apparently de- prerequisite is technology able to express the prob- rivative, work in Asia in later centuries [Hart, 2011; lems of interest. The Nine Chapters had counting Libbrecht, 1973; Martzloff, 1997] are the only treat- tables, and Europe had symbolic algebra. Possess- ments of general linear problems until early mod- ing the expressive capability does not mean it is ern Europe. The main surviving work of Greek and used immediately. Some time was needed to de- Roman algebra is the Arithmetica problem book velop the concept of equations [Heeffer, 2011], and by Diophantus, which is believed to be from the even then, of 107 algebras printed between 1550 third century. Problem 19 in chapter (or book) 1 and 1660 in the late Renaissance, only four books is to find four numbers such that the sum of any had simultaneous linear equations [Kloyda, 1938]. three exceeds the other by a given amount [Heath, The earliest example found by Kloyda was from 1910, p. 136]. Diophantus reduced the repetitive Jacques Peletier du Mans [1554]. He solved a prob- conditions to one with a new unknown, such as the lem of Girolamo Cardano to find the money held by three men when each man’s amount plus a 1Exercise: Solve the problem in your head by any means fraction of the others’ is given. Peletier first took and then consult Høyrup [2002, pp. 81–82] for the the approach of Cardano. This solution was by scribe’s method. Discuss. verbal reasoning in which the symbolic algebra is a

June/July 2011 Notices of the AMS 783 convenient shorthand. The discourse has variables for just two amounts, and it represents the third by the given formula of the other two. Peletier [p. 111] then re-solved the problem almost as we do, starting from three variables and equations and using just symbolic manipulation (restated here with modern symbols): 2R + A + B = 64 R + 3A + B = 84 R + A + 4B = 124 2R + 4A + 5B = 208 3A + 4B = 144 3R + 4A + 2B = 148 3R + 2A + 5B = 188 6R + 6A + 7B = 336 6R + 6A + 24B = 744 17B = 408 Figure 3. Isaac Newton (1643–1727) closed a Peletier’s overlong calculation suggests that gap in algebra textbooks. 1689 portrait by removing unknowns systematically was a further Godfrey Kneller. advance. That step was soon made by Jean Bor- rel, who wrote in Latin as Johannes Buteo [1560, While Newton’s notes awaited publication, p. 190]. Borrel and the Nine Chapters both used Michel Rolle [1690, pp. 42ff.] also explained how the same double-multiply elimination process to solve simultaneous, specifically linear, equa- (restated with modern symbols): tions. He arranged the work in two columns with 3A + B + C = 42 strict patterns of substitutions. We may speculate A + 4B + C = 32 that Rolle’s emphasis survived in the “method of A + B + 5C = 40 substitution” and that his “columne du retour” is remembered as “backward” substitution. Nev- 11B + 2C = 54 ertheless, Newton influenced later authors more 2B + 14C = 78 strongly than Rolle. 150C = 750 In the eighteenth century many textbooks ap- Lecturing on the algebra in Renaissance texts peared “all more closely resembling the algebra of became the job of Isaac Newton (Figure 3) upon Newton than those of earlier writers” [Macomber, his promotion to the Lucasian professorship. In 1923, p. 132]. Newton’s direct influence is marked 1669–1670 Newton wrote a note saying that he by his choice of words. He wrote “extermino” in intended to close a gap in the algebra textbooks: his Latin notes [Whiteside, 1968–1982, v. II, p. 401, “This bee omitted by all that have writ introduc- no. 63] that became “exterminate” in the English tions to this Art, yet I judge it very propper & edition and the derivative texts. A prominent ex- necessary to make an introduction compleate” ample is the algebra of Thomas Simpson [1755, pp. [Whiteside, 1968–1982, v. II, p. 400, n. 62]. New- 63ff.]. He augmented Newton’s lessons for “the Ex- ton’s contribution lay unnoticed for many years termination of unknown quantities” with the rule until his notes were published in Latin in 1707 and of addition and/or subtraction (linear combination then in English in 1720. Newton stated the recur- of equations). sive strategy for solving simultaneous equations Among many similar algebras, Sylvestre Lacroix whereby one equation is used to remove a variable (Figure 4) made an important contribution to the from the others. nomenclature. His polished textbooks presented the best material in a consistent style [Domingues, And you are to know, that by each Æquation 2008], which included a piquant name for each one unknown Quantity may be taken away, concept. Accordingly, Lacroix [1804, p. 114] wrote, and consequently, when there are as many “This operation, by which one of the unknowns Æquations and unknown Quantities, all at is removed, is called elimination” (Cette opéra- length may be reduc’d into one, in which tion, par laquelle on chasse une des inconnues, se there shall be only one Quantity unknown. nomme élimination). The first algebra printed in — Newton [1720, pp. 60–61] the United States was a translation by John Farrar Newton meant to solve any simultaneous algebraic of Harvard College [Lacroix, 1818]. As derivative equations. He included rules to remove one vari- texts were written, “this is called elimination” able from two equations which need not be lin- became a fixture of American algebras. ear: substitution (solve an equation for a variable Gaussian elimination for the purpose of school- and place the formula in the other) and equality-of- books was thus complete by the turn of the nine- values (solve in both and set the formulas equal). teenth century. It was truly schoolbook elimination,

784 Notices of the AMS Volume 58, Number 6 A is ai d’Angers. David June/July matrix a for first-order statistics, the to given minimum. still the for is the conditions differential equations Gauss, Following “Normal- normal 84]. and name p. 214], 1822, p. [Gauss, 1809, gleichungen” “elimina- [Gauss, 73], vulgarem” p. tionem 1805, [Legendre, ordinares” “méthodes 2 of technology the computation. Gaussian hand with soon professional evolve that would method elimination least-squares the of “ordi- elimination. by meaning schoolbook elimination”, said, “common or respectively methods” they nary which solved, equations”, be “normal the could sense Gauss’s of or “equations least-squares Legendre’s minimum” from the obtained Gauss ap- in were The once solutions 1986]. led dispute [Stigler, proximate which calculations priority his model, planet unfortunate disclosed is linear dwarf an general what to “lost” the stated called succinctly the to now Legendre of methods be- Then unstated orbit Ceres. Gauss sum using the discrepancies. by the calculate the celebrity minimizing of a squares came by the equations of simultane- linear overdetermined, in ous unknowns squares inferences the statistical draw “car- for to least method modern a was of quarrés,” It rés”). moindres method des what the (“méthode invented named 5) Legendre (Figure Legendre Adrien-Marie [1809] Gauss when and [1805] arose equa- linear finally simultaneous solve tions to need societal A Elimination Professional readily algebra. provide symbolic to in developed exercises undertaken been had it because called (1765–1843) it Lacroix Sylvestre 4. Figure ie ounvectors column sized h utdprssae déutosd iiu”and minimum” du “d’équations are: phrases quoted The t min b élimination nmdr oaino ueia nlssrte than rather analysis numerical of notation modern In owiheiiainwsapplied. was elimination which to , J. G. Reinis, The Portrait Medallions of David D’Angers. x b − Ax 2011 mg fa 81bsrle by bas-relief 1841 an of Image . 2 n h omleutosare equations normal the and , b A and ffl ounrn n suitably and rank column full of 2 x uhwsteimportance the was Such h es-qae problem least-squares the , A t Ax oie fteAMS the of Notices = ySefidDte edxnfrte1828 the of for frontispiece Bendixen Detlev Siegfried by Lithograph elimination”. “common method, replacing professional first (1777–1855) the Gauss devised Friedrich Carl 5. Figure .2]woeteoedtrie qain as [1810, equations overdetermined Gauss the wrote [1759]. 22] Lagrange p. the of in form squares of canonical sum reformulated the reexpress He to problem calculations. the own his he passage described one in but 2004, justifications, statistical dealt with mostly [Dunnington, publications least-squares calculations numbers His 138]. prodigious p. million his a that involved estimated who n oo,i em fwihh osrce linear parameters: constructed fewer he successively which of of combinations terms in on, so and as hs nunmdnotation, unnamed an chose Gauss hs u fsquares, of sum whose insi h omleutos qiaety in equivalently, coeffi- form equations, numeric quadratic normal the the the represent in to cients used he which rce oainto notation bracket htdffrwt ahosre ntneo the of of quantities growing instance the The observed by primes). indicated each (as with model differ that h iermdl while model, linear the where A C B as isl a nicriil computer, incorrigible an was himself Gauss = = = n n n [an] [bn, [cn, ,q ,... . . r, q, p, [xy] ′ ′′ [xy, [xy, + + + 2 1 ] ] p a a a = 2 1 ′ ′′ + ,w w, ] ] p p [aa]p xy = = srnmsh Nachrichten Astronomische + + + r h nnw aaeesof parameters unknown the are [xy, [xy] ′ + w , q b b b Ω ′ ′′ x + 1 ′′ as hnetne his extended then Gauss . q q − etc., etc. ′ ] . . . , y + [bb, ,a ,c . . . c, b, a, n, + + + − [ax][ay] ′ Ω [ab]q r c c c [bx, st eminimized. be to is , + [aa] r h discrepancies the are ′ ′′ 1 x r r ]q [bb, ′′ 1 + + + y ][by, + + ′′ , ...... 1 + [bc, [cc, ] + r numbers are [ac]r = = = 1 , . . . ] 1 2 w w w , ]r ]r ′ ′′ + + + ...... 785 These formulas complete the squares of successive parameters in the quadratic form, leaving A2 B2 C2 Ω = + + + + [nn, µ], [aa] [bb, 1] [cc, 2] where µ is the quantity of variables. Thus A = 0, B = 0, C = 0, … can be solved in reverse order to obtain values for p, q, r, . . . , also in reverse order, at which Ω attains its minimum, [nn, µ]. Solving least-squares problems by Gauss’s method required the numbers [xy, k] for in- creasing indices k. Gauss halved the work of schoolbook elimination, because he needed x, y only in alphabetic order. More importantly, his notation discarded the equations of symbolic al-

gebra, thereby freeing computers to organize the Provided by the family ofCourtesy Adele also Chauvenet of Hanifin. the United States Naval Academy. work more efficiently. When Gauss calculated, he Figure 6. William Chauvenet (1820–1870) and simply wrote down lists of numbers using his others taught Gauss’s computing methods. bracket notation to identify the values [Gauss, Portrait circa 1839 by Eunice Makepeace 1810, p. 25]. Towle. The subsequent evolution of Gaussian elimination illustrates the meta-disciplinary nature of mathematics [Grcar, 2011c]. People such as Gauss with significant knowledge of fields besides math- feat was astounding, because in principle it re- ≈ 3 ematics were responsible for these developments. quires about 23000 n /3 arithmetic operations The advances do not superficially resemble either for n = 41. Doolittle dispensed with Gauss’s brack- schoolbook elimination or what is taught at uni- ets and instead identified the numbers by their versities today, but they are no less important, positions in tables. He replaced the divisions in because through them social needs were met. the bracket formulas by reciprocal multiplications. Professional elimination began to develop due Doolittle’s tables colocated values to help him use to the economic and military value of cartogra- the product tables of August Leopold Crelle [1864]. phy. Gauss took government funds to map the An unrecognized but prescient feature of Doolit- German principality where he lived. Maps were tle’s work combined graphs with algebra to reduce drawn from networks of triangles using angles computational complexity. He used the carto- that were measured by surveyors. The angles had graphic triangulation to suggest an ordering of to be adjusted to make the triangles consistent. To the overdetermined equations (equivalently, an that end, Gauss [1826] devised a method to find arrangement of coefficients in the normal equa- least-squares solutions of underdetermined equations) to preserve zeroes in the work. The zeroes tions by a quadratic-form calculation similar obviated many of the 23000 operations. 3 The next innovations were made by the French to the overdetermined case. Once Friedrich military geodesist André-Louis Cholesky [Benoit, Bessel [1838] popularized Gauss’s approach, 1924] (Figure 8). He, too, addressed the nor- cartographic bureaus adopted the bracket no- mal equations of the underdetermined, angle- tation. Gauss’s calculations became part of the adjustment problem. Although Cholesky is re- mathematics curriculum for geodesists. membered for the square roots in his formulas,4 By whatever method of elimination is per- his innovation was to order the arithmetic steps formed, we shall necessarily arrive at the to exploit a feature of multiplying calculators. The same final values …; but when the number machines were mass-produced starting in 1890 of equations is considerable, the method of [Apokin, 2001], and Cholesky personally used a substitution, with Gauss’s convenient nota- Dactyle (Figure 9). A side effect of long multi- tion, is universally followed. plication is that the machines could internally — Chauvenet [1868, p. 514] (Figure 6) 5 accumulate sums of products. Calculations thus The first innovations after Gauss came from arranged were quicker to perform because fewer Myrick Doolittle [1881] (Figure 7). He was a computer at the United States Coast Survey who could 4Cholesky originated a construction that is now inter- solve forty-one normal equations in a week. This preted as decomposing a symmetric and positive definite matrix into a product LLt , where L is a lower triangular 3For A of full row rank, these problems were matrix and t is transposition. 5 minAx=b x2, where x are the angle adjustments The sums of products became known as dot-, inner-, or and Ax = b are the consistency conditions. The solu- scalar-products of vectors after the adoption of matrix tion was given by correlate equations x = At u, where notation. Calculations employing sums were described as AAt u = b are again normal equations to be solved. “abbreviated” or “compact”.

786 Notices of the AMS Volume 58, Number 6 ubr oprdt Doolittle’s to compared numbers June/July paper. on for recorded tables be Cholesky’s to had results intermediate 1895. (1875–1918), circa Cholesky André-Louis 8. Figure (1830–1911), 1862. Doolittle circa Hascall Myrick 7. Figure Ldra,14]oc twsrivne ythe by reinvented [1945]. used Dwyer was Paul widely statistician it American become The once methods. did 1948] own [Laderman, method” its root develop to “square statis- begun [Brezinski, after geodesy had notice in tics appeared little it received because 2006] it was but method superior, Cholesky’s in 1989]. analyses [Fox, used regression econometrics department for The methods 1927]. least-squares Ezekiel, Agriculture and of from Department [Tolley the moved to Survey they Coast example, when the on method For insisted Doolittle’s successfully statistics. employees using government it to States that transition Doolittle’sUnited prominent the War. so World made was First the method least-squares after for use problems primary the became analysis ttsia nlssi h omo regression of form the in analysis Statistical

© École Polytechnique. Courtesy of Antiochiana, Antioch College. 2011 n qain edonly held equations O (n 3 ) . O (n 2 oie fteAMS the of Notices ) i n oiiedfiie h edt routinely to need The definite. modern positive symmet- In be and must equations. coefficients ric of normal matrix the for terms, only sufficed in Bros. Chateau Foncine-le-Haut. by 1905 Braunschweig from in and Brunsviga by made resold machines Dactyle Odhner. Willgodt Russian-Swedish entrepreneur the the of on design based pinwheel calculator Dactyle 9. Figure ounpout hc stentrlwyt cal- hand. to by way figures of natural columns column-by- the with is culate a which have product, They column Cracovians. the called in matrices form invent independently to 12) (Figure [1938a,b] Banachiewicz moti- Tadeusz computing astronomer vated Manual bookstore. in the campus presently are your to that University textbooks analysis Jagiellonian numerical the of observatory computers. electronic the programming for of calculations purpose organize help Eventually would decompositions. matrices matrix re- through trivially lated were pro- algorithms the elimination all liferating rep- that new showed the technology hand, by resentational compute to needed not were century matrices Although nineteenth 2008]. the 1977a,b, of 1975, [Hawkins, half second the main developed oftrices modify had adoption authors to Several the algebra. algebra with matrix ended symbolic elimination using Gaussian of milieu The Interpretation Matrix machines 11). calculating (Figure of manufacturer publicized leading was a method by reorganized accumulate Crout’s products. to 10), of Technology, sums elimination (Figure of Gaussian [1941] mathematician schoolbook Institute Crout a Massachusetts Cholesky, Prescott the by of at work Unaware earlier engineering. the in developed equations linear gradually simultaneous of kinds more solve h ehd fGus oltl,adCholesky and Doolittle, Gauss, of methods The yCaoin n h hoyb matrices. by theory the made and are Cracovians computations in by the may, that It said …. be column fact, by row column by multiply prac- column to multiply than in to easier that is conceded it tice be however, must, It astronomical the from leads development This esn[1944] Jensen —

Photo by Freddy Haeghens, [email protected]. 787 788 w epe er esn[94 Fgr 3 of pictograms, used 13) Institut (Figure Geodætisk [1944] Danish Jensen by the Henry algebra matrix people. idea Cayley’s The two Arthur 1933. in as realized early was as calculations represent n“ffiin a fbidn p oe“so-called some up” building was University of method the way Doolittle’s “efficient of that an 14) showed Michigan (Figure of year [1944] same Dwyer to the In Paul [1938]: matrices”. al. “elementary by multipli- et cation through Frazer operations arithmetic by represent presen- suggested Jensen’s was of tation aspect Cholesky’s noteworthy and A method, the method. Cracovian product: the triangular Gauss’s brackets), a with calculation as (the express- algorithm” matrix “Gaussian to square a amounted ing equations normal solving Oakland, in manufactured was It 10ACT. model iue1.Cotsmto a efre with performed was method Crout’s 11. Figure iue1.Pect uadCot(1907–1984), Crout Durand Prescott 10. Figure = aaheizavctduigCaoin to Cracovians using advocated Banachiewicz Courtesy of John Wolff, ahnssc sti acatcalculator, Marchant this as such machines www.vicnet.net.au/~wolff. Courtesy of MIT Museum. oepaieta he loihsfor algorithms three that emphasize to , aiona nte13sad1940s. and 1930s the in California, oie fteASVolume AMS the of Notices ic 1936. circa r 6 on esnspprthrough paper Jensen’s found 16) ure ie by aided opoewa o emn i not. did Neumann von what prove to 6 decompo- matrix the sition. introduced part analysis initial their The of elimination. Gaussian Golds- study and to Neumann tine von motivated machines of the efficacy programmable, the first over Concerns the computers. building electronic col- of process his the were and others in and 15) They Goldstine. (Figure Herman Neumann laborator von Cayleyan John of from terms in is, that the matrices. form, roughly are in modern Dwyer elimination the Gaussian and depict Jensen to earliest of The inter- Banachiewicz. papers of similar work coincident the no in found except pretation He matrices. triangular” cuae oprbeerrbudhsytt be to case. general yet the has in bound be established error to achieved. comparable observed been a is accurate, has elimination that Gaussian best Although the their remains definite, positive bound and When symmetric is available. matrix became the computers al- once mechanized of the gorithm errors be rounding would anticipated the they on what bounds establish to bra sdCaoinagbat eciecomputing describe to algebra Cracovian used oeecs:Teedrsl fmc td s onttry not do is, study much of result end The exercise: No iue1.TduzBnciwc (1882–1954) Banachiewicz Tadeusz 12. Figure h etse otecmu oktr was bookstore campus the to step next The ehda h obnto ftotricks: two decomposes of it combination First, elimination the … the as interpret method therefore may We came interpretation matrix the for use deep A o emn n odtn sdmti alge- matrix used Goldstine and Neumann Von inductive explicit, simple, process. a by forms inverses it their second] [and … matrices [triangular] o emn n odtn [1947] Goldstine and Neumann von — ahmtclReviews Mathematical ehd.Poofo 1946. from Photo methods. A noapouto two of product a into 6 onTd (Fig- Todd John . 58 Number ,

MR Courtesy of Adam Strzałkowski. and 6 icpieta a tfis ouae ythose June/July by populated first at was that discipline a clear ideas made wrote them. that overworking expression Turing without of of matrices. brevity manner a elementary with the in with case and general Jensen elimination the Neu- treating schoolbook von described by of of Goldstine He manner and the cited. mann in [1948] Gold- elimination and Turing Gaussian Neumann Todd von whom to through visit stine, a matrix Jensen during the also Turing from and of Alan learned both evidently Phys- was interpretation who National them 17), the Among (Figure at Laboratory. the staff communicated ical to auditor [Taussky formalism An London matrix 2006]. College Todd, King’s at and it on lectured 1960s. Photo circa algebra. matrix (1901–1982) used Dwyer independently Sumner Paul 14. Figure Cayleyan use algebra. to Banachiewicz was by (1915–1974) inspired Jensen Henry 13. Figure h neto feetoi optr created computers electronic of invention The

Courtesy of Prof. Ingram Olkin, Dept. of Statistics, Stanford University. Courtesy of the University of Copenhagen (Andersen 1975). 2011 oie fteAMS the of Notices tecmiaino w rcs.Poofrom Photo 1947. tricks”. March two of saw combination (1903–1957) Neumann “the von John 15. Figure An Coda they textbooks then presentation. Turing’s interpretation, reprised analysis matrix [1967] the featured Moler numerical that and influential 1960s. Forsythe the and wrote until [1964] taught Fox algebra commonly When Linear not was slowly. itself more descriptions matrix decade. a within used widely was tab. name 2011a, The first [Grcar, 1]. the elimination” been “Gaussian it have call to to appears and Society, Gauss elimi- to Mathematical nation school” “high American misattributed [1953] an the Forsythe In “elimination”. to for simply address named origin was an then suggested The what Gauss describe discipline. of to calculations new used the had the geodesists of science” that matter terminology “computer subject credence the lent named to involvement who have Gauss’s mathematician 1972]. to [Knuth, visionary reputed a is Forsythe was 1972; George 18) [Traub, them, (Figure Among calculations 1970]. scientific Wilkinson, made who uaigmcie,teaihei tp a be can steps arithmetic cal- the for products machines, of culating sums Cholesky emphasized While Crout 1983]. and Reid, parallel and with [Duff 1970] computers of [Irons, were equations analysis the use methods solve finite-element to for other recently more method tables, developed a multiplication developed calculations with Doolittle least-squares then pro- for first the and the and method developed problem Gauss fessional the as to Just cal- adapted the computer. the well because whether is on way, culation depends al- computing straightforward of an a speed becomes in seldom gorithm elimination Gaussian interpretation matrix of The problem. mathematical h nvriymteaiscriuu adopted curriculum mathematics university The algorithm sasre fsesfrsliga solving for steps of series a is

Courtesy of the Los Alamos National Laboratory Archives. 789 790 oerdcltasomtosaepsil that possible solve to are work the transformations reduce com- 1998]. radical diﬀerent Dongarra, More and suit [Whaley architectures to puter automatically reordered aobr[93,adWieie[1968–1982] Whiteside and [1938], for pleasure [1923], Kloyda a referees of is Macomber scholarship the It the and article. acknowledge editor this to improved the that to advice grateful am I archetype, Acknowledgments Platonic technique. a their evolving an being is is elimination algorithms than Gaussian the future Rather Perhaps about name. 2007]. al., certainty et only and Demmel Cohn 2003; 1969; Umans, [Strassen, operations arithmetic iue1.Jh od(9120)lcue on lectured (1911–2007) Todd John 16. Figure iue1.Aa uig(9215)described (1912–1954) Turing Alan 17. Figure

Courtesy of the Royal Society Archives, Courtesy of the Archives, California Institute of Technology.

ti h a nvriisnwtahit. teach now universities way the in it © Godfrey Argent Studio. n htgahcra1951. circa Photograph esnswr n1946. in work Jensen’s qain below equations oie fteASVolume AMS the of Notices O (n 3 ) .D Crout D. P. Cohn H. .L Crelle L. A. .Andersen E. to possible References it made which contribution. 21), Newton’s identify 20, 19, (Figures .Buteo J. Brezinski C. .Chauvenet W. .G Bashmakova G. I. Benoit Banachiewicz T. .A Apokin A. I. .W Bessel W. F. iue1.Gog oste(9717)called (1917–1972) Forsythe George 18. Figure emr eln n dto,1864. edition, 2nd ersparen Berlin, ganz Reimer, Tausend unter Zahlen mit Dividiren 1868. Philadelphia, Co., & Lippincott B. J. in 1974, n ovn ytm flna qain …, equations Engrs. linear Elect. Inst. of Amer. systems solving and 2003. Sci. IEEE, Comput. of Foundations in on Symp. multiplication, matrix fast to c Math. Sc. …, linéaires ooas,SreA c Math. Sc. A. Série Polonaise, carrés, moindres des méthode la ueia Algorithms Numerical … Ostpreussen so.Ae. ahntn C 2000. DC, Washington, Algebra Amer., Assoc. of Evolution and 2001. Braunschweig/Wiesbaden, Vieweg, atCholesky), dant nvrie,1975. Universitet, foer,i .Toean .Y iusv and Nitussov, Y. A. Trogemann, editors, G. Ernst, in W. ifmometr”, 94 h uhri dnie nya Commandant as only identiﬁed is Benoit. author The 1924, éhd erslto uéiu e équations des numérique résolution de Méthode , oesruemtoe…(rcd uComman- du (Procédé … méthode une sur Note , , Logistica and ae 9–0,1938b. 393–404, pages , ro 1973–1974 Ârbog hr ehdfreautn determinants evaluating for method short A , , er esn coe 951,August 1915–10, October 7 Jensen, Henry , ilotTepi dnradhs“Ar- his and Odhner Theophil Willgodt , h ieadwr fAdéCholesky, André of work and life The , ehnaenwlh le utpiie und Multipliciren alles welche Rechentafeln .Umans C. ul nen elAa.Plnie éi A. Série Polonaise, l’Acad. de Intern. Bull. , and raieo h ehdo es Squares Least of Method the on Treatise öi.Aa.Ws. eln 1838. Berlin, Wiss., Akad. König. , rnie ’n ovletcnqede technique nouvelle d’une Principes , ai,1560. Paris, , and ultngéodésique Bulletin optn nRussia in Computing .J Baeyer J. J. .S Smirnova S. G. 33:7–8,2006. 43(3):279–288, , ru-hoei approach group-theoretic A , 013–21 1941. 60:1235–1241, , t“asinelimination”. “Gaussian it ae 24.Københavns 42–45. pages , T.A hnte) Math. Shenitzer), A. (Tr. ae 3–3,1938a. 134–135, pages , , ul nen el’Acad. de Intern. Bull. rc 4hAn IEEE Ann. 44th Proc. rdesn in Gradmessung 2:77,April (2):67–77, , , ae 438–449, pages , 58 h Beginnings The ae 39–46. pages , Number , Trans. G. , Josè Mercado/Stanford News Service. 6 , June/July 1920. yearbook, California, School Redding, High County primary Shasta the sources. examine to neglected academicians who later prominent more and bested textbooks American to historical antecedents of She importance record”. the on anticipated method this of earliest appearance “the was solving equations for simultaneous lesson Newton’s [1923] that Macomber observed Louise Gertrude 20. Figure information. electronic the of before age assembled more being the for all impressive is that of sources collection primary unrivaled is an dissertation by Her supported 2009]. LaDuke, and mathematics [Green American a of was woman She pioneering texts. Renaissance in few examples the Kempis discovered à (1896–1977) Thomas Kloyda Mary Sister 19. Figure

Courtesy of the Shasta County Historical Society. Courtesy of Sisters of Saint Francis, Rochester, Minnesota. 2011 oie fteAMS the of Notices .S Dwyer S. P. .W Dunnington W. G. .S Duff S. I. Doolittle H. M. .C Domingues C. J. olcino i papers. his monumental of a collection in wrote Newton what remembered 2008] [Guicciardini, Whiteside (1932–2008) Thomas Derek 21. Figure .Dme,I Dumitriu I. Demmel, J. .A Fox A. K. .A rzr .J Duncan J. W. Frazer, A. R. .Fox L. .E Forsythe E. G. Forsythe E. G. .Euler L. n orlto hoywt arxjsicto fim- solution, of of methods justification proved matrix with theory correlation and 2004. ence ah Software Math. systems, linear symmetric sparse definite 28,1944. 82–89, 1881. DC, 1878 Washington, June, Office, Printing with Government Ending 115–120. … Survey Geodetic triangu- and a of adjustment in the larization, in and equations normal 2008. Basel, ler sstable, is algebra Revolution, 1967. Jersey, New Cliffs, Systems Algebraic Linear of regression, and lrno rs,Ofr,1964. Oxford, Press, Clarendon 1:37,1989. (1):53–70, 1953. interesting, be abig,1938. Cambridge, and Dynamics Equations to Applications Differential Some and Matrices tary 1945. ah so.Ae. ahntn C n edition, 2nd DC, Washington, Amer., Assoc. Math. , , h qaero ehdadisuei correlation in use its and method root square The , , nItouto oNmrclLna Algebra Linear Numerical to Introduction An giutrlEooit nteEconometric the in Economists Agricultural , and netn u Algebra zur Anleitung arxpeetto flatsquares least of presentation matrix A , .K Reid K. J. xodEooi aes e Series New Papers, Economic Oxford eoto h ueitneto h Coast the of Superintendent the of Report ovn ieragbaceutoscan equations algebraic linear Solving , , ()3235 1983. 9(3):302–325, , and ehdepoe nteslto of solution the in employed Method , ul mr ah Soc. Math. Amer. Bull. ari n h Calculus the and Lacroix .Ae.Sait Assoc. Statist. Amer. J. ue.Math. Numer. , alFidihGus ia fSci- of Titan Gauss: Friedrich Carl .B Moler B. C. h utfotlslto fin- of solution multifrontal The , abig nvriyPress, University Cambridge , and , and , rnieHl,Englewood Prentice-Hall, , n.Mt.Statist. Math. Ann. .Holtz O. ud 1771. Lund, , 0()5–1 2007. 108(1):59–91, , .R Collar R. A. , optrSolution Computer 59(4):299–329, , atlinear Fast , 40:493–503, , Birkhäuser, , C Trans. ACM , Courtesy of Dr. Piers Bursill-Hall, Cambridge University. Elemen- pages , 15(1): , 41 , , 791 C. F. Gauss, Theoria Motus Corporum Coelestium in J. Laderman, The square root method for solving simul- Sectionibus Conicis Solum Ambientium, Perthes and taneous linear equations, Math. Tables Aids Comp., 3 Besser, Hamburg, 1809. (21):13–16, 1948. , Disquisitio de elementis ellipticis Palladis, Com- J. L. Lagrange, Recherches sur la méthode de maximis et mentationes Societatis Regiae Scientiarum Gottingensis minimis, Miscellanea Taurinensia, 1, 1759. Reprinted recentiores: Commentationes classis mathematicae, 1 in Œuvres de Lagrange, (J.-A. Serret, editor), vol. 1, (1808–1811):1–26, 1810. pages 1–16, Gauthier–Villars, Paris, 1867. , Anwendung der Wahrscheinlichkeitsrech- A. M. Legendre, Nouvelle méthodes pour la détermi- nung auf eine Aufgabe der practischen Geometrie, nation des orbites des comètes, Chez Didot, Paris, Astronomische Nachrichten, 1(6):81–86, January 1822. 1805. , Supplementum theoriae combinationis observa- U. Libbrecht, Chinese Mathematics in the Thir- tionum erroribus minimis obnoxiae, Commentationes teenth Century, MIT Press, Cambridge, 1973. Societatis Regiae Scientiarum Gottingensis recentiores: G. L. Macomber, The influence of the English and French Commentationes classis mathematicae, 6 (1823–1827): writers of the sixteenth, seventeenth, and eighteenth 57–98, 1826. centuries on the teaching of algebra, M.A. thesis, J. F. Grcar, How ordinary elimination became Gaussian University of California, Berkeley, 1923. elimination, Historia Math. 38(2):163Ð218, 201a. doi: J.-C. Martzloff, A History of Chinese Mathematics (Tr. 10.1016/j.hm.2010.06.003. S. S. Wilson), Springer, Berlin, 1997. , John von Neumann’s Analysis of Gaussian elim- I. Newton, Universal Arithmetick, Senex, Taylor et al., ination and the origins of modern numerical analysis, London, 1720. SIAM Rev., to appear, 2011b. J. Peletier du Mans, L’Algebre, Lyon, 1554. , Mathematics turned inside out: The intensive K. Plofker, Mathematics in India, Princeton Univ. Press, faculty versus the extensive faculty, Higher Education, 2009. 61(6), 2011c. doi: 10.1007/s10734-010-9358-y. R. R¯ashid, Entre arithmétique et algèbre: Recherches sur J. Green and J. LaDuke, Pioneering Women in American l’histoire des mathématiques arabes, Société d’Édition Mathematics: The Pre-1940 Ph.D.’s, Amer. Math. Soc., Les Belles Lettres, Paris, 1984. Providence, 2009. J. G. Reinis, The Portrait Medallions of David d’Angers, N. Guicciardini, Derek Thomas Whiteside (1932–2008), Polymath Press, New York, 1999. Historia Math., 36(1):4–9, 2008. E. Robson, Mathematics in Ancient Iraq: A Social History, R. Hart, The Chinese Roots of Linear Algebra, Johns Princeton Univ. Press, 2008. Hopkins University Press, Baltimore, 2011. M. Rolle, Traité d’algèbre, E. Michallet, Paris, 1690. T. Hawkins, Cauchy and the spectral theory of matrices, K. Shen, J. N. Crossley, and A. W.-C. Lun, The Nine Historia Math., 2:1–29, 1975. Chapters of the Mathematical Art Companion and , Another look at Cayley and the theory of matri- Commentary, Oxford Univ. Press, New York, 1999. ces, Archives Internationales d’Histoire des Sciences, 27 T. Simpson, A Treatise of Algebra, John Nourse, London, (100):82–112, 1977a. 2nd edition, 1755. , Weierstrass and the theory of matrices, Arch. S. M. Stigler, The History of Statistics: The Measurement Hist. Exact Sci., 17(2):119–164, 1977b. of Uncertainty before 1900, Harvard University Press, , Frobenius and the symbolical algebra of Cambridge, 1986. matrices, Arch. Hist. Exact Sci., 62(1):23–57, 2008. V. Strassen, Gaussian elimination is not optimal, T. L. Heath, Diophantus of Alexandria, Cambridge Numer. Math., 13(4):354–356, 1969. University Press, Cambridge, 2nd edition, 1910. O. Taussky and J. Todd, Cholesky, Toeplitz and A. Heeffer, From the second unknown to the sym- the triangular factorization of symmetric matrices, bolic equation, in A. Heeffer and M. Van Dyck, Numerical Algorithms, 41:197–202, 2006. editors, Philosophical Aspects of Symbolic Reasoning H. R. Tolley and M. Ezekiel, The Doolittle Method …, J. in Early Modern Mathematics, pages 57–101, College Amer. Statist. Assoc., 22(160):497–500, 1927. Publications, London, 2011. J. F. Traub, Numerical mathematics and computer J. Høyrup, Lengths, Widths, Surfaces: A Portrait of Old science, Comm. ACM, 15(7):537–541, 1972. Babylonian Algebra and Its Kin, Springer-Verlag, New A. M. Turing, Rounding-off errors in matrix processes, York, 2002. Quart. J. Mech. Appl. Math., 1(3):287–308, 1948. B. M. Irons, A frontal solution program for fi- J. von Neumann and H. H. Goldstine, Numerical in- nite element analysis, Int. J. Numer. Methods Eng., verting of matrices of high order, Bull. Amer. Math. 2(1):5–32, 1970. Soc., 53(11):1021–1099, 1947. H. Jensen, An attempt at a systematic classification of C. Whaley and J. Dongarra, Automatically tuned lin- some methods for the solution of normal equations, ear algebra software, in Proc. 1998 ACM/IEEE SC98 Meddelelse 18, Geodætisk Institut, Copenhagen, 1944. Conference, pages 1–27, IEEE, 1998. M. Thomas à Kempis Kloyda, Linear and Quadratic D. T. Whiteside, editor, The Mathematical Papers of Equations 1550–1660, Edwards Brothers, Ann Arbor, Isaac Newton, 8 vols., Cambridge University Press, 1938, University of Michigan dissertation. 1968–1982. D. E. Knuth, George Forsythe and the Development of J. H. Wilkinson, Some comments from a numerical Computer Science, Comm. ACM, 15(8):721–726, 1972. analyst, J. ACM, 18(2):137–147, 1970. S. F. Lacroix, Elements of Algebra (Tr. J. Farrar), F. Woepcke, Extrait du Fakhrî, Paris, 1853. University Press, Cambridge, Massachusetts, 1818. , Elemens d’algèbre, Chez Courcier, Paris, 5th edition, 1804.

792 Notices of the AMS Volume 58, Number 6