Methods of Conjugate Gradients for Solving Linear Systems
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
NBS-INA-The Institute for Numerical Analysis
t PUBUCATiONS fl^ United States Department of Commerce I I^^^V" I ^1 I National Institute of Standards and Tectinology NAT L INST. OF STAND 4 TECH R.I.C. A111D3 733115 NIST Special Publication 730 NBS-INA — The Institute for Numerical Analysis - UCLA 1947-1954 Magnus R, Hestenes and John Todd -QC- 100 .U57 #730 1991 C.2 i I NIST Special Publication 730 NBS-INA — The Institute for Numerical Analysis - UCLA 1947-1954 Magnus R. Hestenes John Todd Department of Mathematics Department of Mathematics University of California California Institute of Technology at Los Angeles Pasadena, CA 91109 Los Angeles, CA 90078 Sponsored in part by: The Mathematical Association of America 1529 Eighteenth Street, N.W. Washington, DC 20036 August 1991 U.S. Department of Commerce Robert A. Mosbacher, Secretary National Institute of Standards and Technology John W. Lyons, Director National Institute of Standards U.S. Government Printing Office For sale by the Superintendent and Technology Washington: 1991 of Documents Special Publication 730 U.S. Government Printing Office Natl. Inst. Stand. Technol. Washington, DC 20402 Spec. Publ. 730 182 pages (Aug. 1991) CODEN: NSPUE2 ABSTRACT This is a history of the Institute for Numerical Analysis (INA) with special emphasis on its research program during the period 1947 to 1956. The Institute for Numerical Analysis was located on the campus of the University of California, Los Angeles. It was a section of the National Applied Mathematics Laboratories, which formed the Applied Mathematics Division of the National Bureau of Standards (now the National Institute of Standards and Technology), under the U.S. -
Key Moments in the History of Numerical Analysis
Key Moments in the History of Numerical Analysis Michele Benzi Department of Mathematics and Computer Science Emory University Atlanta, GA http://www.mathcs.emory.edu/ benzi ∼ Outline Part I: Broad historical outline • 1. The prehistory of Numerical Analysis 2. Key moments of 20th Century Numerical Analysis Part II: The early history of matrix iterations • 1. Iterative methods prior to about 1930 2. Mauro Picone and Italian applied mathematics in the Thirties 3. Lamberto Cesari's work on iterative methods 4. Gianfranco Cimmino and his method 5. Cimmino's legacy PART I Broad historical outline The prehistory of Numerical Analysis In contrast to more classical fields of mathematics, like Analysis, Number Theory or Algebraic Geometry, Numerical Analysis (NA) became an independent mathematical disci- pline only in the course of the 20th Century. This is perhaps surprising, given that effective methods of computing approximate numerical solutions to mathematical problems are already found in antiquity (well before Euclid!), and were especially prevalent in ancient India and China. While algorithmic mathematics thrived in ancient Asia, classical Greek mathematicians showed relatively little interest in it and cultivated Ge- ometry instead. Nevertheless, Archimedes (3rd Century BCE) was a master calculator. The prehistory of Numerical Analysis (cont.) Many numerical methods studied in introductory NA courses bear the name of great mathematicians including Newton, Euler, Lagrange, Gauss, Jacobi, Fourier, Chebyshev, and so forth. However, it should be kept in mind that well into the 19th Century, the distinction between mathematics and natural philosophy (including physics, chemistry, astronomy etc.) was almost non-existent. Scientific specialization is a modern phenomenon, and nearly all major mathematicians were also physicists and astronomers. -
SAM History: Overview
SAM History: Overview The precurser of the Seminar for Applied Mathematics (SAM) (in German: Seminar f¨urAngewandte Mathematik) at ETH Zurich, the Institute for Applied Mathematics (IAM) was founded on January 1, 1948, at the request of Professor Eduard Stiefel (21.4.1909{25.11.1978), who had been a full profes- sor of mathematics since 1943, well known for his earlier contributions to pure mathematics. His vision was to introduce applied mathematics at ETH based on the usage of electronic computers as they were being developed in the United States, England, and the Netherlands. Among his early goals was the construction of a Swiss programmable electronic computer. As his first collaborators he hired two perma- nent assistants Heinz Rutishauser (30.1.1918{10.11.1970), a mathematician, and Ambros P. Speiser 13.11.1922{10.5.2003), an electrical engineer. For gathering information on the current status of computer technology, Stiefel went in October 1948 for five months on a fact finding mission to Amsterdam and sev- eral places of the United States. He also sent his two assistants for a year to the two top U.S. computer research centers: Harvard and Princeton. When Stiefel returned from the U.S., he learned that the Z4 computer, the fourth computer the ingenious German Konrad Zuse (22.6.1910{18.12.1995) had constructed during WWII had survived the war and was available for rent. He decided to rent it for a period of five years. It was installed at ETH in August 1950. Thanks to the Z4, Stiefel and his collaborators could quickly start working on numerical methods and their applications. -
NBS-INA-The Institute for Numerical Analysis
t PUBUCATiONS fl^ United States Department of Commerce I I^^^V" I ^1 I National Institute of Standards and Tectinology NAT L INST. OF STAND 4 TECH R.I.C. A111D3 733115 NIST Special Publication 730 NBS-INA — The Institute for Numerical Analysis - UCLA 1947-1954 Magnus R, Hestenes and John Todd -QC- 100 .U57 #730 1991 C.2 i I NIST Special Publication 730 NBS-INA — The Institute for Numerical Analysis - UCLA 1947-1954 Magnus R. Hestenes John Todd Department of Mathematics Department of Mathematics University of California California Institute of Technology at Los Angeles Pasadena, CA 91109 Los Angeles, CA 90078 Sponsored in part by: The Mathematical Association of America 1529 Eighteenth Street, N.W. Washington, DC 20036 August 1991 U.S. Department of Commerce Robert A. Mosbacher, Secretary National Institute of Standards and Technology John W. Lyons, Director National Institute of Standards U.S. Government Printing Office For sale by the Superintendent and Technology Washington: 1991 of Documents Special Publication 730 U.S. Government Printing Office Natl. Inst. Stand. Technol. Washington, DC 20402 Spec. Publ. 730 182 pages (Aug. 1991) CODEN: NSPUE2 ABSTRACT This is a history of the Institute for Numerical Analysis (INA) with special emphasis on its research program during the period 1947 to 1956. The Institute for Numerical Analysis was located on the campus of the University of California, Los Angeles. It was a section of the National Applied Mathematics Laboratories, which formed the Applied Mathematics Division of the National Bureau of Standards (now the National Institute of Standards and Technology), under the U.S. -
Discovery of Two Historical Computers in Switzerland: Zuse
Discovery of Two Historical Computers in Switzerland: Zuse Machine M9 and Contraves Cora and Discovery of Unknown Documents on the Early History of Computing at the ETH Archives Herbert Bruderer Swiss Federal Institute of Technology, Zurich [email protected], [email protected] Abstract. Since 2009 we have been trying to find out more about the beginnings of computer science in Switzerland and abroad. Most eyewitnesses have already died or are over 80 years old. In general, early analog and digital computers have been dismantled long ago. Subject to exceptions, the few remaining devices are no longer functioning. Many documents have gone lost. Therefore investigations are difficult and very time-consuming. However, we were able to discover several unknown historical computers and hundreds of exciting documents of the 1950s. In 2012, the results have been published in a first book; a second volume will be forthcoming next year. Keywords: Cora, ERMETH, ETH archives, Hasler AG, Heinz Rutishauser, Konrad Zuse, M9 (=Z9), Mithra AG, Paillard SA, Remington Rand AG, Unesco, Z4. 1 Introduction My investigations into the history of computing started in 2009 in connection with the Zuse centenary (100th birthday of the German computer pioneer in 2010). The goal was to get a more comprehensive and deeper understanding of the events of the pioneering days. Coincidentally and to our surprise we discovered the program- controlled Zuse relay calculator M9 whose existence was unknown even to computer scientists. Shortly afterwards another machine emerged: the Swiss made transistor computer Cora. From 2010 to 2013 several contemporary eyewitness discussions concerning M9 and Cora took place. -
Numerical Analysis in Zurich – 50 Years Ago
2 Martin H. Gutknecht is a professor and senior scientist at ETH Zurich. He was also Scientific Director of the Swiss Center for Scientific Computing (CSCS/SCSC). Numerical Analysis in Zurich – 50 Years Ago Surely, applied mathematics originated in ancient gathering of this sort ever – as an opportunity to times and slowly matured through the centuries, recall some of the local contributions. We focus on but it started to blossom colorfully only when those in numerical analysis and scientific comput- electronic computers became available in the late ing, but we will also touch computers, computer 1940s and early 1950s. This was the gold miner’s languages, and compilers. time of computer builders and numerical analysts. Responsible for establishing (electronic) The venue was not the far west of the United scientific computing in Switzerland was primarily States, but rather some places in its eastern part, Eduard Stiefel (1909–1978): he took the initiative, such as Boston, Princeton, Philadelphia, and New raised the money, hired the right people, directed York, and also places in Europe, most notably, the projects, and, last but not least, made his own Manchester, Amsterdam, and Zurich. Only long lasting contributions to pure and applied math- after these projects had begun did it become ematics. Stiefel got his Dr. sc. math. from ETH in known that the electronic computer had been 1935 with a dissertation on Richtungsfelder und invented earlier by clever individuals: 1937–1939 Fernparallelismus in n-dimensionalen Mannig- by John V. Atanasoff and his gradute student faltigkeiten written under the famous Heinz Hopf. Clifford Berry at Iowa State College, Ames, Iowa, It culminated in the introduction of the Stiefel and, independently, 1935–1941 by Konrad Zuse in (-Whitney) classes, certain characteristic classes Berlin. -
Who Invented the Great Numerical Algorithms?
Nick Trefethen Oxford Mathematical Institute Who invented the great numerical algorithms? 1/50 A discussion over coffee. Ivory tower or coal face? Stigler’s Law of Eponymy No scientific law is named after its original discoverer. 2/50 SOME MAJOR DEVELOPMENTS IN SCIENTIFIC COMPUTING (29 of them) Before 1940 Newton's method least-squares fitting orthogonal linear algebra Gaussian elimination QR algorithm Gauss quadrature Fast Fourier Transform Adams formulae quasi-Newton iterations Runge-Kutta formulae finite differences 1970-2000 preconditioning 1940-1970 spectral methods floating-point arithmetic MATLAB splines multigrid methods Monte Carlo methods IEEE arithmetic simplex algorithm nonsymmetric Krylov iterations conjugate gradients & Lanczos interior point methods Fortran fast multipole methods stiff ODE solvers wavelets finite elements automatic differentiation 3/50 Before 1940 4/50 Newton’s Method for nonlinear eqs. Heron, al-Tusi 12c, Al Kashi 15c, Viète 1600, Briggs 1633… Isaac Newton 1642-1727 Mathematician and physicist Trinity College, Cambridge, 1661-1696 (BA 1665, Fellow 1667, Lucasian Professor of Mathematics 1669) De analysi per aequationes numero terminorum infinitas 1669 (published 1711) After 1696, Master of the Mint Joseph Raphson 1648-1715 Mathematician at Jesus College, Cambridge Analysis Aequationum universalis 1690 Raphson’s formulation was better than Newton’s (“plus simple” - Lagrange 1798) FRS 1691, M.A. 1692 Supporter of Newton in the calculus wars — History of Fluxions, 1715 Thomas Simpson 1710-1761 1740: Essays -
Ganderlatsis Rutishauser
My years with Rutishauser By Friedrich L. Bauer, TU Munich February 19, 1952 Heinz Rutishauser (1918-1970), who died far too early, was one of my very best friends; among the professionals of my age class that I became aquainted with in the fifties, he ranks in one line with Klaus Samelson (1918-1980), my Munich study colleague, and with Jim Wilkinson (1919-1986). Rutishauser gave a seminal lecture entitled Automatische Rechenplanfertigung bei programmgesteuerten ” Rechenmaschinen“ at the GaMM meeting in Freiburg in Breisgau, at the end of March, 1951, which I missed — I was still assistent of Fritz Bopp, the University of Munich Theoretical Physicist, successor of the famous Arnold Sommerfeld. I became aware of the lecture when it was published mid-1952 as an ETH-Mitteilung. Therefore, when I met Heinz Rutishauser in February 1952 for the first time, I was absolutely ignorant about his success, and we did not talk about the subject. It just happens that we met on February 19, 1952; tomorrow it will be 50 years ago. You may ask yourself, how our cooperation developed. It happened via Konrad Zuse and Eduard Stiefel. In 1949 — June 27, to be precise — Zuse had presented in the seminar of Wilhelm Britzlmayr (1892– 1970), logician and Honorary Professor (whose regular occupation was director of a bank) at the Ludwig- Maximilians-Universitat¨ in Munich a report on his ’Plankalkul¨ ’. Britzlmayr, who found out that I was interested in the topic and who had kindred relations in Switzerland, kept me informed from time to time on the further development, and mentioned one day that in July 1949 a professor at the ETH Zurich¨ had signed a contract to rent Zuse’s relay calculator »’Z4’, and that in August 1950 the machine was operational. -
Discovery of Two Historical Computers In
Discovery of Two Historical Computers in Switzerland: Zuse Machine M9 and Contraves Cora and Discovery of Unknown Documents on the Early History of Computing at the ETH Archives Herbert Bruderer To cite this version: Herbert Bruderer. Discovery of Two Historical Computers in Switzerland: Zuse Machine M9 and Contraves Cora and Discovery of Unknown Documents on the Early History of Computing at the ETH Archives. International Conference on History of Computing (HC), Jun 2013, London, United Kingdom. pp.96-104, 10.1007/978-3-642-41650-7_9. hal-01455272 HAL Id: hal-01455272 https://hal.inria.fr/hal-01455272 Submitted on 3 Feb 2017 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Distributed under a Creative Commons Attribution| 4.0 International License Discovery of Two Historical Computers in Switzerland: Zuse Machine M9 and Contraves Cora and Discovery of Unknown Documents on the Early History of Computing at the ETH Archives Herbert Bruderer Swiss Federal Institute of Technology, Zurich [email protected] Abstract. Since 2009 we have been trying to find out more about the beginnings of computer science in Switzerland and abroad. Most eyewitnesses have already died or are over 80 years old. -
The Pioneer Days of Scientific Computing in Switzerland
The Pioneer Days of Scientific Computing in Switzerland Martin H. Gutknecht Eidgenbssische Technische Hochschule CH-8092 Ziirich Abstract. Scientific computing was established in computational mathematics. Moreover, during World Switzerland by E. Stiefel, assisted by H. Rutishauser, AP. War II Stiefel, as an officer of the Swiss Army, had to Speiser, and others. We cover the years from the foundation of some extent worked on computational problems. the Institute for Applied Mathematics at the ETH in 1948 to the When after the war he became aware of the completion of the ERMETH, the electronic computer buih in this institute, in 1956/57. In this period, Stiefel’s team also solved a development of computers and algorithms in other large number of real-world computational problems on another countries, in particular the USA, he realized the computer, Zuse’s 24, rented by the institute. Along with this scientific and economic importance of this research for work went major contributions to numerical analysis by a highly industrialized country, and, through his Rutishauser and Stiefel, and Rutishauser’s seminal work on personal initiative, he achieved the foundation of the compiling programs, which was later followed by his strong Institute for Applied Mathematics. Its aim and commitment in ALGOL. purpose were the introduction of scientific computing We have tried to include some background information and to complement H.R. Schwarz’s article [Scw81] on the same on programmable machines in Switzerland. From the subject. beginning Stiefel was backed up in his basic decisions by a Committee for the Development of Computers in Switzerland and by the Board of Directors 1. -
A Bibliography of Publications of Heinz Rutishauser
A Bibliography of Publications of Heinz Rutishauser Nelson H. F. Beebe University of Utah Department of Mathematics, 110 LCB 155 S 1400 E RM 233 Salt Lake City, UT 84112-0090 USA Tel: +1 801 581 5254 FAX: +1 801 581 4148 E-mail: [email protected], [email protected], [email protected] (Internet) WWW URL: http://www.math.utah.edu/~beebe/ 30 January 2021 Version 1.04 Abstract This bibliography records publications of Heinz Rutishauser (30 Jan- uary 1918–10 November 1970). Title word cross-reference 1 [Nak66a,R Rut66a]. 2 [Nak66b, Rut66b]. $24.50 [G.70]. $3.95 [Str61]. $49 [I.91]. eb(x+acosx) dx [Rut51a, Rut51a]. LR [Rut58b, Rut60g, RS63, TT15]. n [Rut55a, Rut60e]. QD [AN68, Rut62c, Ban63a, Dmi13, AC10]. R4 [Rut47b]. -algorithm [AC10, Dmi13, Rut62c]. -Algorithmus [Ban63a]. -ter [Rut55a, Rut60e]. -th [Rut55a, Rut60e]. -Transformation [Rut60g, Rut58b]. 104 [Rut62a]. 125 [Rut62b]. 135 [Sch69]. 150 [Rut63a, Rut63f]. 1960 [AGG60]. 1969 [Alb71]. 1980 [Ano80]. 1 2 2 [Rut76b, S.78]. 314 [Cul72b].ˇ 34 [H.70]. 34- [H.70]. 367 [Cul72a].ˇ 50 [Gep66]. 6-ter [HRG59a]. 60 [BBG+60, BBG+63c, BBG+63a, BBG+76, BHO+67, Bec71, Cul72b,ˇ Cul72a,ˇ Gep66, GHL67, NBB+60, NBB+63, NBB+65, NBB+97, Rut63g, Rut63h, van19]. 6th [HRG59a]. 6x [Rut65]. Abbildungen [Rut50b, Rut50c]. Accelerating [Rut58a]. according [Rut59e]. Adjoint [EGRS59b, Str61]. again [Rut68c]. ago [Gut07]. Akademie [Gep66]. Akademie-Verlag [Gep66]. Al [Rut59b, Rut59b]. Al-Salam [Rut59b, Rut59b]. Algebra [Rut66d, Rut70a, WR71]. algebraic [Rut59d]. ALGOL [BBG+63c, Bec71, Cul72b,ˇ Cul72a,ˇ Rut13, Gep66, BBG+63c, BBG+63a, BBG+63b, BBG+76, BHO+67, GHL67, NBB+60, NBB+63, NBB+65, Nau68, Rut61, Rut63g, Rut63h, Rut65, Rut13, van19, BBG+60, NBB+97]. -
Some History of Conjugate Gradients and Other Krylov Subspace Methods October 2009 SIAM Applied Linear Algebra Meeting 2009 Dianne P
Some History of Conjugate Gradients and Other Krylov Subspace Methods October 2009 SIAM Applied Linear Algebra Meeting 2009 Dianne P. O'Leary c 2009 1 Some History of Conjugate Gradients Dianne P. O'Leary Computer Science Dept. and Institute for Advanced Computer Studies University of Maryland [email protected] http://www.cs.umd.edu/users/oleary 2 A Tale of 2 Cities Los Angeles, California Z¨urich,Switzerland 3 ... and a Tale of 2 Men 4 ... or maybe 3 Men 5 ... or maybe 5 Men 6 The Conjugate Gradient Algorithm 7 Notation • We solve the linear system Ax∗ = b where A 2 Rn×n is symmetric positive definite and b 2 Rn. • We assume, without loss of generality, that our initial guess for the solution is x(0) = 0 : • We denote the Krylov subspace of dimension m by m−1 Km(A; b) = spanfb; Ab; : : : ; A bg : (k) • Then the conjugate gradient algorithm chooses x 2 Kk(A; b) to minimize (x − x∗)T A(x − x∗). { work per iteration: 1 matrix-vector multiply and a few dot products and combinations of vectors. { storage: 3 vectors, plus original data. 8 Context for the Conjugate Gradient Algorithm Hestenes and Stiefel (1952) presented the conjugate gradient algorithm in the Journal of Research of the NBS. Context: Two existing classes of algorithms • Direct methods, like Gauss elimination, modified a tableau of matrix entries in a systematic way in order to compute the solution. These methods were finite but required a rather large amount of computational effort, with work growing as the cube of the number of unknowns.