<<

NumericalResearch Linear Matters in the UK: From Cayley to Exascale Computing February 25, 2009

NickNick Higham Higham DirectorSchool of of Research The University of Manchester School of Mathematics [email protected] http://www.ma.man.ac.uk/~higham/

ENUMATH Conference 2011 September 5–9, 2011 Leicester 1 / 6 Outline

1 Matrices

2 Applications

3 History

4 Machines and Computation

5 Towards Exascale

MIMS Nick Higham in the UK 2 / 53 What is a ? Matrix = array = table of numbers. E.g.  −4 −11 3 −6   −17 12 2 22    .  1 12 −2 −1  3 0 7 1 Penguin Dictionary of Mathematics (4th ed., 2008): A set of quantities arranged in a rectangular array, with certain rules governing their combination. Term “matrix” coined in 1850 by James Joseph Sylvester, FRS (1814–1897).

MIMS Nick Higham Linear Algebra in the UK 3 / 53 Correlation Matrix

An n × n matrix A for which aij is the correlation between variables i and j. E.g.

 1.00 −0.28 0.75 −0.09   −0.28 1.00 0.25 −0.53    .  0.75 0.25 1.00 −0.08  −0.09 −0.53 −0.08 1.00 Some properties: symmetric, 1s on the diagonal, off-diagonal elements between −1 and 1.

MIMS Nick Higham Linear Algebra in the UK 4 / 53 Correlation Matrix

An n × n matrix A for which aij is the correlation between variables i and j. E.g.

 1.00 −0.28 0.75 −0.09   −0.28 1.00 0.25 −0.53    .  0.75 0.25 1.00 −0.08  −0.09 −0.53 −0.08 1.00 Some properties: symmetric, 1s on the diagonal, off-diagonal elements between −1 and 1.

MIMS Nick Higham Linear Algebra in the UK 4 / 53 Leslie Matrix

Model for growth of female portion of an animal population; P. H. Leslie (1945). Model with 4 age classes:

 0 9 12 6   1/3 0 0 0  L =   .  0 1/2 0 0  0 0 1/4 0

Row 1: average births per age class. Subdiagonal: survival rates from one age class to next.

MIMS Nick Higham Linear Algebra in the UK 5 / 53 Magic Square

Dürer’s Melencolia I (1514)

MIMS Nick Higham Linear Algebra in the UK 6 / 53 Magic Square

MIMS Nick Higham Linear Algebra in the UK 6 / 53 Matrix (1)

Consider set of all Web pages on the internet. Define gij = 1 if page i links to page j: 1 2 3 4 1 0 1 1 0  2 1 0 1 0   . 3 0 1 1 1  4 0 0 1 0 Then scale rows so they sum to 1 (stochastic matrix): 1 2 3 4 1 0 1/2 1/2 0  2 1/2 0 1/2 0   . 3 0 1/3 1/3 1/3  4 0 0 1 0

MIMS Nick Higham Linear Algebra in the UK 7 / 53 Matrix (2)

Google matrix for http://www.manchester.ac.uk:

MIMS Nick Higham Linear Algebra in the UK 8 / 53 Popularity

Number of hits from Google search on exact phrase:

2011 2007 Correlation Matrix 1,030,000 702,000 Magic Square 915,000 418,000 Leslie Matrix 35,900 37,600 Google Matrix 46,000 928

MIMS Nick Higham Linear Algebra in the UK 9 / 53 Outline

1 Matrices

2 Applications

3 History

4 Machines and Computation

5 Towards Exascale

MIMS Nick Higham Linear Algebra in the UK 10 / 53 “Matrices offer some of the most powerful techniques in modern mathematics. In the social they provide fresh insights into an astonishing variety of topics.”

Penguin, 1986.

Chapter 4: Matrices and Matri- mony in Tribal Soci- eties. Nonlinear least squares, Levenberg–Marquardt:

T T (J J + λD)d = J , J ∈ R3200×32 for 8 images.

Alan Turing Building Panorama

MIMS Nick Higham Linear Algebra in the UK 12 / 53 Nonlinear least squares, Levenberg–Marquardt: T T (J J + λD)d = J r, J ∈ R3200×32 for 8 images.

Alan Turing Building Panorama

MIMS Nick Higham Linear Algebra in the UK 12 / 53 Nonlinear least squares, Levenberg–Marquardt:

T T (J J + λD)d = J r, J ∈ R3200×32 for 8 images.

Alan Turing Building Panorama

MIMS Nick Higham Linear Algebra in the UK 12 / 53 Alan Turing Building Panorama

Nonlinear least squares, Levenberg–Marquardt:

T T (J J + λD)d = J r, J ∈ R3200×32 for 8 images.

MIMS Nick Higham Linear Algebra in the UK 12 / 53 Jpeg Image Format

Jpeg compression first converts from RGB to YCbCr colour space where Y = luminance, Cb, Cr = blue, red chrominances, by

 Y   0.299 0.587 0.114   R   Cb  =  −0.1687 −0.3313 0.5   G  . Cr 0.5 −0.4187 −0.0813 B

Vision has poor response to spatial detail in coloured areas of same luminance ⇒ Cb, Cr can take greater compression.

MIMS Nick Higham Linear Algebra in the UK 13 / 53 Outline

1 Matrices

2 Applications

3 History

4 Machines and Computation

5 Towards Exascale

MIMS Nick Higham Linear Algebra in the UK 14 / 53 Linear System

Jiu Zhang Suanshu (Nine Chapters of the Mathematical Art), around 1 AD.

T /M/L = sheaves of rice stalks from top/medium/low grade paddies.

Find yield (in dou) for each quality of sheaf, given overall yields as follows:

3T + 2M + L = 39, 2T + 3M + L = 34, T + 2M + 3L = 26.

MIMS Nick Higham Linear Algebra in the UK 15 / 53 Cayley and Sylvester

Term “matrix” coined in 1850 by James Joseph Sylvester, FRS (1814–1897).

Matrix algebra developed by Arthur Cayley, FRS (1821– 1895). Memoir on the Theory of Ma- trices (1858).

MIMS Nick Higham Linear Algebra in the UK 16 / 53 Cayley Sylvester Enter Cambridge Trinity College, St. John’s College, University 1838 1831 Wrangler in Tripos Senior Wrangler, Second wrangler, 1837 examinations 1842 Work in London Pupil barrister from Actuary from 1844; 1846; called to the pupil barrister from Bar in 1849 1846; called to the Bar in 1850 Elected Fellow of the 1852 1839 Royal Society President of the 1868–1869 1866–1867 London Mathematical Society Awarded Royal 1882 1880 Society Copley Medal Awarded LMS De 1884 1887 Morgan Medal British Assoc. for the President, 1883 Vice President, Advancement of 1863–1865; President of Section A, 1869 Academic Positions

Cayley Sylvester • Sadleirian Chair, • UCL, 1838 Cambridge 1863 • U Virginia, 1841 • Royal Military Academy, Woolwich, London, 1855 • Johns Hopkins University 1876 • Savilian Chair of , Oxford, 1883

MIMS Nick Higham Linear Algebra in the UK 18 / 53 Biographies

Tony Crilly, Arthur Cayley: Mathemati- cian Laureate of the Victorian Age, 2006.

Karen Hunger Parshall, James Joseph Sylvester. Jewish in a Victorian World, 2006.

MIMS Nick Higham Linear Algebra in the UK 19 / 53 Matrices in

Frazer, Duncan & Collar, Aerodynamics Division of NPL: aircraft flutter, matrix . Elementary Matrices & Some Applications to Dynamics and Differential , 1938. Emphasizes importance of eA.

Arthur Roderick Collar, FRS (1908–1986): “First book to treat matrices as a branch of applied mathematics”.

MIMS Nick Higham Linear Algebra in the UK 20 / 53 History of

Chinese used variant of GE in Nine Chapters of the Mathematical Art. Gauss developed GE for his work in theory. GE first appears in Theoria Motus (1809). Variants of GE went by various names in the first half of 20th century: the bordering method, the escalator method (for matrix inversion), the square root method (Cholesky factorization), pivotal condensation, Doolittle’s method and Crout’s method.

MIMS Nick Higham Linear Algebra in the UK 21 / 53 1

MIMS Nick Higham Linear Algebra in the UK 22 / 53 1

MIMS Nick Higham Linear Algebra in the UK 22 / 53 1

MIMS Nick Higham Linear Algebra in the UK 22 / 53 1

MIMS Nick Higham Linear Algebra in the UK 22 / 53 1

MIMS Nick Higham Linear Algebra in the UK 22 / 53 MIMS1 Nick Higham Linear Algebra in the UK 22 / 53 MIMS Nick Higham Linear Algebra in the UK 23 / 53 MIMS Nick Higham Linear Algebra in the UK 23 / 53 MIMS Nick Higham Linear Algebra in the UK 23 / 53 MIMS Nick Higham Linear Algebra in the UK 23 / 53 MIMS Nick Higham Linear Algebra in the UK 23 / 53 Outline

1 Matrices

2 Applications

3 History

4 Machines and Computation

5 Towards Exascale

MIMS Nick Higham Linear Algebra in the UK 24 / 53 William Thomson (Lord Kelvin, 1824–1907)

On a Machine for the Solution of Simultaneous Equations, Proc Roy Soc, 1878.

Proposed a system involving tilting plates, cords, pulleys, for 8–10 unknowns. Suggested iterative refinement: “There is, of course, no limit to the accuracy thus obtainable by successive approximations.” Actual system for 9 unknowns built by Wilbur (1936) at MIT. Tapes 60ft long. For 3 sig figs, about 3 times faster than human with desk calculator.

MIMS Nick Higham Linear Algebra in the UK 25 / 53 R. R. M. Mallock’s Machine (1933)

Experimental analogue m/ (vari- able coil transformers) for solving 6 lin eqns built & tested in 1931.

M/c for 10 equations built by Cambridge Instrument Co. Accurate to ≈ 1% of largest component. Cost ≈ £2000. Aware of conditioning issue: “if the equations are ill-conditioned, these errors may be serious”. Used equilibration and iterative refinement. “The machine could not adequately deal with ill conditioned equations, letting out a very sharp whistle when equilibrium could not be reached” (Croarken).

MIMS Nick Higham Linear Algebra in the UK 26 / 53 Lord Vivian Bowden (1910–1989)

Many years ago we made out of half a dozen transformers a simple and rather inaccurate machine for solving simultaneous equations—the solutions being represented as flux in the cores of the transformers. During the course of our experiments we set the machine to solve the equations— X + Y + Z = 1 X + Y + Z = 2 X + Y + Z = 3 The machine reacted sharply— it blew the main fuse and put all the lights out. — B. V. BOWDEN, The Organization of a Typical Machine (1953)

MIMS Nick Higham Linear Algebra in the UK 27 / 53 Lewis Fry Richardson (1881–1953)

Num soln of PDEs (1910): finite- difference methods, Richardson ex- trapolation (“deferred approach to the limit”). Richardson’s method: xk+1 = xk + αk (Axk − b).

Met. office, 1913–1916; Paisley College, 1929–1940. First to apply mathematics, in particular the method of finite differences, to weather prediction: Weather Prediction by Numerical Process, 1922 (2ed, 2007).

MIMS Nick Higham Linear Algebra in the UK 28 / 53 “Perhaps some day in the dim future it will be possible to advance the computations faster than the weather advances . . . But that is a dream.” “Imagine a large hall like a theatre. . . the walls of this chamber are painted to form a map of the globe.. . . A myriad are at work upon the weather of the part of the map where each sits, but each attends only to one or part of an equation.”

Richardson on Weather Forecasting

“The detailed example of Ch. IX was worked out in France in the intervals of transporting wounded in 1916–1918. During the battle of Champagne in April 1917 the working copy was sent to the rear, where it became lost, to be re-discovered some months later under a heap of coal.”

MIMS Nick Higham Linear Algebra in the UK 29 / 53 “Imagine a large hall like a theatre. . . the walls of this chamber are painted to form a map of the globe.. . . A myriad computers are at work upon the weather of the part of the map where each sits, but each computer attends only to one equation or part of an equation.”

Richardson on Weather Forecasting

“The detailed example of Ch. IX was worked out in France in the intervals of transporting wounded in 1916–1918. During the battle of Champagne in April 1917 the working copy was sent to the rear, where it became lost, to be re-discovered some months later under a heap of coal.” “Perhaps some day in the dim future it will be possible to advance the computations faster than the weather advances . . . But that is a dream.”

MIMS Nick Higham Linear Algebra in the UK 29 / 53 Richardson on Weather Forecasting

“The detailed example of Ch. IX was worked out in France in the intervals of transporting wounded in 1916–1918. During the battle of Champagne in April 1917 the working copy was sent to the rear, where it became lost, to be re-discovered some months later under a heap of coal.” “Perhaps some day in the dim future it will be possible to advance the computations faster than the weather advances . . . But that is a dream.” “Imagine a large hall like a theatre. . . the walls of this chamber are painted to form a map of the globe.. . . A myriad computers are at work upon the weather of the part of the map where each sits, but each computer attends only to one equation or part of an equation.”

MIMS Nick Higham Linear Algebra in the UK 29 / 53 Forecast Factory

Artist’s impression: Francois Schuiten. MIMS Nick Higham Linear Algebra in the UK 30 / 53 Richard Vynne Southwell (1888–1970)

Relaxation method for Ax = b. Examine patterns & relative magni- tudes of residuals, identify best way to reduce them. “Like a game of chess” (Fox). Cgce acceleration: terms overrelax- ation, underrelaxation coined. Multigrid ideas used. Relaxation Methods in Science, 1940. Relaxation Methods in Theoretical , 1946. “Any attempt to mechanize relaxation methods would be a waste of time” (quoted by Young, 1990).

MIMS Nick Higham Linear Algebra in the UK 31 / 53 MIMS Nick Higham Linear Algebra in the UK 32 / 53 2D Flow Round Aerofoil

MIMS Nick Higham Linear Algebra in the UK 33 / 53 (1918–1992)

PhD (1942) with Southwell. “Outstand- ing exponent of relaxation method”.

Mathematics Division, NPL, 1945–1956. Set up Oxford Univ Computing Lab., 1957. 1950s papers on A−1, Ax = b. An Introduction to , 1964. Early textbook treatment of com- putational aspects. First textbook to describe Wilkin- son’s backward error analysis.

MIMS Nick Higham Linear Algebra in the UK 34 / 53 James Hardy Wilkinson (1919–1986)

MIMS Nick Higham Linear Algebra in the UK 35 / 53 Turing Wilkinson Enter Cambridge Kings College, Trinity College, 1936 University 1931 Second World War Bletchley Park; Ordnance Board of breaks the Enigma Ministry of Supply code. National Physical 1945: Senior 1946: Working half time Laboratory Scientific Officer each with Turing and Desk Computing Group

Proposal for 1948–51: Head of Pilot Development . . . of ACE group. ACE 1951–56: Works on exploitation of Pilot ACE for solving scientific problems Elected Fellow of 1951 1969 the Royal Society Gaussian Elimination at NPL

1946 Fox, Goodwin, Turing & Wilkinson solve 18 × 18 system on desk calculator in 2 weeks. Obtained small residual. 1948 Fox, Huskey & Wilkinson give empirical evidence in support of GE, even for ill conditioned matri- ces. 1948 Wilkinson’s confidential NPL report on the Au- tomatic Computing Engine (ACE) gives program implementing GE with partial pivoting and itera- tive refinement. 1963 Wilkinson’s backward error analysis: Rounding Errors in Algebraic Processes.

MIMS Nick Higham Linear Algebra in the UK 37 / 53 MIMS Nick Higham Linear Algebra in the UK 38 / 53 Turing’s Paper

Rounding-Off Errors in Matrix Processes, Quart. J. Mech. and Applied Math., 1948.

Proves ∃ce of A = LU; shows GE computes it. Introduces term “”. Uses term “preconditioning”. Describes iterative refinement for linear systems. Exploits backward error ideas.

MIMS Nick Higham Linear Algebra in the UK 39 / 53 Pilot Ace

1950 The Pilot ACE at the National Physical Laboratory runs for the first time.

MIMS Nick Higham Linear Algebra in the UK 40 / 53 Linear Equation Solvers on the Pilot ACE

An interesting feature of the codes is that they made a very intensive use of subroutines; the addition of two vectors, multiplication of a vector by a , inner products, etc., were all coded in this way — J. H. Wilkinson

MIMS Nick Higham Linear Algebra in the UK 41 / 53 Daily Mirror, 1952

MIMS Nick Higham Linear Algebra in the UK 42 / 53 MIMS Nick Higham Linear Algebra in the UK 43 / 53 on Flutter

Olga Taussky, in Frazer’s group at NPL, 1940s. 6 × 6 QEPs from flutter in supersonic aircraft. Used Gershgorin. Peter Lancaster, English Electric Co., 1950s. QEPs, 2 ≤ n ≤ 20.

MIMS Nick Higham Linear Algebra in the UK 44 / 53 Outline

1 Matrices

2 Applications

3 History

4 Machines and Computation

5 Towards Exascale

MIMS Nick Higham Linear Algebra in the UK 45 / 53 TOP500, http://www.top500.org

Ranks world’s fastest computers by their performance on the LINPACK benchmark. Performance measured in flops (floating point operations) per second. Updated twice a year: SC (Nov, USA) and Germany (June). User can tune provided code (C and MPI). Must obtain correct result (small residual). Must not use Strassen’s method.

Tera: 1012, Peta: 1015, Exa: 1018.

MIMS Nick Higham Linear Algebra in the UK 46 / 53 Rmax % of Power GFlops/ Site Computer Country Cores [Pflops] Peak [MW] Watt

RIKEN Advanced Inst K Computer Fujitsu SPARC64 1 Japan 548,352 8.16 93 9.9 824 for Comp Sci VIIIfx + custom Nat. Tianhe-1A, NUDT 2 China 186,368 2.57 55 4.04 636 Center in Tianjin Intel + Nvidia GPU + custom DOE / OS Jaguar, Cray 3 USA 224,162 1.76 75 7.0 251 Oak Ridge Nat Lab AMD + custom

Nat. Supercomputer Nebulea, Dawning 4 China 120,640 1.27 43 2.58 493 Center in Shenzhen Intel + Nvidia GPU + IB

GSIC Center, Tokyo Tusbame 2.0, HP 5 Japan 73,278 1.19 52 1.40 850 Institute of Technology Intel + Nvidia GPU + IB

DOE / NNSA Cielo, Cray 6 USA 142,272 1.11 81 3.98 279 LANL & SNL AMD + custom NASA Ames Research Plelades SGI Altix ICE 7 USA 111,104 1.09 83 4.10 265 Center/NAS 8200EX/8400EX + IB DOE / OS Hopper, Cray 8 Lawrence Berkeley Nat USA 153,408 1.054 82 2.91 362 AMD + custom Lab Commissariat a Tera-10, Bull 9 l'Energie Atomique France 138,368 1.050 84 4.59 229 Intel + IB (CEA) DOE / NNSA Roadrunner, IBM 10 USA 122,400 1.04 76 2.35 446 Los Alamos Nat Lab AMD + Cell GPU + IB

MIMS Nick Higham Linear Algebra in the UK 47 / 53 28 in the UK

Rmax Rank Site Computer Cores Tflop/s 24 University of Edinburgh Cray XE6 12-core 2.1 GHz 44376 279 65 Atomic Weapons Establishment BullxB500 Cluster, Xeon X56xx 2.8Ghz, QDR Infiniband 12936 124 69 ECMWF Power 575, p6 4.7 GHz, Infiniband 8320 115 70 ECMWF Power 575, p6 4.7 GHz, Infiniband 8320 115 93 University of Edinburgh Cray XT4, 2.3 GHz 12288 95 154 University of Southampton iDataPlex, XeonQC 2.26 GHz,Ifband, Windows HPC2008 R2 8000 66 160 IT Service Provider Cluster Platform 4000 BL685c G7, Opteron12C 2.2 Ghz, GigE 14556 65 186 IT Service Provider Cluster Platform 3000 BL460c G7, Xeon X5670 2.93 Ghz, GigE 9768 59 190 Computacenter (UK) LTD Cluster Platform 3000 BL460c G1, Xeon L5420 2.5 GHz, GigE 11280 58 191 Classified xSeries x3650 Cluster Xeon QC GT 2.66 GHz, Infiniband 6368 58 211 Classified BladeCenterHS22 Cluster, WM Xeon 6-core 2.66Ghz,Ifband 5880 55 212 Classified BladeCenterHS22 Cluster, WM Xeon 6-core 2.66Ghz,Ifband 5880 55 213 Classified BladeCenterHS22 Cluster, WM Xeon 6-core 2.66Ghz,Ifband 5880 55 228 IT Service Provider Cluster Platform 4000 BL685c G7, Opteron12C 2.1 Ghz, GigE 12552 54 233 Financial Institution iDataPlex, Xeon X56xx 6C 2.66 GHz, GigE 9480 53 234 Financial Institution iDataPlex, Xeon X56xx 6C 2.66 GHz, GigE 9480 53 278 UK Meteorological Office Power 575, p6 4.7 GHz, Infiniband 3520 51 279 UK Meteorological Office Power 575, p6 4.7 GHz, Infiniband 3520 51 Cluster Platform 3000 BL460c, Xeon 54xx 3.0GHz, 339 Computacenter (UK) LTD GigEthernet 7560 47 351 AsdaStores BladeCenterHS22 Cluster, WM Xeon 6-core 2.93Ghz, GigE 8352 47 365 Financial Services xSeriesx3650M2 Cluster, Xeon QC E55xx 2.53 Ghz, GigE 8096 46 404 Financial Institution BladeCenterHS22 Cluster, Xeon QC GT 2.53 GHz, GigEthernet 7872 44 405 Financial Institution BladeCenterHS22 Cluster, Xeon QC GT 2.53 GHz, GigEthernet 7872 44 415 Bank xSeries x3650M3, Xeon X56xx 2.93 GHz, GigE 7728 43 416 Bank xSeries x3650M3, Xeon X56xx 2.93 GHz, GigE 7728 43 482 IT Service Provider Cluster Platform 3000 BL460c G6, Xeon L5520 2.26 GHz, GigE 8568 40 484 IT Service Provider Cluster Platform 3000 BL460c G6, Xeon X5670 2.93 GHz, 10G 4392 40

MIMS Nick Higham Linear Algebra in the UK 48 / 53 59 PFlop/s 100000000100 Pflop/s

10 Pflop/s 8.2 PFlop/s 10000000

1000000 1 Pflop/s

100 Tflop/s 100000 SUM 41 TFlop/s

10 Tflop/s 10000 N=1 1 Tflop/s 1000 1.17 TFlop/s 6-8 years

100 Gflop/s 100 N=500 59.7 GFlop/s 10 Gflop/s 10 My Laptop (6 Gflop/s)

1 Gflop/s 1 My iPad2 (620 Mflop/s) 400 MFlop/s 100 Mflop/s 0.1 1993 1995 1997 1999 2001 2003 2005 2007 2009 2011

MIMS Nick Higham Linear Algebra in the UK 49 / 53 MIMS Nick Higham Linear Algebra in the UK 50 / 53 Need for Software Redesign

Fast ascent terascale → petascale → exascale. Extreme parallelism & hybrid design. Limits on power/clock speed. Reducing communication essential. Fault tolerance required.

MIMS Nick Higham Linear Algebra in the UK 51 / 53 Issues for Exascale

Synchronization-reducing . Communication-reducing algorithms. Fault resilient algorithms. Mixed precision methods. Reproducibility of results.

Rethink our approach to Ax = b.

MIMS Nick Higham Linear Algebra in the UK 52 / 53 Asynchronous Jacobi EPSRC project Novel Asynchronous Algorithms and Software for Large Sparse Systems, 2010–2014 (Manchester, Edinburgh, Hull, Leeds & Strathclyde).

Avge iterations 250

200

150

100

50 0 2 4 6 8 10 12 14 16 18 20 22 24 Time (secs) 8000

7000

6000

5000

4000 0 2 4 6 8 10 12 14 16 18 20 22 24

MIMS Nick Higham Linear Algebra in the UK 53 / 53 ReferencesI

D. J. Albers and G. L. Alexanderson, editors. Mathematical People: Profiles and Interviews. Birkhäuser, Boston, MA, USA, 1985. B. V. Bowden. The organization of a typical machine. In B. V. Bowden, editor, Faster than Thought: A Symposium on Digital Computing Machines, pages 67–77. Pitman, London, 1953. M. Brown and D. G. Lowe. Automatic panoramic image stitching using invariant features. Int. J. Computer Vision, 74(1):59–73, 2007.

MIMS Nick Higham Linear Algebra in the UK 44 / 53 ReferencesII

T. Crilly. Arthur Cayley: Mathematician Laureate of the Victorian Age. Johns Hopkins University Press, Baltimore, MD, USA, 2006. M. R. Cullen. Linear Models in . Ellis Horwood, Chichester, 1985. J. J. Dongarra, P. Luszczek, and A. Petitet. The LINPACK benchmark: Past, present and future. Concurrency and Computation: Practice and Experience, 15:803–820, 2003.

MIMS Nick Higham Linear Algebra in the UK 45 / 53 References III

L. Fox. Early in the . In S. G. Nash, editor, A History of Scientific Computing, pages 280–300. Addison-Wesley, Reading, MA, USA, 1990. R. A. Frazer, W. J. Duncan, and A. R. Collar. Elementary Matrices and Some Applications to Dynamics and Differential Equations. Cambridge University Press, Cambridge, UK, 1938. 1963 printing. J. F. Grcar. How ordinary elimination became Gaussian elimination. Historia Mathematica, 38(2):163–218, 2011.

MIMS Nick Higham Linear Algebra in the UK 46 / 53 ReferencesIV

J. F. Grcar. Mathematics of Gaussian elimination. Notices Amer. Math. Soc., 58(8):782–792, 2011. D. A. Grier. When Computers Were Human. Princeton University Press, Princeton, NJ, USA, 2005. D. J. Higham and A. Taylor. The sleekest link . Mathematics Today, 39(6):192–197, 2003.

MIMS Nick Higham Linear Algebra in the UK 47 / 53 ReferencesV

N. J. Higham. An interview with Peter Lancaster. Numerical Analysis Report No. 468, Manchester Centre for Computational Mathematics, Manchester, England, June 2005. D. C. Joyce. Survey of extrapolation processes in numerical analysis.

SIAM Rev., 13(4):435–490, 1971. JPEG file interchange format, version 1.02. http: //www.w3.org/Graphics/JPEG/jfif3.pdf.

MIMS Nick Higham Linear Algebra in the UK 48 / 53 ReferencesVI

P. Lancaster. Lambda-Matrices and Vibrating Systems. Pergamon Press, Oxford, 1966. Reprinted by Dover, New York, 2002. A. N. Langville and C. D. Meyer. Google’s PageRank and Beyond: The Science of Search Engine Rankings. Princeton University Press, Princeton, NJ, USA, 2006. R. R. M. Mallock. An electrical calculating machine. Proc. Royal Society, Series A, 140(841):457–483, 1933.

MIMS Nick Higham Linear Algebra in the UK 49 / 53 References VII

National Physical Laboratory. Modern Computing Methods. Number 16 in Notes on Applied Science. Her Majesty’s Stationery Office, London, 1957. K. H. Parshall. James Joseph Sylvester. Jewish Mathematician in a Victorian World. Johns Hopkins University Press, Baltimore, MD, USA, 2006. L. F. Richardson. Weather Prediction by Numerical Process. Cambridge University Press, Cambridge, UK, 1922.

MIMS Nick Higham Linear Algebra in the UK 50 / 53 References VIII

R. V. Southwell. Relaxation Methods in Engineering Science: A Treatise on Approximate Computation. Oxford University Press, 1940. R. V. Southwell. Relaxation Methods in Theoretical Physics: A Continuation of the Treatise ‘Relaxation Methods in Engineering Science’. Oxford University Press, 1946. O. Taussky. How I became a torchbearer for matrix theory. Amer. Math. Monthly, 95(9):801–812, Nov. 1988.

MIMS Nick Higham Linear Algebra in the UK 51 / 53 ReferencesIX

O. Taussky and J. Todd. Cholesky, Toeplitz and the triangular factorization of symmetric matrices. Numer. Algorithms, 41:197–202, 2006. W. Thomson. On a machine for the solution of simultaneous equations. Proc. Royal Society, 28:111–113, 1878-1879. J. B. Wilbur. The mechanical solution of simultaneous equations. J. Franklin Inst., 222(6):715–724, 1936.

MIMS Nick Higham Linear Algebra in the UK 52 / 53 ReferencesX

D. M. Young. A historical review of iterative methods. In S. G. Nash, editor, A History of Scientific Computing, pages 180–194. Addison-Wesley, Reading, MA, USA, 1990.

MIMS Nick Higham Linear Algebra in the UK 53 / 53