NumericalResearch Linear Matters Algebra in the UK: From Cayley to Exascale Computing February 25, 2009 NickNick Higham Higham DirectorSchool of of Research Mathematics The University of Manchester School of Mathematics [email protected] http://www.ma.man.ac.uk/~higham/ ENUMATH Conference 2011 September 5–9, 2011 Leicester 1 / 6 Outline 1 Matrices 2 Applications 3 History 4 Machines and Computation 5 Towards Exascale MIMS Nick Higham Linear Algebra in the UK 2 / 53 What is a Matrix? Matrix = array = table of numbers. E.g. 2 −4 −11 3 −6 3 6 −17 12 2 22 7 6 7 : 4 1 12 −2 −1 5 3 0 7 1 Penguin Dictionary of Mathematics (4th ed., 2008): A set of quantities arranged in a rectangular array, with certain rules governing their combination. Term “matrix” coined in 1850 by James Joseph Sylvester, FRS (1814–1897). MIMS Nick Higham Linear Algebra in the UK 3 / 53 Correlation Matrix An n × n matrix A for which aij is the correlation between variables i and j. E.g. 2 1:00 −0:28 0:75 −0:09 3 6 −0:28 1:00 0:25 −0:53 7 6 7 : 4 0:75 0:25 1:00 −0:08 5 −0:09 −0:53 −0:08 1:00 Some properties: symmetric, 1s on the diagonal, off-diagonal elements between −1 and 1. MIMS Nick Higham Linear Algebra in the UK 4 / 53 Correlation Matrix An n × n matrix A for which aij is the correlation between variables i and j. E.g. 2 1:00 −0:28 0:75 −0:09 3 6 −0:28 1:00 0:25 −0:53 7 6 7 : 4 0:75 0:25 1:00 −0:08 5 −0:09 −0:53 −0:08 1:00 Some properties: symmetric, 1s on the diagonal, off-diagonal elements between −1 and 1. MIMS Nick Higham Linear Algebra in the UK 4 / 53 Leslie Matrix Model for growth of female portion of an animal population; P. H. Leslie (1945). Model with 4 age classes: 2 0 9 12 6 3 6 1=3 0 0 0 7 L = 6 7 : 4 0 1=2 0 0 5 0 0 1=4 0 Row 1: average births per age class. Subdiagonal: survival rates from one age class to next. MIMS Nick Higham Linear Algebra in the UK 5 / 53 Magic Square Dürer’s Melencolia I (1514) MIMS Nick Higham Linear Algebra in the UK 6 / 53 Magic Square MIMS Nick Higham Linear Algebra in the UK 6 / 53 Matrix (1) Consider set of all Web pages on the internet. Define gij = 1 if page i links to page j: 1 2 3 4 12 0 1 1 0 3 26 1 0 1 0 7 6 7: 34 0 1 1 1 5 4 0 0 1 0 Then scale rows so they sum to 1 (stochastic matrix): 1 2 3 4 12 0 1=2 1=2 0 3 26 1=2 0 1=2 0 7 6 7: 34 0 1=3 1=3 1=3 5 4 0 0 1 0 MIMS Nick Higham Linear Algebra in the UK 7 / 53 Matrix (2) Google matrix for http://www.manchester.ac.uk: MIMS Nick Higham Linear Algebra in the UK 8 / 53 Popularity Number of hits from Google search on exact phrase: 2011 2007 Correlation Matrix 1,030,000 702,000 Magic Square 915,000 418,000 Leslie Matrix 35,900 37,600 Google Matrix 46,000 928 MIMS Nick Higham Linear Algebra in the UK 9 / 53 Outline 1 Matrices 2 Applications 3 History 4 Machines and Computation 5 Towards Exascale MIMS Nick Higham Linear Algebra in the UK 10 / 53 “Matrices offer some of the most powerful techniques in modern mathematics. In the social sciences they provide fresh insights into an astonishing variety of topics.” Penguin, 1986. Chapter 4: Matrices and Matri- mony in Tribal Soci- eties. Nonlinear least squares, Levenberg–Marquardt: T T (J J + λD)d = J r; J 2 R3200×32 for 8 images: Alan Turing Building Panorama MIMS Nick Higham Linear Algebra in the UK 12 / 53 Nonlinear least squares, Levenberg–Marquardt: T T (J J + λD)d = J r; J 2 R3200×32 for 8 images: Alan Turing Building Panorama MIMS Nick Higham Linear Algebra in the UK 12 / 53 Nonlinear least squares, Levenberg–Marquardt: T T (J J + λD)d = J r; J 2 R3200×32 for 8 images: Alan Turing Building Panorama MIMS Nick Higham Linear Algebra in the UK 12 / 53 Alan Turing Building Panorama Nonlinear least squares, Levenberg–Marquardt: T T (J J + λD)d = J r; J 2 R3200×32 for 8 images: MIMS Nick Higham Linear Algebra in the UK 12 / 53 Jpeg Image Format Jpeg compression first converts from RGB to YCbCr colour space where Y = luminance, Cb; Cr = blue, red chrominances, by 2 Y 3 2 0:299 0:587 0:114 3 2 R 3 4 Cb 5 = 4 −0:1687 −0:3313 0:5 5 4 G 5 : Cr 0:5 −0:4187 −0:0813 B Vision has poor response to spatial detail in coloured areas of same luminance ) Cb, Cr can take greater compression. MIMS Nick Higham Linear Algebra in the UK 13 / 53 Outline 1 Matrices 2 Applications 3 History 4 Machines and Computation 5 Towards Exascale MIMS Nick Higham Linear Algebra in the UK 14 / 53 Linear System Jiu Zhang Suanshu (Nine Chapters of the Mathematical Art), around 1 AD. T =M=L = sheaves of rice stalks from top/medium/low grade paddies. Find yield (in dou) for each quality of sheaf, given overall yields as follows: 3T + 2M + L = 39; 2T + 3M + L = 34; T + 2M + 3L = 26: MIMS Nick Higham Linear Algebra in the UK 15 / 53 Cayley and Sylvester Term “matrix” coined in 1850 by James Joseph Sylvester, FRS (1814–1897). Matrix algebra developed by Arthur Cayley, FRS (1821– 1895). Memoir on the Theory of Ma- trices (1858). MIMS Nick Higham Linear Algebra in the UK 16 / 53 Cayley Sylvester Enter Cambridge Trinity College, St. John’s College, University 1838 1831 Wrangler in Tripos Senior Wrangler, Second wrangler, 1837 examinations 1842 Work in London Pupil barrister from Actuary from 1844; 1846; called to the pupil barrister from Bar in 1849 1846; called to the Bar in 1850 Elected Fellow of the 1852 1839 Royal Society President of the 1868–1869 1866–1867 London Mathematical Society Awarded Royal 1882 1880 Society Copley Medal Awarded LMS De 1884 1887 Morgan Medal British Assoc. for the President, 1883 Vice President, Advancement of 1863–1865; President Science of Section A, 1869 Academic Positions Cayley Sylvester • Sadleirian Chair, • UCL, 1838 Cambridge 1863 • U Virginia, 1841 • Royal Military Academy, Woolwich, London, 1855 • Johns Hopkins University 1876 • Savilian Chair of Geometry, Oxford, 1883 MIMS Nick Higham Linear Algebra in the UK 18 / 53 Biographies Tony Crilly, Arthur Cayley: Mathemati- cian Laureate of the Victorian Age, 2006. Karen Hunger Parshall, James Joseph Sylvester. Jewish Mathematician in a Victorian World, 2006. MIMS Nick Higham Linear Algebra in the UK 19 / 53 Matrices in Applied Mathematics Frazer, Duncan & Collar, Aerodynamics Division of NPL: aircraft flutter, matrix structural analysis. Elementary Matrices & Some Applications to Dynamics and Differential Equations, 1938. Emphasizes importance of eA. Arthur Roderick Collar, FRS (1908–1986): “First book to treat matrices as a branch of applied mathematics”. MIMS Nick Higham Linear Algebra in the UK 20 / 53 History of Gaussian Elimination Chinese used variant of GE in Nine Chapters of the Mathematical Art. Gauss developed GE for his work in linear regression theory. GE first appears in Theoria Motus (1809). Variants of GE went by various names in the first half of 20th century: the bordering method, the escalator method (for matrix inversion), the square root method (Cholesky factorization), pivotal condensation, Doolittle’s method and Crout’s method. MIMS Nick Higham Linear Algebra in the UK 21 / 53 1 MIMS Nick Higham Linear Algebra in the UK 22 / 53 1 MIMS Nick Higham Linear Algebra in the UK 22 / 53 1 MIMS Nick Higham Linear Algebra in the UK 22 / 53 1 MIMS Nick Higham Linear Algebra in the UK 22 / 53 1 MIMS Nick Higham Linear Algebra in the UK 22 / 53 MIMS1 Nick Higham Linear Algebra in the UK 22 / 53 MIMS Nick Higham Linear Algebra in the UK 23 / 53 MIMS Nick Higham Linear Algebra in the UK 23 / 53 MIMS Nick Higham Linear Algebra in the UK 23 / 53 MIMS Nick Higham Linear Algebra in the UK 23 / 53 MIMS Nick Higham Linear Algebra in the UK 23 / 53 Outline 1 Matrices 2 Applications 3 History 4 Machines and Computation 5 Towards Exascale MIMS Nick Higham Linear Algebra in the UK 24 / 53 William Thomson (Lord Kelvin, 1824–1907) On a Machine for the Solution of Simultaneous Equations, Proc Roy Soc, 1878. Proposed a system involving tilting plates, cords, pulleys, for 8–10 unknowns. Suggested iterative refinement: “There is, of course, no limit to the accuracy thus obtainable by successive approximations.” Actual system for 9 unknowns built by Wilbur (1936) at MIT. Tapes 60ft long. For 3 sig figs, about 3 times faster than human with desk calculator. MIMS Nick Higham Linear Algebra in the UK 25 / 53 R. R. M. Mallock’s Machine (1933) Experimental analogue m/c (vari- able coil transformers) for solving 6 lin eqns built & tested in 1931. M/c for 10 equations built by Cambridge Instrument Co. Accurate to ≈ 1% of largest component. Cost ≈ £2000. Aware of conditioning issue: “if the equations are ill-conditioned, these errors may be serious”.