Math 715 Applied Mathematics 1 Lecture 1 Introduction to Numerical Linear Algebra

Nathan Albin Nathan Albin

Kansas State University Kansas State University 234 Cardwell Hall 234 Cardwell Hall [email protected] [email protected]

Fall 2014 August 25, 2014

1 / 19 2 / 19

Objectives Table of Contents

1 What is numerical analysis? historical and modern views examples introduce numerical analysis key concepts and numerical linear algebra

describe class structure 2 About this course course objectives topics discussion of course details

3 / 19 4 / 19 Numerical analysis predates computers Numerical analysis

Calculating without calculators! How would you approximate Some classical problems √ rootﬁnding 2 or cos(3) approximating lengths, areas, volumes or solve approximate solution to diﬀerential equations

x3 − 5x2 + 2x − 1 = 0? A modern view Numerical analysis is the area of mathematics and computer Famous names in numerical analysis √ 2 ≈ 1.41421296 science that creates, analyzes, and implements algorithms for Isaac Newton (1643–1727) solving numerically the problems of continuous mathematics. Leonhard Euler (1707–1783) (1800–1600 BCE) Joseph-Louis Lagrange (1736–1813) – Kendall E. Atkinson Carl Friedrich Gauss (1777–1855)

5 / 19 6 / 19

√ √ Example: Computing 2 Example: Computing 2 Newton’s iteration Newton’s iteration

Goal Method Given a guess, x , for the root of f (x) = x2 − 2, compute a better guess, 2 k xk − 2 xk 1 x xk+1 = xk − = + k+1 2xk 2 xk

Idea A few iterations k 0 1 2 3 4 f (x ) 0 k x 1 1.5 1.41666666667 1.41421568627 1.41421356237 f (x) ≈ f (xk ) + f (xk )(x − xk ) = 0 if x = xk+1 := xk − 0 k f (xk ) Actual value √ http://en.wikipedia.org/wiki/Newton’s_method#mediaviewer/File: 2 = 1.41421356237... NewtonIteration_Ani.gif

7 / 19 8 / 19 Example: Computing cos(3) Common themes in numerical analysis Taylor approximation

Taylor expansion about a = π 1 1 1 cos(x) = −1 + (x − π)2 − (x − π)4 + (x − π)6 + ··· 2 24 720 errors are often unavoidable

I modeling error 3 − π ≈ −0.14159 I approximation error I representation error Approximations errors propagate k 0 2 4 6 eﬃciency matters pk (3) -1 -0.98998 -0.98999 -0.98999

Actual value cos(3) = −0.98999 ...

9 / 19 10 / 19

Values are approximated, not exact Eﬀects of precision Exact vs. approximate

√ To 16 significant digits 2 vs. 1.4142135623730951 − cos(3) vs. 0.98999249660044542 cos(3) ≈ −0.9899924966004454 Z 1 x2 e dx vs. 1.462651745907182 Approximate cos(3) with 3 − π ≈ −0.14159 0 k 0 2 4 6 Why approximate? pk (3) -1 -0.98997613595 -0.98999288225841 -0.98999287106756 exact value can be difficult to obtain (e.g., solve e−x = cos(x)) 2 exact value may not be useful in context (e.g., eπ ) Approximate cos(3) with 3 − π ≈ −0.14 only finitely many real numbers can be represented on a computer k 0 2 4 6 exact evaluation can be very slow on a computer pk (3) -1 -0.9902 -0.99021600666667 -0.99021599620898 I Maple, Mathematica, etc. are symbolic (exact) I Matlab is numerical (approximate) I but Matlab can do symbolic, so be careful!

11 / 19 12 / 19 Errors propagate Eﬃciency matters

Supposex ˜ is an approximation of x and |x˜ − x| ≤ 0.5 × 10−5 (5 signiﬁcant digits) Problem |x| 4 3 2 e.g. p(x) = 2x − 3x + 5x + 2x − 1 evaluate p(x) |x˜ − x| x = π, x˜ = 3.1416, ≈ 0.234 × 10−5 |x| Straightforward method Question |f (˜x) − f (x)| p(x) = 2 · x · x · x · x − 3 · x · x · x + 5 · x · x + 2 · x − 1 f (x) = 1000x , ≤ 0.5 × 10? |f (x)|

Answer 10 multiplications 4 additions/subtractions f (x) = 2659365073.955536..., f (˜x) = 2659500032.721517...

3 signiﬁcant digits (loss of signiﬁcance)

13 / 19 14 / 19

Eﬃciency matters Table of Contents

Problem p(x) = 2x4 − 3x3 + 5x2 + 2x − 1 evaluate p(x) 1 What is numerical analysis? Horner’s method historical and modern views examples p(x) = −1 + x · (2 + 5 · x − 3 · x · x + 2 · x · x · x) key concepts = −1 + x · (2 + x · (5 − 3 · x + 2 · x · x)) = −1 + x · (2 + x · (5 + x · (−3 + 2 · x)) 2 About this course course objectives topics discussion of course details 4 multiplications 4 additions/subtractions

15 / 19 16 / 19 Course objectives Math 715 Topics

Linear algebra topics linear systems (Ax = b) In this course, students will least squares problems (minimize kAx − bk) review and practice linear algebra theory. eigenvalue problems (Ax = λx) learn standard techniques for the numerical treatment of common singular value problems (A = UΣV T ) linear algebra problems. obtain a thorough understanding of how these problems are solved and some experience in solving them. examples & applications! acquire the essential knowledge necessary in order to use the methods correctly and eﬀectively in their own research work. Numerical analysis topics algorithmic complexity error propagation and stability direct vs. iterative methods

17 / 19 18 / 19

Course website

http://www.math.ksu.edu/~albin/teaching/math715_fall2014.html

19 / 19