sign in sign up
<<
Home , Numerical analysis

Lecture 1 Introduction to Numerical Numerical Computing?

J. Chaudhry (Zeb) Numerical ?

Department of and University of New Mexico

J. Chaudhry (Zeb) (UNM) Math/CS 375 1 / 29 J. Chaudhry (Zeb) (UNM) Math/CS 375 2 / 29 Numerical Computation vs. Symbolic Computation Analytic Solution vs. Numerical Solution

Analytic Solution (a.k.a. symbolic): The exact numerical or symbolic representation of the solution Numerical Computation: involve numbers directly I may use special characters such as π, e, or tan (83) I manipulate numbers to produce a numerical result Numerical Solution: The computational representation of the solution Symbolic Computation: symbols represent numbers I entirely numerical I manipulate symbols according to mathematical rules to produce a symbolic result Example (analytic) Example (numerical) Example (numerical) Example (symbolic) 1 4 0.25 (17.36)2 − 1 x2 − 1 = 16.36 = x − 1 1 0.33333 ... (?) 17.36 + 1 x + 1 3 3.14159 ... (?) π 0.88472 ... (?) tan (83)

J. Chaudhry (Zeb) (UNM) Math/CS 375 3 / 29 J. Chaudhry (Zeb) (UNM) Math/CS 375 4 / 29 Numerical Computation and Approximation Method vs. vs. Implementation

Numerical Approximation is needed to carry out the steps in the numerical calculation. The overall process is a numerical computation. Method: a general (mathematical) framework describing the solution process Example (symbolic computation, numerical solution) Algorithm: a detailed description of executing the method 1 1 1 1 Implementation: a particular instantiation of the algorithm + + − 1 = = 0.083333333 ... 2 3 4 12 Is it a “good” method? Example (numerical computation, numerical approximation) Is it a robust algorithm? Is it a fast implementation? 0.500 + 0.333 + 0.250 − 1.000 = 0.083

J. Chaudhry (Zeb) (UNM) Math/CS 375 5 / 29 J. Chaudhry (Zeb) (UNM) Math/CS 375 6 / 29 The Big Theme History: Numerical

date to 1650 BC: The Rhind Papyrus of ancient Egypt contains 85 problems; many use numerical algorithms (T. Chartier, Davidson)

Approximates π with (8/9)2 4 3.1605 ∗ ≈

Accuracy Cost

J. Chaudhry (Zeb) (UNM) Math/CS 375 7 / 29 J. Chaudhry (Zeb) (UNM) Math/CS 375 8 / 29 History: Archimedes History: Method of Exhaustion

A circle is not a polygon 287-212BC developed the ”Method of A circle is a polygon with an infinite number of sides Exhaustion” Cn = circumference of n-sided polygon inscribed in circle of radius 1/2 Method for determining π limn Cn = π I find the length of the permieter of a → Archimedes deterimined polygon inscribed inside a circle of ∞ radius 1/2 223 22 I find the perimiter of a polygon <π < circumscribed outside a circle of 71 7 radius 1/2 3.1408 <π < 3.1429 I the value of π is between these two lengths two places of accuracy.... http://www.pbs.org/wgbh/nova/archimedes/pi.html

J. Chaudhry (Zeb) (UNM) Math/CS 375 9 / 29 J. Chaudhry (Zeb) (UNM) Math/CS 375 10 / 29 History: Method of Machin Numerical Analysis

Around 1700, John Machin discovered the trig identity

1 1 Definition (Trefethen) π = 16 arctan ( ) − 4 arctan ( ) 5 239 Study of algorithms for the problems of continuous mathematics

Led to calculation of the first 100 digits of π We’ve been doing this since (and before!) Uses the Taylor of arctan in the algorithm for calculating a definite x3 x5 x7 Newton’s Method arctan (x) = x − + − ... 3 5 7 Taylor’s Series expansion + truncation Used until 1973 to find the first Million digits

J. Chaudhry (Zeb) (UNM) Math/CS 375 11 / 29 J. Chaudhry (Zeb) (UNM) Math/CS 375 12 / 29 Big Questions Numerical Analysis

How algorithms work and how they fail A Numerical Analyst needs Why algorithms work and why they fail computational knowledge (e.g. programming skills) understanding of the application (physical intuition for validation) Connects mathematics and science mathematical ability to construct a meaningful algorithm Need mathematical theory, , and scientific inquiry

J. Chaudhry (Zeb) (UNM) Math/CS 375 13 / 29 J. Chaudhry (Zeb) (UNM) Math/CS 375 14 / 29 Numerical Analysis Numerical Analysis

Why? Numerical methods improve scientific simulation Numerical focus: Some disasters attributable to bad numerical computing (Douglas Arnold) Approximation An approximate solution is sought. How close is this to the I The Patriot Missile failure, in Dharan, Saudi Arabia, on February 25, 1991 desired solution? which resulted in 28 deaths, is ultimately attributable to poor handling of Efficiency How fast and cheap (memory) can we compute a solution? rounding errors. Stability Is the solution sensitive to small variations in the problem setup? I The explosion of the Ariane 5 rocket just after lift-off on its maiden voyage off French Guiana, on June 4, 1996, was ultimately the consequence of a Error What is the role of finite precision of our ? simple overflow. I The sinking of the Sleipner A offshore platform in Gandsfjorden near Stavanger, Norway, on August 23, 1991, resulted in a loss of nearly one billion dollars. It was found to be the result of inaccurate finite element analysis.

J. Chaudhry (Zeb) (UNM) Math/CS 375 15 / 29 J. Chaudhry (Zeb) (UNM) Math/CS 375 16 / 29 I thought we were studying ”Numerical Computing”?! Applications: Mathematics, Engineering, ... Numerical analysis is the study of numerical computing

Numerical Analysis: understanding general behavior of numerical In Engineering computing Numerical Computing: understanding how to apply certain methods to solve specific tasks As computational scientists, we need to understand the concepts of numerical analysis and implementation aspects of the numerical computing We thus focus on Matlab implementation I fast learning curve Biomolecular Rocket Crack I quick time-to-production: low development times Systems Simulations propagation I a major development environment in scientific computing I integrated graphics Errors in computation Specific methods for solving linear and nonlinear systems, root finding, integrating, , etc.

J. Chaudhry (Zeb) (UNM) Math/CS 375 17 / 29 J. Chaudhry (Zeb) (UNM) Math/CS 375 18 / 29 Applications: Mathematics, Engineering, Computer Course Info Science...

In Computer Science? http://math.unm.edu/˜jehanzeb/math375fa19.html : Training deep neural networks (iterative methods) AI: transitions, state systems, patterns (eigenvalues, linear ) Book: Numerical Mathematics and Computing, Seventh : Google , Amazon recommendations Edition, by Cheney and Kincaid (or 6th) (eigenvalues, , sparse matrices, iterative methods) Graphics: image compression, representation of curves/surfaces/lighting (interpolation, differentiation, etc) Zeb’s contact: Security: ssh (random numbers) Office: SMLC 310 Economics/Finance: modeling/simulation of financial data (monte carlo) Office Hours: Tuesday: 11:00 am - 1:00 pm, Wednesday: 10:00 am - 11:00 am Scientific computing: design of algorithms for high email: [email protected] performance/parallel computing.

J. Chaudhry (Zeb) (UNM) Math/CS 375 19 / 29 J. Chaudhry (Zeb) (UNM) Math/CS 375 20 / 29 Assessment Finally. . .

http://math.unm.edu/ jehanzeb/math375fa19.html ˜ http://math.unm.edu/˜jehanzeb/math375fa19.html Homeworks 50% Grades: Midterm 20% Final Exam 30% Homework: Schedule and Notes: Assigned approximately weekly on the web submitted as a report everything is tentative source code has to be included final exam is fixed no graphics by hand Grades posted on UNM Learn accepted for 50% credit up to one week later no dropped scores (or maybe at most one) collaborate, groups of 2 Questions? cite collaborators at the top, submit one report

J. Chaudhry (Zeb) (UNM) Math/CS 375 21 / 29 J. Chaudhry (Zeb) (UNM) Math/CS 375 22 / 29 Preliminaries... Typical Process

Goal A computational experiment... x Find f 0(1.0) for f (x) = e . relative vs absolute error Matlab Problem Goals Don’t (want to need to) know f 0(x) explicitly. To think like a numerical analyst Understanding error in numerical computing Method Utilizing Matlab Use definition. f (x + h) − f (x) f 0(x) = lim h 0 h →

J. Chaudhry (Zeb) (UNM) Math/CS 375 23 / 29 J. Chaudhry (Zeb) (UNM) Math/CS 375 24 / 29 Typical Process continued Measuring Error

Method Here we used the absolute error: Use definition. f (x + h) − f (x) Error(xa) = |xt − xa| f 0(x) = lim h 0 h = |true value − approximate value| → This doesn’t tell the whole story. For example, if the values are large, like Listing 2: Implementation (Matlab) Listing 1: Algorithm (psuedocode) billions, then an Error of 100 is small. If the values are smaller, say 1 f= @(x) (exp(x)); 1 around 10, then an Error of 100 is large. We need the relative error: 2 df= @(x) (exp(x)); 2 h=1 3 h=1; xt − xa 3 x=1 ( ) = 4 x=1; Rel xa 4 for i=1 to 20 do xt 5 for i=1:20 5 h = h/2 true value − approximate value 6 h=h/2; = 6 y = (f (x + h) − f (x))/h 7 y=(f(x+h) - f(x))/h; true value 7 err = |(f 0(x) − y)| 8 err(i) = abs(df(x)-y); 8 end 9 end More on types of errors in future...

J. Chaudhry (Zeb) (UNM) Math/CS 375 25 / 29 J. Chaudhry (Zeb) (UNM) Math/CS 375 26 / 29 What is MATLAB? Why Matlab?

The Good: Fast development times both a computing environment and a language no compiling, easy debugging initially developed as an easy interface to LAPACK: Linear Algebra Package ( libraries) accessible syntax and language constructs MATrix LABoratory in-house graphics capabilities Written in . For matrix computations, it calls C/Fortran libraries == Fast tons of basic ”libraries” or functions available Matlab + “Toolboxes” many more complicated “toolboxes” can be added I Symbolic Math Toolbox: mathematical manipulation of symbols The Bad: I Partial Differential Toolbox: tools for solving PDEs in 2- small coding mistakes can result in slow code I Statistics Toolbox: statistical data analysis I Image processing toolbox: and image analysis loops are computationally intensive I toolbox: computational molecular biology language is limited: no templates etc. I : application development The Ugly: I many many more. http://www.mathworks.com proprietary (but the language format is open) expensive (but free for UNM students/faculty?) the open source substitute, Octave, is not fully compatible

J. Chaudhry (Zeb) (UNM) Math/CS 375 27 / 29 J. Chaudhry (Zeb) (UNM) Math/CS 375 28 / 29 What is Python?

Full fledged, easy to use The Good: All the good stuff that Matlab has! Large range of libraries Free to use forever! Language has full object oriented support Jupyter notebooks are great for displaying text, code and figures in the same file The Bad: Installing it and the associated libraries can be a pain (although many good installers now available) Slightly steeper learning curve compared to Matlab

J. Chaudhry (Zeb) (UNM) Math/CS 375 29 / 29