Data types in science Discrete data (data tables) Experiment Interpolation Observations Calculations Continuous data Analytics functions Analytic solutions 1 2 From continuous to discrete … From discrete to continuous? ?? soon after hurricane Isabel (September 2003) 3 4 If you think that the data values f in the data table What do we want from discrete sets i are free from errors, of data? then interpolation lets you find an approximate 5 value for the function f(x) at any point x within the Data points interval x1...xn. 4 Quite often we need to know function values at 3 any arbitrary point x y(x) 2 Can we generate values 1 for any x between x1 and xn from a data table? 0 0123456 x 5 6 1 Key point!!! Applications of approximating functions The idea of interpolation is to select a function g(x) such that Data points 1. g(xi)=fi for each data 4 point i 2. this function is a good 3 approximation for any interpolation differentiation integration other x between y(x) 2 original data points 1 0123456 x 7 8 What is a good approximation? Important to remember What can we consider as Interpolation ≡ Approximation a good approximation to There is no exact and unique solution the original data if we do The actual function is NOT known and CANNOT not know the original be determined from the tabular data. function? Data points may be interpolated by an infinite number of functions 9 10 Step 1: selecting a function g(x) Two step procedure We should have a guideline to select a Select a function g(x) reasonable function g(x). We need some ideas about data!!! Find coefficients g(x) may have some standard form or be specific for the problem 11 12 2 Linear combination is the most common Some ideas for selecting g(x) form of g(x) & linear combination of elementary functions, or Most interpolation methods trigonometric, or exponential functions, or are grounded on rational functions, … ‘smoothness’of g(x) a h (x) a h (x) a h (x) ... interpolated functions. = 1 1 + 2 2 + 3 3 + However, it does not work all the time Three of most common approximating functions & Polynomials Practical approach, i.e. & Trigonometric functions what physics lies beneath & Exponential functions the data 13 14 Approximating functions should have following properties Linear Interpolation: Idea & It should be easy to determine & It should be easy to evaluate & It should be easy to differentiate & It should be easy to integrate 15 16 Example: C++ Linear Interpolation: coefficients double int1(double x, double xi[], double yi[], int imax) { double y; int j; // if x is ouside the xi[] interval if (x <= xi[0]) return y = yi[0]; if (x >= xi[imax-1]) return y = yi[imax-1]; // loop to find j so that x[j-1] < x < x[j] j = 0; while (j <= imax-1) { if (xi[j] >= x) break; j = j + 1; } y = yi[j-1]+(yi[j]-yi[j-1])*(x-xi[j-1])/(xi[j]-xi[j-1]); return y; } 17 18 bisection approach is much more efficient to search an array 3 Linear interpolation: conclusions 1.0 The linear interpolation may work well for very smooth functions when the second and higher derivatives are 0.5 small. It is worthwhile to note that for the each data interval ) 2 0.0 one has a different set of coefficients a0 and a1. sin(x This is the principal difference from data fitting where -0.5 the same function, with the same coefficients, is used f(x)=sin(x2) Data points to fit the data points on the whole interval [x1,xn]. linear interpolation -1.0 We may improve quality of linear interpolation by increasing number of data points x on the interval. 0123 i x HOWEVER!!! It is much better to use higher-odrder 19 interpolations. 20 Linear vs. Quadratic interpolations Polynomial Interpolation example from F.S.Acton “Numerical methods that work” “A table of sin(x) covering the first quadrant, for example, requires 541 pages if it is to be linearly interpolable to eight decimal places. If quadratic interpolation is used, the same table takes only one page having entries at one-degree intervals.” The number of data points minus one defines the order of interpolation. Thus, linear (or two-point interpolation) is the first order interpolation 21 22 Properties of polynomials System of equations for the second order Weierstrass theorem: If f(x) is a continuous function in the closed interval a ≤ x ≤ b then for every ε > 0 there exists a polynomial Pn(x), where the value on n depends on the value of ε , such that for all x in the closed interval a ≤ x ≤ b Ö We need three points for the second order interpolation Pn (x) − f (x) < ε Ö Freedom to choose points? Ö This system can be solved analytically 23 24 4 Lagrange Interpolation: C++ Solutions for the second order double polint(double x, double xi[], double yi[], int isize, int npoints) { double lambda[isize]; double y; int j, is, il; // check order of interpolation if (npoints > isize) npoints = isize; and any arbitrary order (Lagrange interpolation) // if x is ouside the xi[] interval if (x <= xi[0]) return y = yi[0]; if (x >= xi[isize-1]) return y = yi[isize-1]; f (x) ≈ f1λ1(x) + f2λ2 (x) + fnλn (x) K // loop to find j so that x[j-1] < x < x[j] n x − x j = 0; (x) j λi = ∏ while (j <= isize-1) j(≠i)=1 xi − x j { if (xi[j] >= x) break; j = j + 1; 25 } 26 Example: C++ (cont.) 2.5 // shift j to correspond to (npoint-1)th interpolation f(x) j = j - npoints/2; 2.0 data points 1-st order // if j is ouside of the range [0, ... isize-1] 1.5 3-rd order if (j < 0) j=0; 5-th order ] if (j+npoints-1 > isize-1 ) j=isize-npoints; 2 1.0 7-th order y = 0.0; for (is = j; is <= j+npoints-1; is = is+1) 0.5 { 0.0 lambda[is] = 1.0; for (il = j; il <= j+ npoints-1; il = il + 1) -0.5 { cos(3x)/[0.4+(x-2) -1.0 if(il != is) lambda[is] = lambda[is]* (x-xi[il])/(xi[is]-xi[il]); -1.5 } y = y + yi[is]*lambda[is]; -2.0 } 0123 return y; x } 27 28 note on Lagrange interpolation Important! It is possible to use Lagrange formula Moving from the first -order to the third and 5th order straightforwardly, like in the example above improves interpolated values to the original function. However, the 7th order interpolation instead being There is a better algorithm - Neville’s algorithm closer to the function f(x) produces wild oscillations. (for constructing the same interpolating This situation is not uncommon for high-order polynomial) polynomial interpolation. Sometimes Neville’s algorithm confused with Rule of thumb: do not use high order interpolation. Aitken’s algorithm Fifth order may be considered as a practical limit. (the latter now considered obsolete). If you believe that the accuracy of the 5th order interpolation is not sufficient for you, then you should rather consider some other method of interpolation. 29 30 5 Cubic Spline Spline vs. Polynomials Ö The idea of spline interpolation is reminiscent of very old mechanical devices used by draftsmen to Ö One of the principal drawbacks of the polynomial get a smooth shape. interpolation is related to discontinuity of derivatives at Ö It is like securing a strip of elastic material (metal data points xj. or plastic ruler) between knots (or nails). Ö The procedure for deriving coefficients of spline Ö The final shape is quite smooth. interpolations uses information from all data points, Ö Cubic spline interpolation is used in most plotting i.e. nonlocal information to guarantee global smoothness software. (cubic spline gives most smooth result) in the interpolated function up to some order of derivatives. 31 32 Equations Coefficients for spline interpolation the interpolated function on [xj,xj+1] interval is presented in a cubic spline form For the each interval we need to have a set of three parameters bj, cj and dj. Since there are (n-1) intervals, one has to have 3n-3 equations for deriving the coefficients for j=1,n-1. What is different from polynomial interpolation? The fact that gj(xj)=fj(xj) imposes (n-1) equations. … the way we are looking for the coefficients! Polynomial interpolation: 3 coefficient for 4 data points Spline interpolation: 3 coefficients for the each interval 33 34 Central idea to spline interpolation Now we have more equations (n-1) + 2(n-2) = 3n-5 equations the interpolated function g(x) has continuous the first Data points and second derivatives at each of the n-2 interior 4 points xj. 3 Data points 4 y(x) 2 3 1 y(x) 2 0123456 but we need 3n-3 equations x 1 0123456 x 35 36 6 a little math … Cubic spline boundary conditions c d s (x) = a + b (x − x ) + i (x − x )2 + i (x − x )3 (1) i i i i 2 i 6 i Possibilities to fix two more conditions xi−1 ≤ x ≤ xi i = 1,2,KN. Need to find ai , bi , ci , di . • Natural spline - the second order derivatives are d s '(x) = b + c (x − x ) + i (x − x )2 zero on boundaries. i i i i 2 i • Input values for the first order f(1)(x) derivates at si ''(x) = ci + di (x − xi ) boundaries si '''(x) = di • Input values for the second order f(2)(x) derivates at by the definition (interpolation) a = f (x ) boundaries i i 1) s(x) must be continuous at xi si (xi ) = si+1(xi ) c d a = a + b (x − x ) + i+1 (x − x )2 + i+1 (x − x )3 i i+1 i+1 i i+1 2 i i+1 6 i i+1 37 38 The system of equations to find coefficients hidi = ci − ci−1 i = 1,2,KN c0 = cN = 0 (5) using hi = xi − xi−1 2 2 3 hi hi hi hici − di = bi − bi−1 i = 2,3,KN (6) hibi − ci + di = fi − fi−1 (2) 2 2 6 2 3 hi hi 2) si '(xi ) = si+1'(xi ) i = 1,2,KN −1 h b − c + d = f − f i = 1,2, N (7) i i 2 i 6 i i i−1 K d c h − i h2 = b − b i = 2,3, N (3) Solve the system for c i = 1,2, N −1 i i 2 i i i−1 K i K Consider two equations (7) for points i and i −1 3) si ''(xi ) = si+1''(xi ) 2 hi hi fi − fi−1 dihi = ci − ci−1 i = 2,3,KN (4) b = c + d = i 2 i 6 i h additional equations (conditions at the ends) i h h2 f f s''(a) = s''(b) = 0 i−1 i−1 i−1 − i−2 bi−1 = ci−1 + di−1 = 2 6 hi−1 s1''(x0 ) = 0, sN ''(xN ) = 0 c1 − d1h1 = 0, cN = 0 and sustract the second equation from the first 1 1 f − f f − f b b (h c h c ) (h2d h2 d ) i i−1 i−1 i−2 39 i − i−1 = i i − i−1 i−1 − i i − i−1 i−1 + − 40 2 6 hi hi−1 Use the difference bi − bi−1 in the right side of (6) h2 2h2 ⎛ f − f f − f ⎞ h c + h c − i−1 d − i d = 2⎜ i i−1 − i−1 i−2 ⎟ (8) i i i−1 i−1 i−1 i ⎜ ⎟ Implementation 2 3 ⎝ hi hi−1 ⎠ Then from (5) 2 Recommendation – do not write your own hi di = hi (ci − ci−1), 2 spline program but get one from ….