CS321-001
Introduction to Numerical Methods
Lecture 3
Interpolation and Numerical Differentiation
Professor Jun Zhang
Department of Computer Science University of Kentucky Lexington, KY 40506-0633 Polynomial Interpolation
Given a set of discrete values, how can we estimate other values between these data
The method that we will use is called polynomial interpolation.
We assume the data we had are from the evaluation of a smooth function. We may be able to use a polynomial p(x) to approximate this function, at least locally.
A condition: the polynomial p(x) takes the given values at the given points (nodes), i.e., p(xi) = yi with 0 ≤ i ≤ n. The polynomial is said to interpolate the table, since we do not know the function.
2 Polynomial Interpolation
Note that all the points are passed through by the curve 3 Polynomial Interpolation
We do not know the original function, the interpolation may not be accurate 4 Order of Interpolating Polynomial
A polynomial of degree 0, a constant function, interpolates one set of data
If we have two sets of data, we can have an interpolating polynomial of degree 1, a linear function x x1 x x0 p(x) y0 y1 x0 x1 x1 x0
y1 y0 y0 (x x0 ) x1 x0 Review carefully if the interpolation condition is satisfied
Interpolating polynomials can be written in several forms, the most well known ones are the Lagrange form and Newton form. Each has some advantages
5 Lagrange Form
For a set of fixed nodes x0, x1, …, xn, the cardinal functions, l0, l1,…, ln, are defined as 0 if i j li (x j ) ij 1 if i j
We can interpolate any function f(x) by the Lagrange form of the interpolating polynomial of degree ≤ n n pn (x) li (x) f (xi ) i0
Note that li(x) is of order n, so pn(x) is of order ≤ n, and n pn (x j ) li (x j )f(xi ) l j (x j )f(x j ) f(x j ) i0 The (point exact) interpolation condition is satisfied
6 Cardinal Functions
The cardinal function is n x x j li (x) (0 i n) ji, j0 xi x j What it looks like x x0 x x1 li (x) xi x0 xi x1
x xi1 x xi1 xi xi1 xi xi1
x xn xi xn
Note that li (x j ) 0, for i j and li (xi ) 1 7 An Example
Find the interpolating polynomial for this table
Lagrange form (x 1)(x 1) l (x) (x 1)(x 1) 0 (0 1)(0 1) (x 0)(x 1) 1 l (x) x(x 1) 1 (1 0)(1 1) 2 (x 0)(x 1) 1 l (x) x(x 1) 2 (1 0)(1 1) 2 The interpolating polynomial is 3 15 p (x) 5(x 1)(x 1) x(x 1) x(x 1) 2 2 2
8 Five Lagrange Basis Polynomials
These are the 5 Lagrange basis polynomials for N=5 9 Representation by Lagrange Polynomials
Five weighted polynomials and their sum (red line) for a set of 5 random samples (red points) 10 Step by Step Construction
For any table of data, we can construct a Lagrange interpolating polynomial. Its evaluation is a little bit costly, but we can always do that. The existence of the interpolating polynomial is guaranteed
Can we construct the interpolating polynomial step by step, or if we discover some new data, can we add those data to make the interpolation more accurate?
We can use Newton form of the interpolating polynomial
Let pk(x) be an interpolating polynomial for the data set {(xi,yi)} with 0 ≤ i ≤ k such that pk(xi) = yi
11 Newton Form - Cont.
We want to add another data (xk+1, yk+1) to have a new interpolating polynomial pk+1(x) such that pk+1(xi) = yi for 0 ≤ i ≤ (k + 1). Let
pk1 (x) pk (x) c(x x0 )(x x1 )(x xk ) where c is an undetermined constant
Sincepk1(xk1 ) yk1 , we have
pk (xk1 ) c(xk1 x0 )(xk1 x1 )
(xk1 xk ) yk1
We can solve this equation for c, with the condition that x0, x1, …, xk+1 are all distinct y p (x ) c k1 k k1 (xk1 x0 )(xk1 x1 )(xk1 xk )
12 Newton Polynomial Interpolation
13 Uniqueness of Polynomial
Is the interpolating polynomial unique?
If p and q are interpolating polynomials for the data set {(xi,yi)} for 0 ≤ i ≤ n such that p(xi) = q(xi) = yi
Then the polynomial r(x) = p(x) – q(x) of degree at most n is zero at x0, x1, …, xn. Note that a polynomial of degree n can have at most n roots, we must have r(x) = 0, or p – q = 0. Hence p = q
The interpolating polynomial is unique
It may be written in different forms
14 An Example
Find the interpolating polynomial for this table
Lagrange form (x 1)(x 1) l (x) (x 1)(x 1) 0 (0 1)(0 1) (x 0)(x 1) 1 l (x) x(x 1) 1 (1 0)(1 1) 2 (x 0)(x 1) 1 l (x) x(x 1) 2 (1 0)(1 1) 2 The interpolating polynomial is 3 15 p (x) 5(x 1)(x 1) x(x 1) x(x 1) 2 2 2
15 Newton Form
The zeroth order polynomial is
p0 (x) 5 Let the 1st order interpolating polynomial be
p1 (x) p0 c(x x0 ) 5 c(x 0)
We want p1(x1) = -3, hence -5 + c(1 – 0) = -3, we have c = 2, it follows that
p1 (x) 5 2x Let the 2nd order interpolating polynomial be
p2 (x) p1(x) c(x x0 )(x x1 )
Put p2(-1) = -15, i.e., -5+2(-1) + c(-1 – 0)(-1 – 1) = -15. We have c = -4. The Newton form of the interpolating polynomial is
p2 (x) 5 2x 4x(x 1)
16 Nested Form
For easy programming and efficient computation, we can write Newton form of the interpolating polynomial in nested form
p(x) a0 a1[(x x0 )] a2[(x x0 )(x x1 )] a3[(x x0 )(x x1 )(x x2 )] an[(x x0 )(x x1 )(x xn1 )] Or, using standard product notations as n i1 p(x) a0 ai (x x j ) i1 j1 Using successive factorization, the nested form is p(x) a0 (x x0 )(a1 (x x1 )(a2
(x xn1 )an )))
(((an (x xn1 ) an1 )(x xn2 )
an2 ))(x x0 ) a0 17 Computation Procedure
To evaluate p(x) for a given x, we start from the innermost parentheses, forming successively some intermediate quantities
v0 an v1 v0 (x xn1 ) an1 v2 v1(x xn2 ) an2 vi vi1(x xni ) ani vn vn1 (x x0 ) a0
A pseudocode is
real array (ai )0:n , (xi )0:n integer i, n real x, v v an for i n 1 to 0 step -1 do v v(x xi ) ai end for 18 Divided Difference
The coefficients ai in Newton form of the interpolating polynomial need to be computed. A notation is introduced to facilitate such computation
ak f[x0,x1,,xk ] which is called the divided difference of order k for f
Newton form interpolating polynomial is
pn (x) a0 a1 (x x0 ) a2 (x x0 )(x x1 )
an (x x0 )(x xn1 ) Or written in a compact form n i1 pn (x) ai (x x j ) i0 j0 with the convention 1 (x x j ) 1 j0 19 Computing Coefficients ai
We want pn(xi) = f(xi). So we have f (x0 ) a0
f (x1) a0 a1(x1 x0 )
f (x2 ) a0 a1(x2 x0 ) a2 (x2 x0 )(x2 x1) The solution of this system is a0 f (x0 )
f (x1 ) a0 a1 x1 x0 f (x ) f (x ) 1 0 x1 x0 The divided difference of order 1 is f (x1 ) f (x0 ) f [x0 , x1 ] x1 x0 k Note that f[x0, x1,…, xk] is the coefficient of x in the polynomial pk of degree ≤ k 20 Computing Coefficients – Cont.
f (x2 ) a0 a1 (x2 x0 ) a2 (x2 x0 )(x2 x1 ) f (x ) f [x ] f [x , x ](x x ) 2 0 0 1 2 0 (x2 x0 )(x2 x1 ) f [x0 , x1 , x2 ] In general, we have f [x0 , x1 ,, xk ] k1 i1 f (xk ) i0 f [x0 , x1 ,, xi ] j0 (xk x j ) k1 j0 (xk x j )
Computational algorithm
•Set f[x0] = f(x0)
•For k = 1, 2,…, n, compute f[x0, x1,…, xk] using the above equation 21 Recursive Formula
The divided difference has a recursive formula
f[x0 , x1,, x k ]
f[x1, x 2 ,, x k ] f[x0, x1,, x k1]
x k x 0 Proof:
k f[x0, x1,…, xk] is the coefficient of x in the polynomial pk of degree ≤ k, which interpolates f at x0, x1,…, xk
k-1 f[x1, x2,…, xk] is the coefficient of x in the polynomial qk-1 of degree ≤ (k – 1), which interpolates f at x1, x2,…, xk
k-1 f[x0, x1,…, xk-1] is the coefficient of x in the polynomial pk-1of degree ≤ (k – 1), which interpolates f at x0, x1,…, xk-1
22 Recursive Formula - Proof
We have
x xk pk (x) qk1(x) [qk1 (x) pk1(x)] xk x0 To prove this identity, it suffices to show that it holds at (k + 1) different points, since the left-hand side and the right-hand side are polynomials of degree ≤ k. Note that the left-hand side is pk(xi) = f(xi) for i = 0, 1,…, k
Check the right-hand side at point x0
x0 xk qk1 (x0 ) [qk1 (x0 ) pk1(x0 )] xk x0
qk1 (x0 ) [qk1(x0 ) pk1(x0 )]
pk1(x0 ) f (x0 )
23 Formula Proof - Cont.
Check for points 1 ≤ i ≤ (k – 1), xi xk qk1 (xi ) [qk1(xi ) pk1(xi )] xk x0
xi xk f (xi ) [ f (xi ) f (xi )] f (xi ) xk x0
Check the right-hand side at point xk xk xk qk1 (xk ) [qk1(xk ) pk1(xk )] xk x0
qk1 (xk ) f (xk ) Hence the said identity holds
We take the coefficients of xk on both sides, which yields the desired recursive formula
24 Invariance Theorem
The divided difference f[x0, x1,…, xk] is invariant under all permutations of the arguments x0, x1,…, xk
k This is because f[x0, x1,…, xk] is the coefficient of x of the polynomial pk(x) of degree ≤ k that interpolates f at x0, x1,…, xk. f[x1, x0,…, xk] is the k coefficient of x of the polynomial pk(x) of degree ≤ k that interpolates f at x1, x0,…, xk. These two polynomials are the same
The generic recursive formula is
f [xi , xi1 ,, x j1 , x j ]
f xi1 , xi2 ,, x j f xi , xi1 ,, x j1
x j xi
25 Divided Difference Table
We can construct a divided difference table for f to facilitate computation of the coefficients of the interpolating polynomial
The coefficients along the top diagonal are the ones needed to form the Newton form of the interpolating polynomial 20 Divided Difference Table
20 Divided Difference Algorithm
A pseudocode for computing divided difference is real array (aij )0:n0:n , (xi )0:n integer i, j, n for i 0 to n do ai0 f (xi ) end for for j 1 to n do for i 0 to n j do aij (ai1, j1 ai, j1 ) /(xi j xi ) end for end for
This algorithm computes and stores all components of the divided difference. The coefficients of the Newton interpolating polynomial are stored in the first row of the array (aij)0:n×0:n, i.e., in a(0 : n, 0)
28 Computing the Coefficients only
If we compute divided difference only for constructing Newton interpolating polynomial, there is no need to store the unnecessary divided difference terms. (But they will be computed, used, and discarded)
real array (ai )0:n , (xi )0:n integer i, j, n for i 0 to n do
ai f (xi ) end for for j 1 to n do for i n to j step 1 do
ai (ai ai1) /(xi xi j ) end for end for
The two algorithms assume the same computational cost
29 Memory Allocations
Here we show how the memory is occupied and updated in computing coefficients of Newton interpolating polynomial
Computation must be done backward to avoid erasing needed memory locations
30 Divided Difference Table
20 Newton Polynomial i 0 1 234 0 1 2 3 4 0 1 8 27 64
20 Inverse Interpolation
It is also possible to use a polynomial to approximate the inverse of a function y = f(x). Given a table
An interpolation polynomial n i1 p( y) ci ( y y j ) i0 j0
Can be constructed such that p(yi) = xi. This interpolating polynomial is useful to find the approximate location of a root of a function f(x). E.g., there is a root in [4.0,5.0] for this table
33 Neville’s Algorithm
Neville proposed a different scheme to construct interpolation polynomial step by step. Start with zero degree polynomials Pi(x) = f(xi), we construct higher degree interpolation polynomials by the recurrence relation x x i j Sij (x) Si, j1(x) xi xi j xi x Si1, j1 (x) xi xi j
With Si0(x) = Pi(x) = f(xi). The relation table can be written as
34 Interpolation Property
(0) Redefine constant polynomials as Pi (x) yi for 0 i n, we can define higher order polynomial as x x P ( j) (x) i j P ( j1) (x) i i xi xi j xi x ( j1) Pi1 (x) xi xi j The range of j is 1 ≤ j ≤ n and that of i is j ≤ i ≤ n
The interpolation properties of these polynomials are:
(j) The polynomials Pi defined above interpolate the original table as follows (see p. 153 for a proof) ( j) Pi (xk ) yk (0 i j k i n)
35 Higher Dimensional Interpolation
It is possible to define interpolation polynomials of several variables. The tensor-product interpolation is used on rectangular domain [a,b] × [α,β]. Select n nodes in [a,b] and define the Lagrange polynomials as n x x j li (x) (1 i n). ji, j1 xi x j Select m nodes in [α,β] and define n y y j hi (y) (1 i m). ji, j1 yi y j
Then function n m P(x, y) f (xi , y j )li (x)h j (y) i1 j 1 Interpolates the two dimensional table with data
(xi , y j , f (xi , y j ))
36 2D Interpolation
20 Errors of Interpolation
If p is the polynomial of degree at most n that interpolates f at the n+1 ( ) distinct nodes belonging to an interval [a,b] and if is continuous for each x in [a,b], there is a in (a, b) for which
n 1 (n1) f (x) p(x) f ( )(x xi ) (n 1)! i0
What kind of function may have a small interpolation error?
38 Computing First Derivative
The first derivative can be approximated as 1 f '(x) [ f (x h) f (x)] (1) h For accurate approximation, h should be small. Thus f(x + h) and f(x) are close to each other. This may cause loss of significant digits in finite precision computation
Using Taylor’s theorem, we have 1 f (x h) f (x) hf '(x) h2 f "( ) 2 For ξ between x and x + h. It follows that 1 1 f '(x) [ f (x h) f (x)] hf "( ) h 2 1 hf "( ), The approximation error of (1) is2 or of order O(h). This is a first order (or sided) approximation of first derivative. The error goes to 0 as fast as h → 0 39 1st Order Approximation
20 Higher Order Approximation
It is desirable to have some higher order (faster) approximation schemes 1 f (x h) f (x) hf '(x) h2 f "(x) 2! 1 1 h3 f '''(x) h4 f (4) (x) 3! 4! 1 f (x h) f (x) hf '(x) h2 f "(x) 2! 1 1 h3 f '''(x) h4 f (4) (x) 3! 4! Subtracting these two equations, we have f (x h) f (x h) 2hf '(x) 2 2 h3 f '''(x) h5 f (5) (x) 3! 5!
41 2nd Order Approximation
It follows that 1 f '(x) [ f (x h) f (x h)] 2h h2 h4 f '''(x) f (5) (x) 3! 5! After dropping the higher order terms, we have a second order approximation formula as 1 f '(x) [ f (x h) f (x h)] 2h
2 The leading truncated terms of this approximation scheme is h f '''(x). 6 Hence the approximation is of O(h2). The approximation error goes to 0 as fast as h2 → 0. The exact truncation error is 1 f '''( ) f '''( ) 1 h2 1 2 h2 f '''( ) 6 2 6 42 2nd Order Approximation
20 Central DIfference Approximation
20 Richardson Extrapolation (I)
The first derivative can be approximated as f '(x) 1 [ f (x h) f (x h)] 2h 2 4 6 a2h a4h a6h
In which the constants a2,a4,… depend on the higher order derivatives of f and the value of x. When such information is available, it is possible to construct much more accurate approximation schemes
Define a function (h) 1 [ f (x h) f (x h)] 2h which is an approximation to f’(x) with error of order O(h2). This approximation becomes accurate as h → 0. So we can study the quantity limh → 0 ψ(h)
45 Richardson Extrapolation (II)
Richardson extrapolation estimates the value of ψ(0) from some computed values of ψ(h) near 0
2 4 6 (h) f '(x) a2h a4h a6h 2 4 6 h h h h f '(x) a2 a4 a6 2 2 2 2 Multiply the 2nd equation by 4 and subtract it from the 1st equation
h 3 4 15 6 (h) 4 3 f '(x) a4h a6h 2 4 16 Hence
h 1 h a4 4 5 6 (h) f '(x) h a6h 2 3 2 4 16 f’(x) can be computed as accurate as O(h4)
46 General Approach
A general Richardson extrapolation form 2k (h) L a2k h k1 where we assume that ψ(h) is computable for any h > 0 and we want to approximate L as accurately as possible h Choose a special sequence define 2n h D(n,0) (n 0) 2n Then, we have 2k h D(n,0) L A(k,0) n k1 2
With A(k,0) = -a2k. D(n,0) is a rough approximate of L = limx→0ψ(x) 47 Richardson Theorem The extrapolation formula is 4m D(n,m) D(n,m 1) 4m 1 1 D(n 1,m 1) (1 m n) 4m 1 Richardson Extrapolation Theorem: 2k h D(n,m) L A(k,m) n km1 2 For 0 ≤ m ≤ n
The proof of this theorem is based on induction on m, see p. 175 of the Book.
Not that D(n,m) approximates L at the order of O(h2m). The convergence rate is fast 48 Computational Procedure
Richardson extrapolation computational procedure:
1.) write a procedure to compute ψ(h)
2.) decide on suitable values for n and h
3.) for i = 0, 1,…, n, compute
D(i,0) (h / 2i )
4.) for 0 ≤ i ≤ j ≤ n, compute
D(i, j) D(i, j 1) (4 j 1)1[D(i, j 1) D(i 1, j 1)]
49 Using Interpolation Polynomial
We can approximate the function f(x) by a polynomial pn(x) of order n, such that pn(x) ≈ f(x)
To compute f’(x), we use the approximation f’(x) ≈ p’n(x)
Higher order polynomials are avoided because of oscillation
Let p interpolates f at two points, x0 and x1
p1(x) f (x0 ) f [x0 , x1](x x0 )
The first derivative of p1(x) is
f (x1 ) f (x0 ) p'1 (x) f [x0 , x1 ] f '(x) x1 x0
50 1st and 2nd Order Approx.
Let x0 = x and x1 = x + h, we have 1 f '(x) [ f (x h) f (x)] h This is just the O(h) order sided approximation formula
2 Put x0 = x – h and x1 = x + h, we have the O(h ) approximation scheme 1 f '(x) [ f (x h) f (x h)] 2h A three point polynomial interpolation is
p2 (x) f (x0 ) f [x0 , x1 ](x x0 )
f [x0 , x1 , x2 ](x x0 )(x x1 ) We have corrected approximation
p'2 (x) f [x0 , x1 ] f [x0 , x1 , x2 ](2x x0 x1 )
51 Second Derivative
If we have first derivative, we can use 1 f "(x) [ f '(x h) f '(x h)] 2h To approximate the second derivative to O(h2)
A direct approximation would be using Taylor expansion f (x h) f (x h) 1 2 f (x) h2 f "(x) 2 h4 f (4) (x) 4! Hence, we have 1 f "(x) [ f (x h) 2 f (x) f (x h)] h2 This approximation is of O(h2) accuracy
52