Chapter 7: Interpolation

❑ Topics:

❑ Examples

❑ Polynomial Interpolation – bases, error, Chebyshev, piecewise

❑ Orthogonal Polynomials

❑ Splines – error, end conditions

❑ Parametric interpolation

❑ Multivariate interpolation: f(x,y) Interpolation Motivation Polynomial Interpolation Choosing Interpolant Piecewise Polynomial Interpolation Existence and Uniqueness Interpolation

Basic interpolation problem: for given data (t ,y ), (t ,y ),...(t ,y ) with t

Purposes for InterpolationInterpolation Motivation Polynomial Interpolation Choosing Interpolant Piecewise Polynomial Interpolation Existence and Uniqueness Purposes for Interpolation

Plotting smooth curve through discrete data points

Reading between lines of table Plotting smooth curve through discrete data points Differentiating or integrating tabular data Reading between lines of table Quick and easy evaluation of mathematical function Differentiating or integrating tabular data Replacing complicated function by simple one Quick and easy evaluation of mathematical function ¡ Basis functions for function approximation in numerical solution of Replacingordinary and complicated partial differential function equations by simple (ODEs one and PDEs). ¡ Basis functions for developing integration rules. ¡ Basis functions for developing differentiation techniques. (Not just tabular Michaeldata…) T. Heath Scientific Computing 4 / 56

Michael T. Heath Scientific Computing 4 / 56 Interpolation Motivation Polynomial Interpolation Choosing Interpolant Piecewise Polynomial Interpolation Existence and Uniqueness Interpolation vs Approximation

By definition, interpolating function fits given data points exactly

Interpolation is inappropriate if data points subject to significant errors

It is usually preferable to smooth noisy data, for example by least squares approximation

Approximation is also more appropriate for special function libraries

Michael T. Heath Scientific Computing 5 / 56 Interpolation Motivation Polynomial Interpolation Choosing Interpolant Piecewise Polynomial Interpolation Existence and Uniqueness Issues in Interpolation

Arbitrarily many functions interpolate given set of data points

What form should interpolating function have?

How should interpolant behave between data points?

Should interpolant inherit properties of data, such as monotonicity, convexity, or periodicity?

Are parameters that define interpolating function meaningful? For example, function values, slopes, etc. ?

If function and data are plotted, should results be visually pleasing?

Michael T. Heath Scientific Computing 6 / 56 Interpolation Motivation Polynomial Interpolation Choosing Interpolant Piecewise Polynomial Interpolation Existence and Uniqueness Choosing Interpolant

Choice of function for interpolation based on How easy interpolating function is to work with determining its parameters ! Conditioning? !! evaluating interpolant differentiating or integrating interpolant

How well properties of interpolant match properties of data to be fit (smoothness, monotonicity, convexity, periodicity, etc.)

Michael T. Heath Scientific Computing 7 / 56 Interpolation Motivation Polynomial Interpolation Choosing Interpolant Piecewise Polynomial Interpolation Existence and Uniqueness Functions for Interpolation

Families of functions commonly used for interpolation include Polynomials Piecewise polynomials Trigonometric functions Exponential functions Rational functions

For now we will focus on interpolation by polynomials and piecewise polynomials

We will consider trigonometric interpolation (DFT) later

Michael T. Heath Scientific Computing 8 / 56 A Classic Polynomial Interpolation Problem

• Suppose you’re asked to tabulate data such that linear interpolation between tabulated values is correct to 4 digits.

• How many entries are required on, say, [0,1]?

• How many digits should you have in the tabulated data? A Classic Polynomial Interpolation Problem

An important polynomial interpolation result for f(x) Cn: 2 If p(x) lP n 1 and p(xj)=f(xj), j =1,...,n, then there 2 exists a ✓ [x ,x ,...,x ,x] such that 2 1 2 n f n(✓) f(x) p(x)= (x x )(x x ) (x x ). n! 1 2 ··· n

In particular, for linear interpolation, we have f (✓) f(x) p(x)= 00 (x x )(x x ) 2 1 2 2 2 f 00 h h f 00 f(x) p(x) max | | =max | | | |  [x1:x2] 2 4 [x1:x2] 8 where the latter result pertains to x [x ,x ]. 2 1 2

2 A Classic Polynomial Interpolation Problem

Example: f(x)=cos(x)

We know that f 00 1andthus,forlinearinterpolation | |  h2 f(x) p(x) . | |  8 If we want 4 decimal places of accuracy, accounting for rounding, we need

2 h 1 4 f(x) p(x) 10 | |  8  2 ⇥ 2 4 h 4 10  ⇥ h 0.02 

xcosx 0.00 1.00000 0.02 0.99980 0.04 0.99920 0.06 0.99820 0.08 0.99680

3 Interpolation Motivation Polynomial Interpolation Choosing Interpolant Piecewise Polynomial Interpolation Existence and Uniqueness Basis Functions

Family of functions for interpolating given data points is spanned by set of basis functions 1(t),...,n(t) Interpolating function f is chosen as linear combination of basis functions, n f(t)= xjj(t) Xj=1 Requiring f to interpolate data (ti,yi) means n f(ti)= xjj(ti)=yi,i=1,...,m Xj=1 which is system of linear equations Ax = y for n-vector x of parameters x , where entries of m n matrix A are j ⇥ given by aij = j(ti)

Michael T. Heath Scientific Computing 9 / 56 Interpolation Motivation Polynomial Interpolation Choosing Interpolant Piecewise Polynomial Interpolation Existence and Uniqueness Existence, Uniqueness, and Conditioning

Existence and uniqueness of interpolant depend on number of data points m and number of basis functions n

If m>n, interpolant usually doesn’t exist

If m

If m = n, then basis matrix A is nonsingular provided data points ti are distinct, so data can be fit exactly

Sensitivity of parameters x to perturbations in data depends on cond(A), which depends in turn on choice of basis functions

Michael T. Heath Scientific Computing 10 / 56 Interpolation Monomial, Lagrange, and Newton Interpolation Polynomial Interpolation Orthogonal Polynomials Piecewise Polynomial Interpolation Accuracy and Convergence Polynomial Interpolation

Simplest and most common type of interpolation uses polynomials

Unique polynomial of degree at most n 1 passes through n data points (ti,yi), i =1,...,n, where ti are distinct

There are many ways to represent or compute interpolating polynomial, but in theory all must give same result

(i.e., in infinite< interactive precision example arithmetic) >

Michael T. Heath Scientific Computing 11 / 56 Interpolation Monomial, Lagrange, and Newton Interpolation Polynomial Interpolation Orthogonal Polynomials Piecewise Polynomial Interpolation Accuracy and Convergence Monomial Basis

Monomial basis functions

j 1 j(t)=t ,j=1,...,n

give interpolating polynomial of form

n 1 pn 1(t)=x1 + x2t + + xnt ··· with coefficients x given by n n linear system ⇥ n 1 1 t1 t1 x1 y1 ··· n 1 1 t2 t2 x2 y2 Ax = 2. . ···. . 3 2 . 3 = 2 . 3 = y ...... 6 n 17 6 7 6 7 61 t t 7 6xn7 6yn7 6 n ··· n 7 6 7 6 7 4 5 4 5 4 5 Matrix of this form is called Vandermonde matrix

Michael T. Heath Scientific Computing 12 / 56 Interpolation Monomial, Lagrange, and Newton Interpolation Polynomial Interpolation Orthogonal Polynomials Piecewise Polynomial Interpolation Accuracy and Convergence Example: Monomial Basis

Determine polynomial of degree two interpolating three data points ( 2, 27), (0, 1), (1, 0) Using monomial basis, linear system is 2 1 t1 t1 x1 y1 2 Ax = 1 t2 t2 x2 = y2 = y 2 23 2 3 2 3 1 t3 t3 x3 y3 For these particular4 data, system5 4 is5 4 5 1 24 x 27 1 100x = 1 2 3 2 23 2 3 111x3 0 4 5 4 5 T 4 5 whose solution is x = 15 4 , so interpolating polynomial is ⇥ ⇤ p (t)= 1+5t 4t2 2 Michael T. Heath Scientific Computing 13 / 56 Interpolation Monomial, Lagrange, and Newton Interpolation Polynomial Interpolation Orthogonal Polynomials Piecewise Polynomial Interpolation Accuracy and Convergence Monomial Basis, continued

< interactive example >

Solving system Ax = y using standard linear equation solver to determine coefficients x of interpolating polynomial requires (n3) work O Michael T. Heath Scientific Computing 14 / 56 Interpolation Monomial, Lagrange, and Newton Interpolation Polynomial Interpolation Orthogonal Polynomials Piecewise Polynomial Interpolation Accuracy and Convergence Monomial Basis, continued

For monomial basis, matrix A is increasingly ill-conditioned as degree increases Ill-conditioning does not prevent fitting data points well, since residual for linear system solution will be small But it does mean that values of coefficients are poorly determined Both conditioning of linear system and amount of computational work required to solve it can be improved by using different basis Change of basis still gives same interpolating polynomial for given data, but representation of polynomial will be different

Michael T. Heath Scientific Computing 15 / 56 Interpolation Monomial, Lagrange, and Newton Interpolation Polynomial Interpolation Orthogonal Polynomials Piecewise Polynomial Interpolation Accuracy and Convergence Monomial Basis, continued

Conditioning with monomial basis can be improved by shifting and scaling independent variable t j 1 t c (t)= j d ✓ ◆ where, c =(t + t )/2 is midpoint and d =(t t )/2 is 1 n n 1 half of range of data New independent variable lies in interval [ 1, 1], which also helps avoid overflow or harmful underflow Even with optimal shifting and scaling, monomial basis usually is still poorly conditioned, and we must seek better alternatives < interactive example >

Michael T. Heath Scientific Computing 16 / 56 Polynomial Interpolation

❑ Two types: Global or Piecewise

❑ Choices: ❑ A: points are given to you ❑ B: you choose the points

❑ Case A: piecewise polynomials are most common – STABLE. ❑ Piecewise linear ❑ Splines ❑ Hermite (matlab “pchip” – piecewise cubic Hermite int. polynomial)

❑ Case B: high-order polynomials are OK if points chosen wisely ❑ Roots of orthogonal polynomials ❑ Convergence is exponential: err ~ Ce-¾ n, instead of algebraic: err~Cn-k Piecewise Polynomial Interpolation

❑ Example – Given the table below,

xj fj 0.6 1.2 0.8 2.0 1.0 2.4

❑ Q: What is f(x=0.75)Polynomial ? interpolation error formula:

If p(x) lP n 1 and p(xj)=f(xj), j =1,...,n,thenthere 2 exists a ✓ [x ,x ,...,x ,x] such that 2 1 2 n f n(✓) f(x) p(x)= (x x )(x x ) (x x ) n! 1 2 ··· n f n(✓) = q (x),q(x) lP . n! n n 2 n

1 Polynomial Interpolation

❑ Example – Given the table below,

xj fj 0.6 1.2 0.8 2.0 1.0 2.4

❑ Q: What is f(x=0.75)Polynomial ? interpolation error formula:

❑ If p(x) lP n 1 and p(xj)=f(xj), j =1,...,n,thenthere A: 1.8 --- You’ve just done2 (piecewise) linear interpolation. exists a ✓ [x ,x ,...,x ,x] such that 2 1 2 n ❑ Moreover, you know the error is · (0.2)2 f’’ / 8. f n(✓) f(x) p(x)= (x x )(x x ) (x x ) n! 1 2 ··· n f n(✓) = q (x),q(x) lP . n! n n 2 n

1 xj fj 0.6 1.2 0.8 2.0 1.0 2.4 General Polynomial Interpolation

Whether interpolating on segments or globally, error formula applies over • the interval. If p(t) lP n 1 and p(tj)=f(tj), j =1,...,n, then there 2 exists a ✓ [t ,t ,...,t ,t] such that 2 1 2 n f n(✓) f(t) p(t)= (t t )(t t ) (t t ) n! 1 2 ··· n f n(✓) = q (t),q(t) lP . n! n n 2 n We generally have no control over f n(✓), so instead seek to optimize • choice of the tj in order to minimize

max qn(t) . t [t1,tn] | | 2 Such a problem is called a minimax problem and the solution is given • by the tjsbeingtherootsofaChebyshevpolynomial,aswewilldiscuss shortly.

First, however, we turn to the problem of constructing p(t) lP n 1(t). • 2

1 Constructing High-Order Polynomial Interpolants

❑ Lagrange Polynomials n

p(t)= fj lj(t) j=1 X

lj(t)=1 t = tj l (t )=0 t = t ,i= j j i i 6 lj(t) lP n 1(t) 2

The lj1(t) polynomials are chosen so that p(tj) = f(tj) := fj lj(t)= (t t1)(t t2) (t tj 1)(t tj+1) (t tn) C ··· ··· The lj(t)s are sometimes called the Lagrange cardinal functions. C =(tj t1)(tj t2) (tj tj 1)(tj tj+1) (tj tn) ··· ···

t t1 t t2 t tj 1 t tj+1 t tn lj(t)= . tj t1 tj t2 ··· tj tj 1 tj tj+1 ··· tj tn ✓ ◆✓ ◆ ✓ ◆✓ ◆ ✓ ◆

1 Constructing High-Order Polynomial Interpolants

n ❑ Lagrange Polynomials p(t)= fj lj(t) j=1 X Construct p(t)= f l (tl).j(t)=1 t = tj • j j j lj(ti)=0 t = ti,i= j lj(t)givenbyabove.P 6 • lj(t) lP n 1(t) Error formula f(t) p(t)givenasbefore.2 • Can choose tj’s to minimize error polynomial qn(t). • 1 lj(t)= (t t1)(t t2) (t tj 1)(t tj+1) (t tn) C ··· ··· lj(t)isapolynomialofdegreen 1 • C =(tj t1)(tj t2) (tj tj 1)(tj tj+1) (tj tn) ··· ··· It is zero at t = ti, i = j. • t t16 t t2 t tj 1 t tj+1 t tn lj(t)= . Choose C so thattj itt1 is 1 attj t t=2 t···j. tj tj 1 tj tj+1 ··· tj tn • ✓ ◆✓ ◆ ✓ ◆✓ ◆ ✓ ◆

n

p(t)= fj lj(t) j=1 X

lj(t)=1 t = tj l (t )=0 t = t ,i= j j i i 6 lj(t) lP n 1(t) 2

1 lj(t)= (t t1)(t t2) (t tj 1)(t tj+1) (t tn) C ··· ···

C =(tj t1)(tj t2) (tj tj 1)(tj tj+1) (tj tn) ··· ···

t t1 t t2 t tj 1 t tj+1 t tn lj(t)= . tj t1 tj t2 ··· tj t1j 1 tj tj+1 ··· tj tn ✓ ◆✓ ◆ ✓ ◆✓ ◆ ✓ ◆

1 Construct p(t)= f l (t). • j j j lj(t)givenbyabove.P • Error formula f(t) p(t)givenasbefore. • CanConstructing choose t ’s High-Order to minimize Polynomial error polynomial Interpolantsq (t). • j n

l (t)isapolynomialofdegreen 1 • j It is zero at t = t , i = j. • i 6 Choose C so that it is 1 at t = t . • j

n

p(t)= fj lj(t) j=1 X

lj(t)=1 t = tj l (t )=0 t = t ,i= j j i i 6 lj(t) lP n 1(t) 2

1 lj(t)= (t t1)(t t2) (t tj 1)(t tj+1) (t tn) C ··· ···

C =(tj t1)(tj t2) (tj tj 1)(tj tj+1) (tj tn) ··· ···

t t1 t t2 t tj 1 t tj+1 t tn lj(t)= . tj t1 tj t2 ··· tj tj 1 tj tj+1 ··· tj tn ✓ ◆✓ ◆ ✓ ◆✓ ◆ ✓ ◆

1 Construct p(t)= f l (t). • j j j lj(t)givenbyabove.P • Error formula f(t) p(t)givenasbefore. • CanConstructing choose t ’s High-Order to minimize Polynomial error polynomial Interpolantsq (t). • j n

l (t)isapolynomialofdegreen 1 • j It is zero at t = t , i = j. • i 6 Choose C so that it is 1 at t = t . • j

n

p(t)= fj lj(t) j=1 X

lj(t)=1 t = tj l (t )=0 t = t ,i= j j i i 6 lj(t) lP n 1(t) 2

1 lj(t)= (t t1)(t t2) (t tj 1)(t tj+1) (t tn) C ··· ···

C =(tj t1)(tj t2) (tj tj 1)(tj tj+1) (tj tn) ··· ···

t t1 t t2 t tj 1 t tj+1 t tn lj(t)= . tj t1 tj t2 ··· tj tj 1 tj tj+1 ··· tj tn ✓ ◆✓ ◆ ✓ ◆✓ ◆ ✓ ◆

1 Constructing High-Order Polynomial Interpolants

❑ Although a bit tedious to do by hand, these formulas are relatively easy to evaluate with a computer.

❑ So, to recap – Lagrange polynomial interpolation:

Construct p(t)= f l (t). • j j j lj(t)givenbyabove.P • Error formula f(t) p(t)givenasbefore. • Can choose t ’s to minimize error polynomial q (t). • j n

l (t)isapolynomialofdegreen 1 • j It is zero at t = t , i = j. • i 6 Choose C so that it is 1 at t = t . • j

n

p(t)= fj lj(t) j=1 X

lj(t)=1 t = tj l (t )=0 t = t ,i= j j i i 6 lj(t) lP n 1(t) 2

1 lj(t)= (t t1)(t t2) (t tj 1)(t tj+1) (t tn) C ··· ···

C =(tj t1)(tj t2) (tj tj 1)(tj tj+1) (tj tn) ··· ···

t t1 t t2 t tj 1 t tj+1 t tn lj(t)= . tj t1 tj t2 ··· tj tj 1 tj tj+1 ··· tj tn ✓ ◆✓ ◆ ✓ ◆✓ ◆ ✓ ◆

1 Lagrange Basis Functions, n=2 (linear) Lagrange Basis Functions, n=3 (quadratic) Also have the Newton Basis

Given pn(tj)=fj, j =1,...,n and pn lP n 1. • 2 Let • p (t):=p (t)+C(t t )(t t ) (t t ) n+1 n 1 2 ··· n

such that pn+1(tn+1)=fn+1

Set • f p (t ) C = n+1 n n+1 , qn(tn+1)

with q (t):=(t t )(t t ) (t t ) lP n 1 2 ··· n 2 n

❑ These formulas are interesting because they are adaptive. (More details are in the text, but this is the essence of the method.)

1 Interpolation Monomial, Lagrange, and Newton Interpolation Polynomial Interpolation Orthogonal Polynomials Piecewise Polynomial Interpolation Accuracy and Convergence Lagrange Interpolation

For given set of data points (ti,yi), i =1,...,n, Lagrange basis functions are defined by n n ` (t)= (t t ) / (t t ),j=1,...,n j k j k k=1,k=j k=1,k=j Y6 Y6 For Lagrange basis, 1ifi = j ` (t )= ,i,j=1,...,n j i 0ifi = j ⇢ 6 so matrix of linear system Ax = y is identity matrix

Thus, Lagrange polynomial interpolating data points (ti,yi) is given by

pn 1(t)=y1`1(t)+y2`2(t)+ + yn`n(t) ···

Michael T. Heath Scientific Computing 18 / 56 Interpolation Monomial, Lagrange, and Newton Interpolation Polynomial Interpolation Orthogonal Polynomials Piecewise Polynomial Interpolation Accuracy and Convergence Lagrange Basis Functions

< interactive example >

Lagrange interpolant is easy to determine but more expensive to evaluate for given argument, compared with monomial basis representation TheseLagrangian concerns form are isimportant also more when difficult computing to differentiate, by hand, but notintegrate, important etc. when using a computer.

Michael T. Heath Scientific Computing 19 / 56 Interpolation Monomial, Lagrange, and Newton Interpolation Polynomial Interpolation Orthogonal Polynomials Piecewise Polynomial Interpolation Accuracy and Convergence Example: Lagrange Interpolation

Use Lagrange interpolation to determine interpolating polynomial for three data points ( 2, 27), (0, 1), (1, 0) Lagrange polynomial of degree two interpolating three points (t1,y1), (t2,y2), (t3,y3) is given by p2(t)= (t t )(t t ) (t t )(t t ) (t t )(t t ) y 2 3 + y 1 3 + y 1 2 1 (t t )(t t ) 2 (t t )(t t ) 3 (t t )(t t ) 1 2 1 3 2 1 2 3 3 1 3 2 For these particular data, this becomes

t(t 1) (t + 2)(t 1) p (t)= 27 +( 1) 2 ( 2)( 2 1) (2)( 1)

Michael T. Heath Scientific Computing 20 / 56 Interpolation Monomial, Lagrange, and Newton Interpolation Polynomial Interpolation Orthogonal Polynomials Piecewise Polynomial Interpolation Accuracy and Convergence Interpolating Continuous Functions

If data points are discrete sample of continuous function, how well does interpolant approximate that function between sample points?

If f is smooth function, and pn 1 is polynomial of degree at most n 1 interpolating f at n points t ,...,t , then 1 n f (n)(✓) f(t) pn 1(t)= (t t1)(t t2) (t tn) n! ···

where ✓ is some (unknown) point in interval [t1,tn]

Since point ✓ is unknown, this result is not particularly useful unless bound on appropriate derivative of f is known

Michael T. Heath Scientific Computing 33 / 56 Interpolation Monomial, Lagrange, and Newton Interpolation Polynomial Interpolation Orthogonal Polynomials Piecewise Polynomial Interpolation Accuracy and Convergence Interpolating Continuous Functions, continued

If f (n)(t) M for all t [t ,t ], and | |  2 1 n h = max t t : i =1,...,n 1 , then { i+1 i } Mhn max f(t) pn 1(t) t [t1,tn] | |  4n 2 Error diminishes with increasing n and decreasing h, but only if f (n)(t) does not grow too rapidly with n | | < interactive example >

Michael T. Heath Scientific Computing 34 / 56 Interpolation Monomial, Lagrange, and Newton Interpolation Polynomial Interpolation Orthogonal Polynomials Piecewise Polynomial Interpolation Accuracy and Convergence Convergence

Polynomial interpolating continuous function may not converge to function as number of data points and polynomial degree increases

Equally spaced interpolation points often yield unsatisfactory results near ends of interval

If points are bunched near ends of interval, more satisfactory results are likely to be obtained with polynomial interpolation

Use of Chebyshev points distributes error evenly and yields convergence throughout interval for any sufficiently smooth function

Michael T. Heath Scientific Computing 36 / 56 Unstable and Stable Interpolating Basis Sets

❑ Examples of unstable bases are: i ❑ Monomials (modal): φi = x ❑ High-order Lagrange interpolants (nodal) on uniformly-spaced points.

❑ Examples of stable bases are: ❑ Orthogonal polynomials (modal), e.g.,

❑ Legendre polynomials: Lk(x), or

❑ bubble functions: φk(x) := Lk+1(x) – Lk-1(x). ❑ Lagrange (nodal) polynomials based on Gauss quadrature points (e.g., Gauss-Legendre, Gauss-Chebyshev, Gauss-Lobatto-Legendre, etc.)

❑ Can map back and forth between stable nodal bases and Legendre or bubble function modal bases, with minimal information loss.

3 7 Unstable and Stable Interpolating Basis Sets

Key idea for Chebyshev interpolation is to choose points that minimize • max q (x) on interval := [ 1, 1]. | n+1 | I q (x):=(x x )(x x ) ...(x x ) n+1 0 1 n n n 1 := x + cn 1x + ... + c0 which is a monic polynomial of degree n +1. The roots of the Chebyshev polynomial T (x)yieldsuchasetofpoints • n+1 by clustering near the endpoints.

3 8 Lagrange Polynomials: Good and Bad Point Distributions

N=4

N=7

φ2 φ4

N=8

Uniform Gauss-Lobatto-Legendre

3 9

Here, we see the max qn+1 for uniform (red) and Chebyshev points.

Chebyshev converges much more rapidly. Nth-order Gauss-Chebyshev Points

❑ Roots of Nth-order Chebyshev polynomial are projections of equispaced points on the circle, starting with µ = ±µ/2, then µ = 3±µ/2,…,¼-±µ/2. N+1 Gauss-Lobatto Chebyshev Points

❑ N+1 GLC points are projections of equispaced points on the circle, starting with µ = 0, then µ = ¼/N, 2¼/N, … , k¼/N, … , ¼. Nth-Order Gauss Chebyshev Points

❑ Matlab Demo Interpolation Monomial, Lagrange, and Newton Interpolation Polynomial Interpolation Orthogonal Polynomials Piecewise Polynomial Interpolation Accuracy and Convergence Example: Runge’s Function

Polynomial interpolants of Runge’s function at equally spaced points do not converge

< interactive example >

Michael T. Heath Scientific Computing 37 / 56 Interpolation Monomial, Lagrange, and Newton Interpolation Polynomial Interpolation Orthogonal Polynomials Piecewise Polynomial Interpolation Accuracy and Convergence Example: Runge’s Function

Polynomial interpolants of Runge’s function at Chebyshev points do converge

Chebyshev Convergence< interactive is exponential example > for smooth f(t).

Michael T. Heath Scientific Computing 38 / 56 Interpolation Testing

Try a variety of methods for a variety of functions. • Inspect by plotting the function and the interpolant. • Compare with theoretical bounds. (Which are accurate!) •

Typical Interpolation Experiment

Given f(t), evaluate f := f(t ), j =1,...,n. • j j Construct interpolant: • n

p(t)= pˆj j(t). j=1 X

Evaluate p(t)att˜i, i =1,...,m, m n. (Fine mesh, for plotting, say.) • 4 To check error, compare with original function on fine mesh, t˜ . • i 6 e := p(t˜ ) f(t˜ ) i i i max e e := i | i| max max f i | i| max p f | |. ⇡ max f | | (Remember, it’s an experiment.) Interpolation Testing

Try a variety of methods for a variety of functions. • Inspect by plotting the function and the interpolant. • Compare with theoretical bounds. (Which are accurate!) •

Typical Interpolation Experiment

Given f(t), evaluate f := f(t ), j =1,...,n. • j j Construct interpolant: • n

p(t)= pˆj j(t). j=1 X Evaluate p(t)att˜ , i =1,...,m, m n. (Fine mesh, for plotting, say.) • i To check error, compare with original function on fine mesh, t˜ . • i e := p(t˜ ) f(t˜ ) i i i max e e := i | i| max max f i | i| max p f | |. ⇡ max f | | (Remember, it’s an experiment.)

4 7 Preceding description is for one trial. • Repeat for increasing n and plot e (n)onalog-logorsemilogplot. • max Compare with other methods and with theory: • – methods –identifybestmethodforgivenfunction/requirements – theory – verify that experiment is correctly implemented Repeat with a di↵erent function. •

4 8 Summary of Key Theoretical Results

Piecewise linear interpolation: • 2 h M := max✓ [a,b] f 00(✓) max p f M, 2 | | t [a,b] | |  8 ( h := maxj [2,...,n](tj tj 1),tj 1

Polynomial interpolation through n points: • q (✓) max p f n M, t [a,b] | |  n! 2 hn M, (for t [a, b]),  4n 2 with M := max f n(✓) . ✓ [a,b,t] | | 2 – Here, q (✓):=(✓ t )(✓ t ) (✓ t ). n 1 2 ··· n –Thefirstresultalsoholdstrueforextrapolation, i.e., t [a, b]. 62

Natural cubic spline (s00(a)=s00(b)=0): • 2 max p f Ch M, M =max f 00(✓) , t [a,b] | |  ✓ [a,b] | | 2 2 4

unless f 00(a)=f 00(b)=0,orotherluckycircumstances. 9

Clamped cubic spline (s0(a)=f 0(a), s0(b)=f 0(b)): • max p f Ch4 M, M = max f iv(✓) . t [a,b] | |  ✓ [a,b] | | 2 2 Nyquist sampling theorem: • Roughly: The maximum frequency that can be resolved with n points is N = n/2. There are other conditions, such as limits on the spacing of the sampling. Summary of Key Theoretical Results

Piecewise linear interpolation: • 2 h M := max✓ [a,b] f 00(✓) max p f M, 2 | | t [a,b] | |  8 ( h := maxj [2,...,n](tj tj 1),tj 1

Polynomial interpolation through n points: • q (✓) max p f n M, t [a,b] | |  n! 2 hn M, (for t [a, b]),  4n 2 with M := max f n(✓) . ✓ [a,b,t] | | 2 – Here, q (✓):=(✓ t )(✓ t ) (✓ t ). n 1 2 ··· n –Thefirstresultalsoholdstrueforextrapolation, i.e., t [a, b]. 62

Natural cubic spline (s00(a)=s00(b)=0): • 2 max p f Ch M, M =max f 00(✓) , t [a,b] | |  ✓ [a,b] | | 2 2

unless f 00(a)=f 00(b)=0,orotherluckycircumstances.

Clamped cubic spline (s0(a)=f 0(a), s0(b)=f 0(b)): • max p f Ch4 M, M = max f iv(✓) . t [a,b] | |  ✓ [a,b] | | 2 2 Nyquist sampling theorem: • Roughly: The maximum frequency that can be resolved with n points is N = n/2. There are other conditions, such as limits on the spacing of the sampling.

5 0 Methods: • – piecewise linear – polynomial on uniform points – polynomial on Chebyshev points – natural cubic spline

Tests: • – et – ecos t – sin t on [0, ⇡] ⇡ – sin t on [0, 2 ] – sin 15t on [0, 2⇡] – ecos 11ton [0, 2⇡] 1 – Runge function: 1+25t2 on [0, 1] – Runge function: 1 on [ 1, 1] 1+25t2 – Semi-circle: p1 t2 on [ 1, 1] – Polynomial: tn – Extrapolation – Other interp_test.m interp_test_runge.m 5 1 Methods: • – piecewise linear – polynomial on uniform points – polynomial on Chebyshev points – natural cubic spline

Tests: • – et – ecos t – sin t on [0, ⇡] ⇡ – sin t on [0, 2 ] – sin 15t on [0, 2⇡] – ecos 11ton [0, 2⇡] 1 – Runge function: 1+25t2 on [0, 1] – Runge function: 1 on [ 1, 1] 1+25t2 – Semi-circle: p1 t2 on [ 1, 1] – Polynomial: tn – Extrapolation – Other interp_test.m interp_test_runge.m 5 2 Interpolation Piecewise Polynomial Interpolation Polynomial Interpolation Hermite Cubic Interpolation Piecewise Polynomial Interpolation Cubic Spline Interpolation Piecewise Polynomial Interpolation

Fitting single polynomial to large number of data points is likely to yield unsatisfactory oscillating behavior in interpolant Piecewise polynomials provide alternative to practical and theoretical difficulties with high-degree polynomial interpolation Main advantage of piecewise polynomial interpolation is that large number of data points can be fit with low-degree polynomials

In piecewise interpolation of given data points (ti,yi), different function is used in each subinterval [ti,ti+1]

Abscissas ti are called knots or breakpoints, at which interpolant changes from one function to another

Michael T. Heath Scientific Computing 40 / 56 Interpolation Piecewise Polynomial Interpolation Polynomial Interpolation Hermite Cubic Interpolation Piecewise Polynomial Interpolation Cubic Spline Interpolation Piecewise Interpolation, continued

Simplest example is piecewise linear interpolation, in which successive pairs of data points are connected by straight lines

Although piecewise interpolation eliminates excessive oscillation and nonconvergence, it appears to sacrifice smoothness of interpolating function

We have many degrees of freedom in choosing piecewise polynomial interpolant, however, which can be exploited to obtain smooth interpolating function despite its piecewise nature

< interactive example >

Michael T. Heath Scientific Computing 41 / 56 Piecewise Polynomial Bases: Linear and Quadratic

5 5 Interpolation Piecewise Polynomial Interpolation Polynomial Interpolation Hermite Cubic Interpolation Piecewise Polynomial Interpolation Cubic Spline Interpolation Cubic Spline Interpolation

Spline is piecewise polynomial of degree k that is k 1 times continuously differentiable For example, linear spline is of degree 1 and has 0 continuous derivatives, i.e., it is continuous, but not smooth, and could be described as “broken line” Cubic spline is piecewise cubic polynomial that is twice continuously differentiable As with Hermite cubic, interpolating given data and requiring one continuous derivative imposes 3n 4 constraints on cubic spline Requiring continuous second derivative imposes n 2 additional constraints, leaving 2 remaining free parameters

Michael T. Heath Scientific Computing 44 / 56

Piecewise cubics:

Interval j =[xj 1,xj],j=1,...,n • I p (x) lP (x)on j 2 3 Ij 2 3 pj(x)=aj + bjx + cjx + djx

4n unknowns •

pj(xj 1)=fj 1,j=1,...,n

pj(xj)=fj,j=1,...,n

p0 (xj)=p0 (xj),j=1,...,n 1 j j+1 Spline p00(x )=p00 (x ),j=1,...,n 1 j j j+1 j conditions 4n 2equations • Interpolation Piecewise Polynomial Interpolation Polynomial Interpolation Hermite Cubic Interpolation Piecewise Polynomial Interpolation Cubic Spline Interpolation Cubic Splines, continued

Final two parameters can be fixed in various ways

Specify first derivative at endpoints t1 and tn

Force second derivative to be zero at endpoints, which gives natural spline

Enforce “not-a-knot” condition, which forces two consecutive cubic pieces to be same

Force first derivatives, as well as second derivatives, to Force first derivatives at endpoints to match y’(x) – clamped spline. • match at endpoints t1 and tn (if spline is to be periodic)

Michael T. Heath Scientific Computing 45 / 56 Interpolation Piecewise Polynomial Interpolation Polynomial Interpolation Hermite Cubic Interpolation Piecewise Polynomial Interpolation Cubic Spline Interpolation Example: Cubic Spline Interpolation

Determine natural cubic spline interpolating three data points (ti,yi), i =1, 2, 3

Required interpolant is piecewise cubic function defined by separate cubic polynomials in each of two intervals [t1,t2] and [t2,t3]

Denote these two polynomials by

2 3 p1(t)=↵1 + ↵2t + ↵3t + ↵4t

2 3 p2(t)=1 + 2t + 3t + 4t Eight parameters are to be determined, so we need eight equations

Michael T. Heath Scientific Computing 46 / 56 Cubic Spline Formulation – 2 Segments

Interpolatory 8 Unknowns p1(t1)=y1 2 3 p1(t)=↵1 + ↵2t + ↵3t + ↵4t p1(t2)=y2 2 3 p2(t)=1 + 2t + 3t + 4t p2(t2)=y2

8Equations p2(t3)=y3

Interpolatory ContinuityInterpolatory of Derivatives p1(t1)=y1 p10 (t2)=p1(tp120)=(t2) y1 p1(t2)=y2 p100(t2)=p1(tp2)=200(t2) y2 p (t )=y 8 Unknowns 2 2 2 End Conditionsp2(t2)=y2 2 3 p1(t)=↵1 + ↵2t + ↵3t + ↵4t p2(t3)=y3 p2(t3)=y3 p100(t1)=0 2 3 p2(t)=1 + 2t + 3t + 4t Continuity of Derivativesp200(t3)=0Continuity of Derivatives p0 (t2)=p0 (t2) 1 2 (Naturalp10 ( Splinet2)=) p20 (t2) p00(t2)=p00(t2) 1 2 8Equationsp100(t2)=p200(t2)

End Conditions End Conditions Interpolatory p00(t1)=0 p1(t1)=y1 1 p100(t1)=0 p00(t3)=0 p1(t2)=y2 2 p200(t3)=0

p2(t2)=y2

p2(t3)=y3

Continuity of Derivatives

p10 (t2)=p20 (t2)

p100(t2)=p200(t2)

End Conditions

p100(t1)=0 1 p200(t3)=0

1

1

1 Some Cubic Spline Properties

❑ Continuous ❑ 1st derivative: continuous ❑ 2nd derivative: continuous

❑ “Natural Spline” minimizes integrated curvature: over all twice-differentiable f(x)

passing through (xj,fj), j=1,…,n.

❑ Robust / Stable (unlike high-order polynomial interpolation) ❑ Commonly used in computer graphics, CAD software, etc. ❑ Usually used in parametric form (DEMO) ❑ There are other forms, e.g., tension-splines, that are also useful. ❑ For clamped boundary conditions, convergence is O(h4) ❑ For small displacements, natural spline is like a physical spline. (DEMO) Interpolation Piecewise Polynomial Interpolation Polynomial Interpolation Hermite Cubic Interpolation Piecewise Polynomial Interpolation Cubic Spline Interpolation Hermite Cubic vs Spline Interpolation

Choice between Hermite cubic and spline interpolation depends on data to be fit and on purpose for doing interpolation If smoothness is of paramount importance, then spline interpolation may be most appropriate But Hermite cubic interpolant may have more pleasing visual appearance and allows flexibility to preserve monotonicity if original data are monotonic In any case, it is advisable to plot interpolant and data to help assess how well interpolating function captures behavior of original data

< interactive example >

Michael T. Heath Scientific Computing 49 / 56 Interpolation Piecewise Polynomial Interpolation Polynomial Interpolation Hermite Cubic Interpolation Piecewise Polynomial Interpolation Cubic Spline Interpolation Hermite Cubic vs Spline Interpolation

Matlab “pchip()” function

Michael T. Heath Scientific Computing 50 / 56 Parametric Interpolation

❑ Important when y(x) is not a function of x; then, define [ x(t), y(t) ] such that both are (preferably smooth) functions of t.

❑ Example 1: a circle. Parametric Interpolation: Example 2

❑ Suppose we want to approximate a cursive letter.

❑ Use (minimally curvy) splines, parameterized. Parametric Interpolation: Example 2

❑ Once we have our (xi , yi ) pairs, we still need to pick ti .

❑ One possibility: ti = i , but usually it’s better to parameterize by arclength, if x Stable Lagrangeand y have Interpolation the same units.

An approximate arclength is • ❑ An approximate arclength is: i

si = dsj,dsi := xi xi 1 2 || || j=0 X

❑ Note – can also have Lagrange parametric interpolation… Parametric Interpolation: Example 2

Pseudo-Arclength-Based Index-Based Multidimensional Interpolation

❑ Multidimensional interpolation has many applications in computer aided design (CAD), partial differential equations, high-parameter data fitting/assimilation.

❑ Costs considerations can be dramatically different (and of course, higher) than in the 1D case.

2D basis function, N=10 Multidimensional Interpolation

❑ There are many strategies for interpolating f(x,y) [ or f(x,y,z), etc.]. ❑ One easy one is to use tensor products of one-dimensional interpolants, such as bicubic splines or tensor-product Lagrange polynomials. n n

pn(s, t)= li(s) lj(t) fij i=0 j=0 X X n n

= li(s) fij lj(t) i=0 j=0 X X

2D Example: n=2

1 Consider 1D Interpolation

n

f˜(s)= lj(s) fj j=1 X n

f˜(s)= lj(s) fj j=1 X n

f˜i = lij fj,lij := lj(si) j=1 X f˜ = Lf

s – fine mesh (i.e., target gridpoints) • i L is the matrix of Lagrange cardinal polynomials (or, say, spline • bases) evaluated at the target points, si, i =1,...,m.

1 Two-Dimensional Case (say, n x n à m x m)

n n

f˜(s, t)= li(s) lj(t) fij i=1 j=1 X X n n

= li(s) fij lj(t) i=1 j=1 X X

n n

f˜pq := f˜(sp,tq):= lpi fij lqj i=1 j=1 X X n n T := lpi fij ljq i=1 j=1 X X F˜ = LF LT ! matrix-matrix product, fast

Note that the storage of L is mn < m2 +n2,whichisthestorage • of F˜ and F combined. That is, in higher space dimensions, the operator cost (L)isless • than the data cost (F˜, F ). This is even more dramatic in 3D, where the relative cost is mn • to m3 + n3. Observation: It is dicult to assess relevant operator costs • based on 1D model problems.

1 n n

f˜(s, t)= li(s) lj(t) fij i=1 j=1 X X n n

= li(s) fij lj(t) i=1 j=1 X X

n n

f˜pq := f˜(sp,tq):= lpi fij lqj i=1 j=1 X X n n T := lpi fij ljq i=1 j=1 X X Two-DimensionalF ˜Case= LF(say, LT n x n à m x m)

Note that the storage of L is mn < m2 +n2,whichisthestorage • of F˜ and F combined. That is, in higher space dimensions, the operator cost (L)isless • than the data cost (F˜, F ). This is even more dramatic in 3D, where the relative cost is mn • to m3 + n3. Observation: It is dicult to assess relevant operator costs • based on 1D model problems.

1 Aside: GLL Points and Legendre Polynomials

7 3