Interpolation
• In interpolation or extrapolation we usually want to do the following
- We have data points xi yi , i = 12} N
- We want to know the y value at xxz i
- In interpolation x1 xxN and in extrapolation xx 1 or xx! N - Extrapolation is dangerous; it is used e.g. in solving differential equations.
- Data set may have noise: the interpolate should go smoothly through the data set not necessarily through all points.
- One application is approximating (special) functions - In this case we have an infinite number of points available.
- In some cases interpolation is done by using a few points in the neighborhood of x . - This may result in noncontinuous derivative of the interpolate. - In spline interpolation one condition is that also the derivative is continuous.
- In polynomial interpolation one is not particularly interested in the polynomial coefficients only in its values. - Calculating coefficients is rather error prone.
Scientific computing III 2011: 6. Interpolation 1
Interpolation
• Interpolation vs. curve fitting:
Scientific computing III 2011: 6. Interpolation 2 Interpolation
• Degree of interpolation is (number of points used)-1. - Below an example of interpolation of a smooth function
original function
low degree interpolation
high degree interpolation
- When the original function has sharp corners an interpolation polynomial with a lower degree may work better
Scientific computing III 2011: 6. Interpolation 3
Interpolation: polynomials
• We have a data set xi yi , yi = fx()i , i = 1} N
- We have to find a polynomial P N – 1 x that fulfills the condition
PN – 1()xi = yi , i = 1} N
- It is easy to show that the polynomial is at most of degree N – 1 and it is unique if all x i are different.
- A straightforward way to determine the coefficients is to use the methods we have already learned
- Let the polynomial be of the form
2 N–1 yc=1+c2xc+3x ++}cNx
- From the above condition we get a group of linear equations
2 N–1 °c1+c2x1+c3x1 ++}cNx1 =y1 ° °c+cx+cx2 ++}cxN–1=y ®12232 N2 2 ° } ° °c+cx+cx2 ++}cxN–1=y ¯12N3N NN N
Scientific computing III 2011: 6. Interpolation 4 Interpolation: polynomials
• Theorem: existence of interpolating polynomial:
If points x1 x2 } xN are distinct, then for arbitrary real values y1 y2 } yN there is a unique polynomial P of
degree d N – 1 such that Px i = yi for 1 d i d N .
- Proof by induction:
Suppose we have already a polynomial P that reproduces a part of the data set: Px i = yi , 1 ddik ,
(For example: this can be a constant polynomial for data point 1: Px = y1 .)
Then we add another term to P so that it will go through the point xk + 1 yk + 1 :
Qx = Px + cxx – 1 xx– 2 } xx– k
Q reproduces data points 12 } k because P does and the added term is zero for all these points.
Now we adjust constant c so that Q reproduces the data point k + 1 :
Qx k + 1 = Px k + 1 + cx k + 1 – x1 xk + 1 – x2 } xk + 1 – xk = yk + 1 .
From this equation we can solve c because all x i are distinct.
QED.
Scientific computing III 2011: 6. Interpolation 5
Interpolation: polynomial
- In matrix form:
1 x x2 } xN – 1 1 1 1 c1 y1 2 N – 1 1 x2 x2 } x2 c2 y2 2 N – 1 c = y 1 x3 x3 } x3 3 3 }}}} } } } 2 N–1 cN yN 1 xNxN}xN
- This square matrix has a name of its own: Vandermonde matrix and its a little bad behaving:
>> v=1:5 >> v=1:8 v = v = 1 2 3 4 5 1 2 3 4 5 6 7 8 >> m=vander(v) >> m=vander(v) m = m = 1 1 1 1 1 1 1 1 1 1 1 ... 16 8 4 2 1 128 64 32 16 8 4 ... 81 27 9 3 1 2187 729 243 81 27 9 ... 256 64 16 4 1 16384 4096 1024 256 64 16 ... 625 125 25 5 1 78125 15625 3125 625 125 25 ... >> rcond(m) 279936 46656 7776 1296 216 36 ... ans = 823543 117649 16807 2401 343 49 ... 2.2699e-05 2097152 262144 32768 4096 512 64 ... >> rcond(m) ans = 6.0171e-10
Scientific computing III 2011: 6. Interpolation 6 Interpolation: polynomial
- A better way is to calculate P N – 1 by using so called Lagrange’s polynomials.
- Let the polynomials l1 l2 } lN be defined as
N§·xx– l ()x = ¨¸------j, i=1}N i ©¹x–x j=1i j jiz
- These functions have the property
li()xj= Gij
- Now we can write P N – 1 x as
N P ()x = fx()l()x N – 1 ¦ i i i=1
- It is easy to check that P N–1 x goes through all the data points xiyi .
- Because functions l i have degree less than N it follows that P N–1 also has degree less than N .
Scientific computing III 2011: 6. Interpolation 7
Interpolation: polynomial
- Example:
x : 13e 14e 1 43e y : 2 –1 7 2
1 4 §·x– - -- x–1§·x– - -- ©¹4 ©¹3 93 69 l x = ------= 1 8 x 3 – ------x 2 + ------x – 6 1 11 1 14 2 2 §·- -- – - -- §·---1– §·- -- – - -- ©¹34 ©¹3©¹33 1 4 §·x– - -- x–1§·x– - -- ©¹3 ©¹3 192 512 1216 256 l x = ------= – ------x 3 + ------x 2 – ------x + ------2 11 1 14 13 13 39 39 §·- -- – - -- §·---1– §·- -- – - -- ©¹43 ©¹4©¹43 1 1 4 §·x– - -- §·x– - -- §·x– - -- ©¹3©¹4©¹3 23 31 2 l x = ------= – 6 x 3 + ------x 2 – ------x + - -- 3 1 1 4 2 6 3 §·1– - -- §·1– - -- §·1– - -- ©¹3©¹4©¹3 1 1 §·x– - -- §·x– - -- x–1 ©¹3©¹4 36 57 24 3 l x = ------= ------x 3 – ------x 2 + ------x – ------4 1 41 4 13 13 13 13 §·-4 -- – - -- §·- -- – - -- §·---1– ©¹33 ©¹34 ©¹3 186 1577 5281 560 P x = 2l x–l x+7l x+2l x = ------x 3 – ------x 2 + ------x – ------3 1 2 3 4 13 26 78 39
Scientific computing III 2011: 6. Interpolation 8 Interpolation: polynomials
- So we end up with the interpolation polynomial (let’s leave the subscript out)
xx–2 xx–3 } xx–N Px()= ------y 1 x1–x2 x1–x3 } x1 – xN xx–1 xx–3 } xx–N + ------y 2 x2–x1 x2–x3 } x2 – xN + }
xx–1 xx–2 } xx–N–1 + ------y N xN–x1 xN–x2 } xN – xN – 1
- The error estimation of the above polynomial can be given as follows:
- Let x 1 x 2 } x N be distinct numbers in >@ab and let’s assume that f has N continuous derivatives in >@ab - Then for each x >@ ab [ x ab so that
N f N [ x fx =Px +------xx– (1) N! i i=1
Scientific computing III 2011: 6. Interpolation 9
Interpolation: polynomials
- Proof:
- For xx= k , k = 12 } N fx k = Px k and any [ xk ab fulfills (1)
- For x k z x we define function gt as tx– gt = ft –Pt – >@fx –Px N ------i(2) i = 1 xx–i
- Since f has N continuous derivatives and P has all derivatives continuous and xx z k o gt has N continuous deriv- atives in >@ab
- For tx= k Generalized Rolle’s theorem: N x–x gx = fx –Px – >@fx –Px ------ki= 0 Assume k k k 1. fx continuous on >@ab , xx–i i = 1 2. derivatives f 1 }x f N x exist in ab , 3. x x x ab, - For tx= 01 } N >@ 4. fx j = 0 , for j = 01} N . N xx– Then , , such that N . gx = fx –Px – >@fx –Px ------i= 0 c acb f c = 0 xx– i = 1 i
- Thus g vanishes at N + 1 points xx 1 x 2 } x N in >@ab . - Generalized Rolle’s theorem says that [[ { x in ab for which g N [ =0 . - From (2) we get dN½ tx– 0=g N [ =f N [–P N [–>@fx –Px ------N ------i (3) N®¾ i = 1 dt ¯¿ xx–i t=[
Scientific computing III 2011: 6. Interpolation 10 Interpolation: polynomials
- Now P is at most of degree N – 1 o P N { 0 - The product term is a polynomial of degree N o
N N –1 N tx– dN½ tx– N! ------i= xx– tN+Ot N–1 o ------i = ------ i N®¾ N xx–i dt ¯¿ xx–i xx– i = 1 i=1 i = 1 i=1 i - (3) now becomes N! 0 = f N [– >@fx –Px ------N xx– i=1 i - And solving for fx we get f N [ fx =Px +------N xx– N! i=1 i
QED.
- Note the analogy between the error formula of the Taylor series: f N [ ------xx– N N! 0 and f N ------[ xx– xx– } xx– N! 1 2 N
Scientific computing III 2011: 6. Interpolation 11
Interpolation: polynomials
- Example: - Prepare a table for function fx =ex , x>@01 . - Precision d decimals, step size h - What step size is needed for linear interpolation to give absolute error no more than 10–6 ?
- Let x >@01 and xj d x d xj + 1 - Error is now f 2 f 2 f 2 fx –Px d------[ xx– xx– =------[ xx– xx– =------[ x– jh xj– +1h 2! j j+1 2 j j+1 2 1 fx –Px d---max f 2 [ max x– jh xj– +1h 2 [ >@01 xj ddxxj+1 Maximum of x– jh xj– +1h is at xj= +12eh with value h2e4
eh2 o fx –Px d ------8
- We want the error to be less than 10–6 : eh2 8 –3 ------d 1 0 – 6 h 2 d ---10–6 h 1.72u10 8 e
- So we could choose h=0.001
Scientific computing III 2011: 6. Interpolation 12 Interpolation: polynomials
- As a figure:
Scientific computing III 2011: 6. Interpolation 13
Interpolation: polynomials
- In principle one could calculate the interpolating values from the above equation we obtained:
xx–2 xx–3 } xx–N Px()= ------y 1 x1–x2 x1–x3 } x1 – xN xx–1 xx–3 } xx–N + ------y 2 x2–x1 x2–x3 } x2 – xN + }
xx–1 xx–2 } xx–N–1 + ------y N xN–x1 xN–x2 } xN – xN – 1
- However, there are more efficient ways to do that: so called Neville’s algorithm:
- Let P 1 be a zero-degree polynomial going throught the first point x 1 y 1 ; i.e. P 1 = y 1 .
- In the same way define P 2 P 3 } P N
- Let P 12 be a first-degree polynomial going through points x 1 y1 and x2 y2 ,
- In the same way define P23P34 } P N + 1 N
- Going further we can define polynomials with higher degrees until we get P 123}N which is what we want.
Scientific computing III 2011: 6. Interpolation 14 Interpolation: polynomials
- Different P ‘s can be written as an array with parents and children; for N = 3 :
x1 y1 = P1
P12
x2 y2 = P2 P123
P23
x3 y3 = P3
- Neville’s algorithm fills the above array from left to right one column at a time.
- The recursion formula for the P ‘s is
Pii +1 } im+ xx– P + x–xP = ------im+ ii +1 } im+ –1 i i+1 } im+ xi–xim+
- This recursion works because the parents have the same values at points x i + 1 } x im+ –1
Scientific computing III 2011: 6. Interpolation 15
Interpolation: polynomials
- Neville’s algorithm is based on Aitken’s lemma:
Let P 1n x be a Lagrange’s polynomial with degree n – 1 which satisfies
P1n()xi = yi , i = 1} n
Let P 2 n + 1 x be a Lagrange’s polynomial with degree n – 1 which satisfies
P2 n + 1()xi = yi , i = 2} n + 1
Now the polynomial P1 n + 1()x formed from the data set xi yi , i = 1} n + 1 can be obtained from the formula
xx– 1P2n+1()x – xx– n+1P1n()x P1 n + 1()x = ------xn+1–x1
- This is trivially true since P 1 n + 1 ()x is a polynomial of degree n satisfying P 1 n + 1 ()x i = y i , i = 1 } n + 1
- This can be generalized to any interval ij :
xx–ij+Pii +1 } ij+–1+ xi–xP i+1 i+2 } ij+ P ii +1 } ij+ = ------xi–xij+
Scientific computing III 2011: 6. Interpolation 16 Interpolation: polynomials
- Graphically: i i + 1 Pii +1 } ij+–1½ i + 2 ¾Pii +2 } ij+ } P ¿ i+1 i+2 } ij+ ij+–2 ij+–1 ij+
- For practical calculations it is better to simplify the notation:
Sij =Pii +1 i+2 } ij+ - Now the recursion relation becomes
xx–ij+Sij –1 + xi–xS i+1 j–1 Sij = ------with S i 0 = y i . xi–xij+ - For example
y 1 = S 10
S11
y2 = S20 S12
S21 S13
y3 = S30 S22
S31
y4 = S40
Scientific computing III 2011: 6. Interpolation 17
Interpolation: polynomials
- In Fortran90 this looks like program neville implicit none integer,parameter :: rk=selected_real_kind(15,100) real(rk),allocatable :: xa(:),ya(:),s(:,:) real(rk) :: x,y,x1,x2,dx integer :: n,i,j,ix,ixmax character(len=80) :: argu if (iargc()/=3) then call getarg(0,argu) write(0,’(a,a,a)’) ’usage: ’,trim(argu),’ x1 x2 dx’ stop end if call getarg(1,argu); read(argu,*) x1 ! First x call getarg(2,argu); read(argu,*) x2 ! Last x call getarg(3,argu); read(argu,*) dx ! Step of x read(5,*) n ! allocate(xa(n),ya(n),s(n,0:n-1)) ! Read in do i=1,n ! data xi yi read(5,*) xa(i),ya(i) ! end do ! s(1:n,0)=ya ixmax=(x2-x1)/dx do ix=0,ixmax ! Loop over x x=x1+dx*ix do j=1,n-1 ! Neville’s recursion loops to calculate Sij do i=1,n-j s(i,j)=((x-xa(i+j))*s(i,j-1)+(xa(i)-x)*s(i+1,j-1))/(xa(i)-xa(i+j)) end do end do write(6,*) x,s(1,n-1) end do stop end program neville
Scientific computing III 2011: 6. Interpolation 18 Interpolation: polynomials
- Example: sin x 4 points 8 points
Scientific computing III 2011: 6. Interpolation 19
Interpolation: polynomials
1 - Another example - -- x
4points 8 points 16 points
Scientific computing III 2011: 6. Interpolation 20 Interpolation: polynomials
- In practice the polynomial is often calculated not using the previous recursion relation but using the differences C mi
and Dmi :
Cmi =Pi} im+ –Pi} im+ –1, Dmi =Pi} im+ –P i+1 } im+ - Based on the recursion relation for the P ‘s it is easy to derive the recursion relation for these differences
xim++1–x Cmi +1–Dmi Dm+1i= ------xi–xim++1
x i – x C mi +1–Dmi C m + 1 i = ------xi–xim++1
- On level m the corrections D and C take the polynomial one degree higher.
- Final polynomial P 12}N is thus obtained by traversing the array starting from any y i and summing up the corrections.
x1 y1 = P1
P12
x2 y2 = P2 P123 Cm + 1 i P23 P1234 Dm + 1 i x3 y3 = P3 P234
P34
x4 y4 = P4
Scientific computing III 2011: 6. Interpolation 21
Interpolation: polynomials
- An example implementation: void polint(double xa[], double ya[], int n, double x, double *y, double *dy) { int i,m,ns=1; double den,dif,dift,ho,hp,w; double *c,*d; dif=fabs(x-xa[1]); c=vector(1,n); d=vector(1,n); for (i=1;i<=n;i++) { if ( (dift=fabs(x-xa[i])) < dif) { // Find the table entry nearest to x ns=i; dif=dift; } c[i]=ya[i]; d[i]=ya[i]; } *y=ya[ns--]; for (m=1;m Scientific computing III 2011: 6. Interpolation 22 Interpolation: polynomials • Neville’s algorithm does not give us the coefficients of the polynomial. - They can be calculated by using so called divided differences. - The proof of the theorem on existence of interpolating polynomial provides us a method for constructing it: Newton’s algorithm - We start at zero degree polynomial: P0 x = y0 - Add the second term: Note: Now indexing of the data points starts from 0: xi yi , i = 012} N . P1 x = P0 x + a1 xx– 0 = y0 – a1 xx– 0 ; y1–y0 P1 x1 =y1a1= ------x1–x0 - Add the third term: P 2 x = P 1 x + a 2 xx– 0 xx– 1 = y 0 + a 1 xx– 0 + a 2 xx– 0 xx– 1 By setting P 2 x 2 = y 2 we obtain constant a 2 . - Iteration for the k th polynomial is P k x = P k – 1 x + a k xx– 0 xx– 1 } xx– k – 1 - The final polynomial going through all the points is P N x = a 0 + a 1 xx– 0 +a 2 xx– 0 xx– 1 } ++a N xx– 0 xx– 1 } xx– N – 1 , where a 0 = y 0 . Scientific computing III 2011: 6. Interpolation 23 Interpolation: polynomials - Example: Scientific computing III 2011: 6. Interpolation 24 Interpolation: polynomials - The same data using the Lagrange polynomial: Scientific computing III 2011: 6. Interpolation 25 Interpolation: polynomials - Due to the uniqueness of the interpolating polynomial the Lagrange’s and Newton’s polynomials are the same: 1 1 1 1 Lagrange: Px =–36§·x – - -- x–1–16§·x – - -- x–1+14§·x – - -- §·x– - -- ©¹4 ©¹3 ©¹3 ©¹4 1 1 1 Newton: Px =2+ 36§·x – - -- –38§·x – - -- §·x– - -- ©¹3 ©¹3 ©¹4 Emacs package imaxima: uses TeX to output maxima results. Scientific computing III 2011: 6. Interpolation 26 Interpolation: polynomials - An efficient way to evaluate Newton’s polynomial is nested multiplication. - Newton’s polynomial Px = a0 + a1 xx– 0 +a2 xx– 0 xx– 1 } ++aN xx– 0 } xx– N can be written in the form Ni–1 Px =a+a xx– 0 ¦i j i=1j=0 - Using the common terms xx–i for factoring we obtain the nested form Px =a0+ xx–0 a1+ xx–1 a2+ xx–2 a3+} xx–N–1aN }) - For evaluating Px for xt = we begin from the innermost parenthesis Q0=aN Q1=Q0 tx–N–1+aN–1 Q2=Q1 tx–N–2+aN–2 } QN=QN–1 tx–0+a0 - Q N is the value of the polynomial. Scientific computing III 2011: 6. Interpolation 27 Interpolation: polynomials - An efficient way to compute the coefficients of the Newton’s polynomial is based on divided differences. - We have N + 1 points xi fx i i = 0} N - Newton’s polynomial interpolating these points is Ni–1 P x =a xx– ( –1 xx– is interpreted as 1 .) N ¦i j j=0 j i=0j=0 - Coefficients a i do not depend on N: P N is obtained from P N–1 by adding one more term without changing PN–1 . - A systematic way to obtain a i is to set x equal to points x i one at a time: fx 0 =a0 ° °fx 1 =a0+a1 x1–x0 ® °fx =a+a x–x+a x–x x–x °2 0 11 0 22021 ¯} or in compact form ki–1 fx k=¦ai xk–xj 0 kN i=0j=0 Scientific computing III 2011: 6. Interpolation 28 Interpolation: polynomials - We can now see that a k depends on the values of f at points x0x1 } xk ; formally: ak = fx>@0x1 } xk - Quantity fx>@ 0x1 } xk is called the divided difference of order k for f . Scientific computing III 2011: 6. Interpolation 29 Interpolation: polynomials - Newton’s polynomial now takes the form Ni–1 ½ PN x =®¾fx>@0x1}xi xx–j ¦¯¿ i=0j=0 - Divided differences can be computed recursively: ki–1 k–1 k–1 i–1 fx k=¦ai xk–xj =fx>@0}xk xk–xj+¦fx>@0}xi xk–xj i=0j=0 j=0 i=0 j=0 i–1 fx –k–1fx>@}x x–x k ¦i=00 i kj j=0 o fx>@0}xk= ------k–1 x–x j=0kj - Possible algorithm: 1. Set fx>@0 = fx 0 2. For k = 12} N compute fx>@0 } x k from the equation above. - Well, there are more efficient ways to do this. Scientific computing III 2011: 6. Interpolation 30 Interpolation: polynomials - Recursive property of divided differences: fx>@1}xk –fx>@0}xk–1 fx>@0}xk = ------xk–x0 - Uniqueness of the polynomial: no change if points x i y i permuted o The divided difference fx>@ 0 } x k is invariant under permutations of arguments x 0 } x k . - This means that the recursive relation can be written as fx>@i+1}xj–fx>@i}xj–1 fx>@ixi+1}xj–1xj = ------xj–xi - Now the divided differences can be evaluated as follows: fx>@i = fx i fx>@i + 1 = fx i + 1 fx>@i + 2 = fx i + 2 fx>@i+1–fx>@i fx>@i+2–fx>@i+1 fx>@ixi+1 = ------fx>@i+1xi+2 = ------xi+1–xi xi+2–xi+1 fx>@i+1xi+2–fx>@ixi+1 fx>@ixi+1xi+2 = ------xi+2–xi Scientific computing III 2011: 6. Interpolation 31 Interpolation: polynomials - We can now construct a similar table for f>@} ‘s as in the Neville’s method: x0 fx>@0 fx>@0 x1 x1 fx>@1 fx>@0x1 x2 fx>@1 x2 fx>@0x1 x2 x3 x2 fx>@2 fx>@1x2 x3 fx>@2 x3 x3 fx>@3 - To summarize the story so far: Interpolating polynomial Ni–1 PN x =¦ai xx–j =a0+a1 xx–0+a2 xx–0 xx–1 } ++aN xx–0 } xx–N–1 i=0j=0 where ak=fx>@0}xk, fx>@1}xk–fx>@0}xk–1 fx>@0}xk = ------xk–x0 fx>@k = fx k . Scientific computing III 2011: 6. Interpolation 32 Interpolation: polynomials - Example: Scientific computing III 2011: 6. Interpolation 33 Interpolation: polynomials Scientific computing III 2011: 6. Interpolation 34 Interpolation: polynomials Scientific computing III 2011: 6. Interpolation 35 Interpolation: polynomials - Using the bold face values from the final table we can construct the Newton’s polynomial 3i–1 ½ PN x =®¾fx>@0x1}xi xx–j ¦¯¿ i=0j=0 =fx>@0 +fx>@0x1 xx–0++fx>@0x1x2 xx–0 xx–1 fx>@0x1x2x3 xx–0 xx–1 xx–2 11 3 3 1 ==§·3+- -- x – 1 +- -- x – 1 §·x– - -- –2 x–1§·x– - -- x–0 –---6 x3– 16x2 + 10x – 9 ©¹23 ©¹2 ©¹2 3 Scientific computing III 2011: 6. Interpolation 36 Interpolation: polynomials - Algorithm that computes all the divided differences can now be constructed. - There is no need to store all f>@ } because only fx>@0 , fx>@0 x1 , ..., fx>@ 0} xN are needed. - Denote ai = fx>@0x1 } xi Newton’s polynomial can be written as Ni–1 PN x =¦ai xx–j i=0j=0 - We can use 1D array ai(0:N) to store divided differences - At each step the new divided difference is obtained by ai–ai–1 aim ------xi–xij– Scientific computing III 2011: 6. Interpolation 37 Interpolation: polynomials - And in C: void Coeffs(int n, double *x, double *y, double *a) { int i, j; A simpler but more memory consuming way: for (i=0;i<=n;i++) a[i]=y[i]; double x[n+1],y[n+1],a[n+1][n+1]; for (i=0;i<=n;i++) a[i]=y[i]; for (j=1;j<=n;j++) for(j=1;j<=n;j++) for (i=n;i>=j;i--) for(i=0;i<=n-j;i++) a[i]=(a[i]-a[i-1])/(x[i]-x[i-j]); a[i][j]=(a[i+1][j+1]-a[i][j-1])/ (x[i+j]-x[i]); } - Evaluation of the polynomial: a[0] double Evaluate(int n, double *x, 0 double *a, double t) { int i; a[1] double pt; pt = a[n]; 1 01 for (i=n-1;i>=0;i--) pt=pt*(t-x[i])+a[i]; a[2] return(pt); } 2 12 012 a[3] 3 23 123 0123 Scientific computing III 2011: 6. Interpolation 38 Interpolation: polynomials - Example: - sin x in >@0 1.6875 - Interpolation based on 10 equidistant points Polynomial coefficients Scientific computing III 2011: 6. Interpolation 39 Interpolation: polynomials - Interpolation error: Scientific computing III 2011: 6. Interpolation 40 Interpolation: polynomials • Inverse interpolation: - Approximation to inverse of a function - Values yi = fx i calculated for x0x1 } xN - Form the interpolation polynomial Ni–1 PN y =¦ci yy–j i=0j=0 - It is easy to use the abovementioned routines to compute ci - Inverse interpolation can also be used to locate the root of a given function. Scientific computing III 2011: 6. Interpolation 41 Interpolation: polynomials • Finally a word of warning on using high-degree polynomials: they do go through the data points but may oscillate wildly between them: Scientific computing III 2011: 6. Interpolation 42 Interpolation: rational functions • In many cases it is impossible to get a good interpolation using only polynomials - This means that the polynomial oscillates wildly between points - This may be caused by a singularity in the function to be interpo- lated. 1 - It need not be in the interpolating interval; it may be near it or y = ------1 + x2 not at all on the real axis. - In these cases a good choice might be a rational function - We define the rational function R ii +1 } im+ so that it goes through points xiyi } xim+yim+ : P()x p + p x ++} p x P R ()x= ------P = ------0 1 P ii +1 } im+ Q()x Q Q q0+q1x++}qQx - You have to give the order of the numerator and denominator when specifying the interpolating function. - Since there are PQ + + 1 unknowns we must have m + 1 = PQ+ + 1 - It is possible to derive a recursion relation as in the case of Neville’s algorithm: R i+1 } im+ –Ri} im+ –1 R ii +1 } im+ = R i+1 } im+ + ------§·xx–i §·R i+1 } im+ –Ri} im+ –1 ¨¸------¨¸1– ------–1 ©¹xx–im+ ©¹R i+1 } im+ –R i+1 } im+ –1 - It is started with R i = y i and R ii +1 } im+ =0 , when m=–1 . Scientific computing III 2011: 6. Interpolation 43 Interpolation: polynomials • A sidenote: As expected, polynomial interpolation is used in computing transcendental functions in math libraries - E.g. function sin x from the libm sources: /* @(#)k_sin.c 1.3 95/01/18 */ * For better accuracy, let /* * 3 2 2 2 2 * ======* r = x *(S2+x *(S3+x *(S4+x *(S5+x *S6)))) * Copyright (C) 1993 by Sun Microsystems, Inc. All rights reserved. * then 3 2 * * sin(x) = x + (S1*x + (x *(r-y/2)+y)) * Developed at SunSoft, a Sun Microsystems, Inc. business. */ * Permission to use, copy, modify, and distribute this * software is freely granted, provided that this notice * is preserved. * ======*/ /* __kernel_sin( x, y, iy) * kernel sin function on [-pi/4, pi/4], pi/4 ~ 0.7854 * Input x is assumed to be bounded by ~pi/4 in magnitude. * Input y is the tail of x. * Input iy indicates whether y is 0. (if iy=0, y assume to be 0). * * Algorithm * 1. Since sin(-x) = -sin(x), we need only to consider positive x. * 2. if x < 2^-27 (hx<0x3e400000 0), return x with inexact if x!=0. * 3. sin(x) is approximated by a polynomial of degree 13 on * [0,pi/4] * 3 13 * sin(x) ~ x + S1*x + ... + S6*x * where * * |sin(x) 2 4 6 8 10 12 | -58 * |------(1+S1*x +S2*x +S3*x +S4*x +S5*x +S6*x ) | <= 2 * | x | * * 4. sin(x+y) = sin(x) + sin’(x’)*y * ~ sin(x) + (1-x*x/2)*y Scientific computing III 2011: 6. Interpolation 44 Interpolation: rational functions - Below two examples where rational function interpolation works better than polynomial: Scientific computing III 2011: 6. Interpolation 45 Interpolation: numerical differentiation • Numerical differentiation is not a trivial task. - Subtracting almost equal numbers causes loss of significant figures. • The crudest approximation is based on the definition of the derivative fc : fxh + –fx fc x | ------h - Error truncation estimation can be obtained from the remainder term fxh+ –fx 1 f ' x =------–- -- hf 2 [ h 2 - This converges as Oh ; for Oh 2 we have to take more terms. - Consider the following Taylor’s series 1 1 1 fxh + =fx +hf' x ++++----- h 2 f 2 x ----- h 3 f 3 x ----- h 4 f 4 }x ° 2! 3! 4! ® 1 1 1 °fxh – =fx –hf' x +----- h 2 f 2 x–----- h 3 f 3 x++----- h 4 f 4 }x ¯ 2! 3! 4! - By subtraction we get 2 2 fxh + –fxh – =2hf' x +----- h 3 f 3 x+----- h 5 f 5 x 3! 5! fxh + –fxh – 1 1 o f' x =------–----- h 2 f 3 x–----- h 4 f 5 x–} i.e. Oh 2 ! 2h 3! 5! Scientific computing III 2011: 6. Interpolation 46 Interpolation: numerical differentiation - Including the truncation term we get fxh+ –fxh– 1 f' x =------–- -- h 2 f 3 [ , [>@xh–xh+ 2h 6 - We can write fxh+ –fxh– 1 1 f' x =------–----- h 2 f 3 x–----- h 4 f 5 x–} 2h 3! 5! fxh()+ –fxh()– as f'()x =------+ a h 2 +++a h 4 a h 6 } 2h 2 4 6 - Here the coefficients a 2n only depend on f and x not on h. - A method called Richardson extrapolation is based on this fact. - Let’s define function I in a following way ( f and x now constants) fxh()+ –fxh()– I()h= ------2h - This is the approximation to derivative f c x and its error behaves as Oh 2 . - Our goal is to compute the limit lim I()h=f'()x ho0 Scientific computing III 2011: 6. Interpolation 47 Interpolation: numerical differentiation - We can calculate I for any set of h n and use these as the approximation to fc x . - However, we are smart and want to utilize the previous results in the series hn to calculate the next member. - Assume we have approximations I h and I h e 2 . fxh()+ –fxh()– - As we have f'()x = ------+ a h 2 +++a h 4 a h 6 } we can write 2h 2 4 6 I()h= f'()x–ah2–ah4–ah6– } (1) ° 2 4 6 ® h h2 h4 h6 °I()- -- = f'()x–a§·- -- –a§·- -- –a§·- -- – } (2) ¯ 2 2©¹2 4©¹2 6©¹2 - Now calculate (1) – 4u (2) h 3 15 o I()4h – I()- -- = –3f'()x– - -- a h 4 – ------a h 6 – } 2 4 4 16 6 h 1 h 1 5 o I ()- -- +- -- I ()- -- –I()h =f'()x+ - -- a h 4 ++------a h 6 } 2 3 2 4 4 16 6 - So, by adding the term 13 e I>@()he2–I()h to I() he 2 we have obtained a more accurate estimation to fc x! 4 - We could go on by killing the error term a 4he4 from the series above. - This is called Richardson extrapolation (RE). (Extrapolation in the sense ho0 .) Scientific computing III 2011: 6. Interpolation 48 Interpolation: numerical differentiation - Generally: Assume we have a function I h : f I()h =La– h2k (1) ¦2k k=1 - Coefficients a 2k are unknown. - Goal is to approximate quantity L using function I h : we want to calculate the limit L=lim I h ho0 - Choose a value for h and calculate the numbers h Dn I1 =§·----- , n=12} ©¹2n - According to (1) h2k Dn 1 =L+ Ak 1§·----- , ¦ ©¹2n where Ak 1=–a2k - Dn 1 give a rough estimate for L. Scientific computing III 2011: 6. Interpolation 49 Interpolation: numerical differentiation - We can get more accurate estimates by RE using the recursion relation: 4m 1 Dnm() +1 =------Dnm() –------Dn()–1m 4m–1 4m–1 - It can be shown that the quantities Dnm have the form f h2k Dnm() =L+ Akm() §·----- ¦ ©¹2n km= - Essential here is that the sum over k starts with term km= , so that the first error term is Oh 2m . - Numbers Dnm can be arranged into triangular form: D()11 D()21 D()22 D()31 D()32 D()33 }}} DN()1 DN()2 DN()3 } DNN() Scientific computing III 2011: 6. Interpolation 50 Interpolation: numerical differentiation - A practical algorithm for RE is the following: 1. Write a subroutine for function I h . 2. Choose suitable values for N and h . 3. For i = 12} N compute Di I1 = he2i 4. For 1ddd ijN compute Dij() +1 =Dij()4 + j–1–1>@Dij() –Di()–1j - The algorithm is easy to implement: void Derivative(double (*f)(double x), double x, int N, double h, double D[NMAX][NMAX]) { int i, j; double hh; hh=h; for (i=0; i<=N; i++) { D[i][0]=(f(x+hh)-f(x-hh))/(2.0*hh); for (j=0; j<=i-1; j++) D[i][j+1] = D[i][j]+(D[i][j]-D[i-1][j])/(pow(4.0,j+1.0)-1.0); hh = hh/2.0; } } Scientific computing III 2011: 6. Interpolation 51 Interpolation: numerical differentiation - And the main program: #include main (int argc, char **argv) { double x,h,D[NMAX][NMAX]; int N,i,j; if (argc!=4) { fprintf(stderr,"Usage: %s N x h\n",argv[0]); return (1); } N=atoi(*++argv); x=atof(*++argv); Function fx : h=atof(*++argv); if (N>=NMAX) { double f(double x) { fprintf(stderr,"N should be smaller than %d.\n",NMAX); return exp(-x*x); return(-1); } } for (i=0;i Scientific computing III 2011: 6. Interpolation 52 Interpolation: numerical differentiation - Results from the run: progs> ./richardson 6 1.0 0.5 i: 0 h/ 1 : -0.6734015585 i: 1 h/ 2 : -0.7203428752 -0.7359899807 i: 2 h/ 4 : -0.7319209458 -0.7357803026 -0.7357663241 i: 3 h/ 8 : -0.7348004908 -0.7357603391 -0.7357590082 -0.7357588921 i: 4 h/ 16 : -0.7355193541 -0.7357589753 -0.7357588843 -0.7357588824 -0.7357588823 i: 5 h/ 32 : -0.7356990047 -0.7357588882 -0.7357588824 -0.7357588823 -0.7357588823 -0.7357588823 2 2 2 - Accurate value: fx =e–x o fc x = –2xe–x o fc 1 = – - -- = –0.73575888 e Scientific computing III 2011: 6. Interpolation 53 Interpolation: numerical differentiation • Numerical derivatives can also be computed using the derivative if the interpolated polynomial: fc x | Pc x - In practice the interpolating polynomial is determined only using a couple of points near the argument. - For two nodes: P1 x = fx 0 + fx>@0 x1 xx– 0 fx 1–fx 0 o fc x |Pc x = fx>@0x1 = ------x1–x0 - If x 0 = x and x 1 = xh+ we get fxh + –fx f c x | ------h - For x 0 = xh – and x 1 = xh+ fxh+ –fxh– f c x | ------(central difference form) 2h Scientific computing III 2011: 6. Interpolation 54 Interpolation: numerical differentiation - Now consider three nodes: P2 x = fx 0 + fx>@0 x1 xx– 0 + fx>@0x1 x2 xx– 0 xx– 1 - The derivative is P2c x = fx>@0 x1 + fx>@0x1 x2 2xx– 0 – x1 - This is the same as above but with the correction fx>@0x1 x2 2xx– 0 – x1 . - The accuracy of the leading term fx>@ 0 x1 depends on the choice of x 0 and x1 . - If xx= 0 + x1 e 2 o 2xx – 0 – x1 = 0 and correction term is zero. - This is the reason for the greater accuracy of the central difference form of derivative. - In general we can analyze the error by using the error bound for the interpolating polynomial: N f N+1 [ x fx –Px =------xx– N+1! i i=0 Scientific computing III 2011: 6. Interpolation 55 Interpolation: numerical differentiation - Let’s assume that P N is the polynomial of the least degree that interpolates f at points x0x1 x2 } xN . - Now we can write the error bound as: f N+1 [ fx –P x =------Z x N N+1! N where Z x = xx–i and [[= x i=0 - Differentiation gives 1 d 1 fc x–Pc x =------Z x f N+1 [+------f N + 1 [ Zc x N N+1!dx N+1! - If we can set x equal to a node the expression is simplified because Z xi =0 1 fc x =Pc x+------f N + 1 [ Zc x i N i N+1! i - Or if we can choose x such that Zc x=0 : 1 d fc x=Pc x+------Z x f N+1 [ N N+1!dx Scientific computing III 2011: 6. Interpolation 56 Interpolation: numerical differentiation - Example: Derive the first-derivative formula using the interpolating polynomial P 3 x obtained with four nodes x0x1 x2 x3 . Use central difference in choosing the nodes. - Newton form of the polynomial P3 x =fx 0 +fx>@0 x1 xx– 0 +fx>@0x1 x2 xx– 0 xx– 1 +fx>@0x1x2x3 xx–0 xx–1 xx–2 - And its derivative Note that: Pc x=fx>@x +fx>@xx^` xx– + xx– 3 01 012 0 1 Zc x = xx– +fx>@x x x ^` xx– xx– + xx– xx– + xx– xx– ¦ j 0 1 2 3 1 2 0 2 0 1 i jiz - Formulas for the divided differences are obtained in a familiar way: fx>@i=fx i fx>@i+1–fx>@i fx>@ixi+1 = ------xi+1–xi fx>@i+1xi+2–fx>@ixi+1 fx>@ixi+1xi+2 = ------xi+2–xi Scientific computing III 2011: 6. Interpolation 57 Interpolation: numerical differentiation - Now we choose the nodes using the central difference; i.e. x is in the middle of the nodes: x0 = xh– , x1 = xh+ , x2 = x – 2h , x3 = x + 2h - This allows us to write down the divided differences: fx 1–fx 0fxh + –fxh – fx>@0x1 = ------= ------x1–x0 2h fx 2–fx 1fx –2h –fxh + fx>@1x2 = ------= ------x2–x1 –3h fx 3 –fx 2 fx +2h –fx –2h fx>@2x3 = ------= ------x3–x2 4h fx>@1x2 –fx>@0x1 1 ½fx –2h –fxh + fxh + –fxh – fx>@0x1x2 =------=------®¾------– ------x2–x0 – h ¯¿–3h 2h 1 =------>@ 2 fx –2h –3fxh – +fxh + 6h2 fx>@2x3 –fx>@1x2 1 ½fx +2h –fx –2h fx –2h –fxh + fx>@1x2x3 =------=- -- ®¾------– ------x3–x1 h ¯¿4h –3h 1 =------>@ 3 fx +2h –4fxh + +fx –2h 12h2 Scientific computing III 2011: 6. Interpolation 58 Interpolation: numerical differentiation - And the last one: fx>@1x2x3 –fx>@0x1x2 fx>@0x1x2x3 =------x3–x0 1 ½3 fx +2h –4fxh + +fx –2h 2 fx –2h –3fxh – +fxh + =------®¾------– ------3 h ¯¿12h2 6h2 1 =------>@fx +2h –2fxh + +2fxh – –fx –2h 12h3 - The following two factors are still needed: xx–0+ xx–1=2xx–1–x0=0 2 xx–1 xx–2+ xx–0 xx–2+ xx–0 xx–1=–h - The derivative of the polynomial P3c x= fx>@0x1 +fx>@0x1x2^` xx–0+ xx–1 +fx>@0x1x2x3^` xx–1 xx–2+ xx–0 xx–2+ xx–0 xx–1 = fx>@0x1 +fx>@0x1x2x3^` xx–1 xx–2+ xx–0 xx–2+ xx–0 xx–1 fxh+ –fxh– fx+2h –2fxh+ +2fxh– –fx–2h = ------– >@------2h 12h Scientific computing III 2011: 6. Interpolation 59 Interpolation: numerical differentiation - Approximation for the derivative is then fx–2h –8fxh– +8fxh+ –fx+2h fc x | ------12h - This can be used in the Richardson extrapolation to speed up the convergence. - Example: fx = sinx , x = 1.2309594 , h = 1 D(n,0) D(n,1) D(n,2) D(n,3) D(n,4) 0.32347058 0.33265926 0.33572215 0.33329025 0.33350059 0.33335248 0.33333063 0.33334409 0.33333365 0.33333335 0.33333317 0.33333401 0.33333334 0.33333334 0.33333334 - For comparison when only the first term fc x | >@fxh + – fxh – e 2h is used we get the result D(n,0) D(n,1) D(n,2) D(n,3) D(n,4) 0.28049033 0.31961703 0.33265926 0.32987195 0.33329025 0.33333232 0.33246596 0.33333063 0.33333332 0.33333333 0.33311636 0.33333317 0.33333333 0.33333334 0.33333334 Scientific computing III 2011: 6. Interpolation 60 Interpolation: Hermite interpolation • The interpolating function goes through data points xi fx i , i = 01} N : Px i = fx i • We can generalize this by adding requirements that the interpolating function also 2 reproduces derivatives: f1 x =x mi mi P xi =f xi . - The polynomial P is osculating to the function f . - We have N+ 1points xi yi and nonnegative integers m0m1 }mN . - The polynomial fulfilling these conditions has a degree at most N Mm=i+N. ¦ 2 4 i=0 f2 x =x –x k k - So our osculating polynomial interpolates the function f such that f1 0 =f2 0 for k =012 k k d d Px i = fx i for each i=01 }N and k=01}mi . dxk dxk Scientific computing III 2011: 6. Interpolation 61 Interpolation: Hermite interpolation - Note: 1) When N = 0 we have the m0 th Taylor polynomial for f at x0 . 2) When m i = 0 for i = 0} N we have the N th order interpolating polynomial for f - When m i = 1 for i = 0} N the interpolating polynomials are called Hermite polynomials: Px i = fx i and Pc xi = fc xi . - One can derive the following form for the polynomial using the divided differences1 2 2 2 2 HN x =Q00 +Q11 xx–0+++Q11 xx–0 Q33 xx–0 xx–1 Q44 xx–0 xx–1 2 2 2 2 ++}Q2n+12 n+1 xx–0 xx–1 xx–2} xx–n–1 xx–n - Coefficients Q ii are obtained by the following algorithm: 1. For i= 01}N do a) Set z2i=xi, z2i+1=xi , Q2i0=fx i , Q2i+01=fx i , Q2i+11=fc xi Q2i0–Q2i–10 b) If i z0 set Q2i1= ------z2i–z2i–1 Qij –1–Qi–1j–1 2. For i = 23} 2 N + 1 , for j = 23 } i set Q ij = ------zi–zij– 1. See e.g. Burden & Faires: Numerical Analysis, Ch. 3.5 Scientific computing III 2011: 6. Interpolation 62 Interpolation: Hermite interpolation - An in Fortran90 it reads program hermite end do implicit none do i=2,2*n+1 integer,parameter :: & do j=2,i &rk=selected_real_kind(15,100) q(i,j)=(q(i,j-1)-q(i-1,j-1))/(z(i)-z(i-j)) end do real(rk),allocatable :: xa(:),ya(:),& end do &da(:),q(:,:),z(:) real(rk) :: x,y,x1,x2,dx,p ! Use the coefficients to calculate P(x) integer :: n,i,j,ix,ixmax,jmax,iargc ixmax=(x2-x1)/dx character(len=80) :: argu do ix=0,ixmax call getarg(1,argu); read(argu,*) x1 x=x1+dx*ix call getarg(2,argu); read(argu,*) x2 y=q(0,0)+q(1,1)*(x-xa(0)) call getarg(3,argu); read(argu,*) dx do i=2,2*n+1 read(5,*) n p=q(i,i) allocate(xa(0:n-1),ya(0:n-1),da(0:n-1),& jmax=i/2-1 &q(0:2*n-1,0:2*n-1),z(0:2*n-1)) if (ix==0) write(0,*) i,jmax do i=0,n-1 do j=0,jmax read(5,*) xa(i),ya(i),da(i) p=p*(x-xa(j))**2 end do end do n=n-1 if (mod(i,2)==1) p=p*(x-xa(jmax+1)) q=0.0 y=y+p z=0.0 end do do i=0,n write(6,’(f10.4,g20.10)’) x,y z(2*i)=xa(i) end do z(2*i+1)=xa(i) q(2*i,0)=ya(i) stop q(2*i+1,0)=ya(i) end program hermite q(2*i+1,1)=da(i) if (i/=0) then q(2*i,1)=(q(2*i,0)-q(2*i-1,0))/& &(z(2*i)-z(2*i-1)) end if Scientific computing III 2011: 6. Interpolation 63 Interpolation: Hermite interpolation - Graphically: interpolating fx =e–x2 : - Hermite polynomial is often used in the form of cubic Hermite polynomial: Pa =fa ° °Pc a =fc a ® °Pb =fb ° ¯Pc b =fc b - The divided difference formula for this is 2 H3 x =fa + xa– f' a– xa– faab>@ 2 + xa– xb– faabb>@ Note: The following definition applies to divided differ- ences with repetition of the same argument n : n–1 f x fx>@x}x= ------i iii n–1! Scientific computing III 2011: 6. Interpolation 64 Interpolation: piecewise interpolation • High degree polynomial o oscillations. • Remedy: construct interpolating polynomials for subintervals of the data ^`xi fy i : piecewise polynomials or splines - The simplest spline is of the first order: linear interpolation S0 x xt>@0t1 ° °S1 x xt>@1t2 - In explicit form: Sx =® ° } ° ¯SN–1 x xt>@N–1tN Scientific computing III 2011: 6. Interpolation 65 Interpolation: piecewise interpolation - Where each piece of S is linear: Si x = aixb– i - In general a spline S of degree k fullfills the following conditions 1. The domain of S is interval >@ab 2. S, Sc, Scc , S 3 , } , S k – 1 are all continuous on >@ab . 3. There are points xi (knots of S) such that ax= 0 x1 } xn = b and that S is a polynomial of degree at most k on each subinterval >@xi xi + 1 . - A quadratic spline Qx is a continuously differentiable piecewise quadratic function. - Example: x2 xd0 ° Qx =®–x2 0ddx 1 ° ¯12–x xt1 Scientific computing III 2011: 6. Interpolation 66 Interpolation: piecewise interpolation - Ensure that the function Qx really is a quadratic spline. Check all knots: lim Qx = limx2 = 0 lim Qx = lim–x2 = 0 xo0– x o 0– xo0+ x o 0+ lim Qx = lim–x2 = – 1 limQx = lim 12–x= – 1 xo1– x o 1– xo1+ xo1+ lim2Qc x =lim0x= lim2Qc x = lim–x= 0 xo0– xo0–xo0+ xo0+ lim2Qc x=lim2–x=–lim2Qc x = lim –= – 2 xo1– xo1–xo1+ xo1+ - Let’s assume we have the data set ^`xi yi , i = 01} N and we want to interpolate it using quadratic splines. 2 o we have to determine N functions Qi x =aix +bixc+i , i=01}N–1 . o we have to determine 3 N coefficients - On each subinterval >@x ix i+1Q must satisfy: Qi xi=yi and Qi xi+1=yi+1 o 2N equations - Continuity of the first derivative gives Qci–1 xi=Qci xi, i=12}N–1 o N– 1equations - Only 3N –1 equations. Additional conditions e.g. Qc x0=0 . Scientific computing III 2011: 6. Interpolation 67 Interpolation: piecewise interpolation - So, we seek for a piecewise quadratic function Q0 x xx>0x1 ° °Q1 x xx>1x2 Qx =® °} ° ¯QN–1 x xx>N–1xN@ which is continuously differentiable on the interval >@ x0xN and which interpolates the table: Qx i=yi - Denote zi=Qc xi . We can write z –z i+1 i 2 Qi x=------xx–i +zi xx–i+yi 2 xi+1–xi - It easy to verify that Qi xi=yi Qci xi=zi Qci xi+1=zi+1 Scientific computing III 2011: 6. Interpolation 68 Interpolation: piecewise interpolation - In order to Q to be continuous and interpolate the data table ^` xi yi it is necessary and sufficient that Qi xi + 1 = yi + 1 - This gives us §·yi + 1 – yi zi + 1 =– zi +2¨¸------, 0 ddiN –1. ©¹xi + 1 – xi T - The vector >@ z0z1}zN can now be constructed by starting from (an arbitrary) value of z0=Qc x0 - Well, the recursion relation above is a group of linear equations (though a trivial one): z Qc x 1000}0 0 0 z y–ye x–x –1 1 00}0 1 1 0 10 z y–ye x–x 01– 10}0 3 = 2121 001–1}0 } } } zN–1 yN–1–yN–2e xN–1–xN–2 00}01–1 zN yN–yN–1e xN–xN–1 Scientific computing III 2011: 6. Interpolation 69 Interpolation: piecewise interpolation - The program that does the interpolation: program qspline z(0)=Qp0 do i=0,n-1 implicit none z(i+1)=-z(i)+2.0*(ya(i+1)& integer,parameter :: rk=selected_real_kind(15,100) & -ya(i))/(xa(i+1)-xa(i)) end do real(rk),allocatable :: xa(:),ya(:),z(:) real(rk) :: x,y,x1,x2,dx,Qp0 ixmax=(x2-x1)/dx integer :: n,i,j,ix,ixmax character(len=80) :: argu do ix=0,ixmax x=x1+dx*ix if (iargc()/=4) then if (x read(5,*) n allocate(xa(0:n),ya(0:n),z(0:n)) do i=0,n-1 read(5,*) xa(i),ya(i) end do n=n-1 Scientific computing III 2011: 6. Interpolation 70 Interpolation: piecewise interpolation - Random sets of knots created with gawk one-liner: gawk ’BEGIN {srand(); for (x=0;x<=5.01;x+=0.5) print x,rand()}’ - Graphically: 11 knots 41 knots Scientific computing III 2011: 6. Interpolation 71 Interpolation: piecewise interpolation • Splines of degree 3 are cubic splines and they a probably the most commonly used splines. - Cubic splines have more flexibility than quadratic splines o ‘smoother’ interpolant. - In general for splines the following theorem applies: m Let fx C >@ab and S 2m – 1 x be the interpolating spline of f. If gx is an m times continuously differentiable function that fulfills the same interpolating conditions as S 2m–1 then b b 2 m 2 S m x dxd g x dx ³ 2m–1 ³ a a - For cubic splines (m=2 ) we can write this in the form b b 2 2 S'' x dxd g'' x dx ³ 3 ³ a a - Now the curvature of a curve yx is defined as y'' x N x = ------232e ^`1+>@y' x - If we assume that y ' x « 1 in >@ab we can write N x | y '' x Scientific computing III 2011: 6. Interpolation 72 Interpolation: piecewise interpolation - So, for cubic splines we can say that they are the interpolants that have — on the average — the smallest curvature. - Another way to depict this is to imagine the spline as a solid wire that is forced to go through the knots and minimizes its bending energy. - As usual we have the data set or knots x x0 x1 } xN y y0 y1 } yN - We assume that knots are distinct and in ascending order: x0 x1 } xN - The interpolating function Sx consists of N cubic polynomials: S0 x xx>0x1 ° °S1 x xx>1x2 Sx =® i.e. S i x interpolates the subinterval >@xixi+1 °} ° ¯SN–1 x xx>N–1xN@ Scientific computing III 2011: 6. Interpolation 73 Interpolation: piecewise interpolation - The interpolation conditions are Si xi = yi , 0 ddiN - The continuity conditions imposed only at the interior knots x1 x2 } xN – 1 are lim S k x = lim S k x , k = 012 – i + i xxoi xxoi - Are these enough: We have N 3rd order polynomials o 4N parameters. Interpolation conditions o N + 1 equations. Continuity conditions o 3 N – 1 equations. Total 4 N – 2 equations. o need two more equations. - A common choice is S'' x0 = S'' xN = 0 - The resulting function is termed a natural cubic spline. Scientific computing III 2011: 6. Interpolation 74 Interpolation: piecewise interpolation - Example: Determine parameters a , b , c , d , e , f , g , and h so that Sx is a natural cubic spline, where ax3 +bx2 ++cxd x>@–10 Sx =® ¯ex3 +fx2 ++gxh x>@01 and with interpolation conditions: S –1=1 , S 0=2 , S 1=–1 . - Let the two polynomials be S 0 x and S1 x . - From interpolation conditions we get S0 0=d=2 S1 0=h=2 S0 –1 =–a+bc– =–1 S1 1=efg++3=– - The 1st derivative of Sx is 3ax2 +2bxc+ S' x=® ¯3ex2 +2fxg+ - Continuity of S' x implies S'0 =cg= - The 2nd derivative is 6ax + 2b S'' x = ® ¯ 6ex + 2f Scientific computing III 2011: 6. Interpolation 75 Interpolation: piecewise interpolation - From the continuity of S '' x we get S''0 = bf= - And the two extra conditions S''1 – =–6a+2b=0 ® ¯S''1 =–6e+2f=0 - From all these equations we get a=–1, b = –3 , c = –1 , d = 2, e = 1, f = –3 , g = –1 , h = 2 Scientific computing III 2011: 6. Interpolation 76 Interpolation: piecewise interpolation - In what follows we will develop a general algorithm to determine the natural cubic spline. - Based on the continuity of S '' x we can unambiguously define the following numbers zi { S'' xi , 0 ddiN - The two extra conditions say that z 0 = zN = 0 but the other values are still unknown. - We know that on each subinterval >@ xi xi + 1 the second derivative S '' x is linear, and takes the values z i and z i + 1 at the end points: zi+1 zi Si'' x =------xx–i+---- x i + 1 – x , hi hi where we have defined hi=xi+1–xi , 0ddiN–1 - Integrating this twice we get zi+1 2zi 2 Si' x=------xx–i–------x i + 1 – x +c 2hi 2hi zi+1 3zi 3 Si x=------xx–i +------x i + 1 – x ++cxd 6hi 6hi where c and d are integration constants. Scientific computing III 2011: 6. Interpolation 77 Interpolation: piecewise interpolation - By adjusting the constants we get a more convenient form of the spline z z i+1 3i 3 Si x =------xx–i +------x i + 1 – x ++Ci xx–i Di xi+1–x 6hi 6hi - Constants C i and D i can be determined by applying the interpolation conditions: Si xi=yi , Si xi+1=yi+1 z z yh z yh z i+1 3i 3§·i+1i i + 1 §·ii i o Si x=------xx–i +------x i + 1 – x ++¨¸------– ------xx–i ¨¸---- – ------xi+1–x 6hi 6hi ©¹hi6 ©¹hi6 - We still have to find out the values of zi . - This is done by applying the remaining condition – namely the continuity of S' x : - At the interior knots xi , 1 ddiN–1 , we must have S'i–1 xi=S'i xi - From above we get the derivative: z z yh z y h z i+1 2i 2i+1i i + 1 i i i S'i x= ------xx–i –------x i + 1 – x + ------– ------– ---- + ------2hi 2hi hi6 h i 6 Scientific computing III 2011: 6. Interpolation 78 Interpolation: piecewise interpolation - And hi h i y i + 1 – y i S'i xi = – ---- z i + 1 – ---- z i + b i , b i = ------6 3 h i hi–1 h i – 1 y i – y i – 1 S ' i – 1 x i = – ------z i – 1 – ------z i + b i – 1 , b i – 1 = ------6 3 h i – 1 - When we set these equal we get – after a little rearrangement h i – 1 z i – 1 + 2 h i – 1 – h i z i +6h i z i + 1 = b i – b i – 1 , 1 ddiN– 1 - By letting u i = 2 h i – 1 + h i ® ¯v i =6 b i – b i – 1 we get a tridiagonal system of equations for zi : z0=0 ° ®hi–1zi–1+uizi+hizi+1=vi1ddiN–1 ° ¯zN=0 Scientific computing III 2011: 6. Interpolation 79 Interpolation: piecewise interpolation - In matrix form this is 10 z0 0 h0u1h1 z1 v1 huh z v 122 2 = 2 } } } hN–1uN–1hN–1zN–1 vN–1 0 1 zN 0 - We can eliminate the first and the last equations and get finally the form: u1h1 z1 v1 h1u2h2 z2 v2 }} }=} hN–3uN–2hN–2zN–2 vN–2 hN–2uN–1zN–1 vN–1 which is a symmetric tridiagonal system of order N–1 . Scientific computing III 2011: 6. Interpolation 80 Interpolation: piecewise interpolation - This system can be readily solved by 2 h ° u m u – ------i – 1 ° i i u the forward elimination: ® i – 1 i =23}N–1 ° h v ° i – 1 i – 1 v i m v i – ------¯ u i – 1 vN–1 ° zN–1m ------° uN–1 and the back substitution: ® iN= –2N–3 }1 ° v i – h i z i + 1 ° z i m ------¯ u i Scientific computing III 2011: 6. Interpolation 81 Interpolation: piecewise interpolation - Finally we can write down the algorithm for the natural cubic spline for data set ^`xi yi , i = 01} N : 1. For i = 01 } N – 1 compute hi = xi + 1 – xi ° ® yi + 1 – yi ° bi = ------¯ hi 2. Set u 1 = 2 h 1 + h 0 ® ¯v 1 =6 b 1 – b 0 3. For i=23 }N–1compute 2 hi–1 ° ui= 2 hi+hi–1– ------° ui–1 ® ° hi–1vi–1 ° v i = 6 b i – b i – 1 – ------¯ ui–1 4. Set z N = 0 ® ¯ z 0 = 0 5. For iN= – 1 N – 2 } 1 compute v i – h i z i + 1 z i = ------. u i Scientific computing III 2011: 6. Interpolation 82 Interpolation: piecewise interpolation - This algorithm might fail if u i = 0 for some i . - It is clear that u1 !!h1 0 . - Now if ui – 1 ! hi – 1 then ui ! hi because 2 hi–1 ui =2 hi + hi – 1 – ------!!2 h i + h i – 1 –hi–1hi ui–1 - So, by induction: ui!0 , i=12}N–1 . - Now we have the coefficients z i of the spline z z yh z yh z i+1 3i 3§·i+1i i + 1 §·ii i Si x=------xx–i +------x i + 1 – x ++¨¸------– ------xx–i ¨¸---- – ------xi+1–x 6hi 6hi ©¹hi6 ©¹hi6 - However, this is not the best possible form to evaluate the spline. - In order to evaluate it using the nested multiplication we have to massage it into the form 2 3 Si x=Ai+Bi xx–i++Ci xx–i Di xx–i Scientific computing III 2011: 6. Interpolation 83 Interpolation: piecewise interpolation - Notice that the equation above is in the form of Taylor expansion of S i around point xi , Hence, 1 1 A = S x , B = S ' x , C = - -- S '' x , D = - -- S ''' x i i i i i i i 2 i i i 6i i - Therefore z A=y and C= ---i . i i i 2 3 - The coefficient of x in the form above is D i and in the previous form z i + 1 – z i e 6 h i z i + 1 – z i o D i = ------6 h i - Finally the value S i ' x i is give as h i z i + 1 h i z i y i + 1 – y i B i = – ------– ------+ ------6 3 h i - The nested form of S i x is then finally – after a few steps of algebra: ½zi zi+1–zi S i x =yi+ xx–i®¾Bi+ xx–i--- + xx–i------¯¿2 6hi Scientific computing III 2011: 6. Interpolation 84 Interpolation: piecewise interpolation - Then to the implementation. - The following routine solves the coefficients z i given the input parameters xi , yi , N . void Spline3_Coeff(int n,double *x,double *y,double *z) { int i; double *h, *b, *u, *v; h = (double *) malloc((size_t) (n*sizeof(double))); b = (double *) malloc((size_t) (n*sizeof(double))); u = (double *) malloc((size_t) (n*sizeof(double))); v = (double *) malloc((size_t) (n*sizeof(double))); for (i=0; i Scientific computing III 2011: 6. Interpolation 85 Interpolation: piecewise interpolation - And the evaluation routine computes the spline value at x0 given as an input argument. double Spline3_Eval(int n, double *x, double *y, double *z, double x0) { int i; double h, tmp, result; for (i=n-1; i>=0; i--) { if (x0-x[i]>=0) break; } h = x[i+1]-x[i]; tmp = 0.5*z[i] + (x0-x[i])*(z[i+1]-z[i])/(6.0*h); tmp = -(h/6.0)*(z[i+1]+2.0*z[i])+(y[i+1]-y[i])/h + (x0-x[i])*tmp; result = y[i] + (x0-x[i])*tmp; return(result); } Scientific computing III 2011: 6. Interpolation 86 Interpolation: piecewise interpolation - Example: Function sin x at the interval >@0 1.6875 , with ten equidistant knots: Comparison with Newton polynomial Scientific computing III 2011: 6. Interpolation 87 Interpolation: piecewise interpolation - Another example: The serpentine curve x y = ------14e +x2 - And in parametric form 1 °x = - -- tanT ® 2 ° ¯y = sin2T - Let’s take knots evenly spaced in T so that they are non-uniformly spaced in x axis: S T = n------, n = – 5 } 5 12 - We can interpolate the parametric form by constructing splines for both x T and y T - Note that the knots must be in ascending order. Scientific computing III 2011: 6. Interpolation 88 Interpolation: piecewise interpolation - Below are the results comparing the interpolation with a spline and with a polynomial. Scientific computing III 2011: 6. Interpolation 89 Interpolation: Bézier curves • One way of writing the interpolating polynomial is by using basis functions bi x : N Px = cb x ¦ ii i=0 - Note that if we have one polynomial going through all data points it is unique and and does not depend on the basis func- tions used. - As examples of basis functions we have used are monomials i–1 bi x =x , i=0}N and Lagrange’s polynomials N §·xx–j bi()x = ¨¸------, i=0}N ©¹x–x j=0i j jiz - Note that here the monomials do not depend on the data set but Lagrange’s polynomials depend on the positions of the nodes ^`xi. Scientific computing III 2011: 6. Interpolation 90 Interpolation: Bézier curves - Graphically Monomials Lagrange Scientific computing III 2011: 6. Interpolation 91 Interpolation: Bézier curves • The Weierstrass approximation theorem: For any continuous function f on a closed interval, say >@01 , and any H ! 0 , there is a polynomial P such that for any x >@ 01 fx – Px H n - It was proven by Bernstein using polynomials Bi x : n i f§·- -- Bn xofx , when nof , x>@01 ¦©¹n i i=0 where the Bernstein polynomial (BP) is n n n! Bn x =§· 1–xni–xi, §·= ------i ©¹i ©¹i i! ni–! - These can be generalized to any interval >@ab : ni– i n §·n bx– xa– B i x = ------©¹i ba– n i - It is easy to show that the polynomial B n x has a maximum at the position xa=+- -- ba– i n Scientific computing III 2011: 6. Interpolation 92 Interpolation: Bézier curves - Below are examples of various BP’s at the interval >@01 5 Bi 4 Bi 3 3 3 2 3 2 B0 x = 1–x B1 x=3x 1–x B2 x=3x 1–x 3 B 3 3 i B3 x=x 2 B2 x = 1–x2 B2 x=2x 1–x B2 x=x2 Bi 0 1 2 1 1 1 B0 x =1–x B1 x =x Bi Scientific computing III 2011: 6. Interpolation 93 Interpolation: Bézier curves - BP’s are often used as a basis for interpolation of function fx : N P x = pBN x N ¦ i i i=0 where coefficients p i are obtained from interpolation conditions: PN xi =fx i , i=01}N - Why are BP’s useful? Coefficients p i give a hint of how the interpolate P N looks like. - Let’s associate with each p i a control point t ipi such that i t =a+- -- ba– i n - Note that t 0p 0= x0y0 and t NpN= xNyN but the control points between these limits do not necessarily correspond to the data set ^`xiyi, i=1}N–1 . Scientific computing III 2011: 6. Interpolation 94 Interpolation: Bézier curves - In Fortran90 the interpolation is done as follows: program bernstein mx=1.0/dx implicit none integer,parameter :: & do m=0,mx rk=selected_real_kind(12,100) x=dx*m y=0.0 real(rk),allocatable :: b(:),p(:) call bernstein_poly(n,x,b) real(rk) :: x,y,dx do i=0,n integer :: iargc,n,mx,i,m,k write(i+10,’(2g18.10)’) x,b(i) character(len=80) :: argu,file1,file2 y=y+p(i)*b(i) end do if (iargc()<5) then write(200,’(2g18.10)’) x,y write(0,’(a,a)’) ’usage: bernstein ‘,& end do ‘cpointfile interpfile dx n p0 p2 p3 .. pn’ stop close(100) end if close(200) call getarg(1,file1) stop call getarg(2,file2) open(100,file=file1,status=’new’) end program bernstein open(200,file=file2,status=’new’) call getarg(3,argu); read(argu,*) dx call getarg(4,argu); read(argu,*) n allocate(b(0:n),p(0:n)) do i=0,n call getarg(i+5,argu); read(argu,*) p(i) write(100,’(2g18.10)’) & real(i,rk)/real(n,rk),p(i) end do Scientific computing III 2011: 6. Interpolation 95 Interpolation: Bézier curves - Program uses the polpak library for calculating Bernstein polynomials. - The library is available in Fortran90, C and Matlab and can be downloaded from http://www.csit.fsu.edu/~burkardt/f_src/polpak/polpak.html - Compiling and linking a program using the library is easy (after you have created the library itself and assuming the library is in the current directory): gfortran -o bezier bezier.f90 -L. -lpolpak Scientific computing III 2011: 6. Interpolation 96 Interpolation: Bézier curves - Now plot the control points ^`ti pi , i = 0} N and the interpolating polynomial PN x - For example: - This means that by specifying the control points we can easily construct a smooth curve that resembles a line going through those points. - The more we have points the nearer the curve comes to the control points. - Note that here we are no more talking about interpo- lation. - The polynomial does not go through the control points. Scientific computing III 2011: 6. Interpolation 97 Interpolation: Bézier curves - What if we have an arbitrary curve on the xy plane (i.e. not a function y= fx )? y - We can express the curve in the parametric form x= xt ® , t >@ 01 (for example) x ¯y= yt - If the set of control points is ^` xiyi the interpolated curve is N x °P t = xBN t x ° N i i xP= t ¦ ° N ° i=0 ® where ® y N °yP= t ° y ¯ N °P t = yBN t ° N ¦ i i ¯ i=0 - This curve is called the Bézier curve - Note that ^` x iyi are now the coefficients of the ‘interpolating’ polynomials. Scientific computing III 2011: 6. Interpolation 98 Interpolation: Bézier curves - The previous program only needs a little modification to compute the Bézier curve: program bezier mt=1.0/dt implicit none integer,parameter ::& do m=0,mt & rk=selected_real_kind(12,100) t=dt*m x=0.0 real(rk),allocatable :: b(:),xa(:),ya(:) y=0.0 real(rk) :: x,y,t,dt call bernstein_poly(n,t,b) integer :: iargc,n,mt,i,m,k do i=0,n character(len=80) :: argu,file1,file2 y=y+ya(i)*b(i) end do if (iargc()/=2) then call bernstein_poly(n,t,b) write(0,’(a)’) ’usage: bezier n dt’ do i=0,n stop x=x+xa(i)*b(i) end if end do write(6,’(2g18.10)’) x,y call getarg(1,argu); read(argu,*) n end do call getarg(2,argu); read(argu,*) dt n=n-1 stop allocate(b(0:n),xa(0:n),ya(0:n)) do i=0,n end program bezier read(5,*) xa(i),ya(i) end do Scientific computing III 2011: 6. Interpolation 99 Interpolation: Bézier curves - Two examples: Scientific computing III 2011: 6. Interpolation 100 Interpolation: B-splines • The basis functions b i x of the interpolate (note the slight change in notation: Nko ) k Px = ¦cibi x i=0 may also be splines itself: they are so called B-splines (B for basis). k - A B-spline of order k is denoted as N i x and it depends on the set of nodes xi . 3 For example in the case of cubic B-splines the functions N i x have two continuous derivatives and span four points around xi . - B-splines can be computed recursively 1 xdxx 0 i i+1 Ni x = ® ¯0 otherwise xx– x –x k §·i k–1§·ik++1 k–1 Ni x =¨¸------Ni x+¨¸------Ni+1 x, where k =12} and i=01r r}2 ©¹xik+–xi ©¹xik++1–xi+1 - B-splines can be used in a similar fashion as Bézier curves. - With B-splines one can model continuous curves with points where derivatives are discontinuous. - For more information see Kahaner, Moler, Nash: Numerical Methods and Software, or Haataja et al., Numeeriset menetelmät käytännössä. Scientific computing III 2011: 6. Interpolation 101 Interpolation: 2D • Let the function depend on two variables: y= fx 1 x2 i j - The interpolation data is now a set of points in a rectangular grid ^`yijx1 x2 , i=1}m , j=1}n - Let’s use the following notation: ya(j,k) = y()x1a(j) x2a(k) - First we must find the right position in the grid where 4 3 (j,k+1) (j+1,k+1) x1a(j) ddx1 x1a(j+1) x x x2a(k) ddx2 x2a(k+1) 1 2 - Also let the function values at the corners be 1 2 (j,k) (j+1,k) y1 { ya(j,k) y2 { ya(j+1,k) y3 { ya(j+1.k+1) y4 { ya(j,k+1) Scientific computing III 2011: 6. Interpolation 102 Interpolation: 2D - The desired y value is obtained by using bilinear inter- polation: x – x1a(j) t = ------1 x1a(j+1)– x1a(j) x – x2a(k) u = ------2 x2a(k+1)– x2a(k) yx(1 x 2 )1= – t 1 – u y 1 + t 1 – u y 2 ++tuy3 1 – t uy4 - For higher accuracy/smoothess one can generalize the spline interpolation: 1 1. Construct m 1D interpolates 2 in x direction. } 2. Based on these construct one 1D interpolate at the desired y value. m – 1 - For more details see e.g. Numerical Recipes paragraph m 3.6 : desired interpolation point : spline in x direction : spline in y direction Scientific computing III 2011: 6. Interpolation 103 Interpolation: practical tools • If you are using C then GSL has many interpolation routines: Spline interpolation #include int main (int argc, char **argv) { int i,nk,np; double *x,*y,x1,x2,dx,xi,yi; gsl_interp_accel *acc; //Index look-up acceleration gsl_spline *spline; if (argc!=4) { BEGIN { fprintf(stderr,"Usage: %s x1 x2 dx\n",argv[0]); return (1); srand(); } x1=0; x1=atof(*++argv);a x2=100.0; x2=atof(*++argv); dx=1.0; dx=atof(*++argv); n=(x2-x1)/dx+1; fscanf(stdin,"%d",&nk); print n; x = (double *) malloc((size_t) (nk*sizeof(double))); for (i=0;i np=(x2-x1)/dx; for (i=0;i Scientific computing III 2011: 6. Interpolation 104 Interpolation: practical tools • For Fortran users the is e.g. the SLATEC package: http://www.netlib.org/slatec/ - At http://www.csit.fsu.edu/~burkardt/f_src/slatec/slatec.html you can find some documentation and test programs. - Installed on punk.it.helsinki.fi at /usr/local/lib (linking: gfortran prog.f90 -lslatec) - If you want to build the library yourself 1) Download the package: http://www.physics.helsinki.fi/courses/s/tl3/progs/slatecpackage.tgz 2) In files f90split.csh and slatec.csh change the appropriate string for the Fortran compiler: set FORT=gfortran (or f90, ...) 3) Split the one large file to smaller pieces and build the library: ./f90split.csh; ./slatec.csh • Matlab has the function interp1 that does 1D interpolation using the method of choice (nearest, linear, spline, ...) • Function spline computes the spline coefficients and they can be used to evaluate using function ppval: pp=spline(x,y) yi=ppval(pp,xi) Scientific computing III 2011: 6. Interpolation 105 Interpolation: practical tools - A simple example using SLATEC: program spline_slatec ixmax=(x2-x1)/dx implicit none ideriv=0 integer,parameter :: rk=selected_real_kind(15,100) inbv=1 real(rk),allocatable :: xa(:),ya(:),w(:),t(:),bcoef(:) do ix=1,ixmax real(rk) :: x,y,x1,x2,dx,fbcl,fbcr x=x1+dx*ix integer :: ibcl,ibcr,kntopt,ncoef,k,ideriv,inbv y=dbvalu(t,bcoef,n,k,ideriv,x,inbv,w) integer :: n,i,j,ix,ixmax write(6,*) x,y character(len=80) :: argu end do real(rk),external :: dbvalu stop if (iargc()/=3) then end program spline_slatec call getarg(0,argu) write(0,’(a,a,a)’) ’usage: ’,trim(argu),’ x1 x2 dx’ spline> ifort spline_slatec.f90 -lslatec -L../../slatec/ stop spline> a.out 1.0 8.0 0.01 < sin_per_1+x2.dat > end if sin_per_1+x2.interp call getarg(1,argu); read(argu,*) x1 call getarg(2,argu); read(argu,*) x2 call getarg(3,argu); read(argu,*) dx read(5,*) n allocate(xa(1:n),ya(1:n),w(1:5*(n+2)),t(1:n+4),bcoef(1:n+2)) do i=1,n read(5,*) xa(i),ya(i) end do ibcl=2 ibcr=2 fbcl=0.0 fbcr=0.0 kntopt=2 k=4 w=0.0 call dbint4(xa,ya,n,ibcl,ibcr,fbcl,fbcr,& & kntopt,t,bcoef,ncoef,k,w) Scientific computing III 2011: 6. Interpolation 106