Polynomial Interpolation and Approximation

Errors using inadequate data are much less than those using no data at all.

Charles Babbage

1 Fall 2010 Topics to Be Discussed zThis unit only requires the knowledge of simple polynomial algebra and algebraic manipulations. zThe following topics will be presented: ¾A naïve interpolation method ¾The Lagrange form ¾The Newton form ¾Divided Difference ¾Least square polynomial approximation

2 Polynomial Interpolation: 1/5 zIn many applications, we know a function but don’t know its exact form. For example:

π /2 1 K()kdx= ∫0 22 1sin− ()kxsin () zThe above shows a function of k. But, what is this function K(k)? How do we plot K(k) ? Do we have to plug a value of k into the formula and integrate with respect to x to find K(k)?

3 Polynomial Interpolation: 2/5 zIf K(k) is simple, integration may be a viable approach; however, most integrations are difficult. This is a problem. zOne way to overcome this problem is to evaluate K(k) at some values of k and fit them with a simpler function (e.g., polynomial). zOne may use k0, k1, …, kn tltto evaluate K(k0), K(k1), …, K(kn), and fit (k0, K(k0)), (k1, K(k1)), …, (kn, K(kn)) with a polynomial y=P(x). zThen, when K(t) is needed, one uses P(t) instead!

4 Polynomial Interpolation: 3/5 zSince ppyolynomial y=P(x) pppasses all data points (ki,K(ki)) for i = 0, 2, …, n, P(x) is said to interpolate function K(k) at k0, k1, …, kn. zSince a degree n polynomial, P(x) = a0 + a1x + 2 n a2x + … + anx , has n+1 coefficients, n+1 distinct data points (ki,K(ki)) are needed to have a unique solution to a0, a1, a2, …, an. zPolynomials may have other forms, each of which requires a different computation. zWe will discuss the Lagrange and Newton forms. zInterpolating polynomials are unique. They j ust h ave diff eren t forms. 5 Polynomial Interpolation: 4/5 z Polyypnomial Interpolation:

‰Given a set of data points (x0,y0), (x1,y1), …, (xn,yn), where xi’s are all distinct, find a polynomial y = P(x) that interpolates the data points (i.e., yi = P(xi) for i = 0, 1, …, n). ‰The degree of the interpolating polynomial P(x) is n. ‰Then, we may use P(x) as if it is the unknown function that generates data points (x0,y0), (x1,y1), …, (xn,yn). ‰All transcendental functions (e.g., sin(), exp(), log())i) are computed this way. 6 Polynomial Interpolation: 5/5 zA number of fundamental issues have to be addressed properly: ‰ How easy can we determine the interpolating polynomial from the input data points? ‰ HtHow easy can we compute P(x) for a new x after the polynomial is determined? ‰ How easy can we add new data points after P(x) is determined? ‰ How efficient can we solve these three issues?

7 A Naïve Method: 1/3 zLet us tryygg a naïve method before getting into good ones. zSuppose the inter polatin g pol ynomial for n+1 2 n data points is P(x) = a0 + a1x + a2x + … + anx . In other words, P(x) is in the power form. zLet the data points be (x0,y0), (x1,y1), …, (xn,yn). zFor each xi,,g we have the following:

⎡a0 ⎤ ⎢a ⎥ ⎢⎥1 22…nn⎡⎤⎢ ⎥ yPxaaxaxii==++++=()01 ii 2 ax niiii⎣⎦ 1,, xx , , x ⋅ a 2 ⎢ ⎥ ⎢ ⎥ ⎢⎥ ⎣an ⎦ 8 A Naïve Method: 2/3

2 n zCollecting all [1,xi,xi ,…,xi ]’sintoas into a (n+1)×(n+1) matrix yields: 2 n ⎡⎤ya00⎡⎤1 xx00 x 0 ⎡⎤ ⎢ ⎥⎢⎥⎢ 2 n ⎥ ya1 xx x ⎢ 11⎥⎢⎥⎢⎥11 1 2 n ⎢⎥y22=⋅⎢1 xx x⎥ ⎢⎥a ⎢⎥⎢⎥22 2 ⎢⎥ ⎢ ⎥⎢⎥⎢⎥ ⎢ ⎥⎢⎥⎢⎥2 n ⎣⎦ynn⎣⎦1 xxnn x n ⎣⎦a zSince the column matrix [yj](n+1)×1 and the matrix [xi,j](n+1)×(n+1) are known ( i.e., input data), Gaussian elimination may be used to solve for

the a0, a1, a2,,…, …, an.

9 A Naïve Method: 3/3 zIssue #1: Determininggp pol ynomial P(x) is not difficult with Gaussian elimination; however, it is an O(n3) method (i.e., slow)! zIssue #2: EltiEvaluating P(x) with a gi ven x is efficient if one uses the nested form which requires n multiplications ( i.e., O(n)):

aaa012++++( ( ( annn−− 2 + ( a 1 + axxxxx ))) )) zIssue #3: Adding a new data point requires solving a new system of linear equations. zMoreover, pivoting may be needed.

10 Lagrange Polynomials: 1/14 zLagggrange used a s pecial form of P(x). zEach term of P(x) is a degree n polynomial, and the i-th term does not include x-xi.

Px10110()=−+− a ( x x ) a ( x x )

P2012102201()()()()()()()x=− axxxx −+− axx xx −+− axxxx −

Pxnn()=− a01 ( x x )( x − x 2 ) ( x −+ x )

ax10()()()−− x x x 2 x −+ xn ax()()()()−−− x x x x x x −+ x 20 1 3 n Pk(x) means degree k axnn()()()−− x01 x x x − x− 1 n P ()xa⎡⎤n ( xx ) General form nijjij= ∑ ⎣⎦∏ =≠0, − i=0 11 Lagrange Polynomials: 2/14 zWhyygg is the Lagrange form useful? zIt is easy to compute coefficients a0, a1, …, an. zThe i-th term of Pn(x) does not have x-xi:

axiiin()()()()()−− x01 x x x − x−+ 1 x − x 1 x − x zTherefore, plugging one of x=x0, x=x1, …, x=xi-1, x=xi+1, …, x=xn into this term yields a zero. zHowever, plugging x=xi into this term yields:

axii()()()()()−− x01 x i x x i − x i−+ 1 x i − x i 1 x i − x n zHence, xi makthkes the i-th term non-zero andlld all other terms zero.

12 Lagrange Polynomials: 3/14 zRecall that the Lagggrange form is n P ()xa⎡ n ( xx )⎤ nijjij=∏∑ ⎣ =≠0, −⎦ i=0 zIf we plug xi into Pn(xi) , all but the i-th term become zero, and the i-th term is

axii()()()()()−− x01 x i x x i − x i−+ 1 x i − x i 1 x i − x n zTherefore, we have yPxaxxxxi==−− niii() (01 )( i ) ( xx ii −−+ 1 )( xx ii − 1 ) ( xx in − ) and ai is:

yi ai = ()()()()()xxxxi−−01 i xx ii −−+ 1 xx ii − 1 xx i −n 13 Lagrange Polynomials: 4/14

zPlugging the comp uted ai’s back to Pn(x) yields the Lagrange interpolating polynomial.

n ()()()()()xx−−01 xx xx −ii−+ 1 xx − 1 xx − n Pxni()= ∑ y i=0 ()()()()()xxxxi−−01 i xx ii −−+ 1 xx ii − 1 xx in −

zGiven (xi,yi), i=0,1,…,n, the following computes all coefficients ai’s in array a(0:n):

DO i = 0, n ! Compute ai a(i) = 1.0

DO j = 0, n ! For each xj, j != i IF (i /= j) a(i) = a(i)*(x(i) – x(j)) END DO a(i) = y(i)/a(i) END DO 14 Lagrange Polynomials: 5/14 zHow many multiplications are there? ‰For each i, n multiplications are needed in the inner loop because it iterates n+1 times and no mu ltip licati on i s need ed if i = j. ‰Since the outer DO loops for n+1 times, the total number of multiplications is n(n+1). ‰This is high; but, it is easy to do!

DO i = 0, n ! Compute ai a(i) = 1.0

DO j = 0, n ! For each xj, j != i IF (i /= j) a(i) = a(i)*(x(i) – x(j)) END DO a(i) = y(i)/a(i) END DO 15 Lagrange Polynomials: 6/14 zConsider three data points (-1,1), (0,2) and (1,5).

Thus, x0=-1, y0=1, x1=0, y1=2 and x2=1, y2=5. zHere are the ai’s and the interpolating polynomial of degree 2:

y0 11 a0 == = (xxxx0102−− )( ) (( −−×−− 1) 0) (( 1) 1) 2

y1 2 a1 == =−2 (xxxx1012−− )( ) (0 −−×− ( 1)) (0 1)

y2 55 a2 === (xxxx2021−− )( ) (1 −−×− ( 1)) (1 0) 2 15 P2 ()xxxxx=−−+−++ (1)2(1)(1)(1) xx 2216 Lagrange Polynomials: 7/14

zHow to find Pn(x) for an arbitrary x? zThe polynomial Pn(x) is: n Pxni()= ∑ ax (−− x01 )( x x ) ( x − x iin−+ 1 )( x − x 1 ) ( x − x ) i=0 zThe number of multiplications is still n(n+1)!

! x is the input With some programming Px = 0.0 trick, one may reduce the DO i = 0, n number of multiplications s = a(i) to O(n)! How? DO j = 0, n (Hint: use division) IF (i /= j) s = s * (x – x(j)) END DO However, this approach Px = Px + s is risky. Why??? END DO 17 Lagrange Polynomials: 8/14

zWe compppgpyuted the interpolating polynomial P2(x) for (-1,1), (0,2) and (1,5) as follows: 15 Px()=−− xx (1)2(1)(1)(1) x++++ x− ++ x x 2 22

zThen, we may use P2(x) to find other values:

15 P (3)=−− 3(3 1) 2(3+++++ 1)(3− 1)++ (3 1)3=− 3 16+ 30= 17 2 22 15 P (−= 2) ( − 2)(( −−− 2) 1) 2(( −+ 2) 1)(( −−+ 2) 1) (( −+−=−+= 2) 1)( 2) 3 6 5 2 2 22 15 P (0.5)=−−+−++=−++= 0.5(0.5 1) 2(0.5 1)(0.5 1) (0.5 1)0.5 0.125 1.5 1.875 3.25 2 22

18 Lagrange Polynomials: 9/14 zUse a polynomial to evaluate x y the sin() function. i i zIt is sufficient to restrict the 0.0 0.0 0.3 0.29552021 range to about [0,1.5]. A 0.6 0.5646425 table of (xi,yi)’s is generated. 0.9 0.7833269 zThis is a degree 5 polynomial 1.2 0.9320391

with coeffi ci ent s ai’s. 151.5 0.997495 zPlot both functions to see how close they are!

a0 a1 a2 a3 a4 a5 0.0 5.067218 -19.363594 26.863062 -15.981465 3.4207656 19 Lagrange Polynomials: 10/14 zWith the coefficients below, we may compute or “approximate” sin() with a polynomial.

a0 a1 a2 a3 a4 a5 0.0 5.067218 -19.363594 26.863062 -15.981465 3.4207656

xP5(x) sin(x) error 0.2 0.19866316 0.19866933 6.1690807E-6 0.4 0.38942143 0.38941833 -3.0994415E-6 0.6 0.5646424 0.5646425 5.9604644E-8 1.0 0.84147375 0.84147095 -2.8014183E-6 1.4 0.98543715 0.98544973 1.257658E-5

20 Lagrange Polynomials: 11/14 zNow use 0, π/8, π/6, π/4, π/3 x y and π/2. i i zIt is easy to calculate the 0.0 0.0 π/8 0.38268345 sin() at these six values. π/6 0.5 zWhWe have a degree 5 π/4 0.70710676 interpolating polynomial π/3 0.8660254 with coeffi ci ent s ai’s as π/2 101.0 shown below.

a0 a1 a2 a3 a4 a5 0.0 24.586205 -50.82025 42.590042 -17.604652 1.2548212

21 Lagrange Polynomials: 12/14 zWith the coefficients below, we may compute or “approximate” sin() with a polynomial.

a0 a1 a2 a3 a4 a5 0.0 24.586205 -50.82025 42.590042 -17.604652 1.2548212

xP5(x) sin(x) error 0.2 0.19866273 0.19866933 6.6012144E-6 0.4 0.3894185 0.38941833 -1.7881393E-7 0.6 0.5646418 0.5646425 7.1525573E-7 1.0 0.8414726 0.8414711 -1.5497208E-6 1.4 0.9854038 0.9854498 4.6014785E-5

22 Lagrange Polynomials: 13/14 zWhat if we have a new data point (xn+1,yn+1)? zWe need a new term an+1 that does not have x-xn+1:

yn+1 an+1 = ()()()xxxxxxnn++10−− 11 nn + 1 − zWe also need to update each old term a : i new yi ai = ()()()()()()xxxxi−−01 i xx ii −−+ 1 xx ii − 1 xxxx inin −− + 1 zWe need n+1 multiplications to compute an+1, and one multipli cati on ( or di vi si on) f or each ai (0≤i≤n). The total is (n+1)+(n+1) = 2(n+1) !

the original ai 23 Lagrange Polynomials: 14/14

zWe obtained the interppgpyolating polynomial P2(x) for (-1,1), (0,2) and (1,5) as follows: 15 Px()=−− xx (1)2(1)(1)(1) x++++ x− ++ x x 2 22 zIf a new data point (2,10) is introduced. Then, 10 5 a == 3 (2−−×−×− ( 1)) (2 0) (2 1) 3

a2 5/2 5 a2 ===− xx23−−12 2

a1 −2 a1 ===1 xx13−−02

a0 1/2 1 a0 == =− xx03−−−(1)2 6 −155 P( x )=+++++= xx (−− 1)( x 2)++ ( x 1)( x−−− 1)( x 2) ( x+ 1) xx (− 2)++ ( x 1) xx (− 2) 3 623 24 new Newton Divided Difference: 1/19 zNewton chose to use a different form as follows:

Px1010()=+ a a ( x − x )

Px2010201()=+ a ax ( − x ) + a ( x − x )( x − x )

Px30102013012()()()()()()()=+ a axx − + axx − xx − + axxxxxx − − −

Pnnn()()()()xaaxxaxxxx=+−+−−++−−010201 axxxx ()()() 01 xx −− 1 zHere are some observations: zPk(x) has x-x0, x-x1, …, x-xk-1, and does not depend on x-xk+1, x-xk+2, …, x-xn. zHence, Pk(x) only depends on x0, x1, …, xk. zPk(x) is the sum of Pk-1(x) and ak(x-x0)…(x-xk-1).

25 Newton Divided Difference: 2/19

zFor all k,since, since Pk(x0) = a0, we have a0 = y0.

zFrom P1(x), we have y1 = P1(x1) = a0+a1(x1-x0) and a1 =(= (y1-a0)/(x1-x0)=() = (y1-y0)/(x1-x0).

zFrom P2(x2) and the computed a0 and a1, a2 is yaaxx20121−[ + ()− ] a2 = ()()xxxx2021−−

Px1010()=+ a a ( x − x )

Px2010201()=+ a ax ( − x ) + a ( x − x )( x − x )

P30102013012()()()()()()()x=+ a axx − + axxxx − − + axx − xxxx − −

Pnnn()()()()xaaxxaxxxx=+−+−−++−−010201 axxxx ()()() 01 xx −− 1 26 Newton Divided Difference: 3/19

zFrom P3(x) , we have P3(x3) = y3 and a3 is

yaaxxaxxxx3013023031−+[ ()()() −+ − −] a3 = ()()()xxxxxx303132−−−

zSimilarly, we can compute all ai’s (0 ≤ i ≤ n).

Px1010()=+ a a ( x− x )

Px2010201()=+ a ax ( − x ) + a ( x − x )( x − x )

Px30102013012()()()()()()()= a+++ axx− + axx−− xx + axxxxxx−−−

Pnnn()()()()xaaxxaxxxx=+−+−−++−−010201 axxxx ()()() 01 xx −− 1

27 Newton Divided Difference: 4/19

zSuppose we have a0, a1,…,, …, ai-1 and xi, and wish to compute ai in Pi(x).

zFrom Pi(x) , we have

Piii()()()()xaaxxaxxxx=+−+−−++−−010201 axxxx ()()() 01 xx −− 1

zPlug the ak’s (0 ≤ k ≤ i-1) and xi into Pi(x) and solve for ai as shown below:

yii−+[ aaxx01()()() −+ 0 axxxx 2 iiiiii − 0 −++ 1 axx−− 1 ()() − 0 xx − 2] ai = ()()()xxxxii−−01 xx ii −− 1

28 Newton Divided Difference: 5/19

zThe following subroutine takes the ak’s (0 ≤ k ≤ i-1) and (xk,yk)’s (0 ≤ k ≤ i) and computes ai.

yii−+[aaxxaxxxx01()()() −+ 0 2 iiiiii − 0 −++ 1 axx−− 1 ()() − 0 xx − 2] ai = ()()()xxxxii−−01 xx ii −− 1

SUBROUTINE Term-i(…)

sum = a0 ! initialize sum prod = 1.0 ! holds (xi-x0)..(xi-xi-1) DO j = 1, i-1 ! add the terms

prod = prod * (xi –xj-1) ! update the product sum = sum + aj*prod ! add the term END DO

ai = (yi – sum)/(prod*(xi –xi-1)) ! Final touch END SUBROUTINE

2(i-1)+1 = 2i-1 multiplications 1 division are needed 29 Newton Divided Difference: 6/19 zWith subroutine Term-i(…), the following computes a0, a1, a2, …, an one at a time. zSince Term-i(…) uses 2i-1 multiplications for a given i, thttlthe total num ber o f mu ltilitiltiplications i s

nnn ⎛⎞ (1)nn+ 2 ∑∑∑(()21ii−=) 212⎜⎟ − = −= nn iii===111⎝⎠ 2 zNot bad, slightly better than Lagrange method, which uses n(n+1) multiplications

a0 = y0 DO i1i = 1, n

CALL Term-i(……) ! compute ai from a0,…ai-1 END DO

30 Newton Divided Difference: 7/19 zConsider three data points (-1,,)1), ((,)0,2), ((,2,10) .

zThe Newton interpolating polynomial P2(x) is Px2010201012()()()()=+ a axx − + ax − x x − x =+ a ax (1)(1) ++ ax + x

P2 (1)1−=

P2 (0)= 2

P2 (2)= 10 zWe have x0=-1, y0=1, x1=0, y1=2, x2=2, y2=10.

ay00==1

ya10− 21− a1 = = = 1 xx10− 01−−()

yaaxx20120−+()() − 10−+×−−((())() 1 1 2 1 10−+ 1 3 a2 = = = =1 ()()x2021−−x x x (())()2120 −−− 32 × 31 Newton Divided Difference: 8/19 zThe nested form should be used to evaluate a Newton interpolating polynomial efficiently. zHere are two simple examples:

Px3012321()=+ a { a + [ a + axx ( − )]( xx − )}( xx − 0 )

Px40123432( )=+ a [ a + { a + [ a + a ( x − x )]( x − x )}( x − x 1 )]( x − x 0 )

zThe following computes Pn(x) for a given x.

Px = an DO i = n-1, 0, -1

Px = ai + Px*(x – xi) END DO zNumber of multiplications: n. zNewton method is faster than the Lagrange metho d,which requ ires n(n+1). 32 Newton Divided Difference: 9/19

zAdding one more data point (xn+1,yn+1) ? zPn+1(x) = Pn(x)+an+1(x-x0)(x-x1)…(x-xn). zSince ai’s and (xi,yi)’s (0 ≤ i ≤ n) are known, the use of subroutine Term-i() to compute an+1 requires 2(n+1)-1 = 2n +1 multiplications, similar to Lagrange method .

zSuppose we wish to add (4,16) to P2(x) computed earlier.

zSince P2(4)=26, we have a3 in P3(x) as follows:

yPx323− () 16−− 26 1 a3 == = (xxxxxx303132−−− )( )( ) (4 −−×−×− ( 1)) (4 0) (4 2) 4 1 P3 (x )=+ 1 ( x + 1) + ( xx + 1) − ( xxx + 1) ( − 2) 4 33 Newton Divided Difference: 10/19 zIn summary, we have the following : zIssue #1: n2 multiplications are needed to ggpgpyenerate the interpolating polynomial. Not bad! zIssue #2: n multiplications are required to compute Pn(x). Excellent! zIssue #3: 2n+1 multiplications are needed to add a new data point (xn+1,yn+1). Excellent! zCan we make issue #1 and #3 a little better or easier? Yes,,q this is the technique of divided difference.

34 Newton Divided Difference: 11/19 zThe leadinggff coefficient of a ppyolynomial is the coefficient of the highest degree term.

zThe leading coefficient of Pn(x)=a0+a1(x- x0)+…+ an(x-x0)(x-x1)()…(x-xn-1) is an. zSince an only depends on x0, x1, …, xn, we use a new symbol for an: an = f[x0, x1, …, xn]. zTherefore, Pi(x)’s can be rewritten as:

P00(x) = f [x ]

Px10010()=+ fx [] fx [,]( x x − x )

P2(x) = f [x 0] + f [x 01, x ](x − x 0) + f [x 012,,x x ](x −−x 0)(x x 1) n ⎡ i−1 ⎤ Pxnij()=+ fx []001 fx [,,,] x x⎢ ( x − x )⎥ ∑ ∏ 35 =i 1 ⎣ j =0 ⎦ Newton Divided Difference: 12/19 zHow can the new symbol help? zLet Pk-1(x) be a degree k-1 polynomial interpolating x0, …, xk-1 and Qk-1(x) be a degree k-1 polilitltilynomial interpolating x1, …, xk. zConsider the following polynomial of degree k.

xx−−0 xk x Pxkk()=+ Q−−11 () x P k () x xxkk−−00 xx

zPk(x) interpolates x0, …, xk (i.e., Pk(xi)=yi) because Pk(x0)=) = Pk-1(x0) = y0, Pk(xk)=) = Qk-1(xk)=) = yk, and for all 1≤ i ≤ k-1,

xxikiiki−−−−00 x x xx x x PQPki()x =+=+=Q ki−−11 ()x P ki ()x yyy i i i xxkkkk−−−−0000 xx xx xx 36 Newton Divided Difference: 13/19

zSince Pk(x) interpolates x0, x1, …, xk, its leading coefficient f[x0, x1, …, xk] is the sum of leading coefficients of xQk-1(x) and –xPk-1(x) .

xx−−0 xk x Pxkk()=+ Q−−11 () x P k () x xxkk−−00 xx degree k

zSince xQk-1(x)’s leading coefficient is f[x1, …, xk]/(xk-x0) and -xPk-1(x)’siss is -f[x0, …, xk-1]/(xk-x0), the leading coefficient of Pk(x) is

11f [,xxfxx101 ,kk ]− [, ,− ] fx[,,,]01 x xkkk=− fx [,,] 1 x fx [,, 0 x− 1 ] = xxkk−−00 xx xx k − 0

37 Newton Divided Difference: 14/19 zWhat does the following relation mean?

f [,xx101 ,kk ]− f [,xx ,− ] fx[][ 01,, x , xk ] = xxk − 0 zIt means we have a very simply way to compute all coefficients (i.e., ai’s). zThe re lati on sh ows how two adjacent f[x1,…,xk] and f[x0,…,xk-1] can be used to compute a new, expanded one f[x0,…,xk] with two subtractions and one division.

38 Newton Divided Difference: 15/19

zThe relation among f[x1,,,…,xk], f[x0,,,…,xk-1] and f[x0,…,xk] is triangular:

fx[,,,01 x xk − 1 ]

fx[,12 x , , xkkk ]→ fx [,, 01 x , x− 1 , x ] zWe may arrange all xi’s and f[xi]’soncolumns on column 0 and 1, respectively, and take two adjacent f[xi]’s to compute the f[xi,xi+1]’s on column 2. zSimilarly, column k-1 has all f[ ]’s with k-1 xi’s and two adjjpacent ones are used to compute all f[]’s with kxi’s. The results are on column k. zThis table looks like the following (()next slide):

39 Newton Divided Difference: 16/19

! initially di’s have yi’s ! at the end, d0, d1, …, dn are the ai’s DO j = 1, n ! column j DO i = n, j, -1 ! working upward

di = (di – di-1)/(xi –xi-j) END DO ! new di’s overwrites old di’s xfx00[] END DO

coefficient a ’s xfx11[]→ fxx [,] 01 i fff [][][xx, ][− f xx, ] fxx[, , x ]= 23 12 x fx[]→→ fxx [,] fxxx [,,] 123 2 2 12 012 xx31−

x33 fx[]→→ fxx [,] 23 fxxx [,,] 123 → fxxxx [,,,] 0123

x44f [x ] → f [x 34, x ] → f [x 234,,x x ] → f [x 1234,,,x x x ] → f [x 0123,,,,x x x x4 ] col 1 col 2 col 3 col 4 col 5 40 Newton Divided Difference: 17/19

zInterpolate (-1,1), (0,2), (2,10) and (4,16) using Newton method.

Px3 ()= fx[]0 + xi yi f[x0] fx[,](01 x x−+ x 0 ) − 11 fx[]()()[ 012,, xx]( x−− x 0 )( x x 1 ) + f [xxxxxx , , , ]( )( xxxx )( ) f[x ,x ] 0123−−− 0 1 2 02→ 1 0 1 1 1 1 1 f[x0,x1,x2] 2 1 1 1 2 10 → 4 → 1 10 4 1 1 16 3 -¼ -¼ 1 1 416→→−→− 3 f[x0,x1,x2,x3] 4 4 41 Newton Divided Difference: 18/19

zHow many divisions are there?

zStarting with f[xi], there are n+1 columns to compute f[x0,x1,…,xn]. zEach horizontal arrow Æ requires one division. zThe second column needs n, the third need n-1, …, and the n-th needs 1. The total is 1+2+…+n = n(n+1)/2 division. This is better than n2.

xfx00[]

xfx11[]→ fxx [,] 01

x2 fx[] 2→→ fxx [,] 12 fxxx [,,] 012

xfx3[] 3→→ fxx [,] 23 fxxx [,,] 123 → fxxxx [,,,] 0123 42 x4 fx[] 4→→ fxx [,] 34 fxxx [,,] 234 → fxxxx [,,,] 1234 → fxxxxx [,,,, 0123 4 ] Newton Divided Difference: 19/19

zHow about adding new data points? zOne may use the technique discussed earlier to

compute an+1 = f[x0,x1,..,xn+1] . zThe last row in the table must be saved to use divid ed difference. Do it yourself! zNumber of divisions is n, faster than 2n+1!

x00f [x ]

xfx11[]→ fxx [,] 01

xfx2[] 2→→ fxx [,] 12 fxxx [,,] 012 last row wish to compute this

x3 f [x3 ] →→f [x2 , x3 ] f [x1 , x2 , x3 ] →f [x0 , x1 , x2 , x3 ]

x fx[] fxx [,] fxxx [,,] fxxxx [,,,] fxxxxx [,,,, ] 43 new data 4 4→→ 34 234 → 1234 → 0123 4 Efficiency of Methods: A Summary

zThe following table summarizes the efficiency of each discussed method, based on the number of multiplications.

Naive Lagrange Newton Divided Difference 3 2 Computing ai’s O(n ) n(n+1) n n(n+1)/2

Computing P(x) nn(n+1) nn Adding a New Point O(n3)2(n+1) 2n+1 n

44 Example 1: 1/2 zSome functions are difficult to interpolate well with polynomials. The following is for √x at 0, 0.2, 0.4, 0.6, 0.8 and 1 with divided difference. Divided Difference

a0 a1 a2 a3 a4 a5 0.0 2.23606798 -3.2746457 4.5598096 -5.1583509 4.8266509

x P5(x) sqrt(x) |error| 0.1 0.28283819 0.31622777 0.03338958 0.3 0.55208708 0. 54772256 0000.00436452 0.5 0.70514379 0.70710678 0.00196299 070.7 0.83867091 0. 83666003 0.00201088 0.9 0.94392207 0.94868330 0.00476123 45 Example 1: 2/2 Interpolating polynomials can wiggle around

y = x

yx=

P(x)

f(x)

wiggling

0 1 0.8 1 46 Example 2: 1/3 zThis “wiggling” can be worse. Consider Runge’s example 1/(1+x2) at -5, -2.5, 0, 2.5 and 5. Divided Difference

a0 a1 a2 a3 a4 0.038461538 0.039787798 0.061007958 -0.026525199 5.3050398E-3

2 x P4(x) 1/(1+x ) |error| ±4 -0.37931034 0.058823529 0.43813387 ±3 -0.11007958 0.1 0.21007958 ±2 0.40053050 0.2 0.20053050 ±1 0.83421751 0.5 0.33421751 Note that this function is symmetric about the origin 47 Example 2: 2/3 0

1 -2.5 f ()x = 2.5 -5 1+ x2 5

48 Example 2: 3/3

1 fx()= 1+ x2

x = -5, -3.75, -2.5, -1.25, 0, 1.25, 2.5, 3.75, 5

Situation can become worse with more points! 49 Interpolation Error zLet Pn(x) be the polynomial interpreting an unknown function f(x) at x0, x1, …, xn. Then, we have f [1]n+ ()µ fxPx()−= () ( xx −− )( xx ) ( xx − ) nn(1)!n + 01 where µ lies in the smallest interval containing x0, x1, …, xn and x (i.e., in the interval [min(x,x0,…,xn), max(x,x0,…,xn)]). zTherefore, if f[n+1](x) is bounded in the indicated interval, interpolating error comes, mostly,

from (x-x0)(x- x1)…(x- xn). 50 Chebyshev Points (Nodes) zRather than usinggq equall ypy spaced points , Chebyshev suggested the following: ⎛⎞2(ni− )+ 1 xi = cos⎜⎟π where in= 010,1,,… ⎝⎠22n + zSince Chebyshev points are in [-1,1], one must scale them before use. zFor example, to use Chebyshev points on interval [a,b], the scaled xi is a+((b-a)/2)*(xi+1), where xi is a Chebyypshev point. zUse the Chebyshev points xi’s to find yi’s, and interpolate (xi, yi)’s with a ppyolynomial.

51 Example: 1/3 zConsider Runge’s example 1/(1+x2) with 5 Chebyshev points. Divided Difference

a0 a1 a2 a3 a4 0.042350069 0.033811409 0.057019170 -0.020896600 4.3943971E-3

2 xi P4(x)1/(1+x ) |error| ±5 0.2035155 2 0.038461538 0.1650539 8 ±4 -0.14254326 0.058823529 0.20136679 ±3 0.080472398 0.1 0.019527602 ±2 0.50343312 0.2 0.30343312 ±1 0.86267509 0.5 0.36267509

Note that this function is symmetric about the origin 52 Example: 2/3 equally spaced Chebyshev points

1 1+ x2 Chebyshev points

equally spaced

Degree 4 interpolation 53 Example: 3/3

1 1+ x2

Chebyshev points

equally spaced

Degree 8 interpolation 54 A Short Summary zThe Newton divided difference method is the best of the three discussed here. zConsider the use of Chebyypshev points if the degree of an interpolating polynomial is high. zThere are more advanced methods: rational functions and spline functions. zA rational function is the quotient of two polynomials. zA spline function is a collection of lower degree polynomials strung together acting as a single function.

55 Polynomial Approximation: 1/8 zWe mayyyp have many data points (xi,yi)’s, each of which may be noisy. zIn this case, interpolation is not the best way because (1) the degree can be too hig h, an d (2) interpolating noisy data is useless. zApproximation means the curve follows the trend of the data points closely rather than passing through all of them. zA polynomial P(x) of degree n requires n+1 data points for interpolation; however, n+1 data poitints can b e approx imat tded w ith a po lynomi ilal of degree m < n. Now, m is a parameter! zA simple way is the least-square method. 56 Polynomial Approximation: 2/8 zSuppose we have n+1 data points (x0,y0), (x1,y1), …, (xn,yn) and wish to find an approximation polynomial P(x) of degree m, where m < n. zConsider the “error” ei at point (xi,yi) .

ei = yi - P(xi) error at xi : ei = yi - P(xi) y=P(x)

(xi,yi)

P(xj) ej = yj - P(xj)

P(xi) (xj,yj)

57 Polynomial Approximation: 3/8 zThe least-square method minimizes the sum of squared errors: mm 22 min∑eyPxiii=−∑ ( ( )) ii==00 zHow to f?find the minimum? You perhaps learned it in calculus and/or linear algebra. 2 m zSince polynomial P(x)=a0+a1x+a2x +…+amx has unknowns a0, a1, …, am, minimization means: find a0, a1, …, am so that the above sum of squared errors reaches a minimum!

58 Polynomial Approximation: 4/8 zRecall the following fact: ⎡⎤a0 ⎢ a ⎥ ⎢ 1 ⎥ 22 mm⎡⎤… ⎢⎥ Px()iiiiiiiiii= a01++ ax ax 2 ++ axm iiii= ⎣⎦ 1,,,, x x x a 2 ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣am ⎦ zWe can collect all xi’s into a matrix form: (n+1)×(m+1) 2 m (n+1)×1(⎡⎤P()xa00⎡⎤1 xx00 x 0 ⎡⎤m+1)×1 ⎢⎥⎢⎥2 m ⎢⎥ P()xa1 xx x ⎢⎥11⎢ 11 1⎥ ⎢⎥ 2 m ⎢⎥P()xa22= ⎢⎥1 xx xi ⎢⎥ ⎢⎥⎢⎥22 2 ⎢⎥ ⎢⎥⎢ ⎥ ⎢⎥ ⎢⎥P()xa⎢⎥1 xx2 xm ⎢⎥ ⎣⎦nm⎣⎦nn n ⎣⎦ 59 Polynomial Approximation: 5/8 zThe error terms, also in matrix form, are: E Y A X2 m ⎡eyPxy00⎤⎡⎤⎡() 0 ⎤⎡⎤ 0⎡⎤1 xx00 x 0 ⎡⎤ a 0 ⎢⎥⎢⎥⎢ ⎥⎢⎥⎢⎥2 m ⎢⎥ eyP()xy1 xx x a ⎢ 11⎥⎢⎥⎢ 1 ⎥⎢⎥ 1⎢⎥11 1 ⎢⎥ 1 2 m ⎢eyPxy22⎥⎢⎥⎢=−() 2 ⎥⎢⎥ =− 2⎢⎥1 xx xi ⎢⎥ a 2 ⎢⎥⎢⎥⎢ ⎥⎢⎥⎢⎥22 2 ⎢⎥ ⎢⎥⎢⎥⎢ ⎥⎢⎥ ⎢⎥ ⎢ ⎥ ⎢⎥⎢⎥⎢ ⎥⎢⎥⎢⎥2 m ⎢⎥ ⎣eyPxynn⎦⎣⎦⎣() n ⎦⎣⎦ n⎣⎦1 xxnn x n ⎣⎦ a m zIf we use E, Y, X and A to indicate the error

vector, y vector, matrix of [xi,j] and the unknown coefficients [ai], the above becomes: EYXA=−⋅

60 Polynomial Approximation: 6/8

zNote that ET⋅E is the sum of squared errors:

⎡⎤e0 ⎢⎥e n ⎢⎥1 2 ⎢⎥ T ∑eeeeeeEEin==[]012,,,…i , 2 i i−1 ⎢⎥ ⎢⎥ ⎢⎥ ⎣⎦en zPlugging equation EEY=Y-X⋅A into the above yields:

m 2 TTTTT ∑eYXAYXAYYXYAAXXAi =−⋅()()2 −⋅=⋅− ⋅⋅+ () i=0

61 Polynomial Approximation: 7/8 zNote that ET⋅E is the following: m 2 TT TT ∑eYYXYAAXXAi =⋅−2() ⋅⋅+ i=0 zDifferentiating with respect to A yields: m ⎛⎞2 ∂ ⎜⎟∑ei ⎝⎠i=0 TT =−22(X YXXA) + ( ) ∂A zSetting the above to zero and solving for A gives the minimum:

TT XX A(mm 1)1= XY ( 1)1 ( )(1)(1)mm+× + +× +× 62 Polynomial Approximation: 8/8 zSince (XTX)A = XTY,,yg we have a way of finding A that minimizes the sum of squared errors:

‹Read in (xi,yi)’s (0 ≤ i ≤ n))g and the degree m ‹Form matrices X(n+1)×(n+1) and Y(n+1)×1 T T ‹Compute (X X)(m+1)×(m+1) and X Y(m+1)×1 ‹Solve A from (XTX)A = XTY

‹Matrix A(m+1)×1 gives the coefficients of polynomial P(x) of degree m zMultiplications used for computing XTX, XTY and A are O(n3), O(n2) and O(m3). Actually, it is O(n3), because in approximation n > m.

63 Example: 1/3

zSuppose we have 11 data xi yi points as shown in the table. 0.0 3.12308121 0.4 5.15545177 zTherefore, n =10= 10. 0.8 3.98456621 zBut, degree 10 may be too 1.2 3.51668167 high f or practi ca l purpose. 1.6 1.83907247 zLeast square with a degree 3 2.0 -0.35763431 polynomial yields the 242.4 -1.48452199 following result: 2.8 -1.51532280 323.2 -1.24526966 23 fx()=+341285 . 4 . 85546 x − 543021 . x + 108126 . x 3.6 1.20831132 4.0 4.96375036

64 Example: 2/3 zThis diagram shows the approximation polynomial of degree 3 and data points. zThe approximation polynomial follows the shape of the data points closely, but it does not pass through all of them.

65 Example: 3/3 zThis diagram shows both degree 2 and degree 3 degree 3 polynomials. zDegree 2 is obviously inadequate because the curve is too far away from the data points . zDegree 4 is a bit better; degree 2 btthibut, the improvemen tit is not significant in this case. 66 The End

67