CS321

Numerical Analysis

Lecture 6

Approximation by Spline Functions

Professor Jun Zhang

Department of Computer Science University of Kentucky Lexington, KY 40506-0046 Lower Degree Splines

Given a set of data, it is possible to construct a polynomial to interpolate these data. The disadvantage of higher order polynomials is its oscillation behavior

A spline function is a function that consists of polynomial pieces joined together with certain smoothness conditions

First degree spline function is a polygonal function with linear polynomials joined together to achieve continuity

The points t0, t1,…, tn at which the function changes its character are called knots

Note that knots do not have to be nodes 2 High Order Polynomials

Interpolation uses just one polynomial which may be oscillatory if the order of the polynomial is high 3 High Order Polynomial Interpolations

Red curve is the Runge function to be interpolated. Blue curve is the 5th order polynomial, green is the 9th order polynomial interpolations. 4 Splines

A spline function is a function that consists of polynomial pieces joined together with certain smoothness conditions 5 Piecewise Linear Polynomial

A piecewise linear function for the spline of degree 1 can be written as

S0 (x) x [t0 ,t1 ] S (x) x [t ,t ] S(x)  1 1 2     Sn1 (x) x [tn1 ,tn ] where Si (x)  ai x  bi is a linear polynomial

The knots t0, t1,…, tn and the coefficients ai, bi, i = 0, 1,…, n – 1 have to be known in order to evaluate S(x)

First to determine which interval x lies, then evaluate the linear function defined on that interval

6 Piecewise Linear Polynomial

To evaluate S(x), first to determine which interval x lies, then evaluate the linear function defined on that interval

7 Spline of Degree 1

A function S(x) is called a spline of degree 1 if

1. The domain of S is an interval [a,b]

2. S is continuous on [a,b]

3. There is a partitioning of the interval a = t0 < t1 < … < tn = b such that S is a linear polynomial on each subinterval [ti,ti+1]

For outside part of the interval [a,b], we define S(x) = S0(x) when x < a, and S(x) = Sn-1(x) when x > b

It is important that the spline of degree 1 be continuous at the knots, i.e., the left limit and the right limit are equal lim S(x)  lim S(x)  f (t )   i xti xti 8 An Example

Determine if the following function is a first degree spline x  1  x  0.5  S(x)  0.5x  2(x  0.5) 0.5  x  2 x  1.5 2  x  4 Each linear function is continuous on the subinterval it is defined. We need to verify if they are continuous at the two interior knots x = 0.5 and x = 2 lim S(x)  lim x  0.5 x0.5 x0.5 lim S(x)  lim 0.5x  2(x  0.5)  0.25 x0.5 x0.5 The function is not a spline of degree 1, as

lim S(x)  lim S(x) x0.5 x0.5 i.e., S(x) is discontinuous at the knot x = 0.5 9 Construct Spline of Degree 1

Given a data set with t0 < t1 < … < tn

A linear polynomial can be constructed using two pairs of neighboring data

First compute the slope of the line as

yi1  yi mi  ti1  ti The straight line equation is given by the point-slope formula as

Si (x)  yi  mi (x  ti )

It is easy to see that we have 2n degrees of freedom ai and bi and 2n conditions. So the construction of first degree spline is guaranteed 10 Piecewise Linear Interpolation

Piecewise linear interpolation is not smooth at the knot points

11 Modulus of Continuity

The modulus of continuity of a function f is defined as, for a ≤ u ≤ v ≤ b ( f ;h)  sup f (u)  f (v) : u  v  h The quantity is the largest variation of f over a small interval of size h. It measures how much f can change in such an interval

If f is continuous on [a,b] then lim ( f ;h)  0 h0 If f is differentiable on [a,b] then

f (u)  f (v)  f '(c)(u  v)  M1 u  v  M1h where M1 is the maximum value of |f’(x)| on (a,b). It follows that

( f ;h)  M1h

12 Theorems

If p is the first degree polynomial that interpolates a function f at the end points of an interval [a,b], then with h = b – a, we have f (x)  p(x)  ( f ;h) (a  x  b)

Note that the linear function passes through [a,b] can be written as  x  a   b  x  p(x)    f (b)    f (a)  b  a   b  a  Hence  x  a   b  x  f (x)  p(x)   f (x)  f (b)   f (x)  f (a)   b  a   b  a  Note that  x  a   b  x    f (x)    f (x)  f (x)  b  a   b  a  The result follows immediately 13 Accuracy of First Degree Spline

Let p be a first degree spline having knots a = x0 < x1 < … < xn = b. If p interpolates a function f at these knots, then with h = maxi(xi - xi-1) we have f (x)  p(x)  ( f ;h) (a  x  b)

If more knots are inserted such that the maximum spacing h goes to zero, the corresponding first degree spline will converge uniformly to f

If f’ or f” exist and are continuous, we have h f (x)  p(x)  M (a  x  b) 1 2 h2 f (x)  p(x)  M (a  x  b) 2 8 where M1 and M2 are the maximum values of f’ and f” on (a,b), respectively 14 Accuracy of First Degree Spline

From the estimates of the accuracy of the first-degree spline interpolation h f (x)  p(x)  M (a  x  b) 1 2 h2 f (x)  p(x)  M (a  x  b) 2 8 It is obvious that when h approaches zero, with more knot points inserted, the error of the first-degree spline goes to zero

It follows that the spline interpolation can be as accurate as needed, by inserting sufficient number of knot points

The same cannot be said about the polynomial interpolation, because of the oscillations associated with high order polynomials

15 Piecewise Linear Polynomial

The interpolation does not look smooth at knots

16 Second Degree Spline

A function Q is a second degree spline if

1.) the domain of Q is an interval [a,b]

2.) Q and Q’ are continuous on [a,b]

3.) there are points ti such that a = t0 < t1 < … < tn = b and Q is a polynomial of degree at most 2 on each subinterval [ti,ti-1]

A quadratic spline is a continuously differentiable piecewise quadratic function. It is a linear combination of basic functions 1, x, x2. The smoothness condition is stronger than that for the first degree spline

To determine a quadratic spline, we must verify the continuity of both Q and Q’ at the knot points

17 Interpolating Quadratic Spline

Given a set of data

A quadratic spline Q(x) can be constructed to interpolate these data, using t0, t1,…, tn as the knots

A quadratic spline consisting of n separate pieces of quadratic functions of the form 2 Qi (x)  ai x  bi x  ci x  ti ,ti1  There are 3n coefficients to be determined

The interpolating conditions Qi(ti) = yi and Qi(ti+1) = yi+1 give 2n conditions. The continuity of Q’(x) gives another n – 1 conditions at the interior knots. The last condition can be specified (arbitrarily) as Q’(t0) = 0

18 Computational Procedure

Construct a piecewise quadratic function

Q0 (x) x [t0 ,t1 ] Q (x) x [t ,t ] Q(x)  1 1 2     Qn1(x) x [tn1 ,tn ] which is continuously differentiable on [t0,tn] and Q(ti) = yi for 0 ≤ i ≤ n

The computational procedure is based on a recursive procedure on Q’i(ti). Assuming Q’(ti)= zi, then we have zi1  zi 2 Q i (x)  (x  ti )  zi (x  ti )  yi 2(ti1  ti )

This definition can be verified by showing that Qi(ti)= yi, Q’i(ti)= zi, and Q’i(ti+1) =zi+1. Note that zi1  zi Q i '(x)  (x  ti )  zi ti1  ti 19 Computational Procedure

The defined quadratic function Qi(x) does not necessarily satisfy the condition Qi(ti+1) = yi+1 for i = 0, 1,…, n – 1. If we apply this condition, we have zi1  zi 2 yi1  (ti1  ti )  zi (ti1  ti )  yi 2(ti1  ti ) After simplification, we obtain

 yi1  yi  zi1  zi  2  (0  i  n  1)  ti1  ti 

From this equation, we can compute the values of z0, z1,…, zn recursively, by assigning an arbitrary value to z0. Then the second degree spline is written as zi1  zi 2 Qi (x)  (x  ti )  zi (x  ti )  yi 2(ti1  ti ) for each interval [ti,ti+1], i = 0, 1,…, n – 1.

20 Linear and Quadratic Splines

Quadratic polynomial interpolation has better smoothness than the linear polynomial interpolation 21 Subbotin Quadratic Spline

Knots are points where the spline function can change its form, nodes are points where the values of the function is specified

It is not necessary that the knots are nodes.

Subbotin suggests to use the midpoints of the knots as the (interpolation) nodes and the two end points. This yields n + 2 conditions. The continuity of Q and Q’ give another 2(n – 1) conditions, for a total of 3n conditions. No free variable is required

Let the knots be a = t0 < t1 < … < tn = b, we define the interpolation nodes as

p0  t0 pn  tn  1 pi  2 (ti  ti1) (1 i  n)

22 Subbotin Quadratic Spline

The knots are a = t0 < t1 < … < tn = b, the interpolation nodes as

 p0  t0 pn  tn  1  pi  2 (ti  ti1) (1 i  n) 23 Subbotin Quadratic Spline (II)

The quadratic spline function Q(x) should satisfy the interpolation conditions Q( pi )  yi (0  i  n 1)

Let Qi’(ti) = zi, where Qi(x) is defined on the interval [ti,ti+1], which can be written as

1 Qi (x)  yi1  2 (zi1  zi )(x  pi1)  1 (z  z )(x  p )2 2hi i1 i i1 where hi = ti+1 – ti. We can verify that ' ' Qi(pi1) yi1, Qi(ti)zi, Qi(ti1)zi1

We also want to impose the continuity condition at the knots

lim Q (x)  lim Q (x) (1  i  n  1)  i1  i xti xti

24 Subbotin Quadratic Spline (III)

These conditions lead to a recursive formula

hi1zi1  3(hi1  hi )zi  hi zi1  8( yi1  yi ) for i  1,2,,n  1 We impose the first and last interpolation conditions

Q( p0 )  y0 Q( pn1)  yn1 which yield two more equations

3h0 z0  h0 z1  8( y1  y0 )

hn1zn1  3hn1zn  8( yn1  yn )

Now we have n + 1 equations for the n + 1 unknowns z0, z1,…, zn

We need to solve a linear system of the form Ax = b with A being a tridiagonal matrix

25 An Example

Determine if the function is a quadratic spline  x 2 x  0 Q(x)    x x  0 The interval can be viewed as (−∞,∞) which is not a closed interval

Also, the derivative of Q(x) is  2x x  0 Q'(x)    1 x  0 Hence, Q’(x) is discontinuous at x = 0. It follows that Q(x) is not a quadratic spline

How about the function 2 0.1x 0  x  1 Q(x)   2 9.3x  18.4x  9.2 1  x  1.3 26 Natural Cubic Spline

Very high order polynomials oscillate and give undesired feature

Lower order splines do not oscillate, but are not very smooth at the joints (knots) where they may change their characters (shape)

The derivative of the first order spline may not be continuous, the slope of the spline may change abruptly at the knots

The second derivative of the second order spline may not be continuous. The curvature of the quadratic spline may change abruptly at the knots

A better strategy is to use moderate order polynomials to construct spline and to impose smoothness conditions at the knots so that the resulting spline looks smooth but is not oscillatory

27 Linear Spline

Linear polynomial interpolation is not smooth 28 Quadratic and Cubic Splines

Smooth higher order polynomial splines 29 Spline of Degree k

A function S is a spline of degree k if

1.) The domain of S is an interval [a,b]

2.) S, S’,…, S(k-1), are all continuous functions on [a,b]

3.) There are points ti (the knots of S) such that a = t0 < t1 < … < tn = b and such that S is a polynomial of degree at most k on each subinterval [ti,ti+1]

Smoothness is reflected by the order of the continuous derivative at the knots

If we want the approximating spline to have continuous mth derivative, we should choose a spline of degree at least m + 1

30 Cubic Spline

For a spline of degree (m + 1) at any interior knot ti, we have lim S (k ) (x)  lim S (k ) (x) (0  k  m)   xti xti

Since different polynomials are defined at different side of ti, these two polynomials will be the same if they are at most degree m and their zeroth to mth derivatives are equal

In most applications, a spline of degree 3 is chosen (smooth and inexpensive to construct)

This spline has continuous first and second derivatives

Its graph looks smooth, although higher derivatives may be discontinuous

20 Natural Cubic Spline

Cubic spline interpolation is more smooth 32 Natural Cubic Spline

Given a table of data set

where the ti’s are the knots in ascending order

We want to construct a function S with n piecewise cubic polynomials

S0 (x) t0  x  t1 S (x) t  x  t S(x)  1 1 2     Sn1 (x) tn1  x  tn

Si(x) is the cubic polynomial defined on the subinterval [ti,ti+1], the cubic polynomial looks like 3 2 Si (x)  ai x  bi x  ci x  di 0  i  n  1 We have a total of 4n unknowns to be determined

33 Natural Cubic Spline

S(x) must satisfy the interpolation conditions

S(ti )  yi (0  i  n) This will yield 2n conditions for n intervals

S(x), S’(x) and S”(x) have to satisfy the continuity conditions at the interior knots t1, t2,…, tn-1 lim S (k ) (x)  lim S (k ) (x) (k  0,1,2)   xti xti This will yield 2(n – 1) conditions. (Here we only count k = 1, 2).

The remaining two (natural) conditions are imposed at the end knots as

S"(t0 )  S"(tn )  0 These particular conditions give the name of natural cubic spline. Other end knot conditions can also be imposed 34 Computational Algorithm

Since S”(x) is continuous, we define

zi  S"(ti ) (0  i  n) with z0 = zn = 0. For the interval [ti,ti+1], Si”(x) is continuous and takes the values of zi and zi+1 at knots ti and ti+1 (interpolation on interval [ti,ti+1])

zi 1 zi Si "(x)  (x  ti )  (ti1  x) hi hi with hi = ti+1 – ti for 0 ≤ i ≤ n – 1

It can be verified that Si”(ti) = zi, Si”(ti+1) = zi+1 for the interpolation conditions of Si”(x)

If we integrate Si”(x) twice, we get Si(x) zi 1 3 zi 3 Si (x)  (x  ti )  (ti1  x)  cx  d 6hi 6hi with c and d being two constants 35 Computational Algorithm (II)

The cubic polynomial can be rewritten as

zi1 3 zi 3 Si (x)  (x  ti )  (ti1  x)  Ci (x  ti )  Di (ti1  x) 6hi 6hi

Since Si(x) must also interpolate at ti and ti+1 we have Si(ti) = yi and Si(ti+1) = yi+1, the constants Ci and Di can be determined

We now have the polynomial as a function of the zi’s zi1 3 zi 3 Si (x)  (x  ti )  (ti1  x) 6hi 6hi

 yi1 hi     zi1 (x  ti )  hi 6 

 yi hi     zi (ti1  x)  hi 6  36 Computational Algorithm (III)

In order to impose the continuity conditions for S’(x), we have

zi1 2 zi 2 Si '(x)  (x  ti )  (ti1  x) 2hi 2hi

yi1 hi yi hi   zi1   zi hi 6 hi 6

We have at the knot ti h h S '(t )   i z  i z  b i i 6 i1 3 i i 1 bi  ( yi1  yi ) hi and h h S' (t )   i1 z  i1 z  b i1 i 6 i1 3 i i1 1 b  ( y  y ) i1 h i i1 i1 37 Computational Algorithm (IV)

The continuity condition implies Si’(ti) = S’i-1(ti), i.e.,

hi1zi1  2(hi1  hi )zi  hi zi1  6(bi  bi1 ) For 1 ≤ i ≤ n – 1. We have to solve a tridiagonal system of equations

z0  0

hi1zi1  ui zi  hi zi1  vi (1  i  n  1)

zn  0 with ui  2(hi1  hi )

vi  6(bi  bi1 ) Note that we have a tridiagonal system to solve

It can be shown that pivoting is not needed to solve the tridiagonal system, because the coefficient matrix is diagonally dominated

38 Comparison of Interpolations

Comparison of linear and cubic spline interpolation

39 Smoothness of Cubic Spline

The smoothness of an interpolation polynomial can be measured in some sense by the degree of fluctuation of its derivatives

The cubic spline can be shown as “smooth”

If S is the natural cubic spline function that interpolates a twice continuously differentiable function f at knots a = t0 < t1 < … < tn = b, then b b [S"(x)]2 dx  [ f "(x)]2 dx a a

If the graph of a function changes abruptly at some knot, we can construct two different natural cubic splines at different side of that knot to avoid forcing derivative continuity at a point where the function’s derivative is not continuous

40 Smoothness Proof

We define g(x)  f (x)  S(x)

It follows that

g(ti )  0 for 0  i  n because ti are the knot points Furthermore f "(x)  S"(x)  g"(x)

And b b b b ( f ")2 dx  (S")2 dx  (g")2 dx  2 S"g"dx a a a a

41 Smoothness Proof

Look at the last term and use integration by parts b b b b S"g"dx  S"g'|a  S"'g'dx   S"'g'dx a a a We used the facts of natural cubic spline that S"(a)  0 and S"(b)  0 Now, we have b n1 t S"' g'dx  i1 S"' g'dx a  t i0 i Since S is a cubic polynomial in each interval [ti, ti+1], its third derivative is a constant, say ci, so n1 n1 b ti1 S"' g'dx  ci g'dx  ci[g(ti1)  g(ti )]  0 a  t  i0 i i0 42 Smoothness Proof

So from b b b b ( f ")2 dx  (S")2 dx  (g")2 dx  2 S"g"dx a a a a and the fact that b S"g"dx  0 a

We have b b b b ( f ")2 dx  (S")2 dx  (g")2 dx  (S")2 dx a a a a

Thus, the smoothness of the cubic spline is proved

43