Interpolation and Curve Fitting

Interpolation and Curve ﬁtting

Spring 2019

Interpolation and Curve fitting Spring 2019 1 / 19 Curve fitting Find a function that is a good fit to the original data points The function does not have to pass through the original data points.

Interpolation vs Curve ﬁtting

n Given some data points {xi , yi }i=1 and assuming there is some function f (x) describes the quantity of interest at all points.

Interpolation Find a function satisfying

P(xi ) = f (xi ), i = 1,..., n

that allows us to approximate f (x) such that the function values between the data sets may be estimated.

Interpolation and Curve ﬁtting Spring 2019 2 / 19 Interpolation vs Curve ﬁtting

n Given some data points {xi , yi }i=1 and assuming there is some function f (x) describes the quantity of interest at all points.

Interpolation Find a function satisfying

P(xi ) = f (xi ), i = 1,..., n

that allows us to approximate f (x) such that the function values between the data sets may be estimated.

Curve ﬁtting Find a function that is a good ﬁt to the original data points The function does not have to pass through the original data points.

Interpolation and Curve ﬁtting Spring 2019 2 / 19 Piecewise Linear Interpolation

Simplest case: 2 points Basic algebra problem

Given two points (x1, f (x1)) and (x2, f (x2)), ﬁnd P1(x) such that

P1(x1) = f (x1)

P2(x2) = f (x2)

P1 can be written in the form x − x2 x − x1 P1(x) = f (x1) + f (x2) x1 − x2 x2 − x1

Interpolation and Curve ﬁtting Spring 2019 3 / 19 Exercise: Interpolate 10 random data points with values on [0, 10] and evaluate the polynomial on 1:0.1:10 and plot.

Linear Interpolation

6 y 5

1 1 2 3 4 5 6 7 8 9 10 x

Interpolation in MATLAB interp1

vq = interp1(x,v,xq,method)

x – sample points v – values f (x) xq – query points on which the polynomial will be evaluated method – method of interpolation (e.g. linear, cubic, e.t.c)

Interpolation and Curve ﬁtting Spring 2019 4 / 19 Interpolation in MATLAB interp1

vq = interp1(x,v,xq,method)

x – sample points v – values f (x) xq – query points on which the polynomial will be evaluated method – method of interpolation (e.g. linear, cubic, e.t.c) Exercise: Interpolate 10 random data points with values on [0, 10] and evaluate the polynomial on 1:0.1:10 and plot.

Linear Interpolation

6 y 5

1 1 2 3 4 5 6 7 8 9 10 x

Interpolation and Curve ﬁtting Spring 2019 4 / 19 Higher order interpolation : Lagrange interpolant

Idea: Lagrange (1736 - 1813) Given n + 1 data points (x1, f (x1)), (x2, f (x2)),..., (xn, f (xn)), (xn+1, f (xn+1)) , deﬁne a polynomial n+1 X P(x) = f (xj )ϕj (x) i=1 where ( 0 if j = i ϕj (xi ) = δij = 1 if j 6= i Interpolation and Curve ﬁtting Spring 2019 5 / 19 Notice that: 1 0 x − x ¨¨* x − x ¨¨* P (x ) = 1 ¨2 f (x ) + 1 ¨1 f (x ) = f (x ) 1 1 ¨¨ 1 ¨¨ 2 1 ¨x1 − x2 ¨x2 − x1

¨* 0 ¨* 1 x2 − x2¨ x2 − x1¨ P (x ) = ¨¨ f (x ) + ¨¨ f (x ) = f (x ) 1 2 ¨ 1 ¨ 2 2 ¨x1 − x2 ¨x2 − x1

What does ϕ(x) look like?

In the linear case: x − x2 x − x1 P1(x) = f (x1) + f (x2) x1 − x2 x2 − x1

Interpolation and Curve ﬁtting Spring 2019 6 / 19 ¨* 0 ¨* 1 x2 − x2¨ x2 − x1¨ P (x ) = ¨¨ f (x ) + ¨¨ f (x ) = f (x ) 1 2 ¨ 1 ¨ 2 2 ¨x1 − x2 ¨x2 − x1

What does ϕ(x) look like?

In the linear case: x − x2 x − x1 P1(x) = f (x1) + f (x2) x1 − x2 x2 − x1

Notice that: 1 0 x − x ¨¨* x − x ¨¨* P (x ) = 1 ¨2 f (x ) + 1 ¨1 f (x ) = f (x ) 1 1 ¨¨ 1 ¨¨ 2 1 ¨x1 − x2 ¨x2 − x1

Interpolation and Curve ﬁtting Spring 2019 6 / 19 What does ϕ(x) look like?

In the linear case: x − x2 x − x1 P1(x) = f (x1) + f (x2) x1 − x2 x2 − x1

Notice that: 1 0 x − x ¨¨* x − x ¨¨* P (x ) = 1 ¨2 f (x ) + 1 ¨1 f (x ) = f (x ) 1 1 ¨¨ 1 ¨¨ 2 1 ¨x1 − x2 ¨x2 − x1

¨* 0 ¨* 1 x2 − x2¨ x2 − x1¨ P (x ) = ¨¨ f (x ) + ¨¨ f (x ) = f (x ) 1 2 ¨ 1 ¨ 2 2 ¨x1 − x2 ¨x2 − x1

Interpolation and Curve ﬁtting Spring 2019 6 / 19 where:

(x − x2)(x − x3) (x − x1)(x − x3) (x − x1)(x − x2) ϕ1(x) = , ϕ2(x) = , ϕ3(x) = (x1 − x2)(x1 − x3) (x2 − x1)(x2 − x3) (x3 − x1)(x3 − x2)

In the general case, given n + 1 data points we can write down the Lagrange interpolating polynomial of degree n of this form

What does ϕ(x) look like?

Therefore: P1(x) = ϕ1(x)f (x1) + ϕ2(x)f (x2) where x − x2 x − x1 ϕ1(x) = , ϕ2(x) = x1 − x2 x2 − x1 In the case of 3 data points (x1, f (x1)), (x2, f (x2)), (x3, f (x3))

P2(x) = ϕ1(x)f (x1) + ϕ2(x)f (x2) + ϕ3(x)f (x3)

Interpolation and Curve ﬁtting Spring 2019 7 / 19 In the general case, given n + 1 data points we can write down the Lagrange interpolating polynomial of degree n of this form

What does ϕ(x) look like?

Therefore: P1(x) = ϕ1(x)f (x1) + ϕ2(x)f (x2) where x − x2 x − x1 ϕ1(x) = , ϕ2(x) = x1 − x2 x2 − x1 In the case of 3 data points (x1, f (x1)), (x2, f (x2)), (x3, f (x3))

P2(x) = ϕ1(x)f (x1) + ϕ2(x)f (x2) + ϕ3(x)f (x3)

where:

(x − x2)(x − x3) (x − x1)(x − x3) (x − x1)(x − x2) ϕ1(x) = , ϕ2(x) = , ϕ3(x) = (x1 − x2)(x1 − x3) (x2 − x1)(x2 − x3) (x3 − x1)(x3 − x2)

Interpolation and Curve ﬁtting Spring 2019 7 / 19 What does ϕ(x) look like?

Therefore: P1(x) = ϕ1(x)f (x1) + ϕ2(x)f (x2) where x − x2 x − x1 ϕ1(x) = , ϕ2(x) = x1 − x2 x2 − x1 In the case of 3 data points (x1, f (x1)), (x2, f (x2)), (x3, f (x3))

P2(x) = ϕ1(x)f (x1) + ϕ2(x)f (x2) + ϕ3(x)f (x3)

where:

(x − x2)(x − x3) (x − x1)(x − x3) (x − x1)(x − x2) ϕ1(x) = , ϕ2(x) = , ϕ3(x) = (x1 − x2)(x1 − x3) (x2 − x1)(x2 − x3) (x3 − x1)(x3 − x2)

In the general case, given n + 1 data points we can write down the Lagrange interpolating polynomial of degree n of this form Interpolation and Curve ﬁtting Spring 2019 7 / 19 Lagrange interpolating polynomial

Lagrange Interpolant

0 y

-5

-10

-15

1 2 3 4 5 6 7 8 9 10 x

Disadvantage As the polynomial degree increases, the error near the endpoints increases (MA 427)

Interpolation and Curve ﬁtting Spring 2019 8 / 19 Cubic spline interpolants

A cubic spline interpolant is a piecewise cubic that interpolates the function f at x1, x2, x3,..., xn and has two continuous derivatives. Why Splines? Useful in applications that require smooth surfaces, e.g. design of airplane parts or cars. Cublic spline Interpolation

5 y

1 2 3 4 5 6 7 8 9 10 x Interpolation and Curve ﬁtting Spring 2019 9 / 19 The curve can be 1 Polynomials of degree n 2 Trigonometric 3 Exponential

Least Squares curve ﬁtting

A least squares curve ﬁt can be used to obtain a curve such that the squared distance from each point to the curve is minimized.

Interpolation and Curve ﬁtting Spring 2019 10 / 19 Least Squares curve ﬁtting

A least squares curve ﬁt can be used to obtain a curve such that the squared distance from each point to the curve is minimized. The curve can be 1 Polynomials of degree n 2 Trigonometric 3 Exponential

Interpolation and Curve ﬁtting Spring 2019 10 / 19 Linear Regression

Given N data points (xk , yk ), k = 1,..., N we seek a line f (x) = mx + b so that the mean square error (MSE):

N 2 1 X MSE(m, b) = y − f (x ) N k k k=1 N 2 1 X = y − (mx + b) N k k k=1 We can compute partial derivatives with respect to m and b to ﬁnd values of m and b that minimize the error.

Interpolation and Curve ﬁtting Spring 2019 11 / 19 In MATLAB, the function polyfit performs polynomial regression Usage: a = polyfit(X,Y,N)

1 a - row vector of coeﬃcients of the regression, i.e. 2 X and Y - supplied data points 3 N - polynomial degree a(1)*X^N + a(2)*X^(N-1)+. . .+ a(N)*X + a(N+1) To evaluate the polynomial, a at any query points xq Y = polyval(a,xq)

Polynomial Regression

Deﬁne n n−1 f (x) = a1x + a2x + ··· + anx + an+1 that ﬁts the data.

Interpolation and Curve ﬁtting Spring 2019 12 / 19 Usage: a = polyfit(X,Y,N)

Polynomial Regression

Deﬁne n n−1 f (x) = a1x + a2x + ··· + anx + an+1 that ﬁts the data. In MATLAB, the function polyfit performs polynomial regression

Interpolation and Curve ﬁtting Spring 2019 12 / 19 Polynomial Regression

Deﬁne n n−1 f (x) = a1x + a2x + ··· + anx + an+1 that ﬁts the data. In MATLAB, the function polyfit performs polynomial regression Usage: a = polyfit(X,Y,N)

Interpolation and Curve ﬁtting Spring 2019 12 / 19 Linear ﬁt

Linear fit

6 y 5

1 1 2 3 4 5 6 7 8 9 10 x

Interpolation and Curve ﬁtting Spring 2019 13 / 19 Quadratic ﬁt

Quadratic fit

6 y 5

1 1 2 3 4 5 6 7 8 9 10 x

Interpolation and Curve ﬁtting Spring 2019 14 / 19 Exercise

Generate the following plot for x =0:5; and y=[0 10 25 36 52 59];

Linear fit Quadratic fit 70 70

60 60

50 50

40 40

y 30 y 30

20 20

10 10

0 0

-10 -10 0 1 2 3 4 5 0 1 2 3 4 5 x x

Fifth degree fit Tenth degree fit 70 140

60 120

50 100

40 80

y 30 y 60

20 40

10 20

0 0

-10 -20 0 1 2 3 4 5 0 1 2 3 4 5 x x

Interpolation and Curve ﬁtting Spring 2019 15 / 19 The default method is linear interpolation Other methods include 1 cubic 2 spline

Interpolation in 2D

Vq = interp2(X,Y,V,Xq,Yq,method)

Returns Vq, the values of a function of two variables at query points Xq,Yq

Interpolation and Curve ﬁtting Spring 2019 16 / 19 Other methods include 1 cubic 2 spline

Interpolation in 2D

Vq = interp2(X,Y,V,Xq,Yq,method)

Returns Vq, the values of a function of two variables at query points Xq,Yq The default method is linear interpolation

Interpolation and Curve ﬁtting Spring 2019 16 / 19 Interpolation in 2D

Vq = interp2(X,Y,V,Xq,Yq,method)

Returns Vq, the values of a function of two variables at query points Xq,Yq The default method is linear interpolation Other methods include 1 cubic 2 spline

Interpolation and Curve ﬁtting Spring 2019 16 / 19 Sampling and interpolating a Gaussian distribution

Z = peaks(X,Y) – evaluates the peaks function at X and Y. We can use Z as data points and visualize interpolation using various methods

Original sample Linear Interpolation using fine grid

10 10

5 5

0 0

-5 -5 5 5 4 4 0 2 0 2 0 0 -2 -2 -5 -4 -5 -4

Cubic Interpolation using fine grid Spline Interpolation using fine grid

10 10

5 5

0 0

-5 -5 5 5 4 4 0 2 0 2 0 0 -2 -2 -5 -4 -5 -4

Interpolation and Curve ﬁtting Spring 2019 17 / 19 Interpolation and image manipulation

Starting with a random image pic = rand(10,10); Interpolate the image using 64 times as many points in each direction.

Interpolation and Curve ﬁtting Spring 2019 18 / 19 Extrapolation

Deﬁnition Making estimates beyond the obsevation range. 1D – vq = interp1(x,v,xq,method,extrapolation) allows xq to contain points outside the range of x. 2D – q = interp2(X,Y,V,Xq,Yq,method,extrapval) allows for a scalar value extrapval to be assigned to all queries outside the domain of sample points.

Interpolation and Curve ﬁtting Spring 2019 19 / 19