Numerical Methods

Education: Mathematics MAT07 NUMERICAL METHODS

Dr. Ralph W.P. Masenge Numerical Methods

Foreword

The African Virtual University (AVU) is proud to participate in increasing access to education in African countries through the production of quality learning materials. We are also proud to contribute to global knowledge as our Open Educational Resources (OERs) are mostly accessed from outside the African continent. This module was prepared in collaboration with twenty one (21) African partner institutions which participated in the AVU Multinational Project I and II.

From 2005 to 2011, an ICT-integrated Teacher Education Program, funded by the African Development Bank, was developed and offered by 12 universities drawn from 10 countries which worked collaboratively to design, develop, and deliver their own Open Distance and e-Learning (ODeL) programs for teachers in Biology, Chemistry, Physics, Math, ICTs for teachers, and Teacher Education Professional Development. Four Bachelors of Education in mathematics and sciences were developed and peer-reviewed by African Subject Matter Experts (SMEs) from the participating institutions. A total of 73 modules were developed and translated to ensure availability in English, French and Portuguese making it a total of 219 modules. These modules have also been made available as Open Educational Resources (OER) on oer.avu.org, and have since then been accessed over 2 million times.

In 2012 a second phase of this project was launched to build on the existing teacher education modules, learning from the lessons of the existing teacher education program, reviewing the existing modules and creating new ones. This exercise was completed in 2017.

On behalf of the African Virtual University and our patron, our partner institutions, the African Development Bank, I invite you to use this module in your institution, for your own education, to share it as widely as possible, and to participate actively in the AVU communities of practice of your interest. We are committed to be on the frontline of developing and sharing open educational resources.

The African Virtual University (AVU) is a Pan African Intergovernmental Organization established by charter with the mandate of significantly increasing access to quality higher education and training through the innovative use of information communication technologies. A Charter, establishing the AVU as an Intergovernmental Organization, has been signed so far by nineteen (19) African Governments - Kenya, Senegal, Mauritania, Mali, Cote d’Ivoire, Tanzania, Mozambique, Democratic Republic of Congo, Benin, Ghana, Republic of Guinea, Burkina Faso, Niger, South Sudan, Sudan, The Gambia, Guinea-Bissau, Ethiopia and Cape Verde.

The following institutions participated in the teacher education program of the Multinational Project I: University of Nairobi – Kenya, Kyambogo University – Uganda, Open University of Tanzania, University of Zambia, University of Zimbabwe – Zimbabwe, Jimma University – Ethiopia, Amoud University - Somalia; Université Cheikh Anta Diop (UCAD)-Senegal, Université d’ Antananarivo – Madagascar, Universidade Pedagogica – Mozambique, East African University - Somalia, and University of Hargeisa - Somalia

2 The following institutions participated in the teacher education program of the Multinational Project II: University of Juba (UOJ) - South Sudan, University of The Gambia (UTG), University of Port Harcourt (UNIPORT) – Nigeria, Open University of Sudan (OUS) – Sudan, University of Education Winneba (UEW) – Ghana, University of Cape Verde (UniCV) – Cape Verde, Institut des Sciences (IDS) – Burkina Faso, Ecole Normale Supérieure (ENSUP) - Mali, Université Abdou Moumouni (UAM) -

Niger, Institut Supérieur Pédagogique de la Gombe (ISPG) – Democratic Republic of Congo and Escola Normal Superieur Tchicote – Guinea Bissau

Bakary Diallo

The Rector

African Virtual University

3 Numerical Methods

Production Credits

This second edition is the result of the revision of the first edition of this module. The informations provided below, at the exception of the name of the author of the first edition, refer to the second edition.

Author

Ralph W.P. Masenge

Reviewers

Nafy Aidara

Assiedu Addo Samuel

AVU - Academic Coordination

Dr. Marilena Cabral

Module Coordinator

Jeanine Ngalula

Instructional Designers

Elizabeth Mbasu

Diana Tuel

Benta Ochola

Media Team

Sidney McGregor Michal Abigael Koyier

Barry Savala Mercy Tabi Ojwang

Edwin Kiprono Josiah Mutsogu

Kelvin Muriithi Kefa Murimi

Victor Oluoch Otieno Gerisson Mulongo

4 Copyright Notice

This document is published under the conditions of the Creative Commons http://en.wikipedia.org/wiki/Creative_Commons

Attribution http://creativecommons.org/licenses/by/2.5/

Module Template is copyright African Virtual University licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. CC-BY, SA

Supported By

AVU Multinational Project II funded by the African Development Bank.

5 Numerical Methods

Table of Contents

Foreword 2

Production Credits 4

Supported By 5

Numerical Methods 8

Prerequisite Courses Or Knowledge. 8

Time . 8

Material. 8

Module Rationale. 8

Content. 9

Overview 9

Outline: Syllabus (include specific timings here if needed). 10

Graphic Organiser 11

Specific Learning Objectives. 12

Teaching And Learning Activities . 14

Pre-assessment 14

Solutions 18

Pedagogical Comment For Learners. 19

Learning Activities. 20

Types and Sources of Errors 20

Summary. 20

List of required readings. 21

Key Words. 21

Learning Activity 1: Types and Sources of Errors . 22

Introduction 22

Analytic Methods, Numerical Methods and Errors.. 22

Analytic versus numerical methods 22

Need for numerical methods 24

6 Errors 25

Precision and Accuracy 25

Types and Sources of Errors . 26

Methods of reducing errors . 27

How to reduce initial error 27

How to reduce discretization errors 28

How to reduce truncation errors 29

How to Reduce Rounding Errors 30

How to Reduce Rounding Errors 31

Rules for Rounding Numbers 33

Formative Evaluation: . 34

Solutions 35

Learning Activity 2: Interpolation 37

Summary. 37

Specific Learning objectives. 38

List of required readings. 38

Introduction . 39

Key Terms . 39

Learning activity 2: Meaning of Interpolation . 40

Linear interpolation. 41

Error in linear interpolation 42

Interpolation polynomials 42

Lagrangean interpolation polynomial 43

Newton’s divided differences . 44

Finite Difference Operators . 47

Finite Difference Interpolation Polynomials . 50

Tables of Differences. 51

Application Of Difference Tables In Interpolation 53

Application of Newton’s forward difference formula 54

Application of Stirling’s central difference formula 55

5 Numerical Methods

References for the Learning Activity 56

Solutions To Formative Evaluation Questions 56

Learning Activity 3. 59

Numerical Integration Summary 59

List of relevant readings. 59

Key Words. 60

Learning Activity: Numerical Integration . 60

Introduction 60

Need for Numerical Integration Methods 61

Classification of Numerical Integration Methods. . 61

Order of an integration method 61

Newton-Cotes Numerical Integration Methods 61

Error Terms in the Trapezoidal Rule and in Simpson’s Rule 65

Romberg Integration 65

3.7 Gaussian Integration Methods 68

References for the Learning Activity. 70

Learning Activity 4: Roots of Functions Summary 72

List of required readings. 72

Learning Activity : Roots of Functions. 73

Introduction 73

Key Words. 73

Roots or Zeros of a Function 74

Numerical Methods 74

Convergence of the Bisection Method. 75

4.4 Newton’s Method for a Coupled System 82

Reference for the Learning Activity . 89

Solutions To Formative Evaluation Questions 89

XI. Compiled List of all Key Concepts (Glossary). 92

Key Concepts 92

Key Theorems and Principles . 95

6 XIII. Compiled List of (Optional) Multimedia Resources . 96

XIV. Synthesis of the Module . 97

XV. Summative Evaluation. 99

SOLUTIONS 101

XVI. References. 109

XVII. Author of the Module. 110

7 Numerical Methods

Numerical Methods

Prerequisite Courses Or Knowledge

Calculus 2 is prerequisite.

Time

The total time for this module is 120 study hours.

Material

Students should have access to the core readings specified later. Also, they will need a computer to gain full access to the core readings. Additionally, students should be able to install the computer software wxMaxima and use it to practice algebraic concepts.

Module Rationale

A key attribute of mathematics is its applicability in problem solving. The history of the subject is full of evidence that the driving force in its early development was based in trying to solve problems in plane geometry, celestial mechanics and in navigation.

Unfortunately, mathematical formulations (models) of most problems in science and engineering are, in general, difficult to solve analytically either because of the com¬plex nature of the analytical solutions or because such solutions cannot be expressed in terms of combinations of known mathematical functions.

In all such cases, numerical methods have to be resorted to. The mathematics or science student is therefore expected to have a working knowledge of and ability to apply numerical methods in solving some basic mathematical problems such as interpolation, numerical integration and finding roots of functions.

Figure 3: Surfing on the Internet is a key to e-learning

8 Numerical Methods

Content

Overview

First Learning Activity: Types and Causes of Errors

The first learning activity aims at making the learner appreciate the need for numerical methods. It is also felt that this is the right moment to define the concept of a mathe¬matical error, point out the sources and types of errors and mention some practical ways of reducing their cumulative effect on the numerical solution.

Second Learning Activity: Interpolation

The second learning activity deals with the concept of interpolation. Both linear and higher order polynomial interpolation methods, based on Lagrange, Newton’s divided differences and finite difference interpolation techniques are presented.

Third Learning Activity: Numerical Integration

The third learning activity looks at the problem of numerical integration. The discus¬sion is limited to Newton-Cotes formulae. Specific attention is given to the Trapezoidal and Simpson’s rules and the application of Richardson’s extrapolation technique on both the Trapezoidal and Simpson’s rules in deriving Romberg integration schemes.

Fourth Learning Activity: Roots of functions

The fourth and final learning activity presents the root finding problem associated with solving the nonlinear equation f (x) = 0 and solving the coupled system of two nonlinear equations f (x, y) = 0, g(x, y) = 0 .

Figure 4: A Baobab Tree. Number of roots equals number of twigs

9 Numerical Methods

Outline: Syllabus (include specific timings here if needed).

This is a two unit course offered at Level 2 with priority rating A. Mathematics Module 3 is its prerequisite course.

The following is a detailed outline of the contents of each Learning Activity. The timings shown are suitable for students following a unit level course allowing 35 hours per unit. The timings show the length of time the Learner is recommended to spend learning each of its components.

[Students following a module level course should allow 120 hours for the complete module].

Preperation, taking and review for Pre-Assessment (4 hrs)

Learning Activity No.1: Types and Causes of Errors (25 hrs)

Types and sources of errors Need for numerical methods Error sources and types Strategies for reducing errors

Learning Activity No.2: Interpolation (35 hrs)

• Linear interpolation • Lagrangian interpolation • Newton’s divided differences • Finite difference operators • Finite difference tables • Finite difference interpolation polynomials

Learning Activity No.3: Numerical integration (25 hrs)

• Newton Cote’s formulae • Derivation of the trapezoidal and Simpson’s rules Romberg integration Gaussian quadrature formulae

Learning Activity No. 4: Roots of functions. (25 hrs)

• Bisection method • Convergence of the bisection method The Regula Falsi or False Position method Secant method Newton-Raphson’s method • Solving a coupled system of two nonlinear equations Fixed point iterations

Preperation and taking Summative Assessment (6 hrs)

10 Numerical Methods

Graphic Organiser

Figure 5: Graphic Organizer

General Objective(s)

At the end of this module:

You will be equipped with knowledge and understanding of the properties of elementary functions and their various applications necessary to confidently teach these subjects at the secondary school level.

You will have secure knowledge of the related contents of school mathematics to enable you confidently teach these subjects at the secondary school level.

You will acquire knowledge of and the ability to apply available ICT to improve the teaching and learning of school mathematics.

Specifically you will be able to:

1. Distinguish between numerical and analytical methods/solutions

2. Appreciate the need for learning and applying numerical methods

3. Identify the main sources of errors and take measures to eliminate or reduce such errors

4. Derive and apply a number of interpolation methods

5. Derive and apply a number of numerical integration methods

6. Derive and apply a number of numerical methods for finding roots of func¬tion

7. Solve a coupled system of two nonlinear equations in two variables.

11 Numerical Methods

Specific Learning Objectives

As already mentioned in the Module Outline given in Section VI, this is a two units module and its contents will be presented using four learning activities. Each learning activity has a set of specific learning objectives. We are stating them here in advance with a view to enable the learner have a global overview of what lies ahead in terms of what s/he will be able to do on completion of the module.

At the end of the module, the learner will be able to:

S/N Learning Activity Specific Learning Objectives

1 List the chief sources of computational errors and the practical steps to be taken to eliminate or reduce their cumulative effect on the numerical solution

Appreciate the difference between the size (accuracy) and the seriousness Types and sources of errors (precision) of an error.

Understand and apply linear interpolation on a given table of function values

Estimate the error in linear interpolation for a known smooth function

2 Derive, understand and apply the Trapezoidal rule, Simpson’s rule and any other Newton-Cotes type numerical integration formula Numerical Integration Derive, understand and apply Romberg’s numerical integration scheme based on either the Trapezoidal or Simpson’s rule.

12 Numerical Methods

3 Derive and apply the bisection method

Prove convergence of the bisection method

Derive, understand and apply both the Secant and the Regula Falsi methods Roots of functions Derive, understand and apply the Newton Raphson method

Derive, understand and apply the Newton’s method on a pair of nonlinear simultaneous equations.

4 Explain why numerical methods are essential in solving mathematical problems

Write down and apply the Lagrange interpolation polynomial for equally spaced data

Write down and apply Newton’s interpolation based on divided differences Interpolation Define and manipulate the shift, forward, backward, central and mean difference operators

Construct tables of differences for a given function or given data

Derive and apply Newton’s forward, Newton’s backward, and Stirling’s central difference interpolation polynomials.

13 Numerical Methods

Teaching And Learning Activities

Pre-assessment

1. If x is an exact measurement of a certain quantity and x0 is its approximation, the concept of error in the approximation is defined by

2. The absolute error in approximating the exact quantity x = 104 by the approxi¬mate value x0 = 107 is given by

The relative error in the approximation x0 given in Question 1 is

(a) 0.028846

(b) 0.299462

(d) 0.038462

3. One valid reason why the function

is not continuous at x = 0 is

14 Numerical Methods

4. The continuous function x3 - 3x - 3 must have a zero (root) at some point in the interval 2 < x < 3 because

5. The first derivative of the function

6. The truncated cubic Maclaurin series expansion of the

function is

7. The general anti-derivative of the function f (x) = ln(x) is given by

x ln( x) + C

ln(x) - x + C

x ln(x) - x + C

xln(x) + x + C

8. The value of the integral

15 Numerical Methods

9. Working throughout with six decimal places accuracy, the approximate value of using the Trapezoidal rule with h = 0.25 is

(a) 1.146850

(b) 1.416560

(d) 1.146580

10. Working under the same conditions, the approximate value of the integral given in Question 8 using Simpson’s rule is found to be

(a) 1.416039

(b) 1.146083

(d) 1.146580

11. If the 3rd and 13th terms of an arithmetic sequence of numbers are 7 and 27, respectively, then the 52nd element of the sequence is

(a) 103

(b) 105

(d) 106

12. Starting with x0 = 2, the value of x4 using the iteration formula is

(a) 2.104255

(b) 2.103731

(d) 2.103713

13. Given two points P (x0, y0) and Q (Xj, y), the x-coordinate of the point where the secant line AB cuts the x - axis is

16 Numerical Methods

14. The curve of a function y = f (x) passes through two points whose coordinates are (0.2,1.183) and (0.4,1.342). By using the straight line which joins the two points to approximate the curve of the function in the given interval, the approximate value of y at x = 0.3 is found to be

(a) y = 1.2265

(b) y = 1.6225

(d) y = 1.2625

15. Given a point P (x0, f (x0)) on the curve of a differentiable function y = f (x), the x coordinate of the point where the tangent at P cuts the x - axis is

16. The Bisection method is based on the principle that a continuous function which is positive at one point x = a and negative at another point x = b has a root (zero) at some point x = c in the interval a < c < b and that the point which bisects the interval [a, b], is a reasonable approximation of c. Starting with the two points a = 2, b = 3 , application of the bisection method on the function x3 - 3x - 3 gives the value after the third bisection process.

(a) x3 = 2.215

(b) x3 = 2.115

(d) x3 = 2.225

17. The Newton Raphson method for approximating a root of a function f (x) uses the iteration formula If x0 = 2 is an approximation of one of the roots of the function f (x) = x3 - 3x - 3, then application of the Newton-Raphson method on the function gives the value of x2 as:

(a) x2 = 2.130836

(b) x2 = 2.103386

(d) x2 = 2.103836

17 Numerical Methods

18. The coordinates of the point where the curves representing the functions x2 + y2 = 4 ; x2 - y2 = 1 intersect each other and which lies in the first qua¬drant are

(a) x = 1.224745, y = 1.561139

(b) x = 1.581139, y = 1.224745

(d) x = 1,242745,y = 1.561139

19. Starting with x0 = 1.5, y0 = 1.2, the values x3 and y3 obtained using the pair formulae are

(a) x3 = 1.562050, y3 = 1.322875

(b) x3 = 1.322875, y3 = 1.562050

(e) x3 = 1.233875, y3 = 1.562050

Solutions

1 C 11 B

2 B 12 B

3 D 13 C

4 B 14 D

5 A 15 A

6 D 16 C

7 D 17 D

8 C 18 B

9 C 19 C

10 D

18 Numerical Methods

Pedagogical Comment For Learners

The learner’s confidence to embark on this module will greatly be enhanced by solving these preassessment questions.

Questions 1,2 are on errors and 3 is on continuity. If you have problems in solving them you are advised to look up the definitions given in the Module Glosary.

Questions 4,5,6 and 7 are on the concepts of limits, continuity, differentiation and the anti- derivative. If you fail to solve any of them then work through the relevant Sections of the pre-requisite Module (Mathematics Module 3). The same tip holds if you experience difficulties in solving Questions 8,9,10 and 11 on integration.

Questions 12,13,16,17,18 and 19 relate to the concept of iterations and simula- tneous equations. Here, simply study carefully the given formulae and apply them faithfully.

Questions 15, 16 and 17 are on linear approximation of a function. In case one faces some problems in solving any of these, one should work through the relevant section in your Higher Level Mathmatics books.

Each correct solution has 5 marks, giving a maximum score of 90 marks. A score in the range 41 - 60 is average. A below average score (0 - 40) probably implies that you need a thorough go at the relevant prerequisite material before starting the mo¬dule. An average score may mean you can start the module but with frequent cross reference to some prerequisite materials. With an above average score (61 - 90) the learner should confidently embark on the module in the knowledge that he/she has the required background knowledge to start and successfully complete it.

19 Numerical Methods

Learning Activities

Learning Activity 1 Types and Sources of Errors

Summary

In this learning activity we discuss three important introductory topics. We begin by making a distinction between analytic and numerical solutions. This is followed by discussion of some typical mathematical problems specifically selected to convince the learner why numerical methods are needed. We conclude the activity by defining the concept of errors in computational mathematics, pointing out their main causes, types and presenting practical ways of eliminating or reducing their effect on nume¬rical solutions.

Specific Learning objectives

At the end of this learning activity the learner will be able to:

1. Define the concept of errors in computational mathematics

2. Distinguish between absolute and relative errors and appropriately relate them to the concepts of accuracy and precision

3. List the chief sources of computational errors and the practical steps to be taken to eliminate or reduce their cumulative effect on the numerical solution

4. Appreciate the difference between the size (accuracy) and the seriousness (precision) of an error.

5. Understand and apply linear interpolation on a given table of function va¬lues

6. Estimate the error in linear interpolation for a known smooth function

20 Types and Sources of Errors

List of required readings

Wikipedia: Numerical Methods/Errors Introduction

List of relevant useful links

Wolfram MathWorld (visited 03.04.07) http://mathworld.wolfram.com

Students should search for the entry covering the unit title. Also, search for any key words in the text. Mathworld gives a detailed reference in all cases.

Wikipedia (visited 03.04.07) http://en.wikipedia.org/wiki

MacTutor History of Mathematics (visited 03.03.07) http://www-history.mcs.standrews.ac.uk/ Indexes

The MacTutor Archive is the most comprehensive history of mathematics on the internet. Students should search for their unit title and read the history of the subject. This gives a helpful overview of the importance and context of the topic being studied.

Key Words

Error in an approximation: The difference between the exact and the approximate value.

Absolute error: The error without consideration of the sign (positive or negative).

Relative error: The ratio between the absolute error and the exact quantity.

Initial, discretization, truncation and rounding errors: Different types of errors caused by different sources of error.

21 Numerical Methods

Learning Activity 1: Types and Sources of Errors

Introduction

This first learning activity of the module is intended to provide the learner with answers to the following important questions; questions raised by many students taking a numerical methods course for the first time.

1. What is a numerical solution and how does such a solution differ from an exact (true) or analytical solution?

2. Why should one learn numerical methods? Are numerical methods needed?

3. What are errors in the context of mathematics? What are the main sources of computational errors? How can one eliminate errors or reduce their effect on numerical solutions?

We begin the activity by explaining the difference between an analytic and a numerical solution.

In order to establish the need for numerical methods we discuss a number of mathematical problems specifically selected with a view to convince the learner that there is a real need for learning numerical methods either because analytical solutions cannot be found or they are too complex to be of any practical use.

To answer the questions on errors we define the concept of an error and list a number of sources of errors. In the sequel we also suggest for each type and source of error practical steps to be taken to reduce the error and hence its impact on numerical solution.

Analytic Methods, Numerical Methods and Errors.

Analytic versus numerical methods

What is a numerical solution and how does such a solution differ from an exact (true) or analytical solution?

An analytic method for solving a given mathematical problem is any method based on rigorous mathematical analysis and whose application leads to the true (exact) solution, also known as analytic solution.

A numerical method for solving a given mathematical problem is any method based on rigorous mathematical analysis whose application, in most cases, can only lead to an approximate (non-exact) solution, also known as numerical solution. In some very rare cases, a numerical method may result in an exact solution.

22 Types and Sources of Errors

Example 1.1

The exact solutions of the nonlinear equation x2 - 5 x + 3 = 0 can be obtained using the well known quadratic formula (analytic method)

This gives the analytic solutions

On the other hand, the iteration formula (numerical method)

may also be applied to approximate one of the two solutions of the given quadratic equation. This method can only give an approximate numerical solution.

DO THIS

Given the numerical integration formulas

Trapezoidal rule:

Simpson’s rule:

1. The trapezoidal rule gives exact values of the integral for any

linear function f (x) = ax + b

2. Simpson’s rule gives exact values of the integral for any

cubic function :f (x) = ax 3 + bx2 + cx + d .^

In general, the difference between analytical and numerical solutions can be summed up by the statement: Analytical solutions are exact while numerical methods are only approximate.

23 Numerical Methods

Need for numerical methods

Why should one learn numerical methods? Are numerical methods needed?

Because of the above distinction between analytic and numerical solutions one can easily be tempted to conclude that one should only use analytic methods in solving mathematical problems. In other words, there is no need to learn numerical methods because they can only give approximate solutions. Such a conclusion is misguided. We need to learn numerical methods for the following three main reasons.

• For some problems the analytical solution may not be known. A typical examples is given by the following cases.

x 1 0 1 25 1 5 1 75 2

f ( x) 1 1.118 1.118 1.323 1.414

• An integral, such as is perfectly defined but the anti-derivative of the in

Tegrand f (x) = ex cannot be expressed using known mathematical functions.

• In some cases, it may be possible to find a mathematical expression for the analytical solution of a given problem. However, the expression may be com¬putationally too complicated to handle numerically. A typical problem is that of finding an anti-derivative of the function

After some tedious manipulations involving factorization of the denominator fol¬lowed by application of the method of partial fractions, one finds the general anti- derivative

where C is an arbitrary constant of integration. This complicated result makes the evaluation of a typical associated definite integral almost impossible to carry out with any meaningful degree of accuracy.

DO THIS

• Verify that (using the function F in example 3 above).

• By factorizing 8 - x3 = (2 - x)(x2 + 2x + 4) and applying the method of partial fractions derive the expression for F(x) as an anti-derivative of f (x) .

24 Types and Sources of Errors

Errors

What are errors in the context of mathematics? What are the main sources of com¬putational errors? How can one eliminate errors or reduce their effect on numerical solutions?

In the key words section you can find definitions of the key concepts and stated the key theorems and principles relevant to the topic of numerical methods. This includes the definitions of errors, absolute errors, relative errors, initial errors, discretization errors, truncation errors and rounding errors. For ease of reference we list them here.

Assumption

Let X * be an approximation to an exact (true) quantity X . Then,

The absolute error in X* is defined by X - X

The error in X* is defined by X - X

The percentage error in X* is defined by

Since the exact (true) value X is normally not known, one replaces it with the approximate value X * in the denominator X - X of the expression for the relative and percentage errors.

Precision and Accuracy

Measurements and calculations can be characterized with regard to their accuracy and precision.

Precision refers to how big or how small the absolute error is. The absolute error is therefore a measure of the precision of an approximation.

Accuracy refers to how closely the approximation X agrees with the true value X

Here, what counts is not only the magnitude of the deviation but its size relative to the true value X . Accuracy is therefore measured by the relative error

25 Numerical Methods

Types and Sources of Errors

We now list the sources and types of errors and briefly discuss methods of eliminating or reducing such errors so that the numerical solution we get is not seriously affected by them to the extent of rendering it meaningless.

Initial errors

Any mathematical problem meriting to be solved numerically involves some initial data. Such data may be in the form of coefficients in a mathematical expression or entries in a matrix. If this initial data is not exact, then the deviations from their respec¬tive true values are called initial errors. In some problems, uncertainties in the initial data can have devastating effect on the final numerical solution to the problem.

Discretization error

Most of the literature on the subject of computational errors does not make a distinc¬tion between discretization and truncation errors, the reason being that the two types of errors are almost inseparable. In this presentation we separate the two because truncation errors are special types of discretization errors.

The true (exact) solutions of some mathematical problems are continuous functions y = f (x) of their respective independent variables. In almost all cases, numerical methods for solving such problems approximate the unknown continuous solutionvf (x) by a sequence {f (xn )}of approximate values of the solution at a discrete set of points {xn } in the domain of the solution function f (x) . For example, the continuous function f (x) = x + e-x is the solution of the initial value problem y’ + y = 1 + x, y(0) = 1

A typical numerical method for solving this problem is given by the recurrence relation xo = 0, yo = 1, Yn = (1 - h)yn_1 + (1 + xn_1)h, n = 1,2,3,..., h being a constant distance between two consecutive discrete values of the variable x. The error resulting from such a discretization process is called discretization error.

Truncation error

Truncation errors are special types of discretization errors. The term truncation error refers to the error in a method, which occurs because some infinite process is stopped prematurely (truncated) to a fewer number of terms or iterations in the process.

Such errors are essentially algorithmic errors and one can predict the extent of the error that will occur in the method.

Specifically, the solution obtained using some numerical methods may involve infinite processes.

26 Types and Sources of Errors

For instance, this is the case with all convergent iteration methods and convergent infinite series. Since such infinite processes cannot be carried out indefi¬nitely, one is forced to stop (truncate) the process and hence accept an approximate solution. The error caused though this unavoidable termination of an infinite process is called a truncation error.

Rounding error

Rounding errors are errors introduced during numerical calculations due to the ina¬bility of calculating devices to perform exact arithmetic. For example, if we multiply two numbers, each with six decimal digits, the product will have twelve decimal digits. Unfortunately some calculating devices may not be able to display all twelve decimal digits. In such cases one is forced to work with fewer digits thereby necessitating dropping some of the (less significant) digits on the right of the product. The error so introduced is called a rounding error.

Methods of reducing errors

In the spirit of “prevention is better than cure” we shall attempt in this section to give practical suggestions of ways to eliminate or reduce the impact of various types of computational errors that are encountered in resorting to numerical methods.

How to reduce initial error

Initial errors can have a devastating effect on numerical solutions.

We illustrate a typical case involving an example taken from Francis Sheid, Nume¬rical Analysis, Shaum Outline Series, 1968page 342 involving the solution of the following two simultaneous linear equations. x - y = 1 x - 1.00001y = 0

The true (analytical) solution is x = 100,001, y = 100,000 . In this example, the set of initial data consists of the elements of the coefficient matrix and the right hand side vector

However, if the entry — 1.00001 in the matrix A is changed to — 0.99999 while all other data items remain unchanged, the resulting system of equations

has the drastically changed exact (analytical) solution x = — 99,999 , y = —100,000

This somewhat startling result demonstrates how a small change in the initial data can cause disproportionately large changes in the solution of some problems.

27 Numerical Methods

Thus, the only way to reducing or if possible, to eliminate initial errors is by ensuring that all data given with or computed for use in solving a problem is as accurate as is humanly possible.

How to reduce discretization errors

Different numerical methods for approximating the solution of a given mathematical problem can result in numerical solutions with very different degrees of accuracy due to the magnitudes of their respective discretization errors. Consider the problem of evaluating the definite integral:

The exact (analytical) value of the integral, correct to six decimal places is tan—1(1) = 0.785398.

Let us apply the trapezoidal method and Simpson’s rule using an interval length h = 0.25 . First we evaluate the integrand at the relevant points and get

i 0 1 2 3 4

x 0 0.25 0.5 0.75 1

f (x ) 1 0.941 0.8 0.64 0,500000

The trapezoidal rule

gives the numerical solution 0.782794 .

Simpson’s rule leads to the numerical solution 0.785392.

One observes that, while the solution obtained using the trapezoidal rule is correct to only two decimal places, the solution obtained using Simpson’s rule is correct to four decimal places. This significant difference in the accuracy ofthe two numerical solutions is caused by the differences in the discretization errors of the two nume¬rical methods. Simpson’s rule has a smaller discretization error than the trapezoidal rule.

In general, discretization errors cannot be avoided. However, one can reduce them substantially by being careful in selecting a numerical method whose discretization error is known a priori to be relatively small.

28 Types and Sources of Errors

How to reduce truncation errors

Truncation errors are caused by the unavoidable need to stop a convergent infinite process in efforts to get a solution. The size of the truncation error will therefore depend on the particular infinite process (numerical method) being used an on how far we are prepared to carry on with the infinite process.

The truncation error can be reduced either by

(a) Choosing a numerical method with a small truncation error or

(b) Carrying out the infinite process sufficiently far.

Example 1.2

The continuous function f (x) = x2 - 3x + 1 has a root which lies in the interval 0 < x < 1 (Why?). Using the quadratic formula, the exact value of the root correct to six decimal places is p = 0.381966. A number of iterative methods exist for approximating such a root. Here we consider two such methods:

The bisection method

The Newton-Raphson method

If one performs only three iterations (truncation after three iterations) with each method using the starting values x0 = 0 and x1 = 1 for the bisection method and x1 = 0 for the Newton- Raphson method, one gets the following sequence of ap¬proximations for each method.

Method Initial Values x x. x4

Bisection x0 = 0 x1 = 1 0.5 0.25 0.375

Newton x = 0 0.333 0.381 0.382 Raphson

These results demonstrate that in stopping the infinite process (iteration) after the third iteration, the truncation error of the Newton Raphson method is much smaller than that of the bisection method.

29 Numerical Methods

DO THIS

Continue applying the bisect method on the above example until the solution is correct to three decimal places. How many more iterations did this require?

How to Reduce Rounding Errors

Before we discuss this important last task in our learning activity we shall first intro¬duce a few terms that will frequently be mentioned and used in the process.

What are Figures or Digits

In computational mathematics, the words “figure” and “digit” are synonyms. They are used interchangeably to mean any one of the ten numerals in the set {0, 1, 2, 3, 4, 5, 6, 7, 8, 9}.

In the decimal system of real numbers, a number N is a string or an ordered se¬quence of figures or digits A typical example is the number

N = 00073920600365.00004507000

A number can be viewed as a measure of the size or magnitude of some real or ima¬ginary quantity. The position of each digit in the string of digits has direct bearing on the importance or significance of that digit (figure) in the overall measure of the size or magnitude of the quantity the number represents.

Intuitively we know that the leftmost digit 7 in the number N above is more signi¬ficant than the rightmost digit 7.

Which digits in a number are Significant?

The following rules apply in deciding which digits or figures in a given number are significant.

1. Nonzero integers are always significant figures.

2. Any zeros on the leftmost part of a number are not significant.

3. All zero digits positioned between nonzero digits are significant.

4. Zeros at the rightmost end of a number are counted as significant only if the number contains a decimal point.

30 Types and Sources of Errors

How many Significant Figures are in a given Number

The number of significant figures in a given number is found using the following rule:

Rule 1: The number of significant figures in a purely integer number (with no decimal digits) is obtained by counting, starting with the leftmost nonzero digit and ending with the rightmost nonzero digit.

Example 1.3

The number 541500409 has 9 significant figures.

The number 002507030 has 6 significant figures

Rule 2: The number of significant figures in a number having a decimal part is ob¬tained by counting all the digits, starting with the leftmost nonzero digit.

Example 1.4

The number 6.00213 has 6 significant figures.

The number 6.00213000 has 9 significant figures

NOTE: All zero digits at the end of a decimal number are significant.

How to Reduce Rounding Errors

Armed with the concepts of digits/figures and significant figures in a number we can now comfortably discuss ways of reducing rounding errors.

One obvious method of dealing with the problem of rounding errors is to work with the maximum allowable accuracy on our calculating device at each stage in our calculations.

Example 1.5

Find the sum of the numbers 2.35, 1.48, 4.24 using a calculating device that can only handle numbers with two significant figures.

The exact sum is S = 2.35 + 1.48 + 4.24 = 8.07

If we neglect the second decimal digit from each term and form their sum we find the approximate sum

51 = 2.3 + 1.4 + 4.2 = 7.9

The absolute error in S1 is: |S - SJ = 0.17 A better approximation of S under the same limitations is

52 = 2.4 + 1.5 + 4.2 = 8.1

The absolute error inS2 is: S - S 2 = 0.03 This error is significantly smaller than that in S1.

The immediate question expected to be raised by the learner is “How did one arrive at the two digit terms in the sum S2 ?”

31 Numerical Methods

The answer to the above question is simple. Each term has been obtained from its corresponding three-digit term by rounding.

The learner will soon know how to round numbers

What it means to round a number

To round a number to a fixed number of figures or digits simply means leaving out (dropping) all digits on the right hand side of the number beyond a certain posi¬tion.

If a number is rounded simply by dropping all digits beyond a certain position on the right hand side of the number without making any adjustments to the last retained digit, then one speaks of “rounding off or chopping the number”.

Example 1.6

The sum S1 has been calculated using terms obtained from the original numbers by rounding off (chopping) the third decimal digit from each term. The term 2.35 was rounded to 2.3, the term 1.48 was rounded to 1.4 and the term 4.24 was rounded to 4.2. In each case, the last retained digit (the first decimal place) has not been adjusted in the process of rounding.

Note

The sum S2 has also been obtained through rounding. However, the rounding this time is different. Here, not all the three terms have been rounded off!

The term 2.35 has been rounded to 2.4 The term 1.48 has been rounded to 1.5 The term 4.24 has been rounded to 4.2

We observe that in rounding each of the first two terms 2.35 and 1.48, the digit occu¬pying the second decimal position has been dropped but the digit occupying the first decimal position has been adjusted by increasing it by one (unity). The third digit 4.24 has simply been rounded off.

This practice (or as yet unknown rule for rounding numbers) seems to have some significant advantage over rounding off manifested by the above example in which

S2 is more accurate than S1

32 Types and Sources of Errors

Rules for Rounding Numbers

In order to reduce the error in rounding numbers, the rejection of digits beyond some predetermined position (n) is accompanied by making adjustments to the digit re¬tained in position (n-1). The adjustment involves either leaving the digit in position (n) unchanged or increasing it by one (unity). The decision to retain or increase by 1 the digit occupying position (n-1) is governed by the following rules.

a. If the digit in position (n + 1) is greater than 5 then the digit in position (n) is increased by 1.

b. If the digit in position (n + 1) is 5 and at least one other digit to its right is non zero then the digit in position (n) is increased by 1.

c. If the digit in position ( n + 1 ) is less than 5 then the digit in position ( n ) is left unchanged.

d. If the digit in position (n + 1) is 5 and all other digits to the right of position (n +1) zero, then

(i) The digit in position (n) is increased by 1 if it is an odd number (1,3,5,7,9);

(ii) The digit in position ( n ) is retained unchanged if it is an even number (0,2,4,6,8). Example1.7

Rounding a given number correct to two significant figures

S/N Number Rounded to Rule 2 Significant Used figures

1 8.361 8.4 (a)

2 8.351 8.4 (b)

3 8.35 8.4 (d) (i)

4 8.45 8.4 (d) (ii)

5 8.05 8 (d) (ii)

6 8.349 8.3 (c)

7 2.55 2.6 (d) (i)

8 2.65 2.6 (d) (ii)

9 0.056 0.056 (a)

10 0.055 0.055 (b)

33 Numerical Methods

Formative Evaluation:

Students should work through this exercise carefully writing full solutions for each problem. They should check their work thoroughly using the solutions provided.

Questions

1. (a) Using the method of substitution find the exact solution of the linear system of equations 5 x + 7 y = 12.075 7 x + 10 y = 16.905

(b) Round the values on the right hand side of each equation to two significant figures and then find the exact solution of the resulting system of linear equation.

(c) Use the solutions obtained from the two systems of equations to explain why initial errors need to be avoided as much as possible.

2. (a) How many significant figures are in each of the following numbers:

(i) 00001000020000

(ii) 10000200003004

(iii) 000123.0004500

(b) Round each of the following numbers correct to five significant figures.

(i) 0123.395

(ii) 0123.205

(iii) 0123.206

3. Given the quantity perform the following

calculations:

a. Find the exact value of X correct to five significant figures.

b. Approximate value of X using three digit chopping arithmetic (rounding without making any adjustments).

c. Approximate value of X using three digit rounding arithmetic.

d. Calculate the absolute errors and percentage errors in the approximations obtained in parts (b) and (c).

4. Assuming that the function f (x) has a single root p lying inside the close interval, a < x < b the bisection method for approximating p uses the iteration formula

34 Types and Sources of Errors

a. Prove by mathematical induction that the error in the ith iterate xi is given by

b. If a = 0 and b = 1, how many bisection iterations will be needed to obtain an approximation with an error not greater than 10-3 ?

Solutions

1. (a) x = 2.415, y = 0.

(b) x = 1, y = 1.

(c) Initial errors must be avoided because the solutions of some problems may be very sensitive to relatively small initial errors.

2. (a) (i) 6 (b) (i) 123.40

(ii) 14 (ii) 123.20

points x0 and x1, with x1 = x0 + h ,is given by E (x) = — (x - x0)(x - x1) f (^ )

(iii) 10 (iii) 123.2

3. (a) Exact value X = 0.45606

(b) Approximate value X b = 0.455 X

(d) Absolute error

Absolute error

Percentage errors

4. (a)Mathematical induction

Test : Formula is true for i = 1 because the error in the first bisection is

|b – a|

35 Numerical Methods

Assumption: Let the formula hold for i = k > 1 (fixed). This means

(b) If a = 0 and b = 1, then . This will not exceed 10 if .

36 Learning Activity 2: Interpolation

Learning Activity 2: Interpolation

Summary

The concept of interpolation is important in any introductory course on numerical methods. Numerical approximation addresses both the need to approximate numbers as well as approximating functions. Numerical interpolation approximates functions. We approximate functions for one or several of the following reasons:

• A large number of important mathematical functions may only be known through tables of their values. • Some functions may be known to exist but are computationally too complex to manipulate numerically. • Some functions may be known but the solution of the problem in which they appear may not have an obvious mathematical expression to work with.

Interpolation as it is presented in this Learning Activity will give the Learner an op¬portunity to experience first hand some practical applications of numerical methods in solving the mathematical problem of approximating a function f (x) which is known only through a set of values at a finite number of points.

This Learning Activity covers the following subtopics:

• Linear interpolation • Lagrange interpolation polynomial • Newton’s divided differences interpolation polynomial • Finite differences interpolation polynomials

We start off by defining the concept of interpolation. Linear interpolation is used to illustrate the concept. This is followed by presenting special interpolation polynomials. Specifically, we discuss interpolation polynomials based on Lagrangian interpola¬tion coefficients, Newton’s divided differences and on finite differences.

37 Numerical Methods

Specific Learning objectives

At the end of this Learning Activity the learner will be able to:

• Explain why numerical methods are needed in solving interpolation pro¬blems. • Apply Lagrangean interpolation polynomials. • Apply Newton’s divided differences interpolation polynomial. • Define and manipulate finite difference operators (shift, forward, backward, central and mean difference operators). • Construct tables of differences for tabulated function values. • Derive and apply Newton’s forward, Newton’s backward and Stirling’s central difference interpolation polynomials, and finally • Use finite difference interpolation polynomials in deriving some numerical integration methods.

List of required readings

Wikipedia: Interpolation

List of relevant useful links

Wolfram MathWorld (visited 03.04.07) http://mathworld.wolfram.com

Students should search for the entry covering the unit title. Also, search for any key words in the text. Mathworld gives a detailed reference in all cases. Wikipedia (visited 03.04.07) http://en.wikipedia.org/wiki

As with mathworld, students should search for the entry covering the unit title. Also, search for any key words in the text. Wikipedia generally gives shorter and less complete entries. However, they can be easier to read. MacTutor History of Mathematics (visited 03.03.07) http://www-history.mcs.standrews.ac.uk/Indexes

38 Learning Activity 2: Interpolation

Key Terms

Interpolation: approximation of a function using discrete values.

Interpolation polynomial: a polynomial which interpolates a function using discrete values provided.

Divided Differences: Differences of the function values provided, related to the differences between the discrete points provided.

Finite difference operators: Mathematical operators used in constructing the diffe¬rences in the function values.

Finite difference tables: A table showing different orders of the differences in the function values.

Introduction

How Many Learning Activities are in this Module?

As stated in the summary above, numerical methods are used both for approximating numbers as well as for approximating functions. The contents of this module are covered in four Learning Activities. Two of these Learning Activities are on methods of approximating numbers (the definite integral and roots of functions). Another Learning Activity is on errors and a fourth Learning Activity is on numerical methods for approximating functions.

Why are Polynomials Chosen to Approximate Functions?

Functions are approximated using other functions deemed to be simple to manipulate numerically. Specifically, one uses polynomials to approximate other complicated functions, mainly because polynomials are

• Simple to evaluate • Simple to differentiate, and • Simple to integrate.

We begin the presentation by defining the concept of interpolation and establishing the need for numerical methods for carrying out interpolation. This is followed by a detailed discussion of the simplest polynomial used in approximating functions, liner interpolation. Higher order interpolation polynomials are then introduced, including

39 Numerical Methods

• Lagrangian interpolation polynomials • Newton’s divided differences interpolation polynomials, and finally • Finite difference interpolation polynomials.

At the end of the Learning Activity we briefly discuss the possible use of interpolation polynomials in deriving some numerical integration methods such as the Trapezoidal rule and Simpson’s rule.

Learning activity 2: Meaning of Interpolation

The mathematical concept of interpolation is concerned with the problem of approxi¬mating a function f (x) defined over a closed interval [a, b].

The function f (x) to be approximated is usually defined through a set of its values at n + 1 distinct points lying in the interval [a, b]. Typically one is given the pairs of values

(xk, f (xk)); k = ^A...n; xo = a xn = b

The problem is then to find values (or even derivatives) of the function at some non tabular points lying inside the interval.

These types of problems are quite common in experimental physics and chemistry, where an algebraic expression for a function may not be known but its values for different values of its independent variable may be obtained experimentally through laboratory measurements.

Typical Example (Bajpai et al 1975 pp 205)

In an experiment carried out in a physics laboratory, the length y of a wire was measured for various loads x suspended from it. The results of the experiment are as tabulated hereunder:

Table 1: Load versus extension of a wire

x: Load (kg) 0 1 2 3 4 5

y : Length (mm) 2027.1 2029.4 2031.8 2034.1 2036.5 2039

From the data obtained above one may wish to know the length of the wire for any non tabulated load lying inside the range [0,5]. This is known as interpolating the function y = f (x), where the functional dependence of y on the load x is not explicitly given here. Any attempt to use the data to calculate the length of the wire for a load outside the given range is called extrapolation.

40 Learning Activity 2: Interpolation

Linear interpolation

In linear interpolation we assume that we have two points A( x0, f0) and B( x1, f ) on the curve of a (usually unknown) continuous function y = f (x) and that we now wish to approximate the value of the function at a point.

DO THIS

Revisit and learn the various forms of defining a straight line and writing the equation of the straight line for each form:

• Two-points given • A point and a slope is given

Because a straight line is completely defined by giving two points which lie on it, we approximate the function f (x) locally over the interval [x0, xj by the straight line through the two given points.

The equation of the straight line can be given in different forms. Here we consider three forms:

(i) The usual y-intercept - slope or Newton’s form.

(ii) The Lagrange form and

(iii) The determinant form

Worked Example 1

Using the data obtained from the laboratory experiment of suspending load from a wire and measuring its length, apply the determinant form of linear interpolation to approximate the length of the wire when the load is 2.7 kg

Solution which looks to be quite reasonable in that as may be expected, it is closer to f1 than it is to f0 .

In this example we substitute the values x0 = 2, f0 = 2031.8, x1 = 3, f1 = 2034.1, x = 2.7 into the determinant form of the equation and obtain the value

which looks to be quite reasonable in that as may be expected, it is closer to f1 than it is to f0 .

41 Numerical Methods

DO THIS (2.1)

Apply the other two forms of the linear interpolation function P1 (x) to approximate the length of the wire when the load is 1.35 kg

Error in linear interpolation

We now attempt to answer the question: How big is the error in linear interpola¬tion?

In other words, we wish to know how accurate the answers obtained through the process of linear interpolation are.

An important feature of interpolation polynomial Pn (x) of any degree is that Pn (xk) = f (xk) k = 0,1,2,...,n.

For linear interpolation this implies that P(x0) = f (x0) P(xj) = f (xj) .

This further implies that the error E (x) at any point X S [ X0, xj in linear interpo¬lation must have the form

E1(x) = (x - x)(x - x)C where C is a constant which is independent of x . In a subsequent section of this Lear¬ning Activity the value of C for a function f (x) which is assumed to be sufficiently differentiable is given as C = — f ) where M i n(x0, x1, x) <§ < Max(x0, x1, x) With this result, the error in linear interpolation assumes the form

Interpolation polynomials

As already stated at the beginning of the Learning Activity the main problem of polynomial interpolation may be stated as follows:

Given the values f (xk ) = fk of some continuous function f (x) at n +1 distinct points xk, k = 0,1,2,...,n one is required to construct a polynomial Pn(x) of degree n such that it satisfies the following co-location criteria Pn (xk ) = fk; k = 0,1,2,..., n

One can show that this so called “Interpolation polynomial” does exist and is unique. However there are different forms of representing the interpolation polynomial. In this Learning Activity we give four different forms of the interpolation polynomial.

42 Learning Activity 2: Interpolation

Lagrangean interpolation polynomial

The Lagrangean form of the interpolation polynomial has the general form

in which the terms Li (x) i = 0,1,2,..., n are individually polynomials of degree n in x called are called the Lagrangean interpolation coefficients.

To ensure that Pn (x) satisfies the co-location criteria given above, the Lagrangean interpolation coefficients are constructed such that they satisfy the condition

One can verify that the following definition of Lt (x) meets this equirement:r

With this definition of the Lagrangian interpolation coefficients one can now verify the co-location condition on the interpolation polynomial, for

Worked Example 2

Construct the cubic Lagrangean interpolation polynomial which interpolates the function f(x) given by the following table of values:

Table 2

i 0 1 2 3

1 2 3 4

f 1.54 0.58 0.01 0.35

P3( x) = 1.54 L 0( x) + 0.58 L,( x) + 0.01L 2( x) + 0.35 L 3( x) where

43 Numerical Methods

DO THIS (2.2)

Make use of the above Lagrangean interpolation polynomial to interpolate the func¬tion f (x) at x = 2.6 Observation

1. We note that the second form (ii) of the equation of a straight line given in the DO THIS (2.1) self-exercise problem is a typical example of the Lagrangean linear interpolation formula.

2. In changing from one degree Lagrangian interpolation polynomial to ano¬ther, say from linear to quadratic, all calculations must be done afresh. For instance, none of the expressions used in the linear approximation case (the linear Lagrangian coefficients L0 (x) L1 (x)) are useful in the construction of the quadratic Lagrangian interpolation polynomial. Because of the second observation, efforts have been made to construct interpola¬tion polynomials that are iterative in the sense that, a higher degree interpolation polynomial can be got simply by adding higher-degree terms to an existing lower degree interpolation polynomial. In what follows we shall present two interpolation polynomials in this category.

Newton’s divided differences

Let the function f (x) be given at n + 1 distinct points x0, x1v.., xn such that fk = f (xk ), k = 0,1,2,..., n

We define what are known asNewton’s divided differences

44 Learning Activity 2: Interpolation

A typical table of divided differences would look as follows

Table 3: Table of Divided Differences

Divided Differences

x f ( x) First Second Third Fourth

x0 f ( x0)

f [ ^ x1]

x1 f ( x1) f [x0 , x1, x2 ]

f [X1, x2] f [x0 , x1 , x2 , x3 ]

x2 f ( x2) f [x1, x2 , x3 ] f [x0 , x1, x2 , x3 , x4 ]

f [x2 , x3] f [x1 , x2 , x3 , x4 ]

x3 f ( x3) f [x2, x3, x4] f [x1 , x2 , x3 , x4, x5 ]

f [ x3 , x4] f [x2 , x3 , x4 , x5 ]

x4 f ( x4) f [x3 , x4 , x5 ]

f [ x4 , x5]

x5 f ( x5)

Using these divided differences Newton derived the following n-th degree po¬lynomial Pn (x) which interpolates the function f (x) at the n +1 distinct collocation points xk , k = 0,1,2,...,n : (Burden and Faires, 1989 p 113)

We shall illustrate this approach on the following example.

Worked Example 3

1. Construct a table of divided differences for the function f (x) given by the following data:

45 Numerical Methods

Table 4: Tabulated values of a function

x -3 -2 0 1 2 3

f(x) -17 -25 -13 -5 -1 23 115

2. Use the resulting table of differences to interpolate f (x) at: x = -2.3 using Newton’s linear, quadratic and cubic interpolation polynomials based at x0 = -3

Solution

Table 5: Table of divided differences

Divided Differences

x f ( x) First Second Third Fourth

-3 -17

-8

-2 -25 10

12 -4

-13 -2 1

8 0

0 -5 -2 1

4 4

1 10 1

24 8

2 23 34

3 115

Interpolation based on x = -3.0

Linear Interpolation

Here we take: x0 =-3.0, f (x0) = -17, f [x0, x1] = -8, x = -2.3 and we get f (-2.3) = f (x0) + (x - f [x0, x1] = -22.6

46 Learning Activity 2: Interpolation

Quadratic Interpolation

Because we are interpolating at the same point x = -2.3 we only need to consider the additional term (x - x0)(x - x1) f [x0, x1, x2] whose value at x =-2.3 is (0.7)(-0.3)(10) = -2.1

Adding this correction to the value obtained through linear interpolation we get f (-2.3) = -22.6 - 2.1 = -24.7

Cubic interpolation

Again we only need to calculate a correction to be made in the value obtained using quadra¬tic interpolation. The additional term is (x - x0)(x - x1)(x - x2) f [x0, x1, x2, x3] whose value is (0.7)(-0.3)(-1.3)(-4) = -1.092

Making this correction in the value obtained by quadratic interpolation gives f (-2.3) = -25.792

DO THIS (2.3)

Using the above table of divided differences

1. Write down the value of the divided difference f [x2, x3, x4 ] and show exactly how it has been arrived at.

2. Interpolate f (x) at x = -0.3 using a 4-th degree Newton’s divided differences interpolation polynomial based on the point x = -1.0.

Compare your answer with the value of the function f (x) = x4 + 2x3 - 3x2 + 4x - 5 at x = -0.3

Finite Difference Operators

The third and final set of interpolation polynomials are closely related to those based on Newton’s divided differences.

The basic assumption here is that,

The function values f (xk ) := fk are given at a number of equally spaced and distinct points having a constant interval length h = xk+1 - xk between consecutive collocation points.

A systematic introduction of the method of approach involves the introduction of some difference operators. There are four basic difference operators: The Shift, Forward, Backward and Central difference operators. These are defined as follows.

The Shift Operator E

The shift operator E is defined through the relation

Ef (x) = f (x + h)in the case of a continuous variable x, and through the relation

Ef k = f k+1 in the case of a discrete variable xk .

47 Numerical Methods

Powers of the operator (positive or negative) are defined in a similar manner: Epf (x) = f (x + ph); Epfk = fk+ p

The Forward Difference Operator v

The forward difference operator A is defined through the relation

Af (x) = f (x + h) - f (x) in the case of a continuous variable x, and through the relation

Af k = f k+1 - f k in the case of a discrete variable xk . (1)

Powers of the forward difference operator A can also be defined

Ap+1 fk = Apfk+1 -Apfk p = 0,12,...; A0 fr = fr for any value of r .

The Backward Difference Operator V

The backward difference operator V is defined through the relation in the case of a continuous variable x, and through the relation in the case of a discrete variable xk .

Powers of the backward difference operator can also be defined

for any value of r

The Central Difference Operator 6

The central difference operator 6 is defined through the relation 6f (x) = f (x +1 h) - f (x -1 h) in the case of a continuous variable x, and through the relation in the case of a discrete variable xk .

Powers of the central difference operator can also be defined

for any value of r .

Relationship between Difference Operators

The definition of powers of the Shift Operator E enables us to relate the other three difference operators with one another.

We first note that:

48 Learning Activity 2: Interpolation

Using these three results, all of which are based on the valid definition of the shift operator we now establish the following relationships:

Because

Again, because we conclude that

Similarly, because

we conclude that

In the case of the central difference operator TM it is not possible to express the shift operator E in terms of the central difference operator TM alone. However, we shall do so by involving the other finite difference operators. Specifically we can relate the forward and backward difference operators with one another by involving the shift and central difference operators.

1. Multiplying both sides of equation (7) with 2 E and taking note of the result in equation (5) we get

2. Multiplying both sides of equation (7) with 2E and taking note of the result in equation (6) we get

Having established all these inter-relationships between the operators E , TM one can now define any powers of the operators.

A last useful operator, the Mean (or averaging) Operator is often used in connection with some interpolation formulas that are based on the central difference operator TM.

The mean operator is defined through the relationship

49 Numerical Methods

The mean operator is defined through the relationship

Finite Difference Interpolation Polynomials

Armed with the five difference operators ( E, A, V, 5, ^) we are now ready to list (not derive) a number of interpolation polynomials based on their use. Exactly how they are used will be discussed after introduction of the concept of a table of dif¬ferences . Instead of operating with the variable x, such polynomials are usually written in terms of a digital variable 5 obtained from the transformation of variables

1. Newton’s Forward Difference Interpolation Polynomial

2. Backward Difference Interpolation Polynomial

3. Stirling’s Central Difference Interpolation Polynomial

4. Bessel’s Central Difference Interpolation Polynomial

50 Learning Activity 2: Interpolation

5. Everett’s Central Difference Interpolation Polynomials

Tables of Differences

All five finite difference interpolation polynomials given above contain forward or backward or central differences of powers exceeding one. It is therefore important to know how these quantities can be got. The answer is not difficult. These quantities are obtained from tables of differences constructed using the given values of the function f (x) at the equidistant collocation points x}

The tables of values are obtained in a process similar to that taken in constructing the Newton’s divided differences. The only difference here is that only differences of function values are involved. Such differences are not divided by any quantity.

The following are typical examples of tables of differences.

Table 6: Forward Difference Table

FORWARD DIFFERENCES

x f ( x) First Second Third Fourth Fifth

x0 f0

x1 f1 Af

Af, Af

X2 f2 A2f, Af

Af Af

x f3 a/2 Af

f Af

X4 f4 Af

51 Numerical Methods

X5 f5

Observation

Note the forward slopping nature of the entries based at any point xk in the table of forward differences. The Newton Forward Interpolation polynomial uses value along such a path

Table 7: Backward Difference Table

BACKWARD DIFFERENCES

x f ( x) First Second Third Fourth Fifth

xo fo

Vf,

x1 f1 Vf

Vf2 Vf

x2 f2 Vf V%

Vfs Vf Vf;

x3 f3 Vf Vf

Vf Vf

x4 f4 Vf

Vf;

x- f-

Observation

Note also the backward slopping nature of the entries based at any point xk in the table of backward differences. The Newton Backward Difference Interpolation Po¬lynomial uses values along such a path.

Table 8: Central Difference Table

CENTRAL DIFFERENCES

x f ( X) First Second Third Fourth Fifth

X-3 f-3

f *

52 Learning Activity 2: Interpolation

f. f S2f-2

f * Sf *

S f fl f f-i

Sf- * S3f- * f - *

X S4 0 fo fo

sf* S3f* S5f*

X S 1 fl Sfl fl

Sf * s3f *

X 2 f Sf

f *

Observation

Note further the horizontal slopping nature of the entries based at any point xk in the table of central differences. Stirling’s, Bessel’s and Everret’s formulas use table entries on and in the neighborhood of such a path.

DO THIS (2.4)

(i) In reference to Table 9 (Page 57) and taking xo = o.o give the values of the following finite differences:

2 2 4 1 Af3, V f1, 6f 1 , A f2, V f5, ^5f2 2

(ii) Using appropriate equivalences express 5 6 f 8 in terms of forward and backward differences.

Application Of Difference Tables In Interpolation

We are now ready to illustrate both how to construct a table of differences and also how to use it for interpolation purposes.

Worked Example 4

(i) Construct a table of differences for the function tabulated bellow.

x o o.2 o.4 o.6 o.8 1.o

f(x) 1.oooo o.98o1 o.9211 o.8253 o.6967 o.54o3

53 Numerical Methods

(ii) Use the resulting table to interpolate f(x)

• At x = o.1 using Newton’s forward difference formula • At x = o.9 using Newton’s backward difference formula and • At x = o.5 using Stirling’s central difference formula.

Solution

(i)

Table 9: Table of differences

Differences

x f(x) First Second Third Fourth Fifth

0 1

-0.020

0.2 0.980 -0.039

-0.059 0.002

0.4 0.921 -0.037 0.002

-0.096 0.004 -0.001

0.6 0.825 -0.033 0.001

-0.129 0.005

0.8 0.697 -0.028

-0.156

1 0.540

(ii) Interpolation

Application of Newton’s forward difference formula

We take X0 = 0.0 . Since the interval length being used is h = 0.2 the value of the

01 - 0.0 parameter s is calculated and found to be s

Newton’s forward difference formula is given by

54 Learning Activity 2: Interpolation

The table entries needed for this calculation are highlighted in the above table of differences based on X0 = 0.0 Substitution of these values and the calculated value of s gives the following result

f (0.1) = 1.0000 - 0.00995 + 0.0048875 + 0.00014375 - 0.00006640625 = 0.9950 Application of Newton’s backward difference formula

We take Xn = 1.0. Again since the interval length is h = 0.2 the value of the parameter s is calculated and found to be

Newton’s backward difference formula is given by

The table entries needed for this calculation are highlighted in the above table of differences based at xn = 1.0. Direct substitution of these values and the calculated value of s gives f (0.9) = 0.5403 + 0.0782 + 0.003475 - 0.0003125 - 0.0000390625 = 0.6216 rounded correct to four significant figures.

Application of Stirling’s central difference formula

We take x0 = 0.4 . Since the interval is h = 0.2 the value of the parameter 5 is calculated and found to be s = 05—04 = 0.5

Stirling’s central difference formula is given b

The table entries needed for this calculation are not highlighted in the above table but are relatively easy to pick up. For the inexperienced learner the teething problem may lie in finding the averaged values. To maximize transparency in the calculations we point out the following:

With this clarification we now perform the calculations and find f (0.5) = 0.9211 - 0.0387 - 0.0046 - 0.000196875 - 0.00001328125 = 0.8776

55 Numerical Methods

DO THIS (2.5)

Using the entries in Table 9 apply Everret’s central difference formula based on X0 = 0.4 to interpolate the function f ( X) at x = 0.45 .

References for the Learning Activity

A.C. Bajpai, I.M. Calus and J.A. Fairley, Numerical Methods for Engineers and

Scientists, Taylor & Francis Ltd., London 1975. R.L. Burden and D. Faires, Numerical Analysis, PWS-Kent Publishing Co. Boston, Fifth Edition 1989.

Solutions To Formative Evaluation Questions

DO THIS (2.1)

The load 1.35 kg lies between the loads 1 kg and 2 kg. We therefore take and subs¬titute into the two linear interpolation formulas:

DO THIS (2.2)

P3( x) = 1.54 L 0( x) + 0.58L1( x) + 0.01L 2( x) + 0.35L 3( x)

At x = 2.6 the values of the Lagrangean coefficients are found to be

Substitution into the expression for P3 (2.6) gives the resultP3 (2.6) = 0.15792 .

56 Learning Activity 2: Interpolation

DO THIS (2.3)

(i) f [ x2, x3, x4] = -2

This result is obtained from the definition of divided differences as follows:

(i) Interpolation of f(x)at x = -0.3 using Newton’s 4th degree divided differences interpolation polynomial.

Here x0 = -1.0

We then get the answer

P4(-0.3) = -13.0 + 5.6 + 0.42 +1.092 - 0.6279 = -6.5159

On the other hand, the function f (x) = x4 + 2x3 - 3x2 + 4x - 5 is found to have the same value (-6.5159) at x = - 0.3 .

DO THIS (2.4)

(i) One finds

(ii) This is solved using the equivalence relations One finds:

57 Numerical Methods

DO THIS (2.5)

Using the entries in Table 9 apply Everret’s central difference formula based on x0 = 0.4 to interpolate the function f (x) at x = 0.45 .

We are given the quantities x0 = 0.4; h = 0.2, x = 0.45 and therefore the two pa¬rameters required in the application of the Everret method are

58 Numerical Integration Summary

Learning Activity 3 Numerical Integration Summary In this learning activity we focus on numerical integration methods. A major portion of the presentation is devoted to the Newton-Cotes category of formulae. However, we also dwell on the more accurate family of numerical integration methods clas¬sified as Gaussian integration methods. Specific attention is given to the two best known Newton Cotes methods: Trapezoidal rule, Simpson’s rule and the application of Richardson’s extrapolation technique on both methods in deriving the Romberg integration method. As for Gaussian integration methods we limit our discussion to the Gaussian method based on Legendre polynomials. The Learning Activity is presented under the following headings:

• The need for numerical integration • Classification of methods • Newton-Cotes formulae • The Trapezoidal Rule • Simpson’s rule • Error terms in the Trapezoidal and Simpson’s rules • Romberg integration • Gaussian integration methods

At the end of this learning activity the learner will be able to:

1. Classify numerical integration methods

2. Derive and apply the Trapezoidal rule and Simpson’s rules.

3. Derive and apply Romberg’s integration method based on either the Tra¬pezoidal or Simpson’s rule

4. Apply the Gaussian integration method based on Legendre polynomials.

List of relevant readings

Fundamental Numerical Methods and Data Analysis, George W. Collins, chapter 4. Wikipedia: Numerical Methods/Numerical Integration

List of relevant useful links

Wolfram MathWorld (visited 03.04.07) http://mathworld.wolfram.com

Students should search for the entry covering the unit title. Also, search for any key words in the text. Mathworld gives a detailed reference in all cases.

59 Numerical Methods

Wikipedia (visited 03.04.07)

http:/n.wikipedia.org/wiki

MacTutor History of Mathematics (visited 03.03.07) http://www-history.mcs.standrews.ac.uk/ Indexes

Key Words

Anti-derivative of f (x)

A function F (x) is called an anti-derivative of another function f (x) if

Order of an integration method

A numerical integration method is said to be of order p if it gives exact values of the integral for all polynomial functions f (x) of degree m < p .

Learning Activity: Numerical Integration

Introduction

In this Learning Activity we give the features that distinguish the family of New¬ton-Cotes methods from the Gaussian integration methods. We use a geometrical approach to derive the Trapezoidal (Trapezium) rule and Simpson’s rule. Because at this juncture the Learner has not been exposed to the concept of orthogonal polyno¬mials, we have presented without deriving, the Gaussian integration method based on the Legendre orthogonal polynomials. Illustrative examples are given and solved to assist the learner experience the applications of the methods presented.

60 Numerical Integration Summary

Need for Numerical Integration Methods

In the first learning activity (Section 1.6 (b)) we gave two examples to demonstrate why we need to learn and be able to apply numerical methods in solving some ma¬thematical problems. Part (ii) of the first Example 1 and the whole of the second example were purposely chosen to illustrate the need to learn numerical methods for evaluating some definite integrals.

Classification of Numerical Integration Methods.

Numerical methods for approximating the definite integral have the general form a

The coefficients Wk are called weighting coefficients and xk are the abscissas or nodes taken from the range [a, b] of integration at which the integrand is to be evaluated.

Order of an integration method

A numerical integration method is said to be of order p if it gives exact values of the integral for all polynomial functions f (x) of degree m < p .

The general numerical integration formula given above has n + 1 nodes xk and n + 1 corresponding weights, giving a total of 2n + 2 unknown parameters. Since a polynomial of degree m is completely determined by giving its values at m + 1 distinct points, it follows that the order of the general numerical integration formula given above is at most 2n + 1.

Newton-Cotes Numerical Integration Methods

The Newton-Cotes integration methods are derived from the general formula by:

1. Requiring the n + 1 nodes xk to be uniformly distributed over the range of

in¬tegration [a, b] such that xk = x0 + kh, k = 0,1,2,...n, where x0 = a, xn =b and

2. Determining the n + 1 weights Wk such that the formula gives exact values of the integral for all polynomial functions f (x) of degree at most n . The Trapezoidal rule and Simpson’s rule fall in this category of methods.

(a) The Trapezoidal Rule

61 Numerical Methods

DO THIS (3.1)

(i) What kind of a geometrical figure is a trapezium?

(ii) What sides of a trapezium determine its area?

(iii) How is the area of a trapezium determined?

The Trapezoidal rule (sometimes referred to simply as the Trapezium rule) is the sim¬plest practical numerical integration method. It is based on the principle of finding the area of a trapezium. The principle behind the method is to replace the curve y = f (x) by a straight line (linear approximation) as shown in the figure 3.1.

Typically, we approximate the area A under the curve y = f (x) between the ordinates at x0 and x1 by , where f0 = f (x0) , f1 = f (x1) and h is the distance between x0 and x1 .

Figure 3.1: Trapezoidal Rule

For the integral the trapezoidal rule can also be applied by subdividing the interval [ a ,b] into n subintervals [ xk-1, xk ] , k = 1,2,3,...n , of equal length h = xk - xk_, with a = x0 and b = xn , followed by applying the trapezoidal rule over each subinterval. The area A under the curve y = f (x) between the ordinates at a = x0 and b = xn can then be approximated by the generalized Trapezoidal rule

Observation

In this generalized Trapezoidal Rule, the nodes are chosen to be the equidistant points xk = x0 + (k - 1)h, k = 1,2,3,..., n while the weighting coefficients Wk have been determined such that the formula gives exact value of the integral for all linear functions of the form y = f (x) = ax + b . These turn out to be:

62 Numerical Integration Summary

Example 3.1

Approximate by the Trapezoidal Rule with n = 10 (h= 0.1). x

Solution

In this example: f (x) = — and h = 0.1

We evaluate the function at the points, xi = 1 + (i - 1)0.1, i = 1,2,3,...10 and obtain the pairs of values:

Table 3.1: Value of the function

x f ( x) x f ( x)

1 1 1.6 0.625

1.1 0.909 1.7 0.588

1.2 0.833 1.8 0.556

1.3 0.769 1.9 0.526

1.4 0.714 2 0.5

1.5 0.667

Direct application of the generalized trapezoidal rule gives the approximate value

(b) Simpson’s Rule

To obtain Simpson’s rule we subdivide the interval [a,b] into only two equal subintervals using the points x0, x1, x2, where x2 - x, = x, - x0 = h, and replace the curve of the general function y = f (x) over the interval [ x0, x2 ] by the quadratic Lagrangean interpolation polynomial

63 Numerical Methods

The general definite integral is then approximated by the integral

.Without loss of generality, one takes x0 = 0 and gets the formula

Because the formula uses values at three points (two equal subintervals), a generalized Simpson’s Rule is only possible when the number n of subintervals is even:

[ Xo, X2], [ X2, X4], [ X4, X6], ..., [ Xn - 2, Xn ] .

One applies the formula over each subinterval and adds the results to get the generalized Simpson’s Rule formula

Example 3.1

Approximate using Simpson’s rule with n = io (h = o.1).

Solution

This is the same problem solved above using the Trapezoidal Rule. We certainly can apply Simpson’s Rule because the number of subintervals is even (n = 10). Using the values given in the Table 3.1 one obtains

DO THIS (3.2)

1. What is the exact value of the integral

2. Which of the two approximations of the integral obtained using the Trapezoidal rule and using Simpson’s rule is more accurate?

3. How is the accuracy of the approximations of either the Trapezoidal Rule or Simpson’s Rule affected by taking a smaller interval h (increasing the number of subintervals / refining the partitioning of the range of integration)?

64 Numerical Integration Summary

Error Terms in the Trapezoidal Rule and in Simpson’s Rule

With an exact value of o.69314718 which is the value ofln2, your answers to the other two questions are expected to have been that Simpson’s Rule is more accurate than the Trapezoidal Rule, and that the accuracy of both methods increases with decreasing interval length h . These results are made more vivid by the error terms of the respective methods.

1. Error Term in the Trapezoidal Rule

For reasonably behaved functions f (x) it can be shown (Fox and Mayers, 195 8) that the exact (true) value I ofthe integral Jf(x)dx is related to the value T (h)obtained by the Trapezoidal Rule with an interval of length h through the expression

I -T (h) = ATh2 + BTh4 + CTh6 +...

where AT, BT, CT,... are constants whose values are independent of the interval length h.

2. Error Term in Simpson’s Rule

Similarly, the error in Simpson’s approximation S(h) of the integral I is given by I - S(h) = ASh4 + BSh6 + CSh8 + ... where AS, BS, CS,... are again constants whose values do not depend on the interval length h.

The leading error term in the Trapezoidal Rule is AT h2 while that of the Simpson

Rule is AS h4. This explains why Simpson’s Rule is appreciably more accurate than the Trapezoidal Rule for the same interval length h .

Romberg Integration

Strictly speaking, Romberg integration is not a Newton Cotes method. Romberg integration is a post-processing technique. It is a method which uses previously computed values to produce more accurate results.

The method takes advantage of the knowledge of the error terms in either the Tra¬pezoidal Rule or the Simpson’s Rule to produce a much more accurate approximation of the integral using previously calculated approximate values.

(a) Romberg Integration Using the Trapezoidal Rule

Let — and h2 be two different intervals used with the Trapezoidal Rule. From the above form of the error term for the method we can write

I -T(h) = ATh12 + BTh4 + CTh6 +...

I -T(h2) = AT h22 + BTh24 + CTh26 +...

65 Numerical Methods

Eliminating the constant AT from these two formulae and solving for I we get

Specifically, if we choose (halving the interval) we get

The quantity

is an approximation of the integral I with a much smaller error than either of the two approximations T ( -) and T (h2). Its value is called the Romberg value of the integral with respect to the Trapezoidal Rule.

(b) Romberg Integration Using Simpson’s Rule

Derivation of a Romberg integration formula based on Simpson’s Rule follows the path as that used in connection with the Trapezoidal Rule.

Let - and h2 be two different intervals used with Simpson’s Rule. From the above form of the error term for the method we can write

I - S(hO = AS-2 + BS-4 + CS-6 + ...

I - S(h2) = Ash22 + Bsh24 + Csh26 + ...

Eliminating the constant AS from these two formulae and solving for I we get

Specifically, if we choose (halving the interval) we get

The quantity is an approximation of the integral I with a much smaller error than either of the two approximations S(-) and S(h2). Its value is called the Romberg value of the integral with respect to Simpson’s Rule

66 Numerical Integration Summary

Example 3.3

Approximate by Romberg integration based on the Trapezoidal Rule with h1 - 0.2 and h2 - 0.1

Solution

All the values required for the calculations are given in Table 3.1 One finds

The Romberg value we have obtained by combining two fairly inaccurate values using the Trapezoidal rule has the same high accuracy as the one obtained using Simpson’s Rule with h - 0.1

DO THIS (3.3)

(i) Is it possible to apply Simpson’s Rule on the integral using the interval h - 0.2 ?

(ii) If your answer is YES, apply the method. If your answer is NO explain carefully why not.

Example 3.4

Construct a table of values of the function f (x) = 1 at the equidistant points x xk = 1 + 0.125(k -1); k = 1,2,3,...,9 . Then approximate the integral applying Romberg integration based on Simpson’s Rule with interval lengths h1 = 0.25 and h2 = 0.125 .

67 Numerical Methods

Solution

Table 3.2: Value of the function

With these values one finds:

Using these two Simpson approximations of the integral, Romberg integration leads to the much more accurate value

. S(0.25,0.125) = S(0.125) + 1/15 [S(0.125) - S(0.25)]= 0.6931484

3.7 Gaussian Integration Methods

Gaussian integration methods are derived from the general integration formula by assuming that all the 2n + 2 parameters in the formula (the n + 1 nodes xk and their corresponding n + 1 weights Wk) are to be determined so that the resulting integration formula gives exact values for all poly¬nomial functions f (x) of degree at most 2n + 1.

For functions f (x) defined over the range [-1,1] Gauss developed a numerical integration formula

where the nodes are the roots of some special polynomial (Legendre orthogonal po¬lynomials) which are symmetrically positioned about the origin and all the weighting coefficients are positive.

The above mentioned restriction of the range of integration for the Gaussian integra¬tion methods is not a serious one because we can transform any finite interval [a, b] to [ -1,1] using the transformation

68 Numerical Integration Summary

and hence obtain the transformed integral on the right hand side of the equation

on which the Gaussian integration method can be applied.

Extensive tables giving values of the Gaussian nodes and their corresponding weight for different values of n have been constructed ready for use in solving any given definite integral over a finite range.

We reproduce here a table of the weights and nodes for the Gaussian integration method based on Legendre polynomials for n = 1,2,3,4,5.

Nodes and Weights for Gauss-Legendre Integration Method

Example 3.5

Approximate the integral by the Gaussian integration method using the Gauss-Legendre nodes and weights for n = 4.

69 Numerical Methods

Solution

With a = 2 and b = 4 the transformation of variables from x £ [2,4] to t £ [-1,1] gives: x = t + 3 and dx = dt

Direct calculations yield the following values:

W0 = 0.568889; F (0) = 0.1

W0 F (0) = 0.0568889:

Wj = 0.478629; F(+0.538469) = 0.073960; F (-0.538469) = 0.141660 Wj[F (0.538469) + F (-0.538469)]= 0.1032020

W2 = 0.236927; F(+0.906180) = 0.061507; F (-0.906180) = 0.185733

W2 [F (0.906180) + F (-0.906180)]= 0.0585778

Adding the three products together gives the value I = 0.218667

This is an astonishingly accurate result for, the true value of the integral is tan-1 (4) - tan-1 (2) = 1.325818 -1.107149 = 0.218669

DO THIS (3.4)

Approximate the integral f e-x cos(x)dx by the Gaussian integration method using the Gauss- Legendre-hodes and weights for n = 4 .

References for the Learning Activity

Fox, L. and Mayers, D.F., Computing Methods for Scientists and Engineers, - Oxford University Press, London (1958).

Solutions To Formative Evaluation Questions

DO THIS (3.1)

(i) A trapezium is a quadrilateral which has a pair of parallel sides.

(ii) The sides of a trapezium that determine its area are the pair of parallel sides.

(iii) The area of a trapezium is given by: Area A = h/2[Lx + L2 ], where Ll, L2

are the lengths of the parallel sides and — is the perpendicular distance between them.

70 Numerical Integration Summary

DO THIS (3.2)

(i) The exact value of the integral

(ii) The approximation obtained using Simpson’s rule is more accurate (has a smaller error) than that obtained using the trapezoidal (trapezium) rule.

(iii) The accuracy of both the trapezoidal rule and Simpson’s rule become more accurate as the interval length — gets smaller.

DO THIS (3.3)

(i) It is not possible to apply Simpson’s rule on the integral using the interval h= 0.2.

(ii) The reason why it is not possible is because with h= 0.2 the number of subintervals over the range [1,2] of integration would be odd (5) while Simpson’s rule can only be applied when the number of subintervals is even.

DO THIS (3.4)

The integral is first transformed into using the change of variables:

One uses the following nodes and weighting coefficients

W1 = 0.652145, t1 = 0.339981, -11 =-0.339981

W2 = 0.347855, t2 = 0.861136, -12 = -0.861136

One finds

71 Numerical Methods

Learning Activity 4: Roots of Functions Summary

This is the fourth and final learning activity for this module. In this learning activity we shall discuss a frequently occurring problem in mathematics: the root finding problem for either a nonlinear equation f (x) = 0 involving a single independent variable x or for a coupled system of two nonlinear equations (x, y) = 0, g (x, y) = 0 in two independent variables (x, y).

At the end of this learning activity the learner is expected to be able to derive, apply and prove that the bisection method always converges; derive and apply both the Secant method and the Regula Falsi method; derive and apply the Newton Raphson method and also derive and apply Newton’s method for a coupled system of nonlinear equations. At the end of our presentation we discuss briefly the concept offixed points of a function and the theorems which guarantee their existence and uniqueness and relate these results to the root finding problem in a manner which allows the learner to derive her/his own convergent iteration methods.

List of required readings

Wikipedia: Numerical Methods/Equation Solving

List of relevant useful links

Wolfram MathWorld (visited 03.04.07) http://mathworld.wolfram.com

Students should search for the entry covering the unit title. Also, search for any key words in the text. Mathworld gives a detailed reference in all cases.

Wikipedia (visited 03.04.07) http://en.wikipedia.org/wiki

MacTutor History of Mathematics (visited 03.03.07) http://www-history.mcs.standrews.ac.uk/ Indexes

72 Learning Activity 4: Roots of Functions Summary

Key Words

Root or zero of a function: A value of x at which the function has a value of zero.

Fixed-points of a function: A value of x for which the function has the same value of x (e.g. f2)=2)

Intermediate value theorem for continuous functions: A continuous function takes all values lying between two values of the function.

Learning Activity : Roots of Functions

Introduction

Five numerical methods for finding roots of functions will be presented in this learning activity. The first four methods are for solving the nonlinear equation f (x) = 0 The fifth method is for solving a coupled system of two nonlinear equations in two variables f (x, y) = 0, g (x, y) = 0.

The bisection method will be derived. We prove that the method is always convergent. This will be followed by the method of Regula Falsi, which has a significant similarity with the bisection method but converges slightly faster. The Secant method is immediately presented after the Regula Falsi method because the two methods share a common mathematical formula. However, the Secant method is computationally more efficient. The Newton-Raphson method is presented last and is shown to be a generalized form of the Secant and Regula Falsi methods.

73 Numerical Methods

Roots or Zeros of a Function

For a function of a single independent variable y = f (x), a point x = p is called a root or a zero of f (x) if the value of the function is zero at the point, meaning. f (p) = 0 In Figure 4.1 the points x = x1, x = x2, x = x3 are all roots of the function f (x)

Figure 4.1: Roots of a Function f (x)

Numerical Methods

(a) The Bisection Method

The bisection method for approximating roots of functions is a typical example of an iteration method.

In its simplest form, an iteration method may be defined as a repetitive process of applying a function g (x) on one or several previous approximate values xn-1 ... to produce a new and hopefully more accurate approximation xn of the specific quantity being sought. In mathematical symbolism we write

The bisection method is based on the intermediate value theorem for continuous functions. If f (x)is continuous on an interval [a, b] and if the values f (a) and f (b) differ in sign, (f (a) f (b) < 0), then the equation f (x) = 0 has at least one real root p G [ a, b].

Assuming that the points a and b were chosen to contain only one root we can bisect the interval [a, b] into two halves at the point c = (a + b) and conclude that the root lies

74 Learning Activity 4: Roots of Functions Summary

either in the interval (a, c) or in the interval (c, b) provided that f (c) ^ 0 , in which case c is the required root.

The Bisection method repeats the process of bisecting the interval which contains the root p until we are satisfied that we are close enough to the root.

The above process can be summed up by the following algorithm (Kendal E. Atkinson, 1989 pp 56).

Algorithm “Bisect (f ( x), a, b, P, 8 )”

Steps:

1. Set x1 := a and x2 := b

2. Define x3 = 1 [xj + x2 ]

3. If x2 - x3 <8 , then accept p = x3 and exit.

4. If f (x2) f (x3) < 0 then ^ := x3; otherwise x2 := x3.

5. Go back to step 2.

Convergence of the Bisection Method

Convergence of any iteration method implies that the error in the approximation is tending to zero as the number of iteration increases.

For the Bisection method, the absolute value of the error is bounded by the length of the interval in which the root lies at that particular stage.

Error bound after 1st bisection

Error after 2nd bisection

Error after 3rd bisection

Error after n bisection

Because we conclude that the bisection method always converges.

75 Numerical Methods

Example 4.1

Verify that the function f (x) = X2 + 4 X - 10 has a root inside the interval (1,2) and use the limits of the interval as starting values of the bisection method to approximate the root in 10 bisections.

Solution

We evaluate the function at the two end points of the given interval and find f (1) = - 5 and f (2) = 2. Since f (x) is a continuous function and f (1) f (2) < 0, the intermediate value theorem asserts that f has at least one root in the interval 1 < x < 2. We can therefore carry out the bisection method to find the results tabulated hereunder.

Table 4.1: Bisection Method for the function f (x) = x2 + 4x -10

n a f (a) b f (b) c f (c)

1 -5 2 2 1.5 -1.75

1 1.5 -1.75 2 1.5 1.75 0.0625

2 1.5 -1.75 1.75 0.0625 1.625 -0.859375

3 1.625 -0.859375 1.75 0.0625 1.6875 -0.402344

4 1.6875 -0.402344 1.75 0.0625 1.71875 -0.170898

5 1.71875 -.170898 1.75 0.0625 1.734375 -0.054443

6 1.734375 -0.054443 1.75 0.0625 1.742188 0.003971

7 1.734375 -0.054443 1.742188 0.003971 1.738282 -0.025248

8 1.738282 -0.025248 1.742188 0.003971 1.740235 -0.010642

9 1.740235 -0.010642 1.742188 0.003971 1.741212 -0.003333

10 1.741212 -0.003333 1.742188 0.003971 1.741700 0.000319

The sequence of values under the c-column in the table is convergent. The true value of the root being approached by this sequence (use the quadratic formula) is

p = 1.741657 ...

76 Learning Activity 4: Roots of Functions Summary

DO THIS (4.1)

Verify that the function f (x) = x - cos(x) has a root in the interval [0,1] and hence apply the bisection method, in only five iterations, to approximate the root.

(b) The Regula Falsi Method

The Bisection method we have just presented was a bit wasteful. In it one spends great efforts at calculating values of the function f at two points which are only used for deciding in which subinterval the root lies but are never used in the actual calculation of the approximate value of the root.

The method known as Regula Falsi (or rule of false position) rectifies this anomaly. The method retains the principle of enclosure characteristic of the Bisection method but also makes direct use of the function values at the two points which enclose the root of the function.

The general approach for this and subsequent methods (Secant method and the Newton

Raphson method) is to replace the curve of y = f (x) in the interval where the root lies by the straight line joining the two points.

Specifically, let the root p lie in the interval [ x1, x2 ]. The equation of the secant line joining the two points A( x1, f1) and B( x2, f2) is

Figure 4.2 Regula Falsi Method

77 Numerical Methods

This line intersects the x-axis at the point with coordinates ( x3,0)where

Unlike the Bisection method, the Regula Falsi assumes that the two values x1 and x2 enclose the root being sought by requiring that f1 f2 < 0 and at the same time involves the same function values in calculating a new approximation to the root.

The above process can be repeated several times in an iterative process using the following algorithm.

Algorithm “Regula Falsi ( f (x), a, a, b, P, å )

Example 4.2 x2 : = x3 .

Starting with the values x1 = 1, x2 = 2 apply the Regula Falsi method on the function f (x) = x2 + 4 x -10 to obtain an approximate value of the root enclosed in the interval (x1, x2) in only four (4) iterations.

Table 4.2: The Regula Falsi Method for the function f (x) = x2 + 4x -10

n X1 f (X1) x 2 f (x 2) f (X3)

1 -5 2 2 1.714286 -0.204080

1 1.714286 -0.204080 2 2 1.740741 -0.006857

2 1.740741 -0.006857 2 2 1.741627 -0.000227

3 1.741627 -0.000227 2 2 1.741656 -0.000010

4 1.741656 -0.000010 2 2 1.741657 -0.000003

78 Learning Activity 4: Roots of Functions Summary

Examination of the values of x3 against the background of the exact value of the root p = 1.741657387... demonstrates the faster convergence of the Regula Falsi method compared to the Bisection method.

Observation

The requirement in both the Bisection and the Regula Falsi methods that the two values involved in the calculations, x1, x2 must enclose the root is computationally very restrictive and significantly reduce the efficiency of both methods. In a programmed algorithm, the process of checking whether or not f (x:) f (x2) < 0 is time consuming. As a result, efforts have been made to do without it. Our next method achieves just that.

The Secant method is essentially the same as the Regula Falsi method. The only difference is that in the Secant method the requirement that the two values x: and x2 must enclose the root is dropped. All one needs is to require the two values used in the computation be close enough to the required root. The algorithm for the Secant method is as given hereunder.

Algorithm “Secant Method ( f (x), a, b, p, 8 )”

DO THIS (4.2)

Starting with x0 = 0, x1 = 1, perform 5 iterations using the secant method to approximate the root of the function f (x) = X - cos(x) .

79 Numerical Methods

(d) The Newton-Raphson Method

The Newton Raphson method is by far the most popular numerical method for approximating roots of functions. The method assumes that the function f (x) is differentiable in the neighborhood of the root and that the derivative is not zero anywhere in that neighborhood.

Assuming that x0 is a point which is sufficiently close to the root of the function, the graph of the function y = f (x) is approximated by the tangent to the curve at the point. y

Tangent

Fig ure 4.3: Newton -Rap hson M ethod

The equation of the tangent through point (x0, f (x0) on the curve y = f (x) is y = f0 + (x - x0) f/(x0)

This tangent intersects with the x - axis at the point x1 whose value is

The value x1 is then accepted as a new approximation to the root. The point (x1, f (x1) can be taken as a new point to draw a tangent through. Its intersection with the x - axis, given by is also accepted as a new approximation to the root. This process can be repeated over and over, leading to the iteration method given by the

Newton-Raphson formula

80 Learning Activity 4: Roots of Functions Summary

Each iteration using the Newton Raphson method requires one function evaluation and one first derive evaluation. Compared to the previous three numerical methods, the Newton Raphson method converges very rapidly to the root.

Example 4.3

Starting with x0 = 1 approximate a root of the function f (x) = x2 + 4 x - 10 correct to 6 decimal places.

Solution f (x) = x2 + 4 x -10 = (x + 4) x -10 f’(x) = 2 x + 4

The requirement that we obtain an answer correct to 6 decimal places simply means that we carry out the iteration until the 7th decimal digit in the values being calculated is no longer changing.

Table 4.3: The Newton-Raphson Method for the function f (x) = x2 + 4x -10

n xn f ( xn ) f’(xn)

(x + 4)x - 10 2 xn + 4

v n ‘ n

0 1 -5 6 1.833 333 3

1 1.833 333 3 0.6944442 7.6666666 1.742 753 6

2 1.742 753 6 0.0082045 7.4855072 1.741 657 5

3 1.741 657 5 0 7.483315 1.741 657 5

Because the value of the function at x3 is for all purposes zero, we can conclude that the required approximation is x = 1.741657 rounded to six decimal places. The true value of the root to the same degree of accuracy is p = 1.741657

DO THIS (4.3)

Starting with x0 = 0 apply the Newton-Raphson method in only four iterations to approximate the root of the function f (x) = x - cos(x).

81 Numerical Methods

4.4 Newton’s Method for a Coupled System

Our fifth and last numerical method will focus on solving a system of two simultaneous nonlinear equations of the general form f (x, y) = 0, g (x, y) = 0.

A typical example of such a system is to find the coordinates of the point in the first quadrant where the parabola y = 2x2 - 7 intersects with the circle x2 + y2 = 6 .

Here, we are looking for the pairs of values (x, y) which satisfy the two nonlinear equations f (x, y) = 2x2 - y - 7 = 0, g(x, y) = x2 + y2 - 6 = 0

To obtain a Newton-Raphson type method for approximating the solution of the general problem stated above, we let (x0, y0) be an approximation to the exact solution

(a, p) of the coupled system. To obtain an improved solution (xj, yj ) we assume that the exact coordinates of the point are obtained by making adjustments h and k to our initial values, such that a = x0 + h ; p = y0 + k . and hence, f (x0 + h, y0 + k) = 0 g(x0 + h, y0 + k) = 0 .

Expanding f and g in a Taylor Series about the point (x0, y0) one gets

where we have deliberately left out all higher order terms and retained only the terms linear in the increments h and k .

If we truncate the series on the right hand side of each equation after the linear terms we still get

but . We denote the values of x and y which satisfy the above pair of equations with

X1 = Xo + h ; y1 = yo + k.

82 Learning Activity 4: Roots of Functions Summary

Solving the resulting system of two linear equations:

for the unknown value h and k

The values of h and k are found to be

where h k D, D , D are the three determinants

in which all the quantities (function values and partial derivatives) which appear therein are evaluated at the point (x0, y0)

Once these quantities have been calculated, the values of the new approximate solution ( x1 , y1 ) can be calculated .

The above analysis carried out using the initial approximate solution (x1, yj) can now be repeated using the new pair(Xj,y 2)to lead to a new approximate solution(x2,y2), and so on, and so on in an obvious iterative manner,

The method described above is known as Newton’s method for a system of simultaneous nonlinear equations. Its speed of convergence is the same as its counterpart in solving the single nonlinear equation f (x) = 0 .

Example 4.4

Using analytical methods find the true solutions of the coupled system of equations f (x, y) s 2x2 - y - 7 = 0, g(x, y) = x2 + y2 - 6 = 0

Perform two iterations with Newton’s method to approximate a solution of the coupled pair of equations which is believed to lie close to the point (2,1) .

83 Numerical Methods

Solution

Solving the equation g(x,) = 0 for x2 one gets x2 = 6 - y2, and substituting this expression of x2 into the equation f (x, y) = 0 one gets 2[ - y2 J- y - 7 = 0 This leads to the quadratic equation 2 y + y - 5 = 0

Whose roots are y(1) = 1.350781, y(2) =-1.850781

The corresponding values for x are found to be

First Iteration

Second Iteration xj = 2.05, y1 = 1.4

84 Learning Activity 4: Roots of Functions Summary

DO THIS (4.4)

Perform 2 iterations with Newton’s method to approximate the solution of the coupled system of equations x - x2 - y2 = 0 that lies near the point with coordinates (0.8,0.4). y - x2 + y2 = 0

(4.5) Fixed-Point Iterations

Definition

A number p is called a fixed point of a function g(x) if g (p) = p .

The mathematical problem of finding values of x which satisfy the equation x = g (x) is called the fixed point problem.

(a) Existence Theorem

If g(x) is continuous over[a, b] and g (x) e [a, b] Vx e [a, b], then g(x)has at least one fixed point in [a, b]

To prove this theorem we need the Intermediate Value Theorem for continuous functions.

Proof

If g(a) = a or g(b) = b then the proof is complete because then, a or b , or both are fixed points ofg(x) . However, if g (a) ^ a and g (b) ^ b then from the assumption g (x) e [a, b] made above, the function g(x) satisfies a < g(x) < b . We define a function h(x) = g(x) - x for x e [a, b] . Like g(x), the function h(x) is also continuous over[a, b]. By evaluating h(x) at x = a and x = b we find h(a) = g(a) - a > 0, h(b) = g(b) - b < 0 .

The intermediate value theorem asserts that the function h(x) must vanish (have zero value) at an intermediate point p e (a, b). At such a point x = p we have, h( p ) = g( p ) -p = 0 which implies g (p) = p a result which implies that g (x) has a fixed point p e (a, b).

85 Numerical Methods

(b) Uniqueness Theorem

The above theorem establishes the existence of at least one fixed point. There could therefore be several fixed points. To guarantee that there is only one fixed point we have the following theorem.

If in addition to the assumptions made above, the function g (x) is differentiable in (a, b) and its derivative satisfies the condition

Then, g (x) has a unique fixed point in (a, b).

Proof

Assume that g(x) has two different fixed points: p 1, p 2 with p 1 ^ p 2 . Then, g (p 1) = p 1 and g (p 2) = p 2.

Through subtraction and using the Mean Value Theorem for differentiation we get

By taking absolute values on both sides of this equation and observing the above condition on g1 (x) we find

This is a contradiction which can only result from the assumption we made that We therefore conclude that under the stated assumptions, the function g (x) has a unique (only one) fixed point.

We introduced the concept of a fixed point for the purpose of using it in solving our root- finding problem. The relationship between the two problems is simple.

In the root-finding problem we want to find all values of x which satisfy the equation f (x) = 0.

Let us assume that we are able to express (split) the function f (x) in the form f (x) = x - g (x).

It is obvious that this splitting can be done in many ways. The particular choice of g (x) will be critical in converting the root-finding problem into a useful fixed-point problem.

If p is a root (zero) of f (x) then

86 Learning Activity 4: Roots of Functions Summary

This result implies that the roots of f(x) are the fixed points of g(x) .

What we now need to do is to find the best splitting of the function f(x) .

Such a splitting must result in a function g (x) which has a unique fixed point in a given interval. We shall demonstrate the splitting process on a typical example.

Example 4.5

Consider the root-finding problem f (x) = x2 + 4 x -10 = 0. Using the Intermediate Value Theorem we can show that f (x)has roots in the intervals [-6,-5] , [1,2] For our purposes we shall be interested in finding the root that lies in [1,2].

The equation x2 + 4 x -10 = 0 can be written in many different ways. The following are but a few alternate ways of expressing the equation.

1. x2 = 10 - 4 x

2. x( x + 4) = 10

3. 4 x = 10 - x2

We solve each of these three equations for x and get

These are three possible fixed-point problems for the given root-finding problem given above. The question is which one of these fixed-point problems is most suited for solving the root- finding problem?

To answer this question we find which of the three fixed-point functions g1 (x) g2 (x) g3 (x) satisfies the criteria stated by the uniqueness theorem with respect to the root in [1,2]. Using direct substitution we find that

From the above results we conclude that, only g 2 (x) satisfies the existence criteria.

87 Numerical Methods

Questions

Why do the other two functions fail the test? Does g 2 (x) satisfy the uniqueness criteria?

The Learner is expected to answer the first question by checking whether or not g1 (x), g 3 (x) meet the existence criterion for possessing a fixed point in [1,2]. oT answer the second question we note that

Since we conclude that g2 (x) has a unique fixed point p G (1,2) which is automatically a root of the function f (x) . Approximation of the root

The root p can be approximated iteratively using the iteration

for any starting value x0taken

from the interval (1,2) The following ten iterates were obtained using the starting value x = 1.0

n xn g 2 (xn )

0 1.000 000 2.000 000

1 2.000 000 1.666 667

2 1.666 667 1.764 706

3 1.764 706 1.734 694

4 1.734 694 1.743 772

5 1.743 772 1.741 016

6 1.741 016 1.741 852

7 1.741 852 1.741 598

8 1.741 598 1.741 675

9 1.741 675 1.741 652

10 1.741 652 1.741659

We note that the last value g2(x10) = 1.741659 is very close to the true value ofthe root of f (x) which is p = 1.741657 rounded to 6 decimal places.

88 Learning Activity 4: Roots of Functions Summary

DO THIS (4.5)

1. Show that the function g (x) = cos( x) satisfies the conditions for the existence of a unique fixed point inside the interval [0,1].

2. Use the result obtained in part (a) to find in only ten iterations, the point of intersection of the two curves y = x, y = cos( x). Start the iteration process with x0 = 1.

Reference for the Learning Activity

Kendall E. Atkinson, An Introduction to Numerical Analysis, - John Wiley & Sons, Second Edition (1989).

Solutions To Formative Evaluation Questions

DO THIS (4.1)

Application of the bisection method in 5 iterations on f (x) = x - cos(x) f (0) = -1, f (1) = 0.459698 . Because f (x) is continuous over [0,1] and f (0) f (1) < 0 , then by the Intermediate Value Theorem for continuous functions it follows that f (x) has at least one root in the interval (0,1) The bisection method gives the following iterates:

X0 = 0 f(X0) < 0

x1 = 1 f (X1) > 0

x2 = 0.5 f(X2) < 0

x3 = 0.5 f(X3) > 0

x4 = 0.625 f(X4) < 0

x5 = 0.6875 f (X5) < 0

x6 = 0.71875 f (X6 ) < 0

DO THIS (4.2)

Application of the secant method in 5 iterations on f (x) = x - cos(x).

Formula to be used:

89 Numerical Methods

Starting values given:

One obtains the following iterates:

x2 = 0.685073, f2 = -0.089300

x3 = 0.410979, f3 = -0.505751

x4 = 0.743847, f4 = +0.007979

x5 = 0.738677, f5 = -0,000683

x6 = 0.739085, f6 = +0.000000

DO THIS (4.3)

Application of Newton-Raphson’s method in three iterations on f (x) = x - cos(x)

Formula to be used is:

Starting value given x0 = 0 ; f’ (x) = 1 + sin(x) One gets the iterates

DO THIS (4.4)

Starting values x0 = 0.8, y 0 = 0.4

90 Learning Activity 4: Roots of Functions Summary

First iteration y1 = 0.420339, x1 = 0.772881,

Second iteration

x2 = 0.771846, y2 = 0.419644

DO THIS (4.5)

Given:

Checking existence of a fixed point g (x) is continuous over [0,1] .

Therefore the existence criterion is fulfilled.

Checking uniqueness of a fixed point g /(x) = - sin( x) I

g (x)l< 1 Vx e[0,1]

Therefore the uniqueness criterion is fulfilled.

Computations

Starting value: x 0 = 1.0

The following iterates are obtained

91 Numerical Methods

XI. Compiled List of all Key Concepts (Glossary)

Key Concepts

Each of the (four) learning activities has key concepts, theorems and principles specific to its contents. However, to give the learner an opportunity to have a glimpse at what lies ahead in terms of the general framework and contents of the module we list here, a priori, all the key concepts, theorems and principles relevant to the module.

Reproduction of these concepts under their respective learning activities is meant to enhance the learner’s understanding and appreciation of their importance in the overall understanding of the course.

1. Error in an approximation

Let X be the exact (true) value of some quantity Q and X * be an approximate value of Q obtained through some numerical process. The difference (deviation) between the exact value X and its approximation X * is called the error in X*, and is denoted by Error (X *) = X - X *

2. Absolute error

The error in an approximation may be positive or negative, depending on whether one has underestimated or overestimated the true value of the quantity Q being approximated. In practice, what matters more is the size of the error rather than its sign. In order to ignore the sign and concentrate on the size of the error, one introduces the concept of absolute error, which is defined by: Absolute error in X* = X - X * .

3. Relative error

The absolute error gives the size of the error and thereby serves as a measure of the accuracy of the approximation X *. However, by not relating the error to the true value being approximated one may not be able to gauge the seriousness of the error. To achieve this one introduces the concept of relative error or percentage error. This is defined by: Relative error in Percentage

error in (X *) = 100

% provided that X * 0 .

Since the true value X is normally not known, one replaces it with the approximate value X * in the denominator of the relative (percentage) error.

92 Learning Activity 4: Roots of Functions Summary

4. Arithmetic progression

An arithmetic progression (AP) is a special sequence {an}: n = 0,1,2,... of numbers.

The essential elements of such a sequence are its first element a0 and a constant difference d. Except for the first element, the k - th element of the sequence is given as ak = a0 + (k - 1)d , k = 1,2,3,...

The sum of the first n terms of an AP is easily shown to be

5. Geometric progression.

A geometric progression (GP) is a special sequence {an}: n = 0,1,2,...of numbers.

The essential elements of such a sequence are its first element a0 and a constant multiplication factor r * 1. Except for the first element, the k - th element of the sequence is given by ak = a 0 rk k = 1,2,3,...

The sum of the first n terms of a GP is

6. Limit of a function

A function f (x) is said to approach the value (have the limit) L as x approaches lim f (x) = L a value c in the domain of f, and we write x ^c if for every choice of a small positive number e one can find a corresponding small positive number 5 (e ) such that whenever C <5 (e) then ^ f (x) L < e . Essentially this statement implies that values of the function f (x) will be arbitrarily close to the number L provided the corresponding values of x are close enough to point c .

7. Continuity

A function f (x) is said to be continuous at a point x = c if the limit of the values lim f (x) = f (a) f (x) is the value f (c), that is if x ^c. Essentially what this definition of continuity implies is that, for f (x) to be continuous at point x = c the following three conditions must hold:

93 Numerical Methods

1. The function must be defined at the point: f (c) must exist.

lim f (x) = L

2. The function must have a limit at the point:x ^c

3. The limit must be equal to the function value: f(c) = L

8. Derivative

The derivative of a function f(x) at an arbitrary point x in its domain is the limit of the difference quotient as the change Ax in x tends to zero, that is and calls it the first derivative of f . dx.

If the limit exists one denotes it by

9. Anti-derivative

An anti-derivative of a function f (x) is itself a function F(x) The function F has the property that its derivative must be the function f (x), meaning that F’(x) = f (x).

Essentially, the process of finding an anti-derivative of f (x) is the reverse process to that of differentiating a function. For this reason, one also speaks of anti-differentiation when finding F(x). Symbolically one writes

10. Convergence of a sequence

A sequence of numbers, written {an} is a well defined and ordered finite or infinite set of numbers. The individual members (elements) of the set must be unambiguously defined. Essentially a set is a special function defined over the set of natural (counting) numbers N or over a subset of N . A set {an} is said to converge to a limit L if for every e > 0 one can find a corresponding positive integer number N (e) such that |an - L| < e if only n > N (e ).

11. Fixed point of a function

Given a continuous function g(x) over a closed interval [a, b] any value x = p which satisfies the relation g (p) = p is called a fixed point of the function g(x) . The process of finding fixed points of a function is closely related to that of determining roots of functions.

94 Learning Activity 4: Roots of Functions Summary

12. Secant line

A secant is a line which passes through two given points P( x1, y1 ) and Q( x2, y2) and is given by the equation . The secant equation is used in deriving the Regula Falsi and the Secant methods for approximating roots of functions.

Key Theorems and Principles

1. Intermediate Value Theorem

If f is continuous on a closed interval [a, b] and if k is any number lying between the two values f (a) and f (b), then there is at least one number c in (a, b) such that f (c) = k. This theorem is the basis upon which two important numerical methods for finding roots of a function f (x) , the bisection method and the Regula Falsi (Rule of false position).

2. Mean Value Theorem of Differentiation (MVT)

If f is a continuous function on the closed interval [a, b] and is differentiable on the open interval (a,b), then there exists at least one value of x = c E (a, b) such that

3. Taylor’s Theorem

Let f be a function whose first n derivatives are continuous on the closed interval [c, c + h] (or [c + h, c] if h is negative) and assume that f(n+1) exists in [c, c + h]

(or [c + h, c] if h is negative). Then, there exists a number O, with 0 < O < 1 such that

4. Fundamental Theorem of Calculus

If f is continuous on [a, b] and F (x) = f f (t)dt for each x in [a, b], then F(x) is continuous on [a, b] and differentiable on (a, b) and dF= f (x) . In other words, F(x) is an anti-derivative of f (x) .

95 Numerical Methods

Corollary: If f (x) is continuous on a closed and bounded interval [a, b], and g (x) = f (x), then Ja f (x)dx = g(b) — g(a).

5. Fixed Point Theorem

If the function g(x) is continuous over a < x < b and g (x) e [ a, b] V x e [ a, b] then there exists at least one fixed point x = p e [a, b] such that g(p) = p . If, in addition to the above assumptions, g(x) is differentiable in a < x < b and dg

L < 1 for all x e (a, b) , then g(x) has a unique (only one) fixed point dx pe (a, b).

XII. Compiled List Of Compulsory Readings

Wikipedia: Numerical Analysis Wikipedia: Interpolation

Fundamental Numerical Methods and Data Analysis, George W. Collins, II, chapter 4. (see: http://bifrost.cwru.edu/personal/collins/numbk/)

Wikipedia: Numerical Methods/Numerical Integration

Wikipedia: Numerical Methods/Equation Solving

XIII. Compiled List of (Optional) Multimedia Resources

Reading # 1: Wolfram MathWorld (visited 03.11.06)

Complete reference : http://mathworld.wolfram.com

Abstract : Wolfram MathWorld is a specialised on-line mathematical encyclopedia. Rationale: It provides the most detailed references to any mathematical topic. Stu¬dents should start by using the search facility for the module title. This will find a major article. At any point students should search for key words that they need to understand. The entry should be studied carefully and thoroughly.

96 Learning Activity 4: Roots of Functions Summary

Reading # 2: Wikipedia (visited 03.11.06)

Complete reference : http://en.wikipedia.org/wiki

Abstract : Wikipedia is an on-line encyclopedia. It is written by its own readers. It is extremely up-to-date as entries are contunally revised. Also, it has proved to be extremely accurate. The mathematics entries are very detailed. Rationale: Students should use wikipedia in the same way as MathWorld. However, the entries may be shorter and a little easier to use in the first instance. Thy will, however, not be so detailed.

Reading # 3: MacTutor History of Mathematics (visited 03.11.06)

Complete reference :http://www-history.mcs.standrews.ac.uk/Indexes Abstract : The MacTutor Archive is the most comprehensive history of mathematics on the internet. The resources are organsied by historical characters and by historical themes.

Rationale: Students should search the MacTutor archive for key words in the topics they are studying (or by the module title itself). It is important to get an overview of where the mathematics being studied fits in to the hostory of mathematics. When the student completes the course and is teaching high school mathematics, the cha¬racters in the history of mathematics will bring the subject to life for their students. Particularly, the role of women in the history of mathematics should be studied to help students understand the difficulties women have faced while still making an important contribution.. Equally, the role of the African continent should be studied to share with students in schools: notably the earliest number counting devices (e.g. the Ishango bone) and the role of Egyptian mathematics should be studied.

XIV. Synthesis of the Module

Completion of the fourth learning activity marks the end of this module. At this juncture, and before presenting oneself for the summative evaluation, it is desirable and appropriate that the learner reflects back on the mission, objectives, activities and attempts to form a global picture of what he/she is expected to have achieved as a result of the collective time and efforts invested in the learning process.

As laid out in the module overview in Section 6, the learner is expected to have acquired knowledge of the basic concepts relating to numerical approximation of numbers and functions.

The learner is now expected to be able to comfortably define the concept of an er¬ror in mathematics, identify the sources of errors, the different types of errors and methods of reducing the impact of errors on our final numerical approximation to a mathematical problem.

After the opening learning activity on errors three substantive learning activities on the core issue of approximation were presented. One of these three core learning ac¬tivity was on numerical approximation of functions using interpolation polynomials. Specifically the learner

97 Numerical Methods

is now expected to be reasonably comfortable in using linear interpolation in its various forms and derive an error bound for linear interpolation. The activity also presents the important topic of interpolation polynomials, concentrating on Lagrangean, Newton’s divided differences and the all important finite difference interpolation polynomials.

The remaining two core learning activities (the third and fourth) are on numerical methods for approximating numbers (roots of functions and values of definite inte¬grals). In this regard the learner is expected to have appreciated the need for resorting to numerical methods, but also be able to apply the methods learnt, including the Bisection, Regula-Falsi, Secant and the Newton-Raphson methods for approximating roots of a non-linear function of a single variable. One is also expected to solve a evaluate limits of functions, sequences and infinite series. One is also expected to be able to apply Newton’s method to approximate solutions of a couples system of two nonlinear equations in two variables.

The module has been structured with a view to escorting and guiding the learner through the material with carefully selected examples and references to the core references. A reasonable number of worked out examples have been included in every learning activity to serve as beacons of reference both in understanding the text before them as well as serving as points of reference while solving related formative evaluation problems that appear in the text in the form of DO THIS. The degree of mastery of the module contents will largely depend on the learner’s deliberate and planned efforts to monitor his/her progress by solving the DO THIS problems.

98 Learning Activity 4: Roots of Functions Summary

XV. Summative Evaluation 1. (a) Define the concept of an error in mathematics.

(b) Give the main types of mathematical errors and mention their sources.

(c) Distinguish between absolute and relative errors and attempt to relate them to the concepts of accuracy and precision.

2. (a) The error E(x) in linear interpolation of a function f (x) using the values f0 = f (x0), f1 = f (x1) where x1 - x0 = h is given by

Derive an upper bound for E. 2

(b) Use the error bound obtained in part (a) to determine the smallest interval size h for which linear interpolation of cos(x)will give approximate values with errors not exceeding 0.01 .

3. The following table gives values of a function f (x) at a number of equally spaced points xk .

x 0.0 0.125 0.25 0.375 0.5

f(x) 1.000000 0.984615 0.941176 0.876712 0.800000

x 0.625 0.75 0.875 1.0

f(x) 0.719101 0.640000 0.566372 0.500000

Approximate f (x) by a quadratic Lagrangean interpolation polynomial using the points (0.125,0.984615), (0.25,0.941176), (0.375,0.876712) and hence, interpolate f (x) at x = 0.3 .

4. (a) Construct a table of finite differences for the function f (x) tabulated in Question 3 (a).

(b) With the aid of the table of finite differences interpolate the function at x = 0.4 using Bessel’s interpolation formula centered at x0 = 0.375

5. Apply Newton’s interpolation based on divided differences to interpolate the function tabulated in Question 4 at the point x = 0.4

99 Numerical Methods

6. (a ) Derive a Newton-Raphson iteration formula for approximating the ru’ root of a positive real number A .

(b) If A = 7 apply the formula you have derived to approximate the square root of 7 correct to six decimals. Start the iteration with x0 = 2.

7. (a) Define the concept of a fixed point for a function g (x) which is known to be continuous over [a, b] and differentiable in (a, b)

(b) State without proof, the existence and uniqueness theorem for fixed points of g(x)

(c)Discuss whether or not the function has a unique fixed point in the interval [0,1].

8. (a) Using an analytical method calculate all the solutions of the coupled system 2x2 + y2 = 5, x2 - 2y2 = 2 correct to 6 decimal places.

(b) Starting with x0 = 1.5, y0 = 0.5 approximate one of the solutions by carrying out two iterations with Newton’s method for systems of nonlinear equations.

9. (a) Find the anti-derivative F (x) of the function and

use

it to evaluate the integral dx correct to 6 decimals.

(b) Evaluate the above integral using the Newton-Cotes formula

10. (a) Evaluate the integral f (ex - cos(x))dx analytically correct to 6 decimal places accuracy.

(b) Tabulate values of the integrand in (a) at the points xk = 0.125k

k = 0,1,2,...8 and use the values obtained with the Trapezoidal rule to

approximate the integral, taking h = 0.25 and h = 0.125, respectively.

(c ) Apply Romberg integration on the two Trapezoidal values obtained in (b) to obtain a better approximation of the integral in (a).

100 Learning Activity 4: Roots of Functions Summary

SOLUTIONS

1. (a) If X * is an approximation to an exact (true) quantity X, then the deviation X - X * is called the error in the approximation X *.

(b) The main types of mathematical errors are:

Initial errors. These are errors in the initial data supplied with a mathematical problem.

Discretization errors. These are errors caused by the process of converting a mathematical problem having a continuous solution to a numerical model whose solution is a discrete function in the form of a sequence of numbers.

Truncation errors. These are errors introduced through unavoidable termination (truncation) of an otherwise infinite process such an infinite series or a convergent iteration.

Rounding errors. These are errors introduced because of limitations on the part of the instruments we use in performing arithmetic operations (addition, subtraction, multiplication or division).

by |X - X| is always non-negative.

The relative error is the ratio between the absolute error and the absolute value of the true (exact) quantity being approximated. The relative error is denoted by

The absolute error is a measure of the accuracy in the approximation, while the relative error is a measure of the precision and relates to the seriousness of the error.

101 Numerical Methods

(b) The function being approximated is f (x) = cos(x) In this case, f “ (x) = - cos(x)

and in this case M = 1. We now must determine the step length h such that

. This gives the result h < 0.3

3. The quadratic Lagrangean interpolation polynomial passing through three points (x0, f0) (x1, f ) (x2, f2) is given by the expression

P2 (x) = L0 (x)f0 + L1 (x)f + L2 (x)f2 where L0 (x) L1 (x) L2 (x) are theLa- grangean coefficients of degree 2. In this case we have the following values:

With x = 0.3, direct substitution of the various values of involved into the expressions for the Lagrangean coefficients gives:

L 0(0.3) = -0.12, L1(0.3) = 0.84, results which lead to the solution

P2 (0 3) = L0(0.3) f0 + L1(0.3) f1 + L2(0.3) f2

= (-0.12)(0.984615) + (0.84)(0.941176) + (0.28)(0.876712) = 0.917913

4. (a) Table of Differences

Differences

x f(x) First Second Third Fourth

0.000 1.000000

-0.015385

0.125 0.984615 -0.028054

-0.043439 0.007029

0.250 0.941176 -0.021025 0.001748

-0.064464 0.008777

0.375 0.876712 -0.012248 -0.000716

-0.076712 0.008061

0.500 0.800000 -0.004187 -0.005672

-0.080899 0.002389

0.625 0.719101 -0.001798 0.001286

-0.079101 0.003675

102 Learning Activity 4: Roots of Functions Summary

0.750 0.640000 -0.005473 -0.005458

-0.073628 -0.001783

0.875 0.566372 -0.007256

-0.066372

1.000 0.500000

(b) Since x0 = 0.375, x = 0.4,h = 0.125, then value of the parameter

s = —1—(0.4 - 0.375) = 0.2 0.125

With the aid of the table of finite differences interpolate the function at x = 0.4 using Bessel’s interpolation formula:

Addition of all these terms leads to the answerP(0.4) = 0.862137 .

5. First one constructs the table of Newton’s divided differences. Then one applies the interpolation polynomial

Table of Newton’s Divided Differences

Divided Differences

x f(x) First Second Third Fourth

0.0 1.000000

-0.12308

0.125 0.984615 -0.897728

-0.347512 0.599808

0.25 0.941176 -0.672800 0.298326

-0.515712 0.748971

0.375 0.876712 -0.391936 -0.122198

-0.613696 0.687872

103 Numerical Methods

0.5 0.800000 -0.133984 -0.354304

-0.647192 0.510720

0.625 0.719101 0.057536 -0.394240

-0.632808 0.0.313600

0.75 0.640000 0.175136 -1.251670

-0.589024 -0.312235

0.875 0.566372 0.058048

-0.530976

1.0 0.500000

Direct application of the formula based on x0 = 0.375 and using the values of various divided differences highlighted in the table together with the value x = 0.4 gives the following results:

f (x) = 0.876712

(x - x0) f [x0,xj = -0.0153424

(x - x0)(x - xj) f [x0,xl9 x2] = 0.00033496

(x - x0)(x - x1)(x - x2) f [x0, xl9 x2, x3] = 0.00028728

(x - x0)(x - x1)(x- x2)(x - x3) f [x0,xl3x2,x3,x4] = 0.000077616

Therefore

P4(0.4) « 0.876712 - 0.015342 + 0.000335 + 0.000287 + 0.000078 =, 0.862070

6. (a) If then, xr - A = 0 .

One may therefore take the function whose roots are being sought to be f (x) = xr - A .

The derivative of f (x) is f7 (x) = rxr-1.

Substituting these quantities into the Newton-Raphson formula

we obtain the general Newton-Raphson formula

104 Learning Activity 4: Roots of Functions Summary

The Newton-Raphson formula for finding the square root of a number A = 7 and r = 2 . This gives the special formula

Taking x0 = 2.0 the following iterates are obtained from the formula:

x0 = 2.0 x1 = 2.75 x3 = 2.6477273 x4 = 2.6457520 x5 = 2.6457513 x6 = 2.6457513

Since the 7th decimal digit remains constant in the last two iterates we accept x = 2.645751 as the desired approximation of V7 correct to 6 decimals.

7. (a) A number p is said to be a fixed point of a function g(x)if g (p) = p . (b) If g(x) is a function which satisfies the conditions

• g (x) is continuous over a closed interval [a, b] • g (x) e [a, b] Vx e [a, b] • g (x) is differentiable in (a, b) • Max|g’(x)| = L < 1 Vxe(a,b)

Then g(x) has a unique fixed point p e (a, b) .

Since the only point of discontinuity for the function is x = 6, the function is definitely continuous over the interval [0,1].

With both g (0) = 4/6 and g (1) = 4/5 lying in (0,1) we also note that all the function 6 5 values lie inside the given interval.

The function is also differentiable and g(x) . The maximum value of .36 9 This result proves that the function has a unique fixed point in the interval 0 < x < 1.

105 Numerical Methods

8. (a) Analytical Solution

To obtain the true (analytical) solution one eliminates one of the variables and solves the resulting quadratic equation in the retained variable.

From the first equation one finds y2 = 5 - 2 x2

We use this result to eliminate y from the second equation.

The corresponding values of y is obtained from the relation

The four possible solutions to the coupled system are

(-1.549183,-0.447213), (-1.549183,0.447213)

(1.549163,-0.447213), (1.549183,0.447213)

(b) Approximation of the roots using Newton’s method Let

f (x, y) = 2x2 + y2 - 5 = 0

g(x, y) = x2 - 2y2 - 2 = 0

The partial derivatives of f (x, y) and g(x, y) are

Newton’s method states t that, starting from an approximate solution (xn, yn) an

improved approximation (xn+1, yn+1) can be obtained by setting

Xn+1 = xn + hn , yn+1 = y„ + kn

The increments hn, kn are obtained by solving the linear system of equations

106 Learning Activity 4: Roots of Functions Summary

Application of Newton’s method:

First Iteration

xo =15, yo =0 5

f0 =-0.25, fx = 6.0, fy =1.0

go =-0.25, gx = 3.0, gy =- 2.0

Substituting these quantities into the linear system and solving the resulting

system one gets h0 = 0.083333, k0 = -0.083333 and therefore

x1 = x0 + h0 = 1.583333, y1 = y0 + k0 = 0.416667

Second Iteration

x1 = 1.583333, y1 = 0.416667

f1 = 0.187498., fx = 6.333333, fy = 0.833334

g1 = 0.159721., gx = 3.166666, gy =-1.666668

Substituting these quantities into the linear system and solving the resulting

system one gets h1 = -0.033772 , k1 = 0.031667 and therefore

x2 = x1 + h1 = 1.549561, y2 = y1 + k1 = 0.448334

9. (a) The anti-derivative of the function

107 Numerical Methods

(b) To approximate the same integral I = f dx using the special

x2 + 3 x + 2

Newton-Cotes formula

fk 0.0 0.065934 0.107143 0.133333 6 0.160428 0.166667

0.15

One first tabulates the function:

While the given formula involves values of the function over three consecutive intervals, the specified interval length h = — results in a pair of such consecutive intervals,

[x0, x1, x2, x3 ] and [x3, x4, x5, x6 ]. Therefore, the integral will be approximated by the expression

10. (a) The value of the integral is

(b) Table of values of the integrand f (x) = ex - cos(x) for values at the equidistant points: “ = 0.0(0.125)1.0:

(0.0,0.0), (0.125,0.140951), (0.25,0.315113),(0.375,0.524484)

(0.5,0.771139), (0.625,1.057283)

(0.75,1.385311), (0.875,1.757878), (1.0,2.177980)

108 Learning Activity 4: Roots of Functions Summary

With the aid of the extended trapezoidal rule one finds: T (0.25) = 0.890138, T (0.125) = 0.880144

XVI. References

• G. Stephenson, 1973, Mathematical Methods for Science Students - Second Edition, Pearson Education Ltd. • Brian Bradie, A friendly Introduction to Numerical Analysis, Pearson Education International. • A.C. Bajpai, I.M. Calus and J.A. Fairley, 1975, Numerical Methods for Engineers and Scientists, Taylor & Francis Ltd., London. • Burden and Faires, 1985, Numerical Analysis - Fifth Edition, PWS - KENT Publishing Company. • Fox, L. and Mayers, D.F., 1958, Computing Methods for Scientists and Engineers, - Oxford University Press, London. • Kendall E. Atkinson, 1987, An Introduction to Numerical Analysis, John Wiley & Sons. • M.K. Jain, S.R.K. Iyengar & R.K. Jain, 1993, Numerical Methods - Problems and Solutions, Wiley Eastern Ltd. • Walter Jennings, 1964. First Course in Numerical Methods, Collier - Macmillan Ltd. • Lay, 2002, Linear Algebra and its Applications - Third Edition, Addison Wesley.

109 Numerical Methods

XVII. Author of the Module

Dr. Ralph W.P. Masenge is Professor of Mathematics in the Faculty of Science, Technology and Environmental Studies at the Open University of Tanzania. A graduate in Mathematics, Physics and Astronomy from the Bavarian University ofWuerzburg in the Federal Republic of Germany in 1968, Professor Masenge obtained a Masters degree in Mathematics from Oxford University in the United Kingdom in 1972 and a Ph. D in Mathematics in 1986 from the University of Dar Es Salaam in Tanzania through a sandwich program carried out at the Catholic University of Nijmegen in the Netherlands.

For almost 33 years (May 1968 - October 2000) Professor Masenge was an academic member of Staff in the Department of Mathematics at the University of Dar Es Salaam during which time he climbed up the academic ladder from Tutorial Fellow in 1968 to full Professor in 1990. From 1976 to 1982, Professor Masenge headed the Department of Mathematics at the University of Dar Es Salaam. He also served as Associate Dean in the Faculty of Science and was Acting Chief Administrative Officer of the University of Dar e salaam for one year (1995).

Born in 1940 at Maharo Village situated on the slopes of Africa’s tallest mountain, The Killimanjaro, Professor Masenge retired in 2000 and left the services of the University of Dar Es Salaam to join the Open University of Tanzania where he now heads the Directorate of Research and Postgraduate Studies as well as being in charge of a number of Mathematics Courses in Calculus, Mathematical Logic and Numerical Analysis.

Professor Masenge is married and has four children. His main hobby is working on his small banana and coconut farm at Mlalakuwa Village, situated some 13 km from the City Centre on the outskirts of Dar Es Salaam, the commercial capital of the United Republic of Tanzania.

110 Learning Activity 4: Roots of Functions Summary

111

The African Virtual University Headquarters Cape Office Park Ring Road Kilimani PO Box 25405-00603 Nairobi, Kenya Tel: +254 20 25283333 [email protected] [email protected]

The African Virtual University Regional Office in Dakar Université Virtuelle Africaine Bureau Régional de l’Afrique de l’Ouest Sicap Liberté VI Extension Villa No.8 VDN B.P. 50609 Dakar, Sénégal Tel: +221 338670324 [email protected]