Numerical Methods (Wiki) GU course work

PDF generated using the open source mwlib toolkit. See http://code.pediapress.com/ for more information. PDF generated at: Sun, 21 Sep 2014 15:07:23 UTC Contents

Articles Numerical methods for ordinary differential equations 1 Runge–Kutta methods 7 Euler method 18 Numerical integration 25 Trapezoidal rule 32 Simpson's rule 35 Bisection method 40 Newton's method 44 Root-finding algorithm 57 Monte Carlo method 62 Interpolation 73 Lagrange polynomial 78 References Article Sources and Contributors 84 Image Sources, Licenses and Contributors 86 Article Licenses License 87 Numerical methods for ordinary differential equations 1 Numerical methods for ordinary differential equations

Numerical methods for ordinary differential equations are methods used to find numerical approximations to the solutions of ordinary differential equations (ODEs). Their use is also known as "numerical integration", although this term is sometimes taken to mean the computation of integrals. Many differential equations cannot be solved using symbolic computation ("analysis"). For practical purposes, however – such as in engineering – a numeric approximation to the solution is often sufficient. The algorithms studied here can be used to compute such an approximation. An alternative method is to use techniques from calculus to obtain a series expansion of the solution.

Ordinary differential equations occur in many scientific disciplines, for instance in physics, chemistry, biology, and economics. In addition, some methods in numerical partial differential equations convert the partial differential equation into an ordinary differential equation, which must then be solved. Illustration of numerical integration for the differential equation The problem Blue: the Euler method, green: the midpoint method, red: the exact solution, The A first-order differential equation is an Initial value problem (IVP) of step size is the form,

where f is a function that maps [t ,∞) × Rd to Rd, and the initial 0 condition y ∈ Rd is a given vector. First-order means that only the 0 first derivative of y appears in the equation, and higher derivatives are absent. Without loss of generality to higher-order systems, we restrict ourselves to first-order differential equations, because a higher-order ODE can be converted into a larger system of first-order equations by introducing extra variables. For example, the second-order equation y'' = −y can be rewritten as two first-order equations: y' = z and z' = −y. In this section, we describe numerical methods for IVPs, and remark that boundary value problems (BVPs) require a different set of tools. In a BVP, one defines values, or components of the solution y at more than one point. Because of this, different methods need to be used to solve BVPs. For example, the shooting method (and its variants) or global methods like finite differences, Galerkin methods, or collocation methods are appropriate for that class of problems. The same illustration for It is seen that the midpoint method converges faster than The Picard–Lindelöf theorem states that there is a unique solution, the Euler method. provided f is Lipschitz-continuous. Numerical methods for ordinary differential equations 2

Methods Numerical methods for solving first-order IVPs often fall into one of two large categories: linear multistep methods, or Runge-Kutta methods. A further division can be realized by dividing methods into those that are explicit and those that are implicit. For example, implicit linear multistep methods include Adams-Moulton methods, and backward differentiation methods (BDF), whereas implicit Runge-Kutta methods include diagonally implicit Runge-Kutta (DIRK), singly diagonally implicit runge kutta (SDIRK), and Gauss-Radau (based on Gaussian quadrature) numerical methods. Explicit examples from the linear multistep family include the Adams-Bashforth methods, and any Runge-Kutta method with a lower diagonal Butcher tableau is explicit. A loose rule of thumb dictates that stiff differential equations require the use of implicit schemes, whereas non-stiff problems can be solved more efficiently with explicit schemes. The so-called general linear methods (GLMs) are a generalization of the above two large classes of methods.

Euler method For more details on this topic, see Euler method. From any point on a curve, you can find an approximation of a nearby point on the curve by moving a short distance along a line tangent to the curve. Starting with the differential equation (1), we replace the derivative y' by the finite difference approximation

which when re-arranged yields the following formula

and using (1) gives:

This formula is usually applied in the following way. We choose a step size h, and we construct the sequence t , 0 t = t + h, t = t + 2h, … We denote by y a numerical estimate of the exact solution y(t ). Motivated by (3), we 1 0 2 0 n n compute these estimates by the following recursive scheme

This is the Euler method (or forward Euler method, in contrast with the backward Euler method, to be described below). The method is named after Leonhard Euler who described it in 1768. The Euler method is an example of an explicit method. This means that the new value y is defined in terms of n+1 things that are already known, like y . n

Backward Euler method For more details on this topic, see Backward Euler method. If, instead of (2), we use the approximation

we get the backward Euler method:

The backward Euler method is an implicit method, meaning that we have to solve an equation to find y . One often n+1 uses fixed point iteration or (some modification of) the Newton–Raphson method to achieve this. It costs more time to solve this equation than explicit methods; this cost must be taken into consideration when one selects the method to use. The advantage of implicit methods such as (6) is that they are usually more stable for Numerical methods for ordinary differential equations 3

solving a stiff equation, meaning that a larger step size h can be used.

First-order exponential integrator method For more details on this topic, see Exponential integrators. Exponential integrators describe a large class of integrators that have recently seen a lot of development.[1] They date back to at least the 1960s. In place of (1), we assume the differential equation is either of the form

or it has been locally linearize about a background state to produce a linear term and a nonlinear term . Exponential integrators are constructed by multiplying (7) by , and exactly integrating the result over a time interval :

This approximation is exact, but it doesn't define the integral. The first-order exponential integrator can be realized by holding constant over the full interval:

Generalizations The Euler method is often not accurate enough. In more precise terms, it only has order one (the concept of order is explained below). This caused mathematicians to look for higher-order methods. One possibility is to use not only the previously computed value y to determine y , but to make the solution n n+1 depend on more past values. This yields a so-called multistep method. Perhaps the simplest is the Leapfrog method which is second order and (roughly speaking) relies on two time values. Almost all practical multistep methods fall within the family of linear multistep methods, which have the form

Another possibility is to use more points in the interval [t ,t ]. This leads to the family of Runge–Kutta methods, n n+1 named after Carl Runge and Martin Kutta. One of their fourth-order methods is especially popular.

Advanced features A good implementation of one of these methods for solving an ODE entails more than the time-stepping formula. It is often inefficient to use the same step size all the time, so variable step-size methods have been developed. Usually, the step size is chosen such that the (local) error per step is below some tolerance level. This means that the methods must also compute an error indicator, an estimate of the local error. An extension of this idea is to choose dynamically between different methods of different orders (this is called a variable order method). Methods based on Richardson extrapolation, such as the Bulirsch–Stoer algorithm, are often used to construct various methods of different orders. Other desirable features include: • dense output: cheap numerical approximations for the whole integration interval, and not only at the points t , t , 0 1 t , ... 2 • event location: finding the times where, say, a particular function vanishes. This typically requires the use of a root-finding algorithm. Numerical methods for ordinary differential equations 4

• support for parallel computing. • when used for integrating with respect to time, time reversibility

Alternative methods Many methods do not fall within the framework discussed here. Some classes of alternative methods are: • multiderivative methods, which use not only the function f but also its derivatives. This class includes Hermite–Obreschkoff methods and Fehlberg methods, as well as methods like the Parker–Sochacki method or Bychkov-Scherbakov method, which compute the coefficients of the Taylor series of the solution y recursively. • methods for second order ODEs. We said that all higher-order ODEs can be transformed to first-order ODEs of the form (1). While this is certainly true, it may not be the best way to proceed. In particular, Nyström methods work directly with second-order equations. • geometric integration methods are especially designed for special classes of ODEs (e.g., symplectic integrators for the solution of Hamiltonian equations). They take care that the numerical solution respects the underlying structure or geometry of these classes. • Quantized State Systems Methods are a family of ODE integration methods based on the idea of state quantization. They are efficient when simulating sparse systems with frequent discontinuities.

Analysis Numerical analysis is not only the design of numerical methods, but also their analysis. Three central concepts in this analysis are: • convergence: whether the method approximates the solution, • order: how well it approximates the solution, and • stability: whether errors are damped out.

Convergence A numerical method is said to be convergent if the numerical solution approaches the exact solution as the step size h goes to 0. More precisely, we require that for every ODE (1) with a Lipschitz function f and every t* > 0,

All the methods mentioned above are convergent. In fact, a numerical scheme has to be convergent to be of any use.

Consistency and order For more details on this topic, see Truncation error (numerical integration). Suppose the numerical method is

The local (truncation) error of the method is the error committed by one step of the method. That is, it is the difference between the result given by the method, assuming that no error was made in earlier steps, and the exact solution:

The method is said to be consistent if

The method has order if Numerical methods for ordinary differential equations 5

Hence a method is consistent if it has an order greater than 0. The (forward) Euler method (4) and the backward Euler method (6) introduced above both have order 1, so they are consistent. Most methods being used in practice attain higher order. Consistency is a necessary condition for convergence, but not sufficient; for a method to be convergent, it must be both consistent and zero-stable. A related concept is the global (truncation) error, the error sustained in all the steps one needs to reach a fixed time t. Explicitly, the global error at time t is y − y(t) where N = (t−t )/h. The global error of a pth order one-step method N 0 is O(hp); in particular, such a method is convergent. This statement is not necessarily true for multi-step methods.

Stability and stiffness For more details on this topic, see Stiff equation. For some differential equations, application of standard methods —such as the Euler method, explicit Runge–Kutta methods, or multistep methods (e.g., Adams–Bashforth methods)— exhibit instability in the solutions, though other methods may produce stable solutions. This "difficult behaviour" in the equation (which may not necessarily be complex itself) is described as stiffness, and is often caused by the presence of different time scales in the underlying problem. Stiff problems are ubiquitous in chemical kinetics, control theory, solid mechanics, weather forecasting, biology, plasma physics, and electronics.

History Below is a timeline of some important developments in this field. • 1768 - Leonhard Euler publishes his method. • 1824 - Augustin Louis Cauchy proves convergence of the Euler method. In this proof, Cauchy uses the implicit Euler method. • 1855 - First mention of the multistep methods of John Couch Adams in a letter written by F. Bashforth. • 1895 - Carl Runge publishes the first Runge–Kutta method. • 1905 - Martin Kutta describes the popular fourth-order Runge–Kutta method. • 1910 - Lewis Fry Richardson announces his extrapolation method, Richardson extrapolation. • 1952 - Charles F. Curtiss and Joseph Oakland Hirschfelder coin the term stiff equations. • 1963 - Germund Dahlquist introduces A-stability of integration methods.

Numerical solutions to second-order one-dimensional boundary value problems Boundary value problems (BVPs) are usually solved numerically by solving an approximately equivalent matrix problem obtained by discretizing the original BVP. The most commonly used method for numerically solving BVPs in one dimension is called the Finite Difference Method. This method takes advantage of linear combinations of point values to construct finite difference coefficients that describe derivatives of the function. For example, the second-order central difference approximation to the first derivative is given by:

and the second-order central difference for the second derivative is given by:

In both of these formulae, is the distance between neighbouring x values on the discretized domain. One then constructs a linear system that can then be solved by standard matrix methods. For instance, suppose the equation to be solved is: Numerical methods for ordinary differential equations 6

The next step would be to discretize the problem and use linear derivative approximations such as

and solve the resulting system of linear equations. This would lead to equations such as:

On first viewing, this system of equations appears to have difficulty associated with the fact that the equation involves no terms that are not multiplied by variables, but in fact this is false. At i = 1 and n − 1 there is a term involving the boundary values and and since these two values are known, one can simply substitute them into this equation and as a result have a non-homogeneous linear system of equations that has non-trivial solutions.

Notes [1] This is a modern and extensive review paper for exponential integrators

References • Bradie, Brian (2006). A Friendly Introduction to Numerical Analysis. Upper Saddle River, New Jersey: Pearson Prentice Hall. ISBN 0-13-013054-0. • J. C. Butcher, Numerical methods for ordinary differential equations, ISBN 0-471-96758-0 • Ernst Hairer, Syvert Paul Nørsett and Gerhard Wanner, Solving ordinary differential equations I: Nonstiff problems, second edition, Springer Verlag, Berlin, 1993. ISBN 3-540-56670-8. • Ernst Hairer and Gerhard Wanner, Solving ordinary differential equations II: Stiff and differential-algebraic problems, second edition, Springer Verlag, Berlin, 1996. ISBN 3-540-60452-9. (This two-volume monograph systematically covers all aspects of the field.)

• Hochbruck, Marlis; Ostermann, Alexander (May 2010). Exponential integrators (http:/ / journals. cambridge. org/

action/ displayAbstract?fromPage=online& aid=7701740). pp. 209–286. doi: 10.1017/S0962492910000048

(http:/ / dx. doi. org/ 10. 1017/ S0962492910000048). • Arieh Iserles, A First Course in the Numerical Analysis of Differential Equations, Cambridge University Press, 1996. ISBN 0-521-55376-8 (hardback), ISBN 0-521-55655-4 (paperback). (Textbook, targeting advanced undergraduate and postgraduate students in mathematics, which also discusses numerical partial differential equations.) • John Denholm Lambert, Numerical Methods for Ordinary Differential Systems, John Wiley & Sons, Chichester, 1991. ISBN 0-471-92990-5. (Textbook, slightly more demanding than the book by Iserles.) Numerical methods for ordinary differential equations 7

External links

• Joseph W. Rudmin, Application of the Parker–Sochacki Method to Celestial Mechanics (http:/ / csma31. csm.

jmu. edu/ physics/ rudmin/ ps. pdf), 1998.

• Dominique Tournès, L'intégration approchée des équations différentielles ordinaires (1671-1914) (http:/ / www.

reunion. iufm. fr/ dep/ mathematiques/ calculsavant/ Equipe/ tournes. html), thèse de doctorat de l'université Paris 7 - Denis Diderot, juin 1996. Réimp. Villeneuve d'Ascq : Presses universitaires du Septentrion, 1997, 468 p. (Extensive online material on ODE numerical analysis history, for English-language material on the history of ODE numerical analysis, see e.g. the paper books by Chabert and Goldstine quoted by him.)

Runge–Kutta methods

In numerical analysis, the Runge–Kutta methods (German pronunciation: [ˌʁʊŋəˈkʊta]) are an important family of implicit and explicit iterative methods, which are used in temporal discretization for the approximation of solutions of ordinary differential equations. These techniques were developed around 1900 by the German mathematicians C. Runge and M. W. Kutta. See the article on numerical methods for ordinary differential equations for more background and other methods. See also List of Runge–Kutta methods.

The Runge–Kutta method One member of the family of Runge–Kutta methods is often referred to as "RK4", "classical Runge–Kutta method" or simply as "the Runge–Kutta method". Let an initial value problem be specified as follows.

Here, y is an unknown function (scalar or vector) of time t which we would like to approximate; we are told that , the rate at which y changes, is a function of t and of y itself. At the initial time the corresponding y-value is . The function f and the data , are given. Now pick a step-size h>0 and define

for n = 0, 1, 2, 3, . . . , using

(Note: the above equations have different but equivalent definitions in different texts).[1] Here is the RK4 approximation of , and the next value ( ) is determined by the present value ( ) plus the weighted average of four increments, where each increment is the product of the size of the interval, h, and an estimated slope specified by function f on the right-hand side of the differential equation. • is the increment based on the slope at the beginning of the interval, using , (Euler's method) ; • is the increment based on the slope at the midpoint of the interval, using ; • is again the increment based on the slope at the midpoint, but now using ; • is the increment based on the slope at the end of the interval, using . RungeKutta methods 8

In averaging the four increments, greater weight is given to the increments at the midpoint. The weights are chosen such that if is independent of , so that the differential equation is equivalent to a simple integral, then RK4 is Simpson's rule. The RK4 method is a fourth-order method, meaning that the local truncation error is on the order of , while the total accumulated error is order . Tan Delin and Chen Zheng have developed a general formula for a Runge–Kutta method in the fourth-order as follows:

for n = 0, 1, 2, 3, . . . , using

,

where is a free parameter. Choosing , this is the classical fourth-order Runge-Kutta method. With , this formula produces other fourth-order Runge-Kutta methods.

Explicit Runge–Kutta methods The family of explicit Runge–Kutta methods is a generalization of the RK4 method mentioned above. It is given by

where

(Note: the above equations have different but equivalent definitions in different texts). To specify a particular method, one needs to provide the integer s (the number of stages), and the coefficients a (for ij 1 ≤ j < i ≤ s), b (for i = 1, 2, ..., s) and c (for i = 2, 3, ..., s). The matrix [a ] is called the Runge–Kutta matrix, while i i ij the b and c are known as the weights and the nodes. These data are usually arranged in a mnemonic device, known i i as a Butcher tableau (after John C. Butcher): RungeKutta methods 9

0

The Runge–Kutta method is consistent if

There are also accompanying requirements if one requires the method to have a certain order p, meaning that the local truncation error is O(hp+1). These can be derived from the definition of the truncation error itself. For example, a two-stage method has order 2 if b + b = 1, b c = 1/2, and a = c . 1 2 2 2 21 2

Examples The RK4 method falls in this framework. Its tableau is:

0

1/2 1/2

1/2 0 1/2

1 0 0 1

1/6 1/3 1/3 1/6

A slight variation of "the" Runge–Kutta method is also due to Kutta in 1901 and is called the 3/8-rule.[2] The primary advantage this method has is that almost all of the error coefficients are smaller than the popular method, but it requires slightly more FLOPs (floating point operations) per time step. Its Butcher tableau is given by:

0

1/3 1/3

2/3 −1/3 1

1 1 −1 1

1/8 3/8 3/8 1/8

However, the simplest Runge–Kutta method is the (forward) Euler method, given by the formula . This is the only consistent explicit Runge–Kutta method with one stage. The corresponding tableau is:

0

1 RungeKutta methods 10

Second-order methods with two stages An example of a second-order method with two stages is provided by the midpoint method

The corresponding tableau is:

0

1/2 1/2

0 1

The midpoint method is not the only second-order Runge–Kutta method with two stages; there is a family of such methods, parameterized by α, and given by the formula

Its Butcher tableau is

0

In this family, gives the midpoint method and is Heun's method.

Usage As an example, consider the two-stage second-order Runge–Kutta method with α = 2/3. It is given by the tableau

0

2/3 2/3

1/4 3/4

with the corresponding equations

This method is used to solve the initial-value problem

with step size h = 0.025, so the method needs to take four steps. The method proceeds as follows: RungeKutta methods 11

The numerical solutions correspond to the underlined values.

Adaptive Runge–Kutta methods The adaptive methods are designed to produce an estimate of the local truncation error of a single Runge–Kutta step. This is done by having two methods in the tableau, one with order and one with order . The lower-order step is given by

where the are the same as for the higher-order method. Then the error is

which is . The Butcher tableau for this kind of method is extended to give the values of :

0

The Runge–Kutta–Fehlberg method has two methods of orders 5 and 4. Its extended Butcher tableau is: RungeKutta methods 12

0

1/4 1/4

3/8 3/32 9/32

12/13 1932/2197 −7200/2197 7296/2197

1 439/216 −8 3680/513 -845/4104

1/2 −8/27 2 −3544/2565 1859/4104 −11/40

16/135 0 6656/12825 28561/56430 −9/50 2/55

25/216 0 1408/2565 2197/4104 −1/5 0

However, the simplest adaptive Runge–Kutta method involves combining Heun's method, which is order 2, with the Euler method, which is order 1. Its extended Butcher tableau is:

0

1 1

1/2 1/2

1 0

The error estimate is used to control the step size. Other adaptive Runge–Kutta methods are the Bogacki–Shampine method (orders 3 and 2), the Cash–Karp method and the Dormand–Prince method (both with orders 5 and 4).

Nonconfluent Runge–Kutta methods A Runge–Kutta method is said to be nonconfluent if all the are distinct.

Implicit Runge–Kutta methods All Runge–Kutta methods mentioned up to now are explicit methods. Explicit Runge–Kutta methods are generally unsuitable for the solution of stiff equations because their region of absolute stability is small; in particular, it is bounded. This issue is especially important in the solution of partial differential equations. The instability of explicit Runge–Kutta methods motivates the development of implicit methods. An implicit Runge–Kutta method has the form

where

The difference with an explicit method is that in an explicit method, the sum over j only goes up to i − 1. This also shows up in the Butcher tableau: the coefficient matrix of an explicit method is lower triangular. In an implicit method, the sum over j goes up to s and the coefficient matrix is not triangular, yielding a Butcher tableau of the form RungeKutta methods 13

The consequence of this difference is that at every step, a system of algebraic equations has to be solved. This increases the computational cost considerably. If a method with s stages is used to solve a differential equation with m components, then the system of algebraic equations has ms components. This can be contrasted with implicit linear multistep methods (the other big family of methods for ODEs): an implicit s-step linear multistep method needs to solve a system of algebraic equations with only m components, so the size of the system does not increase as the number of steps increases.

Examples The simplest example of an implicit Runge–Kutta method is the backward Euler method:

The Butcher tableau for this is simply:

This Butcher tableau corresponds to the formulae

which can be re-arranged to get the formula for the backward Euler method listed above. Another example for an implicit Runge–Kutta method is the trapezoidal rule. Its Butcher tableau is:

The trapezoidal rule is a collocation method (as discussed in that article). All collocation methods are implicit Runge–Kutta methods, but not all implicit Runge–Kutta methods are collocation methods. The Gauss–Legendre methods form a family of collocation methods based on Gauss quadrature. A Gauss–Legendre method with s stages has order 2s (thus, methods with arbitrarily high order can be constructed). The method with two stages (and thus order four) has Butcher tableau:

Stability The advantage of implicit Runge–Kutta methods over explicit ones is their greater stability, especially when applied to stiff equations. Consider the linear test equation y' = λy. A Runge–Kutta method applied to this equation reduces to the iteration , with r given by

where e stands for the vector of ones. The function r is called the stability function. It follows from the formula that r is the quotient of two polynomials of degree s if the method has s stages. Explicit methods have a strictly lower triangular matrix A, which implies that det(I − zA) = 1 and that the stability function is a polynomial. RungeKutta methods 14

The numerical solution to the linear test equation decays to zero if | r(z) | < 1 with z = hλ. The set of such z is called the domain of absolute stability. In particular, the method is said to be A-stable if all z with Re(z) < 0 are in the domain of absolute stability. The stability function of an explicit Runge–Kutta method is a polynomial, so explicit Runge–Kutta methods can never be A-stable. If the method has order p, then the stability function satisfies as . Thus, it is of interest to study quotients of polynomials of given degrees that approximate the exponential function the best. These are known as Padé approximants. A Padé approximant with numerator of degree m and denominator of degree n is A-stable if and only if m ≤ n ≤ m + 2. The Gauss–Legendre method with s stages has order 2s, so its stability function is the Padé approximant with m = n = s. It follows that the method is A-stable. This shows that A-stable Runge–Kutta can have arbitrarily high order. In contrast, the order of A-stable linear multistep methods cannot exceed two.[3]

B-stability The A-stability concept for the solution of differential equations is related to the linear autonomous equation . Dahlquist proposed the investigation of stability of numerical schemes when applied to nonlinear systems which satisfies a monotonicity condition. The corresponding concepts were defined as G-stability for multistep methods (and the related one-leg methods) and B-stability (Butcher, 1975) for Runge–Kutta methods. A Runge–Kutta method applied to the non-linear system , which verifies , is called B-stable, if this condition implies for two numerical solutions. Let , and be three matrices defined by

A Runge–Kutta method is said to be algebraically stable if the matrices and are both non-negative definite. A sufficient condition for B-stability is: and are non-negative definite.

Derivation of the Runge–Kutta fourth-order method In general a Runge–Kutta method of order can be written as:

where:

are increments obtained evaluating the derivatives of at the -th order. We develop the derivation[4] for the Runge–Kutta fourth-order method using the general formula with evaluated, as explained above, at the starting point, the midpoint and the end point of any interval , thus we choose: RungeKutta methods 15

and otherwise. We begin by defining the following quantities:

where and If we define:

and for the previous relations we can show that the following equalities holds up to :

where:

is the total derivative of with respect to time. If we now express the general formula using what we just derived we obtain: RungeKutta methods 16

and comparing this with the Taylor series of around :

we obtain a system of constraints on the coefficients:

which when solved gives as stated above.

Notes [1] , , and leave out the factor h in the definition of the stages. , and use the y-values as stages. [2] refer to [3] This result is due to .

[4] PDF (http:/ / www. ss. ncu. edu. tw/ ~lyu/ lecture_files_en/ lyu_NSSP_Notes/ Lyu_NSSP_AppendixC. pdf) reporting this derivation

References • Ascher, Uri M.; Petzold, Linda R. (1998), Computer Methods for Ordinary Differential Equations and Differential-Algebraic Equations, Philadelphia: Society for Industrial and Applied Mathematics, ISBN 978-0-89871-412-8. • Atkinson, Kendall A. (1989), An Introduction to Numerical Analysis (2nd ed.), New York: John Wiley & Sons, ISBN 978-0-471-50023-0.

• Butcher, John C. (May 1963), "Coefficients for the study of Runge-Kutta integration processes" (http:/ / journals.

cambridge. org/ action/ displayAbstract?fromPage=online& aid=4922056) 3 (2), pp. 185–201, doi:

10.1017/S1446788700027932 (http:/ / dx. doi. org/ 10. 1017/ S1446788700027932). • Butcher, John C. (1975), "A stability property of implicit Runge-Kutta methods", BIT 15: 358–361, doi:

10.1007/bf01931672 (http:/ / dx. doi. org/ 10. 1007/ bf01931672). • Butcher, John C. (2003), Numerical Methods for Ordinary Differential Equations, New York: John Wiley & Sons, ISBN 978-0-471-96758-3. • Cellier, F.; Kofman, E. (2006), Continuous System Simulation, Springer Verlag, ISBN 0-387-26102-8. RungeKutta methods 17

• Dahlquist, Germund (1963), "A special stability problem for linear multistep methods", BIT 3: 27–43, doi:

10.1007/BF01963532 (http:/ / dx. doi. org/ 10. 1007/ BF01963532), ISSN 0006-3835 (http:/ / www. worldcat.

org/ issn/ 0006-3835). • Forsythe, George E.; Malcolm, Michael A.; Moler, Cleve B. (1977), Computer Methods for Mathematical Computations, Prentice-Hall (see Chapter 6). • Hairer, Ernst; Nørsett, Syvert Paul; Wanner, Gerhard (1993), Solving ordinary differential equations I: Nonstiff problems, Berlin, New York: Springer-Verlag, ISBN 978-3-540-56670-0. • Hairer, Ernst; Wanner, Gerhard (1996), Solving ordinary differential equations II: Stiff and differential-algebraic problems (2nd ed.), Berlin, New York: Springer-Verlag, ISBN 978-3-540-60452-5. • Iserles, Arieh (1996), A First Course in the Numerical Analysis of Differential Equations, Cambridge University Press, ISBN 978-0-521-55655-2. • Lambert, J.D (1991), Numerical Methods for Ordinary Differential Systems. The Initial Value Problem, John Wiley & Sons, ISBN 0-471-92990-5

• Kaw, Autar; Kalu, Egwu (2008), Numerical Methods with Applications (http:/ / numericalmethods. eng. usf. edu/

topics/ textbook_index. html) (1st ed.), autarkaw.com. • Kutta, Martin Wilhelm (1901), "Beitrag zur näherungsweisen Integration totaler Differentialgleichungen", Zeitschrift für Mathematik und Physik 46: 435–453. • Press, William H.; Flannery, Brian P.; Teukolsky, Saul A.; Vetterling, William T. (2007), "Section 17.1

Runge-Kutta Method" (http:/ / apps. nrbook. com/ empanel/ index. html#pg=907), Numerical Recipes: The Art of Scientific Computing (3rd ed.), Cambridge University Press, ISBN 978-0-521-88068-8. Also, Section 17.2.

Adaptive Stepsize Control for Runge-Kutta (http:/ / apps. nrbook. com/ empanel/ index. html#pg=910). • Stoer, Josef; Bulirsch, Roland (2002), Introduction to Numerical Analysis (3rd ed.), Berlin, New York: Springer-Verlag, ISBN 978-0-387-95452-3. • Süli, Endre; Mayers, David (2003), An Introduction to Numerical Analysis, Cambridge University Press, ISBN 0-521-00794-1.

• Tan, Delin; Chen, Zheng (2012), "On A General Formula of Fourth Order Runge-Kutta Method" (http:/ / msme.

us/ 2012-2-1. pdf), Journal of Mathematical Science & Mathematics Education 7.2: 1–10.

External links

• Hazewinkel, Michiel, ed. (2001), "Runge-Kutta method" (http:/ / www. encyclopediaofmath. org/ index.

php?title=p/ r082810), Encyclopedia of Mathematics, Springer, ISBN 978-1-55608-010-4

• Runge–Kutta 4th-Order Method (http:/ / numericalmethods. eng. usf. edu/ topics/ runge_kutta_4th_method. html)

• Runge Kutta Method for O.D.E.'s (http:/ / math. fullerton. edu/ mathews/ n2003/ RungeKuttaMod. html)

• DotNumerics: Ordinary Differential Equations for C# and VB.NET (http:/ / www. dotnumerics. com/

NumericalLibraries/ DifferentialEquations/ ) — Initial-value problem for nonstiff and stiff ordinary differential equations (explicit Runge–Kutta, implicit Runge–Kutta, Gear's BDF and Adams–Moulton).

• GafferOnGames (http:/ / gafferongames. com/ game-physics/ integration-basics/ ) — A physics resource for computer programmers Euler method 18 Euler method

For integrating with respect to the Euler characteristic, see Euler calculus. For Euler's method for factorizing an integer, see Euler's factorization method. In mathematics and computational science, the Euler method is a SN-order numerical procedure for solving ordinary differential equations (ODEs) with a given initial value. It is the most basic explicit method for numerical integration of ordinary differential equations and is the simplest Runge–Kutta method. The Euler method is named after Leonhard Euler, who treated it in his book Institutionum calculi integralis (published 1768–70).

The Euler method is a first-order method, which means that the local error (error per step) is proportional to the square of the step size, and Illustration of the Euler method. The unknown the global error (error at a given time) is proportional to the step size. curve is in blue, and its polygonal approximation The Euler method often serves as the basis to construct more complex is in red. methods.

Informal geometrical description Consider the problem of calculating the shape of an unknown curve which starts at a given point and satisfies a given differential equation. Here, a differential equation can be thought of as a formula by which the slope of the tangent line to the curve can be computed at any point on the curve, once the position of that point has been calculated. The idea is that while the curve is initially unknown, its starting point, which we denote by is known (see the picture on top right). Then, from the differential equation, the slope to the curve at can be computed, and so, the tangent line. Take a small step along that tangent line up to a point Along this small step, the slope does not change too much, so will be close to the curve. If we pretend that is still on the curve, the same reasoning as for the point above can be used. After several steps, a polygonal curve is computed. In general, this curve does not diverge too far from the original unknown curve, and the error between the two curves can be made small if the step size is small enough and the interval of computation is finite.

Formulation of the method Suppose that we want to approximate the solution of the initial value problem

Choose a value for the size of every step and set . Now, one step of the Euler method from to is

The value of is an approximation of the solution to the ODE at time : . The Euler method is explicit, i.e. the solution is an explicit function of for . While the Euler method integrates a first-order ODE, any ODE of order N can be represented as a first-order ODE: to treat the equation

, Euler method 19

we introduce auxiliary variables and obtain the equivalent equation

This is a first-order system in the variable and can be handled by Euler's method or, in fact, by any other scheme for first-order systems.

Example Given the initial value problem

we would like to use the Euler method to approximate .[1]

Using step size equal to 1 (h = 1)

The Euler method is

so first we must compute . In this simple differential equation, the function is defined by . We have

By doing the above step, we have found the slope of the line that is tangent to the solution curve at the point . Recall that the slope is defined as the change in divided by the change in , or . The next step is to multiply the above value by the step size , which we take equal to one here:

Since the step size is the change in , when we multiply the step size and the slope of the tangent, we get a change in value. This value is then added to the initial value to obtain the next value to be used for Illustration of numerical integration for the computations. equation Blue is the Euler method; green, the midpoint method; red, the exact solution, The step size is The above steps should be repeated to find , and . h = 1.0.

Due to the repetitive nature of this algorithm, it can be helpful to organize computations in a chart form, as seen below, to avoid making errors. Euler method 20

0 1 0 1 1 1 2

1 2 1 2 1 2 4

2 4 2 4 1 4 8

3 8 3 8 1 8 16

The conclusion of this computation is that . The exact solution of the differential equation is , so . Thus, the approximation of the Euler method is not very good in this case. However, as the figure shows, its behaviour is qualitatively right. Using other step sizes

As suggested in the introduction, the Euler method is more accurate if the step size is smaller. The table below shows the result with different step sizes. The top row corresponds to the example in the previous section, and the second row is illustrated in the figure.

The same illustration for h = 0.25.

step size result of Euler's method error

1 16 38.598

0.25 35.53 19.07

0.1 45.26 9.34

0.05 49.56 5.04

0.025 51.98 2.62

0.0125 53.26 1.34

The error recorded in the last column of the table is the difference between the exact solution at and the Euler approximation. In the bottom of the table, the step size is half the step size in the previous row, and the error is also approximately half the error in the previous row. This suggests that the error is roughly proportional to the step size, at least for fairly small values of the step size. This is true in general, also for other equations; see the section Global truncation error for more details. Euler method 21

Other methods, such as the midpoint method also illustrated in the figures, behave more favourably: the error of the midpoint method is roughly proportional to the square of the step size. For this reason, the Euler method is said to be a first-order method, while the midpoint method is second order. We can extrapolate from the above table that the step size needed to get an answer that is correct to three decimal places is approximately 0.00001, meaning that we need 400,000 steps. This large number of steps entails a high computational cost. For this reason, people usually employ alternative, higher-order methods such as Runge–Kutta methods or linear multistep methods, especially if a high accuracy is desired.

Derivation The Euler method can be derived in a number of ways. Firstly, there is the geometrical description mentioned above. Another possibility is to consider the Taylor expansion of the function around :

The differential equation states that . If this is substituted in the Taylor expansion and the quadratic and higher-order terms are ignored, the Euler method arises. The Taylor expansion is used below to analyze the error committed by the Euler method, and it can be extended to produce Runge–Kutta methods. A closely related derivation is to substitute the forward finite difference formula for the derivative,

in the differential equation . Again, this yields the Euler method. A similar computation leads to the midpoint rule and the backward Euler method. Finally, one can integrate the differential equation from to and apply the fundamental theorem of calculus to get:

Now approximate the integral by the left-hand rectangle method (with only one rectangle):

Combining both equations, one finds again the Euler method. This line of thought can be continued to arrive at various linear multistep methods.

Local truncation error The local truncation error of the Euler method is error made in a single step. It is the difference between the numerical solution after one step, , and the exact solution at time . The numerical solution is given by

For the exact solution, we use the Taylor expansion mentioned in the section Derivation above:

The local truncation error (LTE) introduced by the Euler method is given by the difference between these equations:

This result is valid if has a bounded third derivative. Euler method 22

This shows that for small , the local truncation error is approximately proportional to . This makes the Euler method less accurate (for small ) than other higher-order techniques such as Runge-Kutta methods and linear multistep methods, for which the local truncation error is proportial to a higher power of the step size. A slightly different formulation for the local truncation error can be obtained by using the Lagrange form for the remainder term in Taylor's theorem. If has a continuous second derivative, then there exists a such that

In the above expressions for the error, the second derivative of the unknown exact solution can be replaced by an expression involving the right-hand side of the differential equation. Indeed, it follows from the equation that

Global truncation error The global truncation error is the error at a fixed time , after however many steps the methods needs to take to reach that time from the initial time. The global truncation error is the cumulative effect of the local truncation errors committed in each step. The number of steps is easily determined to be , which is proportional to , and the error committed in each step is proportional to (see the previous section). Thus, it is to be expected that the global truncation error will be proportional to . This intuitive reasoning can be made precise. If the solution has a bounded second derivative and is Lipschitz continuous in its second argument, then the global truncation error (GTE) is bounded by

where is an upper bound on the second derivative of on the given interval and is the Lipschitz constant of . The precise form of this bound of little practical importance, as in most cases the bound vastly overestimates the actual error committed by the Euler method. What is important is that it shows that the global truncation error is (approximately) proportional to . For this reason, the Euler method is said to be first order.

Numerical stability

The Euler method can also be numerically unstable, especially for stiff equations, meaning that the numerical solution grows very large for equations where the exact solution does not. This can be illustrated using the linear equation

The exact solution is , which decays to zero as . However, if the Euler method is applied to this equation with step size , then the numerical solution is qualitatively wrong: it oscillates and grows (see the figure). This is what it means to Solution of computed with the be unstable. If a smaller step size is used, for instance , then Euler method with step size (blue the numerical solution does decay to zero. squares) and (red circles). The black curve shows the exact solution. Euler method 23

If the Euler method is applied to the linear equation , then the numerical solution is unstable if the product is outside the region

illustrated on the right. This region is called the (linear) instability region. In the example, equals −2.3, so if then which is outside the stability region, and thus the numerical solution is unstable. This limitation —along with its slow convergence of error with h— The pink disk shows the stability region for the means that the Euler method is not often used, except as a simple Euler method. example of numerical integration.

Rounding errors The discussion up to now has ignored the consequences of rounding error. In step n of the Euler method, the rounding error is roughly of the magnitude εy where ε is the machine epsilon. Assuming that the rounding errors are n all of approximately the same size, the combined rounding error in N steps is roughly Nεy if all errors points in the 0 same direction. Since the number of steps is inversely proportional to the step size h, the total rounding error is proportional to ε / h. In reality, however, it is extremely unlikely that all rounding errors point in the same direction. If instead it is assumed that the rounding errors are independent rounding variables, then the total rounding error is proportional to . Thus, for extremely small values of the step size, the truncation error will be small but the effect of rounding error may be big. Most of the effect of rounding error can be easily avoided if compensated summation is used in the formula for the Euler method.

Modifications and extensions A simple modification of the Euler method which eliminates the stability problems noted in the previous section is the backward Euler method:

This differs from the (standard, or forward) Euler method in that the function is evaluated at the end point of the step, instead of the starting point. The backward Euler method is an implicit method, meaning that the formula for the backward Euler method has on both sides, so when applying the backward Euler method we have to solve an equation. This makes the implementation more costly. Other modifications of the Euler method that help with stability yield the exponential Euler method or the semi-implicit Euler method. More complicated methods can achieve a higher order (and more accuracy). One possibility is to use more function evaluations. This is illustrated by the midpoint method which is already mentioned in this article:

This leads to the family of Runge–Kutta methods. The other possibility is to use more past values, as illustrated by the two-step Adams–Bashforth method:

This leads to the family of linear multistep methods. Euler method 24

Notes [1] See also

References • Atkinson, Kendall A. (1989), An Introduction to Numerical Analysis (2nd ed.), New York: John Wiley & Sons, ISBN 978-0-471-50023-0. • Ascher, Uri M.; Petzold, Linda R. (1998), Computer Methods for Ordinary Differential Equations and Differential-Algebraic Equations, Philadelphia: Society for Industrial and Applied Mathematics, ISBN 978-0-89871-412-8. • Butcher, John C. (2003), Numerical Methods for Ordinary Differential Equations, New York: John Wiley & Sons, ISBN 978-0-471-96758-3. • Hairer, Ernst; Nørsett, Syvert Paul; Wanner, Gerhard (1993), Solving ordinary differential equations I: Nonstiff problems, Berlin, New York: Springer-Verlag, ISBN 978-3-540-56670-0. • Iserles, Arieh (1996), A First Course in the Numerical Analysis of Differential Equations, Cambridge University Press, ISBN 978-0-521-55655-2 • Stoer, Josef; Bulirsch, Roland (2002), Introduction to Numerical Analysis (3rd ed.), Berlin, New York: Springer-Verlag, ISBN 978-0-387-95452-3.

• Lakoba, Taras I. (2012), Simple Euler method and its modifications (http:/ / www. cems. uvm. edu/ ~lakobati/

math337/ notes_1. pdf) (Lecture notes for MATH334, University of Vermont), retrieved 29 February 2012.

External links

The Wikibook Calculus has a page on the topic of: Euler's Method

• Euler's Method for O.D.E.'s (http:/ / math. fullerton. edu/ mathews/ n2003/ Euler'sMethodMod. html), by John H. Matthews, California State University at Fullerton. Numerical integration 25 Numerical integration

In numerical analysis, numerical integration constitutes a broad family of algorithms for calculating the numerical value of a definite integral, and by extension, the term is also sometimes used to describe the numerical solution of differential equations. This article focuses on calculation of definite integrals. The term numerical quadrature (often abbreviated to quadrature) is more or less a synonym for numerical integration, especially as applied to one-dimensional integrals. Numerical integration over more than one dimension is sometimes described as cubature, although the meaning of quadrature is understood for higher-dimensional integration as well. Numerical integration consists of finding The basic problem in numerical integration is to compute an numerical approximations for the value approximate solution to a definite integral

to a given degree of accuracy. If f(x) is a smooth function integrated over a small number of dimensions, and the domain of integration is bounded, there are many methods for approximating the integral to the desired precision.

History Main article: Quadrature (mathematics) Quadrature is a historical mathematical term that means calculating area. Quadrature problems have served as one of the main sources of mathematical analysis. Mathematicians of Ancient Greece, according to the Pythagorean doctrine, understood calculation of area as the process of constructing geometrically a square having the same area (squaring). That is why the process was named quadrature. For example, a quadrature of the circle, Lune of Hippocrates, The Quadrature of the Parabola. This construction must be performed only by means of compass and straightedge. For a quadrature of a rectangle with the sides a and b it is necessary to construct a square with the side (the Geometric mean of a and b). For this purpose it is possible to use the following fact: if we draw the circle with the sum of a and b as the diameter, then the height BH (from a point of their connection to crossing with a circle) equals their geometric mean. The similar geometrical construction solves a problem of a quadrature for a parallelogram and a triangle.

Antique method to find the Geometric mean Numerical integration 26

Problems of quadrature for curvilinear figures are much more difficult. The quadrature of the circle with compass and straightedge had been proved in the 19th century to be impossible. Nevertheless, for some figures (for example Lune of Hippocrates) a quadrature can be performed. The quadratures of a sphere surface and a parabola segment done by Archimedes became the highest achievement of the antique analysis.

• The area of the surface of a sphere is equal to quadruple the area of a great circle of this sphere. • The area of a segment of the parabola cut from it by a straight line is 4/3 the area of the triangle inscribed in this segment. The area of a segment of a parabola For the proof of the results Archimedes used the Method of exhaustion of Eudoxus. In medieval Europe the quadrature meant calculation of area by any method. More often the Method of indivisibles was used; it was less rigorous, but more simple and powerful. With its help Galileo Galilei and Gilles de Roberval found the area of a cycloid arch, Grégoire de Saint-Vincent investigated the area under a hyperbola (Opus Geometricum, 1647), and Alphonse Antonio de Sarasa, de Saint-Vincent's pupil and commentator noted the relation of this area to logarithms. John Wallis algebrised this method: he wrote in his Arithmetica Infinitorum (1656) series that we now call the definite integral, and he calculated their values. Isaac Barrow and James Gregory made further progress: quadratures for some algebraic curves and spirals. Christiaan Huygens successfully performed a quadrature of some Solids of revolution. The quadrature of the hyperbola by Saint-Vincent and de Sarasa provided a new function, the natural logarithm, of critical importance. With the invention of integral calculus came a universal method for area calculation. In response, the term quadrature has become traditional, and instead the modern phrase "computation of a univariate definite integral" is more common.

Reasons for numerical integration There are several reasons for carrying out numerical integration. The integrand f(x) may be known only at certain points, such as obtained by sampling. Some embedded systems and other computer applications may need numerical integration for this reason. A formula for the integrand may be known, but it may be difficult or impossible to find an antiderivative that is an elementary function. An example of such an integrand is f(x) = exp(−x2), the antiderivative of which (the error function, times a constant) cannot be written in elementary form. It may be possible to find an antiderivative symbolically, but it may be easier to compute a numerical approximation than to compute the antiderivative. That may be the case if the antiderivative is given as an infinite series or product, or if its evaluation requires a special function that is not available. Numerical integration 27

Methods for one-dimensional integrals Numerical integration methods can generally be described as combining evaluations of the integrand to get an approximation to the integral. The integrand is evaluated at a finite set of points called integration points and a weighted sum of these values is used to approximate the integral. The integration points and weights depend on the specific method used and the accuracy required from the approximation. An important part of the analysis of any numerical integration method is to study the behavior of the approximation error as a function of the number of integrand evaluations. A method that yields a small error for a small number of evaluations is usually considered superior. Reducing the number of evaluations of the integrand reduces the number of arithmetic operations involved, and therefore reduces the total round-off error. Also, each evaluation takes time, and the integrand may be arbitrarily complicated. A 'brute force' kind of numerical integration can be done, if the integrand is reasonably well-behaved (i.e. piecewise continuous and of bounded variation), by evaluating the integrand with very small increments.

Quadrature rules based on interpolating functions A large class of quadrature rules can be derived by constructing interpolating functions that are easy to integrate. Typically these interpolating functions are polynomials. The simplest method of this type is to let the interpolating function be a constant function (a polynomial of degree zero) that passes through the point ((a+b)/2, f((a+b)/2)). This is called the midpoint rule or rectangle rule. Illustration of the rectangle rule.

The interpolating function may be an affine function (a polynomial of degree 1) that passes through the points (a, f(a)) and (b, f(b)). This is called the trapezoidal rule.

Illustration of the trapezoidal rule.

For either one of these rules, we can make a more accurate approximation by breaking up the interval [a, b] into some number n of subintervals, computing an approximation for each subinterval, then adding up all the results. This is called a composite rule, extended rule, or iterated rule. For example, the composite trapezoidal rule can be Illustration of Simpson's rule. stated as

where the subintervals have the form [k h, (k+1) h], with h = (b−a)/n and k = 0, 1, 2, ..., n−1. Interpolation with polynomials evaluated at equally spaced points in [a, b] yields the Newton–Cotes formulas, of which the rectangle rule and the trapezoidal rule are examples. Simpson's rule, which is based on a polynomial of order 2, is also a Newton–Cotes formula. Quadrature rules with equally spaced points have the very convenient property of nesting. The corresponding rule with each interval subdivided includes all the current points, so those integrand values can be re-used. Numerical integration 28

If we allow the intervals between interpolation points to vary, we find another group of quadrature formulas, such as the Gaussian quadrature formulas. A Gaussian quadrature rule is typically more accurate than a Newton–Cotes rule, which requires the same number of function evaluations, if the integrand is smooth (i.e., if it is sufficiently differentiable). Other quadrature methods with varying intervals include Clenshaw–Curtis quadrature (also called Fejér quadrature) methods, which do nest. Gaussian quadrature rules do not nest, but the related Gauss–Kronrod quadrature formulas do.

Adaptive algorithms For more details on this topic, see Adaptive quadrature. If f(x) does not have many derivatives at all points, or if the derivatives become large, then Gaussian quadrature is often insufficient. In this case, an algorithm similar to the following will perform better:

def calculate_definite_integral_of_f(f, initial_step_size): ''' This algorithm calculates the definite integral of a function from 0 to 1, adaptively, by choosing smaller steps near problematic points. ''' x = 0.0 h = initial_step_size accumulator = 0.0 while x < 1.0: if x + h > 1.0: h = 1.0 - x quad_this_step = if error_too_big_in_quadrature_of_over_range(f, [x,x+h]): h = make_h_smaller(h) else: accumulator += quadrature_of_f_over_range(f, [x,x+h]) x += h if error_too_small_in_quadrature_of_over_range(f, [x,x+h]): h = make_h_larger(h) # Avoid wasting time on tiny steps. return accumulator

Some details of the algorithm require careful thought. For many cases, estimating the error from quadrature over an interval for a function f(x) isn't obvious. One popular solution is to use two different rules of quadrature, and use their difference as an estimate of the error from quadrature. The other problem is deciding what "too large" or "very small" signify. A local criterion for "too large" is that the quadrature error should not be larger than t · h where t, a real number, is the tolerance we wish to set for global error. Then again, if h is already tiny, it may not be worthwhile to make it even smaller even if the quadrature error is apparently large. A global criterion is that the sum of errors on all the intervals should be less than t. This type of error analysis is usually called "a posteriori" since we compute the error after having computed the approximation. Heuristics for adaptive quadrature are discussed by Forsythe et al. (Section 5.4). Numerical integration 29

Extrapolation methods The accuracy of a quadrature rule of the Newton-Cotes type is generally a function of the number of evaluation points. The result is usually more accurate as number of evaluation points increases, or, equivalently, as the width of the step size between the points decreases. It is natural to ask what the result would be if the step size were allowed to approach zero. This can be answered by extrapolating the result from two or more nonzero step sizes, using series acceleration methods such as Richardson extrapolation. The extrapolation function may be a polynomial or rational function. Extrapolation methods are described in more detail by Stoer and Bulirsch (Section 3.4) and are implemented in many of the routines in the QUADPACK library.

Conservative (a priori) error estimation Let f have a bounded first derivative over [a,b]. The mean value theorem for f, where x < b, gives

for some v in [a,x] depending on x. If we integrate in x from a to b on both sides and take the absolute values, we x obtain

We can further approximate the integral on the right-hand side by bringing the absolute value into the integrand, and replacing the term in f' by an upper bound:

(**)

(See supremum.) Hence, if we approximate the integral ∫ b f(x) dx by the quadrature rule (b − a)f(a) our error is no a greater than the right hand side of (**). We can convert this into an error analysis for the Riemann sum (*), giving an upper bound of

for the error term of that particular approximation. (Note that this is precisely the error we calculated for the example .) Using more derivatives, and by tweaking the quadrature, we can do a similar error analysis using a Taylor series (using a partial sum with remainder term) for f. This error analysis gives a strict upper bound on the error, if the derivatives of f are available. This integration method can be combined with interval arithmetic to produce computer proofs and verified calculations.

Integrals over infinite intervals Several methods exist for approximate integration over unbounded intervals. The standard technique involves specially derived quadrature rules, such as Gauss-Hermite quadrature for integrals on the whole real line and Gauss-Laguerre quadrature for integrals on the positive reals. Monte Carlo methods can also be used, or a change of variables to a finite interval; e.g., for the whole line one could use

and for semi-infinite intervals one could use Numerical integration 30

as possible transformations.

Multidimensional integrals The quadrature rules discussed so far are all designed to compute one-dimensional integrals. To compute integrals in multiple dimensions, one approach is to phrase the multiple integral as repeated one-dimensional integrals by appealing to Fubini's theorem. This approach requires the function evaluations to grow exponentially as the number of dimensions increases. Two methods are known to overcome this so-called curse of dimensionality.

Monte Carlo Main article: Monte Carlo integration Monte Carlo methods and quasi-Monte Carlo methods are easy to apply to multi-dimensional integrals, and may yield greater accuracy for the same number of function evaluations than repeated integrations using one-dimensional methods. A large class of useful Monte Carlo methods are the so-called Markov chain Monte Carlo algorithms, which include the Metropolis-Hastings algorithm and Gibbs sampling.

Sparse grids Sparse grids were originally developed by Smolyak for the quadrature of high-dimensional functions. The method is always based on a one-dimensional quadrature rule, but performs a more sophisticated combination of univariate results.

Connection with differential equations The problem of evaluating the integral

can be reduced to an initial value problem for an ordinary differential equation. If the above integral is denoted by F(x), then the function F satisfies

Methods developed for ordinary differential equations, such as Runge–Kutta methods, can be applied to the restated problem and thus be used to evaluate the integral. For instance, the standard fourth-order Runge–Kutta method applied to the differential equation yields Simpson's rule from above. The differential equation F ' (x) = ƒ(x) has a special form: the right-hand side contains only the dependent variable (here x) and not the independent variable (here F). This simplifies the theory and algorithms considerably. The problem of evaluating integrals is thus best studied in its own right. Numerical integration 31

References • Philip J. Davis and Philip Rabinowitz, Methods of Numerical Integration. • George E. Forsythe, Michael A. Malcolm, and Cleve B. Moler, Computer Methods for Mathematical Computations. Englewood Cliffs, NJ: Prentice-Hall, 1977. (See Chapter 5.) • Press, W.H.; Teukolsky, S.A.; Vetterling, W.T.; Flannery, B.P. (2007), "Chapter 4. Integration of Functions"

(http:/ / apps. nrbook. com/ empanel/ index. html?pg=155), Numerical Recipes: The Art of Scientific Computing (3rd ed.), New York: Cambridge University Press, ISBN 978-0-521-88068-8 • Josef Stoer and Roland Bulirsch, Introduction to Numerical Analysis. New York: Springer-Verlag, 1980. (See Chapter 3.) • Boyer, C. B., A History of Mathematics, 2nd ed. rev. by Uta C. Merzbach, New York: Wiley, 1989 ISBN 0-471-09763-2 (1991 pbk ed. ISBN 0-471-54397-7). • Eves, Howard, An Introduction to the History of Mathematics, Saunders, 1990, ISBN 0-03-029558-0,

External links

• Integration: Background, Simulations, etc. (http:/ / numericalmethods. eng. usf. edu/ mws/ gen/ 07int/ index. html) at Holistic Numerical Methods Institute

Free software for numerical integration Numerical integration is one of the most intensively studied problems in numerical analysis. Of the many software implementations, we list a few free and open source software packages here:

• QUADPACK (part of SLATEC): description (http:/ / www. netlib. org/ slatec/ src/ qpdoc. f), source code (http:/ /

www. netlib. org/ slatec/ src). QUADPACK is a collection of algorithms, in Fortran, for numerical integration based on Gaussian quadrature.

• interalg (http:/ / openopt. org/ interalg): a solver from OpenOpt/FuncDesigner frameworks, based on interval analysis, guaranteed precision, license: BSD (free for any purposes)

• GSL (http:/ / www. gnu. org/ software/ gsl/ ): The GNU Scientific Library (GSL) is a numerical library written in C that provides a wide range of mathematical routines, like Monte Carlo integration.

• Numerical integration algorithms are found in GAMS class H2 (http:/ / gams. nist. gov/ serve. cgi/ Class/ H2).

• ALGLIB (http:/ / www. alglib. net/ integral/ ) is a collection of algorithms, in C# / C++ / Delphi / Visual Basic / etc., for numerical integration (includes Bulirsch-Stoer and Runge-Kutta integrators).

• Cuba (http:/ / www. feynarts. de/ cuba/ ) is a free-software library of several multi-dimensional integration algorithms.

• Cubature (http:/ / ab-initio. mit. edu/ wiki/ index. php/ Cubature) code for adaptive multi-dimensional integration.

• Scilab (http:/ / www. scilab. org/ ) is an open source software under CeCILL license (GPL compatible), providing powerful features including numerical integration. Trapezoidal rule 32 Trapezoidal rule

This article is about the quadrature rule for approximating integrals. For the implicit trapezoidal rule for solving initial value problems, see Trapezoidal rule (differential equations). For the explicit trapezoidal rule for solving initial value problems, see Heun's method. In numerical analysis, the trapezoidal rule (also known as the trapezoid rule or trapezium rule) is a technique for approximating the definite integral

The trapezoidal rule works by approximating the region under the graph of the function as a trapezoid and calculating its area. It follows that

The function f(x) (in blue) is approximated by a linear function (in red).

Applicability and alternatives The trapezoidal rule is one of a family of formulas for numerical integration called Newton–Cotes formulas, of which the midpoint rule is similar to the trapezoid rule. Simpson's rule is another member of the same family, and in general has faster convergence than the trapezoidal rule for functions which are twice continuously differentiable, though not in all specific cases. However for various classes of rougher functions (ones with weaker smoothness conditions), the trapezoidal rule has faster convergence in general than Simpson's rule. Moreover, the trapezoidal rule tends to become extremely accurate when periodic functions are integrated over their periods, which can be analyzed in various ways. For non-periodic functions, however, methods with unequally spaced points such as Gaussian quadrature and Clenshaw–Curtis quadrature are generally far more accurate; Clenshaw–Curtis quadrature can be viewed as a change of variables to express arbitrary integrals in terms of periodic integrals, at which point the trapezoidal rule can be applied accurately.

Numerical implementation

Uniform grid

For a domain discretized into N equally spaced panels, or N+1 grid points a = x < x < ... < x = b, where the grid spacing is h=(b-a)/N, 1 2 N+1 the approximation to the integral becomes

Non-uniform grid Illustration of trapezoidal rule used on a sequence When the grid spacing is non-uniform, one can use the formula of samples (in this case, a non-uniform grid). Trapezoidal rule 33

Error analysis The error of the composite trapezoidal rule is the difference between the value of the integral and the numerical result:

There exists a number ξ between a and b, such that

It follows that if the integrand is concave up (and thus has a positive second derivative), then the error is negative and the trapezoidal rule overestimates the true value. This can also be seen from the geometric picture: the trapezoids include all of the area under the curve and extend over it. Similarly, a concave-down function yields an underestimate because area is unaccounted for under the curve, but none is counted above. If the interval of the integral being approximated includes an inflection point, the error is harder to identify. In general, three techniques are used in the analysis of error: 1. Fourier series 2. Residue calculus 3. Euler–Maclaurin summation formula: An asymptotic error estimate for N → ∞ is given by

Further terms in this error estimate are given by the Euler–Maclaurin summation formula. It is argued that the speed of convergence of the trapezoidal rule reflects and can be used as a definition of classes of smoothness of the functions.

Periodic functions The trapezoidal rule often converges very quickly for periodic functions. This can be explained intuitively as: When the function is periodic and one integrates over one full period, there are about as many sections of the graph that are concave up as concave down, so the errors cancel. In the error formula above, f'(a) = f'(b), and only the O(N–3) term remains. More detailed analysis can be found in. Trapezoidal rule 34

"Rough" functions For various classes of functions that are not twice-differentiable, the trapezoidal rule has sharper bounds than Simpson's rule.

Notes

References

The Wikibook A-level Mathematics has a page on the topic of: Trapezium Rule

• Atkinson, Kendall E. (1989), An Introduction to Numerical Analysis (2nd ed.), New York: John Wiley & Sons, ISBN 978-0-471-50023-0. • Rahman, Qazi I.; Schmeisser, Gerhard (December 1990), "Characterization of the speed of convergence of the

trapezoidal rule", Numerische Mathematik 57 (1): 123–138, doi: 10.1007/BF01386402 (http:/ / dx. doi. org/ 10.

1007/ BF01386402), ISSN 0945-3245 (http:/ / www. worldcat. org/ issn/ 0945-3245) • Burden, Richard L.; J. Douglas Faires (2000), Numerical Analysis (7th ed.), Brooks/Cole, ISBN 0-534-38216-9 . • Weideman, J. A. C. (January 2002), "Numerical Integration of Periodic Functions: A Few Examples", The

American Mathematical Monthly 109 (1): 21–36, doi: 10.2307/2695765 (http:/ / dx. doi. org/ 10. 2307/ 2695765),

JSTOR 2695765 (http:/ / www. jstor. org/ stable/ 2695765) • Cruz-Uribe, D.; Neugebauer, C.J. (2002), "Sharp Error Bounds for the Trapezoidal Rule and Simpson's Rule"

(http:/ / www. emis. de/ journals/ JIPAM/ images/ 031_02_JIPAM/ 031_02. pdf), Journal of Inequalities in Pure

and Applied Mathematics (http:/ / jipam. vu. edu. au/ ) 3 (4)

External links

• Trapezoidal Rule for Numerical Integration (http:/ / math. fullerton. edu/ mathews/ n2003/ TrapezoidalRuleMod. html)

• Notes on the convergence of trapezoidal-rule quadrature (http:/ / dedekind. mit. edu/ ~stevenj/ trapezoidal. pdf) Simpson's rule 35 Simpson's rule

For Simpson's voting rule, see Minimax Condorcet. For Simpson's rules used in Ship Stability, see Simpson's rules. In numerical analysis, Simpson's rule is a method for numerical integration, the numerical approximation of definite integrals. Specifically, it is the following approximation:

Simpson's rule also corresponds to the three-point Newton-Cotes quadrature rule.

The method is credited to the mathematician Thomas Simpson Simpson's rule can be derived by approximating (1710–1761) of Leicestershire, England. Kepler used similar formulas the integrand f (x) (in blue) by the quadratic interpolant P(x) (in red). over 100 years prior. In German, the method is sometimes called Keplersche Fassregel for this reason. Simpson's rule is a staple of scientific data analysis and engineering.Wikipedia:Citation needed

Derivation Simpson's rule can be derived in various ways.

Quadratic interpolation One derivation replaces the integrand by the quadratic polynomial (i.e. parabola) which takes the same values as at the end points a and b and the midpoint m = (a + b) / 2. One can use Lagrange polynomial interpolation to find an expression for this polynomial,

An easy (albeit tedious) calculation shows that

[1]

This calculation can be carried out more easily if one first observes that (by scaling) there is no loss of generality in assuming that and .

Averaging the midpoint and the trapezoidal rules Another derivation constructs Simpson's rule from two simpler approximations: the midpoint rule

and the trapezoidal rule

The errors in these approximations are

respectively, where denotes a term asymptotically proportional to . The two terms are not equal; see Big O notation for more details. It follows from the above formulas for the errors of the midpoint and trapezoidal rule that the leading error term vanishes if we take the weighted average Simpson's rule 36

This weighted average is exactly Simpson's rule. Using another approximation (for example, the trapezoidal rule with twice as many points), it is possible to take a suitable weighted average and eliminate another error term. This is Romberg's method.

Undetermined coefficients The third derivation starts from the ansatz

The coefficients α, β and γ can be fixed by requiring that this approximation be exact for all quadratic polynomials. This yields Simpson's rule.

Error The error in approximating an integral by Simpson's rule is

where is some number between and .[2] The error is asymptotically proportional to . However, the above derivations suggest an error proportional to . Simpson's rule gains an extra order because the points at which the integrand is evaluated are distributed symmetrically in the interval [a, b]. Since the error term is proportional to the fourth derivative of f at , this shows that Simpson's rule provides exact results for any polynomial f of degree three or less, since the fourth derivative of such a polynomial is zero at all points.

Composite Simpson's rule If the interval of integration is in some sense "small", then Simpson's rule will provide an adequate approximation to the exact integral. By small, what we really mean is that the function being integrated is relatively smooth over the interval . For such a function, a smooth quadratic interpolant like the one used in Simpson's rule will give good results. However, it is often the case that the function we are trying to integrate is not smooth over the interval. Typically, this means that either the function is highly oscillatory, or it lacks derivatives at certain points. In these cases, Simpson's rule may give very poor results. One common way of handling this problem is by breaking up the interval into a number of small subintervals. Simpson's rule is then applied to each subinterval, with the results being summed to produce an approximation for the integral over the entire interval. This sort of approach is termed the composite Simpson's rule. Suppose that the interval is split up in subintervals, with an even number. Then, the composite Simpson's rule is given by

where for with ; in particular, and . The above formula can also be written as Simpson's rule 37

The error committed by the composite Simpson's rule is bounded (in absolute value) by

where is the "step length", given by [3] This formulation splits the interval in subintervals of equal length. In practice, it is often advantageous to use subintervals of different lengths, and concentrate the efforts on the places where the integrand is less well-behaved. This leads to the adaptive Simpson's method.

Alternative extended Simpson's rule This is another formulation of a composite Simpson's rule: instead of applying Simpson's rule to disjoint segments of the integral to be approximated, Simpson's rule is applied to overlapping segments, yielding:[4]

The formula above is obtained by combining the original composite Simpson's rule with the one consisting in using Simpson's 3/8 rule in the extreme subintervals and the standard 3-point rule in the remaining subintervals. The result is then obtained by taking the mean of the two formulas.

Simpson's 3/8 rule Simpson's 3/8 rule is another method for numerical integration proposed by Thomas Simpson. It is based upon a cubic interpolation rather than a quadratic interpolation. Simpson's 3/8 rule is as follows:

where b - a = 3h. The error of this method is:

where is some number between and . Thus, the 3/8 rule is about twice as accurate as the standard method, but it uses one more function value. A composite 3/8 rule also exists, similarly as above.[5] A further generalization of this concept for interpolation with arbitrary degree polynomials are the Newton–Cotes formulas.

Simpson's 3/8 rule (for n intervals) Defining,

we have

Note, we can only use this if is a multiple of three. A simplified version of Simpson's rules are used in naval architecture. The 3/8th rule is also called Simpson's Second Rule. Simpson's rule 38

Sample implementation An implementation of the composite Simpson's rule in Python 2:

#!/usr/bin/env python2 from __future__ import division

def simpson(f, a, b, n): """Approximates the definite integral of f from a to b by the composite Simpson's rule, using n subintervals""" h = (b - a) / n s = f(a) + f(b)

for i in range(1, n, 2): s += 4 * f(a + i * h) for i in range(2, n-1, 2): s += 2 * f(a + i * h)

return s * h / 3

print simpson(lambda x:x**9, 0.0, 10.0, 100000) # displays 1000000000.0

Notes [1] Atkinson, p. 256; Süli and Mayers, §7.2 [2] Atkinson, equation (5.1.15); Süli and Mayers, Theorem 7.2 [3] Atkinson, pp. 257+258; Süli and Mayers, §7.5 [4] Press (1989), p. 122 [5] Matthews (2004)

References • Atkinson, Kendall E. (1989). An Introduction to Numerical Analysis (2nd ed.). John Wiley & Sons. ISBN 0-471-50023-2. • Burden, Richard L. and Faires, J. Douglas (2000). Numerical Analysis (7th ed.). Brooks/Cole. ISBN 0-534-38216-9. • McCall Pate (1918). The naval artificer's manual: (The naval artificer's handbook revised) text, questions and

general information for deck (http:/ / books. google. com/ books?id=bQc9AAAAYAAJ& pg=PA198). United States. Bureau of Reconstruction and Repair. p. 198.

• Matthews, John H. (2004). "Simpson's 3/8 Rule for Numerical Integration" (http:/ / math. fullerton. edu/ mathews/

n2003/ Simpson38RuleMod. html). Numerical Analysis - Numerical Methods Project. California State University, Fullerton. Retrieved 11 November 2008. • Press, William H., Brian P. Flannery, William T. Vetterling, and Saul A. Teukolsky (1989). Numerical Recipes in

Pascal: The Art of Scientific Computing (http:/ / books. google. com/ ?id=bh5w6E-M-PUC& pg=PA122& dq=extended-simpson's-rule). Cambridge University Press. ISBN 0-521-37516-9. • Süli, Endre and Mayers, David (2003). An Introduction to Numerical Analysis. Cambridge University Press. ISBN 0-521-00794-1.

• Kaw, Autar; Kalu, Egwu (2008). "Numerical Methods with Applications" (1st ed.). (http:/ / www. autarkaw. com). . Simpson's rule 39

• Weisstein, Eric W. (2010). "Newton-Cotes Formulas" (http:/ / mathworld. wolfram. com/

Newton-CotesFormulas. html). MathWorld--A Wolfram Web Resource. MathWorld. Retrieved 2 August 2010.

External links

• Hazewinkel, Michiel, ed. (2001), "Simpson formula" (http:/ / www. encyclopediaofmath. org/ index. php?title=p/ s085450), Encyclopedia of Mathematics, Springer, ISBN 978-1-55608-010-4

• Weisstein, Eric W., "Simpson's Rule" (http:/ / mathworld. wolfram. com/ SimpsonsRule. html), MathWorld.

• Simpson's Rule for Numerical Integration (http:/ / math. fullerton. edu/ mathews/ n2003/ SimpsonsRuleMod. html)

• Application of Simpson's Rule — Earthwork Excavation (http:/ / www. howtoexcel. info/ Civil_Engineering/

Earthwork_Volume. htm) (Note: The formula described in this page is correct but there are errors in the calculation which should give a result of 569m3 and not 623m3 as stated)

• Simpson's 1/3rd rule of integration — Notes, PPT, Mathcad, Matlab, Mathematica, Maple (http:/ /

numericalmethods. eng. usf. edu/ topics/ simpsons_13rd_rule. html) at Numerical Methods for STEM

undergraduate (http:/ / numericalmethods. eng. usf. edu) • A detailed description of a computer implementation is described by Dorai Sitaram in Teach Yourself Scheme in

Fixnum Days, Appendix C (http:/ / www. ccs. neu. edu/ home/ dorai/ t-y-scheme/ t-y-scheme-Z-H-22. html#node_chap_C)

• C Language Program to Implement Simpson's Rule (http:/ / hamsoft-hq. com/ toys/ junk. php?id=4) This article incorporates material from Code for Simpson's rule on PlanetMath, which is licensed under the Creative Commons Attribution/Share-Alike License. Bisection method 40 Bisection method

This article is about searching continuous function values. For searching a finite sorted array, see binary search algorithm. The bisection method in mathematics is a root-finding method that repeatedly bisects an interval and then selects a subinterval in which a root must lie for further processing. It is a very simple and robust method, but it is also relatively slow. Because of this, it is often used to obtain a rough approximation to a solution which is then used as a starting point for more rapidly converging methods. The method is also called the interval halving method,[1] the binary search method, or the dichotomy method.[2]

The method

The method is applicable for solving the equation f(x) = 0 for the real variable x, where f is a continuous function defined on an interval [a, b] and f(a) and f(b) have opposite signs. In this case a and b are said to bracket a root since, by the intermediate value theorem, the continuous function f must have at least one root in A few steps of the bisection method applied over the starting range [a ;b ]. The bigger red dot is the root of the function. the interval (a, b). 1 1

At each step the method divides the interval in two by computing the midpoint c = (a+b) / 2 of the interval and the value of the function f(c) at that point. Unless c is itself a root (which is very unlikely, but possible) there are now two possibilities: either f(a) and f(c) have opposite signs and bracket a root, or f(c) and f(b) have opposite signs and bracket a root. The method selects the subinterval that is a bracket as a new interval to be used in the next step. In this way the interval that contains a zero of f is reduced in width by 50% at each step. The process is continued until the interval is sufficiently small.

Explicitly, if f(a) and f(c) are opposite signs, then the method sets c as the new value for b, and if f(b) and f(c) are opposite signs then the method sets c as the new a. (If f(c)=0 then c may be taken as the solution and the process stops.) In both cases, the new f(a) and f(b) have opposite signs, so the method is applicable to this smaller interval.[3]

Iteration tasks The input for the method is a continuous function f, an interval [a, b], and the function values f(a) and f(b). The function values are of opposite sign (there is at least one zero crossing within the interval). Each iteration performs these steps: 1. Calculate c, the midpoint of the interval, c = 0.5 * (a + b). 2. Calculate the function value at the midpoint, f(c). 3. If convergence is satisfactory (that is, a - c is sufficiently small, or f(c) is sufficiently small), return c and stop iterating. 4. Examine the sign of f(c) and replace either (a, f(a)) or (b, f(b)) with (c, f(c)) so that there is a zero crossing within the new interval. Bisection method 41

When implementing the method on a computer, there can be problems with finite precision, so there are often additional convergence tests or limits to the number of iterations. Although f is continuous, finite precision may preclude a function value ever being zero. For f(x) = x − π, there will never be a finite representation of x that gives zero. Floating point representations also have limited precision, so at some point the midpoint of [a, b] will be either a or b.

Algorithm The method may be written in pseudocode as follows:[4]

INPUT: Function f, endpoint values a, b, tolerance TOL, maximum iterations NMAX CONDITIONS: a < b, either f(a) < 0 and f(b) > 0 or f(a) > 0 and f(b) < 0 OUTPUT: value which differs from a root of f(x)=0 by less than TOL

N ← 1 While N ≤ NMAX # limit iterations to prevent infinite loop c ← (a + b)/2 # new midpoint If f(c) = 0 or (b – a)/2 < TOL then # solution found Output(c) Stop EndIf N ← N + 1 # increment step counter If sign(f(c)) = sign(f(a)) then a ← c else b ← c # new interval EndWhile Output("Method failed.") # max number of steps exceeded

Example: Finding the root of a polynomial Suppose that the bisection method is used to find a root of the polynomial

First, two numbers and have to be found such that and have opposite signs. For the above function, and satisfy this criterion, as

and

Because the function is continuous, there must be a root within the interval [1, 2]. In the first iteration, the end points of the interval which brackets the root are and , so the midpoint is

The function value at the midpoint is . Because is negative, is replaced with for the next iteration to ensure that and have opposite signs. As this continues, the interval between and will become increasingly smaller, converging on the root of the function. See this happen in the table below. Bisection method 42

Iteration

1 1 2 1.5 −0.125

2 1.5 2 1.75 1.6093750

3 1.5 1.75 1.625 0.6660156

4 1.5 1.625 1.5625 0.2521973

5 1.5 1.5625 1.5312500 0.0591125

6 1.5 1.5312500 1.5156250 −0.0340538

7 1.5156250 1.5312500 1.5234375 0.0122504

8 1.5156250 1.5234375 1.5195313 −0.0109712

9 1.5195313 1.5234375 1.5214844 0.0006222

10 1.5195313 1.5214844 1.5205078 −0.0051789

11 1.5205078 1.5214844 1.5209961 −0.0022794

12 1.5209961 1.5214844 1.5212402 −0.0008289

13 1.5212402 1.5214844 1.5213623 −0.0001034

14 1.5213623 1.5214844 1.5214233 0.0002594

15 1.5213623 1.5214233 1.5213928 0.0000780

After 13 iterations, it becomes apparent that there is a convergence to about 1.521: a root for the polynomial.

Analysis The method is guaranteed to converge to a root of f if f is a continuous function on the interval [a, b] and f(a) and f(b) have opposite signs. The absolute error is halved at each step so the method converges linearly, which is comparatively slow. Specifically, if c = (a+b)/2 is the midpoint of the initial interval, and c is the midpoint of the interval in the nth step, 1 n then the difference between c and a solution c is bounded by[5] n

This formula can be used to determine in advance the number of iterations that the bisection method would need to converge to a root to within a certain tolerance. The number of iterations needed, n, to achieve a given error (or tolerance), ε, is given by:

where Therefore, the linear convergence is expressed by Bisection method 43

References

[1] http:/ / siber. cankaya. edu. tr/ NumericalComputations/ ceng375/ node32. html

[2] Encyclopedia of Mathematics (http:/ / www. encyclopediaofmath. org/ index. php/ Dichotomy_method) [3] for section [4] . This version recomputes the function values at each iteration rather than carrying them to the next iterations. [5] , Theorem 2.1 • Burden, Richard L.; Faires, J. Douglas (1985), "2.1 The Bisection Algorithm", Numerical Analysis (3rd ed.), PWS Publishers, ISBN 0-87150-857-5. • Corliss, George (1977), "Which root does the bisection algorithm find?", SIAM Review 19 (2): 325–327, doi:

10.1137/1019044 (http:/ / dx. doi. org/ 10. 1137/ 1019044), ISSN 1095-7200 (http:/ / www. worldcat. org/ issn/ 1095-7200).

• Kaw, Autar; Kalu, Egwu (2008), Numerical Methods with Applications (http:/ / numericalmethods. eng. usf. edu/

topics/ textbook_index. html) (1st ed.)

External links

Wikiversity has learning materials about The bisection method

The Wikibook Numerical Methods has a page on the topic of: Equation Solving

• Weisstein, Eric W., "Bisection" (http:/ / mathworld. wolfram. com/ Bisection. html), MathWorld.

• Bisection Method (http:/ / numericalmethods. eng. usf. edu/ topics/ bisection_method. html) Notes, PPT,

Mathcad, Maple, Matlab, Mathematica from Holistic Numerical Methods Institute (http:/ / numericalmethods.

eng. usf. edu)

• Module for the Bisection Method by John H. Mathews (http:/ / math. fullerton. edu/ mathews/ n2003/

BisectionMod. html)

• Online root finding of a polynomial-Bisection method (http:/ / catc. ac. ir/ mazlumi/ jscodes/ bisection. php) by Farhad Mazlumi Newton's method 44 Newton's method

This article is about Newton's method for finding roots. For Newton's method for finding minima, see Newton's method in optimization. In numerical analysis, Newton's method (also known as the Newton–Raphson method), named after Isaac Newton and Joseph Raphson, is a method for finding successively better approximations to the roots (or zeroes) of a real-valued function.

The Newton–Raphson method in one variable is implemented as follows: Given a function ƒ defined over the reals x, and its derivative ƒ', we begin with a first guess x for a root of the 0 function f. Provided the function satisfies all the assumptions made in the derivation of the formula, a better approximation x is 1

Geometrically, (x , 0) is the intersection with the x-axis of the tangent to the graph of f at (x , f (x )). 1 0 0 The process is repeated as

until a sufficiently accurate value is reached. This algorithm is first in the class of Householder's methods, succeeded by Halley's method. The method can also be extended to complex functions and to systems of equations.

Description

The idea of the method is as follows: one starts with an initial guess which is reasonably close to the true root, then the function is approximated by its tangent line (which can be computed using the tools of calculus), and one computes the x-intercept of this tangent line (which is easily done with elementary algebra). This x-intercept will typically be a better approximation to the function's root than the original guess, and the method can be iterated.

Suppose ƒ : [a, b] → R is a differentiable function defined on the interval [a, b] with The function ƒ is shown in blue and the tangent line is in red. We see that x is a values in the real numbers R. The formula n+1 better approximation than x for the root x of the function f. for converging on the root can be easily n derived. Suppose we have some current approximation x . Then we can derive the formula for a better approximation, x by referring to the diagram on the n n+1 right. The equation of the tangent line to the curve y = ƒ(x) at the point x=x is n

where, ƒ' denotes the derivative of the function ƒ. Newton's method 45

The x-intercept of this line (the value of x such that y=0) is then used as the next approximation to the root, x . In n+1 other words, setting y to zero and x to x gives n+1

Solving for x gives n+1

We start the process off with some arbitrary initial value x . (The closer to the zero, the better. But, in the absence of 0 any intuition about where the zero might lie, a "guess and check" method might narrow the possibilities to a reasonably small interval by appealing to the intermediate value theorem.) The method will usually converge, provided this initial guess is close enough to the unknown zero, and that ƒ'(x ) ≠ 0. Furthermore, for a zero of 0 multiplicity 1, the convergence is at least quadratic (see rate of convergence) in a neighbourhood of the zero, which intuitively means that the number of correct digits roughly at least doubles in every step. More details can be found in the analysis section below. The Householder's methods are similar but have higher order for even faster convergence. However, the extra computations required for each step can slow down the overall performance relative to Newton's method, particularly if f or its derivatives are computationally expensive to evaluate.

History The name "Newton's method" is derived from Isaac Newton's description of a special case of the method in De analysi per aequationes numero terminorum infinitas (written in 1669, published in 1711 by William Jones) and in De metodis fluxionum et serierum infinitarum (written in 1671, translated and published as Method of Fluxions in 1736 by John Colson). However, his method differs substantially from the modern method given above: Newton applies the method only to polynomials. He does not compute the successive approximations , but computes a sequence of polynomials, and only at the end arrives at an approximation for the root x. Finally, Newton views the method as purely algebraic and makes no mention of the connection with calculus. Newton may have derived his method from a similar but less precise method by Vieta. The essence of Vieta's method can be found in the work of the Persian mathematician Sharaf al-Din al-Tusi, while his successor Jamshīd al-Kāshī used a form of Newton's method to solve to find roots of N (Ypma 1995). A special case of Newton's method for calculating square roots was known much earlier and is often called the Babylonian method. Newton's method was used by 17th-century Japanese mathematician Seki Kōwa to solve single-variable equations, though the connection with calculus was missing. Newton's method was first published in 1685 in A Treatise of Algebra both Historical and Practical by John Wallis. In 1690, Joseph Raphson published a simplified description in Analysis aequationum universalis. Raphson again viewed Newton's method purely as an algebraic method and restricted its use to polynomials, but he describes the method in terms of the successive approximations x instead of the more complicated sequence of polynomials used n by Newton. Finally, in 1740, Thomas Simpson described Newton's method as an iterative method for solving general nonlinear equations using calculus, essentially giving the description above. In the same publication, Simpson also gives the generalization to systems of two equations and notes that Newton's method can be used for solving optimization problems by setting the gradient to zero. Arthur Cayley in 1879 in The Newton-Fourier imaginary problem was the first to notice the difficulties in generalizing Newton's method to complex roots of polynomials with degree greater than 2 and complex initial values. This opened the way to the study of the theory of iterations of rational functions. Newton's method 46

Practical considerations Newton's method is an extremely powerful technique—in general the convergence is quadratic: as the method converges on the root, the difference between the root and the approximation is squared (the number of accurate digits roughly doubles) at each step. However, there are some difficulties with the method.

Difficulty in calculating derivative of a function Newton's method requires that the derivative be calculated directly. An analytical expression for the derivative may not be easily obtainable and could be expensive to evaluate. In these situations, it may be appropriate to approximate the derivative by using the slope of a line through two nearby points on the function. Using this approximation would result in something like the secant method whose convergence is slower than that of Newton's method.

Failure of the method to converge to the root It is important to review the proof of quadratic convergence of Newton's Method before implementing it. Specifically, one should review the assumptions made in the proof. For situations where the method fails to converge, it is because the assumptions made in this proof are not met.

Overshoot If the first derivative is not well behaved in the neighborhood of a particular root, the method may overshoot, and diverge from that root. An example of a function with one root, for which the derivative is not well behaved in the neighborhood of the root, is

for which the root will be overshot and the sequence of x will diverge. For a = 1/2, the root will still be overshot, but the sequence will oscillate between two values. For 1/2 < a < 1, the root will still be overshot but the sequence will converge, and for a ≥ 1 the root will not be overshot at all. In some cases, Newton's method can be stabilized by using successive over-relaxation, or the speed of convergence can be increased by using the same method.

Stationary point If a stationary point of the function is encountered, the derivative is zero and the method will terminate due to division by zero.

Poor initial estimate A large error in the initial estimate can contribute to non-convergence of the algorithm.

Mitigation of non-convergence In a robust implementation of Newton's method, it is common to place limits on the number of iterations, bound the solution to an interval known to contain the root, and combine the method with a more robust root finding method.

Slow convergence for roots of multiplicity > 1 If the root being sought has multiplicity greater than one, the convergence rate is merely linear (errors reduced by a constant factor at each step) unless special steps are taken. When there are two or more roots that are close together then it may take many iterations before the iterates get close enough to one of them for the quadratic convergence to be apparent. However, if the multiplicity of the root is known, one can use the following modified algorithm that preserves the quadratic convergence rate: Newton's method 47

This is equivalent to using successive over-relaxation. On the other hand, if the multiplicity of the root is not known, it is possible to estimate after carrying out one or two iterations, and then use that value to increase the rate of convergence.

Analysis Suppose that the function ƒ has a zero at α, i.e., ƒ(α) = 0. If f is continuously differentiable and its derivative is nonzero at α, then there exists a neighborhood of α such that for all starting values x in that neighborhood, the sequence {x } will converge to α.Wikipedia:Citation needed 0 n If the function is continuously differentiable and its derivative is not 0 at α and it has a second derivative at α then the convergence is quadratic or faster. If the second derivative is not 0 at α then the convergence is merely quadratic. If the third derivative exists and is bounded in a neighborhood of α, then:

where If the derivative is 0 at α, then the convergence is usually only linear. Specifically, if ƒ is twice continuously differentiable, ƒ '(α) = 0 and ƒ ''(α) ≠ 0, then there exists a neighborhood of α such that for all starting values x in that 0 neighborhood, the sequence of iterates converges linearly, with rate log 2 (Süli & Mayers, Exercise 1.6). 10 Alternatively if ƒ '(α) = 0 and ƒ '(x) ≠ 0 for x ≠ α, x in a neighborhood U of α, α being a zero of multiplicity r, and if ƒ ∈ Cr(U) then there exists a neighborhood of α such that for all starting values x in that neighborhood, the sequence 0 of iterates converges linearly. However, even linear convergence is not guaranteed in pathological situations. In practice these results are local, and the neighborhood of convergence is not known in advance. But there are also some results on global convergence: for instance, given a right neighborhood U of α, if f is twice differentiable in + U and if , in U , then, for each x in U the sequence x is monotonically decreasing to α. + + 0 + k

Proof of quadratic convergence for Newton's iterative method According to Taylor's theorem, any function f(x) which has a continuous second derivative can be represented by an expansion about a point that is close to a root of f(x). Suppose this root is Then the expansion of f(α) about x is: n

(1)

where the Lagrange form of the Taylor series expansion remainder is

where ξ is in between x and n n Since is the root, (1) becomes:

(2)

Dividing equation (2) by and rearranging gives

(3) Newton's method 48

Remembering that x is defined by n+1

(4)

one finds that

That is,

(5)

Taking absolute value of both sides gives

(6)

Equation (6) shows that the rate of convergence is quadratic if the following conditions are satisfied: 1. 2. 3. sufficiently close to the root The term sufficiently close in this context means the following: (a) Taylor approximation is accurate enough such that we can ignore higher order terms,

(b)

(c)

Finally, (6) can be expressed in the following way:

where M is the supremum of the variable coefficient of on the interval defined in the condition 1, that is:

The initial point has to be chosen such that conditions 1 through 3 are satisfied, where the third condition requires that

Basins of attraction The basins of attraction—the regions of the real number line such that within each region iteration from any point leads to one particular root—can be infinite in number and arbitrarily small. For example,[1] for the function , the following initial conditions are in successive basins of attraction: 2.35287527 converges to 4; 2.35284172 converges to −3; 2.35283735 converges to 4; 2.352836327 converges to −3; 2.352836323 converges to 1. Newton's method 49

Failure analysis Newton's method is only guaranteed to converge if certain conditions are satisfied. If the assumptions made in the proof of quadratic convergence are met, the method will converge. For the following subsections, failure of the method to converge indicates that the assumptions made in the proof were not met.

Bad starting points In some cases the conditions on the function that are necessary for convergence are satisfied, but the point chosen as the initial point is not in the interval where the method converges. This can happen, for example, if the function whose root is sought approaches zero asymptotically as x goes to or . In such cases a different method, such as bisection, should be used to obtain a better estimate for the zero to use as an initial point.

Iteration point is stationary Consider the function:

It has a maximum at x = 0 and solutions of f(x) = 0 at x = ±1. If we start iterating from the stationary point x = 0 0 (where the derivative is zero), x will be undefined, since the tangent at (0,1) is parallel to the x-axis: 1

The same issue occurs if, instead of the starting point, any iteration point is stationary. Even if the derivative is small but not zero, the next iteration will be a far worse approximation.

Starting point enters a cycle

For some functions, some starting points may enter an infinite cycle, preventing convergence. Let

and take 0 as the starting point. The first iteration produces 1 and the second iteration returns to 0 so the sequence will alternate between the two without converging to a root. In fact, this 2-cycle is stable: there are neighborhoods around 0 and around 1 from which all points iterate asymptotically to the 2-cycle (and hence not to the root of the function). In general, the behavior of the sequence can be very complex (see Newton fractal).

Derivative issues

If the function is not continuously The tangent lines of x3 - 2x + 2 at 0 and 1 intersect the x-axis at 1 and 0 differentiable in a neighborhood of the root respectively, illustrating why Newton's method oscillates between these values for then it is possible that Newton's method will some starting points. always diverge and fail, unless the solution is guessed on the first try. Newton's method 50

Derivative does not exist at root A simple example of a function where Newton's method diverges is the cube root, which is continuous and infinitely differentiable, except for x = 0, where its derivative is undefined (this, however, does not affect the algorithm, since it will never require the derivative if the solution is already found):

For any iteration point x , the next iteration point will be: n

The algorithm overshoots the solution and lands on the other side of the y-axis, farther away than it initially was; applying Newton's method actually doubles the distances from the solution at each iteration.

In fact, the iterations diverge to infinity for every , where . In the limiting case of (square root), the iterations will alternate indefinitely between points x and −x , so they do not converge in 0 0 this case either.

Discontinuous derivative If the derivative is not continuous at the root, then convergence may fail to occur in any neighborhood of the root. Consider the function

Its derivative is:

Within any neighborhood of the root, this derivative keeps changing sign as x approaches 0 from the right (or from the left) while f(x) ≥ x − x2 > 0 for 0 < x < 1. So f(x)/f'(x) is unbounded near the root, and Newton's method will diverge almost everywhere in any neighborhood of it, even though: • the function is differentiable (and thus continuous) everywhere; • the derivative at the root is nonzero; • f is infinitely differentiable except at the root; and • the derivative is bounded in a neighborhood of the root (unlike f(x)/f'(x)).

Non-quadratic convergence In some cases the iterates converge but do not converge as quickly as promised. In these cases simpler methods converge just as quickly as Newton's method.

Zero derivative If the first derivative is zero at the root, then convergence will not be quadratic. Indeed, let

then and consequently . So convergence is not quadratic, even though the function is infinitely differentiable everywhere. Similar problems occur even when the root is only "nearly" double. For example, let Newton's method 51

Then the first few iterates starting at x = 1 are 1, 0.500250376, 0.251062828, 0.127507934, 0.067671976, 0 0.041224176, 0.032741218, 0.031642362; it takes six iterations to reach a point where the convergence appears to be quadratic.

No second derivative If there is no second derivative at the root, then convergence may fail to be quadratic. Indeed, let

Then

And

except when where it is undefined. Given ,

which has approximately 4/3 times as many bits of precision as has. This is less than the 2 times as many which would be required for quadratic convergence. So the convergence of Newton's method (in this case) is not quadratic, even though: the function is continuously differentiable everywhere; the derivative is not zero at the root; and is infinitely differentiable except at the desired root.

Generalizations

Complex functions

Main article: Newton fractal When dealing with complex functions, Newton's method can be directly applied to find their zeroes. Each zero has a basin of attraction in the complex plane, the set of all starting values that cause the method to converge to that particular zero. These sets can be mapped as in the image shown. For many complex functions, the boundaries of the basins of attraction are fractals.

In some cases there are regions in the complex plane which are not in any of these basins of attraction, meaning the iterates do not converge. For example,[2] if one uses a real initial condition to seek a root of , all subsequent iterates will be real numbers and so the Basins of attraction for x5 - 1 = 0; darker means iterations cannot converge to either root, since both roots are non-real. more iterations to converge. In this case almost all initial conditions lead to chaotic behavior, while some initial conditions iterate either to infinity or to repeating cycles of any finite length. Newton's method 52

Nonlinear systems of equations

k variables, k functions One may also use Newton's method to solve systems of k (non-linear) equations, which amounts to finding the zeroes of continuously differentiable functions F : Rk → Rk. In the formulation given above, one then has to left multiply with the inverse of the k-by-k Jacobian matrix J (x ) instead of dividing by f '(x ). F n n Rather than actually computing the inverse of this matrix, one can save time by solving the system of linear equations

for the unknown x − x . n+1 n

k variables, m equations, with m > k The k-dimensional Newton's method can be used to solve systems of >k (non-linear) equations as well if the algorithm uses the generalized inverse of the non-square Jacobian matrix J+ = ((JTJ)−1)JT instead of the inverse of J. If the nonlinear system has no solution, the method attempts to find a solution in the non-linear least squares sense. See Gauss–Newton algorithm for more information.

Nonlinear equations in a Banach space Another generalization is Newton's method to find a root of a functional F defined in a Banach space. In this case the formulation is

where is the Fréchet derivative computed at . One needs the Fréchet derivative to be boundedly invertible at each in order for the method to be applicable. A condition for existence of and convergence to a root is given by the Newton–Kantorovich theorem.

Nonlinear equations over p-adic numbers In p-adic analysis, the standard method to show a polynomial equation in one variable has a p-adic root is Hensel's lemma, which uses the recursion from Newton's method on the p-adic numbers. Because of the more stable behavior of addition and multiplication in the p-adic numbers compared to the real numbers (specifically, the unit ball in the p-adics is a ring), convergence in Hensel's lemma can be guaranteed under much simpler hypotheses than in the classical Newton's method on the real line.

Newton-Fourier method Assume that is twice continuously differentiable on and that contains a root in this interval. Assume that on this interval (this is the case for instance if , , and , and on this interval). This guarantees that there is a unique root on this interval, call it . If it is concave down instead of concave up then replace by since they have the same roots. Let be the right endpoint of the interval and let be the left endpoint of the interval. Given ,

define , which is just Newton's method as before. Then define and

note that the denominator has and not . The iterates will be strictly decreasing to the root while

the iterates will be strictly increasing to the root. Also, so that distance

between and decreases quadratically. Newton's method 53

Quasi-Newton methods When the Jacobian is unavailable or too expensive to compute at every iteration, a Quasi-Newton_method can be used.

Applications

Minimization and maximization problems Main article: Newton's method in optimization Newton's method can be used to find a minimum or maximum of a function. The derivative is zero at a minimum or maximum, so minima and maxima can be found by applying Newton's method to the derivative. The iteration becomes:

Multiplicative inverses of numbers and power series An important application is Newton–Raphson division, which can be used to quickly find the reciprocal of a number, using only multiplication and subtraction. Finding the reciprocal of a amounts to finding the root of the function

Newton's iteration is

Therefore, Newton's iteration needs only two multiplications and one subtraction. This method is also very efficient to compute the multiplicative inverse of a power series.

Solving transcendental equations Many transcendental equations can be solved using Newton's method. Given the equation

with g(x) and/or h(x) a transcendental function, one writes

The values of x that solves the original equation are then the roots of f(x), which may be found via Newton's method. Newton's method 54

Examples

Square root of a number Consider the problem of finding the square root of a number. Newton's method is one of many methods of computing square roots. For example, if one wishes to find the square root of 612, this is equivalent to finding the solution to

The function to use in Newton's method is then,

with derivative,

With an initial guess of 10, the sequence given by Newton's method is

Where the correct digits are underlined. With only a few iterations one can obtain a solution accurate to many decimal places.

Solution of cos(x) = x3 Consider the problem of finding the positive number x with cos(x) = x3. We can rephrase that as finding the zero of f(x) = cos(x) − x3. We have f'(x) = −sin(x) − 3x2. Since cos(x) ≤ 1 for all x and x3 > 1 for x > 1, we know that our solution lies between 0 and 1. We try a starting value of x = 0.5. (Note that a starting value of 0 will lead to an 0 undefined result, showing the importance of using a starting point that is close to the solution.)

The correct digits are underlined in the above example. In particular, x is correct to the number of decimal places 6 given. We see that the number of correct digits after the decimal point increases from 2 (for x ) to 5 and 10, 3 illustrating the quadratic convergence. Newton's method 55

Pseudocode The following is an example of using the Newton's Method to help find a root of a function f which has derivative fprime. The initial guess will be and the function will be so that . Each new iterative of Newton's method will be denoted by x1. We will check during the computation whether the denominator (yprime) becomes too small (smaller than epsilon), which would be the case if , since otherwise a large amount of error could be introduced.

%These choices depend on the problem being solved x0 = 1 %The initial value f = @(x) x^2 - 2 %The function whose root we are trying to find fprime = @(x) 2*x %The derivative of f(x) tolerance = 10^(-7) %7 digit accuracy is desired epsilon = 10^(-14) %Don't want to divide by a number smaller than this

maxIterations = 20 %Don't allow the iterations to continue indefinitely haveWeFoundSolution = false %Were we able to find the solution to the desired tolerance? not yet.

for i = 1 : maxIterations

y = f(x0) yprime = fprime(x0)

if(abs(yprime) < epsilon) %Don't want to divide by too small of a number fprintf('WARNING: denominator is too small\n') break; %Leave the loop end

x1 = x0 - y/yprime %Do Newton's computation

if(abs(x1 - x0)/abs(x1) < tolerance) %If the result is within the desired tolerance haveWeFoundSolution = true break; %Done, so leave the loop end

x0 = x1 %Update x0 to start the process again

end

if (haveWeFoundSolution) % We found a solution within tolerance and Newton's method 56

maximum number of iterations fprintf('The root is: %f\n', x1); else %If we weren't able to find a solution to within the desired tolerance fprintf('Warning: Not able to find solution to within the desired tolerance of %f\n', tolerance); fprintf('The last computed approximate root was %f\n', x1) end

References [1] Dence, Thomas, "Cubics, chaos and Newton's method", Mathematical Gazette 81, November 1997, 403-408. [2] Strang, Gilbert, "A chaotic search for i", ''The College Mathematics Journal 22, January 1991, pp. 3-12 (esp. p. 6). • Kendall E. Atkinson, An Introduction to Numerical Analysis, (1989) John Wiley & Sons, Inc, ISBN 0-471-62489-6 • Tjalling J. Ypma, Historical development of the Newton-Raphson method, SIAM Review 37 (4), 531–551, 1995.

doi: 10.1137/1037125 (http:/ / dx. doi. org/ 10. 1137/ 1037125). • Bonnans, J. Frédéric; Gilbert, J. Charles; Lemaréchal, Claude; Sagastizábal, Claudia A. (2006). Numerical

optimization: Theoretical and practical aspects (http:/ / www. springer. com/ mathematics/ applications/ book/ 978-3-540-35445-1). Universitext (Second revised ed. of translation of 1997 French ed.). Berlin: Springer-Verlag.

pp. xiv+490. doi: 10.1007/978-3-540-35447-5 (http:/ / dx. doi. org/ 10. 1007/ 978-3-540-35447-5).

ISBN 3-540-35445-X. MR 2265882 (http:/ / www. ams. org/ mathscinet-getitem?mr=2265882). • P. Deuflhard, Newton Methods for Nonlinear Problems. Affine Invariance and Adaptive Algorithms. Springer Series in Computational Mathematics, Vol. 35. Springer, Berlin, 2004. ISBN 3-540-21099-7. • C. T. Kelley, Solving Nonlinear Equations with Newton's Method, no 1 in Fundamentals of Algorithms, SIAM, 2003. ISBN 0-89871-546-6. • J. M. Ortega, W. C. Rheinboldt, Iterative Solution of Nonlinear Equations in Several Variables. Classics in Applied Mathematics, SIAM, 2000. ISBN 0-89871-461-3. • Press, WH; Teukolsky, SA; Vetterling, WT; Flannery, BP (2007). "Chapter 9. Root Finding and Nonlinear Sets of

Equations Importance Sampling" (http:/ / apps. nrbook. com/ empanel/ index. html#pg=442). Numerical Recipes: The Art of Scientific Computing (3rd ed.). New York: Cambridge University Press. ISBN 978-0-521-88068-8..

See especially Sections 9.4 (http:/ / apps. nrbook. com/ empanel/ index. html#pg=456), 9.6 (http:/ / apps. nrbook.

com/ empanel/ index. html#pg=473), and 9.7 (http:/ / apps. nrbook. com/ empanel/ index. html#pg=477). • Endre Süli and David Mayers, An Introduction to Numerical Analysis, Cambridge University Press, 2003. ISBN 0-521-00794-1. • Kaw, Autar; Kalu, Egwu (2008). Numerical Methods with Applications (1st ed.).

External links

Wikimedia Commons has media related to Newton Method.

• Hazewinkel, Michiel, ed. (2001), "Newton method" (http:/ / www. encyclopediaofmath. org/ index. php?title=p/ n066560), Encyclopedia of Mathematics, Springer, ISBN 978-1-55608-010-4

• Weisstein, Eric W., "Newton's Method" (http:/ / mathworld. wolfram. com/ NewtonsMethod. html), MathWorld. Root-finding algorithm 57 Root-finding algorithm

A root-finding algorithm is a numerical method, or algorithm, for finding a value x such that f(x) = 0, for a given function f. Such an x is called a root of the function f. This article is concerned with finding scalar, real or complex roots, approximated as floating point numbers. Finding integer roots or exact algebraic roots are separate problems, whose algorithms have little in common with those discussed here. (See: Diophantine equation for integer roots) Finding a root of f(x) − g(x) = 0 is the same as solving the equation f(x) = g(x). Here, x is called the unknown in the equation. Conversely, any equation can take the canonical form f(x) = 0, so equation solving is the same thing as computing (or finding) a root of a function. Numerical root-finding methods use iteration, producing a sequence of numbers that hopefully converge towards a limit (the so-called "fixed point") which is a root. The first values of this series are initial guesses. The method computes subsequent values based on the old ones and the function f. The behaviour of root-finding algorithms is studied in numerical analysis. Algorithms perform best when they take advantage of known characteristics of the given function. Thus an algorithm to find isolated real roots of a low-degree polynomial in one variable may bear little resemblance to an algorithm for complex roots of a "black-box" function which is not even known to be differentiable. Questions include ability to separate close roots, robustness against failures of continuity and differentiability, reliability despite inevitable numerical errors, and rate of convergence.

Bracketing methods Bracketing methods track the end points of an interval containing a root. This allows them to provide absolute error bounds on a root's location when the function is known to be continuous. Bracketing methods require two initial conditions, one on either side of the root.

Bisection method The simplest root-finding algorithm is the bisection method. It works when f is a continuous function and it requires previous knowledge of two initial guesses, a and b, such that f(a) and f(b) have opposite signs. Although it is reliable, it converges slowly, gaining one bit of accuracy with each iteration.

False position (regula falsi) The false position method, also called the regula falsi method, is like the secant method. However, instead of retaining the last two points, it makes sure to keep one point on either side of the root. The false position method can be faster than the bisection method and will never diverge like the secant method, but fails to converge under some naive implementations. Ridders' method is a variant on the false-position method that also evaluates the function at the midpoint of the interval, giving faster convergence with similar robustness.

Interpolation Regula falsi is an interpolation method because it approximates the function with a line between two points. Higher degree polynomials can also be used to approximate the function and its root, while bracketing the root. For example, Muller's method can be easily modified so that rather than always keeping the last 3 points, it tracks the last two points to bracket the root and the best current approximation. Such methods combine good average performance with absolute bounds on the worst-case performance. Root-finding algorithm 58

Open methods

Newton's method (and similar derivative-based methods) Newton's method assumes the function f to have a continuous derivative. Newton's method may not converge if started too far away from a root. However, when it does converge, it is faster than the bisection method, and is usually quadratic. Newton's method is also important because it readily generalizes to higher-dimensional problems. Newton-like methods with higher orders of convergence are the Householder's methods. The first one after Newton's method is Halley's method with cubic order of convergence.

Secant method Replacing the derivative in Newton's method with a finite difference, we get the secant method. This method does not require the computation (nor the existence) of a derivative, but the price is slower convergence (the order is approximately 1.6). A generalization of the secant method in higher dimensions is Broyden's method.

Interpolation The secant method also arises if one approximates the unknown function f by linear interpolation. When quadratic interpolation is used instead, one arrives at Muller's method. It converges faster than the secant method. A particular feature of this method is that the iterates x may become complex. n Sidi's method allows for interpolation with an arbitrarily high degree polynomial. The higher the degree of the interpolating polynomial, the faster the convergence. Sidi's method allows for convergence with an order arbitrarily close to 2.

Inverse interpolation The appearance of complex values in interpolation methods can be avoided by interpolating the inverse of f, resulting in the inverse quadratic interpolation method. Again, convergence is asymptotically faster than the secant method, but inverse quadratic interpolation often behaves poorly when the iterates are not close to the root.

Combinations of methods

Brent's method Brent's method is a combination of the bisection method, the secant method and inverse quadratic interpolation. At every iteration, Brent's method decides which method out of these three is likely to do best, and proceeds by doing a step according to that method. This gives a robust and fast method, which therefore enjoys considerable popularity.

Finding roots of polynomials Much attention has been given to the special case that the function f is a polynomial, and there are several root-finding algorithms for polynomials. For univariate polynomials of degrees one (linear polynomial) through four (quartic polynomial), there are closed-form solutions which produce all roots. Linear polynomials are easy to solve, but using the quadratic formula to solve quadratic (second degree) equations may require some care to ensure numerical stability. The closed-form solutions for degrees three (cubic polynomial) and four (quartic polynomial) are complicated and require much more care; consequently, numerical methods such as Bairstow's method may be easier to use. Fifth-degree (quintic) and higher-degree polynomials do not have a general solution according to the Abel–Ruffini theorem (1824, 1799). Root-finding algorithm 59

Finding one root at a time The general idea is to find a root of the polynomial and then apply Horner's method to remove the corresponding factor according to the Ruffini rule. This iterative scheme is numerically unstable; the approximation errors accumulate during the successive factorizations, so that the last roots are determined with a polynomial that deviates widely from a factor of the original polynomial. To reduce this error, it is advisable to find the roots in increasing order of magnitude. Wilkinson's polynomial illustrates that high precision may be necessary when computing the roots of a polynomial given its coefficients: the problem of finding the roots from the coefficients is in general ill-conditioned. The most simple method to find a single root fast is Newton's method. One can use Horner's method twice to efficiently evaluate the value of the polynomial function and its first derivative; this combination is called Birge–Vieta's method. This method provides quadratic convergence for simple roots at the cost of two polynomial evaluations per step. Closely related to Newton's method are Halley's method and Laguerre's method. Using one additional Horner evaluation, the value of the second derivative is used to obtain methods of cubic convergence order for simple roots. If one starts from a point x close to a root and uses the same complexity of 6 function evaluations, these methods perform two steps with a residual of O(|f(x)|9), compared to three steps of Newton's method with an reduction O(|f(x)|8), giving a slight advantage to these methods. When applying these methods to polynomials with real coefficients and real starting points, Newton's and Halley's method stay inside the real number line. One has to chose complex starting points to find complex roots. In contrast, the Laguerre method with a square root in its evaluation will on itself leave the real axis. Another class of methods is based on translating the problem of finding polynomial roots to the problem of finding eigenvalues of the companion matrix of the polynomial. In principle, one can use any eigenvalue algorithm to find the roots of the polynomial. However, for efficiency reasons one prefers methods that employ the structure of the matrix, that is, can be implemented in matrix-free form. Among these methods are the power method, whose application to the transpose of the companion matrix is the classical Bernoulli's method to find the root of greatest modulus. The inverse power method with shifts, which finds some smallest root first, is what drives the complex (cpoly) variant of the Jenkins–Traub method and gives it its numerical stability. Additionally, it is insensitive to multiple roots and has fast convergence with order even in the presence of clustered roots. This fast convergence comes with a cost of three Horner evaluations per step, resulting in a residual of O(|f(x)|2+3Φ), which is slower than three steps of Newton's method.

Finding roots in pairs If the given polynomial only has real coefficients, one may wish to avoid computations with complex numbers. To that effect, one has to find quadratic factors for pairs of conjugate complex roots. The application of the multi-dimensional Newton's method to this task results in Bairstow's method. In the framework of inverse power iterations of the companion matrix, the double shift method of Francis results in the real (rpoly) variant of the Jenkins–Traub method.

Finding all roots at once The simple Durand–Kerner and the slightly more complicated Aberth methods simultaneously find all of the roots using only simple complex number arithmetic. Accelerated algorithms for multi-point evaluation and interpolation similar to the fast Fourier transform can help speed them up for large degrees of the polynomial. It is advisable to chose an asymmetric, but evenly distributed set of initial points. Another method with this style is the Dandelin–Gräffe method (sometimes also falsely ascribed to Lobachevsky), which uses polynomial transformations to repeatedly and implicitly square the roots. This greatly magnifies Root-finding algorithm 60

variances in the roots. Applying Viete's formulas, one obtains easy approximations for the modulus of the roots, and with some more effort, for the roots themselves.

Exclusion and enclosure methods Several fast tests exist that tell if a segment of the real line or a region of the complex plane contains no roots. By bounding the modulus of the roots and recursively subdividing the initial region indicated by these bounds, one can isolate small regions that may contain roots and then apply other methods to locate them exactly. All these methods require to find the coefficients of shifted and scaled versions of the polynomial. For large degrees, FFT-based accelerated methods become viable. For real roots, Sturm's theorem and Descartes' rule of signs with its extension in the Budan–Fourier theorem provide guides to locating and separating roots. This plus interval arithmetic combined with Newton's method yields robust and fast algorithms. The Lehmer–Schur algorithm uses the Schur–Cohn test for circles, Wilf's global bisection algorithm uses a winding number computation for rectangular regions in the complex plane. The splitting circle method uses FFT-based polynomial transformations to find large-degree factors corresponding to clusters of roots. The precision of the factorization is maximized using a Newton-type iteration. This method is useful for finding the roots of polynomials of high degree to arbitrary precision; it has almost optimal complexity in this setting.

Method based on the Budan–Fourier theorem or Sturm chains The methods in this class give for polynomials with rational coefficients, and when carried out in rational arithmetic, provably complete enclosures of all real roots by rational intervals. The test of an interval for real roots using Budan's theorem is computationally simple but may yield false positive results. However, these will be reliably detected in the following transformation and refinement of the interval. The test based on Sturm chains is computationally more involved but gives always the exact number of real roots in the interval. The algorithm for isolating the roots, using Descartes' rule of signs and Vincent's theorem, had been originally called modified Uspensky's algorithm by its inventors Collins and Akritas. After going through names like "Collins–Akritas method" and "Descartes' method" (too confusing if ones considers Fourier's article), it was finally François Boulier, of Lille University, who gave it the name Vincent–Collins–Akritas (VCA) method, p. 24, based on "Uspensky's method" not existing and neither does "Descartes' method". This algorithm has been improved by Rouillier and Zimmerman,[1] and the resulting implementation is, to date,Wikipedia:Manual of Style/Dates and numbers#Chronological items the fastest bisection method. It has the same worst case complexity as the Sturm algorithm, but is almost always much faster. It is the default algorithm of Maple root-finding function fsolve. Another method based on Vincent's theorem is the Vincent–Akritas–Strzeboński (VAS) method; it has been shown that the VAS (continued fractions) method is faster than the fastest implementation of the VCA (bisection) method, which was independently confirmed elsewhere; more precisely, for the Mignotte polynomials of high degree, VAS is about 50,000 times faster than the fastest implementation of VCA. VAS is the default algorithm for root isolation in Mathematica, Sage, SymPy, Xcas. See Budan's theorem for a description of the historical background of these methods. For a comparison between Sturm's method and VAS, use the functions realroot(poly) and time(realroot(poly)) of Xcas. By default, to isolate the real roots of poly realroot uses the VAS method; to use Sturm's method, write realroot(sturm, poly). See also the External links for a pointer to an iPhone/iPod/iPad application that does the same thing. Root-finding algorithm 61

Finding multiple roots of polynomials Main article: Square-free factorization Most root-finding algorithms behave badly when there are multiple roots or very close roots. However, for polynomials whose coefficients are exactly given as integers or rational numbers, there is an efficient method to factorize them into factors that have only simple roots and whose coefficients are also exactly given. This method, called square-free factorization, is based on the multiple roots of a polynomial being the roots of the greatest common divisor of the polynomial and its derivative. The square-free factorization of a polynomial p is a factorization where each is either 1 or a polynomial without multiple roots, and two different do not have any common root. An efficient method to compute this factorization is Yun's algorithm.

References [1] F. Rouillier and P. Zimmerman, Efficient isolation of polynomial's real roots, Journal of Computational and Applied Mathematics 162 (2004) • Press, WH; Teukolsky, SA; Vetterling, WT; Flannery, BP (2007). "Chapter 9. Root Finding and Nonlinear Sets of

Equations" (http:/ / apps. nrbook. com/ empanel/ index. html#pg=442). Numerical Recipes: The Art of Scientific Computing (3rd ed.). New York: Cambridge University Press. ISBN 978-0-521-88068-8.

External links

• Animations for Fixed Point Iteration (http:/ / math. fullerton. edu/ mathews/ a2001/ Animations/ RootFinding/

FixedPoint/ FixedPoint. html)

• GAMS: Roots of polynomials with real coefficients (http:/ / gams. nist. gov/ serve. cgi/ Class/ F1a1/ )

• Free online polynomial root finder for both real and complex coefficients (http:/ / www. hvks. com/ Numerical/

websolver. php) • Kehagias, Spyros: RealRoots, a free App for iPhone, iPod Touch and iPad to compare Sturm's method and VAS

http:/ / itunes. apple. com/ gr/ app/ realroots/ id483609988?mt=8 Monte Carlo method 62 Monte Carlo method

Not to be confused with Monte Carlo algorithm.

Computational physics

Numerical analysis · Simulation Data analysis · Visualization

• v • t [1] • e

Monte Carlo methods (or Monte Carlo experiments) are a broad class of computational algorithms that rely on repeated random sampling to obtain numerical results; typically one runs simulations many times over in order to obtain the distribution of an unknown probabilistic entity. The name comes from the resemblance of the technique to the act of playing and recording results in a real gambling casino. They are often used in physical and mathematical problems and are most useful when it is difficult or impossible to obtain a closed-form expression, or infeasible to apply a deterministic algorithm. Monte Carlo methods are mainly used in three distinct problem classes: optimization, numerical integration and generation of draws from a probability distribution. In physics-related problems, Monte Carlo methods are quite useful for simulating systems with many coupled degrees of freedom, such as fluids, disordered materials, strongly coupled solids, and cellular structures (see cellular Potts model). Other examples include modeling phenomena with significant uncertainty in inputs, such as the calculation of risk in business; and, in math, evaluation of multidimensional definite integrals with complicated boundary conditions. In application to space and oil exploration problems, Monte Carlo–based predictions of failure, cost overruns and schedule overruns are routinely better than human intuition or alternative "soft" methods. The modern version of the Monte Carlo method was invented in the late 1940s by Stanislaw Ulam, while he was working on nuclear weapons projects at the Los Alamos National Laboratory. It was named by Nicholas Metropolis, after the Monte Carlo Casino, where Ulam's uncle often gambled. Immediately after Ulam's breakthrough, John von Neumann understood its importance and programmed the ENIAC computer to carry out Monte Carlo calculations. Monte Carlo method 63

Introduction

Monte Carlo methods vary, but tend to follow a particular pattern: 1. Define a domain of possible inputs. 2. Generate inputs randomly from a probability distribution over the domain. 3. Perform a deterministic computation on the inputs. 4. Aggregate the results. For example, consider a circle inscribed in a unit square. Given that the circle and the square have a ratio of areas that is π/4, the value of π can be approximated using a Monte Carlo method: 1. Draw a square on the ground, then inscribe a circle within it. 2. Uniformly scatter some objects of uniform size (grains of rice or Monte Carlo method applied to approximating sand) over the square. the value of π. After placing 30000 random 3. Count the number of objects inside the circle and the total number points, the estimate for π is within 0.07% of the of objects. actual value. This happens with an approximate 4. The ratio of the two counts is an estimate of the ratio of the two probability of 20%. areas, which is π/4. Multiply the result by 4 to estimate π.

In this procedure the domain of inputs is the square that circumscribes our circle. We generate random inputs by scattering grains over the square then perform a computation on each input (test whether it falls within the circle). Finally, we aggregate the results to obtain our final result, the approximation of π. If the grains are not uniformly distributed, then our approximation will be poor. Secondly, there should be a large number of inputs. The approximation is generally poor if only a few grains are randomly dropped into the whole square. On average, the approximation improves as more grains are dropped.

History Before the Monte Carlo method was developed, simulations tested a previously understood deterministic problem and statistical sampling was used to estimate uncertainties in the simulations. Monte Carlo simulations invert this approach, solving deterministic problems using a probabilistic analog (see Simulated annealing). An early variant of the Monte Carlo method can be seen in the Buffon's needle experiment, in which π can be estimated by dropping needles on a floor made of parallel and equidistant strips. In the 1930s, Enrico Fermi first experimented with the Monte Carlo method while studying neutron diffusion, but did not publish anything on it. In 1946, physicists at Los Alamos Scientific Laboratory were investigating radiation shielding and the distance that neutrons would likely travel through various materials. Despite having most of the necessary data, such as the average distance a neutron would travel in a substance before it collided with an atomic nucleus, and how much energy the neutron was likely to give off following a collision, the Los Alamos physicists were unable to solve the problem using conventional, deterministic mathematical methods. Stanislaw Ulam had the idea of using random experiments. He recounts his inspiration as follows: The first thoughts and attempts I made to practice [the Monte Carlo Method] were suggested by a question which occurred to me in 1946 as I was convalescing from an illness and playing solitaires. The question was what are the chances that a Canfield solitaire laid out with 52 cards will come out successfully? After spending a lot of time trying to estimate them by pure combinatorial calculations, I wondered whether a more practical method than "abstract thinking" might not be to lay it out say one hundred times and simply observe and count the number of successful plays. This was already possible to envisage with the beginning of the new era of fast computers, and I immediately thought of problems Monte Carlo method 64

of neutron diffusion and other questions of mathematical physics, and more generally how to change processes described by certain differential equations into an equivalent form interpretable as a succession of random operations. Later [in 1946], I described the idea to John von Neumann, and we began to plan actual calculations. –Stanislaw Ulam Being secret, the work of von Neumann and Ulam required a code name. Von Neumann chose the name Monte Carlo. The name refers to the Monte Carlo Casino in Monaco where Ulam's uncle would borrow money to gamble. Using lists of "truly random" random numbers was extremely slow, but von Neumann developed a way to calculate pseudorandom numbers, using the middle-square method. Though this method has been criticized as crude, von Neumann was aware of this: he justified it as being faster than any other method at his disposal, and also noted that when it went awry it did so obviously, unlike methods that could be subtly incorrect. Monte Carlo methods were central to the simulations required for the Manhattan Project, though severely limited by the computational tools at the time. In the 1950s they were used at Los Alamos for early work relating to the development of the hydrogen bomb, and became popularized in the fields of physics, physical chemistry, and operations research. The Rand Corporation and the U.S. Air Force were two of the major organizations responsible for funding and disseminating information on Monte Carlo methods during this time, and they began to find a wide application in many different fields. Uses of Monte Carlo methods require large amounts of random numbers, and it was their use that spurred the development of pseudorandom number generators, which were far quicker to use than the tables of random numbers that had been previously used for statistical sampling.

Definitions There is no consensus on how Monte Carlo should be defined. For example, Ripley defines most probabilistic modeling as stochastic simulation, with Monte Carlo being reserved for Monte Carlo integration and Monte Carlo statistical tests. Sawilowsky distinguishes between a simulation, a Monte Carlo method, and a Monte Carlo simulation: a simulation is a fictitious representation of reality, a Monte Carlo method is a technique that can be used to solve a mathematical or statistical problem, and a Monte Carlo simulation uses repeated sampling to determine the properties of some phenomenon (or behavior). Examples: • Simulation: Drawing one pseudo-random uniform variable from the interval (0,1] can be used to simulate the tossing of a coin: If the value is less than or equal to 0.50 designate the outcome as heads, but if the value is greater than 0.50 designate the outcome as tails. This is a simulation, but not a Monte Carlo simulation. • Monte Carlo method: Pouring out a box of coins on a table, and then computing the ratio of coins that land heads versus tails is a Monte Carlo method of determining the behavior of repeated coin tosses, but it is not a simulation. • Monte Carlo simulation: Drawing a large number of pseudo-random uniform variables from the interval (0,1], and assigning values less than or equal to 0.50 as heads and greater than 0.50 as tails, is a Monte Carlo simulation of the behavior of repeatedly tossing a coin. Kalos and Whitlock point out that such distinctions are not always easy to maintain. For example, the emission of radiation from atoms is a natural stochastic process. It can be simulated directly, or its average behavior can be described by stochastic equations that can themselves be solved using Monte Carlo methods. "Indeed, the same computer code can be viewed simultaneously as a 'natural simulation' or as a solution of the equations by natural sampling." Monte Carlo method 65

Monte Carlo and random numbers Monte Carlo simulation methods do not always require truly random numbers to be useful — while for some applications, such as primality testing, unpredictability is vital. Many of the most useful techniques use deterministic, pseudorandom sequences, making it easy to test and re-run simulations. The only quality usually necessary to make good simulations is for the pseudo-random sequence to appear "random enough" in a certain sense. What this means depends on the application, but typically they should pass a series of statistical tests. Testing that the numbers are uniformly distributed or follow another desired distribution when a large enough number of elements of the sequence are considered is one of the simplest, and most common ones. Weak correlations between successive samples is also often desirable/necessary. Sawilowsky lists the characteristics of a high quality Monte Carlo simulation: • the (pseudo-random) number generator has certain characteristics (e.g., a long “period” before the sequence repeats) • the (pseudo-random) number generator produces values that pass tests for randomness • there are enough samples to ensure accurate results • the proper sampling technique is used • the algorithm used is valid for what is being modeled • it simulates the phenomenon in question. Pseudo-random number sampling algorithms are used to transform uniformly distributed pseudo-random numbers into numbers that are distributed according to a given probability distribution. Low-discrepancy sequences are often used instead of random sampling from a space as they ensure even coverage and normally have a faster order of convergence than Monte Carlo simulations using random or pseudorandom sequences. Methods based on their use are called quasi-Monte Carlo methods.

Monte Carlo simulation versus "what if" scenarios There are ways of using probabilities that are definitely not Monte Carlo simulations — for example, deterministic modeling using single-point estimates. Each uncertain variable within a model is assigned a “best guess” estimate. Scenarios (such as best, worst, or most likely case) for each input variable are chosen and the results recorded. By contrast, Monte Carlo simulations sample probability distribution for each variable to produce hundreds or thousands of possible outcomes. The results are analyzed to get probabilities of different outcomes occurring. For example, a comparison of a spreadsheet cost construction model run using traditional “what if” scenarios, and then run again with Monte Carlo simulation and Triangular probability distributions shows that the Monte Carlo analysis has a narrower range than the “what if” analysis.Wikipedia:Please clarify This is because the “what if” analysis gives equal weight to all scenarios (see quantifying uncertainty in corporate finance), while Monte Carlo method hardly samples in the very low probability regions. The samples in such regions are called "rare events".

Applications Monte Carlo methods are especially useful for simulating phenomena with significant uncertainty in inputs and systems with a large number of coupled degrees of freedom. Areas of application include:

Physical sciences See also: Monte Carlo method in statistical physics Monte Carlo methods are very important in computational physics, physical chemistry, and related applied fields, and have diverse applications from complicated quantum chromodynamics calculations to designing heat shields and aerodynamic forms. In statistical physics Monte Carlo molecular modeling is an alternative to computational Monte Carlo method 66

molecular dynamics, and Monte Carlo methods are used to compute statistical field theories of simple particle and polymer systems. Quantum Monte Carlo methods solve the many-body problem for quantum systems. In experimental particle physics, Monte Carlo methods are used for designing detectors, understanding their behavior and comparing experimental data to theory. In astrophysics, they are used in such diverse manners as to model both the evolution of galaxies and the transmission of microwave radiation through a rough planetary surface. Monte Carlo methods are also used in the ensemble models that form the basis of modern weather forecasting.

Engineering Monte Carlo methods are widely used in engineering for sensitivity analysis and quantitative probabilistic analysis in process design. The need arises from the interactive, co-linear and non-linear behavior of typical process simulations. For example, • In microelectronics engineering, Monte Carlo methods are applied to analyze correlated and uncorrelated variations in analog and digital integrated circuits. • In geostatistics and geometallurgy, Monte Carlo methods underpin the design of mineral processing flowsheets and contribute to quantitative risk analysis. • In wind energy yield analysis, the predicted energy output of a wind farm during its lifetime is calculated giving different levels of uncertainty (P90, P50, etc.) • impacts of pollution are simulated and diesel compared with petrol. • In autonomous robotics, Monte Carlo localization can determine the position of a robot. It is often applied to stochastic filters such as the Kalman filter or Particle filter that forms the heart of the SLAM (Simultaneous Localization and Mapping) algorithm. • In telecommunications, when planning a wireless network, design must be proved to work for a wide variety of scenarios that depend mainly on the number of users, their locations and the services they want to use. Monte Carlo methods are typically used to generate these users and their states. The network performance is then evaluated and, if results are not satisfactory, the network design goes through an optimization process. • In reliability engineering, one can use Monte Carlo simulation to generate mean time between failures and mean time to repair for components.

Computational biology Monte Carlo methods are used in computational biology, such for as Bayesian inference in phylogeny. Biological systems such as proteins membranes, images of cancer, are being studied by means of computer simulations. The systems can be studied in the coarse-grained or ab initio frameworks depending on the desired accuracy. Computer simulations allow us to monitor the local environment of a particular molecule to see if some chemical reaction is happening for instance. We can also conduct thought experiments when the physical experiments are not feasible, for instance breaking bonds, introducing impurities at specific sites, changing the local/global structure, or introducing external fields. Monte Carlo method 67

Computer graphics Path Tracing, occasionally referred to as Monte Carlo Ray Tracing, renders a 3D scene by randomly tracing samples of possible light paths. Repeated sampling of any given pixel will eventually cause the average of the samples to converge on the correct solution of the rendering equation, making it one of the most physically accurate 3D graphics rendering methods in existence.

Applied statistics In applied statistics, Monte Carlo methods are generally used for two purposes: 1. To compare competing statistics for small samples under realistic data conditions. Although Type I error and power properties of statistics can be calculated for data drawn from classical theoretical distributions (e.g., normal curve, Cauchy distribution) for asymptotic conditions (i. e, infinite sample size and infinitesimally small treatment effect), real data often do not have such distributions. 2. To provide implementations of hypothesis tests that are more efficient than exact tests such as permutation tests (which are often impossible to compute) while being more accurate than critical values for asymptotic distributions. Monte Carlo methods are also a compromise between approximate randomization and permutation tests. An approximate randomization test is based on a specified subset of all permutations (which entails potentially enormous housekeeping of which permutations have been considered). The Monte Carlo approach is based on a specified number of randomly drawn permutations (exchanging a minor loss in precision if a permutation is drawn twice – or more frequently—for the efficiency of not having to track which permutations have already been selected).

Artificial intelligence for games Main article: Monte Carlo tree search Monte Carlo methods have been developed into a technique called Monte-Carlo tree search that is useful for searching for the best move in a game. Possible moves are organized in a search tree and a large number of random simulations are used to estimate the long-term potential of each move. A black box simulator represents the opponent's moves.[2] The Monte Carlo Tree Search (MCTS) method has four steps:[3] 1. Starting at root node of the tree, select optimal child nodes until a leaf node is reached. 2. Expand the leaf node and choose one of its children. 3. Play a simulated game starting with that node. 4. Use the results of that simulated game to update the node and its ancestors. The net effect, over the course of many simulated games, is that the value of a node representing a move will go up or down, hopefully corresponding to whether or not that node represents a good move. Monte Carlo Tree Search has been used successfully to play games such as Go,[4] Tantrix,[5] Battleship,[6] Havannah,[7] and Arimaa.[8] See also: Computer Go Monte Carlo method 68

Design and visuals Monte Carlo methods are also efficient in solving coupled integral differential equations of radiation fields and energy transport, and thus these methods have been used in global illumination computations that produce photo-realistic images of virtual 3D models, with applications in video games, architecture, design, computer generated films, and cinematic special effects.

Finance and business See also: Monte Carlo methods for option pricing, Stochastic modelling (insurance) and Stochastic asset model Monte Carlo methods in finance are often used to evaluate investments in projects at a business unit or corporate level, or to evaluate financial derivatives. They can be used to model project schedules, where simulations aggregate estimates for worst-case, best-case, and most likely durations for each task to determine outcomes for the overall project.

Use in mathematics In general, Monte Carlo methods are used in mathematics to solve various problems by generating suitable random numbers and observing that fraction of the numbers that obeys some property or properties. The method is useful for obtaining numerical solutions to problems too complicated to solve analytically. The most common application of the Monte Carlo method is Monte Carlo integration.

Integration Main article: Monte Carlo integration Deterministic numerical integration algorithms work well in a small number of dimensions, but encounter two problems when the functions have many variables. First, the number of function evaluations needed increases rapidly with the number of dimensions. For example, if 10 evaluations provide adequate accuracy in one dimension, then 10100 points are needed for 100 dimensions—far too many to be computed. This is called the curse of dimensionality. Second, the boundary of a multidimensional region may be very complicated, so it may not be feasible to reduce the problem to a series of nested one-dimensional integrals. 100 dimensions is by no means unusual, since in many physical problems, a "dimension" is equivalent to a degree of freedom.

Monte Carlo methods provide a way out of this exponential increase in Monte-Carlo integration works by comparing random points with the value of the function computation time. As long as the function in question is reasonably well-behaved, it can be estimated by randomly selecting points in 100-dimensional space, and taking some kind of average of the function values at these points. By the central limit theorem, this method displays convergence—i.e., quadrupling the number of sampled points halves the error, regardless of the number of dimensions. A refinement of this method, known as importance sampling in statistics, involves sampling the points randomly, but more frequently where the integrand is large. To do this precisely one would have to Monte Carlo method 69

already know the integral, but one can approximate the integral by an integral of a similar function or use adaptive routines such as stratified sampling, recursive stratified sampling, adaptive umbrella sampling or the VEGAS algorithm. A similar approach, the quasi-Monte Carlo method, uses low-discrepancy sequences. These sequences "fill" the area better and sample the most important points more frequently, so quasi-Monte Carlo methods can often converge on the integral more quickly.

Errors reduce by a factor of

Another class of methods for sampling points in a volume is to simulate random walks over it (Markov chain Monte Carlo). Such methods include the Metropolis-Hastings algorithm, Gibbs sampling and the Wang and Landau algorithm.

Simulation and optimization Main article: Stochastic optimization Another powerful and very popular application for random numbers in numerical simulation is in numerical optimization. The problem is to minimize (or maximize) functions of some vector that often has a large number of dimensions. Many problems can be phrased in this way: for example, a computer chess program could be seen as trying to find the set of, say, 10 moves that produces the best evaluation function at the end. In the traveling salesman problem the goal is to minimize distance traveled. There are also applications to engineering design, such as multidisciplinary design optimization. The traveling salesman problem is what is called a conventional optimization problem. That is, all the facts (distances between each destination point) needed to determine the optimal path to follow are known with certainty and the goal is to run through the possible travel choices to come up with the one with the lowest total distance. However, let's assume that instead of wanting to minimize the total distance traveled to visit each desired destination, we wanted to minimize the total time needed to reach each destination. This goes beyond conventional optimization since travel time is inherently uncertain (traffic jams, time of day, etc.). As a result, to determine our optimal path we would want to use simulation - optimization to first understand the range of potential times it could take to go from one point to another (represented by a probability distribution in this case rather than a specific distance) and then optimize our travel decisions to identify the best path to follow taking that uncertainty into account. Simulating industry problems using spreadsheet software is a powerful application of Monte Carlo simulation. With a basic spreadsheet tool and some formulas built into the sheet, industry can often attain good solutions without having to procure expensive simulation software.

Inverse problems Probabilistic formulation of inverse problems leads to the definition of a probability distribution in the model space. This probability distribution combines prior information with new information obtained by measuring some observable parameters (data). As, in the general case, the theory linking data with model parameters is nonlinear, the posterior probability in the model space may not be easy to describe (it may be multimodal, some moments may not be defined, etc.). When analyzing an inverse problem, obtaining a maximum likelihood model is usually not sufficient, as we normally also wish to have information on the resolution power of the data. In the general case we may have a large number of model parameters, and an inspection of the marginal probability densities of interest may be impractical, Monte Carlo method 70

or even useless. But it is possible to pseudorandomly generate a large collection of models according to the posterior probability distribution and to analyze and display the models in such a way that information on the relative likelihoods of model properties is conveyed to the spectator. This can be accomplished by means of an efficient Monte Carlo method, even in cases where no explicit formula for the a priori distribution is available. The best-known importance sampling method, the Metropolis algorithm, can be generalized, and this gives a method that allows analysis of (possibly highly nonlinear) inverse problems with complex a priori information and data with an arbitrary noise distribution.

Notes

[1] http:/ / en. wikipedia. org/ w/ index. php?title=Template:Computational_physics& action=edit

[2] http:/ / sander. landofsand. com/ publications/ Monte-Carlo_Tree_Search_-_A_New_Framework_for_Game_AI. pdf

[3] Monte Carlo Tree Search - About (http:/ / mcts. ai/ about/ index. html)

[4] Parallel Monte-Carlo Tree Search - Springer (http:/ / link. springer. com/ chapter/ 10. 1007/ 978-3-540-87608-3_6)

[5] http:/ / www. tantrix. com:4321/ Tantrix/ TRobot/ MCTS%20Final%20Report. pdf

[6] http:/ / www0. cs. ucl. ac. uk/ staff/ D. Silver/ web/ Publications_files/ pomcp. pdf

[7] Improving Monte–Carlo Tree Search in Havannah - Springer (http:/ / link. springer. com/ chapter/ 10. 1007/ 978-3-642-17928-0_10)

[8] http:/ / www. arimaa. com/ arimaa/ papers/ ThomasJakl/ bc-thesis. pdf

References

• Anderson, Herbert L. (1986). "Metropolis, Monte Carlo and the MANIAC" (http:/ / library. lanl. gov/ cgi-bin/

getfile?00326886. pdf). Los Alamos Science 14: 96–108. • Baeurle, Stephan A. (2009). "Multiscale modeling of polymer materials using field-theoretic methodologies: A survey about recent developments". Journal of Mathematical Chemistry 46 (2): 363–426. doi:

10.1007/s10910-008-9467-3 (http:/ / dx. doi. org/ 10. 1007/ s10910-008-9467-3). • Berg, Bernd A. (2004). Markov Chain Monte Carlo Simulations and Their Statistical Analysis (With Web-Based Fortran Code). Hackensack, NJ: World Scientific. ISBN 981-238-935-0. • Binder, Kurt (1995). The Monte Carlo Method in Condensed Matter Physics. New York: Springer. ISBN 0-387-54369-4. • Caflisch, R. E. (1998). Monte Carlo and quasi-Monte Carlo methods. Acta Numerica 7. Cambridge University Press. pp. 1–49. • Davenport, J. H. "Primality testing revisited". Proceeding ISSAC '92 Papers from the international symposium on

Symbolic and algebraic computation: 123 129. doi: 10.1145/143242.143290 (http:/ / dx. doi. org/ 10. 1145/

143242. 143290). ISBN 0-89791-489-9. • Doucet, Arnaud; Freitas, Nando de; Gordon, Neil (2001). Sequential Monte Carlo methods in practice. New York: Springer. ISBN 0-387-95146-6.

• Eckhardt, Roger (1987). "Stan Ulam, John von Neumann, and the Monte Carlo method" (http:/ / www. lanl. gov/

history/ admin/ files/ Stan_Ulam_John_von_Neumann_and_the_Monte_Carlo_Method. pdf). Los Alamos Science, Special Issue (15): 131–137. • Fishman, G. S. (1995). Monte Carlo: Concepts, Algorithms, and Applications. New York: Springer. ISBN 0-387-94527-X. • C. Forastero and L. Zamora and D. Guirado and A. Lallena (2010). "A Monte Carlo tool to simulate breast cancer

screening programmes". Phys. In Med. And Biol. 55 (17): 5213. Bibcode: 2010PMB....55.5213F (http:/ / adsabs.

harvard. edu/ abs/ 2010PMB. . . . 55. 5213F). doi: 10.1088/0031-9155/55/17/021 (http:/ / dx. doi. org/ 10. 1088/

0031-9155/ 55/ 17/ 021). • Golden, Leslie M. (1979). "The Effect of Surface Roughness on the Transmission of Microwave Radiation

Through a Planetary Surface". Icarus 38 (3): 451. Bibcode: 1979Icar...38..451G (http:/ / adsabs. harvard. edu/

abs/ 1979Icar. . . 38. . 451G). doi: 10.1016/0019-1035(79)90199-4 (http:/ / dx. doi. org/ 10. 1016/ 0019-1035(79)90199-4). Monte Carlo method 71

• Gould, Harvey; Tobochnik, Jan (1988). An Introduction to Computer Simulation Methods, Part 2, Applications to Physical Systems. Reading: Addison-Wesley. ISBN 0-201-16504-X. • Grinstead, Charles; Snell, J. Laurie (1997). Introduction to Probability. American Mathematical Society. pp. 10–11. • Hammersley, J. M.; Handscomb, D. C. (1975). Monte Carlo Methods. London: Methuen. ISBN 0-416-52340-4.

• Hartmann, A.K. (2009). Practical Guide to Computer Simulations (http:/ / www. worldscibooks. com/ physics/

6988. html). World Scientific. ISBN 978-981-283-415-7. • Hubbard, Douglas (2007). How to Measure Anything: Finding the Value of Intangibles in Business. John Wiley & Sons. p. 46. • Hubbard, Douglas (2009). The Failure of Risk Management: Why It's Broken and How to Fix It. John Wiley & Sons. • Kahneman, D.; Tversky, A. (1982). Judgement under Uncertainty: Heuristics and Biases. Cambridge University Press. • Kalos, Malvin H.; Whitlock, Paula A. (2008). Monte Carlo Methods. Wiley-VCH. ISBN 978-3-527-40760-6.

• Kroese, D. P.; Taimre, T.; Botev, Z.I. (2011). Handbook of Monte Carlo Methods (http:/ / www.

montecarlohandbook. org). New York: John Wiley & Sons. p. 772. ISBN 0-470-17793-4.

• MacGillivray, H. T.; Dodd, R. J. (1982). "Monte-Carlo simulations of galaxy systems" (http:/ / www.

springerlink. com/ content/ rp3g1q05j176r108/ fulltext. pdf). Astrophysics and Space Science (Springer Netherlands) 86 (2). • MacKeown, P. Kevin (1997). Stochastic Simulation in Physics. New York: Springer. ISBN 981-3083-26-3.

• Metropolis, N. (1987). "The beginning of the Monte Carlo method" (http:/ / library. lanl. gov/ la-pubs/ 00326866. pdf). Los Alamos Science (1987 Special Issue dedicated to Stanislaw Ulam): 125–130. • Metropolis, Nicholas; Rosenbluth, Arianna W.; Rosenbluth, Marshall N.; Teller, Augusta H.; Teller, Edward (1953). "Equation of State Calculations by Fast Computing Machines". Journal of Chemical Physics 21 (6): 1087.

Bibcode: 1953JChPh..21.1087M (http:/ / adsabs. harvard. edu/ abs/ 1953JChPh. . 21. 1087M). doi:

10.1063/1.1699114 (http:/ / dx. doi. org/ 10. 1063/ 1. 1699114). • Metropolis, N.; Ulam, S. (1949). "The Monte Carlo Method". Journal of the American Statistical Association

(American Statistical Association) 44 (247): 335–341. doi: 10.2307/2280232 (http:/ / dx. doi. org/ 10. 2307/

2280232). JSTOR 2280232 (http:/ / www. jstor. org/ stable/ 2280232). PMID 18139350 (http:/ / www. ncbi.

nlm. nih. gov/ pubmed/ 18139350). • M. Milik and J. Skolnick (Jan 1993). "Insertion of peptide chains into lipid membranes: an off-lattice Monte

Carlo dynamics model". Proteins 15 (1): 10–25. doi: 10.1002/prot.340150104 (http:/ / dx. doi. org/ 10. 1002/

prot. 340150104). PMID 8451235 (http:/ / www. ncbi. nlm. nih. gov/ pubmed/ 8451235). • Mosegaard, Klaus; Tarantola, Albert (1995). "Monte Carlo sampling of solutions to inverse problems". J.

Geophys. Res. 100 (B7): 12431–12447. Bibcode: 1995JGR...10012431M (http:/ / adsabs. harvard. edu/ abs/

1995JGR. . . 10012431M). doi: 10.1029/94JB03097 (http:/ / dx. doi. org/ 10. 1029/ 94JB03097). • P. Ojeda and M. Garcia and A. Londono and N.Y. Chen (Feb 2009). "Monte Carlo Simulations of Proteins in Cages: Influence of Confinement on the Stability of Intermediate States". Biophys. Jour. (Biophysical Society) 96

(3): 1076–1082. Bibcode: 2009BpJ....96.1076O (http:/ / adsabs. harvard. edu/ abs/ 2009BpJ. . . . 96. 1076O). doi:

10.1529/biophysj.107.125369 (http:/ / dx. doi. org/ 10. 1529/ biophysj. 107. 125369). • Int Panis L; De Nocker L, De Vlieger I, Torfs R (2001). "Trends and uncertainty in air pollution impacts and external costs of Belgian passenger car traffic International". Journal of Vehicle Design 27 (1–4): 183–194. doi:

10.1504/IJVD.2001.001963 (http:/ / dx. doi. org/ 10. 1504/ IJVD. 2001. 001963). • Int Panis L, Rabl A, De Nocker L, Torfs R (2002). "Diesel or Petrol ? An environmental comparison hampered by uncertainty". In P. Sturm. Mitteilungen Institut für Verbrennungskraftmaschinen und Thermodynamik (Technische Universität Graz Austria). Heft 81 Vol 1: 48–54. Monte Carlo method 72

• Press, William H.; Teukolsky, Saul A.; Vetterling, William T.; Flannery, Brian P. (1996) [1986]. Numerical Recipes in Fortran 77: The Art of Scientific Computing. Fortran Numerical Recipes 1 (Second ed.). Cambridge University Press. ISBN 0-521-43064-X. • Ripley, B. D. (1987). Stochastic Simulation. Wiley & Sons. • Robert, C. P.; Casella, G. (2004). Monte Carlo Statistical Methods (2nd ed.). New York: Springer. ISBN 0-387-21239-6. • Rubinstein, R. Y.; Kroese, D. P. (2007). Simulation and the Monte Carlo Method (2nd ed.). New York: John Wiley & Sons. ISBN 978-0-470-17793-8. • Savvides, Savvakis C. (1994). "Risk Analysis in Investment Appraisal". Project Appraisal Journal 9 (1). doi:

10.2139/ssrn.265905 (http:/ / dx. doi. org/ 10. 2139/ ssrn. 265905). • Sawilowsky, Shlomo S.; Fahoome, Gail C. (2003). Statistics via Monte Carlo Simulation with Fortran. Rochester Hills, MI: JMASM. ISBN 0-9740236-0-4.

• Sawilowsky, Shlomo S. (2003). "You think you've got trivials?" (http:/ / education. wayne. edu/ jmasm/

sawilowsky_effect_size_debate. pdf). Journal of Modern Applied Statistical Methods 2 (1): 218–225.

• Silver, David; Veness, Joel (2010). "Monte-Carlo Planning in Large POMDPs" (http:/ / books. nips. cc/ papers/

files/ nips23/ NIPS2010_0740. pdf). In Lafferty, J.; Williams, C. K. I.; Shawe-Taylor, J.; Zemel, R. S.; Culotta, A. Advances in Neural Information Processing Systems 23. Neural Information Processing Systems Foundation. • Szirmay-Kalos, László (2008). Monte Carlo Methods in Global Illumination - Photo-realistic Rendering with Randomization. VDM Verlag Dr. Mueller e.K. ISBN 978-3-8364-7919-6.

• Tarantola, Albert (2005). Inverse Problem Theory (http:/ / www. ipgp. jussieu. fr/ ~tarantola/ Files/ Professional/

SIAM/ index. html). Philadelphia: Society for Industrial and Applied Mathematics. ISBN 0-89871-572-5. • Vose, David (2008). Risk Analysis, A Quantitative Guide (Third ed.). John Wiley & Sons.

External links

Wikimedia Commons has media related to Monte Carlo method.

• Hazewinkel, Michiel, ed. (2001), "Monte-Carlo method" (http:/ / www. encyclopediaofmath. org/ index.

php?title=p/ m064870), Encyclopedia of Mathematics, Springer, ISBN 978-1-55608-010-4

• Overview and reference list (http:/ / mathworld. wolfram. com/ MonteCarloMethod. html), Mathworld

• Feynman-Kac models and particle Monte Carlo algorithms (http:/ / www. math. u-bordeaux1. fr/ ~delmoral/

simulinks. html)

• Introduction to Monte Carlo Methods (http:/ / www. phy. ornl. gov/ csep/ CSEP/ MC/ MC. html), Computational Science Education Project

• The Basics of Monte Carlo Simulations (http:/ / www. chem. unl. edu/ zeng/ joy/ mclab/ mcintro. html), University of Nebraska-Lincoln

• Introduction to Monte Carlo simulation (http:/ / office. microsoft. com/ en-us/ excel-help/

introduction-to-monte-carlo-simulation-HA010282777. aspx) (for Microsoft Excel), Wayne L. Winston

• Monte Carlo Simulation for MATLAB and Simulink (http:/ / www. mathworks. com/ discovery/

monte-carlo-simulation. html)

• Monte Carlo Methods – Overview and Concept (http:/ / www. brighton-webs. co. uk/ montecarlo/ concept. htm), brighton-webs.co.uk

• Monte Carlo techniques applied in physics (http:/ / personal-pages. ps. ic. ac. uk/ ~achremos/ Applet1-page. htm)

• Approximate And Double Check Probability Problems Using Monte Carlo method (http:/ / orcik. net/

programming/ approximate-and-double-check-probability-problems-using-monte-carlo-method/ ) at Orcik Dot Net Interpolation 73 Interpolation

For other uses, see Interpolation (disambiguation). In the mathematical field of numerical analysis, interpolation is a method of constructing new data points within the range of a discrete set of known data points. In engineering and science, one often has a number of data points, obtained by sampling or experimentation, which represent the values of a function for a limited number of values of the independent variable. It is often required to interpolate (i.e. estimate) the value of that function for an intermediate value of the independent variable. This may be achieved by curve fitting or regression analysis. A different problem which is closely related to interpolation is the approximation of a complicated function by a simple function. Suppose the formula for some given function is known, but too complex to evaluate efficiently. A few known data points from the original function can be used to create an interpolation based on a simpler function. Of course, when a simple function is used to estimate data points from the original, interpolation errors are usually present; however, depending on the problem domain and the interpolation method used, the gain in simplicity may be of greater value than the resultant loss in accuracy. There is also another very different kind of interpolation in mathematics, namely the "interpolation of operators". The classical results about interpolation of operators are the Riesz–Thorin theorem and the Marcinkiewicz theorem. There are also many other subsequent results.

Example

For example, suppose we have a table like this, which gives some values of an unknown function f.

An interpolation of a finite set of points on an epitrochoid. Points through which curve is splined are red; the blue curve connecting them is interpolation. Interpolation 74

Plot of the data points as given in the table.

x f(x)

0 0

1 0 . 8415

2 0 . 9093

3 0 . 1411

4 −0 . 7568

5 −0 . 9589

6 −0 . 2794

Interpolation provides a means of estimating the function at intermediate points, such as x = 2.5. There are many different interpolation methods, some of which are described below. Some of the concerns to take into account when choosing an appropriate algorithm are: How accurate is the method? How expensive is it? How smooth is the interpolant? How many data points are needed?

Piecewise constant interpolation

For more details on this topic, see Nearest-neighbor interpolation. The simplest interpolation method is to locate the nearest data value, and assign the same value. In simple problems, this method is unlikely to be used, as linear interpolation (see below) is almost as easy, but in higher-dimensional multivariate interpolation, this could be a favourable choice for its speed and simplicity.

Piecewise constant interpolation, or nearest-neighbor interpolation. Interpolation 75

Linear interpolation

Main article: Linear interpolation One of the simplest methods is linear interpolation (sometimes known as lerp). Consider the above example of estimating f(2.5). Since 2.5 is midway between 2 and 3, it is reasonable to take f(2.5) midway between f(2) = 0.9093 and f(3) = 0.1411, which yields 0.5252. Generally, linear interpolation takes two data points, say (x ,y ) and (x ,y ), and the interpolant is given by: a a b b

Linear interpolation is quick and easy, but it is not very precise. Another disadvantage is that the interpolant is not Plot of the data with linear interpolation superimposed differentiable at the point x . k The following error estimate shows that linear interpolation is not very precise. Denote the function which we want to interpolate by g, and suppose that x lies between x and x and that g is twice continuously differentiable. Then the a b linear interpolation error is

In words, the error is proportional to the square of the distance between the data points. The error in some other methods, including polynomial interpolation and spline interpolation (described below), is proportional to higher powers of the distance between the data points. These methods also produce smoother interpolants.

Polynomial interpolation

Main article: Polynomial interpolation Polynomial interpolation is a generalization of linear interpolation. Note that the linear interpolant is a linear function. We now replace this interpolant with a polynomial of higher degree. Consider again the problem given above. The following sixth degree polynomial goes through all the seven points:

Substituting x = 2.5, we find that f(2.5) = 0.5965. Generally, if we have n data points, there is exactly one polynomial of degree at most n−1 going through all the data Plot of the data with polynomial interpolation applied points. The interpolation error is proportional to the distance between the data points to the power n. Furthermore, the interpolant is a polynomial and thus infinitely differentiable. So, we see that polynomial interpolation overcomes most of the problems of linear interpolation.

However, polynomial interpolation also has some disadvantages. Calculating the interpolating polynomial is computationally expensive (see computational complexity) compared to linear interpolation. Furthermore, polynomial interpolation may exhibit oscillatory artifacts, especially at the end points (see Runge's phenomenon). More generally, the shape of the resulting curve, especially for very high or low values of the independent variable, Interpolation 76

may be contrary to commonsense, i.e. to what is known about the experimental system which has generated the data points. These disadvantages can be reduced by using spline interpolation or restricting attention to Chebyshev polynomials.

Spline interpolation

Main article: Spline interpolation Remember that linear interpolation uses a linear function for each of intervals [x ,x ]. Spline interpolation uses k k+1 low-degree polynomials in each of the intervals, and chooses the polynomial pieces such that they fit smoothly together. The resulting function is called a spline. For instance, the natural cubic spline is piecewise cubic and twice continuously differentiable. Furthermore, its second derivative is zero at the end points. The natural cubic spline interpolating the points in the table above is given by

Plot of the data with spline interpolation applied

In this case we get f(2.5) = 0.5972. Like polynomial interpolation, spline interpolation incurs a smaller error than linear interpolation and the interpolant is smoother. However, the interpolant is easier to evaluate than the high-degree polynomials used in polynomial interpolation. It also does not suffer from Runge's phenomenon.

Interpolation via Gaussian processes Gaussian process is a powerful non-linear interpolation tool. Many popular interpolation tools are actually equivalent to particular Gaussian processes. Gaussian processes can be used not only for fitting an interpolant that passes exactly through the given data points but also for regression, i.e., for fitting a curve through noisy data. In the geostatistics community Gaussian process regression is also known as Kriging.

Other forms of interpolation Other forms of interpolation can be constructed by picking a different class of interpolants. For instance, rational interpolation is interpolation by rational functions, and trigonometric interpolation is interpolation by trigonometric polynomials. Another possibility is to use wavelets. The Whittaker–Shannon interpolation formula can be used if the number of data points is infinite. Multivariate interpolation is the interpolation of functions of more than one variable. Methods include bilinear interpolation and bicubic interpolation in two dimensions, and trilinear interpolation in three dimensions. Interpolation 77

Sometimes, we know not only the value of the function that we want to interpolate, at some points, but also its derivative. This leads to Hermite interpolation problems. When each data point is itself a function, it can be useful to see the interpolation problem as a partial advection problem between each data point. This idea leads to the displacement interpolation problem used in transportation theory.

Interpolation in digital signal processing In the domain of digital signal processing, the term interpolation refers to the process of converting a sampled digital signal (such as a sampled audio signal) to a higher sampling rate (Upsampling) using various digital filtering techniques (e.g., convolution with a frequency-limited impulse signal). In this application there is a specific requirement that the harmonic content of the original signal be preserved without creating aliased harmonic content of the original signal above the original Nyquist limit of the signal (i.e., above fs/2 of the original signal sample rate). An early and fairly elementary discussion on this subject can be found in Rabiner and Crochiere's book Multirate Digital Signal Processing.[1]

Related concepts The term extrapolation is used if we want to find data points outside the range of known data points. In curve fitting problems, the constraint that the interpolant has to go exactly through the data points is relaxed. It is only required to approach the data points as closely as possible (within some other constraints). This requires parameterizing the potential interpolants and having some way of measuring the error. In the simplest case this leads to least squares approximation. Approximation theory studies how to find the best approximation to a given function by another function from some predetermined class, and how good this approximation is. This clearly yields a bound on how well the interpolant can approximate the unknown function.

References

[1] R.E. Crochiere and L.R. Rabiner. (1983). Multirate Digital Signal Processing. Englewood Cliffs, NJ: Prentice–Hall. (http:/ / www. amazon.

com/ Multirate-Digital-Signal-Processing-Crochiere/ dp/ 0136051626)

External links

• Sol Tutorials - Interpolation Tricks (http:/ / sol. gfxile. net/ interpolation/ index. html) Lagrange polynomial 78 Lagrange polynomial

In numerical analysis, Lagrange polynomials are used for polynomial interpolation. For a given set of distinct points and numbers , the Lagrange polynomial is the polynomial of the least degree that at each point assumes the corresponding value (i.e. the functions coincide at each point). The interpolating polynomial of the least degree is unique, however, and it is therefore more appropriate to speak of "the Lagrange form" of that unique polynomial rather than "the Lagrange interpolation polynomial", since the same polynomial can be arrived at through multiple methods. Although named after Joseph Louis Lagrange, who published it in 1795, it was first discovered in 1779 by Edward Waring and it is also an easy consequence of a formula published in 1783 by Leonhard Euler. Lagrange interpolation is susceptible to Runge's phenomenon, and the fact that changing the interpolation points requires recalculating the entire interpolant can make Newton polynomials easier to use. Lagrange polynomials are used in the Newton–Cotes method of numerical integration and in Shamir's secret sharing scheme in cryptography.

Definition

Given a set of k + 1 data points

where no two are the same, the interpolation polynomial in the Lagrange form is a linear combination

of Lagrange basis polynomials

This image shows, for four points ((−9, 5), (−4, 2), (−1, −2), (7, 9)), the (cubic) interpolation polynomial L(x) (in black), which is the sum of the scaled basis polynomials y ℓ (x), y ℓ (x), 0 0 1 1 y ℓ (x) and y ℓ (x). The interpolation polynomial 2 2 3 3 passes through all four control points, and each scaled basis polynomial passes through its respective control point and is 0 where x corresponds to the other three control points.

where . Note how, given the initial assumption that no two are the same, , so this expression is always well-defined. The reason pairs with are not allowed is that no interpolation function such that would exist; a function can only get one value for each argument . On the other hand, if also , then those two points would actually be one single point. For all , includes the term in the numerator, so the whole product will be zero at :

On the other hand, Lagrange polynomial 79

In other words, all basis polynomials are zero at , except , for which it holds that , because it lacks the term. It follows that , so at each point , , showing that interpolates the function exactly.

Proof The function L(x) being sought is a polynomial in of the least degree that interpolates the given data set; that is, assumes value at the corresponding for all data points :

Observe that: 1. In there are k terms in the product and each term contains one x, so L(x) (which is a sum of these k-degree polynomials) must also be a k-degree polynomial.

2.

We consider what happens when this product is expanded. Because the product skips , if then all

terms are (except where , but that case is impossible, as pointed out in the definition

section—in that term, , and since , , contrary to ). Also if then since

does not preclude it, one term in the product will be for , i.e. , zeroing the entire

product. So

1.

where is the Kronecker delta. So:

Thus the function L(x) is a polynomial with degree at most k and where . Additionally, the interpolating polynomial is unique, as shown by the unisolvence theorem at polynomial interpolation article.

Main idea Solving an interpolation problem leads to a problem in linear algebra amounting to inversion of a matrix. Using a standard monomial basis for our interpolation polynomial L(x) =∑ k x j m , we must invert the Vandermonde j=0 j matrix (x ) j to solve L(x ) = y for the coefficients m of L(x). By choosing a better basis, the Lagrange basis, L(x) = i i i j ∑ k l (x) y , we merely get the identity matrix, δ , which is its own inverse: the Lagrange basis automatically j=0 j j ij inverts the analog of the Vandermonde matrix. This construction is analogous to the Chinese Remainder Theorem. Instead of checking for remainders of integers modulo prime numbers, we are checking for remainders of polynomials when divided by linears. Lagrange polynomial 80

Examples

Example 1

Find an interpolation formula for ƒ(x) = tan(x) given this set of known values:

The tangent function and its interpolant

The Lagrange basis polynomials are:

Thus the interpolating polynomial then is Lagrange polynomial 81

Example 2 We wish to interpolate ƒ(x) = x2 over the range 1 ≤ x ≤ 3, given these three points:

The interpolating polynomial is:

Example 3 We wish to interpolate ƒ(x) = x3 over the range 1 ≤ x ≤ 3, given these three points:

The interpolating polynomial is:

Notes

The Lagrange form of the interpolation polynomial shows the linear character of polynomial interpolation and the uniqueness of the interpolation polynomial. Therefore, it is preferred in proofs and theoretical arguments. Uniqueness can also be seen from the invertibility of the Vandermonde matrix, due to the non-vanishing of the Vandermonde determinant.

But, as can be seen from the Example of Lagrange polynomial interpolation divergence. construction, each time a node x k changes, all Lagrange basis polynomials have to be recalculated. A better form of the interpolation polynomial for practical (or computational) purposes is the barycentric form of the Lagrange interpolation (see below) or Newton polynomials. Lagrange and other interpolation at equally spaced points, as in the example above, yield a polynomial oscillating above and below the true function. This behaviour tends to grow with the number of points, leading to a divergence known as Runge's phenomenon; the problem may be eliminatedWikipedia:Citation needed by choosing interpolation points at Chebyshev nodes. The Lagrange basis polynomials can be used in numerical integration to derive the Newton–Cotes formulas. Lagrange polynomial 82

Barycentric interpolation Using

we can rewrite the Lagrange basis polynomials as

or, by defining the barycentric weights

we can simply write

which is commonly referred to as the first form of the barycentric interpolation formula. The advantage of this representation is that the interpolation polynomial may now be evaluated as

which, if the weights have been pre-computed, requires only operations (evaluating and the weights ) as opposed to for evaluating the Lagrange basis polynomials individually. The barycentric interpolation formula can also easily be updated to incorporate a new node by dividing each of the , by and constructing the new as above. We can further simplify the first form by first considering the barycentric interpolation of the constant function :

Dividing by does not modify the interpolation, yet yields

which is referred to as the second form or true form of the barycentric interpolation formula. This second form has the advantage that need not be evaluated for each evaluation of . Lagrange polynomial 83

Finite fields The Lagrange polynomial can also be computed in finite fields. This has applications in cryptography, such as in Shamir's Secret Sharing scheme.

References

External links

• Hazewinkel, Michiel, ed. (2001), "Lagrange interpolation formula" (http:/ / www. encyclopediaofmath. org/

index. php?title=p/ l057170), Encyclopedia of Mathematics, Springer, ISBN 978-1-55608-010-4

• ALGLIB (http:/ / www. alglib. net/ interpolation/ polynomial. php) has an implementations in C++ / C# / VBA / Pascal.

• GSL (http:/ / www. gnu. org/ software/ gsl/ ) has a polynomial interpolation code in C

• SO (http:/ / stackoverflow. com/ questions/ 11029615/ lagrange-interpolation-method/ 11552763) has a MATLAB example that demonstrates the algorithm and recreates the first image in this article

• Lagrange Method of Interpolation — Notes, PPT, Mathcad, Mathematica, MATLAB, Maple (http:/ /

numericalmethods. eng. usf. edu/ topics/ lagrange_method. html) at Holistic Numerical Methods Institute (http:/ /

numericalmethods. eng. usf. edu)

• Lagrange interpolation polynomial (http:/ / www. math-linux. com/ spip. php?article71) on www.math-linux.com

• Weisstein, Eric W., "Lagrange Interpolating Polynomial" (http:/ / mathworld. wolfram. com/

LagrangeInterpolatingPolynomial. html), MathWorld.

• Estimate of the error in Lagrange Polynomial Approximation (http:/ / www. proofwiki. org/ wiki/

Lagrange_Polynomial_Approximation) at ProofWiki (http:/ / www. proofwiki. org/ )

• Module for Lagrange Polynomials by John H. Mathews (http:/ / math. fullerton. edu/ mathews/ n2003/

LagrangePolyMod. html)

• Dynamic Lagrange interpolation with JSXGraph (http:/ / jsxgraph. uni-bayreuth. de/ wiki/ index. php/ Lagrange_interpolation)

• Numerical computing with functions: The Chebfun Project (http:/ / www. maths. ox. ac. uk/ chebfun/ )

• Excel Worksheet Function for Bicubic Lagrange Interpolation (http:/ / mathformeremortals. wordpress. com/

2013/ 01/ 15/ bicubic-interpolation-excel-worksheet-function/ )

• Lagrange polynomials in Python (http:/ / pastebin. com/ bNVcQt4x) Article Sources and Contributors 84 Article Sources and Contributors

Numerical methods for ordinary differential equations Source: http://en.wikipedia.org/w/index.php?oldid=617579920 Contributors: Akriasas, Akshaysrinivasan, AllanBz, Anita5192, Aoberman, Ardonik, Az1568, Beetstra, BenFrantzDale, Berland, Cengique, Charvest, ChrisGualtieri, Christopher Parham, David.bar, Dhendriks123, Dhollm, Diegotorquemada, Dmitrey, Dysprosia, Eloquence, Emeriste, Fbergero, Fph, Frank101, Fuse809, Giftlite, Gregbard, He7d3r, Holger Blasum, Iorsh, Ivan Ukhov, Jheiv, Jiang, Jitse Niesen, JohnBlackburne, JokerXtreme, KnowledgeOfSelf, LieAfterLie, Lisamh, Lotje, LutzL, Mat cross, MathRocks2012, Michael Hardy, Mogism, Movementarian, Mvdyck, Myasuda, Nbarth, NevilleDNZ, Numsgil, Oleg Alexandrov, Onionmon, Paul Matthews, PavelSolin, Pentti71, Quentar, R.e.b., Randallbritten, Richmondman, RickK, Rookkey, [email protected], Rustyfence, Sardanaphalus, SebastianHelm, Smt3140, Sun Creator, TStein, That Guy, From That Show!, Toytoy, Uncle Milty, Waltpohl, Wavelength, Wolfkeeper, Wolfrock, Yandman, 78 anonymous edits

Runge–Kutta methods Source: http://en.wikipedia.org/w/index.php?oldid=621465037 Contributors: Aarchiba, AllHailZeppelin, Argalooga, Asperal, AxelBoldt, B dalma, Beetstra, Bemoeial, Ben pcc, BenFrantzDale, Berland, BigJohnHenry, BillFlis, Boing! said Zebedee, Bryan Derksen, CSTAR, Campuzano85, Casey Abell, Chokfung, ChrisGualtieri, Christian.Mercat, Chuunen Baka, CiaPan, Connor Behan, Cormstorm, CristianCantoro, Cyp, DD2K, DavidCary, Drew Blount, Dungodung, Dweller, Dynamicdeepak92, Dysprosia, E. H.-A. Gerbracht, EnOreg, Flavius Olimpiu, Foobaz, Francisco Albani, Fredrik, Gagelman, Gak, Gauss, Gco, Giftlite, Gurch, HJ Mitchell, Hanche, Headbomb, Helltopay27, Hyarmendacil, IIBewegung, Ichbinder, InJeans, Isnow, J-Wiki, Jesin, Jitse Niesen, Jmlk17, JohnBlackburne, Jwilty, Kaiwhakahaere, Kawautar, Keilana, Kerfuffler, Khazar2, Kosteltsev, Kri, Kwamikagami, LOL, Lambiam, MFago, MathRocks2012, Mcneja, Michael Hardy, Mikko Laine, Mild Bill Hiccup, Mr. Lefty, Nagle, NativeForeigner, Ncmvocalist, Netheril96, NevilleDNZ, NewInn, Nilsdeppe, Nixdorf, Ntveem, NuclearWarfare, Numsgil, Oleg Alexandrov, Omnipaedista, Patrick, Paul August, PaulGarner, PavelSolin, Petr F. Kartsev, Pie are round, Planb 89, Policron, Prari, Prohlep, Quondum, R.e.b., Randallbritten, Raulshc, RayAYang, Rcgldr, Rchitra, RickK, Rjwilmsi, Rookkey, Rtrace, Ryulong, Scgtrp, SciCompTeacher, Seratna, Shell Kinney, Shirulashem, Silly rabbit, SpaceFlight89, Splibubay, Supermanue, Swpb, The Anome, Thecarrotdude, Thegaboon, Thejimdawg, Tobias Bergemann, Topbanana, Vuurmeester, Wavelength, Widr, WikiUser853, Wile E. Heresiarch, Wolfkeeper, Wolfrock, Yaris678, Zemyla, ZeroG1884, 289 anonymous edits

Euler method Source: http://en.wikipedia.org/w/index.php?oldid=613154090 Contributors: 2coma7, Ale jrb, Altenmann, AnakngAraw, Berland, BiT, BigJohnHenry, Bojeryd91, Calle, Chochopk, Chrisjohnson, Cronholm144, DeC, Giftlite, Gulmammad, He7d3r, IkamusumeFan, Imperator3733, Japanese Searobin, Jitse Niesen, Joe Schmedley, Joseba.makazaga, Kawautar, Kosteltsev, Kreykesje, Kri, Kubieziel, LOL, LaGrange, Lingwitt, Lucien leGrey, Magioladitis, MathRocks2012, Mattopia, Mebden, Michael Hardy, Mikko Laine, Mild Bill Hiccup, Moskvax, Nbarth, Nonenmac, Numsgil, Oleg Alexandrov, PV=nRT, Patrick, Paul August, Paul Matthews, Pozdneev, PuzzletChung, RanBato, Rich Farmbrough, Salih, Sboerm, Siafu, Smedowns, Somegeekintn, Swerty, Tilieu, Vv148408, Wknight94, Wolfkeeper, Wolfrock, Woohookitty, 97 anonymous edits

Numerical integration Source: http://en.wikipedia.org/w/index.php?oldid=622236927 Contributors: ''half-moon'' bubba, Abacenis, Advanpix, Altruism, Anonymous Dissident, Ap, Autarch, Bakken, BenFrantzDale, Berland, Bvanderveen, Canadian Paul, Casey Abell, CatherineMunro, CommonsDelinker, Csigabi, DVD R W, Danger, Daniel.Cardenas, Dmcq, Dungodung, ESkog, Email4mobile, Eric Kvaalen, Euty, Felix Hoffmann, Fgnievinski, Fintor, Fredrik, Freshraisin, Gandalf61, Gesslein, Giftlite, Greg Kuperberg, Hu12, JForget, JJL, Jaredwf, Jason Quinn, Jdpipe, Jfmantis, Jitse Niesen, Jonmsmith, Jordibc, Jrvz, Jtepper, KlappCK, Krishnavedala, LGB, LaguerreLegendre, Lazulilasher, LieAfterLie, Loisel, Mark viking, Mcld, Michael Hardy, Mindmatrix, Myleslong, N.emamy, Nehalem, Nixdorf, O18, Oleg Alexandrov, PV=nRT, Paul August, Pedrito, Quale, Qwfp, RDBury, Rsnvprasad, Ruddyscent, ServiceAT, Silver Spoon, Someone not using his real name, Steven.butlin, Stevenj, Stewartadcock, TStein, TVBZ28, Tanner Swett, Theresa knott, Trashusclay, Trevor MacInnis, Unknown, VictorAnyakin, Waltpohl, Wavelength, Wikipelli, Wile E. Heresiarch, Willkn, Wolfrock, Zeyu Liu, Zundark, 98 anonymous edits

Trapezoidal rule Source: http://en.wikipedia.org/w/index.php?oldid=625916658 Contributors: 48v, 7daysahead, A. B., Alberto da Calvairate, Ant59, Anthony Appleyard, Arthena, Beckner84, Benjamin Dobell, Berland, BigJohnHenry, Boing! said Zebedee, Cammoore, Chadjj, Charles Matthews, Cosfortingham, D.Lazard, Deathgleaner, Discospinster, Dude1818, Email4mobile, Falcon8765, Fredrik, FreeRangeFrog, Gak, Gene Ward Smith, Gesslein, GrantPitel, Horas, InverseHypercube, Isarl, Isheden, Jesin, Jfer91, Jim.belk, Jitse Niesen, JodiChristiansen, Joe Schmedley, JohnBlackburne, K0hlrabi, Karl-Henner, Kawautar, LOL, MR Bacardi, Materialscientist, MathRocks2012, Mcalkins2011, Melikamp, Michael Hardy, Michael.lee.0x2a, Mysdaao, Nanite, Nbarth, New England, Obelix83, Oleg Alexandrov, Olegalexandrov, One.Ouch.Zero, OrangeDeb, Otterfan, Ppntori, Pxma, Rogerbrent, SamShearman, Selinger, Seth Ilys, Simamura, Smjg, Smsarmad, Stephenbez, Stevenj, Sun Creator, Taylanmath, Tesi1700, The Ouroboros, Tide rolls, Tommyjb, Tooyoungdy, UKoch, VictorAnyakin, Vikyel, Wantoun, Wavelength, anonymous edits 183 ,ﺍﺑﺮﺍﻫﯿﻢWikispaghetti, Wile E. Heresiarch, X17bc8, Xiph1980, Yk Yk Yk, Zhoudakan, ۲۰۱۲

Simpson's rule Source: http://en.wikipedia.org/w/index.php?oldid=624997048 Contributors: 777sms, ABF, Acebulf, AdamSmithee, Aeth909, Alansohn, Alembert, AndreasKloeckner, AndrewWTaylor, Anonymous Dissident, Bcnof, Bemoeial, Berland, Beta16, Bharu4hcu, BigJohnHenry, Biglovinb, Bk1 168, Bob man801, Bruce Chen 0010334, Bubba73, Cammoore, Cartoro, Clarince63, Cophus, DaveDixon, Deathgleaner, Diagonalfish, Dicklyon, Diegonc, Dingodog21, Elockid, Embrion, Epachamo, Eternalmatt, Eugene-elgato, Euryalus, Foobarnix, Fredrik, FredrikMeyer, Fsiler, Furrykef, Gandalf61, Gareth Charnock, Giftlite, Glane23, Gregie156, Gringotumadre, Grokmoo, Haemo, Hankwang, HansBrakes, Hauntedpz, Hess88, Hgilbert, Hirak 99, Ibbn, Imaoxyz, Iridescent, Ivanleung95, JakeVortex, Jason Quinn, Jdpipe, Jeff G., Jim1138, JimInTheUSA, Jitse Niesen, Joe407, JohnBlackburne, Johntyree, Jordsan, Jrvz, Jwaixs, Kamitsaha, Katieh5584, Kawautar, Keenan Pepper, Kurt.peek, Lantrix, Lavaka, LcawteHuggle, LittleOldMe, Lupin, MSGJ, Magnus Manske, MarkSutton, MarkSweep, Mecanismo, Megatronium, Michael Hardy, Mike Rosoft, Mike Williamson, Mineadwaly, Ndokos, Nixdorf, Nonagonal Spider, Notthebestusername, Oleg Alexandrov, Olegalexandrov, Orca456, PCHS-NJROTC, PV=nRT, Pedro Fonini, PhilipMW, Policron, Poor Yorick, Qwertyus, Qwfp, Ragnarstroberg, Rajagopalsriram, Reidhoch, Rkaup, Rogerbrent, Sfiguero, Simamura, Sintau.tayua, Smjg, Smsarmad, Snow Blizzard, Soniakeys, Sun Creator, Tarquin, Taxman, Tedzdog, That Guy, From That Show!, Thatsnotaname, Thomson200, Tide rolls, Torquil, Twbaroberts, Ultramandk, Vadixidav, Vianello, Vikyel, Wavelength, Wis66, Wwendler, 335 anonymous edits

Bisection method Source: http://en.wikipedia.org/w/index.php?oldid=623684243 Contributors: Alejo2083, Arithmonic, AzaToth, Beetstra, Berland, Betacommand, BigJohnHenry, CBoeckle, CRGreathouse, Calle, CanadianLinuxUser, Cedar101, ChrisGualtieri, Cobi, Cybercobra, Danger, Decrease789, Diberri, Email4mobile, Emerson7, EricEnfermero, Exp3.14, Faizan, Flyer22, Frap, Giftlite, Glenn L, Glrx, Gombang, Googl, Gryllida, Hh687711, InverseHypercube, J04n, JJL, JabberWok, JamesBWatson, Jaredwf, Jitse Niesen, JorisvS, Jujutacular, Kangaroopower, Kawautar, Kenny TM~, Kpym, Lakshmi7977, Laurentius, Lhf, LiDaobing, Luc4, Lukaskoz, Majromax, MarkSweep, Marshall.Mills, Materialscientist, McKay, Melsaran, Michael Hardy, Mmeijeri, Mohan.chanpur, Mthwiki, NFD9001, NeoUrfahraner, Nicalacla, Obradovic Goran, Oleg Alexandrov, Olegalexandrov, Patrick, Paul August, RDBury, Rursus, Sango123, Sasank.chundi, Sc920505, Simone.rossi, Sparkl3y, Sturkyn, Tnabtaf, Vieque, Wykypydya, Xieyihui, Xutu1116, Zhurov, Zundark, 132 anonymous edits

Newton's method Source: http://en.wikipedia.org/w/index.php?oldid=624124684 Contributors: 123Mike456Winston789, 1exec1, AWTom, Aaron Rotenberg, AaronSw, Adam majewski, Aero-Plex, AgadaUrbanit, Alias Flood, Annne furanku, Anonymous Dissident, Arithmonic, Aude, Avicennasis, AxelBoldt, Baccyak4H, Bdiscoe, Ben pcc, BenFrantzDale, Bentogoa, Berland, BigJohnHenry, Billtubbs, Birge, Blitztall, Blotwell, Bomazi, Borisblue, CBoeckle, CPMartin, CRGreathouse, Capricorn42, Casey Abell, Chiswick Chap, Chogg, Chris the speller, Chris857, Chrisgolden, Christian75, Ckamas, ClarkSims, CnkALTDS, Coffee2theorems, ConMan, Conversion script, Coredesat, Cyfal, Cyp, D.Lazard, DNA Games, David Cooke, Dawn Bard, Dcljr, Dcoetzee, Decrease789, Dekisugi, Dlazesz, Dominus, Draconx, Dreamofthedolphin, Drift chambers, Dulcamara, Duoduoduo, Earlh, EconoPhysicist, Ejrh, Electricmuffin11, Elvek, EmadIV, Epbr123, Eric Le Bigot, Estudiarme, Eweinber, Exir Kamalabadi, Eyrryds, Eyu100, Fintor, Fluffernutter, Flyer22, Formulax, Frau Holle, Fredrik, Gaba p, Galoisgroupie, Gco, Gex999, Giftlite, Ginsuloft, Glenn L, Glrx, Goingstuckey, Gombang, GregorB, GuidoGer, H.ehsaan, Haham hanuka, Headbomb, Henning Makholm, Hike395, Hirzel, Holycow958, Homo logos, Howard nyc, Iamunknown, Illegal604, InverseHypercube, JP.Martin-Flatin, JRSpriggs, JabberWok, Jackzhp, Jagged 85, JamesBWatson, Jaredwf, Jasondet, Jeltz, Jfmantis, Jim.belk, Jimbryho, Jitse Niesen, JohnBlackburne, JohnOwens, Jon Awbrey, Joriki, Jpginn, K.menin, KHamsun, KMic, KSmrq, Katzmik, Kawautar, Kbk, Khazar2, Kiefer.Wolfowitz, KlappCK, Kri, Kutulu, Kyle.drerup, LOL, Lakshmi7977, Laug, Laurentius, Lee Daniel Crocker, Loisel, Looxix, LutzL, M.shahriarinia, Magmait, Manoelramon, MarkSweep, Marquez, Martin Hedegaard, Metaprimer, Michael Hardy, Miguel, Mikhail Ryazanov, Mild Bill Hiccup, Minesweeper, MinorContributor, Mmeijeri, Mohit.del94, Nedunuri, Neilc, Nk, Nonagonal Spider, Nyirenda, Oleg Alexandrov, Olegalexandrov, PDH, Patrick, Paul August, PavelSolin, Pbroks13, Philip Trueman, Pichai Asokan, Pinethicket, Pit, Pitel, Pizza Puzzle, PlantTrees, Point-set topologist, Policron, Poor Yorick, Protonk, Psymun747, Pt, Quibik, Qxukhgiels, RDBury, RMFan1, Rankarana, Redmarkviolinist, RiabzevMichael, Roadrunner, Rob Haelterman, Robert K S, Rrburke, Rs2, Saltine, Sam Hocevar, Saravask, Scuchina, Seaphoto, Selfworm, Shadowjams, Shreevatsa, Shuipzv3, Shultzc, Skizzik, Slaniel, Smarchesini, Some jerk on the Internet, Suffusion of Yellow, Swerty, Talgalili, Tbsmith, Tesspub, TheJesterLaugh, TheObtuseAngleOfDoom, Tide rolls, Titolatif, Tkuvho, Tom Lougheed, Tomisti, Torokun, Toshiki, U+003F, User A1, Vrenator, Wavelength, White Shadows, Wikipedia is Nazism, Wvbailey, Wysprgr2005, Xieyihui, Xooll, Zhurov, Zundark, Пика Пика, 389 anonymous edits

Root-finding algorithm Source: http://en.wikipedia.org/w/index.php?oldid=618220177 Contributors: Access Denied, Adam majewski, Ahmadabdolkader, Ahmadyan, Akritas2, Anonymous Dissident, Army1987, Berland, Bo Jacoby, CRGreathouse, CaritasUbi, Cat2020, Catslash, Ccmcc2012, Charles Matthews, D.Lazard, David Johnson, Decrease789, Denitsa The Truth Seeker, Diberri, Elwikipedista, EmreDuran, Favonian, Fredrik, Frencheigh, Gco, Giftlite, Glrx, Götz, Happyrabbit, Hermitian, Hike395, Hvestermark, Immunize, JRSpriggs, Jaded-view, Jaredwf, Jitse Niesen, JonMcLoone, Joule36e5, Jrvz, Jujutacular, KSmrq, Kakoui, KnightRider, Lakshmi7977, Leonard G., Linas, Lingwitt, Lostella, Lukesalamone, LutzL, Magmait, Mark viking, Marshcmb, Matthewmoncek, Michael Hardy, Mild Bill Hiccup, Mjpnijmeijer, Mmeijeri, Mogism, Niceguyedc, Oleg Alexandrov, Oliphaunt, Oyd11, Pinethicket, Prohlep, QueenCake, Quondum, Qwfp, Raymondwinn, Rhubbarb, Rjwilmsi, Romanski, Ronhjones, Rschwieb, Stevenj, The Anome, Thetwentyone, UKoch, Uscitizenjason, Wonderstruck, YSSYguy, 82 anonymous edits

Monte Carlo method Source: http://en.wikipedia.org/w/index.php?oldid=613883405 Contributors: *drew, 6StringJazzer, A.Cython, ABCD, Aardvark92, Adfred123, Aferistas, Agilemolecule, Akanksh, Alanbly, Albmont, AlexBIOSS, AlexandreCam, AlfredR, Alliance09, Altenmann, Amkilpatrick, Amritkar, Andkore, Andrea Parri, Andreas Kaufmann, AndrewKeenanRichardson, Angelbo, Aniu, Apanag, Aspuru, Astridpowers, Atlant, Avalcarce, Avicennasis, Aznrocket, BAxelrod, BConleyEEPE, BD2412, Banano03, Banus, Bduke, BeRewt, Beatnik8983, Behinddemlune, BenFrantzDale, BenTrotsky, Bender235, Bensaccount, Bgwhite, BillGosset, Bkell, Blotwell, Bmaddy, Bobo192, Boffob, Bomazi, Boredzo, Broquaint, Btyner, C628, CRGreathouse, Caiaffa, Canterbury Tail, Chafe66, Charles Matthews, ChicagoActuary, Chip123456, Chris the speller, Chrisriche, Cibergili, Cm the p, Colonies Chris, Compsim, Coneslayer, Article Sources and Contributors 85

Cretog8, Criter, Crougeaux, Cybercobra, Cython1, DMG413, DVdm, Damistmu, Darkrider2k6, Datta research, David Eppstein, David.conradie, Davikrehalt, Davnor, Dawn Bard, Ddcampayo, Ddxc, Deer*lake, Denis.arnaud, Dhatfield, DianeSteele, Diannaa, Digemedi, Dmcq, Dogface, Donner60, Download, Dratman, Drewnoakes, Drsquirlz, Ds53, Dsimic, Duck ears, Duncharris, Dylanwhs, EEMIV, ERosa, Edstat, Edward, EldKatt, Elpincha, Elwikipedista, Eudaemonic3, Ezrakilty, Fastfission, Fintor, Flammifer, Flyer22, Forderud, Fritsebits, Frozen fish, Furrykef, G716, Gareth Griffith-Jones, Giftlite, Gilliam, Glosser.ca, Goudzovski, GraemeL, GrayCalhoun, Greenyoda, Grestrepo, Gruntfuterk, Gryllida, Gtrmp, Gökhan, Hanksname, Hawaiian717, Headbomb, Herath.sanjeewa, Here, Hokanomono, Hu12, Hubbardaie, Hugh.medal, ILikeThings, IanOsgood, Inrad, Ironboy11, Itub, J.Dong820, J00tel, Jackal irl, Jacobleonardking, Jamesscottbrown, Janpedia, Jason Davies, JavaManAz, Jayjaybillings, Jeffq, JesseAlanGordon, Jitse Niesen, Joey0084, John, John Vandenberg, JohnOwens, Jorgecarleitao, Jorgenumata, Jsarratt, Jugander, Jwchong, Jérôme, K.lee, KSmrq, KaHa242, Karol Langner, Keith.a.j.lewis, Kenmckinley, Kibiusa, Kimys, Kmote, Knordlun, Kri, Kroese, Kummi, Kuru, Lambyte, Lee Sau Dan, LeoTrottier, Lerdthenerd, Levin, Lexor, Lhynard, LizardJr8, LoveMonkey, M-le-mot-dit, Magioladitis, Malatesta, Male1979, ManchotPi, Marcofalcioni, Marie Poise, Mark Foskey, Martinp, Martombo, Masatran, Mathcount, Mattj2, MaxHD, MaxSchumacher, Maxal, Maxentrope, Maylene, Mblumber, Mbmneuro, Mbryantuk, Mean as custard, Melcombe, Michael Hardy, MicioGeremia, Mikael V, Misha Stepanov, Mlpearc, Mnath, Moink, MoonDJ, MrOllie, Mtford, Nagasaka, Nagualdesign, Nanshu, Narayanese, Nasarouf, Nelson50, Nosophorus, Nsaa, Nuno Tavares, Nvartaniucsd, Oceans and oceans, Odoncaoa, Ohnoitsjamie, Oli Filth, Oneboy, OrgasGirl, Ott2, P99am, Paul August, PaulieG, PaulxSA, Pbroks13, Pcb21, Pdelmoral, Pete.Hurd, PeterBoun, Pgreenfinch, Philopp, Phluid61, PhnomPencil, Pibwl, Pinguin.tk, PlantTrees, Pne, Pol098, Policron, Popsracer, Poupoune5, Qadro, Quentar, Qwfp, Qxz, R'n'B, RWillwerth, Ramin Nakisa, Rcsprinter123, Redgolpe, Renesis, Retirementplan io, RexJacobus, Reza1615, Rich Farmbrough, Richie Rocks, Rinconsoleao, Rivanvx, Rjmccall, Rjwilmsi, Robma, RockMagnetist, Rodo82, Ronnotel, Ronz, Rs2, Rygo796, SKelly1313, Saltwolf, Sam Korn, Samratvishaljain, Sankyo68, Seasund, Sergio.ballestrero, Shacharg, Shreevatsa, Siniša Čubrilo, Sjoemetsa, Snegtrail, Snoyes, Somewherepurple, Spellmaster, Splash6, Spotsaurian, SpuriousQ, Stefanez, Stefanomione, StewartMH, Stimpy, Storm Rider, Superninja, Sweetestbilly, Tarantola, Taxman, Tayste, Techhead7890, Tesi1700, Theron110, Thirteenity, ThomasNichols, Thr4wn, Thumperward, Tiger Khan, Tim Starling, Tom harrison, TomFitzhenry, Tooksteps, Toughpkh, Trebor, Twooars, Tyrol5, UBJ 43X, Urdutext, Uwmad, Veszely, Vgarambone, Vipuser, Vividstage, VoseSoftware, Wavelength, Wile E. Heresiarch, William Avery, X-men2011, Yoderj, Zarniwoot, Zoicon5, Zr40, Zuidervled, Zureks, Мурад

97, 杨 中 舟, 508 anonymous edits

Interpolation Source: http://en.wikipedia.org/w/index.php?oldid=603888087 Contributors: A.melkumyan, AHusain314, AlbertSM, Alberto da Calvairate, Algebraist, Andonic, Anna Frodesiak, Auntof6, Awistaiw, BenFrantzDale, Berland, Betacommand, Bobet, Bobo192, CanadianLinuxUser, Ccooke123, Cdc, Chadernook, ChangChienFu, Chaos, Charles Matthews, Chewings72, Coneslayer, Conversion script, Cronholm144, DavidCary, Dcljr, DerHexer, Diannaa, Dick Beldin, Dicklyon, Dino, DoctorW, Don4of4, Dudzcom, Dungodung, Editor2020, Edrowland, Ehrenkater, El C, El-Ahrairah, Elwikipedista, EngineerSteve, Eric119, Everyking, Fgnievinski, Friginator, Giftlite, Gpejic, Grajales, Grunt, Gryllida, HGB, HanPritcher, Harp, Hattrem, Hede2000, HenkvD, Herbee, Heron, Hippopottoman, Hyacinth, InverseHypercube, Ironholds, JJL, JahJah, Jaimeastorga2000, Japo, Jason Quinn, Jeltz, Jezhotwells, Jitse Niesen, John Cline, JohnBlackburne, Jonkerz, JordoCo, Krishnachandranvn, LarryTheAwesomeDude, Lotje, LucasVB, Marcello Quintieri, Martin.Budden, Math Champion, MathFacts, MathMartin, Matt37, Michael Hardy, Mike Peel, Minimac, MuzikJunky, Niffux, Oleg Alexandrov, Oli Filth, Omicronpersei8, PabloCastellano, Patrick, Pdcook, Persian Poet Gal, Pinethicket, Prashantrayarikar, RMFan1, Rich Farmbrough, Rjs.swarnkar, Roger pearse, Rtbehrens, Rudolf.hellmuth, SWAdair, Shreevatsa, Spooky, Supertouch, Sławomir Biały, Thorwald, UFu, Uberjivy, Velella, Vtstarin, Wavelength, Wi-king, Will.doctor1, Woodinville, XJaM, Xiaojeng, Yangli2, Yas, 187 anonymous edits

Lagrange polynomial Source: http://en.wikipedia.org/w/index.php?oldid=626374611 Contributors: A. Pichler, Aboalbiss, Ac44ck, Acdx, Alexey Muranov, Alexeymor, Andris, Arrizaba, Bdmy, Beetstra, Ben pcc, Berland, BigJohnHenry, CRGreathouse, Confront, Cuzkatzimhut, Cyp, Dmcq, DmitriyZotikov, Dspradau, Dysprosia, Dzordzm, Eric-Wester, Flyingonce, Flyingtabmow, Fredrik, Furrykef, GDirichlet, Garion96, Giftlite, GirasoleDE, Ignacioerrico, Jaapkroe, JerroldPease-Atlanta, Jitse Niesen, JohnBlackburne, Julien Tuerlinckx, LOL, Liorma, Loisel, Marcello Quintieri, MathFacts, MathMartin, Matthieu fr, Maxim Razin, Mboverload, Me and, Mecanismo, Michael Hardy, Microbat, Mátyás, Nbarth, Nixdorf, Oarih, Oleg Alexandrov, Olegalexandrov, PV=nRT, PackElend, PaulTanenbaum, PavelSolin, Pedrito, Philip Trueman, Policron, RDBury, Raffamaiden, Rewguy, Rich Farmbrough, Riki, Rooparam, SURIV, Salix alba, Sigmundur, Silly rabbit, Simoneau, Sterrys, Superninja, Sławomir Biały, Tbsmith, The Anome, TheJJJunk, Timwi, Tonsil, Turidoth, V madhu, Xodarap00, 151 anonymous edits Image Sources, Licenses and Contributors 86 Image Sources, Licenses and Contributors

File:Numerical integration illustration, h=1.png Source: http://en.wikipedia.org/w/index.php?title=File:Numerical_integration_illustration,_h=1.png License: Public Domain Contributors: Oleg Alexandrov File:Numerical integration illustration, h=0.25.png Source: http://en.wikipedia.org/w/index.php?title=File:Numerical_integration_illustration,_h=0.25.png License: Public Domain Contributors: Oleg Alexandrov Image:Euler method.svg Source: http://en.wikipedia.org/w/index.php?title=File:Euler_method.svg License: Public Domain Contributors: Oleg Alexandrov Image:Numerical integration illustration, h=1.png Source: http://en.wikipedia.org/w/index.php?title=File:Numerical_integration_illustration,_h=1.png License: Public Domain Contributors: Oleg Alexandrov Image:Numerical integration illustration, h=0.25.png Source: http://en.wikipedia.org/w/index.php?title=File:Numerical_integration_illustration,_h=0.25.png License: Public Domain Contributors: Oleg Alexandrov Image:Instability of Euler's method.svg Source: http://en.wikipedia.org/w/index.php?title=File:Instability_of_Euler's_method.svg License: Creative Commons Zero Contributors: User:Jitse Niesen Image:Stability region for Euler method.svg Source: http://en.wikipedia.org/w/index.php?title=File:Stability_region_for_Euler_method.svg License: Public Domain Contributors: Helmut Podhaisky Image:Wikibooks-logo-en-noslogan.svg Source: http://en.wikipedia.org/w/index.php?title=File:Wikibooks-logo-en-noslogan.svg License: Creative Commons Attribution-Sharealike 3.0 Contributors: User:Bastique, User:Ramac et al. Image:Integral as region under curve.svg Source: http://en.wikipedia.org/w/index.php?title=File:Integral_as_region_under_curve.svg License: Creative Commons Attribution-ShareAlike 3.0 Unported Contributors: 4C File:Geometric mean.svg Source: http://en.wikipedia.org/w/index.php?title=File:Geometric_mean.svg License: Creative Commons Zero Contributors: User:Krishnavedala File:Parabola and inscribed triangle.svg Source: http://en.wikipedia.org/w/index.php?title=File:Parabola_and_inscribed_triangle.svg License: Public domain Contributors: derivative work: Pbroks13 (talk) Parabola-and-inscribed_triangle.png: Mattes Image:Integration rectangle.svg Source: http://en.wikipedia.org/w/index.php?title=File:Integration_rectangle.svg License: Public Domain Contributors: Integration_rectangle.png: Emergie derivative work: Kpengboy (talk) Image:Integration trapezoid.svg Source: http://en.wikipedia.org/w/index.php?title=File:Integration_trapezoid.svg License: Public Domain Contributors: Calkowanie_numeryczne-metoda_trapezow.png: Emergie derivative work: Kpengboy (talk) Image:Integration simpson.png Source: http://en.wikipedia.org/w/index.php?title=File:Integration_simpson.png License: Public Domain Contributors: Emergie Image:trapezoidal rule illustration.png Source: http://en.wikipedia.org/w/index.php?title=File:Trapezoidal_rule_illustration.png License: Public Domain Contributors: Darapti, DieBuche, Docu, Maksim, Oleg Alexandrov File:Composite trapezoidal rule illustration.svg Source: http://en.wikipedia.org/w/index.php?title=File:Composite_trapezoidal_rule_illustration.svg License: Public Domain Contributors: Oleg Alexandrov Image:simpsons method illustration.png Source: http://en.wikipedia.org/w/index.php?title=File:Simpsons_method_illustration.png License: Public domain Contributors: Original uploader was Oleg Alexandrov at en.wikipedia Image:Bisection method.svg Source: http://en.wikipedia.org/w/index.php?title=File:Bisection_method.svg License: Creative Commons Attribution-ShareAlike 3.0 Unported Contributors: Bisection_method.svg: Tokuchan derivative work: Tokuchan (talk) Image:Wikiversity-logo.svg Source: http://en.wikipedia.org/w/index.php?title=File:Wikiversity-logo.svg License: logo Contributors: Snorky (optimized and cleaned up by verdy_p) Image:NewtonIteration Ani.gif Source: http://en.wikipedia.org/w/index.php?title=File:NewtonIteration_Ani.gif License: GNU Free Documentation License Contributors: Ralf Pfeifer Image:NewtonsMethodConvergenceFailure.svg Source: http://en.wikipedia.org/w/index.php?title=File:NewtonsMethodConvergenceFailure.svg License: Public Domain Contributors: Aaron Rotenberg image:newtroot 1 0 0 0 0 m1.png Source: http://en.wikipedia.org/w/index.php?title=File:Newtroot_1_0_0_0_0_m1.png License: GNU General Public License Contributors: Adam majewski, D-Kuru, Dbc334, LutzL, Maksim, Norro Image:Commons-logo.svg Source: http://en.wikipedia.org/w/index.php?title=File:Commons-logo.svg License: logo Contributors: Anomie file:Rayleigh-Taylor instability.jpg Source: http://en.wikipedia.org/w/index.php?title=File:Rayleigh-Taylor_instability.jpg License: Public Domain Contributors: Lawrence Livermore National Laboratory File:Pi 30K.gif Source: http://en.wikipedia.org/w/index.php?title=File:Pi_30K.gif License: Creative Commons Attribution 3.0 Contributors: CaitlinJo File:Monte-carlo2.gif Source: http://en.wikipedia.org/w/index.php?title=File:Monte-carlo2.gif License: unknown Contributors: Zureks File:Monte-Carlo method (errors).png Source: http://en.wikipedia.org/w/index.php?title=File:Monte-Carlo_method_(errors).png License: unknown Contributors: Zureks Image:Splined epitrochoid.svg Source: http://en.wikipedia.org/w/index.php?title=File:Splined_epitrochoid.svg License: Public Domain Contributors: Splined_epitrochoid.png: dino derivative work: InverseHypercube Image:Interpolation Data.svg Source: http://en.wikipedia.org/w/index.php?title=File:Interpolation_Data.svg License: Public Domain Contributors: Berland Image:Piecewise constant.svg Source: http://en.wikipedia.org/w/index.php?title=File:Piecewise_constant.svg License: Public Domain Contributors: Berland Image:Interpolation example linear.svg Source: http://en.wikipedia.org/w/index.php?title=File:Interpolation_example_linear.svg License: Public Domain Contributors: Berland Image:Interpolation example polynomial.svg Source: http://en.wikipedia.org/w/index.php?title=File:Interpolation_example_polynomial.svg License: Public Domain Contributors: Berland Image:Interpolation example spline.svg Source: http://en.wikipedia.org/w/index.php?title=File:Interpolation_example_spline.svg License: Public Domain Contributors: Berland Image:Lagrange polynomial.svg Source: http://en.wikipedia.org/w/index.php?title=File:Lagrange_polynomial.svg License: Creative Commons Attribution-ShareAlike 3.0 Unported Contributors: User:Acdx Image:lagrangeInterp.png Source: http://en.wikipedia.org/w/index.php?title=File:LagrangeInterp.png License: Public domain Contributors: Maksim, Shyam File:Lagrange polynomial Divergence.jpg Source: http://en.wikipedia.org/w/index.php?title=File:Lagrange_polynomial_Divergence.jpg License: Creative Commons Attribution-Sharealike 3.0 Contributors: DavidianSkitzou License 87 License

Creative Commons Attribution-Share Alike 3.0 //creativecommons.org/licenses/by-sa/3.0/