Euler-Maclaurin without analytic derivatives
Bengt Fornberg
University of Colorado, Boulder Department of Applied Mathematics
Slide 1 of 15 The Euler-Maclaurin formulas (for approximating infinite sums)
spacing h x
EM1 – Trapezoidal Rule 24 6 8 ∞ ∞ hh h h h f ()xdxh−≈ fx ( ) −+fx() f(1) () x−+fx (3) () fx (5) () − f (7) ()x + − k =0 k 00 0 0 0 x0 2 12 720 30240 1209600 EM2 – Midpoint Rule 24 6 8 ∞ ∞ hh7 31 h 127 h f ()xdx−≈hf (ξ ) −+fx(1)() fx (3) () − f (5) ()x + f (7) ()x +− k =0 k 00 0 0 x0 24 5760 967680 154828800
James Stirling Colin Maclaurin Leonhard Euler 1692- 1698- 1707-1783 1770 1746
Slide 2 of 15 Derivation of the EM1 formula (Part 1)
24 6 8 ∞ ∞ hh h h h f ()xdxh−≈ fx ( ) −+fx() f(1) () x−+fx (3) () fx (5) () − f (7) ()x + − k =0 k 00 0 0 0 x0 2 12 720 30240 1209600
∞ B − = k hkkf (1)()x k =1 k! 0 z ∞ B = k zk where the Bernoulli numbers Bk are defined by ekz −1! k =0 11 1 1 1 5 BB==−=1, , B , B = 0, B =−= , B 0, B = , B = 0, B =−= , B 0, B = , 012 2 6 3 4 30 5 6 42 7 8 30 9 10 66
Neat identity:
B BB =+++1123 1 0 +++12 2 Multiplying up the denominator: zzzz zz 1! 2! 3! 0! 1! 2!
Equate coefficients: Each coefficient corresponds to a whole diagonal in the identity matrix
BBB − 11 1 011 n 1 0! 1! (n− 1)! 1! 2!n ! B B 11 1 0 1 ×= 0! 1! 1! 2! B0 1 1 × 0! nn× 1! nn× nn
Slide 3 of 15 Derivation of the EM1 formula (case of h = 1) (Part 2) 11 fx( )=+− fk ( ) ( xkfk ) '( ) +− ( xkfk )2 ''( ) + Taylor expand f(x) around x = k: 1! 2!
k +1 This implies: fxdx() 111 f ()k k 1! 2! 3! +− 11 f '(k ) fk(1)() fk = 1! 2! f ''(k ) fk'(+− 1) fk '( ) 1 1! ∞ ∞ f ()k fxdx() 111 k =0 Add these relations for k = 0,1,2,… 0 1! 2! 3! ∞ 11 f '(k ) − f (0) = k =0 1! 2! − 1 ∞ f '(0) f ''(k ) 1! k =0
∞ Use the ‘Neat Identity’: ∞ B BB f ()k 0 12 k =0 B BB 111 fxdx() 0 12 1 0! 1! 2! 0 0! 1! 2! 1! 2! 3! ∞ B B B0 B1 11 1 0 1 − f '(k ) f (0) k =0 0! 1! ×=1! 2! 0! 1! = ∞ B0 1 1 B 0 − 0! 1! f '(0) f ''(k ) k =0 0! Multiply by Bernoulli matrix from left:
Now read off EM1 from the top row in this equation. Slide 4 of 15 Other early Euler-Maclaurin pioneers
Amedeo Plana Niels Henrik Abel Abel-Plana formula (1820): 1781-1864 1802-1829
∞∞−− ∞ =++1()()fit f it = fk() fxdx () f (0) iπ dt k 0 002 e2 t −1 ∞ 1()()∞ fit−− f it (1)()−=+k fk f (0) i dt k=0 22sinh()0 π t
Siméon Poisson 1781-1840 Provided in 1820 the first precise error term For truncated Euler-Maclaurin expansions
Slide 5 of 15 Main complication when applying the EM formulas
24 6 8 ∞ ∞ hh h h h f ()xdxh−≈ fx ( ) −+fx() f(1) () x−+fx (3) () fx (5) () − f (7) ()x + − k =0 k 00 0 0 0 x0 2 12 720 30240 1209600
Formula contains a large number of derivatives (we often want, say, 50 or 100 terms):
In general the calculation of the higher derivatives involved in the Euler- Maclaurin expansion is not possible.
(Süli and Mayers, An Introduction to Numerical Analysis, Cambridge University Press, 2003)
Do we actually need to? Can one simply replace the derivatives in EM with regular finite difference approximations?
Rest of this presentation:
1. Summary of some standard finite difference (FD) schemes 2. Error analysis suggesting that FD actually ought to work just fine in this context 3. Two verification tests 4. Conclusions
Slide 6 of 15 Some finite difference (FD) and Hermite-FD (HFD) formulas ‘Regular’ centered FD weights Centered Hermite weights
Very fast and stable algorithms available to calculate weights also in the case of irregularly spaced nodes. Slide 7 of 15 Apparent difficulty when applying FD formulas
Applying FD formulas for high order derivatives is notoriously ill-conditioned numerically
Say we want to approximate f (50)(0) by FD:
wfnh−−()−++ wfh () −+ wf (0)() + wh ++ wfnh () f (50) (0) ≈ nn101 h50 A derivative is a local property of a function, so the stencil needs to be narrow, which implies - h must be extremely small - h50 is vastly much smaller still - Extreme loss of significant digits in numerator (to produce an O(1)-sized result).
We will NOT assume here that f(z) is available as an analytic function in the complex plane If it is available; two options: 1. Use the Abel-Plana formulas kf!()ξ f ()k ()zd= ξ 2. Use Cauchy’s integral formula: Γ + 2πi ()− z k 1 ξ - No need for path to be close to the center - Trigonometric basis functions are orthogonal - If the path is circular, FFT is available
Slide 8 of 15 EM2 example using ‘regular’ centered FD approximations
∞ Define Fx()=− f ()tdt The EM2 formula then becomes x
∞ 1 7 31 127 fk(+=−+1 ) F (0) F(2) (0) − F (4) (0) + F (6) (0) − F (8) (0) + k=0 2 24 5760 96780 154828800
11 Suppose we want to approximate the RHS using F(x) at 5 nodes x =−{1, − , 0, + , + 1} 22 FF(0)(0)≈ ( 1 (0) ) FFFFFF(2)(0)≈−−+−− 2 2 (1415411 ( 1) ( ) (0) + ( ) − (1) ) 12 3 2 2 3 2 12 FFFFFF(4)(0)2(≈−−−−+ 4 1(1) 4(11 ) 6(0) 4() 1(1)) 22 We can use only μ = 3 terms of EM2. Combine together according to coefficients in the EM2 formula; get weights
13 233 1 −−−,, ,, 30 10 15 10 30 Slide 9 of 15 Generalize to using different number of EM2 terms
73 Coefficients stay much simpler than the EM2 ones; μ = 6, cf. . μ 3503554560 2 + μ−1 (!)n Closed form for coefficients contains w =−(1)k 1 no Bernoulli numbers: ,k nk=||(2nnknk++ 1)( )!( − )!
Weights stay small however far the table is extended, implying no danger ever of numerical cancellations.
Are these approximations accurate? (We have NOT let h -> 0) Slide 10 of 15 Error when using μ terms of the EM2 formula, μ = 1,2,3,… μ B ≈≈2μ (2 )2 (2 ) EM2 with exact analytic derivatives: EM2Error μ FFμ (0) (0) (2 )! (2π )2 μμ − (!)2μπ22 1 FD≈≈FF(2) (0)μ (2)μ (0) FD approximation (using h = 1/2): Error 2μ (2μμ+ 1)!4 2
μμ Only essential difference: Factor (2π )22vs. 4
In words: Given μ (number of EM terms), changing analytical derivatives to regular FD approximations (with h =1/2), loses only about π 2μμ log≈ 0.39 decimal digits. 10 2 All numerical cancellations in high derivative calculations have been eliminated, since all the weights we apply to function values stayed small. Even this accuracy loss becomes virtually eliminated when changing from
regular FD to Hermite-FD approximations. Slide 11 of 15 Regular FD vs. Hermite FD (HFD) approximations Using regular FD, μ = 3 EM2 terms:
17(2) (4) 13233111 −+FF(0) (0) − F (0) ≈−−+−−+ F() 1 F( ) FF (0)( −) F (1) 24 5760 30 1022 15 10 30 Using Hermite FD, μ = 3 EM2 terms:
∞ Recall Fx()=− ftdt () , i.e. Fx'( )= f (x ). x The f(x) function is also available to us without differentiation (the function to be summed).
1(2) 7 (4) 171111 32 17 1 1 −+FF(0) (0) − F (0) ≈ F( −−) FF (0) +( +) f( −+− 0 f) (0) f( ) 24 5760 302222 15 30 10 10 Same formal order of accuracy, and same number of function evaluations, but higher accuracy due to more local information.
Slide 12 of 15 Example 1
∞ fk( )≈ 0.25903856926239039237 Approximate: k =1 2 1 - erfinv arctan 1 x erfinv arctan 1+x2 where 1+ x2 e −1 fx()== , Fx () (2)1xx22++ π
To reach error around 10-16 by direct summation requires approximately 100,000,000 terms.
19 ∞ f ()x f ().x Instead, sum directly k = 1 and apply EM2 to k =20
μ = 5 terms of EM2 requires a total of 9 function evaluations (FD or HFD), or evaluation of up to 7th derivative of f(x) (EM2). The error loss by replacing analytic derivatives with FD/HFD approximations is insignificant. Slide 13 of 15 Example 2
Evaluate to high precision Euler’s constant
n 111∞ γ =−=++−≈lim logn 1 log 1 0.57721566490153286061 n→∞ kk==12 kkk 11 Here: fx()=+ log1 − , Fx ( ) = 2( x − 1)arccoth(2 x −− 1) 1. xx Reaching error 10-100 by direct summation requires about 10100 terms.
N −1 ∞ f ()k f ()k Sum explicitly k = 2 , then apply EM2 to kN = .
Slide 14 of 15 Some conclusions Historical notes: - The pioneering works by Euler, Maclaurin, Plana, Abel, Poisson, etc. were carried out between 200 and 300 years ago. - Surprisingly little (if any) attention has been given to simplifying the numerical use of the Euler-Maclaurin expansions.
Some aspects of the present numerical approach: - FD approximations of low order derivatives to high orders of accuracy is common for ODEs and PDEs. In contrast, FD approximations of very high order derivatives are rare. - High accuracies for EM expansions can be reached with standard FD approximations, and without resorting to small step sizes when approximating the derivatives. - Even when using real-valued arithmetic, it is unnecessary to do any analytic differentiations when using EM expansions to approximate infinite sums.
References:
B.F., Calculation of weights in finite difference formulas, Math. Comp. (1988). B.F., An algorithm for calculating Hermite-based finite difference weights, IMA J. Num. Anal. (2020). B.F. and C. Piret, Complex Variables and Analytic functions: An Illustrated Introduction, SIAM (2020). (with residue calculus derivations of both the EM and the Abel-Plana formulas) B.F., Euler-Maclaurin expansions without analytic derivatives, submitted (2020). Slide 15 of 15