Lecture 3: Backward Error Analysis

Geometric Numerical Integration TU München Ernst Hairer January – February 2010 Lecture 3: Backward error analysis Table of contents 1 Modified differential equation 1 2 Modified Hamiltonian for symplectic integrators 5 3 Near conservation of the total energy 6 4 Counter-example: Takahashi–Imada integrator 10 5 ExistenceofglobalmodifiedHamiltonian 13 6 CompletelyintegrableHamiltoniansystems 15 7 Linear error growth for integrable systems 17 8 Exercises 20

Backward error analysis is the most powerful tool for the study of the long-time behaviour of numerical integrators.

1 Modified differential equation ¨* ϕ (y ) Consider an ordinary differential ¨¨ t 0 ëxact¨ ¨ equation numerical y˙ = f(y), y˙ = f(y) H HH HHHj and a numerical method Φh(y) which y = Φ (y ) ¨¨¨* n+1 h n produces the approximations ëxact¨ ˙ ¨ y = fh(y ) y0,y1,y2,... . A forward error analysis consists of the studye of thee errors y ϕ (y ) (local error) 1 − h 0 and yn ϕnh(y0) (global error) in the solution space. The idea of backward error − ˙ analysis is to search for a modified differential equation y = fh(y ) of the form ˙ 2 y = f(y )+ hf2(y )+ h f3(y )+ ...,e e (1) such that yn = y(nh)e, and toe study thee differencee of the vector fields f(y) and fh(y). We remark that the series in (1) usually diverges and that one has to truncate it suitably for ae rigorous analysis. For the moment we content ourselves with a formal analysis without taking care of convergence issues.

1 For the computation of the modiﬁed equation (1) we put y := y(t) for a ﬁxed t, and we expand the solution of (1) into a Taylor series e 2 y(t + h) = y + h f(y)+ hf2(y)+ h f3(y)+ . . . h2 (2) e + f ′(y)+ hf ′ (y)+ . . . f(y)+ hf (y)+ . . . + ... . 2! 2 2 We assume that the numerical method Φh(y) can be expanded as

2 3 Φh(y)= y + hf(y)+ h d2(y)+ h d3(y)+ . . . (3)

(the coefﬁcient of h is f(y) for consistent methods). The functions dj(y) are known and are typically composed of f(y) and its derivatives. To get y(nh)= yn for all n, we must have y(t + h) = Φh(y). Comparing like powers of h in the e expressions (2) and (3) yields recurrence relations for the functions fj(y): e 1 f (y) = d (y) f ′f(y) (4) 2 2 − 2! 1 ′′ ′ ′ 1 ′ ′ f3(y) = d3(y) f (f, f)(y)+ f f f(y) f f2(y)+ f2f(y) . − 3! − 2! Example 1 Consider the scalar differential equation y˙ = y2, y(0) = 1 with exact solution y(t)=1/(1 t). It has a singularity at t = 1. We apply the explicit − Euler method yn+1 = yn + hf(yn) with step size h =0.02. The above procedure for the computation of the modiﬁed equation is implemented as a Maple script

> fcn := y -> yˆ2: > nn := 6: > fcoe[1] := fcn(y): > for n from 2 by 1 to nn do > modeq := sum(hˆj*fcoe[j+1], j=0..n-2): > diffy[0] := y: > fori from1 by 1 to n do > diffy[i] := diff(diffy[i-1],y)*modeq: > od: > ytilde := sum(hˆk*diffy[k]/k!, k=0..n): > res := ytilde-y-h*fcn(y): > tay := convert(series(res,h=0,n+1),polynom): > fcoe[n] := -coeff(tay,h,n): > od: > simplify(sum(hˆj*fcoe[j+1], j=0..nn-1));

2 20 exact solution solutions of truncated modiﬁed equations

.6 .8 1.0 Figure 1: Solutions of the modiﬁed equation for the problem y˙ = y2, y(0) = 1.

Its output is 3 8 31 157 y˙ = y 2 hy 3 + h2 y 4 h3 y 5 + h4 y 6 h5 y 7 ... . (5) − 2 − 3 6 − 15 ± Figure 1e presentse thee exact solutione (dashede curve),e the numericale solution (thick dots), and the solution of the modified equation, when truncated after 1, 2, 3, and 4 terms. We observe an excellent agreement of the numerical solution with the exact solution of the modified equation. A similar program for the implicit midpoint rule computes the modified equation 1 1 11 3 y˙ = y 2 + h2 y 4 + h4 y 6 + h6 y 8 + h8 y 10 ... , (6) 4 8 192 128 ± and for thee classicale (explicit)e Runge–Kuttae methode of ordere4 1 65 17 19 y˙ = y 2 h4 y 6 + h6 y 8 h7 y 9 + h8 y 10 ... . (7) − 24 576 − 96 144 ± We observee e that the perturbatione e terms in thee modified equatie on are of size (hr), where r is the order of the method. This is true in general. O Theorem 1 Suppose that the method yn+1 = Φh(yn) is of order r, i.e., Φ (y)= ϕ (y)+ hr+1δ (y)+ (hr+2), h h r+1 O r+1 where ϕt(y) denotes the exact flow of y˙ = f(y), and h δr+1(y) the leading term of the local truncation error. The modified equation then satisfies ˙ r r+1 y = f(y )+ h fr+1(y )+ h fr+2(y )+ ..., y(0) = y0 (8) with fr+1(ey)= δre+1(y). e e e

Proof. The construction of the functions fj(y) (see the beginning of this section) shows that f (y)=0 for 2 j r if and only if Φ (y) ϕ (y)= (hr+1). j ≤ ≤ h − h O 3 Example 2 We next consider the Lotka–Volterra equations

q˙ = q(p 1), p˙ = p(2 q), − − and we apply (a) the explicit Euler method, and (b) the symplectic Euler method, both with constant step size h = 0.1. The first terms of their modified equations are h (a) q˙ = q(p 1) q(p2 pq +1)+ (h2), − − 2 − O h p˙ = p(q 2) p(q2 pq 3q +4)+ (h2), − − − 2 − − O h (b) q˙ = q(p 1) q(p2 + pq 4p +1)+ (h2), − − 2 − O h p˙ = p(q 2) + p(q2 + pq 5q +4)+ (h2). − − 2 − O Figure 2 shows the numerical solutions for initial values indicated by a thick dot. In the pictures to the left they are embedded in the exact flow of the differential equation, in those to the right they are embedded in the flow of the modified differential equation, truncated after the h2 terms. For the symplectic Euler method, the solutions of the truncated modified equation are periodic, as is the case for the unperturbed problem.

(a) explicit Euler, h =0.1 p p exact modified 4 flow 4 flow 3 3

2 2

1 1

2 4 6 q 2 4 6 q (b) symplectic Euler, h =0.1 p p exact modified 4 flow 4 flow 3 3

2 2

1 1

2 4 6 q 2 4 6 q

Figure 2: Numerical solution compared to the exact and modiﬁed ﬂows

4 p Euler, h =0.12 p sympl. Euler, h =0.12 4 solutions of 4 solutions of modiﬁed modiﬁed 3 diff. equ. 3 diff. equ. 2 2 1 1

2 4 6 q 2 4 6 q

Figure 3: Study of the truncation in the modiﬁed equation

In Figure 3 we present the numerical solution and the exact solution of the modiﬁed equation, once truncated after the h terms (dashed-dotted), and once truncated after the h2 terms (dotted). The exact solution of the problem is included as a solid curve. Example 3 For a linear differential equation with constant coefﬁcients

y˙ = Ay, y(0) = y0 we consider numerical methods which yield yn+1 = R(hA)yn, where R(z) is the n stability function of the method. In this case we get yn = R(hA) y0, so that t/h t yn = y(nh), where y(t) = R(hA) y0 = exp h ln R(hA) y0 is the solution of the modiﬁed differential equation e 1e y˙ = ln R(hA) y =(A + hb A2 + h2b A3 + . . .) y (9) h 2 3 2 with suitable constantse b2, b3,...e . Since R(z) =1+ z + (ze ) and ln(1 + x)= x x2/2+ (x3) both have a positive radius of convergence,O the series (9) − O converges for h < h with some h > 0. This is an exceptional situation. | | 0 0 2 Modiﬁed Hamiltonian for symplectic integrators

We consider a Hamiltonian system y˙ = J −1 H(y) with smooth H(y), and we show that the modified equation of symplectic∇ methods is also Hamiltonian. Theorem 2 (Existence of a local modified Hamiltonian) If a symplectic method Φ (y) is applied to a Hamiltonian system y˙ = J −1 H(y) with a smooth Hamil- h ∇ tonian H : U R , then the modified equation (1) is locally Hamiltonian. → More precisely, for every y0 U and for all j there exist smooth functions H (y), such that f (y)= J −1 H∈(y) on a suitable neighborhood of y . j j ∇ j 0

5 1 2 −1 Proof. Assume that fj(y)= J Hj(y) for j =1, 2,...,r (this is satisfied for r =1, because f (y)= f(y)= J −∇1 H(y)). We have to prove the existence of a 1 ∇ Hamiltonian Hr+1(y). The idea is to consider the truncated modified equation ˙ r−1 y = f(y )+ hf2(y )+ . . . + h fr(y ), (10) r−1 which is a Hamiltoniane systeme with Hamiltoniane H(y)+hHe 2(y)+. . .+h Hr(y). Its flow ϕr,t(y0), compared to that of (1), satisfies Φ (y )= ϕ (y )+ hr+1f (y )+ (hr+2), h 0 r,h 0 r+1 0 O and also Φ′ (y )= ϕ′ (y )+ hr+1f ′ (y )+ (hr+2). h 0 r,h 0 r+1 0 O By our assumption on the method and by the induction hypothesis, Φh and ϕr,h are symplectic transformations. Together with ϕ′ (y )= I + (h), this implies r,h 0 O ′ T ′ r+1 ′ T ′ r+2 J = Φh(y0) JΦh(y0)= J + h fr+1(y0) J + Jfr+1(y0) + (h ). O ′ The matrix Jfr+1(y) is therefore symmetric, and the (local) existence of Hr+1(y) satisfying f (y)= J −1 H (y) follows from the Integrability Lemma. r+1 ∇ r+1 The application of the Integrability Lemma shows that the modified Hamilto- nian is globally defined on U, if U = R2d, orif U is star-shaped, or if U is simply connected.

3 Near conservation of the total energy

As a ﬁrst application of backward error analysis we study the long-time energy conservation of a symplectic numerical scheme (of order r) applied to Hamilto- nian systems y˙ = J −1 H(y). It follows from Theorem 2 that the corresponding modiﬁed differential equation∇ is also Hamiltonian. After truncation we get

[N] r N−1 H (y) = H(y)+ h Hr+1(y)+ . . . + h HN (y), (11) which we assume to be globally deﬁned on the same open set as the original Hamiltonian H(y).

1G. Benettin & A. Giorgilli, On the Hamiltonian interpolation of near to the identity symplectic mappings with application to symplectic integration algorithms, J. Statist. Phys. 74 (1994) 1117– 1143. 2Y.-F. Tang, Formal energy of a symplectic scheme for Hamiltonian systems and its applica- tions (I), Computers Math. Applic. 27 (1994) 31–39.

6 Theorem 3 Consider a Hamiltonian system with smooth H : U R (where U R2d), apply a symplectic numerical method Φ (y) with step→ size h, and ⊂ h assume that its modified Hamiltonian is globally defined on U. If the numerical solution stays in the compact set K U, then we have asymp- totically for h 0 ⊂ → H[N](y ) = H[N](y )+ (t hN ) n 0 O H(y ) = H(y )+ (hr) n 0 O over time intervals of size t = nh C h−N+r. ≤ Proof. We let ϕN,t(y0) be the flow of the truncated modified equation. The effect N+1 of the truncation is that yj+1 ϕN,h(yj) C h . Since the truncated mod- k − [N] k ≤ [N] ified equation is Hamiltonian with H (y) of (11), we have H ϕN,t(yj) = [N] H (yj) for all times t. Using a global h-independent Lipschitz constant for H[N] on the compact set K, the statement on the long-time conservation of H[N] is a consequence of n−1 H[N](y ) H[N](y ) = H[N](y ) H[N](y ) n − 0 j+1 − j Xj=0 n−1 = H[N](y ) H[N] ϕ (y ) = (nhN+1). j+1 − N,h j O Xj=0

The statement for the Hamiltonian H follows from (11), because Hr+1(y)+ . . . + N−r−1 h HN (y) is uniformly bounded on K independently of h and N. If the Hamiltonian and the integrator are analytic, it is possible to prove exponentially small error bounds on exponentially large time intervals. More precisely, for sufficiently small h there exists N = N(h) h−1 such that in the estimates of the theorem above hN can be replaced by e−γ/h∼. Example 4 The mathematical pendulum is a system with one degree of freedom having the Hamiltonian H(p, q) = 1 p2 cos q. Since the Hamiltonian and the 2 − vector field are 2π-periodic in q, it is natural to consider q as a variable on the circle S1. Hence, the phase space of points (p, q) becomes the cylinder R S1. × Figure 4 shows some level curves of H(p, q) together with numerical solutions. Theorem 3 explains the near conservation of the Hamiltonian with the symplectic Euler method (existence of a global modified Hamiltonian will be discussed later). This is illustrated in Figure 5. The linear drift of the numerical Hamiltonian for non-symplectic methods can be explained by a computation

7 explicitexplicitEuler Euler symplectic symplecticEuler EulerSt¨ormer–Verlet Figure 4: Solutions of the pendulum problem; explicit Euler with step size h = 0.2, initial value (p0, q0) = (0, 0.5); symplectic Euler with h = 0.3 and initial values q0 =0, p0 =0.7, 1.4, 2.1; St¨ormer–Verlet with h =0.6.

.2 Error in total energy

.1 explicit Euler symplectic Euler .0 0 10 20 30 40 50 60 70 80 90 Figure 5: Error in the total energy for the explicit and symplectic Euler methods applied with step size h =0.005 and initial value (p0, q0)=(2.5, 0). similar to that of the proof of Theorem 3. From a Lipschitz condition of the Hamiltonian and from the standard local error estimate, we obtain H(yn+1) r+1 − H(ϕh(yn)) = (h ). Since H(ϕh(yn)) = H(yn), a summation of these terms leads to O H(y ) H(y )= (thr) for t = nh. (12) n − 0 O

Example 5 In the numerical experiment of Figure 6 we study the effect of “large” step sizes for the pendulum problem. We have drawn 200 000 steps of the numerical solution of the implicit midpoint rule for various step sizes h and for initial values (p , q ) = (0, 1.5), (p , q ) = (0, 2.5), (p , q ) = (1.5, π), and 0 0 − 0 0 − 0 0 − (p , q ) = (2.5, π). They are compared to the contour lines of the truncated 0 0 −

8 3 3 3

2 2 2

1 1 1

0 0 0 −3 −2 −1 0 1 2 3 −3 −2 −1 0 1 2 3 −3 −2 −1 0 1 2 3 −1 −1 −1 = 0.70 = 1.04 = 1.465 −2 h −2 h −2 h

3 3 3

2 2 2

1 1 1

0 0 0 −3 −2 −1 0 1 2 3 −3 −2 −1 0 1 2 3 −3 −2 −1 0 1 2 3 −1 −1 −1 = 1.55 = 1.58 = 1.62 −2 h −2 h −2 h Figure 6: Numerical solutions of the implicit midpoint rule with large step sizes modified Hamiltonian p2 h2 H[4](p, q)= cos q + cos(2q) 2p2 cos q . 2 − 48 − This shows that for step sizes as large as h 0.7 the Hamiltonian H[4] is extremely well conserved. Beyond this value, the dynamics≤ of the numerical method soon turns into chaotic behaviour. Example 6 (Modified Hamiltonian of the Stormer–Verlet¨ method) We present the modified Hamiltonian for a separable Hamiltonian H(p, q) = T (p)+ U(q) 1 T T T with T (p) = 2 p p. Using the notation A, B = qA pB pA qB for the Poisson bracket of two functions depending{ } on p∇and q∇, it is− given ∇ by∇ 1 1 H = T + U + h2 T, T, U U, U, T 12 { } − 24 { } 1 1 e + h4 T, T, T, T, U + U, T, T, T, U −720 { } 360 { } 1 1 U, U, T, T, U + T, T, U, U, T + (h6). −480 { } 120 { } O This modified Hamiltonian can be computed with the recursive procedure explained in the beginning of this lecture. A more elegant derivation is by the use of the symmetric Baker–Campbell–Hausdorff formula (cf. Sect. III.4.2 of the mono- graph on “Geometric Numerical Integration”). Since H(p, q) it is composed of derivatives of T and U, it is globally defined. e

9 4 Counter-example: Takahashi–Imada integrator

We consider a second order differential equation q¨ = f(q), and we assume that it is Hamiltonian, i.e., f(q) = U(q). Takahashi & Imada3 noticed that the accuracy of the Störmer–Verlet−∇ discretization is greatly improved by adding the 2 ′ 1 expression αh f (q)f(q) with α = 12 to every force evaluation. This yields h p = p + I + αh2f ′(q ) f(q ) n+1/2 n 2 n n qn+1 = qn + hpn+1/2 (13) h p = p + I + αh2f ′(q ) f(q ), n+1 n+1/2 2 n+1 n+1 which is still symplectic, because for f(q) = U(q) it can be interpreted as applying the Störmer–Verlet method to the Hamiltonian−∇ system with potential α V (q)= U(q) h2 U(q) 2. − 2 k∇ k Simplified Takahashi–Imada method. To avoid the derivative evaluation of the vector field f(q), which corresponds to a Hessian evaluation for f(q)= U(q), we replace I + αh2f ′(q) f(q) with f q + αh2f(q) and thus consider−∇4 h p = p + f q + αh2f(q ) n+1/2 n 2 n n qn+1 = qn + hpn+1/2 (14) h p = p + f q + αh2f(q ) n+1 n+1/2 2 n+1 n+1 which is a (h5) perturbation of the method (13). The method is volume preserving and thusO symplectic for problems with one degree of freedom. To see this, note that (14) is a composition of shears (mappings of the form (p, q) (p + a(q), q) and (p, q) (p, q + hp)), and shears always preserve the volume.7→ 7→ Modified differential equation. Substituting in the modified Hamiltonian of Example 6 the potential U with U α h2 U 2 = U α h2 U, U, T yields − 2 k∇ k − 2 { } the modified Hamiltonian of the symplectic Takahashi–Imadaintegrator (13): 2 4 H(p, q)= H(p, q)+ h H3(p, q)+ h H5(p, q) 3M. Takahashi andb M. Imada. Monte Carlo calculation of quantum systems. II. Higher order correction. J. Phys. Soc. Jpn., 53:3765–3769, 1984. 4J. Wisdom, M. Holman, and J. Touma. Symplectic correctors. In J. E. Marsden, G. W. Patrick, and W. F. Shadwick, editors, Integration Algorithms and Classical Mechanics, pages 217–244. Amer. Math. Soc., Providence R. I., 1996.

10 where (for one degree of freedom)

1 2 1 α H3(p, q) = p Uqq(q) + Uq(q)Uq(q) (15) 12 − 24 2 1 1 H (p, q) = p4 U (q) p2 U (q)U (q)+ . . . (16) 5 −720 qqqq − 120 q qqq Since the method (14) differs from (13) only by a (h5) perturbation, we find that the modified differential equation of (14) satisfiesO 1 p˙ = H(p, q)+ h4α2 f ′′(f, f) (q)+ (h6) −∇q 2 O (17) q˙ = H(bp, q)+ (h6) ∇p O Notice that for potentialsb U(q) that are 2π-periodic, the function H(p, q) is also 2π-periodic in q, but the integral of the perturbation need not be 2π-periodic. b To study energy conservation of the simplified Takahashi–Imada integrator (14) we notice that along the (formal) exact solution of its modified differential equation (17) we have

d 1 T 1 H(p, q)= h4α2 p f ′′(f, f)+ (h6)= h4α2 U (p, U , U )+ (h6), dt 2 O −2 qqq q q O (18) b where we have used f(q) = U(q) = Uq(q). In general, the expression on the right-hand side of (18) cannot−∇ be written− as a total differential. However, this formula yields much insight into the long-time energy conservation of (14).5 Bounded error in the energy without any drift. For problems with one de- • d gree of freedom the right-hand side of (18) can be written as dt F q(t) ′ 2 6 with F (q) = Uqqq(q)Uq(q) (neglecting terms of size (h )). If U(q) is T 2 O T -periodic in q, and 0 Uqqq(q)Uq(q) dq =0, then the antiderivative (or in- 2 1 definite integral) F (qR) of Uqqq(q)Uq(q) is globally defined on the circle S . This is the situation for the pendulum, where U(q) = cos q. We expect that also higher order terms can be treated in this way, so− that no energy drift can be observed in this situation (see Figure 7, where h = 0.2, and initial values q(0) = 0, p(0) = 2.5 are used). Linear energy drift. In general, without any particular assumption on the • potential, the numerical energy will have a drift of size (th4). E.g., for a non-symmetrically perturbed pendulum with U(q)= cosO q +0.2 sin(2q), 2π − for which U (q)U (q)2dq 3.77 (see Figure 7). 0 qqq q ≈ R 5E. Hairer, R.I. McLachlan, and R.D. Skeel. On energy conservation of the simplified Takahashi–Imada method. ESAIM: M2AN 43:631644, 2009.

11 .000 U(q)= cos q − −.005

U(q)= cos q +0.2 sin 2q −.010 − 103 0 1 × 2 2 Figure 7: Error in the shifted modiﬁed energy H(pn, qn)+ h H3(p, q) along the numerical solution of the simpliﬁed Takahashi–Imada method.

Random walk behavior – drift like square root of time. Under the assump- • tion that the solution of the modiﬁed differential equation is ergodic on an invariant set A with respect to an invariant measure µ, we have 1 t lim F p(s), q(s) ds = F (x) µ(dx), (19) t→∞ t Z Z 0 A where x =(p, q) and the function F (x) is the right-hand side of (18). Again there will be a linear drift of size (th4) in general. However, in the pres- ence of symmetries, the integral ofO the right-hand side in (19) is likely to vanish and the numerical Hamiltonian will look like a random walk, so that there will be a drift of size (√t h4). O

6 10−6 × 3 0 −3 −6 104 0 1 × 2 2 Figure 8: Error in the modified energy H(pn, qn)+ h H3(pn, qn) of the N-body problem (N = 9. Lennard–Jones potential) along the numerical solution of the simplified Takahashi–Imada method (50 trajectories). Included is the average over 400 trajectories as a function of time (µ = 0.091 10−6 at t = 2 104) and the standard deviation (σ =3.08 10−6 at t =2 10×4). × × × 12 5 Existence of global modified Hamiltonian

The theory of generating functions shows that every symplectic one-step method Φ :(p, q) (P, Q) can be locally expressed in terms of a function S(P,q,h) as h 7→ p = P + S(P,q,h), Q = q + S(P,q,h). (20) ∇q ∇p This property allows us to prove the existence of a globally defined modified Hamiltonian, whenever the generating function is globally defined.

Theorem 4 Assume that the symplectic method Φh has a generating function 2 3 S(P,q,h) = hS1(P, q)+ h S2(P, q)+ h S3(P, q)+ . . . (21) with smooth Sj(P, q) defined on an open set U. Then, the modified differential equation is a Hamiltonian system with 2 H(p, q)= H(p, q)+ hH2(p, q)+ h H3(p, q)+ ... , (22) where the functionse Hj(p, q) are defined and smooth on the whole of U. Proof. The exact solution (P, Q)= p(t), q(t) of the Hamiltonian system corresponding to H(p, q) is given by e e e p = P + S(P,q,t), Q = q + S(P,q,t), ∇q ∇P where S is the solution of thee Hamilton–Jacobi differentiale equation

e ∂tS(P,q,t) = H P, q + P S(P,q,t) , S(P,q, 0)=0. (23) ∇ Since H dependse on the parametere h, thise is also the casee for S. Our aim is to determine the functions H (p, q) such that the solution S(P,q,t) of (23) coincides e j e for t = h with (21). e We ﬁrst express S(P,q,t) as a series 2 3 S(P,q,t) =et S1(P,q,h)+ t S2(P,q,h)+ t S3(P,q,h)+ ... , insert it intoe (23) and comparee powers ofe t. This allowse us to obtain the functions Sj(p,q,h) recursively in terms of derivatives of H: e S1(p,q,h) = H(p, q) e

e e∂H ∂S1 2 S2(p,q,h) = (p,q,h) (24) ∂qe · ∂Pe e 2 ∂H ∂S2 1 ∂ H ∂S1 ∂S1 3 S3(p,q,h) = (p,q,h)+ 2 , (p,q,h). ∂qe · ∂Pe 2 ∂qe ∂Pe ∂Pe e 13 We then write Sj as a series 2 Sj(ep,q,h)= Sj1(p, q)+ h Sj2(p, q)+ h Sj3(p, q)+ ... , insert it ande the expansione (22) for H intoe (24), ande compare powers of h. This yields S (p, q) = H (p, q) and for j > 1 we see that S (p, q) is a function of 1k k e jk derivatives of H with l

Theorem 5 A symplectic Runge–Kutta method (i.e., biaij + bjaji = bibj for all i, j) applied to a system with smooth Hamiltonian H : U R (with U R2d an → ⊂ arbitrary open set) has a modiﬁed Hamiltonian (22) with smooth functions Hj(y), deﬁned globally on U.

Proof. Let (Pi, Qi) be the internal stages of an implicit Runge–Kutta method (p, q) (P, Q). It follows from implicit differentiation of the Runge–Kutta equa- 7→ tions that, under the condition biaij + bjaji = bibj for all i, j, the Runge–Kutta formulas can be written as (20) with (an idea of Lasagni 1988) s s S(P,q,h)= h b H(P , Q ) h2 b a H (P , Q )T H (P , Q ). i i i − i ij q i i p j j Xi=1 i,jX=1

This shows that the coefficient functions Sj(P, q) can be expressed in terms of derivatives of H(P, q). This theorem extends to partitioned Runge–Kutta methods including the Störmer– Verlet integrators. Theorem 4 implies that all methods based on generating functions (e.g., variational integrators) have a globally defined modified Hamiltonian.

14 6 Completely integrable Hamiltonian systems

There is an interesting class of Hamiltonian problems, for which symplectic integrators have an improved long-time behavior of the global error. Here and in the following, we denote the standard d-dimensional torus by Td = Rd/2πZd = (θ mod 2π,...,θ mod 2π) ; θ R . { 1 d i ∈ } Deﬁnition 1 We call a Hamiltonian system completely integrable, if, for every (p0, q0) in the domain of H(p, q), there exists a symplectic diffeomorphism (p, q)= ψ(a, θ), 2π-periodic in θ

d 2d between V T and U R (where U is a neighborhood of (p0, q0), and V is open), such× that the Hamiltonian⊂ in the new variables becomes H(p, q) = H(ψ(a, θ)) = K(a).

The variables (a, θ) = (a1,...,ad, θ1 mod 2π,...,θd mod 2π) are called action-angle variables. In these variables, the system becomes

a˙ i =0, θ˙i = ωi(a), i =1,...,d with ∂ , and can be solved directly , ωi(a) = ∂ai K(a) ai(t) = ai0 θi(t) = θi0 + ωi(a0) t, so that we get periodic (or quasi-periodic) ﬂow

p(t), q(t) = ψ a0, θ0 + ω(a0) t . The following theorem gives a practical characterization of integrability6. Theorem 6 (Arnold-Liouville) Suppose that for the Hamiltonian H(p, q) there exist smooth functions F = H, F ,...,F : U R, U R2d satisfying 1 2 d → ⊂ (I1) F1,...,Fd are in involution, i.e., Fi, Fj = 0, where the Poisson bracket is given by F,G = F T G { F T} G, { } ∇q ∇p − ∇p ∇q (I2) the gradients of F1,...,Fd are everywhere linearly independent,

(I3) the solution trajectories of the Hamiltonian systems with Hamiltonian Fi (for i =1,...,d) exist for all times and remain in U.

If, in addition, the level sets (p, q) U ; Fi(p, q) = ci, i = 1,...,d are compact, then the Hamiltonian system{ is∈ completely integrable. }

6V.I. Arnold, Mathematical Methods of Classical Mechanics, Springer-Verlag, New York, 1978, second edition 1989.

15 Example 8 (Motion in central ﬁeld) Consider the Hamiltonian

1 2 2 2 2 H = (p1 + p2)+ V (r), r = q1 + q2 , 2 q with a potential V (r) that is defined and smooth for r > 0. The Kepler problem corresponds to V (r) = 1/r, and the perturbed Kepler problem to V (r) = 1/r µ/(3r3). Changing to− polar coordinates (see Exercises 3 and 4) − − q r cos ϕ p cos ϕ sin ϕ p 1 = , r = 1 , (26) q2 r sin ϕ pϕ r sin ϕ r cos ϕ p2 − this becomes 2 1 2 pϕ H(pr,pϕ,r,ϕ)= pr + 2 + V (r). 2 r The system has the angular momentum L = pϕ as a first integral, since H does not depend on ϕ. Clearly, H, L = 0 everywhere. The gradients of H and L { } 2 3 ′ are linearly independent unless both pr = 0 and pϕ = r V (r). By inserting 2 2 pϕ = 2r (H V (r)) and eliminating r this becomes a condition of the form α(H, L)=0,− which for the Kepler problem reads L2(1 +2HL2)=0. The conditions of Theorem 6 are thus satisfied on the domain U = (p ,p ,r,ϕ) ; r > 0, α(H, L) =0 . { r ϕ 6 } Example 9 (Toda lattice) This is a system of particles on a line interacting pair- wise with exponential forces. The motion is determined by the Hamiltonian n 1 H(p, q) = p2 + exp(q q ) 2 k k − k+1 Xk=1 with periodic boundary conditions: q = q . With the notation a = 1 p , n+1 1 k − 2 k b = 1 exp 1 (q q ) , all n eigenvalues of the matrix k 2 2 k − k+1 a1 b1 bn b1 a2 b2 0  .. .. L =  b2 . .   .   0 .. a b   n−1 n−1  b b a   n n−1 n  are first integrals of the system. It can be shown (here without proof) that this system is completely integrable Many important problems in celestial mechanics are small perturbations of integrable systems, e.g., planetary motion.

16 7 Linear error growth for integrable systems

We consider a completely integrable Hamiltonian system p˙ = H(p, q), q˙ = H(p, q). (27) −∇q ∇p with real analytic Hamiltonian. We let (p, q) = ψ(a, θ) be the symplectic diffeomorphism that transforms (27) to action-angle variables, and we denote the inverse transformation by (a, θ) = I(p, q), Θ(p, q) . Consequently, the compo- nents I1,...,Id of I are ﬁrst integrals of the system, i.e., I(p(t), q(t)) = I(p0, q0) for all t. In the action-angle variables, the Hamiltonian is K(a) = H(p, q), and we denote the vector of frequencies by ω(a) = K(a). We consider this in a neighbourhood of some a∗ Rd. ∇ The aim of this section∈ is to prove that for symplectic methods applied to completely integrable systems we have simultaneously linear growth of the global error, • near conservation of all ﬁrst integrals depending only on the action vari- • ables. These properties are illustrated in Figure 9. Non-symplectic methods, like the classical Runge–Kutta method of order 4, show a quadratic growth of the global error and a linear growth of the error in the action variables.

.012 error in the eigenvalues of L .010 RK4, h =0.08 .008 .006 .004 St¨ormer–Verlet, .002 h =0.02 .000 50 100 .8 global error .6 RK4, h =0.08

.4 St¨ormer–Verlet, h =0.02 .2

.0 50 100 Figure 9: Euclidean norm of numerical errors for the Toda lattice with n = 3; initial values are p = 1.5, p =1, p =0.5, and q =1, q =2, q = 1. 1 − 2 3 1 2 3 − 17 Theorem 7 Consider completely integrable Hamiltonian system with real-analytic Hamiltonian • symplectic integrator of order r with globally defined modified Hamiltonian • strong non-resonance condition for ω(a∗) • k ω(a∗) γ k −ν, k Zd, k =0 (28) | · | ≥ | | ∈ 6 I(p , q ) a∗ Const log h −ν−1 • k 0 0 − k ≤ | | −r Then, there exist constants C, h0 such that for h h0 and for t = nh h the numerical solution satisfies ≤ ≤ (p , q ) (p(t), q(t)) C t hr k n n − k ≤ I(p , q ) I(p , q ) C hr. k n n − 0 0 k ≤ Proof. Let us mention (without proof) that for ν>d 1 the set of frequencies − in a fixed ball that do not satisfy (28) has Lebesgue measure bounded by cγ. Therefore, almost all frequencies satisfy (28) for some γ > 0. The main steps of the proof are illustrated in Figure 10. The missing part is

backward error analysis

integrable numerical modiﬁed Hamilton system solution Hamilton system H(p,q) p ,q H(p,q) { n n} action – angle e variables (p,q)= ψ(a, θ) Hamilton modif. Hamilton system system K(a)+ εK1(a, θ) K(a) r Lindstedt Poincar´e with ε = h series a˙ = 0 (a, θ)= χ(b, ϕ) θ˙ = w(a) modif. Hamilton system N K(b)+ εK1(b)+ . . . + ε KN (b) +εN+1R(b, ϕ)

b(t) b CtεN+1 k − 0k≤ ϕ(t) ϕ w (b )t C(t + t2)εN+1 k − 0 − ǫ 0 k≤ Figure 10: Idea of the proof for the linear error growth of symplectic integrators.

18 the recursive elimination of the angle-variables by the use of a Lindstedt–Poincar´e series. This will be discussed in the following (omitting technical details). We consider a perturbed Hamiltonian

K(a, θ)= K(a)+ εK1(a, θ). The problem is to ﬁnd a symplectice transformation (a, θ)= χ(b, ϕ) which elimi- nates the angle variable θ as far as possible. We look for a transformation of the form b = a S(b, θ), ϕ = θ + S(b, θ), − ∇θ ∇b where the generating function is given by a truncated series

2 N S(b, θ)= εS1(b, θ)+ ε S2(b, θ)+ . . . + ε SN (b, θ) with coefﬁcient functions that are 2π-periodic in θ. Such a transformation is (ε) close to the identity. In the new variables the Hamiltonian is O

K(χ(b, ϕ)) = K(a, θ)= K(b + S(b, θ), θ) ∇θ 2 e = K(b)+ εeω(b) θSe1(b, θ)+ K1(b, θ) + (ε ) · ∇ O where ω(b)= K(b). We aim in ﬁnding S such that ∇ 1 ω(b) S (b, θ)+ K (b, θ) · ∇θ 1 1 does not depend on θ. Expanding the periodic functions into Fourier series

i k·θ i k·θ S1(b, θ)= sk(b)e , K1(b, θ)= hk(b)e , kX∈Zd kX∈Zd we obtain a formal solution from h (b) s (b) = k , k =0. k − i k ω(b) 6 · At this point we are struck by the problem of small denominators. For any values of the frequencies ωj(b), the denominator k ω(b) = k1ω1(b)+ + kdωd(b) · d ··· becomes arbitrarily small for some k = (k1,...,kd) Z , and even vanishes if the frequencies are rationally dependent. Using the∈ non-resonace condition (28) and the fast decay of Fourier coefficients for analytic functions permits to overcome this difficulty. Further coefficients of the generating function S(b, θ) can be constructed in a similar way.

19 8 Exercises

1. Change the Maple script of Example 1 in such a way that the modiﬁed equations for the implicit Euler method, the implicit midpoint rule, or the trapezoidal rule are obtained. Observe that for symmetric methods one gets expansions in even powers of h.

2. Compute a ﬁrst integral of the Lotka–Volterra equations (Example 2) and of the truncated modiﬁed equation for the symplectic Euler method.

3. Let Q = χ(q) be a change of position coordinates. Prove that the relation p = χ′(q)TP extend this to a symplectic mapping (p, q) (P, Q). T 7→ Hint. Consider the generating function S(P, q)= P χ(q).

4. Let y = ψ(z) be a symplectic change of coordinates. Prove that it transforms y˙ = J −1 H(y) into z˙ = J −1 K(z) with K(z)= H(y)= H ψ(z) . ∇ ∇ 5. (Field & Nijhoff 2003)7 Apply the symplectic Euler method to the system with Hamiltonian H(p, q) = ln(α + p) + ln(β + q). Compute the modified Hamiltonian and prove that the series converges for sufficiently small step sizes. Hint. The method conserves exactly I(p, q)=(α + p)(β + q). Find linear two-term recursions for p and q , and use the ideas of Example 3. { n} { n} Result. hk I(p, q)−k H(p, q)= H(p, q) . − k(k + 1) Xk≥1 e 6. Consider a differential equation y˙ = f(y) with a divergence-free vector field, and apply a volume-preserving integrator. Show that every truncation of the modified equation has again a divergence-free vector field. Hint. Adapt the proof by induction of Theorem 2.

7C.M. Field & F.W. Nijhoff, A note on modiﬁed Hamiltonians for numerical integrations ad- mitting an exact invariant, Nonlinearity 16 (2003) 1673–1683.