arXiv:chao-dyn/9403001v1 9 Mar 1994 ogtms ecmeto h eeac oamshrcpredi atmospheric 05.45.+b,92.60.Wc to relevance literature. an meteorological the the short on in both comment studied for We exponents, times. Lyapunov predicts long local quantitatively the that from model scaling simple chao a deterministic demonstrate out FailuWe rule cannot growth. scaling exponential predic error of long ponential and evidence attractor—al short obscuring the both even of at times att size errors an the in over scalings near exponential exponents growth local error globa of of and variations saturation local large dire the the between more ponents, difference are The however, predictability. exponents, largest to Lyapunov the by Local governed rate exponent. exponential an at separate will rdcinErr n oa ypnvExponents Lyapunov Local and Errors Prediction ti rqetyasre hti hoi ytmtoinitia two system chaotic a in that asserted frequently is It nttt o olna cec nvriyo aiona Sa California, of University Science Nonlinear for Institute cip nttto fOceanography of Institution Scripps nttt o olna Science Nonlinear for Institute nttt o olna Science Nonlinear for Institute aiePyia Laboratory Physical Marine .J “i” Sidorowich (“Sid”) J. J. er .I Abarbanel I. D. Henry aJla A92093-0402 CA Jolla, La eateto Physics of Department eateto Physics of Department ate .Kennel B. Matthew Nvme ,2018) 7, (November alCd 0402 Code Mail Abstract and and and 1 ∗ † ‡ sa explanation. an as s intms some- times, tion et bev ex- observe to re lblLyapunov global atr n the and ractor, euti non- in result l ypnvex- Lyapunov l yee sn REVT using Typeset l ls points close lly bevderror observed surprisingly d tyrelevant ctly Diego n tblt as ctability E X Chaotic behavior in dynamical systems is intermediate between precisely predictable, regular evolution and completely unpredictable, stochastic evolution. The global Lyapunov exponents [1,2] quantify the evolution of perturbations as the system evolves for long times. A positive global Lyapunov exponent implies that errors grow to the overall size of the attractor limiting predictability to finite times. Given observed data from a chaotic source, one can construct accurate empirical predic- tors [3,4,6] which allow predictions for finite times—without any knowledge of the underlying dynamics. Positive global Lyapunov exponents causes the errors in these predictions to grow exponentially rapidly, and it is conventionally assumed that the prediction error E(t) will grow as

E(t)= E(0) exp(λ1t). (1)

λ1 is the largest global Lyapunov exponent. Indeed this error growth is used as a diagnostic of deterministic chaos in the analysis of observed data [4,5]. This paper is devoted to a closer examination of this common assumption, and we will show it is often incorrect for times where the system is predictable. For t →∞ the largest

global Lyapunov exponent λ1 is defined in terms of a long-term average growth rate, but it does not always reflect the actual growth of prediction errors in finite time. Further for t → ∞ the perturbed orbit, although eventually uncorrelated with the reference orbit, is constrained to stay on the attractor, and thus the size of a perturbation will saturate near the overall size of the attractor. Reconciling these two aspects of error growth leads to our improvement to Equation (1). We briefly review the definition of local Lyapunov exponents. Our discrete time dynam- ical system is y(n +1) = F(y(n)). Small perturbations δy(n) to this orbit evolve L steps forward in time according to the linearized dynamics

δy(n + L)= DFL(y(n)) · δy(n). (2)

L With DFab(y)= ∂Fa(y)/∂yb the Jacobian matrix of the dynamics, we denote DF (y(n)= L−1 DF(y(n + i)). The Oseledec matrix [2], Qi=0

L T L 1/2L O(y, L)= h(DF (y)) · DF (y)i (3)

2 has eigenvalues eλ1(y,L), eλ2(y,L),...,eλd(y,L), in d-dimensions. We order the exponents as

λ1 ≥ λ2 ≥ λ3 ... ≥ λd. These local exponents λa(y, L) address the growth (or decay) over L time steps of perturbations made around some point y in the state space. As L → ∞

λa(y, L) → λa, the global exponents. Taken over an initially isotropic distribution the initial root-mean-squared Euclidean distance of the evolved perturbation vectors to the origin will grow by ( 1 d Λ2)1/2. d Pi=1 i Λa(x, L) = exp[Lλa(x, L)]. Lorenz first derived this formula in an early paper [9]. We employ the analogous quantity appropriate for a geometric mean instead of the arithmetic mean. We found this to be 1 d Λ via numerical experiment, though we do not yet know d Pi=1 i of a simple, general derivation corresponding to that in reference [9]. We note that direct nu- merical computation of the matrix product of many Jacobians leads to ill-conditioning, and thus for practical computation we employ the algorithm of reference [8] to stably compute the local Lyapunov exponents. The quantity E(x, L)= 1 d Λ (x, L), which we denote the “expansion factor”, quan- d Pi=1 i tifies the multiplicative growth of typical predictor errors starting at x, looking ahead L steps. The expansion factor is best calculated from local exponents as

L(λ2(x,L)−λ1(x,L)) L(λ3(x,L)−λ1(x,L)) log E(x, L)= Lλ1(x, L) + log[1 + e + e + ... + eL(λd(x,L)−λ1(x,L))] − log d (4)

For increasing L, λ1(x, L) quickly dominates the expansion factor. This also motivates Equa-

tion (1), but here λ1(x, L), is the finite-time Lyapunov exponent, which does not converge

very quickly to λ1. [7]. The other ingredient in our model for the average prediction error is a saturation cutoff. If we limit the maximal expansion factor for each initial condition to a constant R, we then compute the geometric mean (over reference points on the attractor x(i)) of the expansion factors E(x, L), which is the arithmetic mean of log E(x, L), hard-limited by ρ = log R:

1 N x log X(L, ρ)= X min [log E( (i), L), ρ] − ρ. (5) N i=1

The saturation cutoff models the fact that finite-sized perturbations and prediction errors cannot grow indefinitely. We estimate R is the ratio of the geometric mean of |x(i) − x(j)| over uncorrelated pairs of attractor points to the geometric mean of the initial perturbation

3 magnitude. Larger R corresponds to better predictability. Note that we are averaging individually thresholded expansion factors—not thresholding an average expansion factor. Before the saturation sets in log X(L) ≈ Lλ¯(L), with λ¯(L) the average finite-time Lyapunov exponent [7,8]. We now compare (1) the growth of errors actually incurred by making repeated predic- tions and (2) the growth rate implied by the thresholded expansion factors, Equation (5). This suggests that prediction errors grow as do the separation of initially close trajectories. This means we are considering the errors in prediction that arise from inherent dynamical instability and not inaccuracies as a result of specific features of the prediction scheme. Our measure of average prediction error is the geometric mean over initial conditions of an L-step iterated predictor, a composition of L one-step predictors. The error is normalized by the geometric mean distance between all time-decorrelated pairs of points on the attractor, so that the absence of predictability corresponds to zero logarithmic error. With G(•) the one-step predictor, the normalized prediction error L steps ahead is

1 N 1 N log χ(L)= log |GL(x(i)) − x(i + L)| − log |x(i) − x(j)|. (6) X 2 X N i=1 N i,j=1

As for log X(L, ρ) limL→∞ log χ(L) = 0. The iterated prediction is not rescaled to remain close to the reference trajectory. The main empirical result is that on chaotic attractors log χ(L) = log X(L, ρ) given the correct threshold ρ. We evaluate the local Lyapunov exponents, then compute the single parameter family of curves given by Equation (5) over a range of ρ. We find that once the best value of ρ is found, the resulting log X(L, ρ) quantitatively predicts the scaling of errors throughout the range of time examined. That equation (5) predicts the scaling of errors in the saturation region (L →∞) is somewhat surprising because Lyapunov exponents quantify linear expansion rates of infinitesimal perturbations, but in the saturation region one envisions typically large deviations. A possible explanation is that in the saturation regime average prediction error is controlled by the tail of the distribution of individual expansion factors: those initial conditions that happen to be exceptionally predictable and whose expansion factors thus remain below the threshold for especially long times. This is plausible because local Lyapunov exponents often have wide distributions [7,8].

Figure 1 shows iterated prediction errors,log10(χ(L)), computed using nonlinear kernel regression [6] versus thresholded expansion factors (5) for data from the Ikeda map of the

4 plane to itself [10]. The data set is 20,000 real and imaginary components from the complex- valued map z(n +1) = p + Bz(n) exp[iκ − iα/(1 + |z(n)|2)] with parameters p = 1.0, B = 0.9, κ = 0.4, α = 6.0. At short times, there is an obvious exponential scaling region (a line with a constant slope λ1), as predicted by Equation (1). Starting at L ≈ 10 the slope decreases and curves off to zero, well modeled by the thresholded expansion factor. We did not optimize ρ in any sophisticated manner, but simply stepped ρ by 0.2 and selected the best match. From a time series y(i), i =1 ...N, we compute a prediction for an input vector x as G(x) = N y(i + 1)K(x − y(i)) / N K(x − y(i)) using a Gaussian kernel K(z) = Pi=1  Pi=1  exp(−|z|2/σ2) with σ set to twice the geometric mean nearest-neighbor distance of the y(i). This is a global prediction formula but effectively functions as a localized interpolator. The local Lyapunov exponents were calculated from this same data set [8] without the equations.

Figure 2 shows the same information for data from a three dimensional flow of Lorenz [11]. The dynamical equations arex ˙ = −y2 −z2 −a(x−F ), y˙ = xy −bxz −y +G, z˙ = bxy +xz −z, with parameters set a =0.25, b =4.0, F =8.0, and G =1.0, sampled at intervals δt =0.2. The attractor has a dimension of about 2.5. There is no obvious exponential scaling regime here. Given only the curve of predictor errors, one could not easily identify the Lyapunov exponent, in contrast to the previous example. The appropriately thresholded expansion factor, however, agrees with the observed scaling of prediction errors. For contrast, the graph also shows the results of choosing either too small a threshold (the circles) or too large a threshold (the triangles) for this prediction scheme and data set. Now we examine our main result log χ(L) = log X(L, ρ) with a better controlled ex- periment. This time as our “prediction function” we use the actual dynamical equations of the Lorenz system but start the integration with an initial condition slightly perturbed from the reference point by a small uniformly distributed random vector η(i): GL(x(i)) = FL(x(i)+η(i)). We directly calculate Lyapunov exponents from the known differential equa- tions by simultaneously integrating the equations of motion and the tangent space flow. Fig- ure 3 compares log χ(L) with log X(L, ρ) with varying sizes of the initial perturbation. The growth of the perturbations matches the thresholded expansion factor, with the previously free parameter ρ fixed at the predicted value ρ = hlog |x(i)−x(j)|ii,j=1...N −hlog |η(i)|ii=1...N , confirming our model. This scaling with L is generic to most low dimensional flows: a curve at low L scaling with an initially high slope, because the average largest local exponent is larger than the global

5 exponent [8], an exponential region (constant slope) corresponding to the global Lyapunov exponent, and a long tail curving off to zero as the finite-size threshold takes effect. The size and existence of the region of constant slope depends on the size of the initial error. If the error is large enough, there may be no region of exponential scaling as the curvature due to finite-time exponents merges with the curvature due to thresholds. Higher-dimensional data are likely to have lower initial predictability than very low-dimensional attractors, and therefore likely to show little observably exponential error scaling. Intermittent dynamics will create a wide variation in finite-time Lyapunov exponents, thus moving forward the time where saturation begins to take effect. In analyzing dynamical systems more complex than simple one or two dimensional models, requiring manifestly exponential error scaling as confirming deterministic chaos is unrealistic.

We have identified empirical prediction error with the evolution of perturbations, but the identity does not always strictly hold. Figure 4 shows the prediction errors on the same Lorenz flow [11] data using an accurate local quadratic polynomial technique [4,5]. This time, the prediction error is not satisfactorily matched by the thresholded expansion factor with any ρ, the main difference being a substantially larger slope at small L. We attribute part of this discrepancy to the assumption, used in deriving the expansion factor, that the initial perturbations are distributed uniformly in all directions. When a very accurate model, such as this one, is combined with extremely clean data, short term forecasting errors will be quite small, and the predicted trajectory will closely shadow the attractor, and thus, the unstable manifold. Therefore the initial rate of expansion will be larger than for isotropically distributed errors, being described better by a new “primary” expansion factor Lλ1(x, L) that only considers error expansion due to the single largest local Lyapunov exponent. This quantity increases faster at short times than the standard expansion factor, but still, it fails to precisely match the scaling of forecast error. Another discrepancy not accounted for by the expansion factor is that iterating imperfect models injects new error at every timestep, not only at the beginning. Usually exponential expansion of old error dominates new error, except at the shortest times. This effect will cause yet another increase to the slope at small times. When we add some isotropic noise to the initial condition before applying the iterated local predictor, outstanding agreement with the scaling predicted by Equation (5) is restored. We note that the divergence between straight prediction error and expansion factor is exaggerated by the particular conditions in force here: very good prediction on a

6 data base of very clean low-dimensional data from a smooth chaotic flow. We performed the same test on noisier experimental chaotic data and saw a smaller difference. Is adding noise to the initial condition “cheating”? To be completely rigorous, one ought to model the distribution of prediction errors in the various directions corresponding to the local Lyapunov exponents. This is not possible without knowledge of detailed properties of the specific prediction scheme, and is obviously beyond the scope of this paper. The approximation of isotropic perturbation may often be acceptable, and can be enforced by adding artificial noise. In the context of analyzing observed data, a successful match be- tween prediction errors (even with artificial initial perturbations) and thresholded expansion factors is a novel cross-check that affirms the validity of the modeling procedures, even if one cannot exactly quantify the error scaling for a particular prediction scheme. Accurate evaluation of Lyapunov exponents from degraded or high-dimensional data is somewhat difficult in practice, requiring good estimates of derivatives of the implied evolution func- tion. Various reconstruction parameters, such as embedding and local manifold dimensions and time delay [8,3] may yield substantially different numerical answers for Lyapunov ex- ponents, even if all are topologically acceptable. Further requiring a good match between expansion factors and error scaling provides a criterion for selecting among the otherwise equivalent choices. In general, we have found that prediction error scaling varies less with reconstruction parameters than Lyapunov exponents. Figure 5 shows hlog E(x(i), L)i (log X without the threshold) for the Lorenz flow. At the smallest times, the expansion factor does not in fact possess a constant slope, and only flattens out to a straight line, with slope equal to the infinite-time Lyapunov exponent after an initial transient regime. This curvature is always in the direction seen on this graph (higher slope at shorter times) and is a result of the fact that the average local Lyapunov exponent λ(L) approaches the global exponent from above as L → ∞ [8]. This curvature explains non-exponential scaling of prediction error for early time intervals. Also shown is the mean plus and minus one standard deviation of the distribution of individual expansion factors E(x, L). The Figure demonstrates that the variation in local exponents as a function of initial condition causes a wide variation in expansion factor that increases with increasing L. The interaction of this wide distribution with the threshold results in the saturation of prediction error for longer times. The effects of the finite size of the attractor begin to manifest themselves when an appreciable number of the individual expansion factors approach the cutoff, which occurs well before the mean expansion factor does so.

7 Notice that the width of the distribution increases as L → ∞; we have observed this feature in all systems we examined. An important conclusion is that even though the local Lyapunov exponent converges to a single global exponent which is independent of

initial condition, i.e. limL→∞ λ1(x, L) = λ1, the expansion factors of various individual

initial conditions do not do so: limL→∞ E(x, L) =6 E¯(L). This is because the standard deviation of the distribution of local exponents h(λ(x, L) − λ(L))2i typically decreases at a rate substantially slower than L−1. Of course, considering global saturation, normalized error will eventually converge to unity, but this is an entirely different mechanism. The message is that considering dynamical predictability solely in view of the largest global Lyapunov exponent may be conceptually misleading as well as quantitatively inaccurate. The temporal development of forecast error has been a prime concern of the weather forecasting community for many years, starting with work of Lorenz [9,12]. Common practice has been to hypothesize ad hoc relationships for the average error as a function of time; usually, in the form of a differential equation. Lorenz found [12] that increase of mean deviation between initially close initial atmospheric states could be fitted by an empirical law of the form

E˙ (t)= αE(t) (1 − E(t)/E(∞)) . (7)

A positive Lyapunov exponent motivates the first term; the inevitable saturation at max- imum error, the second. Stroe and Royer [13] compared generalized parameterizations of empirical growth laws that include Lorenz’s with results of large-scale atmospheric simulations and some experimental observations. The scaling of mean error at larger times, in the saturation regime, appeared to be better fit by an exponential law such as d log E(t)/dt = −β log E(t), similar to results seen in our work, but a single empirical rule governing the initial growth of errors was not clearly indicated, either in this or previ- ous studies. We explain this with the fact that the initial growth of error is governed by finite-time, rather than global, Lyapunov exponents. This results in an initial regime of non-exponential growth, second, this error growth rate depends on the coordinate system used [7]. The differing measures of phase-space distance employed by atmospheric scientists will result in different growth rates making the notion of a universal “doubling time for small errors” less useful than commonly believed. Our modeling of the error growth, which holds the mean finite-time Lyapunov exponents responsible at small error, but with their variation most important at saturation, backs up Stroe and Royer’s conclusions that the infinitesimal

8 growth rate cannot be easily deduced from the saturation rate via fitting a single empirical formula like equation (7). One further point that we wish to make is that we have observed that using the arithmetic mean for the ensemble average of both error and expansion fac- tors rather than the geometric results in far more “noisy” curves, and thus is it not clear whether expansion factors accurately match prediction errors with ensemble averaging in the arithmetic sense, though we suspect so. The arithmetic mean, commonly employed in me- teorological literature, seems to be dominated by fluctuations in a few samples in the large error tail of the distribution. Convergence of the arithmetical ensemble average appears to require numbers of points excessively large even for this low-dimensional investigation. Choosing an arithmetical average also appears to accentuate the initial superexponential growth. Still, the qualitative behavior of error scaling seen in large-scale atmospheric simu- lations and experiment [13] appears compatible with our model, which we suggest as a more fundamental explanation for observed error growth. In the nonlinear dynamics literature, one example in Farmer and Sidorowich [4] demonstrated initially faster-than-exponential error scaling, but remained unexplained in that work. With the exception of the work of Lorenz [9], the direct use of local Lyapunov exponents to quantify error growth in atmospheric dynamics is rather recent [14–16] and remains open to further development. The Lyapunov exponents have generally been previously considered only in the context of infinitesimal errors. This present work shows that accounting for a threshold apparently allows finite-time Lyapunov exponents to quantify error scaling at both small and substantially larger levels of error. Work remains concerning more realistic models, of course. Results of other low-dimensional chaotic data sets that we have examined, including experimental data, agree with the conclusions of this paper.

9 REFERENCES

∗ E-mail: [email protected] † E-mail: [email protected] ‡ E-mail: [email protected] [1] Eckmann, J.-P. and D. Ruelle “Ergodic Theory of Chaos and Strange Attractors” Rev. Mod. Phys. 57, 617 (1985). [2] Oseledec, V. I., “A Multiplicative Ergodic Theorem. Lyapunov Characteristic Numbers for Dynamical Systems” Trudy Mosk. Mat. Obsc 19, 197 (1968); Moscow Math. Soc. 19, 197 (1968). [3] Abarbanel, H. D. I., R. Brown, J. (“Sid”) Sidorowich, and Lev Sh. Tsimring, “The Analysis of Observed Chaotic Data in Physical Systems”, Reviews of Modern Physics 65, 1331-1392 (1993). [4] J. D. Farmer and J. Sidorowich, Phys. Rev. Lett., 59, 845 (1987) [5] J. D. Farmer and J. Sidorowich, “Exploiting Chaos to Predict the Future and Reduce Noise”. In Evolution, Learning and Cognition, ed. Y.C. Lee, World Scientific, Singapore (1988) [6] H. D. I. Abarbanel, R. Brown, and J. B. Kadtke, “Prediction in Chaotic Nonlinear Systems: Methods for Time Series with Broadband Fourier Spectra”, Phys. Rev. A 41, 1782 (1990). [7] H. D. I. Abarbanel, R. Brown, and M. B. Kennel , “Variability of Lyapunov Exponents on a Strange Attractor”, Journal of Nonlinear Science, 1, 175-199 (1991). [8] H. D. I. Abarbanel, R. Brown, and M. B. Kennel, “Local Lyapunov Exponents from Observed Data”, Journal of Nonlinear Science 2, 343-365 (1992). [9] E. N. Lorenz, Tellus 17, 321–333 (1965) [10] K. Ikeda, Opt. Commun. 30, 257 (1979). [11] E. N. Lorenz, Tellus 36A, 98-110 (1984). [12] E. N. Lorenz, J. Atmos. Sci. 26 636–646 (1969) [13] R. Stroe, J.F. Royer, Ann. Geophysicae 11, 296–316 (1993) [14] B.F. Farrell J. Atmos. Sci. 47 2409–2416 (1990) [15] J.M. Nese, J.A. Dutton J. Climate 6 185–203 (1993) [16] S. Yoden, M. Nomura J. Atmos. Sci 50 1531–1543 (1993)

10 FIGURES

FIG. 1. log10 χ(L) (solid line) and log10 X(L, ρ) (circles) for the Ikeda map. ρ = 2.4 gives a good fit.

FIG. 2. log10 χ(L) (solid line) and log10 X(L, ρ) for data from the Lorenz flow, with ρ = 1.2 (triangles), ρ = 1.6 (circles), and ρ = 2.0 (diamonds).

FIG. 3. log10 χ(L) (solid lines) and log10 X(L, ρ) (symbols) using known flow equations and varying sizes of initial perturbations.

FIG. 4. log10 χ(L) and log10 X(L, ρ) (symbols) using local quadratic prediction, with no initial perturbation (solid line), and a 1% perturbation (dashed line) before prediction.

FIG. 5. Geometric mean of local expansion factors (solid line), plus one standard deviation of the distribution (dotted line) and minus one standard deviation (dashed line), for Lorenz flow.

11 0.0

-1.0 (L) χ

10 Prediction error ), log ρ ρ Expansion factor, =2.4 X(L, 10 -2.0 log

-3.0 0 10 20 30 L 0.0 (L) χ

10 Prediction error ), log -1.0 ρ ρ Expansion factor, =1.6 Expansion factor, ρ=1.2

X(L, Expansion factor, ρ=2.0 10 log

-2.0 0 50 100 150 200 L 0.0

-1.0 (L) χ

10 ), log ρ

X(L, Prediction error, 1% noise

10 Prediction error, 0.1% noise -2.0 log Expansion factor, ρ=2.0 Expansion factor, ρ=3.0

-3.0 0 50 100 150 200 L 0.0

-1.0 (L) χ -2.0 10 ), log ρ

X(L, -3.0 10 Prediction error, no noise log Prediction error, 1% noise Expansion factor, ρ=2.0 -4.0 Expansion factor, ρ=4.0 ``Primary'' expansion factor, ρ=4.0

-5.0 0 50 100 150 200 L 5.0

average expansion factor (unthresholded) plus one standard deviation 4.0 minus one standard deviation

3.0 E(x,L)> 10 2.0

1.0

0.0 0 50 100 150 200 L

Prediction Errors and Lo cal Lyapunov Exp onents

Matthew B Kennel

Institute for Nonlinear Science

and

Department of Physics

y

Henry D I Abarbanel

Department of Physics

and

Marine Physical Laboratory

Scripps Institution of Oceanography

and

Institute for Nonlinear Science

z

J J Sid Sidorowich

Institute for Nonlinear Science University of California San Diego

Mail Code

La Jol la CA

March

Abstract

It is frequently asserted that in a chaotic system two initially close p oints

will separate at an exp onential rate governed by the largest global Lyapunov

exp onent Lo cal Lyapunov exp onents however are more directly relevant

to predictability The dierence b etween the lo cal and global Lyapunov ex

p onents the large variations of lo cal exp onents over an attractor and the

saturation of error growth near the size of the attractorall result in non

exp onential scalings in errors at b oth short and long prediction times some

times even obscuring evidence of exp onential growth Failure to observe ex

p onential error scaling cannot rule out deterministic chaos as an explanation

We demonstrate a simple mo del that quantitatively predicts observed error

scaling from the lo cal Lyapunov exp onents for b oth short and surprisingly

long times We comment on the relevance to atmospheric predictability as

studied in the meteorological literature

bWc

Typeset using REVT X

E

Chaotic b ehavior in dynamical systems is intermediate b etween precisely predictable

regular evolution and completely unpredictable sto chastic evolution The global Lyapunov

exp onents quantify the evolution of p erturbations as the system evolves for long times

A p ositive global Lyapunov exp onent implies that errors grow to the overall size of the

attractor limiting predictability to nite times

Given observed data from a chaotic source one can construct accurate empirical predic

tors which allow predictions for nite timeswithout any knowledge of the underlying

dynamics Positive global Lyapunov exp onents causes the errors in these predictions to grow

exp onentially rapidly and it is conventionally assumed that the prediction error E t will

grow as

E t E exp t

1

is the largest global Lyapunov exp onent Indeed this error growth is used as a diagnostic

1

of deterministic chaos in the analysis of observed data

This pap er is devoted to a closer examination of this common assumption and we will

show it is often incorrect for times where the system is predictable For t the largest

global Lyapunov exp onent is dened in terms of a longterm average growth rate but

1

it do es not always reect the actual growth of prediction errors in nite time Further for

t the p erturb ed orbit although eventually uncorrelated with the reference orbit is

constrained to stay on the attractor and thus the size of a p erturbation will saturate near

the overall size of the attractor Reconciling these two asp ects of error growth leads to our

improvement to Equation

We briey review the denition of lo cal Lyapunov exp onents Our discrete time dynam

ical system is yn Fyn Small p erturbations yn to this orbit evolve L steps

forward in time according to the linearized dynamics

L

yn L DF yn yn

L

With DF y F y y the Jacobian matrix of the dynamics we denote DF yn

ab a b

Q

L1

DFyn i The Oseledec matrix

i=0

h i

12L

L L

T

O y L DF y DF y

(yL) (yL) (yL)

1 2

d

has eigenvalues e e e in ddimensions We order the exp onents as

These lo cal exp onents y L address the growth or decay over

1 2 3 d a

L time steps of p erturbations made around some p oint y in the state space As L

y L the global exp onents

a a

Taken over an initially isotropic distribution the initial ro otmeansquared Euclidean

P

d 1

2 12

distance of the evolved p erturbation vectors to the origin will grow by

i=1 i

d

x L expL x L Lorenz rst derived this formula in an early pap er We

a a

employ the analogous quantity appropriate for a geometric mean instead of the arithmetic

P

1 d

mean We found this to b e via numerical exp eriment though we do not yet know

i

i=1

d

of a simple general derivation corresp onding to that in reference We note that direct nu

merical computation of the matrix pro duct of many Jacobians leads to illconditioning and

thus for practical computation we employ the algorithm of reference to stably compute

the lo cal Lyapunov exp onents

P

d

1

The quantity E x L x L which we denote the expansion factor quan

i

i=1

d

ties the multiplicative growth of typical predictor errors starting at x lo oking ahead L

steps The expansion factor is b est calculated from lo cal exp onents as

L( (xL) (xL)) L( (xL) (xL))

2 1 3 1

log E x L L x L log e e

1

L( (xL) (xL))

1

d

e log d

For increasing L x L quickly dominates the expansion factor This also motivates Equa

1

tion but here x L is the nitetime Lyapunov exp onent which do es not converge

1

very quickly to

1

The other ingredient in our mo del for the average prediction error is a saturation cuto

If we limit the maximal expansion factor for each initial condition to a constant R we then

compute the geometric mean over reference p oints on the attractor xi of the expansion

factors E x L which is the arithmetic mean of log E x L hardlimited by log R

N

X

log X L min log E xi L

N

i=1

The saturation cuto mo dels the fact that nitesized p erturbations and prediction errors

cannot grow indenitely We estimate R is the ratio of the geometric mean of jxi xj j

over uncorrelated pairs of attractor p oints to the geometric mean of the initial p erturbation

magnitude Larger R corresp onds to b etter predictability Note that we are averaging

individually thresholded expansion factorsnot thresholding an average expansion factor

Before the saturation sets in log X L L L with L the average nitetime Lyapunov

exp onent

We now compare the growth of errors actually incurred by making rep eated predic

tions and the growth rate implied by the thresholded expansion factors Equation

This suggests that prediction errors grow as do the separation of initially close tra jectories

This means we are considering the errors in prediction that arise from inherent dynamical

instability and not inaccuracies as a result of sp ecic features of the prediction scheme

Our measure of average prediction error is the geometric mean over initial conditions of an

Lstep iterated predictor a comp osition of L onestep predictors The error is normalized by

the geometric mean distance b etween all timedecorrelated pairs of p oints on the attractor

so that the absence of predictability corresp onds to zero logarithmic error With G the

onestep predictor the normalized prediction error L steps ahead is

N N

X X

L

log jG xi xi Lj log jxi xj j log L

2

N N

i=1 ij =1

As for log X L lim log L The iterated prediction is not rescaled to remain

L

close to the reference tra jectory

The main empirical result is that on chaotic attractors log L log X L given the

correct threshold We evaluate the lo cal Lyapunov exp onents then compute the single

parameter family of curves given by Equation over a range of We nd that once

the b est value of is found the resulting log X L quantitatively predicts the scaling

of errors throughout the range of time examined That equation predicts the scaling of

errors in the saturation region L is somewhat surprising b ecause Lyapunov exp onents

quantify linear expansion rates of innitesimal p erturbations but in the saturation region

one envisions typically large deviations A p ossible explanation is that in the saturation

regime average prediction error is controlled by the tail of the distribution of individual

expansion factors those initial conditions that happ en to b e exceptionally predictable and

whose expansion factors thus remain b elow the threshold for esp ecially long times This is

plausible b ecause lo cal Lyapunov exp onents often have wide distributions

Figure shows iterated prediction errorslog L computed using nonlinear kernel

10

regression versus thresholded expansion factors for data from the Ikeda map of the

plane to itself The data set is real and imaginary comp onents from the complex

2

valued map z n p B z n expi i jz nj with parameters p B

At short times there is an obvious exp onential scaling region a

line with a constant slop e as predicted by Equation Starting at L the slop e

1

decreases and curves o to zero well mo deled by the thresholded expansion factor We did

not optimize in any sophisticated manner but simply stepp ed by and selected the b est

match From a time series yi i N we compute a prediction for an input vector x as

P P

N N

Gx yi K x yi K x yi using a Gaussian kernel K z

i=1 i=1

2 2

expjzj with set to twice the geometric mean nearestneighbor distance of the yi

This is a global prediction formula but eectively functions as a lo calized interpolator The

lo cal Lyapunov exp onents were calculated from this same data set without the equations

Figure shows the same information for data from a three dimensional ow of Lorenz

2 2

The dynamical equations are x y z ax F y xy bxz y G z bxy xz z

with parameters set a b F and G sampled at intervals t

The attractor has a dimension of ab out There is no obvious exp onential scaling regime

here Given only the curve of predictor errors one could not easily identify the Lyapunov

exp onent in contrast to the previous example The appropriately thresholded expansion

factor however agrees with the observed scaling of prediction errors For contrast the

graph also shows the results of choosing either to o small a threshold the circles or to o

large a threshold the triangles for this prediction scheme and data set

Now we examine our main result log L log X L with a b etter controlled ex

p eriment This time as our prediction function we use the actual dynamical equations

of the Lorenz system but start the integration with an initial condition slightly p erturb ed

L

from the reference p oint by a small uniformly distributed random vector i G xi

L

F xi i We directly calculate Lyapunov exp onents from the known dierential equa

tions by simultaneously integrating the equations of motion and the tangent space ow Fig

ure compares log L with log X L with varying sizes of the initial p erturbation The

growth of the p erturbations matches the thresholded expansion factor with the previously

free parameter xed at the predicted value hlog jxi xj ji hlog j iji

ij =1N i=1N

conrming our mo del

This scaling with L is generic to most low dimensional ows a curve at low L scaling with

an initially high slop e b ecause the average largest lo cal exp onent is larger than the global

exp onent an exp onential region constant slop e corresp onding to the global Lyapunov

exp onent and a long tail curving o to zero as the nitesize threshold takes eect The

size and existence of the region of constant slop e dep ends on the size of the initial error If

the error is large enough there may b e no region of exp onential scaling as the curvature due

to nitetime exp onents merges with the curvature due to thresholds Higherdimensional

data are likely to have lower initial predictability than very lowdimensional attractors and

therefore likely to show little observably exp onential error scaling Intermittent dynamics

will create a wide variation in nitetime Lyapunov exp onents thus moving forward the

time where saturation b egins to take eect In analyzing dynamical systems more complex

than simple one or two dimensional mo dels requiring manifestly exp onential error scaling

as conrming deterministic chaos is unrealistic

We have identied empirical prediction error with the evolution of p erturbations but

the identity do es not always strictly hold Figure shows the prediction errors on the same

Lorenz ow data using an accurate lo cal quadratic p olynomial technique This

time the prediction error is not satisfactorily matched by the thresholded expansion factor

with any the main dierence b eing a substantially larger slop e at small L We attribute

part of this discrepancy to the assumption used in deriving the expansion factor that the

initial p erturbations are distributed uniformly in all directions When a very accurate mo del

such as this one is combined with extremely clean data short term forecasting errors will

b e quite small and the predicted tra jectory will closely shadow the attractor and thus the

unstable manifold Therefore the initial rate of expansion will b e larger than for isotropically

distributed errors b eing describ ed b etter by a new primary expansion factor L x L

1

that only considers error expansion due to the single largest lo cal Lyapunov exp onent This

quantity increases faster at short times than the standard expansion factor but still it fails

to precisely match the scaling of forecast error Another discrepancy not accounted for by

the expansion factor is that iterating imp erfect mo dels injects new error at every timestep

not only at the b eginning Usually exp onential expansion of old error dominates new error

except at the shortest times This eect will cause yet another increase to the slop e at

small times When we add some isotropic noise to the initial condition b efore applying the

iterated lo cal predictor outstanding agreement with the scaling predicted by Equation

is restored We note that the divergence b etween straight prediction error and expansion

factor is exaggerated by the particular conditions in force here very go o d prediction on a

data base of very clean lowdimensional data from a smo oth chaotic ow We p erformed

the same test on noisier exp erimental chaotic data and saw a smaller dierence

Is adding noise to the initial condition cheating To b e completely rigorous one ought

to mo del the distribution of prediction errors in the various directions corresp onding to the

lo cal Lyapunov exp onents This is not p ossible without knowledge of detailed prop erties

of the sp ecic prediction scheme and is obviously b eyond the scop e of this pap er The

approximation of isotropic p erturbation may often b e acceptable and can b e enforced by

adding articial noise In the context of analyzing observed data a successful match b e

tween prediction errors even with articial initial p erturbations and thresholded expansion

factors is a novel crosscheck that arms the validity of the mo deling pro cedures even if

one cannot exactly quantify the error scaling for a particular prediction scheme Accurate

evaluation of Lyapunov exp onents from degraded or highdimensional data is somewhat

dicult in practice requiring go o d estimates of derivatives of the implied evolution func

tion Various reconstruction parameters such as embedding and lo cal manifold dimensions

and time delay may yield substantially dierent numerical answers for Lyapunov ex

p onents even if all are top ologically acceptable Further requiring a go o d match b etween

expansion factors and error scaling provides a criterion for selecting among the otherwise

equivalent choices In general we have found that prediction error scaling varies less with

reconstruction parameters than Lyapunov exp onents

Figure shows hlog E xi Li log X without the threshold for the Lorenz ow At

the smallest times the expansion factor do es not in fact p ossess a constant slop e and only

attens out to a straight line with slop e equal to the innitetime Lyapunov exp onent after

an initial transient regime This curvature is always in the direction seen on this graph

higher slop e at shorter times and is a result of the fact that the average lo cal Lyapunov

exp onent L approaches the global exp onent from ab ove as L This curvature

explains nonexp onential scaling of prediction error for early time intervals Also shown is

the mean plus and minus one standard deviation of the distribution of individual expansion

factors E x L The Figure demonstrates that the variation in lo cal exp onents as a function

of initial condition causes a wide variation in expansion factor that increases with increasing

L The interaction of this wide distribution with the threshold results in the saturation

of prediction error for longer times The eects of the nite size of the attractor b egin

to manifest themselves when an appreciable number of the individual expansion factors

approach the cuto which o ccurs well b efore the mean expansion factor do es so

Notice that the width of the distribution increases as L we have observed this

feature in all systems we examined An imp ortant conclusion is that even though the

lo cal Lyapunov exp onent converges to a single global exp onent which is independent of

initial condition ie lim x L the expansion factors of various individual

L 1 1

initial conditions do not do so lim E x L E L This is b ecause the standard

L

2

deviation of the distribution of lo cal exp onents hx L L i typically decreases at

1

a rate substantially slower than L Of course considering global saturation normalized

error will eventually converge to unity but this is an entirely dierent mechanism The

message is that considering dynamical predictability solely in view of the largest global

Lyapunov exp onent may b e conceptually misleading as well as quantitatively inaccurate

The temp oral development of forecast error has b een a prime concern of the weather

forecasting community for many years starting with work of Lorenz Common practice

has b een to hypothesize ad hoc relationships for the average error as a function of time

usually in the form of a dierential equation Lorenz found that increase of mean

deviation b etween initially close initial atmospheric states could b e tted by an empirical

law of the form

E t E t E tE

A p ositive Lyapunov exp onent motivates the rst term the inevitable saturation at max

imum error the second Stro e and Royer compared generalized parameterizations

of empirical growth laws that include Lorenzs with results of largescale atmospheric

simulations and some exp erimental observations The scaling of mean error at larger

times in the saturation regime app eared to b e b etter t by an exp onential law such as

d log E tdt log E t similar to results seen in our work but a single empirical rule

governing the initial growth of errors was not clearly indicated either in this or previ

ous studies We explain this with the fact that the initial growth of error is governed by

nitetime rather than global Lyapunov exp onents This results in an initial regime of

nonexp onential growth second this error growth rate dep ends on the co ordinate system

used The diering measures of phasespace distance employed by atmospheric scientists

will result in dierent growth rates making the notion of a universal doubling time for small

errors less useful than commonly b elieved Our mo deling of the error growth which holds

the mean nitetime Lyapunov exp onents resp onsible at small error but with their variation

most imp ortant at saturation backs up Stro e and Royers conclusions that the innitesimal

growth rate cannot b e easily deduced from the saturation rate via tting a single empirical

formula like equation One further p oint that we wish to make is that we have observed

that using the arithmetic mean for the ensemble average of b oth error and expansion fac

tors rather than the geometric results in far more noisy curves and thus is it not clear

whether expansion factors accurately match prediction errors with ensemble averaging in the

arithmetic sense though we susp ect so The arithmetic mean commonly employed in me

teorological literature seems to b e dominated by uctuations in a few samples in the large

error tail of the distribution Convergence of the arithmetical ensemble average app ears

to require numbers of p oints excessively large even for this lowdimensional investigation

Cho osing an arithmetical average also app ears to accentuate the initial sup erexp onential

growth Still the qualitative b ehavior of error scaling seen in largescale atmospheric simu

lations and exp eriment app ears compatible with our mo del which we suggest as a more

fundamental explanation for observed error growth In the nonlinear dynamics literature

one example in Farmer and Sidorowich demonstrated initially fasterthanexp onential

error scaling but remained unexplained in that work

With the exception of the work of Lorenz the direct use of lo cal Lyapunov exp onents

to quantify error growth in atmospheric dynamics is rather recent and remains op en

to further development The Lyapunov exp onents have generally b een previously considered

only in the context of innitesimal errors This present work shows that accounting for a

threshold apparently allows nitetime Lyapunov exp onents to quantify error scaling at

b oth small and substantially larger levels of error Work remains concerning more realistic

mo dels of course Results of other lowdimensional chaotic data sets that we have examined

including exp erimental data agree with the conclusions of this pap er

REFERENCES

Email mbkinlsucsdedu

y

Email hdiahamiltonucsdedu

z

Email sidinlsucsdedu

Eckmann JP and D Ruelle Ergo dic Theory of Chaos and Strange Attractors Rev

Mod Phys

Oseledec V I A Multiplicative Ergo dic Theorem Lyapunov Characteristic Numbers

for Dynamical Systems Trudy Mosk Mat Obsc Moscow Math Soc

Abarbanel H D I R Brown J Sid Sidorowich and Lev Sh Tsimring The

Analysis of Observed Chaotic Data in Physical Systems Reviews of Modern Physics

J D Farmer and J Sidorowich Phys Rev Lett

J D Farmer and J Sidorowich Exploiting Chaos to Predict the Future and Reduce

Noise In Evolution Learning and Cognition ed YC Lee World Scientic Singap ore

H D I Abarbanel R Brown and J B Kadtke Prediction in Chaotic Nonlinear

Systems Metho ds for Time Series with Broadband Fourier Sp ectra Phys Rev A

H D I Abarbanel R Brown and M B Kennel Variability of Lyapunov Exp onents

on a Strange Attractor Journal of Nonlinear Science

H D I Abarbanel R Brown and M B Kennel Lo cal Lyapunov Exp onents from

Observed Data Journal of Nonlinear Science

E N Lorenz Tellus

K Ikeda Opt Commun

E N Lorenz Tellus A

E N Lorenz J Atmos Sci

R Stro e JF Royer Ann Geophysicae

BF Farrell J Atmos Sci

JM Nese JA Dutton J Climate

S Yoden M Nomura J Atmos Sci

FIGURES

FIG log L solid line and log X L circles for the Ikeda map gives a

10 10

go o d t

FIG log L solid line and log X L for data from the Lorenz ow with

10 10

triangles circles and diamonds

FIG log L solid lines and log X L symbols using known ow equations and

10 10

varying sizes of initial p erturbations

FIG log L and log X L symbols using lo cal quadratic prediction with no initial

10 10

p erturbation solid line and a p erturbation dashed line b efore prediction

FIG Geometric mean of lo cal expansion factors solid line plus one standard deviation of

the distribution dotted line and minus one standard deviation dashed line for Lorenz ow