Localized Lyapunov Exponents and the Prediction of Predictability

Home , Ikeda map, Lyapunov exponent

¡ ¢

Physics Letters A 9909 2000 xxx

¥ ¤ www.elsevier.nl£ locate pla

Localized Lyapunov exponents and the

prediction of predictability ¦ Christine Ziehmann a, § , Leonard A. Smith a,b,c, JurgenÈ Kurths a

¨ a

Uni© ersitat Potsdam, Institut fur Physik, Nichtlineare Dynamik, Postfach 60 15 53, D-14469, Potsdam, Germany

ÈÈ b

Mathematical Institute,Uni ersity of Oxford, Oxford, OX1 3LB, UK

c London School of Economics, London WC2A 2AE, UK

Received 9 December 1999; received in revised form 19 April 2000; accepted 3 May 2000 Communicated by C.R. Doering

Abstract

Every forecast should include an estimate of its likely accuracy, a current measure of predictability. Two distinct types of localized Lyapunov exponents based on infinitesimal uncertainty dynamics are investigated to reflect this predictability. Regions of high predictability within which any initial uncertainty will decrease are proven to exist in two common chaotic

systems; potential implications of these regions are considered. The relevance of these results for finite size uncertainties is

PACS: 05.45.-a; 05.10.-a; 05.45.Tp

Keywords: Lyapunov exponents; Predictability; Chaos; Weather forecasting; Ensemble prediction; Nonlinear dynamics

1. Introduction ! difficulty in quantifying predictability: 1 the depen-

! dence of measures of predictability upon the particu-

" # $ lar metric adopted 2 the dependence of uncertainty

Prediction of predictability refers to the quantita- !

dynamics upon the magnitude of the uncertainty in

% &

tive attempt to assess the likely error in a particular

the initial condition, and 3 the fact that errors in the forecast a priori. There are at least three sources of

model are often unknown until after predictions are

' ( ) observed to fail. Lyapunov exponents 1±3 quantify * predictability through globally averaged effective + growth rates of uncertainty in the limits of large time

, and small uncertainty; thus by construction they are )

of limited use. To obtain a quantitative estimate of Corresponding author. Tel.: 49-331-977-1302, fax: 49-

the accuracy of a particular forecast, the local dy- 331-977-1142.

E-mail address: [email protected] C. Ziehmann . - namics of uncertainties about that initial condition

/ 0

2 C. Ziehmann et al. 3 Physics Letters A 00 ()2000 000±000

4 5

, are more relevant 4±9 . By allowing better risk not considered in this paper as there is no systematic

D D

assessment, a prediction of predictability is of value manner to include system H model mismatches, thus it ? in 6 any field from physics to economics; weather is assumed throughout the paper that the perfect forecasting provides a particular example relevant to model is known.

7 both fields. + Effecti 8 e growth rates defined over a fixed dura-

tion are also used to quantify predictability; they are I

9 employed daily in the operational weather forecast 2. Localized Lyapunov exponents

: ; < centers of Europe and North America 10,11 . In Section 2, the distinction between what will be called

The dynamics of infinitesimal uncertainties about

, J

finite time exponents and finite sample exponents is K

L M N

= a point x0 in an m-dimensional state space are

> J

shown to lie in the particular initial orientation of +

O P Q >

9 governed by the linear propagator, x0 , t , which

U T

the perturbation each considers for a given initial 9 m

R S

< evolves any infinitesimal initial uncertainty 0 R

J ,

condition; this can result in dramatically different ,

+ 8

8 about x0 forward for a time t along the system's

effecti e growth rates. Both types of exponent are J W

< trajectory to x :

called `Local Lyapunov exponents', and recognizing t

^ _ ` a

> ]

X Y Z [ b the distinction between them resolves some confu- L t x, \ t 0 .1

sion in the literature. In terms of predicting the

forecast accuracy, the finite time exponents are shown c In discrete time maps, the linear propagator over k

to be the more relevant quantities in Section 3, where ? iterations is simply the product of Jacobians along

e f g h

d k l m

J J

n o

L j it is also proven that the mean of the largest finite i

the trajectory, that is x0 ,k xk 1 ...

p q r s

t u

v w J time exponent does not provide an unbiased estimate J

x10x . x ) of the corresponding global exponent, and similarly For high dimensional systems, interest tends to be

for the mean of the smallest finite time exponent. In focused on subspaces which are likely to contain the

y z

, addition, in regions of state space where the largest fastest growing perturbations 4,10,11 . Two orienta-

{ |

* 6

finite time exponent is less than zero all perturbations of particular interest are i that which will

tions will shrink independent of their orientation; this } have grown the most under the linearized dynamics

~ n

d J

is investigated in Section 4 where such regions are , after k steps, k x , and ii the local orientation of

proven to exist in two common chaotic maps. the globally fastest growing direction, lx, which 1 Each class of Lyapunov exponent discussed in is sometimes called the Lyapunov vector 13 . The

this paper assumes the observational uncertainty is first of these orientations is defined by the singular

infinitesimal; of course as long as it remains in- B value decomposition 14 of the propagator: the

n ,

finitesimal it cannot limit predictability, and once it k J x are simply the right singular vectors of

is finite its growth is no longer quantified by Lya- J x,k . Each is associated with a singular value,

* J

punov exponents. Therefore, the rigorous results re- x ; by convention, . The finite time

iii 1

stricted to infinitesimal uncertainties are contrasted Lyapuno 8 exponents 15 are

@ with numerical demonstrations for finite uncertain-

° ±

n ¡ ¢ n £

¤ ¤ ¥ ¥

ª « ¬ ® ¯

$ ¨ © d

ties in Section 4. Part of the popularity of global ¦ 1

§ J J k J k Lyapunov exponents stems from the fact that their i x d log 2 x,k i x

A k

B value does not depend upon the metric or coordinate

º »

n ³

· ¸ ¹

system used; this is not the case for the exponents 1 ¶

J ´

7 k

¼ µ d log x .2 based upon a finite length of trajectory, yet in prac- k 2 i tice only the latter are available. We return to this

issue in Section 5. Finally, there is the question of ½ Properties of these exponents have been noted by

¾ ¿ À Á Â Ã Ä

D model error in nonlinear forecasting, either paramet- Lorenz 4 , Grassberger et al. 16 , Abarbanel 5 and

E F

Å Æ

G ric or structural 12 . Arguably, model error may be references thereof. By Oseledec's Theorem 1 , in

Ç n È É Ê

d Ë Ì

Í < k J

more responsible for poor predictions of real nonlin- the limit k the i x converge to a unique set

9 ) ear systems than `chaos.' Model imperfections are of values, the Lyapunov exponents Î i, which are the

C. Ziehmann et al. Ï Physics Letters A 00 ()2000 000±000 3

n c

a b , d , k same for almost all x with respect to an ergodic and 11approach , yet for the relatively small k measure. ) over which forecasts are typically made their proper-

ties are quite different and neither is constrained by

Ú Û

Ð Ñ Ò Ó n Ô

Õ Ö × Ø

$ 1

Ù e

lim d log , the value of .

Ý Þ ii Ü 2 i 1

k k

à á K i ß 1,2,...,m 3 f 3. Properties of localized Lyapunov exponents If the sum of the â i is negative, a volume

9 element in state space will shrink, on average, as it g

9 evolves along a trajectory, and most initial condi- General constraints on the relative magnitudes of

tions will evolve towards an attractor of dimension the largest and the smallest finite time and finite

ã ä

J )

less than K m 17 . At each point x on such an sample exponents are now derived from the defini-

å æ

! , attractor, the orientation lxdenotes the orientations above, and then illustrated below. By construc-

tion, maximum growth corresponds to , thus

n i j k l n m n o

tion corresponding to 1, that is the orientation h 1

J q r J J

p ?

k x k x for each x, and therefore this in-

s t

u v towards which almost every uncertainty in the 11 n

9 k x

equality also holds for the mean values w

{ | } ~

n 1 z

sufficiently distant past would have evolved, when y

é ê

J ,

, where denotes an arithmetic average

the trajectory reaches x. Similarly, define lxas 1 ¼

taken with respect to the natural measure , and U the orientation corresponding to ì for details, see N

í î ï ð ñ ò ó

, <

L a numerical approximation with sample size N.Ex- 17 . Numerically, lx10and lxU m 0 can be approx-

, amples from two chaotic systems are given in Fig. 1.

imated by evolving the singular vectors of

ô õ ö

÷ ø ù ù

ú û ý ü The Henon map 24 is

x j ,2 j , that is, the 2 j step propagator about

ù ù

* J

th þ

the j pre-image of x , forward j steps until the 2

0 x 1 ax y , y bx ,6

i 1 iii1 i

trajectory reaches x . Thus

0 6

ü ü ¥

¨ ©

§ where a 1.4 and b 0.3; the Jacobian is indepen-

ù ¤

J ¡ ¢ £ J J

j 2 j K

ü ü l x x , j x , i ¦ 1,m.4

i 0 ji j dent of y and has constant determinant equal to b.

ù j The Ikeda map 25

As j , we expect lxi L 0 to approach the orien-

> >

tations of lx for i 1 and i m, leading to the x 1 x cos t y sin t ,

i 0 i ¢ 1 ii

! )

1 8

¦ § ¨ © ª «

> ¥ >

£ ¤ definition of the finite sample Lyapuno exponents y x sin t y cos t ,7

i ¬ 1 ii

, -

$ % & ' ( ) * +

" # d

1 $

J ! J k 12x d log x,k lx1,5with

. n / 0 1 2 3

, 4 J ®

k > U

and x is similarly defined using lx. ¯

8 n ;

mmn t 0.4

° ±

5 22 ,

k , k x y 1

7 9 :

Both the 6 and the are often called `local

< =

² ³

Lyapunov exponents' 5,19±23 . To avoid the confu- , @ n and 0.9 provides a rather more complex Jaco-

k 7 2

> ?

sion of this polysemy, we will call the `finite- ´

A bian, still with constant determinant, equal to in

time' since they are completely defined by a finite

n D

B this case. Note the non-Gaussian shape of all the d k !

segment of trajectory and the E `finite-sample' F distributions in Fig. 1, particularly those for small k. since they sample the growth of an orientation µ

G Contrasting the shapes of the distributions for the ! defined by the global dynamics . Both involve the Henon and Ikeda systems suggests that such distribu-

same linear forward propagator, but each reflects the

H tions will be strongly system dependent. Also note

} d +

growth of a different orientation: the lxfor the ,

n K L M N n O P n Q

J 1 how the distributions sharpen with increasing k and

d T U 7 V

¶ n · ¸ ¹

, J

k k k k J S

11and the x for the 1.Ask both 1that for given k the distributions of º x and

n ¼ ½ ¾

» 1

! J ¿ k

1 x differ.

[

W W X Y 1 Z

If a specific x is not of interest, approximation of lx1 t at

\ ] Z x t along a numerical trajectory can be simply approximated by a very long integration of the system and tangent equations for an 2 À We assume throughout that there exists a unique natural

arbitrary initial uncertainty. The quality of this approximation may measure which is well approximated by the numerical iteration of

^ _ ` be unknown, however, see 18 and the discussion in Section 5. the system.

4 C. Ziehmann et al. Á Physics Letters A 00 ()2000 000±000

Ã Ä Å Æ Ç È É Ê Ë Ã Ì Í Î Ï Ð Ñ Ò Ó Ô Õ Ö

Â k k

Ø Ù Ú Û

Fig. 1. Distributions of × 11a , b and c , d in the Henon and the Ikeda systems for k 1 thick , k 4 thin , and k 64

Ü Ý Þ ß

à á

dashed , each with N 4096. The arrows at the top axis indicate â N . Larger bin widths have been used in the lower panels.

n å æ

ã ä

D ' ( , ) *

+ , - . / 0 1 2 3 4

k 3

The mean values ç N do not increase with matrices and , thus . The equal-

n ê ë ì í n î ï

1¼ 111

è é

? d d

2 k k )

ñ ò 5 6

increasing k. In fact ð 11for any k,as ity holds only if the first left singular vector u1 of

7 8

ó ô :

can be seen by considering the matrix , the product is aligned with the first right singular vector 19

õ ö

; < = > ? @

) J

) A B

÷ ø ù ú û D

of Jacobians xi ,i 1,2, . . . ,2 k. Divide into of i.e. u11C 1 , as in the uniform Baker's

E F G H I

, þ ÿ

D ý

two sub-products ü and , each of length k: map 17 and Baker's Apprentice Maps 7 . From

J K

Eq. 2 , the largest finite time Lyapunov exponent

¨ ©

J J ¤ ¥ J J

n n

£ ¦ §

x ... ¢ x x ... x

2 kk1 k 1

3 This follows immediately from the singular value decomposi-

O N

M T

) # $

Q R S T U V W

tion SVD of a square matrix, P , where the super-

! "

The first singular value 1 of reflects the T

d X

D maximum possible growth over the first k steps; the script denotes the transpose of a matrix. is a diagonal matrix

Y Z [ \ ] whose largest entry is and and are orthonormal rotation

D 1 & first singular value of matrix % must be less than or matrices. Noting that rotation matrices cannot enhance growth 9 equal to the product of the first singular values of the yields the desired result. A brief proof is given in the Appendix

C. Ziehmann et al. ^ Physics Letters A 00 ()2000 000±000 5

ø ù

D `

defined by _ must be less than or equal to the sequences of functions 26,27 . While this does not

a n b c d

ç ü ý þ

ú û

) , g h J

ÿ ¡ 8

6 2 k k

e f i

a erage of those defined by and : x guarantee a monotonic decrease of or in-

n q ç ¤ ¥ ¦

k l m n o p

j 11 1

r s t u v

¢ £

J } § ©

w 11 1k k

n ~

y z z { | ¨

log x log x crease of with increasing k, it does imply

2k 112 k 12 1k 1 m

k J x . Similarly, the smallest finite time expo-

11 ¼

- D J

2 k ç

nent defined by must satisfy x - k Ë k

n m 1

J J

1 k k 11and mmfor all k,10

U x x . As this is true for each

¡ ¢ £ n ¤ ¥ ¦ §

2 mk1 n m 1

, © J 2 k J 2 k

individual ¨ x alternatively x , the

1 imi ç

ª «

)

K k

mean of the largest smallest finite time Lyapunov proving that the mean of the 1 distribution is not

d ? Ë

9 exponent will not increase not decrease as k in- an unbiased estimate of the global Lyapunov expo-

! "

# $ <

creases by a factor of two: nent 1 for any finite k 5,28 .

n ° ± ² ³ n ´ µ ¶ · n ¸ ¹ º » n ¼ ½

® ¯

¾ ¿ À

Á Â Ã

2 k k , 2 k k The mean of the distribution of finite sample

Ä Å

and .8 )

11 mm '

Æ Ç È

(

É Ê Ë Ë

, exponents is equal to by definition, independent

* +

, -

1 ç

When and are of different lengths, k and ¡

1 k / Ì

of k. While in Fig. 1 each . N decreases with

4 5

1 ç

0 1 2 3

6 Ì

k , then Ë

Í Î

2 k /

n n n n

â ã ä

á increasing k as indicated by the arrows , the

Ú Ì Û Ü Ì Ý Þ Ì ß à Ì N

k k k k 1

Ï Ð Ñ Ò Ó Ô Õ Ö × Ø Ù 1 2 2 1 §

k12k 1 k21 k11 9 coincide, providing a consistency check as to whether

ç : ;

ç ç

8 9

å æ

Ë Ë

k k k /

é ê

1 è 2

ì =

and a similar relation is obtained for ë , i.e., N might be large enough so that N approxi-

ç î ï ð ç ñ ò ó ô ç @ A

í m 1

> ?

Ì ö Ì ÷ õ

k k Ë k ì

the k 1 k m are sub-additive super-additive mates the limiting value B 1 .

C D E F

Fig. 2. Contrasting finite sample Lyapunov exponents abscissa with the corresponding finite time Lyapunov exponents ordinate in the

G H I J K L M N

T U

S 2

Q R Ikeda system for k P 1 light grey , k 4 grey , and k 256 black iterations. Note, that in the Ikeda system Det J . 6 C. Ziehmann et al. V Physics Letters A 00 ()2000 000±000

If the Jacobian determinant is constant, then for likely regions of relatively high predictability: ï all

6 ð 6 W

Ë all x infinitesimal initial uncertainties will decrease in

ñ ò

õ Y

X these regions. Recently 9,29 , similar regions have

ó ô

m )

ç Z

^ _ ` a b c

[ \

k W been determined analytically in the Lorenz 30 sys-

õ ö ÷ ø ]

x log ,11õ

i 2 tem see also 20,31 ; we now present new results

g g h i f 1

for the Ikeda system and the Henon system. This

ç i j k l ç m n o

r s ù ú

W u v W

k k û

where det . Recalling that t x x result is then demonstrated numerically to hold in

ç x y z { ç | } ~

w 11

W W Ë k k

and ì x x , it follows that for both the several cases for finite uncertainties, but the exact

ç mm ç

k k

Henon system and the Ikeda system , results below are subject to the caveat of infinitesi-

ç ç ç

k k k

12log 2 1, and mal uncertainties, as are all general arguments re-

« ¬ ®

¨ © ª

§ garding the prediction of deterministic chaotic sys-

¡ ¢ £

¤ ¥

k 1 1 õ ¯ 1 max ¯ x 12log min x 1 ,12 tems.

defining a triangle which bounds the distribution of

ç ² ³ ´ µ ç ¶ · ¸ ¹

° ±

W W º k k

11x , » x , as illustrated in Fig. 2 for the 4.1. Infinitesimal uncertainties

Ì ¼

Ikeda map. With increasing k õ the distributions of

ç ¿ À Á Â ç Ã

Ì Ä Å

k k

ü ¾

points approach the line ½ . For k 256, the

Æ Ç

È É Ê Ë ç Ì Í Î Ï

11ç Naturally, exact results are most easily obtained

W Ñ Ò W

k k ©

Ì Ì ý largest observed value of Ð 11x x was

Ó for small k. Therefore we consider only k 1 and þ

0.03. Yet the widths of each of these distributions Ì k 2 analytically; numerical results are given for

¡ Ó

W ¤

¡ ¢

exceeds 0.3, indicating that this variation stems from 1 £

õ larger values. In a map x 0 implies that the ¦

1 ¥ ¡

the different initial conditions on the attractor, not W

§ ¨

õ largest singular value of the Jacobian x the initial orientation.

is less than one. For the Ikeda system with © in

the range 13 3 2 1, the one-step finite

time Lyapunov exponent 1 x passes through

Ô zero at two circles about the origin with radii r o,i

4. Regions of high predictability in chaotic maps e

& '

" # $ % 2

2 !

6c 1 6c 1 1 where c

( ) *

W 0

+ , -

. 1

1 . In this case / 1 x 0 for all points either

e 2

Lyapunov exponents are often said to reflect pre- 1

3 4 5 6 22

dictability, and a positive global Lyapunov exponent within the inner circle i.e. those with x y ri

7 8

9 5 : 3 22 is often said to destroy any hope of `long-term' or outside the outer circle i.e. x y r o . Fig. 3

predictability. But since they are defined via the shows points on the Ikeda attractor where the sign of

;

6 Ó

= >

1 < A linear propagator Lyapunov exponents need only x is indicated by the grey scale. For @ 0.9,

1 B

Õ D

quantify the growth of infinitesimal uncertainties in the radii are rioC 0.135 and r 7.404 thus this

õ Ë

the initial condition, this is a high price to pay for attractor lies well within r o, and the outer circle is

F G G

# Ë

invariance under a smooth change of coordinate. not visible in Fig. 3 . As H approaches one, the

Ö Ö × Ø Ù

Both the Ikeda system Eq. 6 and the Henon û radius of the inner circle goes to zero.

Ú Ú Û Ü system Eq. 7 are considered chaotic for the pa- In the Henon system there are no regions in which

rameters considered above, since in each case it is ' every uncertainty will shrink after one iteration, that

Ý ç Þ ß à

) Ó

6 M Ó

W N W

K L

k 1 J

â ã

believed that á 0; yet this does not imply x is, x 0 for all x. As shown below, this is not Q

111 e

Ó Ì ä

õ 2 P

0 for any finite k. Indeed it is clear from Fig. 2 the case for O , however. The smallest value of

å 1

W W ¡

T U

that there are many points on the attractor about 1.5 1 S x is found for points x on the y-axis. Here

æ 1

Ì ç

[

W \

Y Z 1 X

% of the Ikeda system for which the leading k 1 1 x 0, and thus all uncertainties shrink except

finite time exponent is negative, i.e. there are states those aligned with 1, which remain unchanged in

è è 5

about which e ery infinitesimal uncertainty will magnitude. Note that for points on the y-axis, 1 is

shrink regardless of its orientation. parallel to the 5 y-axis. We now prove that there exist

]

Ë W

_ ` 2 ^

We now proceed to locate the corresponding re- a finite region within which all points have a 1 x

Ó b

ÿ gions in state space with negative largest finite time 0; the region includes a portion of the first preim-

ç í î

ê Ë

k 5 ì Lyapunov exponents, ë 1 0, which we interpret as age of the y-axis.

C. Ziehmann et al. c Physics Letters A 00 ()2000 000±000 7

i j

f g 1 e

Fig. 3. Regions of decreasing uncertainty in the Ikeda system. Points on the attractor are colored grey if h x 0, black otherwise. Within

k 1

i p i

m n 1 l

the circular region near the origin, o 1 x 0 for all x.

q q r Ë

The preimage of the y-axis is the parabola, y ii, where ® are the roots of the characteristic

t ²

2 T± °

ax 1, and the two step propagator for points polynomial of ¯ . Thus

e v

s x y

¼ ½ »

2 º

µ ¶ ¹

,a 1is 2 ´ 2 2 2 4 ¸

2b 2 a 1 · b 0. 13

e ¿

e ~

Ì Å

{ |

z 2

s } 3 3 À ,a 2 1,k 2 x 0 x We can now test whether ¾ 1 for any ;

Ë Â

alternatively, we can solve for Á 1 to find

e Ã 2 a 1

01 2

Í Î Ï Ð

Å Æ Ä b 1 0.91

b 0

È É Ê Ë Ì

b 0 Ç

1 1.083 14

2 ab 0.84

Ò Ñ

b 0 s

, for a 1.4, b 0.3. Note that the parabola and the

Ó Ô

2 ab b Õ

x-axis intersect at the point 0,-1 . At this intersec-

Ö × Ø

where we have used the fact that in the Henon tion Õ x 0 and it follows from the equation

ç Û ç à

e Ü Ý Þ ß e á â

W Ë

3 k 2 k 2

system x is only a function of x. To locate above that Ù b 1 or, equivalently,

ä ç å ç è

æ ç e é

e 12

ê ë

¡ î ï ð

õ ì í

those x with k 2 0, we determine the singular that k 2 1 log b k 2 0. Showing that a ¡

1¡ 122 2

Ì ª

¡ ¥ ¦ §

£ ¤

s ¨ © ¢ 2 ¢ 2 values iiof ,a 1,k 2 , noting that « point is negative in the range given by Eq. 14 proves

8 C. Ziehmann et al. ñ Physics Letters A 00 ()2000 000±000

that all points on the parabola within these limits preimages of the q y-axis. The first three preimages of

ç ô

Õ q ö

k 2 õ ó

have ò 1 0. By continuity, there exists a finite the x-axis can be obtained analytically. The y-axis is

õ û

region in the neighborhood of the parabola for the preimage of the Õ x-axis, while the parabola noted

e e

2 2 above is the first preimage of the y-axis. The first

÷ ø

1 b 1 b

Õ û ú

ù x , preimage of the parabola is s

2 s ab 2 ab

s s

Õ q s

x , y 1 a , punov exponent is negative. A numerical estimate of bb

õ this region is shown in Fig. 5 a.

2 2

What about larger values of k? Trajectories pass- 1 b 1 b

! ÿ

2 þ

Å Å ý

ü . s

ing through this region of 0 are often found to s

£ ¤

ç 1

¡ ¢ Ì ¥ Å 2 ab 2 ab

have k 0 for k 2 as well; negative values of

¨ ©

§ 1

Ë 4 Õ

1 are clearly visible in Fig. 1 a . In the following, The first 4 preimages of the x-axis are shown to- ÿ

ç $ %

Ì Å

k © k

of points on the attractor with 1 0 for k 2 and while points on the attractor with " 1 0 are shown

Ì & õ

the q y-axis; namely that such points tend to lie near for the specific values k 2,3,4,5 in the four panels. (

Fig. 4. All panels show the Henon attractor, the x-axis, the y-axis and its last three preimages: the parabola ' solid line , its second preimage

. / 0 1 2 3

* + , -

) k

5 6 7

long dashed and its third preimage short dashed . The different panels show points on the attractor with 4 1 0 for k 2a,k 3b,

8 9 : ; = k < 4 c , and k 5d.

C. Ziehmann et al. > Physics Letters A 00 ()2000 000±000 9

ç A B

¼ k @

It is clear that the regions with ? 1 0 are related to Next, we investigate the behavior for even larger

R S

Ì 6

õ the intersections of the attractor and preimages of the k in the Henon system. Grassberger et al. 16 showed

ç V W

q k U

y-axis. We next show that this is even more evident that one should expect T 1 0 for arbitrarily large Ì

for initial conditions in the general vicinity of the k, assuming a behavior essentially like averages of û

Ë attractor. As in Fig. 4, Fig. 5 shows both the preim- random variables correlated only over short times.

X Ë

ages of the Õ x-axis and the attractor; in addition, test This general picture is correct although the details

ç E F

k Ë D

points for which C 1 0 are plotted as well, where are sometimes important, as we have argued else-

Y Z õ the test points were drawn at random from the region © where 15 . Here, we conjecture that, due to the

shown in this figure. Points with decreasing uncer- deterministic nature of the Henon system, this frac-

ç ]

Ì G

õ k \

tainties for k 5 are found in the vicinity of preim- tion of [ 1 decreases more quickly than the random

Ë £

ages of the q y-axis, i.e. they often have trajectories variable argument would suggest. As shown in Fig.

which include a near approach to the q y-axis. Typi- 6, the fraction of initial conditions on the attractor

ç ` a

^ _

cally, this occurs towards the end of that trajectory: © with k 0 is observed to decrease exponentially

K L

e 1

2 © J

points with I 1 0 are close to the first preimage of with k, as are the corresponding fraction when linear

q P

3 O N

the y-axis, points with M 1 0 are close to its first propagators of the map are combined at random. The

Ì b

Ë and second preimage and so forth. random case for k 1,2,4 and 8 are shown; for each,

c d

Fig. 5. All panels show the Henon attractor, the x-axis, the y-axis and its last three preimages: the parabola solid line , its second preimage

. j

f g h i

e k

l m

long dashed and its third preimage short dashed . The different panels show points in the vicinity of the attractor with k 1 0 for k 2

n o p q r s t u

w x a,k v 3b,k 4 c , and k 5d.

10 C. Ziehmann et al. y Physics Letters A 00 ()2000 000±000

. } ~

{ |

z k

Fig. 6. For the Henon system a 1.4,b 0.3 , the fraction of points with 1 0 as a function of k in the deterministic case squares .

Also shown are the results for random matrices, where the matrices are drawn from the distribution of the linear propagators of the Henon

map, that is , for j 1,2,4 and 8. The solid lines reflect the best fit to an exponential decay over the range 8 k 40. For large k, about 2 30 iterations of the map where considered in the deterministic case. Note, that determinism is a strong constraint reducing the

likelihood of finding negative finite time exponents. õ

the fraction decreases less quickly than in the deter- § condition with zero mean and the same standard

ministic case. Disregarding the determinism in the deviation . Taking a point on the attractor at ran-

series of Jacobians leads to frequencies of negative dom, 128 `observations' were generated and pre-

Ì ' k ©

1 which for the larger k exceed those of the dicted forward k steps; if the distance from truth at deterministic case by orders of magnitudes. final time was less than the initial perturbation ap- plied in more than 50% of the 128 cases, then the

4.2. Finite uncertainties ¡ original point on the attractor was considered to be

From the practical point of view of estimating of . For k 1 there is a region not shown of ¡

predictability, the knowledge of such points would high predictability centered on the circle derived

) Ì

be of limited utility for large k, since the shrinking ¢ above and shown in Fig. 3. Points of high pre-

§ £

9 ¢

¥ ¦ region around each point may be very small. Further, dictability have been observed for ¤ 2 and per-

1 ©

recall that all estimates of predictability based upon sist for ¨ 8 . While all are near the origin, many ª

Lyapunov exponents assume an infinitesimal initial fall outside the circle, but this is not surprising as the £

' error. Therefore, we next consider finite uncertainties definition of high predictability in this numerical ' explicitly, first exploring Gaussian distributed uncer- « experiment is much less restrictive than requiring a

õ tainties in the Ikeda map, and then uncertainties of negative finite time Lyapunov exponent, which guar- ¢

uniform magnitude in the Henon map. In each case antees 100% of the uncertainties to shrink if they ¡

Ë and discuss the relation between regions of enhanced k 4 are shown in Fig. 7. The results based upon

predictability and the regions where 1 0. infinitesimal uncertainties shown in the upper left ¼

In the Ikeda map, we consider normally dis- panel are seen to reflect the region of high pre-

® ¯ tributed uncertainties in each coordinate of the initial dictability very well up until 1 8, which is a fair C. Ziehmann et al. ° Physics Letters A 00 ()2000 000±000 11

infinitesimals; of course, structures smaller than ¼ § cannot be detected. It becomes harder to identify ½ regions with differing properties in predictability with increasing prediction time; sensitive dependence on initial condition will limit the prediction of pre- £ dictability as well as prediction itself.

X Thus far, we have only considered the value of an

« ¢ exponent s at a particular value of k K; alterna-

³ tively one might consider regions in which the expo- À nent is negative for all k K, with corresponding

Á uncertainties decreasing monotonically for the total

£ Â duration of ¿ K iterations. While such subsets are of

interest, they are not investigated here since the « existence of small positive values at intermediate k ¢ are consistent with regions of high predictability;

Â indeed the points omitted from the set of points for a

Ã particular k are those that are said to display `return

Ä Å of skill' in meteorology 32 . Although beyond the scope of this paper, it would be interesting to exam-

Â ine the spatial distribution and fraction of initial Æ condition in these subsets, both as a function of k ¢ and the magnitude of the initial uncertainty.

4.3. Coexistence of chaos and regions of high pre- Ç dictability Fig. 7. All panels show the Ikeda attractor; the dots represent the

attractor. Initial conditions on the attractor with enhanced pre- È

Another interesting aspect concerning the regions

Í Î

dictability are marked with a ` ' when more than 50% of 128 Ì

Ê Ë É k

initial uncertainties show decreased magnitudes at final time with 1 0 is the interplay between these regions ¢ k ² 4. The different panels reflecting different initial magnitudes and the location of unstable periodic orbits, which

eps are contrasted with the linear dynamics in the upper left panel. ¢ are believed to form the skeleton of the attractor in

Ï Ð

Ñ many chaotic systems 33 . Clearly, an unstable pe-

riod-k orbit cannot contain a point within a region

Ì Ô Õ

k Ó

fraction of the diameter of the attractor. We stress for which Ò 1 0, since that point would then be ³ that results of this kind will be extremely system stable. In short, each point on each unstable period k

specific. orbit must avoid all regions of the state space in

§ ¶ Ì Ø Ù

Ö ×

4 É k µ

For the Henon map the portrait of negative ´ 1 which 1 0: it is not easy to see how this comes

³ ³ ¢

revealed in Fig. 5 is contrasted with regions of high about if the orbits are dense on the attractor s and the

predictability for finite uncertainties in initial condi- ¢ area of the regions does not vanish. If the regions do ³

tion with a much sharper test than for the Ikeda map. Ú not vanish, then this observation suggests a new ¼

In this case each observation is placed at random on ¢ angle from which to view the extreme sensitivity of

¢ ¢ a circle of radius · about the true state; 1000 such the structure of the attractor to small changes in

`observations' were considered for each true state Ã parameter value.

¢ ¸

and only if the final time distance of ¸ e ery one of It is also interesting to consider the implications

Ì Ý

Û Ü Ñ Ã k them was less than ¹ was the point recorded as positive 1 might have on numerical results when `high predictability.' This is shown for 3 different ³ the true attractor is a stable attracting periodic orbit;

initial magnitudes in Fig. 8. For small magnitudes, no point in a periodic orbit need lie in a region for

Ì à á

º »

É Þ ß â k s 0.001, the cartography is quite similar to that of which 1 0. For a 1.37511006867,b 0.3, the 12 C. Ziehmann et al. ä Physics Letters A 00 ()2000 000±000

Fig. 8. All panels show the Henon attractor; the dots represent initial conditions with enhanced predictability for finite uncertainties. The

å æ ç è é ê ë ì

different panels belong to different initial magnitudes of initial uncertainty: a eps í 0 thus coinciding with Fig. 5c . In panels b , c , and

î ï

d , a dot at i x indicates that the distance between the image of the true state and each one of 1000 inexact `observations' decreased at i k ð 4. The observations were initially distributed on a circle of radius eps centered on x.

Henon system has a stable period 24 orbit. The X This observation suggests an interesting possible

24ó ò

majority of points on this orbit have ñ 0, but parallel between these simple two dimensional maps

ø ù

÷ 1

24 ¢ ö

for several, õ 0; the largest observed value is and the onset of turbulence in laminar fluid flows. It

ü ý

÷ 1

û ú 24

1 0.3, implying a magnification factor of more has long been known that shear flows can become

³ ÿ than a hundred within one cycle. When þ is non- turbulent at Reynolds numbers well below the criti-

normal, the magnitude of the leading singular value ¡ cal value as defined by the classical linear stability

¢ £ ¤ ³

Ñ may be quite large regardless of whether or not the theory based on eigenvalues 34 and references

¥ ¦ § ³

orbit is asymptotically stable. With a slight increase thereof; for a recent overview see 35 . Perturbations ³

in s a the system appears chaotic; the dynamics still in the directions of the singular vectors may grow

¨ ©

resemble those of the stable orbit, but the attractor ½ rapidly for a finite time, exciting nonlinear terms Ú now consists of 24 small, clearly separated chaotic ¢ and thereby dominating the onset of turbulence; the

`regions,' each visited in turn. It would be interesting long term behavior described by the eigenvalues

³ to examine the distribution of leading singular values becomes irrelevant. Non-normality might hold simi-

¢ about stable periodic points on the same orbit, as a lar consequences for the numerical iteration of non-

function of parameter; is there a positive lower bound linear systems. The smallest nonzero numerical per- ³ on the angle between these eigenvectors? turbation is finite, being defined by the numerical

C. Ziehmann et al. Physics Letters A 00 ()2000 000±000 13

] ^

9 ¡

grid; and it could be quite difficult to identify a certainties along lxhave the advantage to be free

stable period k orbit with 1 0 by numerically of transients, but if of finite magnitude they also may Â

iterating the map. The smallest nonzero numerical lie `off the attractor'. And even for finite uncertain- ³

perturbation might well grow sufficiently to bring ties `on the attractor', finite time growth is not bound

³ the nonlinear terms into play, resulting in sustained, by . These facts imply that the `super-Lyapunov' a

1 ` ¡

complicated dynamics up to the time-scales at which growth found by Nicolis et al. 8 is to be expected:

b ¢

the numerical orbit closed exactly upon itself, as all ¢ after time t an uncertainty may be magnified by

³ ¢

Ñ 1t 1t d

trajectories on digital computers will 36 . We do not more than the larger of 2 c and 2 , even if the Â

¡ claim that this is the case in the maps considered initial uncertainty is infinitesimal. Over what dura-

e f ³ ¢ above, but merely note the dynamics might appear tion can realistic i.e. operational uncertainties be

similar and thus stress the value of performing a ³ treated as infinitesimal? Or equivalently, what is the

bifurcation analysis in addition to numerical itera- « extent of the linear regime? This is an interesting ³

tion. ¢ and open question, even in numerical weather fore-

h i ¡ casting 38 .

Q Note that computing exponents for finite time is

5. Discussion and conclusions somewhat gratuitous in that s any increase will yield a

« ¸

positive ¸ effecti e exponent; a positive exponent im- X The results presented in this article hold implica- plies effectively exponential growth then only in the

³ tions for two questions of general interest: the ap- limit of infinite time. For finite time, a positive

¢ «

proximation of largest Lyapunov exponent 1 and exponent implies growth, but not exponential growth; Â

³ the estimation of likely forecast accuracy. Noting it only reflects the time dependence of the uncer-

¡ ¢ k s

that the finite time Lyapunov exponents i can tainty under the additional assumption that the

be computed accurately by standard methods, a lower growth was exponential. The width of the distribu-

k $

" #

bound on the error in assuming ! N is tions in Figs. 1 and 2 does not indicate uniform

' ( ) * + , - . / 0

÷ 11

% &

k $ 2 k k

2 3 4 5 6

given by 1 11NN, while 1pro- exponential growth on these time scales. An alterna-

³ 7

vides an upper bound on 8 1. When a good approxi- tive approach to quantify predictability by computing

9 ³

mation of the `Lyapunov vector' l1 is available, one the time required to reach an uncertainty threshold is

j k ¡

can also require for the difference between finite ¡ contrasted with the use of effective rates in 7,9 ,

= > ? @ A B C

: ; <

k k s

D E F

sample and finite time exponents 11 where examples with both large l 1 and large uncer-

G H

¢ I ¢

. Yet since both l11and are multiplicative tainty doubling times are discussed. As proven in

J K «

ergodic statistics 1 , uncertainty in numerical esti- Section 4, there are initial conditions for which no

L M

Ñ ½

mates of lx1 remains largely unquantified. Simi- perturbations grow for two paradigm attractors; it

larly, inasmuch as matrix multiplication does not Q would be interesting to investigate the relative loca-

o p

¡ k

N n

commute, attempts to estimate the uncertainty in 1 tion of regions within which m 1 0 and unstable

via the standard bootstrap approach must be treated period k orbits for large k, as a function of parame-

O P ³

Q with care for deterministic systems 15 . The goal of ter in a variety of low dimensional maps; the numer-

s t

R r

a sufficient condition for the convergence of 1 ics near stable period k points with q 1 0 may «

estimates remains allusive. Quantitative necessary ¢ also prove of interest. u

¡ conditions, along with explicit tests for convergence In this paper each system has been considered in S

Â in aleatoric systems stochastic systems with positive its natural state space, its original, physically rele-

T U V

¢ 7 1 are discussed elsewhere 28,37 . vant co-ordinate system. It should be noted, that

In terms of identifying the `worst forecast bust', Ú neither the typical measures of forecast error 4 nor

³ X

³ ¢

the 11are more important than the Y simply be- the finite time Lyapunov exponents nor the finite

¡ ¢ cause the Z 1 are larger. While it is sometimes argued sample Lyapunov exponents are invariant under co- ³ that the corresponding singular vectors may point

`off the attractor', the [ 1 remain relevant, as possible

\ uncertainties about the true initial state will also lie w `off the attractor', almost certainly. Infinitesimal un- 4 See v 39 for an atypical approach.

14 C. Ziehmann et al. x Physics Letters A 00 ()2000 000±000

ordinate changes or even changes in a Riemannian Ñ matrix, a diagonal matrix, and another rotation ma- z

metric . In this study, the forecast error is simply the ³ trix, hence the spectral norm of a matrix is identical

À À Á Â

Ä Å Æ Ç È É Ê Ë Ì Í

Euclidean distance between two points in state space, to its first singular value, Ã 1 . Given ,

{ {

T Q

} ~

specifically, | where is the identity then for any nonzero x we have

Î Ï Ð Ñ Ò Ó Ô Õ Ö Ö × Ø Ù Ù Ú Û Ü Ý

ò ó

Þ ß

à á â ² ã ä å æ ç ² è é ê ë ì ² í î ï ð ñ ²

matrix. The singular vector corresponding to the ² x x x x x .

ô õ ö ÷ ÷

largest finite time exponent maximizes this distance ô

ù ú û ü

Dividing by x , substitution yields ø 1

ý þ ÿ ¡ ¢ £ ¤ ¥

at prediction time k. There may be physically more ý

¡ casts. Then the identity might be replaced by another

Riemannian metric , for example the inverse of ³ the covariance matrix may serve as a natural choice References

Q when the different directions in state space display

different variances. Alternatively may be used to 1 V.I. Oseledec, Transactions of the Moscow Mathematical

¢ Society 19 1968 197.

account for different levels of noise on different state

2 J.-P. Eckmann, D. Ruelle, Rev. Mod. Phys. 57 1985 617. space variables, to target the variables whose predic-

³ 3 L. Arnold, Random Dynamical Systems, Springer, Berlin,

tion is of particular concern, or even to decide which 1998.

7 variables to observe in order to minimize the predic- 4 E.N. Lorenz, Tellus 17 1965 321.

³ tion error see 40 and references therein . The 5 H.D. I Abarbanel, R. Brown, M.B. Kennel, Int. J. Mod.

! "

¢ Phys. B 5 1991 1347. $

application determines the choice of metric. # & In conclusion, we again stress that the relevance 6 R. Doerner, B. Hubinger,È W. Martiensen,% S. Grossmann, S.

³ Thomae, Chaos Solitons and Fractals 1 1991 553.

( ) *

of all three types of exponent is restricted to cases ' 7 L. Smith, Phil. Trans. R Soc. Lond. A 1994 .

+ ,

Q where the uncertainties are sufficiently small that 8 C. Nicolis, S. Vannitsem, J.-F. Royer, Q.J.R. Meteorol. Soc.

- .

³ their growth is well approximated by the linear 121 1995 705.

/ 0

9 L. Smith, C. Ziehmann, K. Fraedrich, Q.J.R. Meteorol. Soc.

propagator 38 : is exact only for infinitesimal 1

\ 125 1999 2855.

4 5 6 uncertainties. Behavior of larger finite uncertainties 3

10 Z. Toth, E. Kalnay, Bull. Am. Meteorol. Soc. 74 1993 ½

requires the use of ensembles of initial conditions, 2317.

7 8

« each consistent with the observation; the relative 11 T.N. et al., Palmer, Phil. Trans. R. Soc. Lond., 1994.

9 :

performance of ensembles in the subspaces defined 12 L.A. Smith, Nonlinear Dynamics and Statistics chapter Dis-

by ¢ are contrasted with those defined in the sub- entangling Uncertainty and Error: On the Predictability of 1

9 Nonlinear Systems, Birkhauser,È Boston 2000.

< = > space defined by l1 for several chaotic flows in 9 . ;

X 13 S. Vannitsem, C. Nicolis, J. Atmos. Sci. 54 1997 347. @

The construction of ensembles for forecast evalua- ? 14 G. Strang, Linear algebra and its application, Hartcourt Brace ³

tion in imperfect models remains an important issue Jovanovich, San Diego, 1988.

B C D for all nonlinear systems. A 15 C. Ziehmann, L.A. Smith, J. Kurths, Physica D 126 1999

49. F

E 16 P. Grassberger, R. Badii, A. Politi, J. Statistical Physics 51

G H

1988 135.

I J Appendix A. 17 E. Ott, Chaos in dynamical systems, Cambridge University

Press, Cambridge, New York, Melbourne, 1993.

L M N K 18 G. Froyland, K. Judd, A. Mees, Phys. Rev. E 51 1995

2844. P

Here we establish that for a product of matrices O

Q 19 J.S. Nicolis, G. Mayer-Kress, G. Haubs, Z. Naturforsch. 38a,

¡ ¢ £ ¤ ¥

we have . §

111¦ 1983, p. 1157±1169.

R S T

The spectral norm of a matrix 14 is defined by Q

¨ ¨ © ª « ¬

® ¯

20 J.M. Nese, Physica D 35 1989 237.

U V W X

³ ´ ±

max x 0 ° xxwhere the maximum is taken 21 A.S. Pikovsky, Chaos 3 1993 225.

Y Z [ \ ²

over all nonzero vectors x. It bounds the amplifying 22 B. Eckhardt, D. Yao, Physica D 65 1993 100.

] ^ _ `

¶ ¶ · ¸ ¹ º

² ½ ¾ ¿ ² ¼ power of a matrix, i.e., » x x . The spectral 23 U. Feudel, J. Kurths, A.S. Pikovsky, Physica D 88 1995

176.

b c d norm of a rotation matrix is 1, while that of a a

£ 24 M. Henon, Commun. Math. Phys. 50 1976 69.

f g h

diagonal matrix corresponds to the maximum ele- e 25 K. Ikeda, Opt. Commun. 30 1979 257.

Ñ j

ment. The singular value decomposition decomposes i 26 D. Ruelle, Publications Mathematiques de l'Institut Hautes

k l ¢ any square matrix into the product of a rotation Etudes Scientifiques 50 1979 27.

C. Ziehmann et al. m Physics Letters A 00 ()2000 000±000 15

n o

27 A. Katok, B. Hasselblatt, Introduction to the Modern Theory 34 L. Boberg, U. Brosa, Z. Naturforsch. 43a, 1988.

of Dynamical Systems, Cambridge University Press, 1995. 35 L.N. Trefethen, A.E. Trefethen, S.C. Reddy, T.A. Driscoll, q

p 28 S. Ellner, R. Gallant, D. McGaffrey, D. Nychka, Phys. Lett. Science, 1993, p. 578±584.

r s

A 153 1991 357. 36 L.A. Smith, Lacunarity and Chaos in Nature, PhD thesis u t 29 C. Ziehmann-Schlumbohm, Vorhersagestudien in chaotis- Columbia University, New York, NY. 1987, See Appendix

chen Systemen und in der Praxis, PhD thesis Freie UniversitatÈ 1.

Berlin. Meteorologische Abhandlungen. Neue Folge Serie A. 37 L. Smith, C. Ziehmann, J. Kurths, 2000.

Band 8 Heft 3 1994. 38 I. Gilmour, Nonlinear model evaluation: -shadowing, proba-

v w x y

30 E.N. Lorenz, J. Atmos. Sci. 20 1963 130. bilistic forecasts, and weather forecasting, PhD thesis Oxford {

z 31 H. Mukougawa, M. Kimoto, S. Yoden, J. the Atmospheric University 1998.

| }

Sciences 48 1991 1231. 39 P. McSharry, L.A. Smith, Phys. Rev. Lett. 83 1999 4285.

32 J.L. Anderson, H.M. van den Dool 122 1994 507. 40 J.A. Hansen, L.A. Smith, J. Atmos. Sci., 1999, in press.

33 P. Cvitanovic, Phys. Rev. Letters 61 1988 2729.