Noise Versus Chaos in a Causal Fisher-Shannon Plane

Papers in Physics, vol. 7, art. 070006 (2015)

www.papersinphysics.org

Received: 20 November 2014, Accepted: 1 April 2015 Edited by: C. A. Condat, G. J. Sibona Licence: Creative Commons Attribution 3.0 DOI: http://dx.doi.org/10.4279/PIP.070006 ISSN 1852-4249

Noise versus chaos in a causal Fisher-Shannon plane

Osvaldo A. Rosso,1, 2∗ Felipe Olivares,3 Angelo Plastino4

We revisit the Fisher-Shannon representation plane H × F, evaluated using the Bandt and Pompe recipe to assign a probability distribution to a time series. Several stochastic dynamical (noises with f −k, k ≥ 0, power spectrum) and chaotic processes (27 chaotic maps) are analyzed so as to illustrate the approach. Our main achievement is uncovering the informational properties of the planar location.

I. Introduction quantifiers. Chaotic systems display “sensitivity to ini- Temporal sequences of measurements (or observa- tial conditions” and lead to non-periodic motion tions), that is, time-series (TS), are the basic ele- (chaotic time series). Long-term unpredictability ments for investigating natural phenomena. From arises despite the deterministic character of the tra- TS, one should judiciously extract information on jectories (two neighboring points in the phase space dynamical systems. Those TS arising from chaotic move away exponentially rapidly). Let x (t) and systems share with those generated by stochastic 1 x2(t) be two such points, located within a ball of processes several properties that make them very radius R at time t. Further, assume that these two similar: (1) a wide-band power spectrum (PS), (2) points cannot be resolved within the ball due to a delta-like autocorrelation function, (3) irregular poor instrumental resolution. At some later time t0, behavior of the measured signals, etc. Now, irregu- the distance between the points will typically grow lar and apparently unpredictable behavior is often 0 0 0 to |x1(t ) − x2(t )| ≈ |x1(t) − x2(t)| exp(λ |t − t|), observed in natural TS, which makes interesting the with λ > 0 for a chaotic dynamics, λ the largest establishment of whether the underlying dynami- Lyapunov exponent. When this distance at time t0 cal process is of either deterministic or stochastic exceeds R, the points become experimentally dis- character in order to i) model the associated phe- tinguishable. This implies that instability reveals nomenon and ii) determine which are the relevant some information about the phase space popula- ∗Email: [email protected] tion that was not available at earlier times [1]. One can then think of chaos as an information source. 1 Insitituto Tecnológicode Buenos Aires, Av. Eduardo The associated rate of generated information can be Madero 399, C1106ACD Ciudad Autónomade Buenos cast in precise fashion via the Kolmogorov-Sinai’s Aires, Argentina. entropy [2, 3]. 2 Instituto de F´ısica, Universidade Federal de Alagoas, Maceió,Alagoas, Brazil. One question often emerges: is the system chaotic (low-dimensional deterministic) or stochas- 3 Departamento de F´ısica,Facultad de Ciencias Exactas, Universidad Nacional de La Plata, La Plata, Argentina. tic? If one is able to show that the system is dominated by low-dimensional deterministic chaos, then 4 Instituto de F´ısica,IFLP-CCT, Universidad Nacional de La Plata, La Plata, Argentina. only few (nonlinear and collective) modes are re- quired to describe the pertinent dynamics [4]. If

070006-1 Papers in Physics, vol. 7, art. 070006 (2015) / O. A. Rosso et al. not, then the complex behavior could be modeled a measure of “global character” that is not too by a system dominated by a very large number of sensitive to strong changes in the distribution tak- excited modes which are in general better described ing place on a small-sized region. Such is not the by stochastic or statistical approaches. case with Fisher’s Information Measure (FIM) F Several methodologies for evaluation of Lya- [16,17], which constitutes a measure of the gradient punov exponents and Kolmogorov-Sinai entropies content of the distribution f(x), thus being quite for time-series’ analysis have been proposed (see sensitive even to tiny localized perturbations. It Ref. [5]), but their applicability involves taking reads into account constraints (stationarity, time series length, parameters values election for the method- Z 1 df(x)2 ology, etc.) which in general make the ensuing F[f] = dx ∆ f(x) dx results non-conclusive. Thus, one wishes for new Z dψ(x)2 tools able to distinguish chaos (determinism) from = 4 . (2) noise (stochastic) and this leads to our present in- ∆ dx terest in the computation of quantifiers based on Information Theory, for instance, “entropy”, “sta- FIM can be variously interpreted as a measure of tistical complexity”, “Fisher information”, etc. the ability to estimate a parameter, as the amount These quantifiers can be used to detect deter- of information that can be extracted from a set of minism in time series [6–11]. Different Informa- measurements, and also as a measure of the state tion Theory based measures (normalized Shannon of disorder of a system or phenomenon [17]. In the entropy, statistical complexity, Fisher information) previous definition of FIM (Eq. (2)), the division allow for a better distinction between deterministic by f(x) is not convenient if f(x) → 0 at certain chaotic and stochastic dynamics whenever “causal” x−values. We avoid this if we work with real prob- 2 information is incorporated via the Bandt and ability amplitudes f(x) = ψ (x) [16,17], which is a Pompe’s (BP) methodology [12]. For a review of simpler form (no divisors) and shows that F simply BP’s methodology and its applications to physics, measures the gradient content in ψ(x). The gradi- biomedical and econophysic signals, see [13]. ent operator significantly influences the contribu- Here we revisit, for the purposes previously de- tion of minute local f−variations to FIM’s value. tailed, the so-called causality Fisher–Shannon en- Accordingly, this quantifier is called a “local” one tropy plane, H × F [14], which allows to quantify [17]. the global versus local characteristic of the time Let now P = {pi; i = 1, ··· ,N} be a discrete series generated by the dynamical process under probability distribution, with N the number of pos- study. The two functionals H and F are evalu- sible states of the system under study. The con- ated using the Bandt and Pompe permutation ap- comitant problem of information-loss due to dis- proach. Several stochastic dynamics (noises with cretization has been thoroughly studied and, in par- f −k, k ≥ 0, power spectrum) and chaotic processes ticular, it entails the loss of FIM’s shift-invariance, (27 chaotic maps) are analyzed so as to illustrate which is of no importance for our present purposes the methodology. We will encounter that signifi- [10, 11]. In the discrete case, we define a “normal- cant information is provided by the planar location. ized” Shannon entropy as

( N ) S[P ] 1 X II. Shannon entropy and Fisher in- H[P ] = = − pi ln(pi) , (3) S S formation measure max max i=1

Given a continuous probability distribution func- where the denominator Smax = S[Pe] = ln N is R that attained by a uniform probability distribution tion (PDF) f(x) with x ∈ ∆ ⊂ R and ∆ f(x) dx = 1, its associated Shannon Entropy S [15] is Pe = {pi = 1/N, ∀i = 1, ··· ,N}. For the FIM, we take the expression in term of real probability Z amplitudes as starting point, then a discrete nor- S[f] = − f ln(f) dx , (1) malized FIM convenient for our present purposes is ∆

070006-2 Papers in Physics, vol. 7, art. 070006 (2015) / O. A. Rosso et al.

given by a) Noninvertible maps: (1) Logistic map; (2) Sine N−1 map; (3) Tent map; (4) Linear congruential X 1/2 1/2 2 F[P ] = F0 [(pi+1) − (pi) ] . (4) generator; (5) Cubic map; (6) Ricker’s popu- i=1 lation model; (7) Gauss map; (8) Cusp map; It has been extensively discussed that this dis- (9) Pinchers map; (10) Spence map; (11) Sine- cretization is the best behaved in a discrete envi- circle map; ronment [18]. Here, the normalization constant F0 b) Dissipative maps: (12) Hénonmap; (13) Lozi reads map; (14) Delayed logistic map; (15) Tinker-  ∗  1 if pi∗ = 1 for i = 1 or bell map; (16) Burgers’ map; (17) Holmes ∗ ∗ F0 = i = N and pi = 0 ∀i 6= i (5) cubic map; (18) Dissipative standard map;  1/2 otherwise. (19) Ikeda map; (20) Sinai map; (21) Discrete If our system lies in a very ordered state, which predator-prey map, occurs when almost all the pi – values are zeros, c) Conservative maps: (22) Chirikov standard we have a normalized Shannon entropy H ∼ 0 and map; (23) Hénon area-preserving quadratic a normalized Fisher’s Information Measure F ∼ 1. map; (24) Arnold’s cat map; (25) Gingerbread- On the other hand, when the system under study is man map; (26) Chaotic web map; (27) Lorenz represented by a very disordered state, that is when three-dimensional chaotic map; all the pi – values oscillate around the same value, we obtain H ∼ 1 while F ∼ 0. One can state that Even when the present list of chaotic maps is not the general FIM-behavior of the present discrete exhaustive, it could be taken as representative of version (Eq. (4)), is opposite to that of the Shan- common chaotic systems [19]. non entropy, except for periodic motions [10, 11]. The local sensitivity of FIM for discrete-PDFs is re- ii. Noises with f −k power spectrum flected in the fact that the specific “i−ordering” of The corresponding time series are generated as fol- the discrete values pi must be seriously taken into account in evaluating the sum in Eq. (4). This lows [20]: 1) Using the Mersenne twister genera- c point was extensively discussed by us in previous tor [21] through the Matlab RAND function we 0 works [10, 11]. The summands can be regarded as generate pseudo random numbers yi in the inter- a kind of “distance” between two contiguous prob- val (−0.5, 0.5) with an (a) almost flat power spectra abilities. Thus, a different ordering of the pertinent (PS), (b) uniform PDF, and (c) zero mean value. 1 summands would lead to a different FIM-value, 2) Then, the Fast Fourier Transform (FFT) yi is −k/2 hereby its local nature. In the present work, we first obtained and then multiplied by f , yield- 2 2 follow the lexicographic order described by Lehmer ing yi ; 3) Now, yi is symmetrized so as to obtain a [22] in the generation of Bandt-Pompe PDF. real function. The pertinent inverse FFT is now at our disposal, after discarding the small imaginary components produced by the numerical approxima- III. Description of our chaotic and tions. The resulting time series η(k) exhibits the stochastic systems desired power spectra and, by construction, is representative of non-Gaussian noises. Here we study both chaotic and stochastic systems, selected as illustrative examples of different classes of signals, namely, (a) 27 chaotic dynamic maps IV. Results and discussion [9,19] and (b) truly stochastic processes, noises with f −k power spectrum [9]. In all chaotic maps, we took (see section i.) the same initial conditions and the parameter-values detailed by Sprott. The corresponding initial val- i. Chaotic maps ues are given in the basin of attraction or near In the present work, we consider 27 chaotic maps the attractor for the dissipative systems, or in the described by J. C. Sprott in the appendix of his chaotic sea for the conservative systems [19]. For book [19]. These chaotic maps are grouped as each map’s TS, we discarded the first 105 iterations

070006-3 Papers in Physics, vol. 7, art. 070006 (2015) / O. A. Rosso et al.

1.0

9 10 can use any of these TS for evaluating the dynami- 23-x

D = 6 25

11 6 cal system’s invariants (like correlation dimension,

1,3

12 22-x

2 23-y

0.8 13 Lyapunov exponents, etc.), by appealing to a time

21-y

15-x

26 lag reconstruction [19]. Here we analyzed TS gener- 27

15-y

21-x

8 22-y ated by each one of chaotic maps’ coordinates when

0.6

19-y 16-y the corresponding map is bi- or multi-dimensional. 16-x Due to the fact that the BP-PDF is not a dynam-

19-x

0.4 k =3.5 ical invariant (neither are other quantiﬁers derived

k =3

Nonivertible maps 20-x by Information Theory), some variation could be 20-y

0.2 Dissipative maps 18-x expected in the quantiﬁers’ values computed with

k =2.5

Conservative maps 18-y Fisher Information Measure this PDF, whenever one or other of the TS gener- k =2

k - noise 4

k =0

0.0 ated by these multidimensional coordinate systems.

0.4 0.6 0.0 0.2 0.8 1.0 From Fig. 1, we clearly see that the chaotic maps Normalized Shannon Entropy under study are localized mainly at entropic region lying between 0.35 and 0.9, and reach FIM Figure 1: Localization in the causality Fisher- values from 0.4 to almost 1. A second group of Shannon plane of the 27 chaotic maps considered chaotic maps, constituted by: the Gauss map (7), in the present work. The Bandt-Pompe PDF was linear congruential generator (4), dissipative stan- evaluated following the lexicographic order [22] and dard map (18), Sinai map (20) and Arnold’s cat considering D = 6 (pattern-length), τ = 1 (time map, is localized near the right-lower corner of the 7 lag) and time series length N = 10 data (initial H × F plane, that is in the range 0.95 ≤ H ≤ 1.0 conditions given by Sprott [19]). The inside num- and 0 ≤ F ≤ 0.3. Their localization could be bers represent the corresponding chaotic map enu- understood if one takes into account that when a merated at the beginning of section i.. The let- 2D-graphical representation of them (i.e., a graph ters “X” and “Y” represent the time series coordi- Xn × Xn+1 for one dimensional maps, or Xn × Yn nates maps for which their planar representation is for two dimensional maps) it tends to fulfill the clearly distinguishable. The open circle-dash line space, resembling the behavior of stochastic dy- represents the planar localization (average values namics. However, they are chaotic and present a over ten realizations with different seeds) for the clear structure when the dynamics are represented −k stochastic process: noises with f power spec- in higher dimensional plane. trum. Noises with f −k power spectrum (with 0 ≤ k ≤ 5) exhibit a wide range of entropic values (0.1 ≤ H ≤ 1) and FIM values lying between 0 ≤ F ≤ 0.5. A smooth transition in the pla- and, after that, N = 107 iterations-data were gen- nar location is observed in the passage from un- erated. correlated noise (k = 0 with H ∼ 1 and F ∼ 0) Stochastic dynamics represented by time series to correlated one (k > 0). The correlation de- −k of noises with f power spectrum (0 ≤ k ≤ 3.5 gree grows as the k value increases. From Fig. 1 and ∆k = 0.25) were considered. For each value of we gather that, for stochastic time series with in- k, ten series with different seeds and total length creasing correlation-degree, the associated entropic 6 N = 10 data were generated (see section ii.), and values H decrease, while Fisher’s values F increase. their corresponding average values were reported Taking into account that other stochastic processes, for uncorrelated (k = 0) and correlated (k > 0) like fBm and fGn (not shown), present a quite close noises. behavior to the k-noise analyzed here (see Ref. The BP-PDF was evaluated for each TS of N [11]), we can think that the open circle-dash line data, stochastic and chaotic, following the lexi- represents a division of the plane; above this line cographic pattern-order proposed by Lehmer [22], all the chaotic maps are localized. It is also inter- with pattern-lengths D = 6 and time lag τ = 1. esting to note that, qualitatively, the same results Their corresponding localization in the causality are obtained when the evaluations where made with Fisher-Shannon plane are shown in Fig. 1. One pattern length D = 4 and D = 5, as well as, differ-

070006-4 Papers in Physics, vol. 7, art. 070006 (2015) / O. A. Rosso et al. ent Fisher information measure discretization are and missing ordinal patters, Physica A 391, used. 42 (2012). Summing up, we have presented an extensive series of numerical simulations/computations and [8] O A Rosso, L C Carpi, P M Saco, M Gómez have contrasted the characterizations of determin- Ravetti, H A Larrondo, A Plastino, The Amigó istic chaotic and noisy-stochastic dynamics, as rep- paradigm of forbidden/missing patterns: A de- resented by time series of finite length. Surprisingly tailed analysis, Eur. Phys. J. B 85, 419 (2012). enough, one just has to look at the different planar [9] O A Rosso, F Olivares, L Zunino, L De Micco, locations of our two dynamical regimes. The pla- A L L Aquino, A Plastino, H A Larrondo, nar location is able to tell us whether we deal with Characterization of chaotic maps using the chaotic or stochastic time series. permutation Bandt–Pompe probability distribution, Eur. Phys. J. B 86, 116 (2013). [10] F Olivares, A Plastino, O A Rosso, Ambi- Acknowledgements - O. A. Rosso and A. Plas- guities in Bandt–Pompe’s methodology for lo- tino were supported by Consejo Nacional de Inves- cal entropic quantifiers, Physica A, 391, 2518 tigaciones Cient´ıficasy Técnicas(CONICET), Ar- (2012). gentina. O. A. Rosso acknowledges support as a [11] F Olivares, A Plastino, O A Rosso, Contrast- FAPEAL fellow, Brazil. F. Olivares is supported ing chaos with noise via local versus global in- by Departamento de F´ısica,Facultad de Ciencias formation quantifiers, Phys. Lett A 376, 1577 Exactas, Universidad Nacional de La Plata, Ar- (2012). gentina. [12] C Bandt, B Pompe, Permutation entropy: A natural complexity measure for time series, [1] H D I Abarbanel, Analysis of observed chaotic Phys. Rev. Lett. 88, 174102 (2002). data, Springer-Verlag, New York (1996). [13] M Zanin, L Zunino, O A Rosso, D Papo, Per- [2] A N Kolmogorov, A new metric invariant mutation entropy and its main biomedical and for transitive dynamical systems and automor- econophysics applications: A review, Entropy phisms in lebesgue sapces, Dokl. Akad. Nauk. 14, 1553 (2012). (USSR) 119, 861 (1959). [14] C Vignat, J F Bercher, Analysis of signals in [3] Y G Sinai, On the concept of entropy for a the Fisher-Shannon information plane, Phys. dynamical system, Dokl. Akad. Nauk. (USSR) Lett. A 312, 27 (2003). 124, 768 (1959). [15] C Shannon, W Weaver, The mathematical the- [4] A R Osborne, A Provenzale, Finite correlation ory of communication, University of Illinois dimension for stochastic systems with power- Press, Champaign, USA (1949). law spectra, Physica D 35, 357 (1989). [16] R A Fisher, On the mathematical foundations [5] H Kantz, T Scheiber, Nonlinear time series of theoretical statistics, Philos. Trans. R. Soc. analysis, Cambridge University Press, Cam- Lond. Ser. A 222, 309 (1922). bridge, UK (2002). [17] B R Frieden, Science from Fisher information: [6] O A Rosso, H A Larrondo, M T Mart´ın, A A Unification, Cambridge University Press, Plastino, M A Fuentes, Distinguishing noise Cambridge, UK (2004). from chaos, Phys. Rev. Lett. 99, 154102 (2007). [18] P Sánchez-Moreno, R J Yán˜ez,J S Dehesa, Discrete densities and Fisher information, In: [7] O A Rosso, L C Carpi, P M Saco, M Gómez Proceedings of the 14th International Con- Ravetti, A Plastino, H A Larrondo, Causality ference on Difference Equations and Appli- and the entropy-complexity plane: Robustness cations, Eds. M. Bohner, et al., Pag. 291,

070006-5 Papers in Physics, vol. 7, art. 070006 (2015) / O. A. Rosso et al.

U˘gurBah¸ce¸sehir University Publishing Com- [21] M Matsumoto, T Nishimura, Mersenne pany, Istanbul, Turkey (2009). twister: A 623-dimensionally uniform pseudo- random number gererator, ACM T. Model. [19] J C Sprott, Chaos and time series analy- Comput. S. 8, 3 (1998). sis, Oxford University Press, New York, USA (2003). [22] http://www.keithschwarz.com/interesting/ code/factoradic-permutation/Factoradic [20] H A Larrondo, Matab program: noisefk.m Permutation.hh.html (http://www.mathworks.com/matlabcentral/ fileexchange/35381) (2012).

070006-6