Lessons from the Long Flow Records of the Nile: Determinism Vs

20 Years of Nonlinear Dynamics in Geosciences Rhodes, Greece 11-16 June 2006

Lessons from the long flow records of the Nile: Determinism vs. indeterminism and maximum entropy

Demetris Koutsoyiannis Department of Water Resources, School of Civil Engineering, National Technical University of Athens, Greece Aris Georgakakos Georgia Water Resources Institute, School of Civil and Environmental Engineering, Georgia Institute of Technology, USA Data set 1: 160 (30) 140 Contemporary x x, Annual 120 record of 100 monthly flows at 80 60 Annual Aswan 40 30-year average Data period 1870- 20 40 (30) 2001 (131 years) 35 September

3 x x, Units: km 30 25 20 15 10 5 6 General observations: (30) x , , 5 February Aperiodic fluctuations x on large time scales 4 Behaviours in 3 different months may 2 be like or unlike 1 0 1870 1890 1910 1930 1950 1970 1990 2010 Year D. Koutsoyiannis and A. Georgakakos, Lessons from the long flow records of the Nile 2 Data sets 2 and 3: records of annual maximum and minimum water levels at the Roda Nilometer Data period 640-1452 (813 years): the longest instrumental data set available worldwide – high importance in understanding and modelling hydroclimatic behaviours

10 (30) x , , 8 Units: m Minimum x

0 15 (30) x , , 13 x Maximum 11 9 7 Annual 5 30-year average General observation: 3 Aperiodic fluctuations 600 700 800 900 1000 1100 1200 1300 1400 1500 on large time scales Year AD

D. Koutsoyiannis and A. Georgakakos, Lessons from the long flow records of the Nile 3 Part 1 Seeking determinism 40 Fifty years of monthly flow Analysis of the x i 30 monthly flows 20

0 0 100 200 300 400 500 600 i 50 γ j 600 terms of 30 autocovariance γj vs. 10 lag j -10

-30 0 100 200 300 400 500 600 j 100000 Power spectrum From data 1/12 10000 2/12 From autocovariance 3/12 1000 4/12 General observations: 100 5/12 6/12 10 1. All three plots s (ω k ) highlight the annual 1 cycle 0.1 2. No apparent over- 0.01 0.001 0.01 0.1 1 annual cycle ω k D. Koutsoyiannis and A. Georgakakos, Lessons from the long flow records of the Nile 5 Fifty4 years of monthly flow cyclically standardized Analysis of the x i ' 3 2 standardized 1 0 monthly flows -1 -2 -3 0 100 200 300 400 500 600 i 1 General observations: γ j 0.8 600 terms of 1. The annual cycle 0.6 autocovariance γ vs. lag j cannot be ‘removed’ 0.4 j by standardizing the 0.2 0 monthly time series -0.2 2. ‘Linear’ logics 0 100 200 300 400 500 600 j implying additivity of 1000 Power spectrum the type 1/12 100 ‘periodic + aperiodic’ 10 2/12 or ‘signal + noise’ s (ω k ) may be inappropriate 1 0.1 for natural processes From data 0.01 3. No apparent over- From autocovariance annual cycle 0.001 0.0001

0.001 0.01 0.1 ω k 1 D. Koutsoyiannis and A. Georgakakos, Lessons from the long flow records of the Nile 6 Seeking over-annual cycles

Several studies have claimed 4 .2 2 .2 7 1 2 1 9 6 4 to have detected several 2 5 6 over-annual periods from Study 1 the Nilometer time series Study 2

Here are two recent 5 1 3 2 1 7 6

examples 2 5 6 1 10 100 1000 Period (years)

Authors’ opinion: Such claims may be suspicious for several reasons 1. The plethora of periods, which are numbers asymmetric to each other, may rather indicate a stochastic behaviour (all frequencies are significant) 2. Some studies test the significance of detected periods against white noise; apparently, a white noise hypothesis is totally inconsistent with the Nile behaviour 3. Some studies may have undervalued the estimation uncertainty in stochastic processes with high autocorrelation

D. Koutsoyiannis and A. Georgakakos, Lessons from the long flow records of the Nile 7

10000 Seeking over- annual cycles: 1000 s (ω k ) Aswan and 100 Nilometer annual 10 Entire record First half Second half Aswan, annual series 1 0.01 0.1 1 ω A simple methodology: 100 k A ‘real’ peak in the power spectrum 10 (manifesting s (ω k ) 1 determinism rather than 0.1 a random effect) will appear also at the same 0.01 Nilometer, minimum frequency if we split the 0.001 sample in two halves or 100 in three tertiaries 10

s (ω k ) 1

Conclusion: 0.1 No over-annual cycle 0.01 appears in any of the Nilometer, maximum three time series 0.001 0.001 0.01 0.1 1 ω k D. Koutsoyiannis and A. Georgakakos, Lessons from the long flow records of the Nile 8 Algorithmic details in seeking over-annual cycles

1. We split the sample into two halves; if the sample size n is not even we omit one value so that the half-sample size ν = n/2 is integral 2. We estimate the periodograms of the full time series and its two halves at frequencies ωk = k/ν for integral k such that 1/ν ≤ ωk ≤ 1/2 3. For each ωk we calculate a likelihood measure of a ‘real’ peak η(ωk) as the number of peaks in the three periodograms divided by 3 (a real peak should have η(ωk) = 1) 4. We repeat (analogously) steps 1-3 splitting the sample into three tertiaries (here ν = n/3)

5. We plot η(ωk) vs. ωk for the two cases (halves, tertiaries) and locate frequencies ω in whose neighborhood η(ω) = 1 in both cases 6. For these frequencies, we inspect in the periodogram of the full series whether or not the peak is higher than neighbouring peaks; if yes we can say that the identified ω manifest determinism rather than a random effect

1 η (ω k ) 0.75

Power spectrum peaks 0.5 plot, Nilometer, maximum 0.25

0 0 0.1 0.2 0.3 0.4 0.5 ω k D. Koutsoyiannis and A. Georgakakos, Lessons from the long flow records of the Nile 9 Seeking a chaotic attractor

Natural monthly flows of Standardized monthly Nile, Time lag 1 flows of Nile, Time lag 6 8 )

) 8 1 2 3 4 m m 1 2 3 4 , , 7 ε ε 7 5 6 7 8 ( ( 5 6 7 8 2 2 d d 6 6

5 5

4 4

3 3

2 2

1 1

0 0 0.0001 0.001 0.01 0.1 1 0.0001 0.001 0.01 0.1 1 ε ε Notation: ε = scale length; m = embedding dimensions (from 1 to 8 as indicated in the legends); d2(ε, m) = local slope of correlation sums

Conclusion: No low-dimensional determinism

D. Koutsoyiannis and A. Georgakakos, Lessons from the long flow records of the Nile 10 Part 2 Stochastic description Multi-scale setting of stochastic analysis

A process at continuous time t X(t)

i k (k) 1 ⌠ The averaged process at a scale k X := ⌡ X(t) dt i k (i – 1) k

Properties of the process at an arbitrary observation scale k = 1 (e.g. annual) Standard deviation σ ≡ σ(1)

(1) Autocorrelation function (for lag j) ρj ≡ ρj Power spectrum (for frequency ω) s(ω) ≡ s(1)(ω) Can be derived analytically from Properties of the process at any other scale those at scale k = 1 – depend on ρj Specific properties at any scale of a simple scaling stochastic process (SSS - fractional Gaussian noise) with Hurst exponent H Standard deviation σ(k) = kH – 1 σ (0.5 < H <1)

(k) 2H – 2 Autocorrelation function (for lag j) ρj = ρj ≈ H (2 H – 1) |j| Power spectrum (for frequency ω) s(k)(ω) ≈ 4 (1 – H) σ2 k 2H – 2 (2 ω)1 – 2 H

D. Koutsoyiannis and A. Georgakakos, Lessons from the long flow records of the Nile 12 Manifestation of aperiodic fluctuations into the statistics of the time series

3 3 Month (km ) σ (km ) Cs Ck τ3 τ4 Η ρFGN1 ρ1 ρ2 ρ12 Aug 19.03 4.38 0.06 0.28 0.01 0.11 0.74 0.40 0.58 0.06 0.17 Sep 21.55 5.07 0.21 0.18 0.03 0.11 0.81 0.53 0.74 0.24 0.25 Oct 14.58 4.27 0.17 0.24 0.01 0.11 0.91 0.77 0.85 0.61 0.50 Nov 8.16 2.31 0.64 0.14 0.14 0.09 0.85 0.62 0.86 0.72 0.33 Dec 5.57 1.40 1.21 1.91 0.23 0.20 0.90 0.74 0.90 0.75 0.46 Jan 4.33 1.03 0.74 1.05 0.14 0.19 0.87 0.68 0.90 0.75 0.44 Feb 3.17 0.91 0.64 0.53 0.11 0.13 0.82 0.56 0.92 0.76 0.40 Mar 2.76 0.87 0.79 1.25 0.10 0.14 0.83 0.59 0.90 0.75 0.41 Apr 2.52 1.05 0.43 0.87 0.12 0.01 0.93 0.82 0.80 0.62 0.73 May 2.33 1.01 0.46 1.11 0.14 0.02 0.94 0.83 0.94 0.73 0.79 Jun 2.24 0.82 0.82 0.69 0.15 0.07 0.88 0.69 0.78 0.69 0.46 Jul 5.26 1.76 0.74 0.58 0.13 0.13 0.83 0.59 0.57 0.35 0.39 Average 0.53 0.27 0.10 0.10 0.86 0.65 0.81 0.59 0.44 Annual 91.51 18.38 0.48 0.23 0.11 0.13 0.83 0.58 0.32 0.32

Series (m) σ (m) Cs Ck τ3 τ4 Η ρFGN1 ρFGN2 ρ1 ρ2 Minima 2.70 0.96 0.70 3.20 0.05 0.16 0.84 0.60 0.46 0.46 0.38 Maxima 9.27 0.75 0.13 6.24 0.01 0.22 0.84 0.59 0.45 0.43 0.36

High autocorrelation (ρ) – High Hurst coefficient (H > 0.5)

D. Koutsoyiannis and A. Georgakakos, Lessons from the long flow records of the Nile 13

1.4 0.8 ) ) k k ( Empirical Aswan, annual ( Aswan, September σ σ Log Log statistical Log 1.2 0.6 analysis 1: Standard 1 Empirical, classic 0.4 Empirical, SSS deviation White noise Modelled, SSS Modelled, ME vs. scale 0.8 0.2 0 0.4 0.8 1.2 0 0.4 0.8 1.2 Log k Log k 0.2 )

) 0 k ( k ( σ σ 0 Log Log Observation: Log -0.2 In all cases an SSS -0.2 model seems -0.4 satisfactory -0.4 (White noise is -0.6 -0.6 far from Nilometer, minimum Nilometer, maximum satisfactory) -0.8 -0.8 0 0.5 1 1.5 2 0 0.5 1 1.5 2 Log k Log k D. Koutsoyiannis and A. Georgakakos, Lessons from the long flow records of the Nile 14

(1) 0 (1) j j

Empirical ρ Aswan, annual ρ Aswan, September Log Log statistical -0.4 Log -0.4 analysis 2: Auto- -0.8 -0.8 correlation Empirical, classic -1.2 Empirical, SSS -1.2 vs. lag at Modelled, SSS Modelled, ME scale 1 -1.6 -1.6 0 0.5 1 1.5 0 0.5 1 1.5 j Log Log j 0 0 (1) (1) j j ρ ρ Log Log

Log Log -0.4 -0.4

-0.8 -0.8 Observation: In all cases an SSS -1.2 -1.2 model seems Nilometer, minimum Nilometer, maximum satisfactory -1.6 -1.6 0 0.5 1 1.5 2 0 0.5 1 1.5 2 Log j Log j

D. Koutsoyiannis and A. Georgakakos, Lessons from the long flow records of the Nile 15

) )

k 0 (

k 0 ( 1 Empirical 1 ρ Aswan, annual statistical ρ Log Log -0.2

Log Log -0.2 analysis 3: Auto- -0.4 -0.4 correlation Empirical, classic -0.6 Empirical, SSS -0.6 vs. scale at Modelled, SSS Modelled, ME Aswan, September lag 1 -0.8 -0.8 0 0.4 0.8 1.2 0 0.4 0.8 1.2 Log k Log k ) )

0 k k

( 0 ( 1 1 ρ ρ Log Log

Log Log -0.2 -0.2

-0.4 -0.4 Observation: In all cases even -0.6 -0.6 an SSS model seems not very Nilometer, minimum Nilometer, maximum satisfactory -0.8 -0.8 0 0.5 1 1.5 2 0 0.5 1 1.5 2 Log k Log k D. Koutsoyiannis and A. Georgakakos, Lessons from the long flow records of the Nile 16 Some type-“why?” questions

Why the temperature of this room tends to equal Because this that of the environment (in the absence of behaviour heating/cooling systems) maximizes Why the probability for each outcome of a die is 1/6? entropy Why the normal (non scaling) distribution is so common for processes with relatively low variation? (εντροπία Why variables with high variation tend to have entropy asymmetric inverse-J-shaped (rather than bell- entropie shaped) distributions? Entropie Why variables with high variation tend to have a entropia scaling behaviour in state (i.e. non-normal distribution)? entropía Why stochastic dependence of natural quantities in entropi consecutive time steps appears so commonly to be entrópia linear? entroopia Why the Hurst phenomenon (scaling behaviour in entropija time) is so common in geophysical, biological, socioeconomical and technological processes? энтропия ентропія)

D. Koutsoyiannis and A. Georgakakos, Lessons from the long flow records of the Nile 17 What is entropy?

For a discrete random variable X taking values xj with probability mass function pj ≡ p(xj), the Boltzmann-Gibbs-Shannon (or extensive) entropy is defined as w w φ := Ε[–ln p(Χ)] = –∑ pj ln pj, where ∑ pj = 1 j = 1 j = 1 For a continuous random variable X with probability density function f(x), the entropy is defined as ∞ ∞ φ := Ε[–ln f(Χ)] = –⌡⌠ f(x) ln f(x) dx, where ⌡⌠ f(x) dx = 1 –∞ –∞ Several generalizations of the entropy definition have been proposed; the most promising seems to be that of Tsallis (1988) In both cases the entropy φ is a measure of uncertainty about X and equals the information gained when X is observed Sometimes entropy is regarded as a measure of order or disorder and complexity

D. Koutsoyiannis and A. Georgakakos, Lessons from the long flow records of the Nile 18 Entropic quantities of a stochastic process

The order 1 entropy (or simply entropy or unconditional entropy) refers to the marginal distribution of the∞ process Xi :

φ := Ε[–ln f(Χi)] == –⌡⌠ f(x) ln f(x) dx, The order n entropy refers to the joint–∞ distribution of the vector of variables Xn = (X1, …, Xn) taking values xn = (x1, …, xn):

φn := Ε[–ln f(Χn)] = –⌡⌠ f(xn) ln f(xn) dxn The order m conditional entropyD refers to the distribution of a future variable (for one time step ahead) conditional on known m past and present variables (Papoulis, 1991):

φc,m := Ε[–ln f(Χ1|X0, …, X–m + 1)] = φm – φm - 1 The conditional entropy refers to the case where the entire past is observed:

φc := limm → ∞ φc,m The information gain when present and past are observed is:

ψ := φ – φc The theory is for discrete time stationary processes

D. Koutsoyiannis and A. Georgakakos, Lessons from the long flow records of the Nile 19 What is the principle of maximum entropy (ME)?

In a probabilistic context, the principle of ME was introduced by Jaynes (1957) as a generalization of the “principle of insufficient reason” attributed to Bernoulli (1713) or to Laplace (1829) In a probabilistic context, the ME principle is used to infer unknown probabilities from known information In a physical context, the ME principle determines thermodynamical states The principle postulates that the entropy of a random variable should be at maximum, under some conditions, formulated as constraints, which incorporate the information that is given about this variable Typical constraints used in a probabilistic or physical context are: Mass Mean/Momentum ∞ ∞ Non-negativity ⌠ ⌠ ⌡ f(x) dx = 1, Ε[Χ] = ⌡ x f(x) dx = μ x ≥ 0 –∞ –∞ Variance/Energy Dependence/Stress ∞ ∞ Ε[Χ 2] = ⌡⌠ x2 f(x) dx = σ2 + μ2, Ε[Χi Xi + 1] = ⌡⌠ xi xi + 1 f(xi, xi + 1) dxi dxi + 1 = ρ σ 2 + μ2 –∞ –∞ D. Koutsoyiannis and A. Georgakakos, Lessons from the long flow records of the Nile 20 Application of the ME principle at the basic time scale

Maximization of either φn (for any n) or φc with the mass/mean/ variance constraints results in Gaussian white noise, with maximized entropy

φ = φc = ln(σ 2πe) , φn = n φ and information gain ψ = 0. This result remains valid even with the non-negativity constraint if variation is low (σ/μ << 1) Maximization of either φn (for any n) or φc with the additional constraint of dependence with ρ > 0 (for lag one) results in a Gaussian Markov process with maximized entropy

2 φ = ln(σ 2πe), φc = ln[σ 2 π e (1 – ρ )], φn = φ + (n – 1) φc and information gain ψ = –ln 1 – ρ2 The question is how we should maximize entropy in a continuous time process: At the observation scale? (=> a Markov process – but not reasonable) At multiple time scales simultaneously? (see Koutsoyiannis, 2005 => SSS) At scale tending to zero? (a ‘local’ contemplation) At scale tending to infinity? (a ‘global’ contemplation)

D. Koutsoyiannis and A. Georgakakos, Lessons from the long flow records of the Nile 21 Entropy extremizing solutions - Comparison

4 4 Grey noise Unconditional Conditional Markov/ARMA(1,1) Entropy 3 Entropy 3 Simple scaling Asymptotic scaling White noise 2 2

1 1

0 0

-1 -1 0.01 0.1 1 10 100 0.01 0.1 1 10 100 Scale Scale

• All five solutions are constrained by σ = 1 at the annual scale (k = 1); • All solutions but white noise are also constrained by ρ = 0.449, as in the Nile flow record at the annual scale (k = 1); white noise is plotted just for comparison • To derive the simple scale and asymptotic scaling solutions two additional inequality constraints (restrictions) were used 1. ψ(0) ≥ ψ(k) for any k > 0 (meaning that predictability at any time scale k is lower than that instantaneously after the measurement) 2. ψ(∞) < ∞ (prohibiting an illimitable predictability at very large scales)

D. Koutsoyiannis and A. Georgakakos, Lessons from the long flow records of the Nile 22 Entropy extremizing solutions - Comparison

4 4 Grey noise Unconditional Conditional Markov/ARMA(1,1) Entropy 3 Entropy 3 Simple scaling Asymptotic scaling White noise 2 2

1 1

0 0

-1 -1 0.01 0.1 1 10 100 0.01 0.1 1 10 100 Scale Scale

Small scales (→ 0) Scale 1 Large scales (→ ∞) UE CE CE UE CE Grey noise Max Max Max Min Markov/ARMA(1,1) Min Min Max Min Simple scaling Max/R Max/R (Large) (Large) Asymptotic scaling (Large) Max/R Max/R

Note R: solutions with additional restrictions D. Koutsoyiannis and A. Georgakakos, Lessons from the long flow records of the Nile 23 Entropy 1 1 ) ) k k ( ( j j ρ ρ extremizing 0.1 solutions – 0.1 0.01

Auto- 0.01 correlation 0.001

0.001 0.0001 functions 1 10 100 1 10 100 j j 1 1 ) ) k ( k ( j j ρ ρ 0.1

0.01 0.1

0.001

0.0001 0.01 1 10 100 1 10 100 j j k=1/128 k=1/16 k=1/4 k=1 k=4 k=16 k=128

D. Koutsoyiannis and A. Georgakakos, Lessons from the long flow records of the Nile 24 Conclusions

Apart from the annual cycle, no other signatures of determinism appear in the long records of the Nile River The type of stochastic behaviour observed in the Nile records is very different from white noise and suggests a simple scaling or asymptotic scaling process Both these processes are consistent with the maximum entropy principle and suggest much higher entropy on small and large scales in comparison to Markov processes Thus, the observed behaviour can be interpreted as dominance of uncertainty in nature (… which makes the world interesting, i.e. not boring) Both classical physics and quantum physics are indeterministic Karl Popper (in his book “Quantum Theory and the Schism in Physics”) The future is not contained in the present or the past W. W. Bartley III (in Editor’s Foreward to the same book)

D. Koutsoyiannis and A. Georgakakos, Lessons from the long flow records of the Nile 25 References

Bernoulli, J. (1713), Ars Conjectandi, Reprinted in 1968 by Culture et Civilisation, Brussels Jaynes, E.T. (1957), Information Theory and Statistical Mechanics I, Physical Review, 106, 620-630 Koutsoyiannis, D. (2005), Uncertainty, entropy, scaling and hydrological stochastics, 2, Time dependence of hydrological processes and time scaling, Hydrological Sciences Journal, 50(3), 405- 426, 2005 Laplace, P.S. de, (1829), Essai philosophique sur les Probabilités, H. Remy, 5th édition, Bruxelles Papoulis, A. (1991) Probability, Random Variables, and Stochastic Processes (third edn.), McGraw-Hill, New York, USA Tsallis, C. (1988) Possible generalization of Boltzmann-Gibbs statistics, J. Stat. Phys. 52, 479-487 Both classical physics and quantum physics are indeterministic Karl Popper (in his book “Quantum Theory and the Schism in Physics”) The future is not contained in the present or the past W. W. Bartley III (in Editor’s Foreward to the same book)

D. Koutsoyiannis and A. Georgakakos, Lessons from the long flow records of the Nile 26