sign in sign up
<<
Home , Attractor, Bifurcation diagram, Chaos theory, Dynamical system, Edge of chaos, Logistic map

Communication Visual Analysis of Nonlinear Dynamical Systems: , , Self-Similarity and the Limits of Prediction

Geoff Boeing

Department of City and Regional Planning, University of California, Berkeley, CA 94720, USA; [email protected]; Tel.: +1-510-642-6000

Academic Editor: Ockie Bosch Received: 7 September 2016; Accepted: 7 November 2016; Published: 13 November 2016

Abstract: Nearly all nontrivial real-world systems are nonlinear dynamical systems. Chaos describes certain nonlinear dynamical systems that have a very sensitive dependence on initial conditions. Chaotic systems are always deterministic and may be very simple, yet they produce completely unpredictable and divergent behavior. Systems of nonlinear are difficult to solve analytically, and scientists have relied heavily on visual and qualitative approaches to discover and analyze the dynamics of nonlinearity. Indeed, few fields have drawn as heavily from visualization methods for their seminal : from strange , to bifurcation diagrams, to cobweb plots, to phase diagrams and embedding. Although the social sciences are increasingly studying these types of systems, seminal concepts remain murky or loosely adopted. This article has three aims. First, it argues for several visualization methods to critically analyze and understand the behavior of nonlinear dynamical systems. Second, it uses these visualizations to introduce the foundations of nonlinear dynamics, chaos, fractals, self-similarity and the limits of prediction. Finally, it presents Pynamical, an open-source Python package to easily visualize and explore nonlinear dynamical systems’ behavior.

Keywords: visualization; nonlinear dynamics; chaos; ; ; bifurcation; dynamical systems; prediction; python; logistic

1. Introduction Chaos is a branch of that deals with nonlinear dynamical systems. A is simply a of interacting components that form a larger whole. Nonlinear means that due to or multiplicative effects between the components, the whole becomes something greater than the mere sum of its individual parts. Lastly, dynamical means the system changes over based on its current state. Nearly every nontrivial real-world system is a nonlinear . Chaotic systems are a type of nonlinear dynamical system that may contain very few interacting parts and may follow simple rules, but all have a very sensitive dependence on their initial conditions [1,2]. One might expect that any simple would produce easily-predictable behavior. Yet, despite their deterministic simplicity, over time, these systems can produce wildly unpredictable, divergent and fractal (i.e., infinitely detailed and self-similar without ever actually repeating) behavior due to that sensitivity. such systems’ thus requires an impossible precision of measurement and computation. Chaos fundamentally indicates that there are limits to knowledge and prediction because some futures may be unknowable with any precision. Further, interventions into a system may have unpredictable outcomes even if the intervention is very minor, as tiny effects can compound (or be damped) nonlinearly over time.

Systems 2016, 4, 37; doi:10.3390/systems4040037 www.mdpi.com/journal/systems Systems 2016, 4, 37 2 of 18

Real-world chaotic and fractal systems span the spectrum from leaky faucets [3], to ferns [4], to heart rates [5–7], to [8,9]. Many scholars have studied the implications of nonlinearity, chaos and fractals for the social sciences, including [10,11], urban studies [12–16], [17–21], architecture [22,23] and city planning [24–27]. One constant throughout the interdisciplinary history of nonlinear dynamical systems’ study is that nonlinear systems are extremely difficult to solve analytically because they cannot be broken down into constituent parts, solved individually, then recombined as a solution. Scientists have instead relied heavily on visual and qualitative approaches, a perspective first developed by Henri Poincaré in the late 1800s, to discover and analyze the fascinating dynamics of nonlinearity [28,29]. Information visualization helps analysts detect and examine hidden in complex datasets [30]. In particular, few fields have drawn as heavily from visualization as nonlinear dynamics and chaos have for their pivotal discoveries, from Lorenz’s first visualization of strange attractors [31], to May’s groundbreaking bifurcation diagrams [32], to phase diagrams for discerning higher-dimensional hidden in data [33]. Such nonlinear analysis is particularly useful, yet underutilized for exploring [34,35]. These methods in turn have broad applicability to visual . This article introduces nonlinearity and chaos interdisciplinarily through the methods of data visualization, using a logistic model to dissect the terminology, visualize pertinent features of chaos and fractals and discuss the wide-ranging implications for knowledge and prediction. It makes three primary contributions. First, it reviews and disseminates advanced visualization techniques for the qualitative analysis of nonlinear dynamical systems’ behavior to an interdisciplinary body of systems scholars. Second, it provides a visual introduction to the salient concepts of nonlinearity and chaos to a scholarly audience. Although the social sciences are increasingly studying these types of systems, some of the seminal concepts remain murky or loosely adopted in the theoretical literature [36]. Most formal treatments of chaos and nonlinear dynamics in the scholarly literature are densely technical and geared toward an exclusive audience of mathematicians and physicists. For this article, rather, readers require only a familiarity with . We thus do not cover the rigorous mathematical underpinnings of chaos and nonlinear dynamics, but the references throughout cite both the original foundational publications in this field, as well as recent scholarly developments. Interested readers will be well-rewarded in consulting these works. Third and finally, it presents Pynamical, a free open-source Python software package for the visual analysis of discrete nonlinear dynamical systems. Comparable tools usually must be developed from scratch or rely on expensive commercial software, such as MATLAB [37]. Pynamical provides a fast, simple, reusable and extensible new means for exploring system behavior. The following section provides a background to the and the concepts of and attractors. Then, we introduce several information visualization techniques to explore qualitative system behavior, bifurcations, the path to chaos, fractals and strange attractors. We investigate the difference between chaos and before finally visualizing the famous butterfly effect and discussing its implications for scientific prediction. All of these models and visualizations are developed in Python using Pynamical; for readability, we reserve the technical details of its functionality for AppendixA.

2. Background and Model Edward Lorenz, the father of [38], once described chaos as “when the present determines the , but the approximate present does not approximately determine the future” [39]. Lorenz first discovered chaos by accident while developing a simple of atmospheric convection, using three ordinary differential equations [31]. He found that nearly indistinguishable initial conditions could produce completely divergent outcomes, rendering prediction impossible beyond a time horizon of about a fortnight [40]. How can this possibly happen with a simple deterministic system? We will explore an example using the logistic map, a model based on the common s- logistic that shows how a population grows slowly, then rapidly, before tapering off as it reaches its environment’s carrying Systems 2016, 4, 37 3 of 18 capacity [41,42]. The uses a differential that treats time as continuous. The logistic map instead uses a difference equation to look at discrete time steps [43,44]. It is called the logistic map because it maps the population value at any time step to its value at the next time step: xt+1 = r·xt·(1 − xt). This nonlinear equation defines the rules, or dynamics, of our system: x represents the population at some time t, and r represents the growth rate. Thus, the population level at any given time is a function of the growth rate and the previous time step’s population level. If the growth rate is set too low, the population will die out and go extinct. Higher growth rates might settle toward a stable value or fluctuate across a series of population booms and busts. Chaos can manifest itself in both continuous (i.e., with dynamics defined by differential equations) and discrete (i.e., with dynamics defined by an iterated map) nonlinear dynamical systems. The logistic map is a simple, one-dimensional, discrete equation that produces chaos at certain growth rates. We will explore this in depth momentarily, but first, we use Pynamical to run the logistic model for 20 time steps (we will henceforth call these recursive iterations of the equation generations) for growth rate parameter values of 0.5, 1, 1.5, 2, 2.5, 3 and 3.5. Table1 presents the results. The columns represent growth rates, and the rows represent generations. The model always starts with a population level of 0.5 and represents population as a ratio between zero (extinction) and one (the maximum carrying capacity of our system). If we trace down the column in Table1 under a growth rate of 1.5, we see that the population level eventually settles toward a final value of 0.333 after several generations. In the column for a growth rate of two, we see an unchanging population level of 0.5 across every generation. This makes sense in the real world: if two parents produce two children, the overall population will neither grow nor shrink. Thus, a growth rate parameter value of two represents the replacement rate.

Table 1. Population values produced by the logistic map with 7 growth rate parameter values over 20 generations.

Generation r = 0.5 r = 1.0 r = 1.5 r = 2.0 r = 2.5 r = 3.0 r = 3.5 1 0.500 0.500 0.500 0.500 0.500 0.500 0.500 2 0.125 0.250 0.375 0.500 0.625 0.750 0.875 3 0.055 0.188 0.352 0.500 0.586 0.562 0.383 4 0.026 0.152 0.342 0.500 0.607 0.738 0.827 5 0.013 0.129 0.338 0.500 0.597 0.580 0.501 6 0.006 0.112 0.335 0.500 0.602 0.731 0.875 7 0.003 0.100 0.334 0.500 0.599 0.590 0.383 8 0.002 0.090 0.334 0.500 0.600 0.726 0.827 9 0.001 0.082 0.334 0.500 0.600 0.597 0.501 10 0.000 0.075 0.333 0.500 0.600 0.722 0.875 11 0.000 0.069 0.333 0.500 0.600 0.603 0.383 12 0.000 0.065 0.333 0.500 0.600 0.718 0.827 13 0.000 0.060 0.333 0.500 0.600 0.607 0.501 14 0.000 0.057 0.333 0.500 0.600 0.716 0.875 15 0.000 0.054 0.333 0.500 0.600 0.610 0.383 16 0.000 0.051 0.333 0.500 0.600 0.713 0.827 17 0.000 0.048 0.333 0.500 0.600 0.613 0.501 18 0.000 0.046 0.333 0.500 0.600 0.711 0.875 19 0.000 0.044 0.333 0.500 0.600 0.616 0.383 20 0.000 0.042 0.333 0.500 0.600 0.710 0.827

Figure1 visualizes the resulting time series as a graph produced by Pynamical, with time on the x-axis and the system state on the y-axis. This graph visualizes how the population changes over time at different growth rates. For instance, the violet for a growth rate of 0.5 quickly drops to zero: the population dies out. The teal line that represents a growth rate of two (the replacement rate) stays steady at a population level of 0.5. The growth rates of three and 3.5 are more interesting. While the green line for a growth rate of three seems to slowly converge toward a stable value, the yellow line for a growth rate of 3.5 just seems to repeatedly bounce around four different values. Systems 2016, 4, 37 4 of 18 green line for a growth rate of three seems to slowly converge toward a stable value, the yellow line fSystemsor a growth2016, 4, 37rate of 3.5 just seems to repeatedly bounce around four different values. 4 of 18 An attractor is the value, or set of values, that a system settles toward over time. When the growth rate parameter is set to 0.5, the system has a fixed- attractor at a population level of zero, as depictedAn attractor by the is theviolet value, line. or In set other of values, words, that the a population system settles value toward is drawn over toward time. When a stable the equilibriumgrowth rate parameter of zero over is set time to 0.5, as the the system model has iterates: a fixed-point the logistic attractor equation at a population maps the level value of zero, of a fixedas depicted-point by attractor the violet to line. itself. In Wh otheren words, the growth the population rate parameter value is is drawn set to toward 3.5, the a stable system equilibrium oscillates betweenof zero over four time values as the as modeldepicted iterates: by the the yellow logistic line. equation This oscillating maps the attractor value of is a fixed-pointcalled a limit attractor cycle. However,to itself. When when the we growth adjust the rate growth parameter rate isparameter set to 3.5, beyond the system 3.57, oscillateswe witness between the onset four of valueschaos. asA chaoticdepicted system by the yellowhas a strange line. This attractor oscillating, around attractor which is called the system a . oscillates However, forever when without we adjust ever repeatingthe growth itself rate or parameter settling beyondinto a steady 3.57, we state witness of behavior the onset [45,46] of chaos.. It never A chaotic produces system the has same a strange value twiceattractor,, and around its structure which is thefractal, system meaning oscillates the same forever without exist ever at every repeating scale no itself matter or settling how much into wea steady zoom state into ofit [47] behavior. [45,46]. It never produces the same value twice, and its structure is fractal, meaning the same patterns exist at every scale no matter how much we zoom into it [47].

Figure 1. Time series graph of the logistic map with seven growth rate parameter values over Figure 1. Time series graph of the logistic map with seven growth rate parameter values over 20 generations. 20 generations. 3. System Bifurcations 3. System Bifurcations To show this more clearly, we run the logistic model again, this time for 200 generations across To show this more clearly, we run the logistic model again, this time for 200 generations across 1000 growth rate values between zero and four. When we produced the plot in Figure1, we had only 1000 growth rate values between zero and four. When we produced the plot in Figure 1, we had only seven growth rates. This time, we have 1000, so we need to visualize the results in a different way seven growth rates. This time, we have 1000, so we need to visualize the results in a different way to to make them comprehensible, using a that visualizes a system’s attractors as make them comprehensible, using a bifurcation diagram that visualizes a system’s attractors as a a function of some parameter [32,48,49]. The bifurcation diagram in Figure2 represents 1000 discrete function of some parameter [32,48,49]. The bifurcation diagram in Figure 2 represents 1000 discrete vertical slices, each corresponding to one of 1000 growth rate parameter values evenly spaced between vertical slices, each corresponding to one of 1000 growth rate parameter values evenly spaced zero and four. To produce each of these visual slices, Pynamical ran the model 200 , then threw between zero and four. To produce each of these visual slices, Pynamical ran the model 200 times, away the first 100 results, leaving just the final 100 generations for each growth rate. Each vertical slice then threw away the first 100 results, leaving just the final 100 generations for each growth rate. Each thus visualizes the population values that the logistic map settles toward over time (i.e., the attractor) vertical slice thus visualizes the population values that the logistic map settles toward over time (i.e., for that parameter value. the attractor) for that parameter value. In Figure2, we can see that for growth rates less than one, the system always eventually collapses In Figure 2, we can see that for growth rates less than one, the system always eventually collapses to zero (extinction). For growth rates between one and three, the system always settles into an exact, to zero (extinction). For growth rates between one and three, the system always settles into an exact, stable population level. For instance, in the vertical slice above a growth rate of 2.5, there is only stable population level. For instance, in the vertical slice above a growth rate of 2.5, there is only one one population value represented (0.6), and it corresponds precisely to where the line for a growth rate population value represented (0.6), and it corresponds precisely to where the line for a growth rate of 2.5 settles in Figure1’s time graph. At this parameter value, the system’s attractor is a fixed point of 2.5 settles in Figure 1’s time graph. At this parameter value, the system’s attractor is a fixed point at 0.6. However, for some growth rates, such as 3.9, the plot in Figure2 shows 100 different values; Systems 2016, 4, 37 5 of 18 Systems 2016, 4, 37 5 of 18

at 0.6. However, for some growth rates, such as 3.9, the plot in Figure 2 shows 100 different values; in other words, a different different value value for for each each of of its its 100 100 generations. generations. Here Here,, the the system system never never settles settles into into a afixed fixed point point or or a alimit limit cycle. cycle.

Figure 2. Bifurcation diagram of 100 generations of the logistic map for 1000 growth rate parameterparameter values between zero and four.four. The vertical slice above each growth rate depicts the system’s attractor at that rate.

Why is this visualization called a bifurcation diagram? If we zoom into the growth rates between 2.82.8 and 4four to see to see what what is happening is happening at a at finer a finer scale scale (Figure (Figure3), the 3), possible the possible population population values values fork intofork twointo discretetwo discrete paths paths at the at vertical the vertical slice above slice above a growth a growth rate of rate three. of Atthree a growth. At a growth rate of 3.2,rate the of 3.2, system the oscillatessystem oscillates exclusively exclusively between between two population two population values: one values: around one 0.5 around and the 0.5 other and aroundthe other 0.8. around Thus, at0.8. that Thus, growth at that rate, growth applying rate, theapplying logistic the map logistic to one map of theseto one two of these population two population values yields values the yields other. Justthe other. beyond Just a growth beyond rate a growth of 3.4, therate diagram of 3.4, the bifurcates diagram again bifurcates into four again paths. into Thisfourcorresponds paths. This tocorresponds the yellow to line the inyellow Figure line1: whenin Figure the 1: growth when ratethe growth parameter rate isparameter set to 3.5, is the set systemto 3.5, the oscillates system overoscillates four populationover four population values. These values. are periods,These are just periods like the, just period like ofthe a .period of a Atpendul a growthum. At rate a ofgrowth 3.2, the rate system of 3.2, has the asystem period-two has a attractor.period-two At attractor. a growth At rate a growth of 3.5, the rate system of 3.5, has the asystem period-four has a attractor.period-four Just attractor. beyond a Just growth beyond rate a of growth 3.5, it bifurcates rate of 3.5, again it bifurcates into eight again paths intoas the eight system paths oscillates as the oversystem eight oscillates population over values.eight population These consecutive values. These bifurcations consecutive are phase bifurcations transitions are from phase one transitions behavior, suchfrom asone a fixed-pointbehavior, such attractor, as a fixed to a- qualitativelypoint attractor, different to a qualitatively type of behavior, different such type as a of period-two behavior, limitsuch cycleas a period attractor,-two as limit we vary cycle the attractor, parameter as value.we vary Beyond the parameter a growth value. rate of Beyond 3.57, however, a growth the bifurcationsrate of 3.57, ramphowever, up until the thebifurcations system is ramp capable up of until eventually the system landing is on capable any population of eventually value. landing This is on known any aspopulation the period-doubling value. This is path known to chaos. as the As period we adjust-doubling the growth path to ratechaos. parameter As we adjust upwards, the growth the logistic rate mapparameter will oscillate upwards, between the logistic two, map then will four, oscillate then eight, between then tw 16,o, thenthen 32four, (and then on eight, and on then to infinity)16, then population32 (and on and values. on to infinity) population values.

Systems 2016, 4, 37 6 of 18

Systems 20162016,, 4,, 37 6 of 18

Figure 3. Bifurcation diagram of 100 generations of the logistic map for 1000 growth rate parameter Figurevalues 3.betweenBifurcation 2.8 and diagram 4. The ofvertical 100 generations slice above of each the logisticgrowth maprate depicts for 1000 the growth system’s rate attractor parameter at Figure 3. Bifurcation diagram of 100 generations of the logistic map for 1000 growth rate parameter valuesthat rate. between 2.8 and 4. The vertical slice above each growth rate depicts the system’s attractor at values between 2.8 and 4. The vertical slice above each growth rate depicts the system’s attractor at that rate. thatBy the rate. time we reach a growth rate of 3.99, it has bifurcated so many times that the system now jumps, seemingly randomly, between all population values. We only say seemingly randomly because By the time we reach a growth rate of 3.99, it has bifurcated so many times that the system now it is definitely not truly random. Rather, this model follows very simple deterministic rules yet jumps, seeminglyseemingly randomly,randomly, between between all all population population values. values. We We only only say sayseemingly seeminglyrandomly randomly because because it produces apparent randomness due to its attractor having a period of infinite length. This is chaos: isit definitely is definitely not not truly truly random. random. Rather, Rather, this model this model follows follows very simple very deterministic simple deterministic rules yet producesrules yet deterministic and aperiodic. If we zoom in again, to the narrow slice of growth rates between 3.7 and apparentproduces randomnessapparent randomness due to its attractor due to its having attractor a period having of infinitea period length. of infinite This islength. chaos: This deterministic is chaos: 3.9 (Figure 4), we begin to see the visceral beauty of chaos. Out of the noise emerge strange swirling anddeterministic aperiodic. and If we aperiodic. zoom in If again, we zoom to the in narrow again, to slice the of narrow growth slice rates of betweengrowth rates 3.7 and between 3.9 (Figure 3.7 and4), patterns and thresholds on either side of which the system behaves very differently. For example, we3.9 (Figure begin to 4), see we the begin visceral to see beauty the visceral of chaos. beauty Out of chaos. the noise Out emerge of the noise strange emerge swirling strange patterns swirling and between the growth rates of 3.82 and 3.84, the system moves from chaos back into order, oscillating thresholdspatterns and on thresholds either side on of either which side the systemof which behaves the system very behaves differently. very For differently. example, For between example, the between just three population values: approximately 0.15, 0.55 and 0.95. However, then at growth growthbetween rates the growth of 3.82 andrates 3.84, of 3.82 the and system 3.84, moves the system from chaosmoves back from into chaos order, back oscillating into order, between oscillating just rates beyond 3.86, it bifurcates again and returns to chaos. Indeed any one-dimensional system with threebetween population just three values: population approximately values: approximately 0.15, 0.55 and 0.95. 0.15, However, 0.55 and then 0.95. at However, growth rates then beyond at growth 3.86, a period-three cycle such as this at some parameter value is capable of chaotic behavior at other itrates bifurcates beyond again 3.86, andit bifurcates returns to again chaos. and Indeed returns any to one-dimensional chaos. Indeed any system one-dimen with asional period-three system cyclewith parameter values [50]. sucha period as this-three at somecycle parameter such as this value at some is capable parameter of chaotic value behavior is capable at other of chaotic parameter behavior values at [ 50 other]. parameter values [50].

Figure 4. Bifurcation diagram of 100 generations of the logistic map for 1000 growth rate parameter valuesFigure between4. Bifurcation 3.7 and diagram 3.9. The of system 100 generations moves from of orderthe logistic to chaos map and for back 1000 again growth as therate growth parameter rate isvalues adjusted. between 3.7 and 3.9. The system moves from order to chaos and back again as the growth rate Figure 4. Bifurcation diagram of 100 generations of the logistic map for 1000 growth rate parameter is adjusted. values between 3.7 and 3.9. The system moves from order to chaos and back again as the growth rate is adjusted.

Systems 20162016,, 4,, 37 7 of 18

Universality refers to the phenomenon that very different systems can exhibit very similar Universality refers to the phenomenon that very different systems can exhibit very similar behavior regardless of their underlying dynamics. It is commonly associated with Mitchell behavior regardless of their underlying dynamics. It is commonly associated with ’s discovery that all systems that undergo this period-doubling path to chaos obey a Feigenbaum’s discovery that all systems that undergo this period-doubling path to chaos obey [51,52]: the distance between consecutive bifurcations along the horizontal axis a mathematical constant [51,52]: the distance between consecutive bifurcations along the horizontal axis shrinks by a factor that asymptotically approaches 4.669, now known as Feigenbaum’s constant [44]. shrinks by a factor that asymptotically approaches 4.669, now known as Feigenbaum’s constant [44]. Regardless of the system’s specific dynamics, the ratio of the bifurcations on its road to chaos always Regardless of the system’s specific dynamics, the ratio of the bifurcations on its road to chaos always obeys this constant. obeys this constant. 4. Fractals and Strange Attractors 4. Fractals and Strange Attractors There is also a deep and universal connection between chaos and fractals [37]. In Figure 4, the There is also a deep and universal connection between chaos and fractals [37]. In Figure4, bifurcations around a growth rate of 3.85 may look familiar. If we zoom in to the center one the bifurcations around a growth rate of 3.85 may look familiar. If we zoom in to the center one (Figure 5), we incredibly see the same structure that we saw earlier at the macro-level. In fact, if we (Figure5), we incredibly see the same structure that we saw earlier at the macro-level. In fact, if we keep zooming infinitely in to this visualization, we will continue seeing the same structure and keep zooming infinitely in to this visualization, we will continue seeing the same structure and patterns patterns at finer and finer scales, forever. How can this possibly be? We mentioned earlier that chaotic at finer and finer scales, forever. How can this possibly be? We mentioned earlier that chaotic systems systems have strange attractors and their structure can be characterized as fractal [53–55]. Fractals have strange attractors and their structure can be characterized as fractal [53–55]. Fractals are shapes are shapes that are self-similar, meaning they have the same structure at every scale [56–58]. As we that are self-similar, meaning they have the same structure at every scale [56–58]. As we zoom in zoom in on them, we find smaller copies of the larger macro-structure. The bifurcation diagram (and on them, we find smaller copies of the larger macro-structure. The bifurcation diagram (and thus, thus, the attractor) of the logistic map is a fractal: at the fine scale in Figure 5, we see a tiny reiteration the attractor) of the logistic map is a fractal: at the fine scale in Figure5, we see a tiny reiteration of of the same bifurcations, chaos and limit cycles we saw in Figure 1’s visualization of the full range of the same bifurcations, chaos and limit cycles we saw in Figure1’s visualization of the full range of growth rates. growth rates.

Figure 5. Bifurcation diagram of 100 generations of the logistic map for 1000 growth rate parameter Figure 5. Bifurcation diagram of 100 generations of the logistic map for 1000 growth rate parameter values between 3.84 and 3.856. This is the same structure that we saw earlier at the macro-level in values between 3.84 and 3.856. This is the same structure that we saw earlier at the macro-level in Figure3, because chaotic systems’ strange attractors are fractal. Figure 3, because chaotic systems’ strange attractors are fractal.

AnotherAnother way to visualize this nonlinear time series is with a phase diagram,diagram, using a method called state state-space-space reconstruction reconstruction through through delay delay-coordinate-coordinate embedding embedding [35] [35. Simply]. Simply put, put, this this plots plots the thesystem’s system’s value value at generation at generation t + 1 ont + the 1 on y-axis the y versus-axis versus its value its at value t on the at t xon-axis the [59]x-axis, giving [59], us giving another us anothervisual window visual windowinto the qualitative into the qualitative behavior behavior of the system. of the The system. clever The insight clever of insight this phase of this diagram phase diagramis that it isembeds that it embedsone-dimensional one-dimensional time series time data series from data our from logistic our logistic map into map two into-dimensional two-dimensional state statespace: space: an imaginary an imaginary space spacethat uses that system uses system variables variables as its as its dimensions [33,60,61] [33. ,Each60,61 ].point Each in point state inspace state is spacea system is a state system, or state,in other or inwords, other a words, set of variable a set of variablevalues. While values. traditional While traditional systems’ systems’analysis analysistends to tends focus to on focus visualizing on visualizing time series time seriesas in Figureas in Figure 1, nonlinear1, nonlinear dynamics dynamics tends tends to focus to focus on visualizing these state spaces. Few real-world systems are fully observable, yet the dynamics in a properly reconstructed are identical to the true dynamics of the entire system [34].

Systems 2016, 4, 37 8 of 18

onSystems visualizing 2016, 4, 37 these state spaces. Few real-world systems are fully observable, yet the dynamics8 of in18 a properly reconstructed state space are identical to the true dynamics of the entire system [34]. InIn ourour case,case, thethe twotwo variables variables are: are: (1) (1) the the population population value value at at generation generationt; andt; and (2) (2) the the value value at tat+ t 1.+ 1. For For example, example, with with a a growth growth rate rate of of 3.5, 3.5, the the population population value value at at GenerationGeneration 11 isis 0.5;0.5; thethe valuevalue atat GenerationGeneration 2 2 is is 0.875 0.875;; the the value value at at Generation Generation 3 is 3 0.383 is 0.383;; and and so forth so forth (see (see Table Table 1). 1Therefore). Therefore,, our ourtwo two-dimensional-dimensional phase phase diagram diagram will will have have (x, (xy,) ypoints) points at at(0.5, (0.5, 0.875) 0.875) and and (0.875, (0.875, 0.383) 0.383),, and and so on (Figure(Figure6 6B).B). Remember Rememberthat thatour ourmodel modelfollows follows aasimple simpledeterministic deterministic rule,rule, soso ifif wewe knowknow aa certain certain generation’sgeneration’s population value, value, we we can can easily easily determine determine the the next next generation’s generation’s value. value. Like Like earlier, earlier, to toproduce produce these these phase phase diagrams diagrams,, Pynamical Pynamical runs runs the the logistic logistic model model for for 200 200 generations and then then discardsdiscards thethe firstfirst 100100 rows,rows, toto visualizevisualize onlyonly thosethose valuesvalues thatthat thethe systemsystem settlessettles towardtowardover overtime. time.

FigureFigure 6.6. PhasePhase diagramsdiagrams ofof the the logistic logistic map map over over 200 200 generations generations for for growth growth rate rate parameter parameter values values of: 2.9of: (2.9A); ( 3.5A); (3.5B); ( 3.56B); 3.56 (C); ( andC); and 3.57 3.57 (D). When(D). When the parameter the parameter is set is to set 2.9, to the 2.9, model the mo convergesdel converges at a single at a fixed-point.single fixed- Whenpoint. theWhen parameter the parameter is set to 3.5is set or to higher, 3.5 or the higher, model the oscillates model overoscillates four points,over four then points, eight, andthen on eight, and and on as on it and bifurcates. on as it bifurcates.

In Figure 6A, the phase diagram shows that the logistic map homes in on a fixed-point attractor at 0.655 (on both axes) when the growth rate parameter is set to 2.9. This corresponds to the vertical slice above the x-axis value of 2.9 in the bifurcation diagram in Figure 2. Figure 6B depicts a period- four limit cycle attractor: when the growth rate is set to 3.5, the logistic map oscillates over four points, as shown in this phase diagram (and in Figures 1 and 2). If we adjust the growth rate parameter up

Systems 2016, 4, 37 9 of 18

In Figure6A, the phase diagram shows that the logistic map homes in on a fixed-point attractor at 0.655 (on both axes) when the growth rate parameter is set to 2.9. This corresponds to the vertical slice above the x-axis value of 2.9 in the bifurcation diagram in Figure2. Figure6B depicts a period-four limitSystems cycle 2016, attractor:4, 37 when the growth rate is set to 3.5, the logistic map oscillates over four points,9 of 18 as shown in this phase diagram (and in Figures1 and2). If we adjust the growth rate parameter up toto 3.56,3.56, wewe witnesswitness aa period-doublingperiod-doubling bifurcation:bifurcation: FigureFigure6 6CC shows shows the the system system now now oscillating oscillating over over eighteight points.points. AsAs wewe approachapproach thethe chaoticchaotic regime,regime, thethe rangerange ofof parameterparameter valuesvalues inin whichwhich ourour systemsystem behavesbehaves chaotically,chaotically the, the period-doubling period-doubling bifurcations bifurcations start start to cometo come more more quickly. quickly. Figure Figure6D shows 6D shows that severalthat several additional additional bifurcations bifurcations occurred occurred between betwe theen growth the growth rates rates of 3.56 of and3.56 3.57. and 3.57. AA kindkind ofof structurestructure isis slowlyslowly beingbeing revealedrevealed acrossacross FigureFigure6 6,, but but we we can can see see it it much much more more clearly clearly asas wewe pushpush thethe growthgrowth raterate parameterparameter valuevalue deepdeep intointo thethe chaoticchaotic regime.regime. TheThe phasephase diagramdiagram inin FigureFigure7 A 7A reveals reveals the the system’s system’s attractor attractor at at a a growth growth rate rate of of 3.9. 3.9. Figure Figure7B 7B visualizes visualizes 50 50 different different growthgrowth raterate parameterparameter valuesvalues betweenbetween 3.63.6 andand 4,4, eacheach withwith itsits ownown color.color. ThoseThose ratesrates thatthat exhibitexhibit chaoschaos formform parabolas,parabolas, butbut somesome gapsgaps existexist wherewhere thethe systemsystem occasionallyoccasionally settlessettles downdown intointo periodicperiodic behaviorbehavior (e.g.,(e.g., inin thethe tealteal bandband whenwhen thethe growthgrowth raterate isis setset toto 3.83;3.83; comparecompare thisthis bandband ofof periodicityperiodicity withwith FigureFigure4 4).).

FigureFigure 7.7. CroppedCropped phasephase diagramsdiagrams ofof thethe logisticlogistic mapmap overover 200200 generationsgenerations for:for: ((A)) aa growthgrowth raterate parameterparameter value value of of 3.9; 3.9 and; and (B ) ( 50B) growth 50 growth rate parameter rate parameter values values between between 3.6 and 3 4.6 (the and chaotic 4 (the regime), chaotic eachregime), with each its own with colored its own line colored line

StrangeStrange attractorsattractors areare revealed revealed by by these these shapes shapes as as the the system system is is somehow somehow oddly oddly constrained, constrained, yet yet it neverit never settles settles into into a fixeda fixed point point or or limit limit cycle cycle like like it it did did in in Figure Figure6. 6. Instead Instead it it just just bounces bounces around around differentdifferent populationpopulation valuesvalues (i.e.,(i.e., pointspoints onon thethe parabola)parabola) foreverforever withoutwithout everever repeatingrepeating thethe samesame valuevalue twice.twice. It is is impossible impossible to to predict predict if if any any two two consecutive consecutive observations observations appear appear near near each each othe otherr or orfar far apart apart on on the the parabola. parabola. Further, Further, the the parabolas parabolas in in Figure Figure 7B never overlap overlap due due to to their their fractal fractal geometrygeometry andand thethe deterministicdeterministic naturenature ofof thethe logisticlogistic map.map. Consider: if two different parameter values couldcould everever landland onon thethe exactexact samesame point,point, theirtheir systemssystems wouldwould havehave toto evolveevolve identicallyidentically overover timetime becausebecause thethe logisticlogistic mapmap isis deterministic.deterministic. We cancan seesee inin thesethese visualizationsvisualizations thatthat thisthis indeedindeed nevernever happens.happens. WhileWhile the the dynamics dynamics of of a chaotic system system appear appear to haveto have no patternno whatsoever, whatsoever, in reality, in reality they, conformthey conf toorm a remarkable to a remarkable fractal pattern, fractal patterna strange, a attractor, strange which attractor confines, which the confines system to the a limited system slice to a oflimited state spaceslice of and state ensures space thatand noensures state willthat everno state repeat will [ 62ever]. Fractals repeat [62] are. indeed Fractals strange. are indeed Rather strange. than havingRather than a whole-number having a whole -number suchdimension as two such or three, as two they or three, are characterized they are characterized by a fractional by a (hence,fractional fractal) (hence, dimension fractal) [ 55 dimension,61,63]. The [55,61,63] . The fractal refers dimension to the space-filling refers to characteristics the space-filling of characteristics of a curve that, through self-similarity, becomes a bit more than a one-dimensional line yet a bit less than a two-dimensional . These visualizations have all plotted quantitative data to better explain and understand the qualitative behavior of a nonlinear dynamical system. A cobweb plot is a visualization technique particularly well-suited to revealing the qualitative behavior of one-dimensional maps, allowing us to analyze the long-term of such systems under recursive iteration [37]. The cobweb plots drawn by Pynamical in Figure 8 consist of three lines: a diagonal gray identity line representing

Systems 2016, 4, 37 10 of 18 a curve that, through self-similarity, becomes a bit more than a one-dimensional line yet a bit less than a two-dimensional plane. These visualizations have all plotted quantitative data to better explain and understand the qualitative behavior of a nonlinear dynamical system. A cobweb plot is a visualization technique particularly well-suited to revealing the qualitative behavior of one-dimensional maps, allowing us to analyze the long-term evolution of such systems under recursive iteration [37]. The cobweb plots drawnSystems by 201 Pynamical6, 4, 37 in Figure8 consist of three lines: a diagonal gray identity line representing10 ofy 18= x, a red curve representing the logistic map as y = f(x) for a given parameter value and a blue line tracing y = x, a red curve representing the logistic map as y = f(x) for a given parameter value and a blue line the path of the cobweb. To draw a cobweb: tracing the path of the cobweb. To draw a cobweb: 1. Begin on the x-axis at the point (x, 0) where x is the initial population value (0.5 in our example), 1. Begin on the x-axis at the point (x, 0) where x is the initial population value (0.5 in our example), and draw a vertical line to the red function curve; this new point is at (x, f(x)). and draw a vertical line to the red function curve; this new point is at (x, f(x)). x x 2. 2. DrawDraw a a horizontal horizontal line line from from thisthis pointpoint to the gray identity identity line line;; this this new new point point is isat at(f( (f(x), f(),x f()). )). 3. 3. DrawDraw a a vertical vertical line line from from this this pointpoint toto the red function curve curve;; this this new new point point is isat at(f( (f(x),x f(f(), f(f(x))).x))). 4. 4. RepeatRepeat Steps Steps 2 2 and and 3 3 recursively. recursively. TheThe cobwebscobwebs in Figure Figure 8 were iterated iterated 100 100 times. times.

FigureFigure 8. 8.Cobweb Cobweb plots plots of of the the logistic logistic mapmap forfor growth rate parameter parameter values values of of:: (A ()A 1); 1;(B ()B 2.7) 2.7;; (C) ( C3.5) 3.5;; (D)(D 3.9.) 3.9. The The diagonal diagonal graygray identityidentity line line represents represents y y= =x; xthe; the red red curve curve represents represents the logistic the logistic map mapas as y = f(x)) for each of of the four parameter parameter values values;; and and the the blue blue cobweb cobweb line line represents represents the the system’s system’s trajectorytrajectory over over 100 100 generations. generations.

The blue lines intersect the red curve at those values our system lands on as it iterates from an initial population value of 0.5. In Figure 8A,B, the cobweb shows the system homing in on fixed-point attractors of zero and 0.65, respectively. At a growth rate of 3.5 (Figure 8C), the system oscillates over four points in its limit cycle attractor, denoted by rectangular closed loops. The points where the blue lines intersect the red curve are the same as those revealed by the attractor in Figure 6B for the same parameter value. Finally, Figure 8D visualizes our system’s behavior in the chaotic regime at a growth

Systems 2016, 4, 37 11 of 18

The blue lines intersect the red curve at those values our system lands on as it iterates from an initial population value of 0.5. In Figure8A,B, the cobweb shows the system homing in on fixed-point attractors of zero and 0.65, respectively. At a growth rate of 3.5 (Figure8C), the system oscillates over four points in its limit cycle attractor, denoted by rectangular closed loops. The points where the Systems 2016, 4, 37 11 of 18 blue lines intersect the red curve are the same as those revealed by the attractor in Figure6B for the samerate of parameter 3.9. The value. chaotic Finally, Figure fills the8D plotvisualizes with our rectangles system’s, an behavior eventually in the infinite chaotic number regime at of anever growth-repeating rate of 3.9. The chaotic that form orbit a fills fractal the plotcobweb with throughout rectangles, the an eventuallydiagram. infinite number of never-repeating trajectories that form a fractal cobweb throughout the diagram. 5. Chaos and Randomness 5. Chaos and Randomness Phase diagrams are useful for visually revealing strange attractors in time series data, like that producedPhase by diagrams the logistic are usefulmap, because for visually they revealing embed this strange one-dimensional attractors in timedata seriesinto a data,two- likeor even that producedthree-dimensional by the logistic state space. map, It because can be difficult they embed to ascer thistain one-dimensional if certain time series data intoare deterministic a two- or even or three-dimensionaljust random if we do state not space.fully understand It can be difficult their underlying to ascertain dynamics if certain [64] time. Take series the two are deterministicseries plotted orby justPynamical random in if Figure we do 9 not as an fully example. understand Both theirof the underlying lines seem dynamicsto jump around [64]. Takerandomly. the two The series red plottedline does by depict Pynamical random in data, Figure but9 as the an blue example. line comes Both from of the our lines logistic seem model to jump when around the growth randomly. rate Theis set red to 3.99. line doesThis is depict deterministic random data,chaos, but but the it is blue difficult line comesto differentiate from our from logistic randomness. model when Instead the growthin Figure rate 10 is, we set tovisualize 3.99. This these is deterministic same two datasets chaos, but with it isphase difficult diagrams to differentiate rather than from time randomness. graphs, Insteadgiving us in a Figure clear window 10, we visualizeinto the qualitative these same behavior two datasets of our with systems. phase Now diagrams, we can rather clearly than see time our graphs,chaotic givingsystem us constrained a clear window by its into strange the qualitative attractor. behaviorBy contrast, of our the systems. random Now, data wejust can look clearly like seethe ournoise chaotic that they system actually constrained are. by its strange attractor. By contrast, the random data just look like the noise that they actually are.

Figure 9. Plot of two time series, one deterministic/chaotic from the logistic map (blue), and one Figure 9. Plot of two time series, one deterministic/chaotic from the logistic map (blue), and one random (red). random (red).

This is particularly revealing in a three three-dimensional-dimensional phase diagram from Pynamical (Figure 10BB)) that embeds our our time time series series into into a a three three-dimensional-dimensional state state space space by by plotting plotting the the population population value value at atgeneration generation t +t 2+ versus 2 versus the thevalue value at t at + t1+ versus 1 versus the thevalue value at t at. Thist. This plot plotessentially essentially extrudes extrudes our ourtwo- two-dimensionaldimensional plot (Figure plot (Figure 10A), 10 thenA), pans then and pans rotates and rotates the viewpoint. the viewpoint. In fact, Inif we fact, looked if we stra lookedight straightdown at downthe xy at-plane the xy of-plane the three of the-dimensional three-dimensional plot in plotFigure in Figure10B, it 10 wouldB, it would look identical look identical to the totwo the-dimensional two-dimensional plot in plotFigure in Figure10A (see 10 AppendixA (see Appendix A for anA animatedfor an animated visualization visualization of this). ofStrange this). attractors stretch and fold state space in higher dimensions, allowing their fractal forms to fill space without ever producing the same value twice.

Systems 2016, 4, 37 12 of 18

Strange attractors stretch and fold state space in higher dimensions, allowing their fractal forms to fill spaceSystemsSystems without 201 20166, ,4 4, ,37 37 ever producing the same value twice. 1212 of of 18 18

FigureFigureFigure 10.10. 10. PhasePhase Phase diagramsdiagrams diagrams ofof of thethe the timetime time seriesseries series inin in FigureFigure Figure9 .(9. 9. ( A(AA))) isis is aa a two-dimensionaltwo two--dimensionaldimensional statestate state spacespace space versionversion version (the(the(thexy xy xy-plane)--plane)plane) ofof of thethe the three-dimensionalthree three--dimensionaldimensional one one one ( B( (B).B).).

ToToTo press prepressss this thisthis further, further,further, we wewe can cancan use useuse Pynamical PynamicalPynamical to toto visualize visualizevisualize the thethe rest restrest of ofof the thethe logistic logisticlogistic map’s map’smap’s chaotic chaoticchaotic regimeregimeregime in inin three threethree dimensions: dimensions:dimensions: thethethe phasephasephase diagramdiagramdiagram ininin FigureFigureFigure 11 1111 is isis a aa three-dimensional threethree--dimensionaldimensional version versionversion of ofof the thethe two-dimensionaltwotwo--dimensionaldimensional state statestate space spacespace we wewe saw sawsaw in inin Figure FigureFigure7 B. 77BB The. . TheThe novel novelnovel color colorcolor coding codcodinging exposes exposesexposes the thethe dynamical dynamicaldynamical system’ssystem’ssystem’s behavior behavior behavior across across across the chaotic the the chaotic chaotic regime: regime information regime: : informationinformation virtually impenetrable virtually virtually impenetrable impenetrable without visualization. without without Thevisualization.visualization. beautiful The The structure beautiful beautiful of structure thestructure strange of of the the attractor strange strange isattractor attractor revealed is is revealed revealed as it twists as as it it twists twists and curlsand and curls curls around around around its three-dimensionalitsits three three--dimensionaldimensional state state state space space space (see (s (see Appendixee Appendix AppendixA forA A for for an an animatedan animated animated visualization). visualization). visualization). This This This structure structure structure again again again demonstratesdemonstratesdemonstrates that that that ourour our apparentlyapparently apparently random random random time time time series series series data data datafrom from fromthe the the logisticlogistic logistic modelmodel model isis is notnot not trulytruly truly randomrandom random atatat all. all.all. Instead, Instead,Instead, it itit is isis aperiodic aperiodicaperiodic deterministic deterministicdeterministic chaos, chaos,chaos, constrainedconstrainedconstrained byby aa mind-bendingmindmind--bendingbending strangestrastrangenge attractor.attractor. NoNoNo mattermatter matter howhow how muchmuch much wewe we zoomzoom zoom in,in, in, thethe the parabolasparabolas parabolas nevernever never overlapoverlap overlap andand and no no no point point point ever ever ever repeats repeats repeats itself. itself. itself.

Figure 11. Two different viewing perspectives of a single three-dimensional phase diagram of the FigureFigure 11.11. TwoTwo differentdifferent viewingviewing perspectivesperspectives ofof aa singlesingle threethree--dimensionaldimensional phasephase diagramdiagram ofof thethe logistic map over 200 generations for 50 growth rate parameter values between 3.6 and 4, each with its logisticlogistic map map over over 200 200 generations generations for for 50 50 growth growth rate rate parameter parameter values values between between 3 3.6.6 and and 4 4, ,each each with with own colored line. itsits own own colored colored line. line.

Systems 2016, 4, 37 13 of 18 Systems 2016, 4, 37 13 of 18

6. Unpredictable Unpredictable Systems: Systems: The The Butterfly Butterfly Effect Effect Attractors havehave aa basin of attraction:attraction: a set of points that the system’s dynamics will pull into this attractor overover timetime [[65]65].. This is easily seen with a cobweb plot. Figure 12 shows how the logistic map’s basin of attraction (when the growth rate is 2.7) pulls three different initial population values into the same fixed-point fixed-point attractor. attractor. The The initial initial state state of the of system the system will eventually will eventually become become unknowable, unknowable, because anybecause one ofany many one differentof many possibledifferent points possible in thepoints basin in ofthe attraction basin of couldattraction have could been thehave one been pulled the intoone thepulled attractor. into the attractor. By contrast, contrast, chaotic chaotic systems systems are characterized are characterized by their by sensitive their sensitive dependence dependence on initial conditions. on initial Theirconditions. strange Their attractors strange are attractors globally are stable, globally yet locally stable, unstable:yet locally they unstable: have basinsthey have of attraction, basins of yetattraction, within ayet strange within attractor, a strange infinitesimally attractor, infinitesimally close points close diverge points over diverge time without over time ever without leaving ever the attractor’sleaving the confines. attractor’s This confines. divergence Thiscan divergence be measured can be by measured Lyapunov by exponents Lyapunov [66 exponents], the calculation [66], the of whichcalculation is described of which by is Wolf described et al. [67 by]. IfWolf the Lyapunovet al. [67]. exponent’s If the Lyapunov value is expon positive,ent’s then value the is two positive, points movethen the apart two over points time moveat an exponential apart over rate. time If at the an Lyapunov exponential exponent rate. If is the negative, Lyapunov then these exponent points is convergenegative, exponentiallythen these points quickly, converge such asexponentially toward a fixed quickly, point such or limit as toward cycle. Finally,a fixed thepoint Lyapunov or limit exponentcycle. Finally, is zero the when Lyapunov there isexponent a bifurcation is zero [68 when]. For there example, is a bifurcation with our logistic [68]. For model, example, the Lyapunov with our exponentlogistic model, is zero the when Lyapunov the growth exponent rate is is set zero to onewhen or the three growth because rate they is set are to bifurcation one or three points; because it is negativethey are bifurcation for most growth points; rates, it is suchnegative as 0 for≤ r most< 1 and growth 1 < r rates,< 3, because such as they0 ≤ r have< 1 and fixed-point 1 < r < 3, because or limit cyclethey have attractors; fixed- andpoint it isor positivelimit cycle for theattractors; chaotic and regime it is (exclusive positive for of thosethe chaotic occasional regime windows (exclusive when of thethose system occasional resumes windows brief periodicity,when the system such asresumes when thebrief growth periodicity, rate issuch 3.83). as Awhen positive the growth Lyapunov rate exponentis 3.83). A indicatespositive Lyapunov that the system exponent has indicates a highly sensitivethat the system dependence has a highly on initial sensitive conditions dependence and is aon common initial conditions signature and of chaos is a common [69–71]. signature of chaos [69–71].

Figure 12.12. Cobweb plots of the logistic map pulling initial population values of 0.1 (A),), 0.5 (B)) andand 0.9 (C) into into the the same same fixed fixed-point-point attractor attractor over over time. time. At Atthis this growth growth rate rate parameter parameter value value of 2.7, of 2.7,the theLyapunov exponent is negative. is negative.

This nonlinear divergence of veryvery similarsimilar values values makes makes real-world real-world modeling modeling and prediction prediction difficult,difficult, because because we we must must measure the the parameters and system and statesystem with state infinite with precision. infinite Otherwise, precision. tinyOtherwise, errors in tiny measurement errors in measurement or rounding or are rounding compounded are compounded over time until over the time system until the eventually system divergeseventually drastically diverges drastically from the prediction. from the pred Iniction. the real In world, the real infinite world, precisioninfinite precision is impossible. is impossible. It was throughIt was through one such one rounding such rounding error that error Lorenz that first Lorenz discovered first discovered chaos. Recall chaos. his Recall words his at the words beginning at the ofbeginning this article: of this “the article: present “the determines present thedetermines future, but the the future, approximate but the presentapproximate does not present approximately does not determineapproximately the future” determine [39]. the future” [39].

Systems 2016, 4, 37 14 of 18 Systems 2016, 4, 37 14 of 18

As a demonstration of this, we run thethe logistic model with two very similar initial population values, shown in Figure 1313.. Both have the same growth rate parameter value of 3.9. The blue line represents an an initial initial population population value value of 0.5, of 0.5 and, and the red the line red represents line represents an initial an population initial population of 0.50001. of These0.50001. two These initial two conditions initial conditions are extremely are close extremely to one another,close to and one accordingly, another, and their accordingly trajectories, their look trajectoriesessentially identicallook essentially for thefirst identical 30 generations. for the first After 30 generations. that, however, After the that, minuscule however, difference the minuscule in initial differenceconditions in compounds initial conditions to the pointcompounds that by to the the 40th point generation, that by the the 40th two generation lines show, the little two in lines common. show Whatlittle in began common. as nearly What indistinguishable began as nearly initial indistinguishable conditions produces initial completely conditions different produces outcomes completely over differenttime due outcomes to nonlinearity over time and exponentialdue to nonlinearity divergence. and exponential divergence.

Figure 13. 13. PlotPlot of of two two time time seri serieses with with identical identical dynamics, dynamics, one starting one starting at an atinitial an initialpopulation population value blue red valueof 0.5 ( ofblue 0.5) (and the) and other the starting other starting at 0.50001 at 0.50001 (red). At ( this). Atgrowth this growthrate parameter rate parameter value of value 3.9, the of 3.9, the Lyapunov exponent is positive; thus, the system is chaotic, and we can see the nearby points Lyapunov exponent is positive; thus, the system is chaotic, and we can see the nearby points diverge diverge over time. over time.

If our knowledge of these two systems began at GenerationGeneration 50, we would have no way of guessing that that they they were were nearly nearly identical identical in in the the beginning. beginning. With With chaos, chaos, history history is thus is thus lost to lost time to, time, and predictionand prediction of the of future the future is only is onlyas accurate as accurate as our as measurements. our measurements. Human Human measurements measurements are never are infinitelynever infinitely precise, precise, so in real so- inworld real-world chaotic chaoticsystems, systems, errors compound errors compound,, and the andfuture the becomes future becomes entirely unknowableentirely unknowable given long given enough long time enough horizons. time horizons. This phenomenon This phenomenon is famously is famously known as known the butterf as thely effectbutterfly: a : a flaps butterfly its wings flaps in its wings and in Chinasets off and a tornado sets off in a tornadoTexas. Small in Texas. events Small compound events andcompound irreversibly and alter irreversibly the future alter of the universe. future of In the Figure universe. 13, a tiny In Figure fluctuation 13, a of tiny 0.00001 fluctuation makes an of enormous0.00001 makes difference an enormous in the behavior difference and in the state behavior of the andsystem state 40 of generations the system later. 40 generations Although later. this Althoughsystem’s future this system’s cannot future be predicted, cannot be we predicted, can characterize we can characterizeits dynamics its geometrically dynamics geometrically with phase withdiagrams, phase bifurcation diagrams, bifurcation plots and plots cobweb and plotscobweb; and plots; statistically and statistically with Lyapunov with Lyapunov exponents exponents and fractaland fractal dimensions. dimensions.

77.. Conclusion Conclusionss This article had had three three aims. aims. First, First, it it argued argued for for the the usefulness usefulness of ofseveral several visualization visualization methods methods to analyzeto analyze and and understand understand the the behavior behavior of of nonlinear nonlinear dynamical dynamical systems. systems. Second, Second, it it introduced the foundational concepts concepts of of nonlinear nonlinear dynamics, dynamics, chaos, ch fractals,aos, fractals, self-similarity self-similarity and the limitsand the of prediction. limits of prediction.It argued that It arguedinformation that visualization information visualization is a key way to is engage a key way with to these engage system with concepts. these system Third, concepts. Third, and by way of the first two, it demonstrated the Pynamical software package for visualizing the behavior of discrete nonlinear dynamical systems. In particular, Pynamical translates

Systems 2016, 4, 37 15 of 18 and by way of the first two, it demonstrated the Pynamical software package for visualizing the behavior of discrete nonlinear dynamical systems. In particular, Pynamical translates these concepts and tools into the Python programming language because it is prevalent, fast and easy to use (details in AppendixA). This package fills a need for a free, fast, simple, extensible tool to introduce and analyze nonlinear dynamical systems’ behavior visually, making it a useful platform for research, and pedagogy. Nonlinear systems are extremely difficult to solve analytically because they cannot be broken down into constituent parts. Instead of solving them, this article presented visual geometric methods to explore system behavior and stability. These methods have broad applicability to scholarly and professional visual by revealing hidden structure and patterns in time series whose underlying dynamics may not be well known. In particular, they reveal the qualitative behavior of nonlinear dynamical systems over time and/or in response to parameter variations. This article used the logistic map to define such a set of nonlinear dynamics. As simple as this model was, at different growth rate parameter values, it produced stability, periodic or chaos. We used Pynamical to create bifurcation diagrams and cobweb plots to visualize this behavior across different parameter values. In the chaotic regime, the system jumped seemingly randomly between all population values. Accordingly, we used Pynamical to embed the data into higher-dimensional state space to create phase diagrams to visualize the system’s strange attractor and understand its constrained, deterministic dynamics. Finally, we explored the butterfly effect’s implications of nonlinearity on system sensitivity, as infinitesimal differences in initial conditions compounded over time until nearly identical systems had diverged drastically. Thus, in many nonlinear systems, there are fundamental limits to knowledge and prediction. These visualization methods can help researchers discover, examine and understand the behavior of nonlinear dynamical systems and chaos.

Conflicts of Interest: The authors declare no conflict of interest.

Appendix A Pynamical and all of the code used to develop these models and produce these visualizations are available in a public repository on GitHub at https://github.com/gboeing/pynamical. Pynamical is built on top of Python’s numpy, numba, pandas and matplotlib code libraries:

• numpy is a numerical library that handles the underlying data vectors; • numba provides just-in-time compilation for optimized performance; • pandas handles the higher-level data structures and analysis; • matplotlib is the engine used to produce the visualizations and graphics.

Python was selected for developing Pynamical and the visualizations in this article because it is a standard programming environment for information visualization. It offers three key advantages: prevalence, speed and ease of use. First, Python has become a widely-prevalent language. The TIOBE index, a common measure of programming language popularity, ranks Python as the fifth most popular programming language as of 2016. In contrast, other languages commonly used for data science, visualization and mathematics are ranked much lower (e.g., R, MATLAB and SAS are only ranked 18th, 19th and 21st, respectively, in popularity). Developing tools for exploring, understanding and visualizing dynamical systems in Python makes them available to a much wider audience of systems analysts, researchers and students. Furthermore, unlike many languages, Python is completely free and open-source. It is also multi-purpose, and researchers can use its standard syntax and grammar for everything from statistical modeling, to cartography, to full software development. Becoming a Swiss army knife of the computational science world, Python has grown popular and powerful. Today, innumerable researchers and developers contribute open-source libraries of pre-packaged functionality for all to repurpose. Systems 2016, 4, 37 16 of 18

Second, Pynamical is fast. It is built on top of Python’s pandas, numpy and numba libraries for quick vectorized numerical computation. Although it is an interpreted language, these libraries provide compiled components that make mathematical modeling fast and efficient and can take advantage of the optimized mathematical routines of Intel processors’ Math Kernel Library. Pynamical also takes advantage of just-in-time code compilation to offer additional speed-ups. Third, Python is very easy to use, and this simplicity translates to Pynamical. Most of the visualizations in this article can be created from scratch in just one or two lines of straightforward code. The following examples demonstrate this simplicity. The user merely imports Pynamical into the Python environment, then runs the following code to produce the visualization: • Figure2: bifurcation_plot(simulate(num_rates = 1000)) • Figure4: bifurcation_plot(simulate(rate_min = 3.7, rate_max = 3.9, num_rates = 1000)) • Figure6D: phase_diagram(simulate(num_gens = 100, rate_min = 3.57)) • Figure8D: cobweb_plot(r = 3.9, x0 = 0.5) • Figure 11: phase_diagram_3d(simulate(num_gens = 4000, rate_min = 3.6, num_rates = 50)). Pynamical defines extensible functions to express a discrete map’s equation and encapsulate a model to simulate it iteratively. The logistic map, the Singer map and the cubic map are built-in by default, but any other iterated map can be defined and added by the user. Pynamical also defines a function to convert model output into x-y points, as well as functions to plot these points as a bifurcation diagram, a cobweb plot, an animated cobweb plot, a two-dimensional phase diagram, a three-dimensional phase diagram and an animated three-dimensional phase diagram. Animated cobweb plots of the entire parameter space and animated three-dimensional phase diagrams extending those presented in this study are also available in the GitHub repository. They shed particular on the fractal of strange attractors as they stretch and fold state space, thus serving as an indispensable tool for pedagogy and visual information presentation.

References

1. Hastings, A.; Hom, C.L.; Ellner, S.; Turchin, P.; Godfray, H.C.J. Chaos in : Is Mother Nature a Strange Attractor? Annu. Rev. Ecol. Syst. 1993, 24, 1–33. [CrossRef] 2. Rickles, D.; Hawe, P.; Shiell, A. A Simple Guide to Chaos and . J. Epidemiol. Commun. Health 2007, 61, 933–937. [CrossRef][PubMed] 3. Suetani, H.; Soejima, K.; Matsuoka, R.; Parlitz, U.; Hata, H. Learning Approach for Chaos in the Dripping Faucet. Phys. Rev. E 2012, 86, 036209. [CrossRef][PubMed] 4. Singh, S.L.; Mishra, S.N.; Sinkala, W. A New Iterative Approach to Fractal Models. Commun. Nonlinear Sci. Numer. Simul. 2012, 17, 521–529. [CrossRef] 5. Hoshi, R.A.; Pastre, C.M.; Vanderlei, L.C.M.; Godoy, M.F. Poincaré Plot Indexes of : Relationships with Other Nonlinear Variables. Auton. Neurosci. 2013, 177, 271–274. [CrossRef][PubMed] 6. Babbs, C.F. Initiation of Ventricular Fibrillation by a Single Ectopic Beat in Three Dimensional Numerical Models of Ischemic Heart Disease: Abrupt Transition to Chaos. J. Clin. Exp. Cardiol. 2014, 5, 2–11. [CrossRef] 7. Glass, L. Introduction to Controversial Topics in Nonlinear Science: Is the Normal Heart Rate Chaotic? Chaos 2009, 19, 028501. [CrossRef][PubMed] 8. Hong, Z.; Dong, J. Chaos Theory and Its Application in Modern Cryptography. In Proceedings of the 2010 International Conference on Application and System Modeling (ICCASM 2010), Taiyuan, China, 22–24 October 2010; pp. 332–334. 9. Makris, G.; Antoniou, I. Cryptography with Chaos. In Proceedings of the 5th Chaotic Modeling and Simulation International Conference, Athens, Greece, 12–15 June 2012; pp. 309–318. 10. Guastello, S.J. Chaos, Catastrophe, and Human Affairs: Applications of Nonlinear Dynamics to Work, Organizations, and Social Evolution; Press: New York, NY, USA, 2013. 11. Richards, D. From Individuals to Groups: The Aggregation of Votes and Chaotic Dynamics. In Chaos Theory in the Social Sciences; Kiel, L.D., Elliott, E., Eds.; University of Michigan Press: Ann Arbor, MI, USA, 1996; pp. 89–116. Systems 2016, 4, 37 17 of 18

12. Batty, M.; Longley, P. Fractal Cities: A Geometry of Form and Function; Academic Press: London, UK, 1994. 13. Batty, M.; Xie, Y. Self-Organized Criticality and Urban Development. Discret. Dyn. Nat. Soc. 1999, 3, 109–124. [CrossRef] 14. Benguigui, L.; Czamanski, D.; Marinov, M.; Portugali, Y. When and Where is a City Fractal? Environ. Plan. B 2000, 27, 507–519. [CrossRef] 15. Shen, G. Fractal Dimension and Fractal Growth of Urbanized Areas. Int. J. Geogr. Inf. Sci. 2002, 16, 419–437. [CrossRef] 16. Chen, Y.; Zhou, Y. Scaling Laws and Indications of Self-Organized Criticality in Urban Systems. Chaos Solitons Fractals 2008, 35, 85–98. [CrossRef] 17. Chen, W.C. Nonlinear Dynamics and Chaos in a Fractional-Order Financial System. Chaos Solitons Fractals 2008, 36, 1305–314. [CrossRef] 18. Guégan, D. Chaos in Economics and . Annu. Rev. Control 2009, 33, 89–93. [CrossRef] 19. Puu, T. Attractors, Bifurcations, & Chaos: Nonlinear Phenomena in Economics, 2nd ed.; Springer Science & Business Media: New York, NY, USA, 2013. 20. Rosser, J.B. Chaos Theory and Rationality in Economics. In Chaos Theory in the Social Sciences; Kiel, L.D., Elliott, E., Eds.; University of Michigan Press: Ann Arbor, MI, USA, 1996; pp. 199–213. 21. Oxley, L.; George, D.A.R. Economics on the : Some Pitfalls of Linearizing Complex Systems. Environ. Model. Softw. 2007, 22, 580–589. [CrossRef] 22. Hamouche, M.B. Can Chaos Theory Explain Complexity In Urban Fabric? Applications in Traditional Muslim Settlements. Nexus Netw. J. 2009, 11, 217–242. [CrossRef] 23. Ostwald, M.J. The of Architecture: Calibrating the Box-Counting Method Using Scaling Coefficient and Grid Disposition Variables. Environ. Plan B 2013, 40, 644–663. [CrossRef] 24. Cartwright, T.J. Planning and Chaos Theory. J. Am. Plan. Assoc. 1991, 57, 44–56. [CrossRef] 25. Innes, J.E.; Booher, D.E. Planning with Complexity; Routledge: London, UK, 2010. 26. Batty, M.; Marshall, S. The Origins of Complexity Theory in Cities and Planning. In Complexity of Cities Have Come of Age; Portugali, J., Meyer, H., Stolk, E., Tan, E., Eds.; Springer: Berlin, Germany, 2012; pp. 21–45. 27. Batty, M. The New Science of Cities; MIT Press: Cambridge, MA, USA, 2013. 28. Alpigini, J.J. Dynamical System Visualization and Analysis via Performance Maps. Inf. Vis. 2004, 3, 271–287. [CrossRef] 29. Layek, G.C. An Introduction to Dynamical Systems and Chaos; Springer: New Delhi, India, 2015. 30. Chen, C. Information Visualization: Beyond the Horizon, 2nd ed.; Springer: London, UK, 2006. 31. Lorenz, E.N. Deterministic Nonperiodic . J. Atmos. Sci. 1963, 20, 130–141. [CrossRef] 32. May, R.M. Simple Mathematical Models with Very Complicated Dynamics. Nature 1976, 261, 459–467. [CrossRef][PubMed] 33. Packard, N.H.; Crutchfield, J.P.; Farmer, J.D.; Shaw, R.S. Geometry from a Time Series. Phys. Rev. Lett. 1980, 45, 712–716. [CrossRef] 34. Bradley,E. Time Series Analysis. In Intelligent Data Analysis: An Introduction, 2nd ed.; Hand, D., Berthold, M., Eds.; Springer: Berlin, Germany, 2003. 35. Bradley, E.; Kantz, H. Nonlinear Time-Series Analysis Revisited. Chaos 2015, 25, 097610. [CrossRef][PubMed] 36. Chettiparamb, A. Metaphors in Complexity Theory and Planning. Plan Theory 2006, 5, 71–91. [CrossRef] 37. Tomida, A.G. Matlab Toolbox and GUI for Analyzing One-Dimensional Chaotic Maps. In Proceedings of the 2008 International Conference on Computational Sciences and Its Applications, Perugia, , 30 June–3 July 2008; pp. 321–330. 38. Stewart, I. The Lorenz Attractor Exists. Nature 2000, 406, 948–949. [CrossRef][PubMed] 39. Danforth, C.M. Chaos in an Atmosphere Hanging on a Wall. Available online: http://mpe2013.org/2013/ 03/17/chaos-in-an-atmosphere-hanging-on-a-wall/ (accessed on 1 September 2016). 40. Gleick, J. Chaos: Making a New Science; Cardinal: Indianapolis, IN, USA, 1991. 41. May, R.M. Biological Populations with Nonoverlapping Generations: Stable Points, Stable Cycles, and Chaos. Science 1974, 186, 645–647. [CrossRef][PubMed] 42. Li, W.; Wang, K.; Su, H. Optimal Harvesting Policy for Stochastic Logistic . Appl. Math. Comput. 2011, 218, 157–162. [CrossRef] Systems 2016, 4, 37 18 of 18

43. Pastijn, H. Chaotic Growth with the Logistic Model of P.F. Verhulst. In The Logistic Map and the Route to Chaos; Ausloos, M., Dirickx, M., Eds.; Springer: Berlin, Germany, 2006; pp. 3–11. 44. Strogatz, S.H. Nonlinear Dynamics and Chaos, 2nd ed.; Westview Press: Boulder, CO, USA, 2014. 45. Ruelle, D.; Takens, F. On the Nature of . Commun. Math. Phys. 1971, 20, 167–192. [CrossRef] 46. Shilnikov, L. Bifurcations and Strange Attractors. Proc. Int. Congr. Math. 2002, 3, 349–372. 47. Grebogi, C.; Ott, E.; Yorke, J.A. Chaos, Strange Attractors, and Fractal Basin Boundaries in Nonlinear Dynamics. Science 1987, 238, 632–638. [CrossRef][PubMed] 48. Gershenson, C. Introduction to Chaos in Deterministic Systems. Available online: http://arxiv.org/abs/ nlin/0308023 (accessed on 10 November 2016). 49. Wu, G.C.; Baleanu, D. Discrete Fractional Logistic Map and Its Chaos. Nonlinear Dyn. 2014, 75, 283–287. [CrossRef] 50. Li, T.Y.; Yorke, J.A. Period Three Implies Chaos. Am. Math. Mon. 1975, 82, 985–992. [CrossRef] 51. Feigenbaum, M.J. Quantitative Universality for a Class of Nonlinear Transformations. J. Stat. Phys. 1978, 19, 25–52. [CrossRef] 52. Feigenbaum, M.J. Universal Behavior in Nonlinear Systems. Phys. Nonlinear Phenom. 1983, 7, 16–39. [CrossRef] 53. Hénon, M. A Two-Dimensional Mapping with a Strange Attractor. Commun. Math. Phys. 1976, 50, 69–77. [CrossRef] 54. Farmer, J.D.; Ott, E.; Yorke, J.A. The Dimension of Chaotic Attractors. Phys. Nonlinear Phenom. 1983, 7, 153–180. [CrossRef] 55. Grassberger, P.; Procaccia, I. Characterization of Strange Attractors. Phys. Rev. Lett. 1983, 50, 346–349. [CrossRef] 56. Mandelbrot, B.B. How Long Is the Coast of Britain? Science 1967, 156, 636–638. [CrossRef][PubMed] 57. Mandelbrot, B.B. The Fractal Geometry of Nature; Macmillan: New York, NY, USA, 1983. 58. Mandelbrot, B.B. Multifractals and 1/f Noise; Springer: New York, NY, USA, 1999. 59. Huikuri, H.V.; Mäkikallio, T.H.; Peng, C.K.; Goldberger, A.L.; Hintze, U.; Møller, M. Diamond Study Group. Fractal Correlation Properties of RR Dynamics and Mortality in Patients with Depressed Left Ventricular Function after an Acute Myocardial Infarction. Circulation 2000, 101, 47–53. [CrossRef][PubMed] 60. Takens, F. Detecting Strange Attractors in Turbulence. In Dynamical Systems and Turbulence; Rand, D., Young, L.S., Eds.; Springer: Berlin, Germany, 1981; pp. 366–381. 61. Theiler, J. Estimating Fractal Dimension. J. Opt. Soc. Am. A 1990, 7, 1055–1073. [CrossRef] 62. Kekre, H.B.; Sarode, T.; Halarnkar, P.N. A Study of Period Doubling in Logistic Map for Shift Parameter. Int. J. Eng. Trends Technol. 2014, 13, 281–286. [CrossRef] 63. Clarke, K.C. Computation of the Fractal Dimension of Topographic Surfaces Using the Triangular Prism Area Method. Comput. Geosci. 1986, 12, 713–722. [CrossRef] 64. Sander, E.; Yorke, J.A. The Many Facets of Chaos. Int. J. Bifurc. Chaos 2015, 25, 1530011. [CrossRef] 65. Sprott, J.C.; Xiong, A. Classifying and Quantifying Basins of Attraction. Chaos 2015, 25, 083101. [CrossRef] [PubMed] 66. Brown, T.A. Measuring Chaos Using the Lyapunov Exponent. In Chaos Theory in the Social Sciences; Kiel, L.D., Elliott, E., Eds.; University of Michigan Press: Ann Arbor, MI, USA, 1996; pp. 53–66. 67. Wolf, A.; Swift, J.B.; Swinney, H.L.; Vastano, J.A. Determining Lyapunov Exponents from a Time Series. Phys. Nonlinear Phenom. 1985, 16, 285–317. [CrossRef] 68. Dingwell, J.B. Lyapunov Exponents. In Wiley Encyclopedia of Biomedical Engineering; Akay, M., Ed.; John Wiley & Sons: Hoboken, NJ, USA, 2006. 69. Chan, K.S.; Tong, H. Chaos: A Statistical Perspective; Springer Science & Business Media: New York, NY, USA, 2013. 70. Hunt, B.R.; Ott, E. Defining Chaos. Chaos 2015, 25, 097618. [CrossRef][PubMed] 71. Kantz, H.; Radons, G.; Yang, H. The Problem of Spurious Lyapunov Exponents in Time Series Analysis and Its Solution by Covariant Lyapunov Vectors. J. Phys. Math. Theor. 2013, 46, 254009. [CrossRef]

© 2016 by the author; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC-BY) license (http://creativecommons.org/licenses/by/4.0/).