1999, 2015, 2018 CHAOS, BIFURCATIONS, AND THEIR In the discrete-time setting, a nonlinear dynamical sys- CONTROL tem is described by either a difference equation, ( ) 𝐱 𝐟 𝐱 𝐩 ,𝑘, , 𝑘+1 = 𝑘 𝑘; =0 1 … (2) 1. NONLINEAR DYNAMICS or a map, ( ) Unlike linear systems, many nonlinear dynamical systems 𝐹 𝐱 → 𝑔 𝐱 𝐩 ,𝑘, , do not show orderly, regular, and long-term predictable ∶ 𝑘 𝑘 𝑘; =0 1 … (3) responses to simple inputs. Instead, they display complex, where notation is similarly defined. Repeatedly iterat- random-like, seemingly irregular, yet well-defined output ing the discrete map 𝐹 backward yields behaviors. Such dynamical phenomenon is known as chaos ( ) ( ) ( ) today. 𝐱 𝐹 𝐱 𝐹 𝐹 𝑥 ⋯ 𝐹 𝑘 𝐱 𝑘 = 𝑘−1 = ( 𝑘−2) = = 0 The term chaos, originated from the Greek word 𝜒𝛼𝑜𝜁, was designated “the primeval emptiness of the universe where the map can also be replaced by a function 𝐟,ifthe before things came into being of the abyss of Tartarus, the system is given via a difference equation, leading to underworld. … In the later cosmologies, Chaos generally ( ) ( ) 𝐱 𝐟◦⋯◦𝐟 𝐱 𝐟 𝑘 𝐱 designated the original state of things, however conceived. 𝑘 = ⏟⏟⏟ 0 = 0 The modern meaning of the word is derived from Ovid, 𝑘 times who saw Chaos as the original disordered and formless ◦ mass, from which the maker of the Cosmos produced the where “ ” denotes composition operation of functions or ordered universe” (1). There also is an interpretation of maps. chaos in ancient Chinese literature, which refers to the Dynamical system (eq. 1)issaidtobenonautonomous 𝑡 spirit existing in the center of the universe (2).Inmod- when the time variable appears separately in the system 𝐟 ern scientific terminology, chaos has a fairly precise but function (e.g. a system with an external time-varying rather complicated definition by means of the dynamics of force input); otherwise, it is said to be autonomous and is a generally nonlinear system. For example, in theoretical expressed as ( ) physics, “chaos is a type of moderated randomness that, 𝐱̇ = 𝐟 𝐱; 𝐩 ,𝑡∈(𝑡 , ∞) (4) unlike true randomness, contains complex patterns that 0 are mostly unknown” (3). Bifurcation, as a twin of chaos, is another prominent 1.2. Classification of Equilibria phenomenon of nonlinear dynamical systems: Quantita- For illustration, consider a general two-dimensional au- tive change of system parameters leads to qualitative tonomous system: change of system properties such as the number and the stability of system equilibria. Typical bifurcations include ⎧ 𝑥̇ = 𝑓(𝑥, 𝑦) ⎪ transcritical, saddle-node, pitchfork, hysteresis, and Hopf ⎨ (5) bifurcations. In particular, period-doubling bifurcation is ⎪ ⎩ 𝑦̇ = 𝑔(𝑥, 𝑦) a route to chaos. To introduce the concepts of chaos and bifurcations as well as their control (4,5,6,7,8,9),somepre- 𝑥 ,𝑦 𝑓 𝑔 with given initial conditions ( 0 0),where and are liminaries on nonlinear dynamical systems are in order. two smooth nonlinear functions that together describe the vector field of the system. 1.1. Nonlinear Dynamical Systems The path traveled by a solution of system (eq. 5), start- ing from the initial state 𝑥 ,𝑦 ,isasolution trajectory,or A nonlinear system refers to a set of nonlinear equations, ( 0 0) 𝜑 𝑥 ,𝑦 which can be algebraic, difference, differential, integral, orbit, of the system, and is sometimes denoted 𝑡( 0 0). functional, or abstract operator equations, even a certain For autonomous systems, two different orbits will never combination of these. A nonlinear system is used to de- cross each other (i.e., never intersect) on the x–y plane. 𝑥 𝑦 scribe a physical device or process that otherwise cannot This – coordinate plane is called the (generalized) phase be well defined by a set of linear equations of any kind. plane (phase space, in the higher dimensional case). The Dynamical system is used as a synonym of a mathematical orbit family of a general autonomous system, correspond- or physical system, in which the output behavior evolves ing to all possible initial conditions, is called solution flow with time and/or with other varying system parameters in the phase space. (10). Equilibria,orfixed points,ofsystem(eq.5), if they exist, In general, a continuous-time dynamical system is de- are the solutions of two homogeneous equations: scribed by a differential equation, 𝑓(𝑥, 𝑦)=0 and 𝑔(𝑥, 𝑦)=0 ( ) 𝐱̇ 𝐟 𝐱,𝑡 𝐩 𝑡 𝑡 , 𝑥,̄ 𝑦 ̄ = ; ∈[0 ∞) (1) An equilibrium is denoted ( ).Itisstable if all the nearby orbits of the system, starting from any initial con- where 𝐱 = 𝐱(𝑡) is the state of the system, 𝐩 is a vector of ditions, approach it; it is unstable, if the nearby orbits are variable system parameters,and𝑓 and 𝑔 are continuous moving away from it. Equilibria can be classified, accord- or smooth (differentiable) nonlinear functions of compara- ing to their stabilities, as stable or unstable node or focus, ble dimensions, which have an explicit formulation for a and saddle point or center, as summarized in Figure 1.The 𝜆 specified physical system in interest. type of equilibria is determined by the eigenvalues, 1,2,

J. Webster (ed.), Wiley Encyclopedia of Electrical and Electronics Engineering.Copyright© 2018 John Wiley & Sons, Inc. DOI: 10.1002/047134608X.W1007.pub2 2 Chaos, Bifurcations, and Their Control

y yy

xx x

Stable node Unstable node Stable focus

yy y

x x x

Unstable focus Saddle Center

Figure 1. Classification of two-dimensional equilibria: stabilities are determined by Jacobian eigenvalues.

[ ] 𝑡 → 𝑓𝑥 𝑓𝑦 solution, as ∞, is called the steady state of the solution, of the system Jacobian, 𝐽 ∶= ,with𝑓 ∶= 𝜕𝑓∕𝜕𝑥, 𝑔 𝑔 𝑥 while the solution trajectory between its initial state and 𝑥 𝑦 the steady state is the transient state. 𝑓𝑦 ∶= 𝜕𝑓∕𝜕𝑦, and so on, all evaluated at (𝑥,̄ 𝑦 ̄).Ifthetwo For a given dynamical system, a point 𝐱𝜔 in the state Jacobian eigenvalues have real parts ℜ{𝜆 } ≠ 0, the equi- 1,2 space is an 𝜔-limit point of the system state orbit 𝐱(𝑡) if, for librium (𝑥,̄ 𝑦 ̄), at which the linearization was taken, is said every open neighborhood 𝑈 of 𝑥 , the trajectory of 𝐱(𝑡) will to be hyperbolic. 𝜔 enter 𝑈 at a (large enough) value of 𝑡. Consequently, 𝐱(𝑡) will repeatedly enter 𝑈 infinitely many times, as 𝑡 → ∞. Theorem 1 (Grobman–Hartman) If (𝑥,̄ 𝑦 ̄) is a hyper- The set of all such 𝜔-limit points of 𝐱(𝑡) is called the 𝜔- bolic equilibrium of the nonlinear dynamical system (eq. limit set of 𝐱(𝑡), and is denoted Ω𝑥.An𝜔-limit set of 𝐱(𝑡) 5), then the dynamical behavior of the nonlinear system is is attracting, if there exists an open neighborhood 𝑉 of Ω𝑥 qualitatively the same as (topologically equivalent to) that [ ] [ ] such that whenever system orbit enters 𝑉 , it will approach 𝑥̇ 𝑥 𝑡 → of its linearized system, = 𝐽 , in a neighborhood Ω𝑥 as ∞.Thebasin of attraction of an attracting point 𝑦̇ 𝑦 is the union of all such open neighborhoods. An 𝜔-limit set of the equilibrium (𝑥,̄ 𝑦 ̄). is repelling, if the system orbits always move away from it. 𝐹 𝐱 𝜔 This theorem guarantees that for the hyperbolic case, For a given map and a given{ initial} state 0,an -limit one can study the linearized system instead of the original 𝐹 𝑘 𝑥 𝑘 → 𝜔 set is obtained from the orbit ( 0) as ∞.This - nonlinear system, with regard to the local dynamical be- limit set Ω is an invariant set of the map, in the sense havior of the system within a (small) neighborhood of the 𝑥 that 𝐹 (Ω ) ⊆ Ω . Thus, 𝜔-limit sets include equilibria and equilibrium (𝑥,̄ 𝑦 ̄). In general, “dynamical behavior” refers 𝑥 𝑥 periodic orbits. to such nonlinear phenomena as stabilities and bifurca- An attractor is an 𝜔-limit set having the property that tions, chaos and attractors, equilibria and limit cycles, all orbits nearby have it as their 𝜔-limit sets. Thus, a col- and so on. In other words, there exist some homeomorphic lection of isolated attracting points is not an attractor. maps that transform the orbits of the nonlinear system into orbits of its linearized system in a (small) neighbor- hood of the equilibrium. Here, a homeomorphic map (or a 1.4. Periodic Orbits and Limit Cycles homeomorphism) is a continuous map whose inverse ex- A solution orbit 𝐱(𝑡) of the nonlinear dynamical system ists and is also continuous. However, in the nonhyperbolic (eq. 1)isaperiodic solution if it satisfies 𝐱(𝑡 + 𝑡 )=𝐱(𝑡) case, the situation is much more complicated, where such p for some constant 𝑡 > 0. The minimum value of such 𝑡 is local topological equivalence does not hold in general. p p called the (fundamental) period of the periodic solution, 𝑡 while the solution is said to be p periodic. 1.3. Limit Sets and Attractors A limit cycle of a dynamical system is a periodic solution The most basic problem in studying the general nonlinear of the system that corresponds to a closed orbit on the dynamical system (eq. 1) is to understand and/or to analyze phase plane and possesses certain attracting (or repelling) the system solutions. The asymptotic behavior of a system properties. Figure 2 shows some typical limit cycles: (a) an Chaos, Bifurcations, and Their Control 3

(a) (b) (c)

(d) (e) (f)

Figure 2. Periodic orbits and limit cycles. inner limit cycle, (b) an outer limit cycle,(c)astable limit cycle,(d)anunstable limit cycle, and (e and f), saddle limit cycles.

1.5. Poincar´eMaps P (x) 𝑛 Assume that the general -dimensional nonlinear au- x 𝑡 tonomous system (eq. 4)hasa p periodic orbit, Γ,and Γ let 𝐱∗ be a point on the periodic orbit and Σ be an (𝑛 −1)- x* dimensional hyperplane transversal to Γ at 𝐱∗,asshown 𝑡 in Figure 3.SinceΓ is p periodic, the orbit starting from Σ 𝐱∗ 𝐱∗ 𝑡 will return to in time p. Any orbit starting from a 𝐱 𝑈 𝐱∗ point in a small neighborhood of on Σ will return Figure 3. Illustration of the Poincare´ map and cross section. and hit Γ at a point, denoted 𝑃 (𝐱), in the vicinity 𝑉 of 𝐱∗. Therefore, a map 𝑃 ∶ 𝑈 → 𝑉 can be uniquely defined by 𝐱∗ 𝐱∗ Σ, along with the solution flow of the autonomous system. Suppose that Σs( ) and Σu( ) are cross sections of the This map is called the Poincaremap´ associated with the stable and unstable manifolds of 𝜑𝑡(𝐱), respectively, which system and the cross section Σ. For different choices of the intersect at 𝐱∗. This intersection always includes one con- ∗ cross section Σ,Poincare´ maps are similarly defined. stant orbit, 𝜑𝑡(𝐱)=𝐱 . A nonconstant orbit lying in the in- Note that a Poincare´ map is only locally defined and is tersection is called a homoclinic orbit, as illustrated in 𝐱̄ ≠ 𝐱̄ a diffeomorphism, that is, a differentiable map that has Figure 4a. For two equilibria, 1 2, of either unstable, 𝐱̄ 𝐱̄ an inverse and the inverse is also differentiable. If a cross center, or saddle type, an orbit lying in Σs( 1)∩Σu( 2) or in 𝐱̄ 𝐱̄ section is suitably chosen, the orbit will repeatedly return Σu( 1)∩Σs( 2), is called a heteroclinic orbit. A heteroclinic and pass through the section. The Poincare´ map together orbit is depicted in Figure 4b, which tends to one equi- with the first return orbit is particularly important, which librium as 𝑡 → ∞ but converges to another equilibrium as is called the first return Poincaremap´ .Poincare´ maps can 𝑡 → −∞. also be defined for non-autonomous systems in a similar way where, however, each return map depends on the ini- tial time in a non-uniform fashion.

1.6. Homoclinic and Heteroclinic Orbits x* 𝐱∗ x1 Let be a hyperbolic equilibrium of a diffeomorphism x2 𝑃 ∶ 𝑅𝑛 → 𝑅𝑛, which can be of either unstable, center, or ∗ saddle type, 𝜑𝑡(𝐱) be a solution orbit passing through 𝐱 , and Ω𝑥∗ be the 𝜔-limit set of 𝜑𝑡(𝐱).Thestable manifold of 𝑀 𝑥∗ Ω𝑥∗ , denoted s, is the set of such points that satisfy (a) (b) 𝜑 𝐱∗ → 𝑡 → 𝑀 𝑡( ) Ω𝑥∗ as ∞;theunstable manifold of Ω𝑥∗ , u,is ∗ ∗ the set of such points x that satisfy 𝜑𝑡(𝑥 ) → Ω𝑥∗ as 𝑡 → −∞. Figure 4. Illustration of homoclinic and heteroclinic orbits 4 Chaos, Bifurcations, and Their Control

Γ ear system: 𝐱̇ 𝐟 𝐱,𝑡 , 𝐱 𝑡 𝑥 = ( ) ( 0)= 0 (6) and, by changing variables if necessary, assume that the origin, 𝐱 =0, is an equilibrium of the system. Lyapunov stability theory concerns with various types of stabilities of the zero equilibrium of system (eq. 6). Figure 5. Illustration of a Sil’inkov-type˘ homoclinic orbit. Stability in the Sense of Lyapunov. The equilibrium 𝐱̄ =0 Note that if a stable and an unstable manifold inter- of system (eq. 6)issaidtobestable in the sense of Lyapunov 𝐱 ≠ 𝐱∗ 𝜀> 𝑡 ≥ sect at a point, 0 , then they will do so at infinitely if, for any 0 and any initial time 0 0,thereexistsa 𝐱 ∞ 𝛿 𝛿 𝜀, 𝑡 > many points, denoted { 𝑘}𝑘=−∞, counted in both forward constant, = ( 0) 0, such that and backward directions, which contains 𝐱 . This sequence, 0 ||𝐱 𝑡 || <𝛿 ⟹ ||𝑥 𝑡 || <𝜀, 𝑡 ≥ 𝑡 ( 0) ( ) ∀ 0 (7) {𝐱𝑘},isahomoclinic orbit in which each 𝐱𝑘 is called a homo- clinic point. This special structure is called a homoclinic Here and throughout, || ⋅ || denotes the standard Euclidean structure, in which the two manifolds usually do not in- norm of a vector. tersect transversally. Hence, this structure is unstable in It should be emphasized that the constant 𝛿 in the above the sense that the connection can be destroyed by a very 𝜀 𝑡 generally depends on both and 0. It is particularly im- small perturbation. If they intersect transversally, how- portant to point out that, unlike autonomous systems, one ever, the transversal homoclinic point will imply infinitely 𝑡 cannot simply assume the initial time 0 =0for a nonau- many other homoclinic points. This eventually leads to tonomous system in a general situation. The stability is a picture of stretching and folding of the two manifolds. said to be uniform with respect to the initial time, if this Such complex stretching and folding of manifolds are key 𝛿 𝛿 𝜀 𝑡 constant, = ( ), is indeed independent of 0 over the en- to chaos. This generally implies the existence of a com- tire time interval (0, ∞). plicated Smale horseshoe map, which is supported by the following mathematical theory. Asymptotic Stability. In both theoretical studies and practical applications, the concept of asymptotic stability 𝑛 𝑛 Theorem 2 (Smale–Birkhoff) Let 𝑃 ∶ 𝑅 → 𝑅 be a dif- is of most importance. feomorphism with a hyperbolic equilibrium 𝐱∗. If the cross The equilibrium 𝐱̄ =0 of system (eq. 6)issaidtobe sections of the stable and unstable manifolds, Σ (𝐱∗) and 𝛿 𝛿 𝑡 s asymptotically stable, if there exists a constant, = ( 0) 𝐱∗ 𝐱∗ Σu( ), intersect transversally at a point other than ,then > 0, such that 𝑃 has a horseshoe map embedded within it. ||𝐱 𝑡 || <𝛿 ⟹ ||𝐱 𝑡 || → 𝑡 → ( 0) ( ) 0as ∞ (8) For three-dimensional autonomous systems, the case of This asymptotical stability is said to be uniform,iftheex- an equilibrium with one real eigenvalue 𝜆 and two complex 𝛿 𝑡 , isting constant is independent of 0 over [0 ∞), and is said conjugate eigenvalues 𝛼 ± 𝑗𝛽is especially interesting. For to be global, if the convergence (||𝐱|| → 0) is independent example, the case with 𝜆>0 and 𝛼<0 gives a Sil’nikov-˘ 𝑥 𝑡 of the initial point ( 0) over the entire domain on which type of homoclinic orbit, as illustrated in Figure 5. the system is defined (i.e., when 𝛿 =∞).

𝜑 Theorem 3 (Sil’nikov) Let 𝑡 be the solution flow of a Orbital Stability. The orbital stability differs from the three-dimensional autonomous system that has a Sil’nikov-˘ Lyapunov stabilities in that it concerns with the structural type homoclinic orbit Γ.If|𝛼| < |𝜆|,then𝜑𝑡 can be extremely stability of a system orbit under perturbation. slightly perturbed to 𝜑̃𝑡, such that 𝜑̃𝑡 has a homoclinic orbit Let 𝜑 (𝐱 ) be a 𝑡 periodic solution of the autonomous ̃ 𝑡 0 p Γ,nearΓ, of the same type, and the Poincare´ map defined system: by a cross section, transversal to Γ̃, has a countable set of 𝐱̇ 𝑡 𝐟 𝐱 , 𝐱 𝑡 𝑥 Smale horseshoes. ( )= ( ) ( 0)= 0 (9) 𝜑 𝐱 and let Γ be the closed orbit of 𝑡( 0) in the phase space, namely, 1.7. Stabilities of Systems and Orbits { } Γ= 𝑦 | 𝑦 = 𝜑𝑡(𝐱 ) , 0 ≤ 𝑡<𝑡 Stability theory plays a central role in both dynamical sys- 0 p tems and automatic control. Conceptually, there are differ- 𝜑 𝐱 The solution trajectory 𝑡( 0) is said to be orbitally stable ent types of stabilities, among which Lyapunov stabilities if, for any 𝜀>0,thereexitsa𝛿 = 𝛿(𝜀) > 0 such that for any and the orbital stability are essential for chaos and bifur- 𝐱 0 satisfying cations control. 𝑑 𝐱 , ||𝐱 𝑦|| <𝛿 ( 0 Γ) ∶= inf 0 − 𝑦∈Γ Lyapunov Stabilities. In the following discussion of Lya- the solution 𝜑 (𝐱 ) of the autonomous system satisfies punov stabilities for the general nonautonomous system 𝑡 0 (eq. 1), the parameters are not indicated explicitly for sim- 𝑑 𝜑 𝐱 , <𝜀, 𝑡 ≥ 𝑡 ( 𝑡( 0) Γ) ∀ 0 plicity. Thus, consider the general nonautonomous nonlin- Chaos, Bifurcations, and Their Control 5

𝑉 𝐱,𝑡 D 𝑡 , Lyapunov Stability Theorems. Two cornerstones in the exists a scalar-valued function ( ) defined on ×[0 ∞) Lyapunov stability theory for dynamical systems are the with three functions 𝛼(⋅),𝛽(⋅),𝛾(⋅)∈K, such that Lyapunov first (or indirect) method and the Lyapunov sec- ond (or direct) method. 𝑉 ,𝑡 The Lyapunov first method, known also as the Jacobian (i) (0 0)=0; 𝑉 𝐱,𝑡 > 𝐱 ≠ D 𝑡 ≥ 𝑡 or local linearization method, is applicable to autonomous (ii) ( ) 0,forall 0 in and all 0; 𝛼 ||𝐱|| ≤ 𝑉 𝐱,𝑡 ≤ 𝛽 ||𝐱|| 𝑡 ≥ 𝑡 systems. This method is based on the fact that the sta- (iii) ( ) ( ) ( ),forall 0; bility of an autonomous system, in a neighborhood of an 𝑉̇ 𝐱,𝑡 ≤ 𝛾 ||𝐱|| < 𝑡 ≥ 𝑡 (iv) ( ) − ( ) 0,forall 0. equilibrium of the system, is essentially the same as its linearized model operating at the same point, and under certain conditions local system stability and associated dy- In this theorem, the uniform stability is usually neces- namical behavior are qualitatively the same as (topologi- sary since the solution of a nonautonomous system may cally equivalent to) that of its linearized model (in some depend on the initial time, often sensitively. As a special sense, similar to the Grobman–Hartman theorem). The case for autonomous systems, the above theorem reduces Lyapunov first method provides a theoretical justification to the following version. for applying linear analysis and linear feedback controllers to nonlinear autonomous systems in the study of asymp- Theorem 6 (Lyapunov second method) (for totic stability and stabilization. continuous-time autonomous systems) The Lyapunov second method, on the other hand, orig- Let 𝐱̄ =0be an equilibrium for the autonomous system inated from the concept of energy decay (i.e., dissipation) (eq. 9). This zero equilibrium is globally (over the domain associated with a stable mechanical or electrical system, D ⊆𝑅𝑛 containing the origin) and asymptotically stable, if is applicable to both autonomous and nonautonomous sys- there exists a scalar-valued function 𝑉 (𝐱) defined on D such tems. Hence, the second method is more powerful, also that more useful, for global stability analysis of dynamical sys- tems. For the general autonomous system (eq. 9), under the (i) 𝑉 (0) = 0; 𝐟 D → 𝑅𝑛 assumption that ∶ is continuously differentiable (ii) 𝑉 (𝐱) > 0,forall𝐱 ≠ 0 in D; in a neighborhood, D, of the origin in 𝑅𝑛, the following (iv) 𝑉̇ (𝐱) < 0,forall𝐱 ≠ 0 in D. theorem of stability for the Lyapunov first method is con- venient to use. In the above two theorems, the function 𝑉 is called a Theorem 4 (Lyapunov first method) (for continuous- Lyapunov function, which is generally not unique for a time autonomous systems) given system. In equation 9, let Similar stability theorems can be established for | 𝜕𝐟 | discrete-time systems (via properly replacing derivatives 𝐽 | 0 = 𝜕𝑥 | with differences). |𝑥=x=0̄ To this end, it is important to remark that the Lyapunov be the Jacobian of the system at the equilibrium 𝐱̄ =0.Then theorems only offer sufficient conditions for determining the asymptotic stability. Yet, the power of the Lyapunov (i) 𝐱̄ =0is asymptotically stable if all the eigenvalues of second method lies in its generality: It works for all kinds 𝐽 of dynamical systems (linear and nonlinear, continuous- 0 have negative real parts; 𝐱̄ 𝐽 time and discrete-time, autonomous and nonautonomous, (ii) =0is unstable if one of the eigenvalues of 0 have positive real part. time-delayed, functional, etc.), and it does not require any knowledge of the solution formula of the underlying sys- Note that the region of asymptotic stability given in this tem. In a particular application, the key is to construct theorem is local. It is important to emphasize that this a working Lyapunov function for the system, which can theorem cannot be applied to non-autonomous systems in be technically difficult if the system is higher dimensional general, not even locally. and complicated. For a general nonautonomous system (eq. 6), the follow- ing criterion can be used. 2. CHAOS Theorem 5 (Lyapunov second method) (for Nonlinear systems have various complex behaviors that continuous-time nonautonomous systems) would never be anticipated in the (finite-dimensional) lin- Let 𝐱̄ =0be an equilibrium of the nonautonomous sys- tem (6).Let ear systems. Chaos is a typical one of this kind. { } In the development of chaos theory, the first evidence K 𝑔 𝑡 𝑔 𝑡 ,𝑔𝑡 𝑡 , = ( )∶ ( 0)=0 ( ) is continuous and nondecreasing on [ 0 ∞) of physical chaos is Edward Lorenz’s discovery in 1963 (11). The first underlying mechanism within chaos was The zero equilibrium of the system is globally (over the observed by Mitchell Feigenbaum, who in 1976 found that domain D ⊆𝑅𝑛 containing the origin), uniformly (with re- “when an ordered system begins to break down into chaos, spect to the initial time), and asymptotically stable, if there a consistent pattern of rate doubling occurs” (3). 6 Chaos, Bifurcations, and Their Control

2.1. What Is Chaos Positive Leading Lyapunov Exponents. The sensitive dependence on initial conditions for chaotic systems There is no unified, universally accepted, rigorous defini- possesses exponential growth rate in general. This expo- tion of chaos in the current scientific literature. nential growth is related to the existence of at least one The term chaos was first introduced into mathematics positive Lyapunov exponent, the leading (largest) one. by Li and Yorke (12). Since then, there have been several Among all main characteristics of chaos, positive leading different but closely related proposals for defining chaos, Lyapunov exponent is perhaps the most convenient one to among which Devaney’s definition is perhaps the most pop- verify in engineering applications. ular one (13). It states that a map 𝐹 ∶ 𝑆 → 𝑆,where𝑆 is a To introduce this concept, consider an 𝑛-dimensional set, is said to be chaotic,if discrete-time dynamical system described by a smooth map 𝐟,orthePoincare´ map of a continuous-time system. (1) 𝐹 is transitive on 𝑆: For any pair of nonempty open ∞ The 𝑖th Lyapunov exponent of the orbit {𝐱𝑘} , generated sets 𝑈 and 𝑉 in 𝑆, there is an integer 𝑘>0 such that 𝑘=0 𝑘 by the iterations of the map starting from any given initial 𝐹 (𝑈)∩𝑉 ≠ 𝜙. 𝐱 state 0, is defined to be (2) 𝐹 has sensitive dependence on initial conditions: ( ) 𝛿> 𝐹 1 | | There is a real number 0, depending only on 𝜆𝑖(𝐱 ) = lim ln | 𝜇𝑖 𝐽𝑘(𝐱𝑘) ⋯ 𝐽 (𝐱 ) | ,𝑖=1, ⋯ ,𝑛 (10) 0 𝑘→ 𝑘 | 0 0 | and 𝑆, such that in every nonempty open subset of 𝑆 ∞ ′ there is a pair of points that, through iterations under where 𝐽𝑖 = 𝐟 (𝐱𝑖) is the Jacobian and 𝜇𝑖(⋅) denotes the 𝑖th 𝐹 , are eventually separated by a distance of at least eigenvalue of a matrix (numbered in the decreasing order 𝛿. of magnitudes). (3) The periodic points of 𝐹 are dense in 𝑆. Lyapunov exponents are generalization of eigenvalues of linear systems, which provide a measure for the mean convergence or divergence rate of neighboring orbits of a Another definition requires the set 𝑆 be compact, but dynamical system. For an 𝑛-dimensional continuous-time drops condition (3) from the above. There even is a view system, depending on the direction (but not the position) that only the transitivity property is essential in this defi- of the initial state vector, its 𝑛 Lyapunov exponents, 𝜆 ≥ nition, while another view is that the sensitivity to initial 1 ⋯ ≥ 𝜆 , describe different types of attractors. For example, conditions is so. 𝑛 for nonchaotic attractors (e.g., limit sets), Although a precise and rigorous mathematical defini- tion of chaos does not seem to be available any time soon, 𝜆𝑖 < 0,𝑖=1, … ,𝑛 ⟹ stable equilibrium some fundamental features of chaos are well understood, 𝜆 =0,𝜆𝑖 < 0,𝑖=2, … ,𝑛 ⟹ stable limit cycle which can be used to signify or identify chaos in most cases, 1 𝜆 𝜆 ,𝜆 < ,𝑖 , ,𝑛 ⟹ especially from an application perspective. 1 = 2 =0 𝑖 0 =3 … stable two − torus 𝜆 ⋯ 𝜆 ,𝜆 < ,𝑖 𝑚 , ,𝑛 ⟹ 𝑚 1 = = 𝑚 =0 𝑖 0 = +1 … stable − torus 2.2. Features of Chaos Here, a two-torus is a bagel-shaped surface in the three- A hallmark of chaos is its fundamental property of ex- dimensional space, and an 𝑚-torus is its geometrical gen- treme sensitivity to initial conditions. Other features of eralization in the (𝑚 +1)-dimensional space. chaos include the embedding of a dense set of unsta- Note that for a three-dimensional continuous-time dy- ble periodic orbits in its strange attractor, positive lead- namical system, the only possibility for chaos to exist is ing (maximal) Lyapunov exponent, finite Kolmogorov– that the three-system Lyapunov exponents satisfy Sinai entropy or positive topological entropy, continuous (+, 0, −) ∶= (𝜆 > 0,𝜆 =0,𝜆 < 0) and 𝜆 < −𝜆 power spectrum, positive algorithmic complexity, ergodic- 1 2 3 3 1 ity and mixing (Arnold’s cat map), Smale horseshoe map, Intuitively, this means that the system orbit in the phase a statistical-oriented definition of Shil’nikov,˘ and some un- space expands in one direction but shrinks in another di- usual (strange) limiting properties (5). rection, thereby yielding complex (stretching and folding) dynamical dynamics within a bounded region. Discrete- Extreme Sensitivity to Initial Conditions. The first signa- time case is different, however. A prominent example is ture of chaos is its extreme sensitivity to initial conditions, the one-dimensional logistic map, discussed in more de- associated with its bounded (or compact) region of orbital tail below, which is chaotic but has (the only) one posi- patterns. It implies that two sets of slightly different initial tive Lyapunov exponent. For four-dimensional continuous- conditions can lead to two dramatically different asymp- time systems, there are only three possibilities for chaos totic states of the system orbit after some time. This is the to emerge: so-called butterfly effect, which is a metaphor saying that , , , 𝜆 > ,𝜆 ,𝜆 ≤ 𝜆 < a single flap of a butterfly’s wings in China today may alter (1) (+ 0 − −): 1 0 2 =0 4 3 0;leadingtochaos. , , , 𝜆 ≥ 𝜆 > ,𝜆 ,𝜆 < the initial conditions of the global weather dynamical sys- (2) (+ + 0 −): 1 2 0 3 =0 4 0; leading to “hy- tem, thereby leading to a significantly different weather perchaos.” , , , 𝜆 > ,𝜆 𝜆 ,𝜆 < pattern in Argentina at a future time. In other words, for (3) (+ 0 0 −): 1 0 2 = 3 =0 4 0;leadingtoa a dynamical system to be chaotic, it must have a (large) “chaotic two-torus.” set of such “unstable” initial conditions that cause orbital divergence within a bounded region in the phase space of Simple Zero of the Melnikov Function. The Melnikov the- the system. ory for chaotic dynamics deals with the saddle points of Chaos, Bifurcations, and Their Control 7

Poincare´ maps of continuous solution flows in the phase space. The Melnikov function provides a measure of the distance between the stable and the unstable manifolds near a saddle point. To introduce the Melnikov function, consider a nonlin- ear oscillator described by a Hamiltonian system:

⎧ 𝑝̇ 𝜕𝐻 𝜀𝑓 ⎪ =− 𝜕𝑞 + 1 ⎨ 𝑞̇ 𝜕𝐻 𝜀𝑓 ⎪ = + 2 ⎩ 𝜕𝑝

𝐟 𝑓 𝑝, 𝑞, 𝑡 ,𝑓 𝑝, 𝑞, 𝑡 ⊤ where ∶= [ 1( ) 2( )] has state variables 𝑝 𝑡 ,𝑞 𝑡 𝜀> 𝐻 𝐻 𝑝, 𝑞 𝐸 𝐸 ( ( ) ( )), 0 is small, and = ( )= K + P is the Hamilton function for the undamped, unforced (when 𝜀 𝐸 𝐸 =0) oscillator, in which K and P are the kinetic and potential energies of the system, respectively. Suppose that the unperturbed (unforced and un- damped) oscillator has a saddle-node equilibrium (e.g., the Figure 6. A typical example of strange attractor: the double 𝐟 𝑡 scroll of Chua’s circuit response. undamped pendulum), and that is p periodic with phase frequency 𝜔>0. When the forced motion is described in the three-dimensional phase space (𝑝, 𝑞,) 𝜔𝑡 , the Melnikov 𝑑 ,𝑑 , <𝑑 <𝜀 𝑘 function is defined by of radii 1 2 …, satisfying∑ 0 𝑘 for all . For a con- 𝜌> ∞ 𝑑𝜌 stant 0,consider 𝑘 𝑘 for different coverings, and [ ] ∑ 𝜌 =1 ∞ let inf 𝑑 be the smallest value of the sum over all such 𝐹 𝑑∗ 𝐻 𝑝,̄ 𝑞 ̄ 𝐟 𝑑𝑡 C 𝑘 ( )=∫ ∇ ( ) ∗ (11) 𝜀 → 𝜌<ℎ −∞ coverings. In the limit 0, this value will diverge if but tends to zero if 𝜌>ℎfor some constant ℎ (need not be 𝑝,̄ 𝑞 ̄ where ( ) is the solution of the unperturbed homo- an integer). This value ℎ is called the Hausdorff dimen- clinic orbit starting from the saddle point of the orig- sion of the set 𝑆. If the above limit exists for 𝜌 = ℎ,then inal Hamiltonian system, 𝐟 𝐟 𝑝,̄ 𝑞,𝜔𝑡 ̄ 𝑑∗ ,and 𝐻 ∗ = ( + ) ∇ = the Hausdorff measure of the set 𝑆 is defined to be [𝜕𝐻∕𝜕𝑝, 𝜕𝐻∕𝜕𝑞]. The variable 𝑑∗ gives a measure of the distance between the stable and unstable manifolds near ∑∞ 𝜇 𝑆 𝑑𝜌 the saddle-node equilibrium. h( ) ∶= lim inf 𝑘 𝜀→0 C The Melnikov theory states that chaos is possible if 𝑘=1 the two manifolds intersect, which corresponds to that the Melnikov function has a simple zero: 𝐹 (𝑑∗)=0at a single There is an interesting conjecture that the Lyapunov 𝜆 point, 𝑑∗. exponents { 𝑘} (indicating the dynamics) and the Haus- dorff dimension ℎ (indicating the geometry) of a strange attractor have the relation Strange Attractors. Attractors are typical in nonlinear systems. The most interesting attractors, very closely re- ∑𝑘 ℎ 𝑘 1 𝜆 lated to chaos, are strange attractors. A strange attrac- = + | | 𝑖 |𝜆 | 𝑖 tor is a bounded attractor, which exhibits sensitive depen- | 𝑘+1| =1 dence on initial conditions but cannot be decomposed into ∑ two invariant subsets contained in disjoint open sets. Most 𝑘 𝑘 𝜆 > where is the largest integer that satisfies 𝑖=1 𝑖 0.This chaotic systems have strange attractors; however, not all formula has been mathematically proved for large fami- strange attractors are associated with chaos. lies of three-dimensional continuous-time autonomous sys- Generally speaking, a strange attractor is not any of tems and of two-dimensional discrete-time systems. the stable equilibria or limit cycles, but rather consists of A notion that is closely related to Hausdorff dimension some limit sets associated with some kind of Cantor sets. In is fractal, which was first conned and defined by Mandel- other words, it has a “strange” and complicated structure brot in the 1970s, to be a set with Hausdorff dimension that may possess a noninteger dimension (fractals), and strictly greater than its topological dimension, where the have some special properties of a Cantor set. For instance, latter is always an integer. Roughly, a fractal is a set that a chaotic orbit usually appears to be “strange” in that the has a fractional Hausdorff dimension and possesses cer- orbit moves toward a certain point (or limit set) for some tain self-similarities. An illustration of the concept of self- time but then moves away from it for some other time, similarity and fractal is given in Figure 13. although the orbit repeats this process infinitely and many There is a strong connection between fractal and chaos. time it never settles anywhere. Figure 6 shows a typical Chaotic orbits often possess fractal structures in the phase Chua attractor that has such strange behavior. space. For conservative systems, the Kolmogorov–Arnold– Moser (KAM) theorem implies that the boundary between Fractals. An important concept that is related to Lya- the region of regular motion and that of chaos is fractal. punov exponent is the Hausdorff dimension. Let 𝑆 be a set However, some chaotic systems have nonfractal limit sets, in 𝑅𝑛 and 𝐶 be a covering of 𝑆 by countably many balls and some fractal structures are not chaotic. 8 Chaos, Bifurcations, and Their Control

Finite Kolmogorov–Sinai Entropy. Another important rotor with varying mass, Lorenz model, and Rossler¨ sys- feature of chaos and strange attractors is quantified by tem, electrical and electronics systems (e.g., Chua’s circuit the Kolmogorov–Sinai (KS) entropy, a concept based on and electric power systems), digital filters, celestial me- Shannon’s information theory. chanics (the three-body problem), fluid dynamics, lasers, The classical statistical entropy is defined by plasmas, solid states, quantum mechanics, nonlinear op- ∑ tics, chemical reactions, neural networks, fuzzy systems, 𝐸 𝑐 𝑃 𝑃 =− 𝑘 ln ( 𝑘) economic and financial systems, biological systems (heart, 𝑘 brain, and population models), and various Hamiltonian where 𝑐 is a constant and 𝑃𝑘 is the probability of the sys- systems (5). tem state being at the stage 𝑘 of the process. According to Chaos also exists in many engineering processes Shannon’s information theory, this entropy is a measure of and, perhaps unexpectedly, in some continuous-time and the amount of information needed to determine the state discrete-time feedback control systems. For instance, of the system. This idea can be used to define a measure in the continuous-time case, chaos has been found in for the intensity of a set of system states, which gives the very simple dynamical systems such as a first-order au- mean loss of information at the state of the system when tonomous feedback system with a time-delay feedback it evolves with time. To do so, let 𝑥(𝑡) be a system orbit and channel, surge tank dynamics under a simple liquid level partition its 𝑚-dimensional phase space into cells of small control system with time-delayed feedback, and several 𝜀𝑚 𝑃 volumes, .Let 𝑘𝑖 be the probability of the sampling or- other types of time-delayed feedback control systems. It 𝑥 𝑘 𝑡 > bital state, ( ), with sampling time s 0, that belongs to also exists in automatic gain control loops, which are the 𝑘𝑖th cell at instant 𝑘 = 𝑖𝑡 , 𝑖 =0, 1, … ,𝑛. Similarly, 𝑃𝑘 ⋯𝑘 s 0 𝑖 very popular in industrial applications such as in most denotes the probability of the orbital state moving from the receivers of communication systems. Most fascinating of 𝑘 𝑘 0th cell through the 𝑖th cell. Then, Shannon defined the all, very simple pendula can display complex dynami- information index to be cal phenomena; in particular, pendula subject to linear ∑ feedback controls can exhibit even richer bifurcations and I𝑛 ∶= − 𝑃𝑘 𝑘 ln (𝑃𝑘 ⋯𝑘 ) 0… 𝑛 0 𝑛 chaotic behaviors. As an example, pendulum controlled by 𝑘 ,⋯,𝑘𝑛 0 a proportional-derivative controller can behave chaotically which is proportional to the amount of the informa- when the tracking signal is periodic, with energy dissipa- tion needed to determine the orbit, if the probabilities tion, even for the case of small controller gains. In addi- I I are known. Consequently, 𝑛+1 − 𝑛 gives additional in- tion, chaos has been found in many engineering applica- 𝑥 𝑘 formation for predicting the state ( 𝑛+1),ifthestates tions such as in the designs of control circuits for switched 𝑥 𝑘 , ,𝑥 𝑘 ( 0) … ( 𝑛) are known. This difference is also the in- mode power conversion equipment, high-performance dig- formation lost during the process. The KS entropy is then ital robot controllers, second-order systems containing a defined by relay with hysteresis, and in various biochemical control systems. ∑𝑛−1 𝐸 1 I I Chaos occurs also frequently in discrete-time feedback KS ∶ = lim lim lim ( 𝑖+1 − 𝑖) 𝑡 →0 𝜀→0 𝑛→∞ 𝑛𝑡 s s 𝑖=0 control systems due to sampling, quantization, and round- ∑ off effects. Discrete-time linear control systems with dead- 1 = − lim lim lim 𝑃𝑘 ⋯𝑘 ln (𝑃𝑘 ⋯𝑘 ) (12) zone nonlinearity have global bifurcations, unstable peri- 𝑡 → 𝜀→ 𝑛→∞ 𝑛𝑡 0 𝑛 0 𝑛 s 0 0 𝑘 , ,𝑘 s 0 … 𝑛 odic orbits, scenarios leading to chaotic attractors, crisis of chaotic attractors reducing to periodic orbits, and so on. This entropy, 𝐸 , quantifies the degree of disorder:(i) KS Chaos also exists in digitally controlled systems, feedback- 𝐸 =0indicates regular attractors, such as stable equilib- KS type of digital filtering systems (either with or without ria, limit cycles, and tori; (ii) 𝐸 =∞implies totally ran- KS control), and even the linear Kalman filter when numeri- dom dynamics, which have no correlations in the phase cal issues are involved. space; (iii) 0 <𝐸 < ∞ signifies strange attractors and KS Many adaptive systems are inherently nonlinear, thus chaos. bifurcations and chaos in such systems are often in- It is interesting to note that there is a connection be- evitable. The instances of chaos in adaptive control sys- tween the Lyapunov exponents and the KS entropy: ∑ tems usually come from several possible sources: the non- 𝐸 ≤ 𝜆+ linearities of the plant and the estimation scheme, exter- KS 𝑖 𝑖 nal excitation or disturbances, the adaptation mechanism, and so on. Chaos can occur in typical model-referenced where 𝜆+ are positive Lyapunov exponents of the same 𝑖 adaptive control (MRAC) and self-tuning adaptive control system. (STAC) systems, as well as some other classes of adaptive feedback control systems of arbitrary order that contain 2.3. Chaos in Control Systems unmodeled dynamics and disturbances. In such adaptive Chaos is ubiquitous, indeed. Chaotic behaviors have been control systems, typical failure modes include convergence found in many typical mathematical maps such as the lo- to undesirable local minima and nonlinear self-oscillation, gistic map, Arnold’s circle map, Henon´ map, Lozi map, such as bursting, limit cycling, and chaos. In indirect adap- Ikeda map, and Bernoulli shift; in various physical sys- tive control of linear discrete-time plants, strange system tems, including the Duffing oscillator, van der Pol oscil- behaviors can arise due to unmodeled dynamics (or distur- lator, forced pendula, hopping robot, brushless DC motor, bances), bad combinations of parameter estimation and/or Chaos, Bifurcations, and Their Control 9 control laws, and lack of persistency of excitation. As an x example, chaos is found in setpoint tracking control of a linear discrete-time system of unknown order, where the adaptive control scheme is either to estimate the order of 0 the plant or to directly track the reference. p Chaos also emerges from various types of neural net- works. Similar to biological neural networks, most arti- ficial neural networks can display complex dynamics, in- cluding bifurcations, strange attractors, and chaos. Even t a very simple recurrent two-neuron model with only one self-interaction can produce chaos. A simple three-neuron Figure 7. The transcritical bifurcation. recurrent neural network can also create period-doubling bifurcations leading to chaos. A four-neuron network and in the parameter space,(𝑥, 𝑦, 𝑝). A few examples are given multi-neuron networks, of course, have more chances to below to distinguish several typical bifurcations. produce complex dynamical patterns such as bifurcations and chaos. A typical example in point is the cellular neural networks, which have very rich complex dynamical behav- Transcritical Bifurcation. The one-dimensional system iors. 𝑥̇ 𝑓 𝑥 𝑝 𝑝𝑥 𝑥2 Chaos has also been experienced in some fuzzy con- = ( ; )= − trol systems. The fact that fuzzy logic can produce com- 𝑥̄ 𝑥̄ 𝑝 𝑝 has two equilibria: 1 =0 and 2 = .When is varied, plex dynamics is more or less intuitive, inspired by the there emerge two equilibrium curves as shown in Figure 7. nonlinear nature of the fuzzy systems. This has been jus- Since the Jacobian for this one-dimensional system is sim- tified, not only experimentally but also both mathemati- 𝐽 𝑝 𝑝<𝑝 ply = , it is clear that for 0 =0, the equilibrium cally and logically. Chaos has been observed, for example, 𝑥̄ 𝑝>𝑝 1 =0is stable, but for 0 =0it changes to be unstable. from a coupled simple fuzzy control system, among others. 𝑥̄ ,𝑝 , Thus, ( 1 0)=(00) is a bifurcation point. In the figure, The change in the shapes of the fuzzy membership func- the solid curves indicate stable equilibria and the dashed tions can significantly alter the dynamical behavior of a 𝑥̄ ,𝑝 curves, the unstable ones. Likewise, ( 2 0) is another bi- fuzzy control system, potentially leading to the occurrence furcation point. This type of bifurcation is called the tran- of chaos. scritical bifurcation. Many specific examples of chaos in control systems can be given. Therefore, controlling chaos is not only interest- ing as a subject for scientific research but also very much Saddle-Node Bifurcation. The one-dimensional system relevant to the objectives of traditional control and sys- 𝑥̇ = 𝑓(𝑥; 𝑝)=𝑝 − 𝑥2 tems engineering. Simply put, it is not an issue that can 𝑥̄ 𝑝 be treated with ignorance or neglect. has an equilibrium 1 =0 at 0√=0, and an equilibrium√ 𝑥̄ 2 𝑝 𝑝 ≥ 𝑥̄ 𝑝 𝑥̄ 𝑝 curve = at 0,where 2 = is stable and 3 =− is unstable for 𝑝>𝑝 =0. This bifurcation, as shown in 3. BIFURCATIONS 0 Figure 8, is called the saddle-node bifurcation. Associated with chaos is bifurcation, another typical phe- nomenon of nonlinear dynamical systems that quantifies Pitchfork Bifurcation. The one-dimensional system the change of system properties (such as the number and 𝑥̇ 𝑓 𝑥 𝑝 𝑝𝑥 𝑥3 the stabilities of the system equilibria) due to the variation = ( ; )= − of system parameters. Chaos and bifurcations have a very 𝑥̄ 𝑝 has an equilibrium 1 =0 at 0 =0, and an equilibrium strong connection; oftentimes they coexist in a complex 𝑥̄ 2 𝑝 𝑝 ≥ 𝑥̄ 𝑝>𝑝 curve = at 0. Here, 1 =0is unstable for 0 =0 dynamical system. 𝑝<𝑝 and stable for 0 =0, and the entire equilibrium curve 𝑥̄ 2 = 𝑝 is stable for all 𝑝>0 at which it is defined. This 3.1. Basic Types of Bifurcations situation, as depicted in Figure 9, is called the pitchfork To illustrate various bifurcation phenomena, it is conve- bifurcation. nient to consider a two-dimensional parameterized non- linear dynamical system: { x x2 = p 𝑥̇ = 𝑓(𝑥, 𝑦; 𝑝) (13) 𝑦̇ = 𝑔(𝑥, 𝑦; 𝑝) where 𝑝 is a real and variable system parameter. 0 ( ) p 𝑥,̄ 𝑦 ̄ 𝑥̄ 𝑡 𝑝 ,̄𝑦 𝑡 𝑝 Let ( )= ( ; 0) ( ; 0) be an equilibrium of the sys- 𝑝 𝑝 𝑓 𝑥,̄ 𝑦 ̄ 𝑝 𝑔 𝑥,̄ 𝑦 ̄ 𝑝 tem when = 0,atwhich ( ; 0)=0and ( ; 0)=0.If 𝑝>𝑝 the equilibrium is stable (respectively, unstable) for 0 𝑝<𝑝 𝑝 but unstable (respectively, stable) for 0,then 0 is a 𝑝 , 𝑝 Figure 8. The saddle-node bifurcation. bifurcation value of ,and(0 0; 0) is a bifurcation point 10 Chaos, Bifurcations, and Their Control

x x2 = p

x 0 p

Figure 9. The pitchfork bifurcation. p = p0 xx Note, however, that not all nonlinear parameterized dy- namical systems have bifurcations. A simple example is y y 𝑥̇ = 𝑓(𝑥; 𝑝)=𝑝 − 𝑥3 that has an entire stable equilibrium curve 𝑥̄ = 𝑝1∕3,which p > p0 does not have any bifurcation. xx

Hysteresis Bifurcation. The dynamical system { p 𝑥̇ 𝑥 1 =− 1 p 𝑥̇ 𝑝 𝑥 𝑥3 2 = + 2 − 2 has equilibria 𝑥̄ 𝑝 𝑥̄ 𝑥̄ 3 1 =0 and − 2 + 2 =0 p = p0 y According to different values of 𝑝, there are either one y or three equilibrium solutions, where√ the second equation 𝑝 yields one bifurcation√ point at 0 =±2 3∕9,butthreeequi- x x |𝑝 | < libria for 0 2 3∕9. The stabilities of the equilibrium solutions are shown in Figure 10. This type of bifurcation is called the hysteresis Figure 11. Two types of Hopf bifurcations illustrated in the bifurcation. phase plane.

Hopf Bifurcation and Hopf Theorems. In addition to the bifurcations described above, called static bifurcations,the the real parts of the eigenvalue pair are zero, and the sta- parameterized dynamical system (eq. 13) can have another bility of the existing equilibrium changes to opposite, as type of bifurcation, the Hopf bifurcation (or dynamical bi- showninFigure11. In the meantime, a limit cycle will furcation). emerge. As indicated in the figure, Hopf bifurcation can Hopf bifurcation corresponds to the situation where, be classified as supercritical (respectively, subcritical), if 𝑝 𝑝 the equilibrium is changed from stable to unstable (re- as the parameter is varied to pass the critical value 0, the system Jacobian has one pair of complex conjugate spectively, from unstable to stable). The same terminology eigenvalues moving from the left-half plane to the right, of supercritical and subcritical bifurcations applies also to crossing the imaginary axis, while all the other eigenval- other non-Hopf types of bifurcations. ues remain to be stable. At the moment of the crossing, Hopf Bifurcation Theorems. Consider a general nonlin- ear parameterized autonomous system, x2 𝐱̇ 𝐟 𝐱 𝑝 , 𝐱 𝑡 𝑥 = ( ; ) ( 0)= 0 (14) where 𝐱 ∈ 𝑅𝑛, 𝑝 is a real variable parameter, and 𝐟 is dif- ferentiable. The most fundamental result on the Hopf bifurcation of p this system is the following theorem, which is stated here –p0 p0 only for the special two-dimensional setting.

Theorem 7 (Poincare–Andronov–Hopf)´ Suppose that the two-dimensional system (eq. 14) has a zero equilibrium, | 𝐱̄ 𝐴̄ 𝜕𝑓 | =0, and assume that its associate Jacobian = 𝜕𝑥 | |𝑥=𝐱̄=0 Figure 10. The hysteresis bifurcation. has a pair of purely imaginary eigenvalues, 𝜆(𝑝) and 𝜆∗(𝑝). Chaos, Bifurcations, and Their Control 11

If where 𝑝̃ is the fixed value of the parameter 𝑝, 𝐥⊤ and 𝐫 are | the left and right eigenvectors of [𝐻(𝑗̃𝜔; 𝑝̃)𝐽(𝑝̃)],respec- dℜ{𝜆(p)} | ̂ | > tively, associated with the eigenvalue 𝜆(𝑗̃𝜔; 𝑝̃),and | 0 dp |𝑝 𝑝 = 0 [ ( ) ] 1 1 𝑝 ℜ ⋅ 𝐡 = 𝐷 𝐳 ⊗ 𝐫 + 𝐫∗ ⊗ 𝐳 + 𝐷 𝐫 ⊗ 𝐫 ⊗ 𝐫∗ for some 0, where { } denotes the real part of the complex 1 2 02 2 22 8 3 eigenvalues, then where ∗ denotes the complex conjugate, 𝜔̃ is the frequency 𝑝 𝑝 𝜆̂ (i) = 0 is a bifurcation point of the system; of the intersection between the locus and the negative 𝑝<𝑝 𝐱̄ real axis that is closest to the point (−1 + 𝑗 0), ⊗ is the (ii) for close enough values of 0, the equilibrium =0 is asymptotically stable; tensor product operator, and 𝑝>𝑝 𝐱̄ (iii) for close enough values of 0, the equilibrium =0 | is unstable; 𝜕2𝐠(𝑦; 𝑝̃) | 𝑝 ≠ 𝑝 𝐱̄ 𝐷 = | (iv) for close enough values of 0, the equilibrium√ =0 2 𝜕𝑦2 | |𝑦 𝑦̄ is surrounded by a limit cycle of magnitude 𝑂( |𝑝|). = | 𝜕3𝐠(𝑦; 𝑝̃) | 𝐷 = | Graphical Hopf Bifurcation Theorem. The Hopf bifurca- 3 𝜕𝑦3 | |𝑦=𝑦̄ tion can also be analyzed in the frequency domain setting [ ]−1 (14). In this approach, the nonlinear parameterized au- 𝐳 1 𝐻 𝑝̃ 𝐽 𝑝̃ 𝐺 𝑝̃ 𝐷 𝐫 ⊗ 𝐫∗ 02 =− 1+ (0; ) ( ) (0; ) 2 tonomous system (eq. 14) is first rewritten in the following 4 [ ] Lur’e form: −1 𝐳 1 𝐻 𝑗̃𝜔 𝑝̃ 𝐽 𝑝̃ 𝐻 𝑗̃𝜔 𝑝̃ 𝐷 𝐫 ⊗ 𝐫 22 =− 1+ (2 ; ) ( ) (2 ; ) 2 ⎧ 𝐱̇ = 𝐴(𝑝) 𝐱 + 𝐵(𝑝) 𝑢 4 ⎪ 𝑦 = 𝐳 |𝑝̃− 𝑝 | ⎨ 𝐲 =−𝐶(𝑝) 𝐱 (15) 0 02 0 ⎪ 𝑦 𝐫 |𝑝̃ 𝑝 |1∕2 ⎩ 𝐮 = 𝑔(𝑦; 𝑝) 1 = − 0 𝑦 = 𝐳 |𝑝̃− 𝑝 | where the matrix 𝐴(𝑝) is chosen to be invertible for all 2 22 0 values of 𝑝,and𝐠 ∈ 𝐶4 depends on the chosen matrices The graphical Hopf bifurcation theorem (for SISO sys- 𝐴, 𝐵,and𝐶. Assume that this system has an equilibrium tems) formulated in the frequency domain, based on the solution, 𝐲̄, satisfying generalized Nyquist criterion, is stated as follows. 𝐲̄(𝑡; 𝑝)=−𝐻(0; 𝑝) 𝑔(𝐲̄(𝑡; 𝑝); 𝑝) Theorem 8 (Graphical Hopf Bifurcation Theorem) where 𝜔 𝜉 𝜔̃ ≠ Suppose that, whenever is varied, the vector 1( ) 𝑗 𝐻(0; 𝑝)=−𝐶(𝑝) 𝐴−1(𝑝) 𝐵(𝑝) 0. Assume also that the half-line, starting from −1 + 0 and pointing to the direction parallel to that of 𝜉 (𝜔̃), first | 1 | ̂ ̂ 𝜆̂ 𝑗𝜔 𝑝̃ Let 𝐽(𝑝)=𝜕𝐠∕𝜕𝐲| and let 𝜆 = 𝜆(𝑗𝜔; 𝑝) be the eigenvalue intersects the locus of the eigenvalue ( ; ) at the point |𝐲=𝐲̄ of the matrix [𝐺(𝑗𝜔; 𝑝)𝐽(𝑝)] that satisfies 𝑃̂ 𝜆̂ 𝜔̂ 𝑝̃ 𝜉 𝜔̃ 𝜃2 = ( ; )=−1+ 1( ) √ 𝜆̂ 𝑗𝜔 𝑝 𝑗 ,𝑗 ( 0; 0)=−1+ 0 = −1 at which 𝜔 = 𝜔̂ and the constant 𝜃 = 𝜃(𝜔̂) ≥ 0,asshownin Figure 12. Suppose, furthermore, that the above intersec- Then, fix 𝑝 = 𝑝̃ and let 𝜔 vary. In so doing, a trajectory of tion is transversal, namely, the function 𝜆̂(𝜔; 𝑝̃), the “eigenlocus,” can be obtained. This locus traces out from the frequency 𝜔 ≠ 0. In much the 0 ⎡ ℜ{𝜉 (𝑗̂𝜔)} ℑ{𝜉 (𝑗̂𝜔)} ⎤ same way, a real zero eigenvalue (a condition for the static ⎢ { 1 } { 1 } ⎥ | | ≠ bifurcation) is replaced by a characteristic gain locus that det ⎢ 𝑑 𝑑 ⎥ 0 ℜ 𝜆̂(𝜔; 𝑝̃)| ℑ 𝜆̂(𝜔; 𝑝̃)| 𝑗 𝜔 ⎢ 𝑑𝜔 | 𝑑𝜔 | ⎥ crosses the point (−1 + 0) at frequency 0 =0. ⎣ |𝜔=𝜔̂ |𝜔=𝜔̂ ⎦ For illustration, consider a single-input single-output (SISO) system. In this case, the matrix [𝐻(𝑗𝜔; 𝑝)𝐽(𝑝)] is Then merely a scalar, and { } ∑𝑛 (i) Thenonlinearsystem(eq.15) has a periodic solu- 𝑦(𝑡)≈𝑦̄ + ℜ 𝑦 e𝑗𝑘𝜔𝑡 tion (output) 𝑦(𝑡)=𝑦(𝑡; 𝑦̄). Consequently, there exists a 𝑘 ̇ 𝑘=0 unique limit cycle for the nonlinear equation 𝐱 = 𝐟(𝐱), in a ball of radius 𝑂(1) centered at the equilibrium 𝐱̄. where 𝑦̄ is the equilibrium solution and the complex coeffi- 𝑦 (ii) If the total number of anticlockwise encirclements of cients { 𝑘} are determined as follows. For the approxima- 𝑝 𝑃̂ 𝜀𝜉 𝜔̃ 𝜀> 𝑛 the point 1 = + 1( ), for a small enough 0,is tion with =2, first define an auxiliary vector: 𝐺 𝑠 𝑝 𝐽 𝑝 [ ] equal to the number of poles of [ ( ; ) ( )] that have 𝐥⊤ 𝐻 𝑗̃𝜔 𝑝̃ 𝐡 positive real parts, then the limit cycle is stable. − ( ; ) 1 𝜉 (𝜔̃)= (16) 1 𝐥⊤𝐫 12 Chaos, Bifurcations, and Their Control

{ λω ( ;p∼)}

xn λω ( ;p∼) ω 1.0 λω ( ;p∼) ∼ –1 0 ¬ { λω ( ;p)}

2 ∼ P = –1+θξ 1(ω ) 2 ∼ θξ1(ω ) 0.5

Figure 12. The frequency domain version of the Hopf bifurcation theorem.

0.0 3.2. Period-Doubling Bifurcations to Chaos 2.8 3.01+1 + 1+1+ α There are several routes to chaos from a regular state of 3.56994 4.0 a nonlinear system, provided that the system is chaotic in nature. One route is that after three Hopf bifurcations a regular Figure 13. Period-doubling of the logistic system with self- motion can become highly unstable, leading to a strange similarity. attractor and then to chaos. Actually, even pitchfork and saddle-node bifurcations can be routes to chaos under cer- map, it appears self-similarity of the bifurcation diagram tain circumstances. For motion on a normalized two-torus, of the map, which is a typical fractal structure. if the ratio of the two fundamental frequencies 𝜔 ∕𝜔 = 𝑝∕𝑞 1 2 Figure 14 shows the Lyapunov exponent 𝜆 versus the is rational, then the orbit returns to the same point after parameter 𝑝,overtheintervalof[2.5, 4]. This figure corre- a 𝑞-cycle; but if the ratio is irrational, then this (quasiperi- sponds to the period-doubling diagram shown in Figure 13. odic) orbit never returns to the starting point. Quasiperi- The most significant discovery about the phenomenon odic motion on a two-torus provides another common route of period-doubling bifurcation route to chaos is Feigen- to chaos. baum’s observation in 1978: The convergence of the period- Period-doubling bifurcation is perhaps the most typical doubling bifurcating parameters has a geometric rate, route that leads system dynamics to chaos. Consider, as 𝑘 𝑝 − 𝑝 ∝ 𝛿− ,where an example, the logistic map ∞ 𝑘 𝑝 − 𝑝 ( ) 𝛿 𝑘+1 𝑘 → 𝛿 . ... 𝑘 → 𝑥 𝑝𝑥 𝑥 𝑘 ∶= 𝑝 𝑝 =46692 ( ∞) 𝑘+1 = 𝑘 1− 𝑘 (17) 𝑘+2 − 𝑘+1

where 𝑝>0 is a variable parameter. With 0 <𝑝<1,the which is known as a universal number for a large class of origin 𝑥 =0 is stable, so the orbit approaches it as 𝑘 → chaotic dynamical systems. ∞. However, for 1 <𝑝<3, all points converge to another equilibrium, denoted 𝑥̄. 3.3. Bifurcations in Control Systems The evolution of the system dynamics, as 𝑝 is gradually Not only chaos but also bifurcations can exist in feed- increased from 3.0 to 4.0 by small steps, is mostly interest- back and adaptive control systems. Generally speaking, ing, which is depicted in Figure 13. The figure shows that local instability and complex dynamical behavior can re- at 𝑝 =3, a (stable) period-two orbit is bifurcated out of 𝑥̄, sult from feedback and adaptive mechanisms when ad- which becomes unstable at that moment, and, in addition to 0, there emerge two (stable) equilibria: 1.00 ( √ )/ 𝑥̄ 1,2 = 1+𝑝 ± 𝑝2 −2𝑝 −3 (2𝑝) √ 0.00 When 𝑝 continues to increase to the value of 1+ 6= 3.544090 … , each of these two points bifurcates to other two, as can be seen from the figure. As 𝑝 moves conse- λ –1.00 quently through the values 3.568759 … , 3.569891 … , … , an infinite sequence of bifurcations is generated by such period-doubling, which eventually leads to chaos: –2.00

period 1 → period 2 → period 4 → ⋯ period 2𝑘 –3.00 2.50 2.88 3.25 3.63 4.00 → ⋯ → chaos p

It is also interesting to note that in certain regions (e.g., Figure 14. Lyapunov exponent versus parameter 𝑝 for the logis- the three windows magnified in the figure) of the logistic tic map. Chaos, Bifurcations, and Their Control 13 equate process information is not available for feedback systems also have similar behavior. Even a common road transmission or parameter estimation. In this situation, vehicle driven by a pilot with driver steering control can one or more poles of the linearized closed-loop transfer have Hopf bifurcation when its stability is lost, which may function may move to cross over the stability boundary, also develop chaos and even hyperchaos. A hopping robot, thereby causing signal divergence as the control process or a simple two-degree-of-freedom flexible robot arm, can continues. However, this sometimes may not lead to global respond to strange vibrations undergoing period-doubling unboundedness, but rather to self-excited oscillations or bifurcations, which eventually lead to chaos. An aircraft self-stabilization, leading to very complex dynamical phe- stalls for flight below a critical speed or over a critical nomena. angle-of-attack can respod to various bifurcations. Dynam- Real examples of bifurcations in feedback control sys- ics of a ship can exhibit stability bifurcation according to tems include the automatic gain control loop system, which wave frequencies that are close to the natural frequency has bifurcations transmitting to Smale horseshoe chaos of the ship, which creates oscillations and chaotic motions and the common route of period-doubling bifurcations to leading to ship capsize. Simple nonlinear circuits are rich chaos. Surprising enough, in some situations even a single sources of different types of bifurcations as well as chaos. pendulum controlled by a linear proportional-derivative Other systems that have bifurcation properties include controller can display rich bifurcations in addition to cellular neural networks, lasers, aeroengine compressors, chaos. weather systems, and biological population dynamics, to Adaptive control systems are more likely to produce name but a few. bifurcations than a simple feedback control system, due to the changes of stabilities in adaptation. The complex dynamics emerging from an adaptive control system is of- 4. CONTROLLING CHAOS ten caused by estimation instabilities. Moreover, certain prototypes of MRAC systems can experience various bifur- Understanding chaos has long been the main focus of re- cations. search in the field of nonlinear science. The idea that chaos Bifurcation theory has been employed for analyzing can in fact be controlled is perhaps counterintuitive. In- complex dynamical systems. For instance, in an MRAC deed, the extreme sensitivity of a chaotic system to initial system, a few pathways leading to estimator instability conditions once led to the impression and argument that have been identified via bifurcation analysis: chaotic motion is in general neither predictable nor con- trollable. (i) A sign change in the adaptation law, leading to a However, recent research effort has shown that not only reversal of the gradient direction as well as an infinite (short term) prediction but also (long term) control of chaos linear drift. are possible. It is now well known that most conventional control methods and many special techniques can be used (ii) The instability caused by high control gains, leading for controlling chaos (5,15,16). In this pursuit, no matter to global divergence through period-doubling bifurca- the purpose is to reduce “bad” chaos or to introduce “good” tions. ones, numerous control strategies have been proposed, de- (iii) A Hopf bifurcation-type of instability, leading to pa- veloped, tested, and applied to many case studies. Numeri- rameter drift and bursting in a bounded regime cal and experimental simulations have demonstrated that through a sequence of global bifurcations. chaotic physical systems respond quite well to these con- trols. In about the same time, applications are proposed in Both instabilities of types (i) and (ii) can be avoided by such diverse fields as biology, medicine, physiology, chemi- gain-tuning or simple algorithmic modifications. The third cal engineering, laser physics, electric power systems, fluid instability, however, is generally due to the unmodeled mechanics, aerodynamics, circuits and electronic devices, dynamics and a poor signal-to-noise ratio, and so cannot signal processing and communications, and so on. The fact be avoided by simple tuning methods. This instability is that researchers from vast scientific and engineering back- closely related to the presence of a degenerate set and a grounds are joining together and aiming at one central period-two attractor. theme – bringing order to chaos – indicates that the study Similarly, in the discrete-time case, a simple adaptive of nonlinear dynamics and their control has progressed control system can have rich bifurcation phenomena such into a new era. Much has been accomplished in the past as period-doubling bifurcation (due to high adaptive con- decade, and yet much more remains a challenge for the trol gains) and Hopf and global bifurcations (due to insuf- near future. ficient excitations). Similar to conventional systems control, the concept of Like the omnipresent chaos, bifurcations exist in many “controlling chaos” is first to mean suppressing chaos in the physical systems (5). For instance, power systems gener- sense of stabilizing chaotic system responses, oftentimes ally have various bifurcation dynamics. When the con- unstable periodic outputs. However, controlling chaos has sumers’ demands for power reach peaks, the stability of also encompassed many nontraditional tasks, particularly an electric power network may move to its margin, lead- those in creating or enhancing chaos when it is useful. The ing to serious oscillations and stability bifurcations, which process of chaos control is now understood as a transition may quickly result in voltage collapse. As another exam- between chaos and order and, sometimes, the transition ple, a typical double pendulum can display bifurcations from chaos to chaos, depending on the application at hand. as well as chaotic motions. Some rotational mechanical In fact, the notion of chaos control is neither exclusive of 14 Chaos, Bifurcations, and Their Control nor conflicting with the purposes of conventional control makes the brain different from an artificial-intelligence systems theory. Rather, it targets at better managing the machine” (19). The idea of anticontrolofchaoshas been dynamics of a nonlinear system on a wider scale, with proposed for solving the problem of driving the system the hope that more benefits may thus be derived from the responses of a human brain model away from the stable special features of chaos. direction, and hence away from the stable (saddle-type) equilibrium. As a result, the periodic behavior of neuronal population bursting can be prevented (20). Control tasks 4.1. Why Chaos Control of this type are also nontraditional. There are many practical reasons for controlling or order- Other potential applications of chaos control in biologi- ing chaos. First of all, “chaotic” (messy, irregular, or disor- cal systems have reached out from the brain to elsewhere, dered) system response with little useful information con- particularly to the human heart. In physiology, healthy tent is unlikely to be desirable. Second, chaos can lead sys- dynamics has been regarded as regular and predictable, tems to harmful or even catastrophic situations. In these whereas disease, such as fatal arrhythmias, aging, and troublesome cases, chaos should be reduced as much as drug toxicity, are commonly assumed to produce disorder possible, or be totally suppressed. Traditional engineer- and even chaos. However, recent laboratory studies have ing design always tries to reduce irregular behaviors of a seemingly demonstrated that the complex variability of system and, therefore, completely eliminates chaos. Such healthy dynamics in a variety of physiological systems has “over-design” is needed in the aforementioned situations. features reminiscent of deterministic chaos, and a wide However, this is usually accomplished at the price of loos- class of disease processes (including drug toxicities and ag- ing great opportunities and benefits in achieving high per- ing) may actually decrease (yet not completely eliminate) formance near the stability boundaries or at the expense the amount of chaos or complexity in physiological sys- of radically modifying the original system dynamics. tems, known as “decomplexification.” Thus, in contrast to Conversely, recent research has shown that chaos can the common belief that healthy heartbeats are completely actually be useful under certain circumstances, and there regular, a normal heart rate may fluctuate in a highly er- is growing interest in utilizing the very nature of chaos (5). ratic fashion, even at rest, and may actually be chaotic (21). For example, it was observed (17) that a chaotic attractor It has also been observed that, in the heart, the amount of typically has embedded within it a dense set of unstable intracellular Ca is closely regulated by coupled processes periodic orbits. Thus, if any of these periodic orbits can that cyclically increase or decrease this amount, in a way be stabilized, it may be desirable to select one that gives similar to a system of coupled oscillators. This cyclical fluc- rise to a certain maximal system performance. In other tuation in the amount of intracellular Ca is a cause of words, when the design of a dynamical system is intended after-depolarizations, which triggers activities in the heart for multiple usages, purposely building chaotic dynamics — the so-called arrhythmogenic mechanism. Medical evi- into the system may allow for the desired flexibilities. A dence reveals that controlling (while not completely elim- control design of this kind is certainly nonconventional. inating) the chaotic arrhythmia can be a new, safe, and Fluid mixing is a good example in which chaos is not promising approach to regulating heartbeats (22,23). only useful but actually necessary (18). Chaos is desirable The sensitivity of chaotic systems to small perturba- in many applications of liquid mixing, where two fluids are tions can be used to direct system trajectories to a desired to be thoroughly mixed while the required energy is min- target quickly with very low (or minimum) control energy. imized. For this purpose, it turns out to be much easier if As an example, NASA scientists used small amounts of the dynamics of the particle motion of the two fluids are residual hydrazine fuel to send the spacecraft ISEE-3/IEC strongly chaotic, since it is difficult to obtain rigorous mix- more than 50 million miles across the solar system, achiev- ing properties otherwise, due to the possibility of invariant ing the first scientific cometary encounter. This control ac- two-tori in the flow. This has been one of the main subjects tion utilized the sensitivity to small perturbations of the in fluid mixing, known as “chaotic advection.” Chaotic mix- three-body problem of celestial mechanics, which would ing is also important in applications involving heating, not be possible in a nonchaotic system since it normally such as plasma heating for a nuclear fusion reactor. In requires a huge control effort (24). such plasma heating, heat waves are injected into the re- actor, for which the best result is obtained when the heat 4.2. Chaos Control: An Example convection inside the reactor is chaotic. Within the context of biological systems, the controlled To appreciate the challenge of chaos control, consider the biological chaos seems to be important with the way a hu- one-dimensional logistic map (eq. 17)withtheperiod- man brain executes its tasks. For years, scientists have doubling bifurcations route to chaos as shown in Figure 13. been trying to unravel how our brains endow us with Chaos control problems in this situation include, but inference, thought, perception, reasoning and, most fas- not limited to, the following: cinating of all, emotion such as happiness and sadness. There were experimental suggestions that human brain Is it possible (and, if so, how) to design a simple can process massive information in almost no time, in controller, 𝑢𝑘, for the given system, in the form of which chaotic dynamics could be a fundamental reason: ( ) 𝑥 𝑝𝑥 𝑥 𝑢 “The controlled chaos of the brain is more than an acciden- 𝑘+1 = 𝑘 1− 𝑘 + 𝑘 tal by-product of the brain complexity, including its myriad connections,” but rather “it may be the chief property that such that Chaos, Bifurcations, and Their Control 15

(i) the limiting chaotic behavior of the period- ear systems and affine-nonlinear systems). However, doubling bifurcations process is suppressed? in chaos control, a target for tracking is not limited to (ii) the first bifurcation is delayed to take place, or constant vectors in the state space but often is an un- some bifurcations are changed either in form or stable periodic orbit of the given system. In addition, in stability? the terminal time for chaos control is usually infinite to (iii) when the parameter 𝑝 is currently not in the be meaningful and practical, because many nonlinear bifurcating range, the asymptotic behavior of dynamical behaviors such as steady states, limit cycles, the system becomes chaotic? attractors, and chaos are asymptotic properties. 5. Depending on different situations or purposes, the per- formance measure in chaos control can be different from Many of such nonconventional control problems emerg- those for conventional controls. Chaos control generally ing from chaotic dynamical systems have posed a real uses criteria like Lyapunov exponents, Kolmogorov– challenge to both nonlinear dynamics analysts and con- Sinai entropy, power spectra, ergodicity, bifurcation trol engineers — they have become, in effect, motivation changes, and so on, whereas conventional controls nor- and stimuli for the current endeavor devoted to the new mally emphasize on robustness of the system stability research direction in control systems: controlling bifurca- or control performance, optimality of control energy or tions and chaos. time, ability of disturbances rejection, and so on. 6. Chaos control includes a unique task — anticontrol, 4.3. Some Distinctive Features of Chaos Control required by some unusual applications such as those At this point, it is illuminating to highlight some distinc- in biomedical engineering mentioned above. This anti- tive features of chaos control theory and methodology, in control tries to create, maintain, or enhance chaos for contrast to other conventional approaches regarding such improving system performance. Bifurcation control is issues as objectives, perspectives, problem formulations, another example of this kind, where a bifurcation point and performance measures. is expected to be delayed in case it cannot be avoided or stabilized. This delay can significantly extend the 1. The targets in chaos control are usually unstable peri- operating time (or system parameter range) for a time- odic orbits (including equilibria and limit cycles), per- critical process such as chemical reaction, voltage col- haps of higher periods. The controller is designed to lapse of electric power systems, and compressor of stall stabilize some of these unstable orbits or to drive the of gas turbine jet engines. These are in direct contrast trajectories of the controlled system to switch from one to traditional control tasks such as the typical problem orbit to another. This inter-orbit switching can be ei- of stabilizing an equilibrium position of a nonlinear sys- ther chaos → order, chaos → chaos, order → chaos, or tem. order → order, depending on the application in interest. 7. Due to the inherent association of chaos and bifurca- Conventional control, on the other hand, does not nor- tions with various related issues, the scope of chaos mally investigate such inter-orbit switching problems control and the variety of problems that chaos control of a dynamical system, especially not those problems deals with are quite diverse, including creation and/or that involve guiding a system trajectory to an unstable management of self-similarity and symmetry, pattern or chaotic state by any means. formation, amplitudes of limit cycles and size of attrac- 2. A chaotic system typically has embedded within it a tor basins, birth and change of bifurcations and limit dense set of unstable orbits, and is extremely sensitive cycles, and so on, in addition to some conventional tasks to tiny perturbations to its initial conditions and system such as target tracking and system regulation. parameters. Such a special property, useful for chaos control, is not available in nonchaotic systems and is It is also worth mentioning an additional distinctive not utilized in any forms by conventional control. feature of a controlled chaotic system that differs from an 3. Most conventional control schemes work within the uncontrolled chaotic system. When the controlled chaotic state space framework. In chaos control, however, one system is non-autonomous, it cannot be reformulated as more often deals with parameter space and phase space. an autonomous system by defining the control input as a Poincare´ maps, delay-coordinates embedding, paramet- new state variable, since the control input is physically not ric variation, entropy reduction, bifurcation monitoring, a system state variable and, moreover, it has to be deter- and so on, are some typical but nonconventional tools mined via design for performance specifications. Hence, a for design and analysis. controlled chaotic system is intrinsically much more dif- 4. In conventional control, a target for tracking is usually a ficult to design than it appears; for instance, many in- constant vector in the state space, with very few excep- variant properties of autonomous systems are no longer tions such as some model-referenced control schemes. valid. This observation raises the question of extending This target is generally not a state of the given (uncon- some existing theories and techniques from autonomous trolled) system (otherwise, perhaps no control is needed system dynamics to nonautonomous controlled dynamical or can be easily achieved). Also, the terminal time for systems, including such complex phenomena as degener- the control is usually finite (e.g., the fundamental con- ate bifurcations and hyperchaos in the system dynamics cept of “controllability” is typically defined using a fi- when a controller is involved. Unless suppressing complex nite, and often fixed, terminal time, at least for lin- dynamics in a process is the only purpose for control, un- 16 Chaos, Bifurcations, and Their Control

derstanding and utilizing the rich dynamics of a controlled mance with respect to the dynamical behavior of the sys- chaotic system are very important for design and applica- tem. This target orbit satisfies tions. 𝜉∗ 𝑃 𝜉∗ ,𝑝∗ 𝑓 = ( 𝑓 ) 4.4. Representative Approaches to Chaos Control The iterations of the map near the desired orbit are then There are various conventional and nonconventional con- observed, and the local properties of this chosen periodic trol methods available for bifurcations and chaos control orbit are obtained. To do so, the map is first locally lin- 𝑃 𝜉∗ (5,15,16). To introduce a few representative control tech- earized, yielding a linear approximation of near 𝑓 and 𝑝∗ niques, two major categories of methodologies are briefly ,as described. 𝜉 𝜉∗ 𝐿 𝜉 𝜉∗ 𝐯 𝑝 𝑝∗ 𝑘+1 ≈ 𝑓 + 𝑘( 𝑘 − 𝑓 )+ 𝑘( 𝑘 − ) (20)

Parametric Variation Control. This approach to control- or ling a chaotic dynamical system, proposed by Ott et al. (17), 𝜉 𝐿 𝜉 𝐯 𝑝 known as the OGY method, is to stabilize one of its unsta- Δ 𝑘+1 ≈ 𝑘Δ 𝑘 + 𝑘Δ 𝑘 (21) ble periodic orbits embedded in an existing chaotic attrac- where tor, via small time-dependent perturbations of a variable ∗ ∗ system parameter. This methodology utilizes the special Δ𝜉𝑘 = 𝜉𝑘 − 𝜉 , Δ𝑝𝑘 = 𝑝𝑘 − 𝑝 𝑓 / / feature of chaos that a chaotic attractor typically has em- 𝐿 = 𝜕𝑃(𝜉∗ ,𝑝∗) 𝜕𝜉 , 𝐯 = 𝜕𝑃(𝜉∗ ,𝑝∗) 𝜕𝑝 bedded within it a dense set of unstable periodic orbits. 𝑘 𝑓 𝑘 𝑘 𝑓 𝑘 To introduce this control strategy, consider a general 𝜆 𝜆 The stable and unstable eigenvalues, s,𝑘 and u,𝑘 satisfying continuous-time parameterized nonlinear autonomous |𝜆 | < < |𝜆 | 𝐿 s,𝑘 1 u,𝑘 , can be calculated from the Jacobian 𝑘. system: 𝑀 𝑀 Let s and u be the stable and unstable manifolds, whose 𝑒 𝑒 𝐱̇ (𝑡)=𝐟(𝐱(𝑡),𝑝) (18) directions are specified by the eigenvectors s,𝑘 and u,𝑘 that 𝜆 𝜆 𝑔 𝑔 are associated with s,𝑘 and u,𝑘, respectively. If s,𝑘 and u,𝑘 where, for illustration, 𝐱 =[𝑥𝑦𝑧]⊤ denotes the state vec- are the basis vectors defined by tor and 𝑝 is a system parameter accessible for adjustment. ⊤ ⊤ ∗ 𝐠 𝐞 𝐠 𝐞 Assume that when 𝑝 = 𝑝 the system is chaotic, and it is de- s,𝑘 s,𝑘 = u,𝑘 u,𝑘 =1 sired to control the system orbit, 𝐱(𝑡), to reach a saddle-type ⊤ ⊤ 𝐠 𝐞 = 𝐠 𝐞 =0 equilibrium (or periodic orbit), Γ, which otherwise would s,𝑘 u,𝑘 u,𝑘 s,𝑘

not be able to arrive by the system orbits due to its unstable then the Jacobain 𝐿𝑘 can be expressed as nature. 𝑝∗ 𝐿 𝜆 𝐞 𝐠⊤ 𝜆 𝐞 𝐠⊤ Suppose that within a small neighborhood of ,thatis, 𝑘 = u,𝑘 u,𝑘 u,𝑘 + s,𝑘 s,𝑘 s,𝑘 (22) 𝑝∗ 𝑝 <𝑝<𝑝∗ 𝑝 −Δ max +Δ max (19) To start the parametric variation control scheme, one may open a window covering the target equilibrium, and 𝑝 > where Δ max 0 is the maximum allowable perturbation, wait until the system orbit travels into the window (i.e., till both the chaotic attractor and the target orbit Γ do not 𝜉 𝜉∗ 𝑘 falls close enough to 𝑓 ).Duetotheergodicityofchaos, ∗ disappear (i.e., within this small neighborhood of 𝑝 ,there this is always possible. To that end, the nominal value of are no bifurcation points of the periodic orbit Γ). By the the parameter 𝑝𝑘 is adjusted by a small amount, Δ𝑝𝑘,using structural stability of chaotic systems, this can be guaran- a control formula given below. In so doing, both the loca- teed. Then, let 𝑃 be the underlying Poincare´ map and Σ tion of the orbit and its stable manifold are changed, such be a surface of cross section of Γ. For simplicity, assume 𝜉 that the next iteration, represented by 𝑘+1 in the surface of that this two-dimensional hyperplane is orthogonal to the cross section, is forced toward the local stable manifold of third axis, and thus is given by the original equilibrium point. Since the system has been { } [ ]⊤ linearized, this control action is usually unable to bring 𝛼𝛽𝛾 𝑅3 𝛾 𝑧 Σ= ∈ ∶ = 0 (a constant) the moving orbit to the target at one iteration step. As a matter of fact, the controlled orbit will leave the small Moreover, let 𝜉 be the coordinates of the surface of cross neighborhood of the equilibrium, and continue to wander section, that is, a vector satisfying chaotically as if there was no control on it at all. However, since the chaotic attractor has a dense set of unstable pe- 𝜉 𝑃 𝜉 ,𝑝 𝑘+1 = ( 𝑘 𝑘) riodic orbits embedded within it, sooner or later the orbit returns to the window again, but would be closer to the where target due to the control effect. Then, the next cycle of it- ∗ eration is applied, with an even smaller control action, to 𝑝𝑘 = 𝑝 +Δ𝑝𝑘 , |Δ𝑝𝑘| ≤ Δ𝑝 max nudge the orbit to move toward the target. At each iteration, 𝑝 = 𝑝𝑘 is chosen to be a constant. For the case of a saddle-node equilibrium target, this Many distinct unstable periodic orbits within the control procedure is illustrated by Figure 15. chaotic attractor can be determined by the Poincaremap.´ Now, suppose that 𝜉𝑘 has approached sufficiently close 𝜉∗ 𝜉 𝜉 Suppose that an unstable period-one orbit 𝑓 has been to 𝑓 ,sothat(eq.20) holds. For the next iteration, 𝑘+1, 𝜉∗ selected, which maximizes certain desired system perfor- to fall onto the local stable manifold of 𝑓 ,theparameter Chaos, Bifurcations, and Their Control 17

the region of control. While this algorithm is effective, it ld fo generally requires good knowledge of the equations gov- ni a erning the system, so that computing Δ𝑝 by (eq. 23)is m 𝑘 Un le stab b possible. In the case where only time series data from the le m ta a S system are available, the delay-coordinate technique may nifo ld be used to construct a faithful dynamical model for control Target (25).

Engineering Feedback Controls. From a control theoretic k + 1 iterate point of view, if only suppression of chaos is concerned, without chaos control may be considered as a particular nonlinear perturbation control problem, and so may not be much harder than con- ventional nonlinear systems control. This was not clear a k th iterate few years ago, however, since even if chaos could be con- trolled was questionable in the old days. k + 1 iterate with parameter A distinctive characteristic of control engineering is perturbation that it always employs some kind of feedback mechanism. In fact, feedback is pervasive in modern control theories Figure 15. Schematic diagram for the parametric variation con- and technologies. For instance, the parametric variation trol method. control method discussed above is a special type of feed- back control method by its very nature. In engineering con- ∗ 𝑝𝑘 = 𝑝 +Δ𝑝𝑘 has to be chosen such that trol systems, conventional feedback controllers are used to be designed for non-chaotic systems. In particular, lin- 𝐠⊤ Δ𝜉 = 𝐠⊤ (𝜉 − 𝜉∗ )=0 u,𝑘 𝑘+1 u,𝑘 𝑘+1 𝑓 ear feedback controllers are typically designed for linear This simply means that the direction of the next iteration is systems. It has been widely experienced that with care- perpendicular to the direction of the current local unstable ful designs of various conventional controllers, controlling manifold. For this purpose, taking the inner product of chaotic systems using feedback strategies is not only pos- equation 21 with 𝑔𝑢,𝑘 and using equation 22 yields sible but indeed quite successful. One basic reason for the success is that chaotic systems, although nonlinear and 𝐠⊤ 𝜉 ,𝑘Δ 𝑘 sensitive to initial conditions with complex dynamical be- Δ𝑝 =−𝜆 u (23) 𝑘 u,𝑘 𝐠⊤ 𝐯 haviors, belong to deterministic systems without stochas- ,𝑘 𝑘 u tic components or random parameters. 𝑔⊤ 𝑣 ≠ where it is assumed that u,𝑘 𝑘 0. This is the control for- mula for determining the variations of the adjustable sys- Some Features of Feedback Control. Feedback is one of tem parameter 𝑝 at each step, 𝑘 =1, 2, …. The controlled the most fundamental principles prevalent in the world. 𝜉∗ orbit thus is expected to approach 𝑓 at a geometrical The idea of using feedback, originated from Isaac Newton 𝜆 rate, s. and Gottfried Leibniz some 300 years ago, has been applied Note that this calculated Δ𝑝𝑘 is used to adjust the pa- in various forms to natural science and modern technology. 𝑝 𝑝 ≤ 𝑝 𝑝 > 𝑝 rameter only if Δ 𝑘 Δ max.WhenΔ 𝑘 Δ max, however, One basic feature of conventional feedback control is, 𝑝 𝜉 one should set Δ 𝑘 =0. Also, when 𝑘+1 falls on a local stable while achieving target tracking, that it can guarantee the 𝜉∗ 𝑝 manifold of 𝑓 ,onesetΔ 𝑘 =0, because the stable manifold stability of the controlled system even if the original uncon- would lead the orbit directly to the target. trolled system is unstable. This implies its intrinsic robust- Note also that the above derivation is based on the as- ness against external disturbances or internal variations sumption that the Poincaremap,´ 𝑃 , always possesses a to a certain extent, which is desirable and often necessary stable and an unstable direction (saddle-type orbits). This for the well performance of a control system. The intu- may not be the case in many systems, particularly those ition of feedback control always consuming strong control with higher periodic orbits. Moreover, it is necessary that energy perhaps lead to a false impression that feedback the number of accessible parameters for control is at least mechanism may not be suitable for chaos control due to equal to the number of unstable eigenvalues of the periodic the extreme sensitive nature of chaos. However, feedback orbit to be stabilized. In particular, when some of such key control under certain optimality criteria, such as a min- system parameters are unaccessible, the algorithm is not imum control energy requirement, can provide the best applicable or has to be modified. Also, if a system has multi- possible performance including the lowest control energy attractors, the system orbit may never return to the opened consumption. This is not only supported by theory but also window but, instead, move to another nontarget (attract- confirmed by experiments with comparison. ing) limit set. In addition, the technique is successful only Another advantage of using feedback control is that it if the control is applied after the system orbit moves into normally does not change the structure and parameters the small window covering the target orbit, over which the of the given system and, whenever the feedback is dis- local linear approximation is still valid. In this case, the connected, the given system retains the original form and waiting time can be quite long for some chaotic systems. dynamics without modification. In most engineering ap- To improve this situation, a method called targeting may plications, the system parameters are not accessible or not help persuade the system dynamics to quickly approach allowed for direct tuning and modifying. In such cases, 18 Chaos, Bifurcations, and Their Control

parameter variation method cannot be used, but feedback control provides a practical approach. An additional advantage of feedback control is its automatic fashion in processing control tasks without further human interference after being designed and implemented. As long as a feedback controller is correctly designed to satisfy the stability criteria and performance specifications, it works off hand. This is important for au- Figure 16. Configuration of a general feedback control system. tomation, reducing the dependence on human operator’s skills and avoiding human errors in monitoring control. 𝐫 𝐱 with a given target trajectory { 𝑘} and initial state 0, find A shortcoming of feedback control methods employing a (nonlinear) controller tracking errors is the explicit or implicit use of reference signals. This has never been a problem in conventional 𝐮𝑘 = 𝐠𝑘(𝐱𝑘) (29) feedback control of non-chaotic systems where the refer- to achieve the tracking-control goal: ence signals are always some designated, well-behaved ones. However, in chaos control, typically a reference sig- lim || 𝐱𝑘 − 𝑟𝑘 || =0 (30) nal is an unstable equilibrium or unstable limit cycle, 𝑘→∞ which is difficult if not impossible to be implemented phys- A closed-loop continuous-time feedback control system ically as a reference input. This critical issue has stimu- has a configuration as shown in Figure 16,where𝐞𝐱 ∶= lated some new research efforts, for instance, to use an 𝐫 − 𝐱 is the tracking error, 𝐟 is the given system, and 𝐠 is auxiliary reference as control input in a self-tuning feed- the feedback controller to be designed, in which 𝐟 and 𝐠 back manner. can be either linear or nonlinear. In particular, for a linear Engineering feedback control approaches have seen an system in the state-space form in connection with a linear alluring future in more advanced theories and applications state-feedback controller, namely, in controlling complex dynamics. Utilization of feedback is ( ) among the most inspiring concepts that engineering mind 𝐱̇ 𝐴 𝐱 𝐵 𝐮 𝐴 𝐱 𝐵𝐾 𝐫 𝐱 , = + = + c − has ever contributed to modern sciences and advanced technologies. 𝐾 where c is a constant control gain matrix to be deter- mined, the closed-loop configuration is shown in Figure 17. A Typical Feedback Control Problem. Only the simple tracking control problem of suppressing chaos is discussed A Control Engineer’s Perspective. In controllers design, here for illustration. particularly in finding a nonlinear controller for a system, A general feedback approach to controlling a dynamical it is important to emphasize that the designed controller system, not necessarily chaotic nor even nonlinear, can be should be (much) simpler than the given system to make illustrated by starting from the following general form of sense of the world. an 𝑛-dimensional control system: For instance, suppose that one wants to find a nonlin- ear controller, 𝑢(𝑡), in the continuous-time setting, to guide 𝐱̇ (𝑡)=𝐟(𝐱, 𝐮,𝑡) , 𝐱(0) = 𝐱 (24) [ ]⊤ 0 𝐱 𝑡 𝑥 𝑡 , ,𝑥 𝑡 the state vector ( )= 1( ) … 𝑛( ) of a given nonlinear 𝐱 𝑡 𝐮 𝐱 where ( ) is the system state, is the controller, 0 is control system: a given initial state, and 𝐟 is a piecewise continuous or smooth nonlinear function. ⎧ 𝑥̇ (𝑡)=𝑥 (𝑡) ⎪ 1 2 Given a reference signal, 𝐫(𝑡), which can be either a con- 𝑥̇ 𝑡 𝑥 𝑡 ⎪ 2( )= 3( ) stant (set point) or a function (time-varying trajectory), ⎨ ⋮ the automatic feedback control problem is to design a con- ⎪ ( ) ⎪ 𝑥̇ 𝑡 𝑓 𝑥 𝑡 , ,𝑥 𝑡 𝑢 𝑡 troller in the following state-feedback form: ⎩ 𝑛( )= 1( ) … 𝑛( ) + ( )

𝐮(𝑡)=𝐠(𝐱,𝑡) (25) to a target state, 𝑦,namely, where 𝐠 is generally a piecewise continuous nonlinear func- 𝐱(𝑡) → 𝐲 as 𝑡 → ∞ . tion, such that the feedback-controlled system ( ) 𝐱̇ (𝑡)=𝐟 𝐱, 𝐠(𝐱,𝑡),𝑡 (26)

can achieve the goal of tracking:

lim || 𝐱(𝑡)−𝐫(𝑡) || =0 (27) 𝑡→∞ For discrete-time systems, the problem and notation are similar. More precisely, for a system

𝐱 𝐟 𝐱 , 𝐮 Figure 17. Configuration of a state feedback control system. 𝑘+1 = 𝑘( 𝑘 𝑘) (28) Chaos, Bifurcations, and Their Control 19

It is then mathematically straightforward to use the “con- Subtracting equation 31 from equation 24 yields the error troller” dynamics: ( ) ( ) ̇ ̄ 𝑢 𝑡 𝑓 𝑥 𝑡 , ,𝑥 𝑡 𝑘 𝑥 𝑡 𝑦 𝐞𝐱 = 𝐟𝐞(𝐞𝐱, 𝑚𝑎𝑡ℎ𝑏𝑓𝑥,) 𝑡 (32) ( )=− 1( ) … 𝑛( ) + c 𝑛( )− 𝑛 𝑘 < where with an arbitrary constant c 0. This controller leads to ( ) ( ) 𝐞 (𝑡)=𝐱(𝑡)−𝐱̄(𝑡) , 𝐟 (𝐞 , 𝐱̄,𝑡)=𝐟 𝐱, 𝐠(𝐱, 𝐱̄(𝑡),𝑡) − 𝐟(𝐱̄,𝑡) 𝑥̇ 𝑡 𝑘 𝑥 𝑡 𝑦 𝐱 𝐞 𝐱 𝑛( )= c 𝑛( )− 𝑛 Here, it is important to note that in order to perform 𝑒 𝑡 𝑥 𝑡 𝑦 → 𝑡 → which yields 𝑛( )∶= 𝑛( )− 𝑛 0 as ∞. Overall, it re- correct stability analysis later on, in the error dynamical sults in a completely controllable linear system, so that system (eq. 32) the function 𝐟𝐞 must not explicitly contain 𝐱; 𝐱(𝑡) → 𝐲 as 𝑡 → ∞. if so, 𝐱 should be replaced by 𝐞𝐱 + 𝐱̄. This is because system Another example is that for the control system 𝐞 𝐱 ( ) (eq. 32) should only contain the dynamics of 𝐱 but not , while the system may contain 𝐱̄ that merely is a specified 𝐱̇ (𝑡)=𝐟 𝐱(𝑡),𝑡 + 𝐮(𝑡) , time function but not a system variable. use Thus, the design problem becomes to determine the con- ( ) troller, 𝐮(𝑡), such that 𝐮(𝑡)=−𝐟(𝐱(𝑡),𝑡)+𝐲̇ (𝑡)+𝐾 𝐱(𝑡)−𝐲(𝑡)

lim || 𝐞𝐱(𝑡) || =0 (33) with a stable constant gain matrix 𝐾, to drive its trajectory 𝑡→∞ to the target 𝐲(𝑡) directly. which implies that the goal of tracking control described This kind of “design,” however, is unimplementable, be- by equation 27 is achieved. cause the controller is even more complicated than the It is clear from equations 32 and 33 that if zero is an given system. Physically, it needs to cancel the nonlinear equilibrium of the error dynamical system (eq. 32), then elements in the given system, which means to remove part the original control problem has been converted to the of the given machine, and then put a new part into the asymptotic stability problem for this equilibrium. As a re- given system, which will change the given system to an- sult, Lyapunov stability methods and theorems can be di- other one. rectly applied or modified to obtain rigorous mathematical In the discrete-time setting, for a given nonlinear sys- techniques for the controllers design, as discussed in more 𝐱 𝐟 𝐱 𝐮 tem, 𝑘+1 = 𝑘( 𝑘)+ 𝑘, one may also find a similar nonlin- detail in the following section. ear feedback controller, or even simpler, use 𝐮 =−𝐟𝑘(𝐱𝑘)+ 𝐠 𝐱 𝐱 𝑘( 𝑘) to achieve any desired dynamics satisfying 𝑘+1 = Chaos Control via Lyapunov Methods. The key in apply- 𝐠 𝐱 𝑘( 𝑘) in just one step! ing the Lyapunov second method to a nonlinear dynamical Simple “mathematical tricks” like these are certainly system is to construct a Lyapunov function that describes not of any engineering design, nor any valuable method- some kind of energy governing the system motion. If this ology, for any real-world application other than illusive function is constructed appropriately, so that it decays computer simulations. monotonically to zero as time evolves, then the system Therefore, in designing a feedback controller, it is very motion, which falls on the surface of this decaying func- important to come out with a simplest possible working tion, will be asymptotically stabilized to zero. A controller, controller: If a linear controller can be designed to do the then, may be designed to force this Lyapunov function of job, use a linear controller; otherwise, try the simplest the system, stable or not originally, to decay to zero. As possible nonlinear controllers (starting, for example, from a result, the stability of tracking error equilibrium, hence piecewise linear or quadratic controllers). Whether or not the original goal of trajectory tracking, is achieved. can one find a simple, physically meaningful, easily im- For a chaos control problem with a target trajectory 𝐱̄, plementable, low-cost, and effective controller for a desig- typically an unstable periodic solution of the given system, nated control task can be quite technical, which relies on a design can be carried out by determining the controller the designer’s theoretical background and practical expe- 𝑢(𝑡) via the Lyapunov second method such that the zero rience. equilibrium of the error dynamics, 𝐞𝐱 =0, is asymptotically stable. In this approach, since a linear feedback controller A General Approach to Feedback Control of Chaos. To alone is usually not sufficient for the control of a nonlinear outline the basic idea of a general feedback approach to system, particularly a chaotic one, it is desirable to find chaos suppression and tracking control, consider system some criteria for the design of simple nonlinear feedback (eq. 24) that is now assumed to be chaotic and possess an controllers. 𝐱̄ 𝑡 > unstable periodic orbit (or equilibrium), ,ofperiod p 0, In so doing, consider the feedback controller candidate 𝐱̄ 𝑡 𝑡 𝐱̄ 𝑡 𝑡 ≤ 𝑡< namely, ( + p)= ( ), 0 ∞. The task is to design a of the form feedback controller in the form of equation 25, such that ̄ 𝐮 𝑡 𝐾 𝐱 𝐱̄ 𝐠 𝐱 𝐱̄, 𝐤 ,𝑡 the tracking control goal (eq. 26), with 𝐫 = 𝐱 therein, is ( )= c( − )+ ( − c ) (34) achieved. where 𝐾 is a constant matrix, which can be zero, and 𝐠 Since the target periodic orbit 𝐱̄ is itself a solution of the c original system, it satisfies is a simple nonlinear function with constant parameters 𝐤 𝐠 ,𝑘 ,𝑡 𝑡 ≥ 𝑡 𝐾 𝐤 𝑐, satisfying (0 𝑐 )=0for all 0.Both c and c are 𝐱̄̇ = 𝐟(𝐱̄,𝑡) (31) determined in the design. Adding this controller to the 20 Chaos, Bifurcations, and Their Control

given system gives that the fundamental matrix (consisting of 𝑛 independent solution vectors) has the expression 𝐱̇ 𝐟 𝐱,𝑡 𝐮 𝐟 𝐱,𝑡 𝐾 𝐱 𝐱̄ 𝐠 𝐱 𝐱̄, 𝐤 ,𝑡 = ( )+ = ( )+ c( − )+ ( − c ) (35) Φ(𝐱̄,𝑡)=𝑀(𝐱̄,𝑡) 𝑒𝑡𝑄 The controller is required to drive the orbit of the controlled 𝑡 𝑄 system (eq. 35) to approach the target trajectory 𝑥̄. The eigenvalues of the constant matrix 𝑒 p are called the The error dynamics (eq. 32) now takes the following Floquet multipliers of the system matrix 𝐴(𝐱̄,𝑡). form: In system (eq. 37), assume that 𝐡(0,𝐾 , 𝐤 ,𝑡)=0and 𝐞̇ 𝐟 𝐞 ,𝑡 𝐾 𝐞 𝐠 𝑒 , 𝐤 ,𝑡 c c 𝑥 = 𝐞( 𝐱 )+ c 𝑥 + ( 𝑥 c ) (36) 𝐡 𝐞 ,𝐾 , 𝐤 ,𝑡 𝜕𝐡 𝐞 ,𝐾 , 𝐤 ,𝑡 𝜕𝐞 ( 𝐱 c c ) and ( 𝐱 c c )∕ 𝐱 both are contin- 𝑅𝑛 where uous in a bounded neighborhood of the origin in . Assume also that

𝐞𝐱 = 𝐱 − 𝐱̄ , 𝐟𝐞(𝐞𝐱,𝑡)=𝐟(𝑒𝐱 + 𝐱̄,𝑡)−𝐟(𝐱̄,𝑡) || 𝐡(𝑒𝐱,𝐾 , 𝐤 ,𝑡) || lim c c =0 ||𝐞 ||→ 𝐟 ,𝑡 𝑡 𝑡 , 𝐞 𝐱 0 || 𝑒𝐱|| It is clear that 𝐞(0 )=0for all ∈[0 ∞),namely, 𝑥 =0 is an equilibrium of the tracking-error dynamical system 𝑡 𝑡 , uniformly with respect to ∈[0 ∞). If the nonlinear (eq. 36). controller (eq. 34) is designed such that all Floquet Next, Taylor expand the right-hand side of the con- multipliers {𝜆𝑖} of the system matrix 𝐴(𝐱̄,𝑡) satisfy trolled system (eq. 36)at𝐞𝐱 =0(i.e., at 𝐱 = 𝐱̄) and remem- | | ber that the nonlinear controller will be designed to satisfy | 𝜆𝑖(𝑡) | < 1 ,𝑖=1, … ,𝑛, ∀ 𝑡 ∈[𝑡 , ∞) 𝐠 , 𝐤 ,𝑡 | | 0 (0 c )=0. Then, the error dynamics are reduced to then the controller will drive the chaotic orbit 𝐱 of the 𝐞̇ 𝐴 𝐱̄,𝑡 𝐞 𝐡 𝐞 ,𝐾 , 𝐤 ,𝑡 𝐱 = ( ) 𝐱 + ( 𝐱 c c ) (37) controlled system (eq. 35) to the target trajectory, 𝐱̄, as 𝑡 → ∞. where [ ] 𝜕𝐟 (𝑒 ,𝑡) 𝐴 𝐱̄,𝑡 𝐞 𝐱 Various Feedback Methods for Chaos Control. In addi- ( )= 𝜕𝐞 𝐱 𝐞𝐱=0 tion to the general nonlinear feedback control approach described above, adaptive and intelligent controls are two 𝐡 𝐞 ,𝐾 , 𝐤 ,𝑡 and ( 𝐱 c c ) contains the rest of the Taylor expansion. large classes of engineering feedback control methods The design is then to determine both the constant con- that have been shown to be effective for chaos control. 𝐾 𝐤 trol gains c and c as well as the nonlinear function Other successful feedback control methods include optimal 𝐠 ⋅, ⋅,𝑡 ( ), based on the linearized model (eq. 37), such that control, sliding-mode, and robust controls (e.g., 𝐻 ∞ con- 𝐞 → 𝑡 → 𝐱 0 as ∞. When this controller is applied to the orig- trol), digital controls, occasionally proportional and time- inal system, the goal of both chaos suppression and target delayed feedback controls, and so on. Linear feedback con- tracking will be achieved. trols are also useful, but generally for simple chaotic sys- For illustration, two controllability conditions estab- tems. Various variants of classical control methods that lished based on the boundedness of the chaotic attractors have demonstrated great potentials for controlling chaos as well as the Lyapunov first and second methods, respec- include distortion control, dissipative energy method, ab- tively, are summarized below (5). sorber as a controller, external weak periodic forcing, Kolmogorov–Sinai entropy reduction, stochastic controls, 𝐡 ,𝐾 ,𝑡 In system (eq. 37), suppose that (0 c )=0 and chaos filtering, and so on (5). 𝐴(𝐱̄,𝑡)=𝐴 is a constant matrix with eigenvalues hav- Finally, it should be noted that there are indeed many ing negative real parts, and let 𝑃 be the positive defi- valuable ideas and methodologies that, by their nature nite and symmetric matrix solution of the Lyapunov and forms, cannot be well classified into any of the afore- equation: mentioned categories, not to mention that many novel ap- proaches are still emerging, improving, and developing (5). 𝑃𝐴+ 𝐴⊤𝑃 =−𝐼 𝐼 𝐾 where is the identity matrix. If c is designed to 5. CONTROLLING BIFURCATIONS satisfy Ordering chaos via bifurcations control has never been a || 𝐡 𝐞 ,𝐾 , 𝐤 ,𝑡 || ≤ 𝑐 || 𝐞 || ( 𝐱 c c ) 𝐱 subject in conventional control studies. This seems to be a 𝑐< 1 𝜆 𝑃 𝑡 ≤ 𝑡< unique approach valid only for those nonlinear dynamical for a constant max( ) for 0 ∞,where 𝜆 𝑃 2 𝑃 systems that possess some special characteristics with a max( ) is the maximum eigenvalue of , then the controller 𝐮(𝑡), defined in equation 34, will drive the route to chaos from bifurcations. orbits 𝐱 of the controlled system (eq. 35)tothetarget trajectory, 𝐱̄,as𝑡 → ∞. 5.1. Why Bifurcation Control Bifurcation and chaos are often twins and, in particu- 𝐱̄ 𝑡 For system (eq. 37), since is p periodic, associated lar, period-doubling bifurcation is a route to chaos. Hence, 𝐴 𝐱̄,𝑡 𝑡 with the matrix ( ),therealwaysexista p periodic by monitoring and managing bifurcations, one can expect nonsingular matrix 𝑀(𝑥,̄ ) 𝑡 and a constant matrix 𝑄,such achieving certain type of control over chaotic dynamics. Chaos, Bifurcations, and Their Control 21

Even bifurcation control itself is very important. In some physical systems such as a stressed system, delay- ( ) 𝑄 𝐽 𝑄 𝐱, 𝐱 𝑝 𝑄 𝐽 𝐱,𝐽 𝐱 𝑝 ing bifurcations offers an opportunity to obtain stable op- 0 = (0) ( ; )+ (0) (0) ; erating conditions for the machine beyond the margin of 𝐶 𝐽 𝐶 𝐱, 𝐱, 𝐱 𝑝 0 = (0) ( ; ) operability in a normal situation. Also, relocating and en- ( ) suring stability of bifurcated limit cycles can be applied +2𝑄 𝐽(0)𝐱,𝑄(𝐱, 𝐱; 𝑝) to some conventional control problems, such as thermal ( ) convection, to obtain better results. Other examples in- +𝐶 𝐽(0)𝐱,𝐽(0)𝐱,𝑄(𝐱, 𝐱; 𝑝); 𝑝 clude stabilization of some critical situations for tethered [ ] satellites, magnetic bearing systems, voltage dynamics of −1 𝐽 − 𝐽 ⊤ 𝐽 𝐥𝐥⊤ 𝐽 ⊤ electric power systems, and compressor stall in gas turbine 0 = (0) (0) + (0) jet engines (5). Bifurcation control essentially means to design a con- Now, consider system (eq. 38) with a control input: troller for a system to obtain some desired behaviors such ( ) 𝐱 𝐟 𝐱 𝑝, 𝑢 ,𝑘 , , as stabilization of bifurcated dynamics, to modify some 𝑘+1 = 𝑘; 𝑘 =0 1 … properties of bifurcations, or to tame chaos via controlling bifurcations. Typical examples include delaying the onset whichisassumedtosatisfythesameassumptionswhen 𝐮 of an inherent bifurcation, relocating an existing bifurca- 𝑘 =0. If the critical eigenvalue −1 is controllable for the tion, changing the shape or type of a bifurcation, introduc- associated linearized system, then there is a feedback con- 𝐮 𝐱 ing a bifurcation at a desired parameter value, stabilizing troller, 𝑘( 𝑘), containing only third-order terms in the 𝐱 (at least locally) a bifurcated periodic orbit, optimizing the components of 𝑘, such that the controlled system has a 𝑝 performance near a bifurcation point for a system, or a locally stable bifurcated period-two orbit for near zero. 𝑝 combination of some of these. Such tasks have practical Also, this feedback stabilizes the origin for =0.If,how- values and great potentials in many nontraditional real- ever, −1 is uncontrollable for the associated linearized sys- 𝐮 𝐱 world applications. tem, then generically there is a feedback controller, 𝑘( 𝑘), containing only second-order terms in the components of 𝐱𝑘, such that the controlled system has a locally stable 5.2. Bifurcation Control via Feedback bifurcated period-two orbit for 𝑝 near 0. Moreover, this Bifurcations can be controlled by different methods, among feedback controller stabilizes the origin for 𝑝 =0(8). which the feedback strategy is especially effective. For illustration, consider a general discrete-time para- 5.3. Bifurcation Control via Harmonic Balance metric nonlinear system: ( ) For continuous-time systems, limit cycles in general can- not be expressed in analytic forms, and so limit cycles cor- 𝐱 = 𝐟 𝐱 ; 𝑝 ,𝑘=0, 1, … (38) 𝑘+1 𝑘 responding to period-two orbits in the period-doubling bi- furcation diagram have to be approximated in analysis and where 𝐟 is assumed to be sufficiently smooth with respect to applications. In this case, the harmonic balance approxi- both the state 𝐱 ∈ 𝑅𝑛 and the parameter 𝑝 ∈ 𝑅, and has an 𝑘 mation technique (14) can be applied, which is also useful equilibrium at (𝐱̄,̄𝑝)=(0, 0). In addition, assume that the for controlling bifurcations such as delaying or stabilizing Jacobian of the linearized system of equation 38,evaluated the onset of period-doubling bifurcations (26). at the equilibrium that is the continuous extension of the Consider a feedback control system in the Lur’e form origin, has an eigenvalue 𝜆 (𝑝) satisfying 𝜆 (0) = −1 and 1 1 described by 𝜆′ ≠ (0) 0, while all remaining eigenvalues have magnitude ( ) strictly less than 1. Under these conditions, the nonlinear 𝐟 ∗ 𝐠◦𝐲 + 𝐾 𝑦 + 𝐲 =0 function has a Taylor expansion: c ( ) where ∗ and ◦ represent the convolution and composition 𝐟 𝐱 𝑝 𝐽 𝑝 𝐱 𝑄 𝐱,𝑥 𝑝 𝐶 𝐱, 𝐱, 𝐱 𝑝 ⋯ ; = ( ) + ( ; )+ ( ; )+ operations, respectively, as shown in Figure 18. First, suppose that a system 𝑆 = 𝑆(𝐟, 𝐠) is given as where 𝐽(𝑝) is the parametric Jacobian, and 𝑄 and 𝐶 are 𝐾 shown in the figure without the feedback controller, c. quadratic and cubic terms in symmetric bilinear and tri- 𝑝 𝑝 Assume also that two system parameter values, h and c, linear forms, respectively. are specified, which define a Hopf bifurcation and a su- This system has the following property (8):Aperiod- percritical predicted period-doubling bifurcation, respec- doubling orbit can bifurcate from the origin of system (eq. tively. Moreover, assume that the system has a family of 𝑝 38)at =0; the period-doubling bifurcation is supercritical predicted first-order limit cycles, which are stable within and stable if 𝛽<0 but is subcritical and unstable if 𝛽>0, 𝑝 <𝑝<𝑝 the range of h c. where Under this framework, the objective now is to de- [ ( )] sign a feedback controller, 𝐾 , added to the system as 𝛽 =2𝐥⊤ 𝐶 (𝐫, 𝐫, 𝐫; 𝑝)−2𝑄 𝐫,𝐽−𝑄 (𝐫, 𝐫; 𝑝) c 0 0 0 0 shown( in Figure) 18, such that the controlled system, ∗ ∗ ⊤ 𝑆 = 𝑆 𝐟, 𝐠,𝐾 , has the following properties: in which 𝐥 is the left eigenvector and 𝐫 is the right c eigenvector of 𝐽(0), respectively, both associated with the 𝑆∗ 𝑝∗ 𝑝 eigenvalue −1,and (i) has a Hopf bifurcation at h = h. 22 Chaos, Bifurcations, and Their Control

be conveniently accomplished in the frequency domain set- ting. Again, consider the feedback system (eq. 15), which can be illustrated by a variant of Figure 18. For harmonic ex- pansion of the system output, 𝐲(𝑡), the first-order formula is (14) 𝐲1 𝜃 𝐫 𝜃3𝑧 𝜃5𝐳 ⋯ = + 13 + 15 + Figure 18. A feedback system in the Lur’e form. where 𝜃 isshowninFigure12, 𝑟 is defined in equation 𝐳 , ,𝑧 𝐫 𝑚 16, and 13 … 1,2𝑚+1 are some vectors orthogonal to , = ∗ (ii) 𝑆 has a supercritical predicted period-doubling bi- 1, 2, …, given by explicit formulas (14). 𝑝∗ >𝑝 furcation for c c. Observe that for a given value of 𝜔̂, defined in the graph- (iii) 𝑆∗ has a one-parameter family of stable predicted ical Hopf theorem, the SISO system transfer function sat- 𝑝∗ <𝑝<𝑝∗ limit cycles for h c . isfies (iv) 𝑆∗ has the same set of equilibria as 𝑆. 1 𝐻(𝑗̂𝜔)=𝐻(𝑠)+(−𝛼 + 𝑗𝛿𝜔)𝐻 ′(𝑠)+ (−𝛼 + 𝑗𝛿𝜔)2𝐻 ′′(𝑠)+⋯ 2 Only the one-dimensional case is discussed here for il- (39) lustration. First, one can design a washout filter with the transfer function 𝑠∕(𝑠 + 𝑎),where𝑎>0, such that it pre- where 𝛿𝜔 = 𝜔̂ − 𝜔,with𝜔 being the imaginary part of the serves the equilibria of the given nonlinear system. Then, bifurcating eigenvalues, and 𝐻 ′(𝑠) and 𝐻 ′′(𝑠) are the first note that any predicted first-order limit cycle can be well and second derivatives of 𝐻(𝑠), defined in equation 15, approximated by respectively. On the other hand, with the higher order ap- proximations, the following equation of harmonic balance 𝑦(1) 𝑡 𝑦 𝑦 𝜔𝑡 ( )= 0 + 1 sin( ) can be derived: 𝑚 𝑚 In so doing, the controller transfer function becomes ∑ ∑ 𝐻 𝑗𝜔𝐽 𝐼 𝐳 𝜃2𝑖+1 𝐻 𝑗𝜔 𝐫 𝜃2𝑖+1 ( ) [ ( ) + ] 1,2𝑖+1 =− ( ) 1,2𝑖+1 𝑠 𝑠2 𝜔2 𝑝 𝑖=0 𝑖=1 + ( h) 𝐾 (𝑠)=𝑘 where 𝐳 = 𝐫 and 𝐫 , 𝑚 = 𝐡𝑚, 𝑚 =1, 2, …,inwhich𝐡 has c c 𝑠 𝑎 3 11 1 2 +1 1 ( + ) the formula shown in equation 16, and the others also have 𝑘 𝜔 𝑝 where c is the constant control gain and ( h) is the fre- explicit formulas (14). quency of the limit cycle emerged from the Hopf bifur- In a general situation, the following equation has to be 𝑝 𝑝 solved: cation at the point = h. This controller also preserves ( ) the Hopf bifurcation at the same point. More importantly, [𝐻(𝑗̂𝜔)𝐽 + 𝐼] 𝐫 𝜃 + 𝑧 𝜃3 + 𝐳 𝜃5 + ⋯ since 𝑎>0, the controller is stable, so by continuity in a 13 15 [ ] small neighborhood of 𝑘 , the Hopf bifurcation of 𝑆∗ not c 𝐻 𝑗̂𝜔 𝐡 𝜃3 𝐡 𝜃5 ⋯ . (40) only remains supercritical but also has a supercritical pre- =− ( ) 1 + 2 + dicted period-doubling bifurcation (say, at 𝑝 𝑘 ,closeto c( c) In so doing, by substituting equation 39 into equation 40, 𝑝 h) and a one-parameter family of stable predicted limit one obtains the expansion 𝑝 <𝑝<𝑝 𝑘 cycles for h c( c). 𝑘 𝛼 𝑗𝛿𝜔 𝛾 𝜃2 𝛾 𝜃4 𝛾 𝜃6 𝛾 𝜃8 𝑂 𝜃9 The design is then to determine c such that the pre- ( − )= 1 + 2 + 3 + 4 + ( ) (41) dicted period-doubling bifurcation can be delayed to a de- in which all the coefficients 𝛾 , 𝑖 =1, 2, 3, 4, can be calculated sired parameter value 𝑝∗. For this purpose, the harmonic 𝑖 c explicitly (14). Then, taking the real part of equation 41 balance approximation method (14) is useful, which leads gives 𝑦(1) 𝑦 𝑦 𝜔 to a solution of by obtaining values of 0, 1,and 𝑝 𝑘 𝑎 𝛼 𝜎 𝜃2 𝜎 𝜃4 𝜎 𝜃6 𝜎 𝜃8 ⋯ (they are functions of , depending on c and , within the =− 1 − 2 − 3 − 4 − range 𝑝 <𝑝<𝑝∗). The harmonic balance also yields con- h c where 𝜎 =−ℜ{𝛾 } are the curvature coefficients of the ex- ditions, in terms of 𝑘 , 𝑎, and a new parameter, for the 𝑖 𝑖 c pansion. period-doubling prediction to occur at the point 𝑝∗.Tothis c To this end, notice that multiple limit cycles will emerge end, the controller design can be completed by choosing a when the curvature coefficients are varied near the value suitable value for 𝑘 to satisfy such conditions (26). c zero, after alternating the signs of the curvature coeffi- cients in increasing (or decreasing) order. For example, to 5.4. Controlling Multiple Limit Cycles have four limit cycles in the vicinity of a type of degener- 𝜎 𝜎 𝜎 𝜎 ≠ As indicated by the Hopf bifurcation theorem, limit cycles ate Hopf bifurcation that has 1 = 2 = 3 =0but 4 0 at are frequently associated with bifurcations. In fact, one the criticality, the system parameters have to be varied in 𝛼> 𝜎 < 𝜎 > 𝜎 < type of degenerate (or singular) Hopf bifurcations (when such a way that, for example, 0, 1 0, 2 0, 3 0, 𝜎 > some of the conditions stated in the Hopf theorems are not and 4 0. This condition suggests a methodology for con- satisfied) determines the birth of multiple limit cycles un- trolling the birth of multiple limit cycles associated with der system parameters variation. Hence, the appearance degenerate Hopf bifurcations. of multiple limit cycles can be controlled by managing the One advantage of this methodology is that there is no corresponding degenerate Hopf bifurcations. This task can need to modify the feedback control path by adding any Chaos, Bifurcations, and Their Control 23 nonlinear components, in order to drive the system orbit 6.2. Some Approaches to Anticontrolling Chaos to a desired region: One can simply modify the system pa- Anticontrol of chaos is a new research direction, which has rameters, a kind of parametric variation control, accord- just started to develop. Different methods for anticontrol- ing to the expressions of the curvature coefficients, so as ling chaos are quite possible (5), but only two preliminary to achieve the goal of controlling bifurcations and limit approaches are presented here for illustration. cycles. Preserving Chaos by Small Control Perturbations. Con- sider an 𝑛-dimensional discrete-time nonlinear system: 6. ANTICONTROL OF CHAOS ( ) 𝐱 𝐟 𝐱 ,𝑝,𝑢 𝑘+1 = 𝑘 𝑘 Anticontrol of chaos, in contrast to the main stream of 𝐱 𝑢 ordering or suppressing chaos, is to make a nonchaotic dy- where 𝑘 is the system state, 𝑘 is a scalar-valued control namical system chaotic, or to retain/enhance the existing input, 𝑝 is a variable parameter, and 𝐟 is a locally invertible 𝑢 chaos of a chaotic system. Anticontrol of chaos as one of nonlinear map. Assume that with 𝑘 =0, the system orbit the unique features of chaos control has emerged as a the- behaves chaotically at some value of 𝑝,andthatwhen𝑝 𝑝 oretically attractive and potentially useful new subject in increases and passes a critical value, c, inverse bifurcation systems control theory and time-critical or energy-critical emerges leading the chaotic state to becoming periodic. high-performance applications. Within the biological context, such a bifurcation is of- ten undesirable: There are many cases where the loss of complexity and the emergence of periodicity are associated with pathology (dynamical disease). The question, then, is whether it is possible (and, if so, how) to keep the system 𝑝>𝑝 6.1. Why Anticontrol of Chaos state chaotic even if c, by using small control inputs, {𝑢𝑘}. Chaos has long been considered as a disaster phenomenon It is known that there are at least three common bi- and so is very fearsome to many. However, chaos “is dy- furcations that can lead chaotic motions directly to low- namics freed from the shackles of order and predictabil- periodic attracting orbits: (i) crises, (ii) saddle-node type of ity.” Under good conditions or suitable control, it “permits intermittency, and (iii) inverse period-doubling type of in- systems to randomly explore their every dynamical possi- termittency. Here, crises refer to a sudden change caused bility. It is exciting variety, richness of choice, a cornucopia by the collision of an attractor with an unstable periodic or- of opportunities” (27). bit; intermittency is a special route to chaos, where regular Today, chaos theory has been anticipated to be poten- orbital behavior is intermittently interrupted by a finite- tially useful in many novel and time- or energy-critical duration “burst,” in which the orbit behaves in a decidedly applications. In addition to those potential utilization of different fashion; inverse period-doubling bifurcation has chaos that mentioned earlier in the discussion of chaos a diagram in reverse form as that shown in Figure 13 (i.e., control, to name but a few more, it is worth mentioning from chaos back to less and less bifurcating points, leading navigation in the multibody planetary system, secure in- back to a periodic motion), while the parameter remains formation processing via chaos synchronization, dynamic increasing. crisis management, critical decision-making in political, In all these cases, one can identify a loss region, 𝐺, economical, as well as military events, and so on In par- which has the property that after the orbit falls into 𝐺, ticular, it has been observed that a transition of a bio- it is rapidly drawn to a periodic orbit. Thus, a strategy to logical system’s state from being chaotic to being patho- retain chaos for 𝑝>𝑝𝑐 is to avoid this from happening by physiologically periodic can cause the so-called dynamical successively iterating 𝐺 in such a way that disease, and so is undesirable. Examples of dynamical dis- ( ) 𝐺 𝐟 −1 𝐺, 𝑝, eases include cell counts in hematological disorder, stim- 1 = 0 ulant drug-induced abnormalities in time of the patterns ( ) ( ) 𝐺 𝑓 −1 𝐺 ,𝑝, 𝐟 −2 𝐺, 𝑝, of brain enzymes, receptors, and animal explorations in 2 = 1 0 = 0 space, cardiac interbeat interval patterns in a variety of ⋮ cardiac disorders, the resting record in a variety of signal- ( ) −𝑚 sensitive biological systems following desensitization, ex- 𝐺𝑚 = 𝐟 𝐺, 𝑝, 0 perimental epilepsy, hormone release patterns correlated with the spontaneous mutation of a neuroendocrine cell to As 𝑚 increases, the width of 𝐺𝑚 in the unstable direction(s) a neoplastic tumor, prediction of immunological rejection has a general tendency to shrink exponentially. This sug- of heart transplants, electroencephalographic behaviors of gests the following control scheme (28): the human brain in the presence of neurodegenerative 𝑚 𝑚 disorder, neuroendocrine, cardiac, and electroencephalo- Pick a suitable value of , denoted 0. Assume that graphic changes with aging, and imminent ventricular fib- the orbit initially starts from outside the region rillation in human subjects, and so on (28). Hence, preserv- 𝐺𝑚 ∪ 𝐺𝑚 ∪ ⋯ ∪ 𝐺 ∪ 𝐺. If the orbit lands in 𝐺𝑚 0+1 0 1 0+1 ing chaos in these cases is important and healthy, which at iterate 𝓁, the control 𝑢𝓁 is applied to kick the orbit presents a real challenge for creative research on anticon- out of 𝐺𝑚 at the next iterate. Since 𝐺𝑚 is thin, this 0 0 trol of chaos (5). control can be very small. After the orbit is kicked 24 Chaos, Bifurcations, and Their Control

′ out of 𝐺𝑚 , it is expected to behave chaotically, until that all the Jacobians 𝐟 (𝐱𝑘) are uniformly bounded: 0 𝑘 it falls again into 𝐺𝑚 , and at that moment another 0+1 || || 𝐟 ′ 𝐱 ≤ 𝛾 < small control is applied, and so on. This procedure sup || 𝑘( 𝑘) || ∞ (43) ≤𝑘≤ || || f can keep the motion chaotic. 0 ∞ To come up with a design methodology, first observe { }𝑛 𝜃(𝑘) 𝑇 𝐱 Anticontrol of Chaos via State Feedback. An approach to that if 𝑖 are the singular values of the matrix 𝑘( 0), 𝑖=1 anticontrol of discrete-time systems can be made math- 𝑘 then 𝜃( ) ≥ 0 for all 𝑖 =1, … ,𝑛 and 𝑘 =0, 1, …,withthere- ematically rigorous by means of applying the engineer- 𝑖 lationship ing feedback control strategy. This anticontrol technique ( ) is first to make the Lyapunov exponents of the controlled 1 (𝑘) (𝑘) 𝜎𝑖 = lim ln 𝜃𝑖 for 𝜃𝑖 > 0 ,𝑖=1, … ,𝑛 system either strictly positive, or arbitrarily assigned (pos- 𝑘→∞ 𝑘 itive, zero, and negative in any desired order), and then ap- 𝑘 𝑘 ( ) (𝑘+1)𝜎𝑖 ( ) plying the simple mod-operations, which keeps the orbits Clearly, if 𝜃𝑖 =e is used in the design, then all 𝜃𝑖 to remain bounded (5,29). This task can be accomplished will not be zero for any finite values of 𝜎𝑖,forall𝑖 =1, … ,𝑛 𝑘 , , 𝑇 𝐱 for any given higher dimensional discrete-time dynamical and =0 1 …. Thus, 𝑘( 0) is always nonsingular. Con- system that could be originally nonchaotic or even asymp- sequently, a control-gain sequence {𝐵𝑘} can be designed totically stable. The argument used is purely algebraic and such that the singular values of the product matrix 𝑇𝑘(𝐱 ) { }𝑛 0 the design procedure is completely schematic, in which no are exactly equal to 𝑒𝑘𝜎𝑖 :atthe𝑘th step, 𝑘 =0, 1, 2, …, approximations are needed. 𝑖=1 one may choose the control gain matrix to be Specifically, consider a nonlinear dynamical system, not 𝜎 necessarily chaotic nor unstable to start with, in the gen- ⎡ 𝑒 1 ⎤ eral form of ⎢ ⎥ 𝐵 𝐟 ′ 𝐱 ⋱ ,𝑘 , , 𝑘 =− 𝑘( 𝑘)+⎢ ⎥ =0 1 … 𝐱 𝐟 𝐱 ⎢ 𝜎 ⎥ 𝑘+1 = 𝑘( 𝑘) (42) ⎣ 𝑒 𝑛 ⎦

𝐱 𝑅𝑛 𝐱 𝑓 𝑛 where 𝑘 ∈ , 0 is given, and 𝑘 isassumedtobecon- 𝜎 where { 𝑖}𝑖=1 are the desired Lyapunov exponents. tinuously differentiable, at least locally in the region of Actually, if the objective is only to have all the resulting interest. Lyapunov exponents of the controlled system orbit be finite The anticontrol problem for this dynamical system is to and strictly positive: design a linear state-feedback control sequence, 𝐮𝑘 = 𝐵𝑘𝐱𝑘, <𝑐≤ 𝜆 𝐱 < ,𝑖, ,𝑛 with uniformly bounded constant control gain matrices, 0 𝑖( 0) ∞ =1 … (44) ||𝐵 || ≤ 𝛾 < ∞,where|| ⋅ || is the spectral norm for a ma- 𝑘 s u s then one can simply use trix, such that the output states of the controlled system, ( ) 𝑐 𝐵𝑘 = 𝛾 + 𝑒 𝐼𝑛 , for all 𝑘 =0, 1, 2, … f 𝐱𝑘 = 𝐟𝑘(𝐱𝑘)+𝐮𝑘 +1 where the constants 𝑐 and 𝛾 are given in equations 44 and f behaves chaotically within a bounded region. Here, chaotic 43, respectively (29). behavior is in the mathematical sense of Devaney de- Finally, in conjunction with the above-designed con- scribed above, namely, the controlled map (i) is transitive, troller, that is, (ii) has sensitive dependence on initial conditions, and (iii) ( ) 𝑐 𝐮𝑘 = 𝐵𝑘𝐱𝑘 = 𝛾 + 𝑒 𝐱𝑘 has a dense set of periodic solutions. f In the controlled system anti-control is accomplished by imposing the mod- 𝐱 𝐟 𝐱 𝐵 𝐱 𝑘+1 = 𝑘( 𝑘)+ 𝑘 𝑘 operation onto the controlled system: 𝐱 𝐟 𝐱 𝐮 let 𝑘+1 = 𝑘( 𝑘)+ 𝑘 (mod 1) 𝐽 𝐱 𝐟 ′ 𝐱 𝐵 𝑘( 𝑘)= 𝑘( 𝑘)+ 𝑘 This results in the expected chaotic system with trajecto- ries remaining within a bounded region in the phase space be the system Jacobian, and let and, moreover, satisfying the aforementioned three basic properties that together define discrete chaos. 𝑇𝑘(𝐱 )∶=𝑇𝑗 (𝐱 , … , 𝐱𝑘)=𝐽𝑘(𝑥𝑘) ⋯ 𝐽 (𝐱 )𝐽 (𝐱 ) ,𝑘=0, 1, 2, … 0 0 1 1 0 0 This approach yields rigorous anticontrol of chaos for 𝜇𝑘 𝜇 𝑇 ⊤𝑇 𝑖 𝑘 Moreover, let 𝑖 [= 𝑖(]𝑘 𝑘) be the th eigenvalue of the th any given discrete-time systems, including all higher di- 𝑇 ⊤𝑇 𝑖 , ,𝑛 𝑘 , , , mensional, linear time-invariant systems, that is, with product matrix 𝑘 𝑘 ,where =1 … and =0 1 2 …. 𝐟𝑘(𝐱𝑘)=𝐴 𝐱𝑘 in system (eq. 42), where the constant matrix The first attempt is to determine the constant control 𝐴 can be arbitrary (even asymptotically stable). gain matrices, {𝐵𝑘}, such that the Lyapunov exponents of Although 𝐮𝑘 = 𝐵𝑘𝐱𝑘 is a linear state-feedback controller, the controlled system orbit can be arbitrarily assigned: the mod-operation is inherently nonsmooth. Recently, other types of (smooth) feedback controllers have been de- 𝜆𝑖(𝐱 )=𝜎𝑖 (arbitrarily given),𝑖=1, … ,𝑛 0 veloped for rigorous anticontrol of chaos, particularly for where each value 𝜎𝑖 may be positive, zero, or negative. It continuous-time dynamical systems (30) and hyperchaotic turns out that this is possible under a natural condition systems (31,32). Chaos, Bifurcations, and Their Control 25

BIBLIOGRAPHY 18. J. M. Ottino, The Kinematics of Mixing: Stretching, Chaos, and Transport. Cambridge University Press, 1989. 1. The New Encyclopaedia Britannica (Ready Reference and In- 19. W. J. Freeman, Sci. Am., 1991, 264, pp 78–85. dex), Micropaedia, Vol. 2, Encyclopaedia Britannica, 1984; 20. S. J. Schiff, K. Jerger, D. H. Duong, T. Chang, M. L. Spano, p. 743. and W. L. Ditto, Controlling chaos in the brain. Nature 1994, 2. T. Matsumoto, Proc. IEEE, 1987, 75, pp 1033–1057. 370, pp 615–620. 3. Academic American Encyclopaedia, vol. 4, Grolier Inc., 1986; 21. A. L. Goldberger, Applied Chaos, Academic Press, 1992; pp p. 283. 321–331. 4. G. Chen (ed.). Controlling Chaos and Bifurcations in Engi- 22. A. Garfinkel, M. L. Spano, W. L. Ditto, and J. N. Weiss, Sci- neering Systems. CRC Press, 2000. ence, 1992, 257, pp 1230–1235. 5. G. Chen and X. Dong. From Chaos to Order: Perspec- 23. J. N. Weiss, A. Garfinkel, M. L. Spano, and W. L. Ditto, J. tives, Methodologies, and Applications. World Scientific, 1997; Clin. Invest. 93, pp 1355–1360. reprinted, 2014. 24.T.Shinbrot,C.Grebogi,E.Ott,andJ.A.Yorke,Nature 1993, 6. G. Chen, D. J. Hill, and X. Yu (eds.). Bifurcation Control. 363, pp 411–417. Springer, 2003. 25. D. Auerbach, C. Grebogi, E. Ott and J. A. Yorke, Phys. Rev. 7. G. Chen and J. L. Moiola, J. Franklin Ins. 1994, 331B,pp Lett., 1992, 69, pp 3479–3482. 819–858. 26.A.Tesi,E.H.Abed,R.Genesio,andH.O.Wang,Automatica, 8. G. Chen, J. L. Moiola, and H. O. Wang, Int. J. Bifur. Chaos 1996, 32, pp 1255–1271. 2000, 10, pp 511–548. 27. J. Ford, In The New Physics; P. Davies, Ed.; Cambridge Uni- 9. G. Chen and X. Yu (eds.). Chaos Control. Springer, 2003. versity Press, 1989; pp 348–372. 10. G. Chen, The Circuits and Filters Handbook. CRC Press, 28. W. Yang, M. Ding, A. J. Mandell, and E. Ott, Phys. Rev. E. 1995; pp 973–1006. 1995, 51, pp 102–110. 11. E. N. Lorenz, J. Atmos. Sci. 1963, 20, pp 130–141. 29. G. Chen and D. Lai, Int. J. Bifur. Chaos 1996, 6, pp 1341– 12. T. Y. Li and J. A. Yorke, Am. Math. Mon. 1975, 82, pp 481– 1349. 485. 30. C. Shen, S. M. Yu, J. H. Lu, and G. Chen, IEEE Trans. Circ. 13. R. L. Devaney, An Introduction to Chaotic Dynamical Sys- Syst. I 2014, 61, pp 854–864. tems. Addison-Wesley, 1987. 31. C. Shen, S. M. Yu, J. H. Lu, and G. Chen, Int. J. Circ. Theory 14. J. L. Moiola and G. Chen. Hopf Bifurcation Analysis: A Fre- Appl. 2015, 43, pp 2039–2056. quency Domain Approach. World Scientific, 1996. 32. G. Chen and X. Yu (eds.). Chaos Control. Springer, 2003. 15. G. Chen and X. Dong. Int. J. Bifur. Chaos 1993, 3, pp 1363– 1409. GUANRONG CHEN 16. M. J. Ogorzalek. IEEE Trans. Circuits Syst. I 1993, 40,pp City University of Hong Kong, 700–706. Hong Kong SAR, China 17. E. Ott, C. Grebogi, and J. A. Yorke, Controlling chaos. Phys. Rev. Lett. 1990, 64, pp 1196–1199.