CHAOTIC COMPUTATION

By ABRAHAM MILIOTIS

A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2009

1 c 2009 Abraham Miliotis

2 TABLE OF CONTENTS page LIST OF TABLES ...... 5 LIST OF FIGURES ...... 7 ABSTRACT ...... 10

CHAPTER 1 INTRODUCTION ...... 11 1.1 Dissertation Overview ...... 14 1.2 Chaos ...... 15 1.2.1 Logistic Map, Topological Transitivity and Period Three ...... 15 1.2.2 The Tent Map, Topological Conjugacy and Universality ...... 38 1.2.3 Threshold Control and Excess Overflow Propagation ...... 43 1.3 Conclusion ...... 47 2 INTRODUCTION TO CHAOTIC COMPUTATION ...... 49 2.1 Number Encoding ...... 49 2.1.1 Excess Overflow as a Number ...... 49 2.1.2 Periodic Orbits for Number Representation ...... 50 2.1.3 Representation of Numbers in Binary ...... 52 2.2 Arithmetic Operations ...... 53 2.2.1 Decimal Addition ...... 53 2.2.2 Binary Addition ...... 56 2.2.3 Decimal Multiplication and Least Common Multiple ...... 59 2.3 Binary Operations ...... 60 2.3.1 Logic Gates ...... 61 2.3.2 Parallel Logic and the Half Adder ...... 64 2.3.3 The Deutsch-Jozsa Problem ...... 67 2.4 Conclusion ...... 73 3 SEARCHING AN UNSORTED DATABASE ...... 76 3.1 Encoding and Storing Information ...... 77 3.2 Searching for Information ...... 80 3.3 Encoding, Storing and Searching: An Example ...... 83 3.4 Discussion ...... 84 4 A SIMPLE ELECTRONIC IMPLEMENTATION OF CHAOTIC COMPUTATION 90 4.1 An Iterated Nonlinear Map ...... 90 4.2 Threshold Control Chaos into Different Periods ...... 90 4.3 Electronic Analog Circuit: Experimental Results ...... 92

3 4.4 Fundamental Logic Gates with a Chaotic Circuit ...... 97 4.5 Encoding and Searching a Database Using Chaotic Elements ...... 100 4.6 Conclusion ...... 104 5 LOGIC OPERATIONS FROM EVOLUTION OF DYNAMICAL SYSTEMS . . 106 5.1 Generation of a Sequence of (2-input) Logic Gate Operations ...... 106 5.2 The Full Adder and 3-Input XOR and NXOR ...... 110 5.3 Conclusion ...... 113 6 MANIPULATING TIME FOR COMPUTATION ...... 115 6.1 Introduction ...... 115 6.1.1 Flexible Logic Gates ...... 115 6.1.2 Search Algorithm ...... 120 6.2 Neural Implementation ...... 122 6.2.1 Neural Models ...... 122 6.2.2 Algorithm Implementations ...... 127 6.3 Electronic Implementation ...... 132 6.4 Discussion ...... 136 7 CONCLUSION ...... 139 REFERENCES ...... 141 BIOGRAPHICAL SKETCH ...... 147

4 LIST OF TABLES Table page 1-1 Summary of the transitions in behaviour of the different intervals described in Section 1.2...... 37 1-2 Topological conjugacy...... 41 1-3 Experimental measurements of Feigenbaum’s constant (δ) in different systems based on their period doubling...... 44 2-1 Truth-table for AND, OR, XOR, NOR, NAND, NOT, and WIRE...... 63 2-2 Necessary and sufficient conditions for a chaotic element to satisfy AND,OR, XOR, NOR, NAND, NOT, WIRE...... 63

prog 2-3 Initial values, x , and threshold values, x∗, required to implement the logic gates AND, OR, XOR, NOR, NAND, NOT, and the identity operation (WIRE), with δ = 0.25...... 64 2-4 Truth table for XOR and AND logic gates on the same set of inputs...... 65 2-5 Truth table for two AND gates operating on independent inputs...... 66 2-6 Required conditions to satisfied parallel implementation of the XOR and AND gate...... 66 2-7 Required conditions for implementing two AND gates on independent sets of inputs...... 66

prog prog 2-8 Examples of initial values, x , y , and thresholds x∗, y∗, yielding the parallel operation of XOR and AND gates...... 67 2-9 Examples of initial values xprog, yprog, and thresholds , yielding operation of two AND gates on independent inputs...... 67 4-1 Truth-table for the five fundamental logic gates NOR, NAND, AND, ORand XOR...... 98 4-2 Necessary and sufficient conditions to be satisfied by a chaotic element inorder to implement the logical operations NOR, NAND, AND, OR and XOR...... 99 4-3 Numerical values of xprog for implementing logical operations NOR, NAND, AND, OR and XOR...... 99

4-4 Updated state values, x1 = f(x0), of a chaotic element in order to implement the logical operations NOR, NAND, AND, OR and XOR...... 99 5-1 The truth table of the five basic logic operations NAND, AND, NOR, XOR,OR. 107

5 5-2 Necessary and sufficient conditions to be satisfied by a chaotic element inorder to implement NAND, AND, NOR, XOR and OR on subsequent iterations. . . . 108 5-3 The truth table of full adder, and necessary conditions to be satisfied...... 112 5-4 The truth table of the 3-input XOR and NXOR logic operations, necessary and sufficient conditions to be satisfied by themap...... 114 6-1 The truth table of each of the five fundamental logic gates, AND, NAND, OR, NOR, XOR ...... 118 6-2 Time values, in arbitrary time “units” for each of the five gates considered . . . 119 6-3 Appropriate time sample instances, based on simulation points, to perform each of the five gates considered ...... 128 6-4 Appropriate R values so time shift an action potential in order to perform each of the five gates considered ...... 131 6-5 Delay times for Vs to implement each of the five gates considered ...... 134

6 LIST OF FIGURES Figure page 1-1 Bifurcation diagram for the Logistic map...... 18

1-2 Forward and Backward evolution of F4...... 19

1-3 Indicative behaviour of I under multiple applications of Fμ for, (a) μ < 1 and (b) 1 < μ...... 22

1-4 Plots of F2 and F2.5...... 25

1-5 Exhibition of the “trapping” of two points in two different configurations of Fμ. 28

2 1-6 Renormalization of two cases of Fμ ...... 29 1-7 Demonstration of birth of odd periodicity fixed points...... 33

1-8 Plot of the function F : I I...... 36 → 1-9 Plot of the function T : I I...... 39 → 1-10 Bifurcation diagram for the Tent map...... 40 1-11 Topological conjugacy between evolved states, up to n = 5...... 42

1-12 Logistic map bifurcation diagram for some values within 3 < μ < μ ...... 44 ∞ 1-13 Threshold Control Mechanism...... 46 1-14 Threshold values for confining the logistic map on orbits of periodicity 2to50. . 47 2-1 Emitted excess by thresholding the logistic map in the interval [0, 0.75]...... 51 2-2 Encoding the set of integers 0, 1, ..., 100 ...... 52 { } 2-3 Number encoding in binary format...... 54 2-4 Serial Addition...... 55 2-5 Decimal parallel addition...... 57 2-6 The branching algorithm can be extended to a larger treelike structure...... 57 2-7 Schematic representation of the serial addition method for binary numbers. . . . 58 2-8 Schematic representation of the parallel addition method for binary numbers. . . 59 2-9 Schematic representation of the method for the Least Common Multiple of four numbers...... 61 2-10 Basis function T - Tent Map...... 71

7 2-11 Basis function T - Inverted Tent Map...... 72

2-12 Four realizationse of the chaotic Deutsch-Jozsa algorithm for the case k = 3. . . . 74 2-13 The total excess emitted from each of the 72 functions...... 75 3-1 The Tent map under the threshold mechanism...... 78 3-2 Schematic representation of the changes in the state of different elements. . . . . 82 3-3 Searching for “l”...... 85 3-4 Searching for “e”...... 86 3-5 Searching for “x”...... 87 4-1 Bifurcation diagram of the iterated map for various values of α and β...... 91 4-2 Graphical form of the map to be implemented by an electronic circuit ...... 92

4-3 Effect of threshold value x∗ on the dynamics of the system...... 93 4-4 Circuit diagram of the nonlinear device of Equation 4–3...... 94 4-5 Voltage response characteristics of the nonlinear device...... 95 4-6 Schematic diagram for implementing the threshold controlled nonlinear map. . . 96 4-7 Circuit diagram of the threshold controller...... 96 4-8 PSPICE simulation results of the experimental circuit...... 97 4-9 Searching for “b”...... 103 4-10 Searching for “o”...... 104 4-11 Searching for “d”...... 105 5-1 Graphical representation of five iterations of the Logistic map...... 109 5-2 Patterns of binary two input symmetric operations...... 111 6-1 Schematic representation of a flexible 2-input logic gate ...... 116 6-2 Construction of a “generic” signal ...... 119 6-3 Schematic representation of the time based search method ...... 122 6-4 Time Delay Unit (TDU) ...... 126 6-5 Demonstration of operating a NOR gate, with a single neuron using different sampling times ...... 128

8 6-6 Demonstration of operating a NOR gate, with a neural circuit using different delay times ...... 130 6-7 Schematic representation of an electronic circuit for logic using time ...... 133 6-8 Demonstration of operating a NOR gate using an electronic circuit utilizing time dependant computation ...... 134 6-9 Schematic representation of an electronic circuit for the time dependant search method ...... 136 6-10 Demonstration of performing a search using an electronic circuit utilizing time dependant computation ...... 137

9 Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy CHAOTIC COMPUTATION By Abraham Miliotis May 2009 Chair: William L. Ditto Major: Biomedical Engineering Chaotic Computation is the exploitation of chaotic systems to perform computational tasks. The abundance of uncountable distinct behaviours by chaotic systems, along with their embedded determinism, position such systems as perfect candidates for developing a new computational environment. The present dissertation focuses on algorithms developed over the past decade within the realm of Chaotic Computation. After a brief exposition of general Chaos Theory, we proceed to give detailed instructions for performing such algorithms, as well as specific examples of implementations. We begin with multiple methods for number representation and basic arithmetic manipulations, providing from the start evidence of the flexibility of Chaotic Computation. The compatibility with Turing machines is subsequently shown through an algorithm for logic operations whose general form is a recurrent theme. We soon though, proceed further than Turing machines and present a solution to the Deutsch-Jozsa problem, of arbitrary binary functions. Even more, a practical issue is also handled by showing how chaotic systems have a natural way for selecting matches of a searched item from within an unsorted database. Finally we present our latest results in handling “prolonged” evolution of chaotic systems. Specifically we demonstrate the dominance of selecting the appropriate behaviour, for a computational task, over being exact with specific state values, or even confined to specific physical quantities.

10 CHAPTER 1 INTRODUCTION Over the last five decades the advancement of computational machines has been an invaluable achievement by our society and extremely beneficial. Moore‘s law [1] either as a self-fulfilling prophecy, or as a simple description of this progress, hasbeen obeyed up to now closely. The immense importance of preserving this progress has driven many to investigate possible reasons that could hinder further increase in computational performance. Obviously the ultimate limits imposed by the physical nature of our world were the first to be identified. The three most important include the speed of light limiton transmission of information, the limit on the amount of information a finite system can store, and the thermodynamic limit of thermal energy dissipation by the erasure of information [2], the latter being the most relevant to current semiconductor technology. Perhaps most important though, at this time, are the limitations inherent in semiconductor technology, which is the basis of contemporary computers. Regularly over the last decade the International Technology Roadmap for Semiconductors (ITRS) has been published with the aim to indicate the tread of specific features of semiconductor technology [3], for example lithography resolution, transistor size and connecting wire diameter. Current technology is “relatively” far from fundamental limits for these attributes (molecular sizes and thermal noise), but approaching them at an alarming rate; for example Austin et. al. report a method for creating memory half-pitch size equivalent to the size of an insulin molecule(6 nm) [4]. As the need for alternative computational paradigms is becoming progressively evident many physical systems have been proposed as alternative basis for computational machines. The most prominent being quantum computing [5, 6] and DNA computing [7–10]. Even after many years of intense research though, both quantum and DNA computing are still facing some fundamental problems. The main problem with both

11 those paradigms is that they are confined, by their nature, to a single physical realization. In each case this confinement presents different problems that are difficult to overcome. For quantum computing the dominant issue is that of thermal noise, which randomizes individual qubits and leads to “decoherence” of the system. DNA computing is mainly restricted by the time-scale with which the necessary chemical reactions take place, making any computational task very slow in producing a result. Regardless of the physical system used for computation it is clear that there will be ultimate limitations in effectiveness for accomplishing any individual task. The common theme in all alternative computational paradigms is flexibility and parallelization; this is also pursued by conventional computer science research. The common goal is the construction of a methodology that will provide a computational model that will be able to solve each computational problem the most efficient way and, if possible, multiple problems simultaneously. The main driving force to achieve such a model is the fact that the most powerful computational machine we have, the human brain, is able to both, solve problems in multiple ways, and handle multiple problems simultaneously. Chaotic systems over the last few decades have attracted the attention of a large portion of the scientific community. The main reason is the abundance of such systemsin nature and the extensive repertoire of behaviours they exhibit [11, 12]; compounded by the fact that they are deterministic systems and can be described by a small set of equations. More recently there has been intense research in methods to control chaotic systems, inspired primarily by the work of Ott, Grebogi and Yorke [13]. Control mechanisms have provided the means to confine a chaotic system to a specific behaviour. Such techniques have been taken advantage by many fields of contemporary research including control, synchronization, communications and information encoding [14, 15]. Many attempts have been made since before the 90‘s, and many are still active fields of research, for bridging dynamics and computation [16–23]. Probably though the clearest suggestion that a chaotic system, specifically, can be used for computation

12 can be attributed to C. Moore [24]. It wasn’t until late 90‘s though, that the use of a non-feedback control mechanism on a chaotic system to perform specific computational tasks was demonstrated [25, 26]. Over the last nine years non-feedback control has been utilized to show how chaotic systems can perform arithmetic and binary logic operations, and even further to solve more complex problems like the Deutsch-Jozsa problem and searching an unsorted database [27, 28]. Non-feedback control typically is achieved either by the use of a threshold on a state variable or by selecting specific values for the system parameters, in either case confining a system to a specific subset of all available points. The key idea is that from the wide variety of behaviours embedded in a chaotic system one can find a specific pattern of behaviour that can accomplish a specific task. Control can confine a system to the required specific behavioural pattern, without loss of the ability to switch to a different behaviour and perform a different task. Thisis accomplished by rigorous investigation of a chaotic system to identify the required behaviours that can reliably represent computational tasks; and the state variable thresholds, or parameter values, that will confine the system to evolve in the required manner. This allows us to envision a computational machine which has as its building block a chaotic system; each element being identical to all others, providing redundancy and reliability; each element able to perform a multitude of functions, providing flexibility; and each element independent of all others, providing parallelism of tasks. The reason we are specific about which chaotic system to use is the Universality of chaos [29]. Chaotic systems, even if their governing equations differ, behaviourally are the same. This not only allows us to choose the most convenient system for theoretical development of algorithms, but more importantly does not confine the physical realization to a specific technology. As mentioned above, chaotic systems exist in abundance in nature including high-speed electronic circuits, lasers, and even neurons; any of which can be used as the building block of a chaos based computer.

13 1.1 Dissertation Overview

This dissertation can be considered to be of four parts. The first part is the remaining of this introductory chapter, the second part chapters two and three, while the fourth chapter can be considered to be on its own. Finally chapters five and six comprise the final part of our exposition of Chaotic Computation. In the remaining of this introductory chapter we provide necessary parts of Chaos Theory so that to relate it with computation. Specifically we expose the reader to aview of Chaos Theory from a “loose” Set Theory approach, with the intension to imply to the reader the connections from Set Theory, to Chaos Theory, to Computation and general Mathematical Logic. As this dissertation is concerned solely with Chaotic Computation we do not delve in depth on the peripheral issues and direct the reader to appropriate references [30, 31]. Chapters two and three can be considered to be the main part of this dissertation; they are an exposition of established algorithms of Chaotic Computation [25–28, 32, 33]. We begin in chapter two with the earliest algorithms of Chaotic Computation developed over the last nine years by Ditto et. al., algorithms for number representation, arithmetic operations and a solution to the Deutsch-Jozsa problem. While the third chapter presents exclusively the recent algorithm for searching an unsorted database [28]. In the relatively small third part that is chapter four, we present a recent implementation of most abilities of Chaotic Computation with an extremely simple electronic circuit [34]. Despite its length, the aims of this part are twofold; first and foremost to give substance of realization to the whole concept of Chaotic Computation through a physical system implementation; and beyond, to present a concrete demonstration of the Universality of chaos, and the ease with which we can translate our results to any chaotic system. The final part is our most recent results, the “cutting edge” developments inChaotic Computation [33]. Chapter five, while focuses solely on binary operations, introduces the importance of what can be considered to be “the time dimension”; in addition it

14 expands on our treatment of “state values”, by showing more concretely the importance of specific behaviours over exact state values. Finally chapter six expands further the idea of manipulating time. Through demonstrations, with electronic circuits, and mainly neural circuits we exhibit implementations of Chaotic Computation algorithms using “time instances” as the medium for computational commands. We wish to the reader the encounter with this exposition to be both informative and enjoyable. 1.2 Chaos

Chaos Theory is the third major “revolution” of Physics of the 20th Century. Even though its “birth date” can be placed as contemporary to Relativity and Quantum Mechanics, circa 1900, it wasn’t until much later, in the early 1960s, that the field actually attracted enough attention to attain “critical mass”, a slightly American biased popular science recount of the history of Chaos Theory can be found in [35]. This dissertation is not about Chaos Theory as such, but more about some of the “features” of Chaos Theory. We utilize much of what is defined, predicted, and expected from Chaos Theory to propose a whole new realm for computation. The problem is that Chaos Theory is a very closely packed theory, with each part relating to some other part and the whole to the details (a concept which is a result of the theory as well). We will present the “features” of Chaos Theory that are necessary for projecting to the reader our results and direct the reader to: Devaney (1982) [36] for a solid mathematical exploration of Chaos and the origin of one of the most accepted definitions of Chaos; Peitgen, J¨ugens and Saupe (1992) [37] for a more hands on demonstration with heavy emphasis on fractals, and Ott (1993) [38] for a midpoint approach to Chaos Theory, with the addition of an exposition of Quantum Chaos. 1.2.1 Logistic Map, Topological Transitivity and Period Three

In this introductory chapter we provide a somewhat “non-traditional” view of chaos. The approach we take is one closer to a mathematician‘s rather than a physicist‘s, since

15 computation, our main concern, is more of a mathematical concept than physical. We present chaos through in a “loose” set theory context, that is we treat topological spaces, intervals and even state variables as sets of points, we try to avoid any “linearizing” tool, like differentiation, and any representation that involves discretization of our set ofpoints, like statistical manipulations. At the same time our exposition is “loose”, since we do not go into extreme formal mathematical precision, that is we omit providing proofs and extensive definitions of terms used. We direct the reader to two seminal sources, Principia Mathematica of Whitehead and Russell [30] and Kurt G¨odel‘sever important paper, “On formally undecidable propositions of principia mathematica and related systems” (1931) [31], for both more background information of our approach, especially the missing mathematical details, and more importantly for justification for our approach and howit relates to the concept of computation. Our main “tool” for presenting our necessary parts of Chaos Theory is the discrete time Logistic map:

n+1 F (xn) = xn+1 = μ xn (1 xn), (1–1) μ ∙ ∙ − where x is the state of the system, n1 is the time step and 0 < μ can be thought of as a growth rate. Actually the origins of this quadratic equation is from population dynamics, specifically it is modeling the behaviour of a population with limited resources. Inthat context x represents the current fraction of the population with respect to the maximum possible sustainable population (x = 1). In its continuous-time form, the model can be solved analytically and gives rise to the most common sigmoid function, a “well behaved” function of continuous growth and eventual saturation2 . When we consider discrete time

1 We shall drop the superscript for the cases of n n + 1, unless we wish to emphasise → the use of a single application of Fμ. 2 A theorem by Poincar´eand Bendixson [39] guarantees that continuous time two dimensional planar systems can not exhibit chaos.

16 though, for example n denoting the current generation of beings in our population, we have Equation 1–1 and the behaviour of x is no longer simple continuous growth leading to eventual saturation; in fact in this discrete case if the population does manage to reach saturation it collapses to zero. For our purposes we need a closer look at Equation 1–1. With a first glance it is clear the function is governed by the three variables x, μ, n, each in principle unbounded. There is no need though to go to such lengths, as we can confine ourselves to the domains 0 x 1, 0 < μ 4, 0 < n. This is truly a very small portion of the whole available ≤ ≤ ≤ space, so briefly lets justify this confinement, and we will provide more justification aswe focus more in the above domains. Starting with 0 < n, we can set the “beginning of time” at any “moment in time”, we will actually see below that this is totally arbitrary; actually for some results of Chaos Theory reversing the arrow of time, n n 1, is very useful. In → − the case of μ things are a bit more complicated, except for the case μ = 0; for μ < 0 we can show the same behaviours as for 0 < μ; for 4 < μ most x values escape to infinity and the ones that are “trapped” form a Cantor set which is the prototypical fractal and, again, shares many of the “features” that are present in 0 < μ 4, subsequently we will confine ≤ μ further. Finally x, with the other two variables confined to the above domains, we have x < 0 and 1 < x escaping to infinity or eventually collapsing to zero. Therefore our concern is what happens to a state x [0, 1] for values of μ (0, 4], in ∈ ∈ the “forward” direction of time n n + 1. An immediate view of the behaviours in this → domain can be obtained through a bifurcation diagram, see Figure 1-1. Such a diagram is

1 3 obtained by evolving an initial x0 state, 2 in this case, for a large number of time steps and plotting the state at the final few time steps, in this case we evolved the map 500time steps and plotted the last 200 states. From this empirical approach we can already see the

3 Using the critical point of the map we are guaranteed to evolve into an attracting state, if one exists.

17 richness that this simple map provides, at least three behaviours are evident: collapse to zero, attraction to a non-zero fixed point and attraction to a periodic orbit.

Figure 1-1. Bifurcation diagram for the Logistic map. Eventual behaviour of an initial state x0 under repeated applications of Fμ.

This static picture of what happens, even though extremely rich, is a generalization of the details of the behaviours of x. For example it should be clear, but it is not clearly shown, that this picture is “almost” completely invariant to the initial x0, that is what happens to a state x [0, 1] happens to “almost” any other initial state, the map has ∈ a local effect that is global, and visa versa. It is not clear though what happens atthe onset of chaos, when “almost” all points behave the same way and at the same time all the “rest” of the points behave in a different way! To see the details inside the bifurcation diagram we take a more “dynamic” approach with respect to the three available variables. Starting with n we will consider the question “Where do states come from?”, we are not going to address the issue of “time reversal”,

18 but rather take a step “back in time” and consider:

x0 = μ x 1 (1 x 1) = Fμ(x 1), (1–2) ∙ − ∙ − − −

1 where x0 [0, 1] and 0 < μ 4. Figure 1-2 shows two plots of the function F each ∈ ≤ 4 showing how the function evolves in either “direction of time”.

Figure 1-2. Forward and Backward evolution of F4. (a) Forward evolution - Empty circle, , marks an arbitrary initial point x0; we can track its forward evolution by #moving “up” (or “down”) to meet the function F4 and then “right” (or “left”) back to the diagonal, each full cycle of these two steps is equivalent to a single 5 application of F4; thus the full circle, , marks the point F4 (x0) = x5. (b) Backward evolution - The steps of the forward evolution can also be reversed; i.e. from an arbitrary initial point x0, empty circle , we can move “left” and “right” (or “left” twice) to the two points on F4 and# then “up” and “down” 1 + (or “down” twice) to meet the diagonal on the points of F4− (x0) = x 1, x−1 , { − − } the two empty squares, o; and of course further back to the four points of 2 F4− , the four full squares, I, and so on.

By solving Equation 1–2 we observe the following: by setting x0 = x 1 we find the − 1 1 fixed points at x = 0 and x = 1 ; now setting x0 = 0 and x0 = 1 we find for each ∞ ∞ − μ − μ 1 1 1 fixed point its two pre-images, specifically x 1 0, 1 0 and x 1 , 1 1 ; − ∈ { } → − ∈ { μ − μ } → − μ

19 2 √μ 4 μ x0 1 − ∙ ∙ and finally by solving the quadratic for any x0 we reach x 1 = 2 2 μ , which leads − ± ∙ μ μ 1 to < xn xn 1 C, that is x > ,Fμ− (x) C; the pre-images of any x value greater 4 ⇒ − ∈ ∀ 4 ∈ than a fourth of the “growth factor” (μ), of the applied map, are in the complex plane. This is an interesting result and the start of another long story, it shows how the dynamics of the logistic map extend in the complex plane. This is beyond our current scope, but in

2 passing we note that the logistic map is a reduction of the quadratic maps Qc(z) = z + c

where z, c C, the source of the famous Mandelbrot and Julia fractals [40], this is an ∈ initial hint to the concept of Universality, which we will address in Section 1.2.2. We are now ready to begin to investigate how μ changes the behaviour of x [0, 1]. ∈ We can not explain what happens at every value of μ so we consider the change before and after a point of transition. Clearly at μ = 1 we have a major change, the fixed point x = 1 1 enters the domain [0, 1] and becomes an attractive fixed point, actually the ∞ − μ “big event” at this μ-value is that the two fixed points cross paths. At the same time0 becomes a repelling fixed point; these conclusions can be drawn by taking the absolute

dF (x ) value of the derivative of the map at the fixed points, dx∞ , a standard technique for characterization of dynamical systems. This is though a local linearization of the map, we prefer to show this, and subsequent transitions, in terms of global effects, thus prepare the ground for what happens at higher values of μ.

We need to deviate for a while to single out some special points on the I = [0, 1] interval. We already seen the four points that map to the fixed points, 0, 1, 1 , 1 1 , { μ − μ } 1 1 3 1 and we add to this list , , . In general is the most important point of Fμ, not { 4 2 4 } 2 because of its value, but since it is the critical point of the map and its backward and forward evolution can actually characterize the map extensively; the theory behind the evolution of the critical point is called kneading theory [36]. The importance of the

1 3 1 1 3 1 3 points 4 and 4 is topological, following Fμ( 4 )Fμ( 2 ) = 4 and of course Fμ( 4 ) = Fμ( 4 ), 1 in addition to Fμ( 2 ) = sup Fμ(I), visually this means what will happen to the interval [ 1 , 3 ] F n+2([ 1 , 3 ]) is what happened previously to the interval [ 3 , 1] F n+1([ 3 , 1]) and 4 4 → μ 4 4 4 → μ 4

20 what is going to happen to [ 3 , 1] F n+1([ 3 , 1]) is what happened to [0, 1 ] F n+1([0, 1 ]), 4 → μ 4 4 → μ 4 1 in a normalized sense. Consider the interval [0, 1] separated in three parts I1 = [0, 4 ), 1 3 3 I2 = [ 4 , 4 ], I3 = ( 4 , 1]. No matter the μ-value we know the following: Fμ(0) = 0, 1 3 3 μ 1 1 μ 1 1 3 3 1 1 Fμ( ) = Fμ( ) = ∙ , Fμ( ) = ∙ and Fμ(1) = 0 so Fμ( 0, , , , 1 ) 0, ( μ), ( μ) , 4 4 16 2 4 { 4 2 4 } → { 4 ∙ 4 ∙ 4 ∙ } divide out by a factor of ( 1 μ) and its like nothing happened! We need to emphasise that 4 ∙ this picture is only for visualization purposes and it applies only to a single application of the map, a “snapshot” if you like that we can keep track of while iterating n , below → ∞ we will discuss the proper way to view evolution of intervals, the renormalization operator. Returning to the question “What happens as μ < 1 1 < μ?”. There are multiple → ways we can analyse the situation, we look at it with respect to changes to the behaviour of the aforementioned intervals. Globally before the point of transition (μ < 1) we simply

have Fμ(I) < I so its clear that in the limit n the whole interval will collapse to → ∞ n+1 n+1 zero. To “see” the change at the transition consider Fμ(I1) I I , that is I1 will → 1 ∪ 2 become the union of what can be considered to be the I1 and I2 of the next application (n + 1) of the map, at which F n+1(In+1) In+2 and F n+2(In+2) In+3 In+3 μ 2 → 3 μ 3 → 1 ∪ 2 completing the “circle”. Actually it is a double spiral with a moving pivot, the sequence

1 3 1 1 of post-images of 4 and 4 denoted as ∗x. So we have at μ < 1, Fμ( 2 ) < 4 , which implies n that F (I2) I2 = , this is enough in some sense to characterize the evolution since we μ ∩ have seen that in some sense both I3 and I1 end up, at least partly, in an interval that can

m n be called I , another way to describe this evolution is F (∗x) = 0 as n . With this 2 μ → ∞ description of the evolution of Fμ for μ < 1 we can see what changes at μ = 1, specifically

1 1 Fμ(I2) I2 = and at the single point magnification F1( ) = . The consequence of this ∩ 6 2 4 n n n+1 change can be expressed in multiple ways: since the post-images of I1 and I3 are in I2

n n 1 it means that the relationship Fμ(I2) I2 = can be extended to F (I2) I − = ∩ 6 μ ∩ 2 6 even as n so I∞ contains a single point, which is the eventual post-image of all these → ∞ 2 m n 1 points that get “trapped” in the sequence of I ; or simply as n , F (∗x) 1 2 → ∞ μ → − μ (= 0, for μ = 1); and even though the “pivot” point does converge to zero eventually, there

21 will always be at least one more “point” between its value and 0, these “points” we will see are cental to what is chaos. So now an increase in μ toppling it over 1 separates 0 and 1 1 by a more “concrete” amount, giving substance to the fixed point at 1 1 , and to − μ − μ the points inside the interval (0, 1 1 ), for a view of the evolution of points in either of − μ these two regions (μ < 1 or 1 < μ) see Figure 1-3. We will come across transitions like the one we just described an infinite number of times going from μ = 1 to μ = 4.

Figure 1-3. Indicative behaviour of I under multiple applications of Fμ for, (a) μ < 1 and 1 (b) 1 < μ. (a) (μ = 2 ) The two fixed points are one at 0, and the other outside I. (b) (μ = 1 1 ) 0 remains a fixed point and now 1 1 I; with one pre-image 2 − μ ∈ being itself at 1 1 = 1 and the other at ( μ <) 1 = 2 . Empty circles, , mark − μ 3 4 μ 3 initial points and full circles, , final points (not the fixed point). #

Now we are in the 1 < μ region and, even though the bifurcation diagram doesn’t show it, the next point of transition is at μ = 2. We have seen how the behaviour of

the whole interval I changed at μ = 1, and with it we should change our view of its

subintervals. We could continue the discussion with I1, I2 and I3, and we will return to them when necessary. Our aim though is to make the behaviour of the whole interval I as

22 visual as possible, and so given that now 1 1 exists in I (as well as both its pre-images − μ 1 1 1 1 , ) we have a better choice of subintervals to use, specifically J˜1 = [0, 1 ), { − μ μ } − μ 1 1 1 J˜2 = [1 , ], J˜3 = ( , 1]. As we have done for the previous transition, first we will − μ μ μ see how the interval I behaves with 1 < μ < 2. A quick side-note, taking the derivative at the fixed points we find the “nature” of 0 to be repelling and1 1 attracting. The − μ details though make all the difference, as we have defined the intervals J˜i we see that 2 ˜0 ˜1 1 ˜0 ˜ ˜ ˜ Fμ (J3 ) = J1 = Fμ (J1 ). The interval J3 gets mapped onto J1 and J1 onto itself, so it seems

4 that J˜1 is not changing and even more it seems that J˜1 and J˜3 are “disconnected” from

1 1 J˜2. So it seems we have three types of points, the set of four points 0, 1 , , 1 , and the { − μ μ } two intervals (1 1 , 1 ) and (0, 1 1 ) ( 1 , 1), the easy part is the fixed points and their − μ μ − μ ∪ μ 1 1 1 μ 1 pre-images Fμ( 0, 1 ) = 0 and Fμ( 1 , ) = 1 and in addition, since < , 1 , { } { − μ μ } − μ 4 { μ } we know all other pre-images of the fixed points are in the complex plane. For(1 1 , 1 ) − μ μ 1 1 1 1 1 1 1 since Fμ : (1 , ) (1 , μ) and 1 < μ < we see that all points in the − μ μ → − μ 4 ∙ − μ 4 ∙ μ limit n are “squeezed” closer and closer to 1 1 . Finally for (0, 1 1 ) ( 1 , 1), the → ∞ − μ − μ ∪ μ simple picture is that x (0, 1 1 ) and y ( 1 , 1), F n(x) > x so as n , x 1 1 ∀ ∈ − μ ∀ ∈ μ μ → ∞ → − μ n+1 1 and as we have seen x : Fμ(y) = x, so F (y) 1 , therefore every point moves ∀ μ → − μ 1 away from zero (or one) and towards 1 . We have though seen that Fμ(J˜1) = J˜1, note − μ μ that x J˜1, x < so every point in J˜1 has two real pre-images, one of them is of course ∀ ∈ 4 in J˜3 and it has no pre-images of its own, but the other one is in J˜1 and x 1 < x0 so the − pre-image also has two pre-images, and so on. So the complete picture of the evolution of

1 J˜1 is that each x0 does evolve to x1(> x0) and eventually, at n , to 1 , but on → ∞ − μ every application of the map it is “replaced” by two other points; in other words, points from the neighbourhood of 0, and of course 1, eventually escape this neighbourhood and evolve into the neighbourhood of 1 1 , 0 is a repelling fixed point and 1 1 attracting. − μ − μ

4 n ˜ 1 “Disconnected” here is used as in every day language, in fact since Fμ (J3) 1 μ as n the intervals are of course still connected. → − → ∞

23 As soon as we cross from μ < 2 into 2 < μ we shall see more clearly the importance in considering this behaviour. At μ = 2 we reach the second transition point, 1 1 = 1 = 1 , in some sense as − μ μ 2 at μ = 1 we had 1 1 = 0, two of the points that lead to a fixed point cross paths; − μ the difference in this case is two points that lead to the same fixed point(1 1 and 1 ). − μ μ Before we go on to the exciting changes that happen in 2 < μ, we note in passing that at

1 1 μ = 2 the derivative at the fixed point is 0 and so 2 is called “super-attractive” point; 2 has no pre-images real or imaginary, in a sense all points (even from the complex plane) 1 ˜ converge to 2 ; more importantly though, it is the only point in J2 as a result we now have,

in addition to F2(J˜3) = F2(J˜1) = J˜1, F2(J˜2) = J˜2, that is all our intervals are “eventually invariant”5 , see Figure 1-4(a) for a view of the behaviour at μ = 2. We are now in the 2 < μ region; there are two important general changes in behaviour, first of the fixed point acquires now an infinite number of pre-images, since

1 μ μ < 4 , and as a consequence, topological “mixing” is now possible, that is any closed interval contains points that have pre-images, or even post-images, outside the interval, the set of points that lead to the fixed point has more than one point; more visually notice

that J˜1 J˜2 J˜3 = , see Figure 1-4(b). This leads us to the first concept that isa ∩ ∩ 6 requirement for chaos, topological transitivity. Definition 1.1. Topological Transitivity. Given a map f : J J, f is said to be topologically transitive on J, if for any two → open subintervals U, V J there exists a x U and a y V such that f n(x) = y, for ⊂ ∈ ∈ some 0 < n. In other words, given an interval any point in it can, eventually, be mapped on any other point in the interval. This is not active for values 2 < μ < 3.5699456..., but the fact

5 We shall see that the transition from “eventually invariant” to invariant makes all the difference.

24 that we have intervals “mixing” is the first step, it is the difference between “eventually invariant” and invariant. Before we proceed let us redefine our intervals so that they are not “mixed” before we even apply the map and to keep the picture of how the intervals

1 1 1 1 evolve as it has been up to now. So we re-define J1 = [0, ), J2 = [ , 1 ], J3 = (1 , 1], μ μ − μ − μ basically acknowledging the fact that the pre-images of the fixed point (1 1 ) have − μ 1 ˜ switched sides around 2 , see Figure 1-4(b) for the “mixing” of the Ji intervals and their redefinition as Ji, where i 1, 2, 3 . The behaviour of these intervals is straightforward, ∈ { } 1 we still have Fμ(J3) = Fμ(J1), but now Fμ(J1) = J1 J2 and Fμ(J2) = J6 1 J3. ∪ ⊂ { − μ } ∪ 1 Visually we have J3 stretched and rotated around 1 onto J1 J2, J1 is simply stretched − μ ∪ 1 onto J1 J2 and J2 folded inside itself, with as the pivot of the folding, and again rotated ∪ 2 1 1 around 1 to land inside J3; overall all points are spiralling around and inwards 1 , − μ − μ which is the situation we had at μ < 1, so it should come as no surprise what happens at μ = 3.

1 Figure 1-4. Plots of F2 and F2.5. (a) Display of the “super-attractive” case when 2 is the fixed point. (b) Display of the six intervals J˜1, J˜2, J˜3 and J1,J2,J3 , the { } { } overlap of J˜i shows the initiation of topological “mixing”.

25 As the points are spiralling inwards the fixed point, and μ is gradually raised we have in J3 what was happening to I, the points collapsing towards the fixed point, until there is

1 1 1 1 6 a point left out, specifically when Fμ( ) (1 ) + 1 (1 ) ; or more visually 2 ≥ − μ 4 ∙ − − μ h i when the stretching of J3 (and J1) “creates” more points than the “available” points in

1 7 J1 J2 and so “a” point is left out that will never spiral on 1 . All this happens at ∪ − μ 1 μ = 3 and, as at μ = 1, the point left out is a post-image of 2 only this time we have, for 2n 1 1 2n 1 0 < n, F − ( ) J3 and F ( ) J2 as n , so we have a fixed point created in J3 3 2 ∈ 3 2 ∈ → ∞ and one in J2, thus initiating the famous period-doubling route to chaos. Before we follow this route, we need to note what happens to 1 1 our “current” fixed point. Just as at − μ μ = 1, the fixed point at 0 changed form from attracting to repelling, in the same manner the nature of 1 1 changes, only with an important difference as we have already noted, − μ unlike for 0 the number of pre-images for 1 1 is infinite, but at least countable, for now. − μ It is time to provide the promised further justification of neglecting the regions 4 < μ and μ < 0. The creation of the two fixed points by “entrapment” is exactly what happens to points on I for 4 < μ, with the slight difference that the trapped points for4 < μ collapse to 0, basically the top part of the parabola is protruding over some value on which the two points that happen to be on, are captured, Figure 1-5 displays this process.

2 1 For μ < 0 consider F3 (J2), it is an inverted parabola, differing from F0 for 0 < μ only

2 by a shift of 3 ; and a small curvature deviation caused by the higher power terms of x, 2 see Figure 1-6(b). In fact we can take F3 , or for any μ for that matter, and using the

renormalization operator translate it to a form for I, as follows: Define: x = 1 1 andx ˆ = 1 . ∞ − μ ∞ μ

6 1 Compare with F1(0.5) 0 + (1 0). ≥ 4 ∙ − 7 This is really a simplistic picture as it can be argued that there are always enough points in an interval, but its visual aid is of great value.

26 (y x ) 1 Lμ(y) = xˆ − x∞ and Lμ− (y) = x + (ˆx x ) y. ∞− ∞ ∞ ∞ − ∞ ∙ We have:

2 1 2 1 RF (x) = Lμ F L− (x) = Lμ F L− (x). μ μ ◦ μ ◦ μ 2 1 1 Where RFμ is the renormalized

function of F that translates J2 = [ , 1 ] onto μ μ − μ I = [0, 1], both by reflecting it upwards and rotating around so that1 1 = 0 and 1 = 1, − μ μ see Figure 1-6 for two explicit examples of renormalization, relating to the issues of μ < 0

1 and 4 < μ. This should make it clear how everything that happened to Fμ will happen

2 1 to Fμ ; of course appropriate Lμ and Lμ− can be found for all n, but this is beyond our current scope, see the work of Kenneth G. Wilson [41, 42]. This sidetrack8 was intended to show how the actual value of μ is very “relative”, as well as the actual interval of x; or to be more precise the actual set of x values. Returning to the 3 < μ region and how the complexity of behaviours increases exponentially with increasing μ. At the moment we have two attracting fixed points, each with its two pre-images, one of them being the other fixed point and the other pre-image has its own two pre-images and so on. The process we have explored up to now will repeat, in fact infinite times, as μ is raised further. We will have a “super-attractive” case, when F 2( 1 ) = 1 , at μ 3.23..., and of course the trapping of yet another two points for μ 2 2 ≈ each one of the “current” two cycle fixed points, and thus the creation of the four fixed point cycle, at μ = 1 + √6 3.44... and of course the change of the two cycle fixed points ≈ into repelling points. The four cycle will become eight cycle and so on for all powers of 2. To our visual picture, we still have the action of the primordial fixed point, 1 1 , even − μ though now, at 3 < μ it is repelling, its pivot action in the spiraling-in still exists. In addition we have more “local” effects from the newborn fixed points acting as pivots fora spiralling-in of their local surrounding points. In effect we have vortices within vortices.

8 Another reason for this diversion is to provide a small glimpse, without going into 2k details, to the picture of Fμ with k , which we will meet soon. ∞ → ∞

27 Figure 1-5. Exhibition of the “trapping” of two points in two different configurations of 1 2 Fμ. (a) Plot of both F3.2, solid line, and F3.2, dashed line showing the eventual evolution of different initial points, marked by , to the 2-cycle fixed points marked by . The solid triangles, , mark the# sequence between the two fixed points. Note that F 2 and the diagonalK can be used like F 1 in Figure 1-2 to n F 1 track the 2 -step evolution of points. (b) Plot of 2+√5 as well as the evolutions of∙ some different initial points, marked by , under multiple

applications of F2+√5. Note that the two points marked# by , are “trapped” F 1 I F 2 by 2+√5, while their four pre-images, marked by , are “trapped” by 2+√5, F F 2∞ and so on to 2+∞√5, and even further to 2+√5 .

As more and more fixed points are created the gaps between them for all this swirling to take place are getting smaller and smaller, leading to smaller and smaller increments of μ to create the subsequent doubling, we will return to this evolution in Section 1.2.2 when we talk about Universality. For now we are interested in what happens at the accumulation point, also know as Feigenbaum point, named after its discoverer Mitchell

Feigenbaum, μ 3.569945671.... Let us attempt to account all the points in I and their ∞ ≈ behaviour starting with the easiest part 0, 1 the two points that collapse to 0. We have { } the fixed point of F 1, that is 1 1 , with it we still have its pre-image 1 and “going further μ − μ μ

28 2 2 Figure 1-6. Renormalization of two cases of Fμ . (a) Case of F3.8, for which 2 1 L3.8 F3.8 L3−.8(x) translates into (b) A plot of RFμ(x), showing how representative◦ ◦ points, marked by symbols, are translated onto points of similar topology. It should also make clear how cases of 4 < μ are “contained” in some 2 2 1 form in 0 < μ 4. (c) Case of F , for which L3 F L− (x) translates into ≤ 3 ◦ 3 ◦ 3 (d) A plot of RFμ(x), showing how representative points, marked by symbols, are translated onto points of similar topology. It should also make clear how cases of μ < 0 are “contained” in some form in 0 < μ 4. ≤

back time” all the points that lead to it, as we mentioned infinite number of them. From

F 2 we have the two fixed points, of course different from1 1 and 1 , each of these two μ − μ μ fixed points has the other one as one of its pre-images, but in addition each pointhasone more pre-image, which of course has its own pre-images, once again in infinite number. Continuing this for each F n with n = 2k as k we can account for all fixed points of μ → ∞ periodicity a power of 2, as already explained above through the period-doubling route, and all their pre-images. With all these points accounted for, it might seem we have run

2k out of points for Fμ at k , to have any fixed points, in fact though we still have just ∞ → ∞ as many points as we accounted for, we have a Cantor set once again and technically our

2∞ fist instance of chaos. The details of the behaviour at the Feigenbaum point(μ ) of Fμ , ∞ ∞

29 1 and for that matter of Fμ , are complicated to describe, both visually and mathematically, ∞ without the introduction of symbolic dynamics, further exposition of Cantor sets and renormalization group operator, concepts which are beyond our current scope. Instead

1 we will proceed in increasing μ further and show how Fμ becomes chaotic on the whole

2∞ interval I = [0, 1] at μ = 4, at which point the behaviours are the same as for Fμ at μ , ∞ ∞ but easier to describe and visualize. We utilize the period doubling route to the Feigenbaum point to introduce the second requirement for chaos, fixed points of periodicity three. The original definition achaotic system by Devaney [36] required of a system: (i) Topological Transitivity, (ii) the set of periodic points to be dense in the space of the system, and (iii) sensitive dependance on initial conditions. Banks et. al. [43] though, showed how transitivity and density of the set of periodic points imply sensitive dependance on initial conditions, while Li and Yorke [44] proved that existence of period three fixed point implies the set of periodic points is dense; in fact they also show how the set of “non-periodic” points is also dense. Before Li and Yorke though, it was Sharkovsky [45, 46] that introduced his famous theorem9 that actually claims more than just period three implies all other periodicities, it actually provides an ordering of the natural10 numbers that will guide us to find fixed points of periodicities other than a power of 2. Theorem 1.1. Sharkovsky’s Theorem.

Given a continuous map f : R R, the ordering of the natural numbers: 1 C 2 C → n n n n 22 C 23 C ... C 2 C ... C 7 2 C 5 2 C 3 2 C ... ∙ ∙ ∙ ... C 7 2 C 5 2 C 3 2... C 9 C 7 C 5 C 3, ∙ ∙ ∙

9 Even though the theorem applies only for maps on the real line, its strong conclusion makes it important to note in any exposition of chaos.

10 Actually the ordering is for more than just the Natural (N) numbers, it covers as many periodicities as all the Reals (R)[47].

30 and if the map has a fixed point of periodicity q, where p C q, then the map also has a fixed point of periodicity p. If the map does not have a fixed point of periodicity p, where p C q, then the map does not have a fixed point of periodicity q. Further, for any p a continuous map does exist which has fixed points of periodicity p, but no fixed points of any periodicity q. The most obvious implication of Sharkovsky’s theorem, which is analogue to the Li-Yorke theorem, is that if a map has a fixed point of period three then it has fixed points of all number periodicities. Of course the specification of the ordering allows usto say more, if a map has a finite number of periodic points then they are all a power of2, and the converse if a map has a periodic point that is not a power of 2 then it has infinite number of periodic points. The second part of the theorem allows for the existence of maps with an infinite number of points of distinct periodicity and yet without all natural number periodicities. Actually the existence of such maps can be utilized in the proof of the theorem to establish this number ordering, and perhaps more importantly to guarantee the success of the threshold control mechanism for chaotic systems, see Section 1.2.3. Furthermore there is a corollary to Sharkovsky’s theorem, which states that if the map f is dependant on a varying parameter the ordering of the “birth” of new period fixed points is given by the Sharkovsky ordering, which brings usbackto Fμ and the further changes as we increase μ. We have already seen the birth of all periods of power of 2 up to the Feigenbaum point, our next step is to show the birth of all other fixed

11 points of even periodicity by increasing μ over the Feigenbaum point to μ2 3.67... and ≈ showing how this region of μ allows only even periodicities.

2 1 1 Readjusting the intervals under observation we define J4 = [Fμ ( 2 ),Fμ( 2 )], or more 1 μ μ μ μ 1 clearly using Fμ h and h( ) = , J4 = [h( ), ]; in addition we define J5 = [h( ), 1 ] ≡ 2 4 4 4 4 − μ

11 2 4 1 √ 3 The algebraic expression for μ2 = 3 [ u +u+1], where u = (19+3 33) ; obtained either 2 1 1 1 1 μ by F ( ) = 1 or sup F − ( ) . μ 2 − μ { μ μ } ≥ 4

31 1 μ and J6 = [1 , ], therefore we have Fμ(J5) J6 and Fμ(J6) J5, for μ < μ < μ2. − μ 4 ⊂ ⊂ ∞ Before focusing on these intervals let us consider the rest of the points in I, of course

μ μ n μ Fμ ( 0, 1 ) = 0 nothing new there; Fμ :( , 1) 0, h( ) and F : 0, h( ) J4 { } 4 → 4 μ 4 → as n . So it is clear that all points in I eventually are
mapped inside J4,
once → ∞ there we have ...J5 J6 J5... so it is impossible to have a point that is mapped → → back on itself after an odd number of applications of the map, furthermore points of all

2 even periodicities are possible and exist. At μ2 we can see what happens to Fμ and it is precisely what has happened to each other even periodicity while increasing μ from

1 the Feigenbaum point, with a small but important difference in the behaviour of 2 . F 2 1 1 1 Specifically μ2 ( 2 ) = μ that is 2 is mapped to the pre-image of the primordial fixed 1 point, so 2 is a pre-image of a fixed point, but not itself fixed as in the “super-attractive” case. Visually this means that there is at least one interval that is completely wrapped

1 around onto itself, in contrast to previously when 2 was “eventually” mapped onto a fixed point. We can also now see the difference between “eventually invariant” and invariant,

2 2 Fμ2 (J5) = J5 and Fμ2 (J6) = J6, note the equality, and see Figure 1-7(a) for a visual of 1 μ the intervals. Perhaps most importantly both pre-images of μ are now less than 4 , which implies that both have two real pre-images, and at least two of those pre-images each have two pre-images of their own, and so on; the fixed point 1 1 has now a set of pre-images − μ which form a Cantor set. When we put all these implications together, along with the fact

2 1 that the Sharkovsky ordering for Fμ is the same as the ordering for Fμ without the last leg

2 2 of the odd integers (so all number periodicities exist for Fμ2 ), it should be clear that Fμ2 is

2 12 chaotic. It is important to note that Fμ2 is chaotic on J4 not the whole I. Therefore now we are left with the last leg of the Sharkovsky ordering, the odd number periodicities. Obviously the most important being period three and the easiest one to visualize since it is relatively “short”. But before we focus on period three lets see

12 2 Actually Fμ is independently chaotic on J5 and J6.

32 how odd periodicities become possible in the first place. So at μ2 < μ we have seen that

2 1 1 2 Fμ ( ) < , so Fμ (J2) J1 = ∅ or to relate two sets from different views J4 J1 = ∅, lets 2 2 μ ∩ 6 ∩ 6 1 1 call this intersection J7 J5, most importantly J7 < Fμ(J7) < 1 Fμ(J7) J5 ⊂ μ ⇒ − μ ⇒ ⊂ ... J6 J5 ... so we have J7 J5 → → → J6 J7, hence the “highest” odd → → infinite9 even times → → periodicity orbit is “born”,| see Figure{z1-7(b). }

Figure 1-7. Demonstration of birth of odd periodicity fixed points. (a) Plot of Fμ2, solid 2 line, and Fμ2, dashed line. We also mark the intervals J4, J5 and J6, it should 2 be clear that Fμ : J5 J6 and Fμ : J6 J5; Fμ is chaotic on both these 2 → 2 → 2 intervals. Note also that Fμ2 : I 0, 1 J4 J4. (b) Plot of Fμ, solid line, 2 { } → and Fμ , dashed line, at μ = μ3 + 0.02, a value for which the period three fixed points are attractive. Note how J5 J1 = J7 = ∅. Within J7 we have one of ∩ 6 3 the period three trapping regions, see inset. (c) Inset, showing a plot of Fμ (x), for μ = μ3 + 0.02 and x [0.1358, 0.1722], topologically the same as some ∈ Fμ(x), for x I. ∈

9 Or none in the case of period three.

33 1 We return now to the issue of the birth of the fixed points of periodicity three for Fμ .

14 At μ3 = 1 + 2√2 we have the birth of the period three orbit, the simplest way to view it is that the solutions to the equation F 3 x = 0, become real from complex. In terms μ − 1 of our visual picture, we have a growth factor which is large enough for 2 to fall outside the “trapping” region. The situation is very similar to what happened at μ = 1 when we

had the introduction of a “new” fixed point in I by F1(I2) I2 = , only this time the ∩ 6 1 1 set of this intersection does not contain a post-image of , F1( ) / F1(I2) I2, so this set 2 2 ∈ ∩ contains some other point or more correctly points. Since now it is not the maximal point that gets captured it means that the intersection can not be collapsed to a single point, but there will always be at least two points, marking the boundaries of this intersection interval, and since at least both these points are “trapped” we have two new fixed points,

3 for each trapping region. Once we consider that we are dealing with Fμ , there are three “trapping” regions, so a total of six new fixed points, along with the two primordial,

0, 1 1 , eight in total agreeing with the fact that F 3 is an eighth degree polynomial15 . { − μ } μ The nature of these points is one of them attracting and the other repelling, since we are talking about an interval being “trapped” the points that actually exist between the two fixed points are also “trapped” by eventually collapsing to the attracting fixed point.So the overall situation in a trapping region is “identical” to the situation we had in I for 1 < μ, see inset (c) of Figure 1-7. Therefore it should be of no surprise that everything that we have talked about up to now for I happens within each “trapping” region over and over again as we increase μ even further; in some sense this is the reason why period three implies chaos and not simply the fact that it allows all other periodicities; even though one argument implies the other.

14 Derivation of this value can be found in [48–50].

15 k k In fact since Fμ is a second order polynomial any Fμ will be a polynomial of order 2 , so solutions to any case of F k x = 0, will have an even number of real roots. μ −

34 Finally μ3 < μ 4, with μ = 4 the most important part, given on Figure 1-8. In ≤ the region between μ3 and 4 everything we have seen repeats infinite number of times.

In some subset of I or other, we have the creation of new other fixed points that turn from stable to unstable giving birth to more and more fixed points in the same way, all the way to the creation of the “period three orbit”, for the subset in question, when, of

1 16 course, we start over again! At μ = 4 we have the final brick on the wall, F4( 2 ) = 1! 1 We have seen the consequences of this before, at μ2, 2 is mapped to 0, the other fixed point, and as before, more importantly 1 μ the pre-images of 0 form a Cantor set; and ≤ 4 after this long journey we finally have F : I I, F is chaotic on the whole interval I. → We have seen all that has been created as we increased μ, and everything is still there, so we will not repeat ourselves. Instead, and in conclusion to this section, we will provide a holistic picture of the points in I. We have a set of points which are fixed, of some period or other, this set is dense in I, there is a difficult issue here, we can exclude from

2∞ 17 this set the fixed points of Fμ , we still have the fixed points of all periodicities, even infinity, but we are only considering the enumerable number periodicities, and this setis

2∞ still dense in I. The set of fixed points of Fμ we can now put on their own and it is in itself a Cantor set, and of course dense in I, but even more, uncountable. We are not done though, there are still “many” points left and specifically the pre-images of all fixed points of enumerable periodicity, which as we have seen for each fixed point form a Cantor set on

2∞ their own. Hence we can consider the following two sets, the fixed points of Fμ (and their pre-images in some sense), and the set of all other fixed points and their pre-images; both sets are an infinite collection of Cantor sets, both dense in I, both uncountable. For its

16 Subsequently we shall drop the subscript on F4 and refer to the function of this parameter simply as F . 17 Note that μ here is not the Feigenbaum point value of μ , instead we are considering ∞ all points of all F 2∞ -functions, even the ones created after period three restarts the process.

35 immense importance, our final point in this section is to emphasise the following, wehave seen how a function on its own is not chaotic, as a set on its own is not chaotic, it is the combination of a many-one function on a specific set, under specific parameters that create chaos.

Figure 1-8. Plot of the function F : I I. Also marked are the intervals I1, I3 and I2, the → last broken up into I2L and I2R , which are the two intervals that are mapped completely onto each other.

36 Table 1-1. Summary of the transitions in behaviour of the different intervals described in Section 1.2. Each arrow represents a single application of Fμ. x is used to indicate that more than one point is trapped in the interval, while y is used to indicate that a single point is trapped (typically an endpoint of the interval).

I = (J1 J5) μ I = I1 I2 I3 = J˜ J˜ I = J1 J2 J3 ∪ ∪ ∪ I 1 3 ∪ ∪ J5 J6 (J3 J6) ∪ ∪ ∪ x I1 I2 0 1 ← → I↑3 y x x x I1 I2 J˜1 J˜2 1 2 → → → I↑3 % ∗ J↑˜3 y x x˜ x˜ y y I1 I2 J1 J3 J1 J2 y y 2 3 → → J5J6 → ↑ % ˜ ˜ ˜ ↑ %. I3 J2 = J1 J3 J3 ∩ y x x˜ x˜ y y I1 I2 J1 J3 J1 J2 3 μ → y y ∞ → → ˜ ↑ %. J5J6 I↑ %. J2 = J1 J3 J3 3 ∩ y x x˜ x˜ y y I1 I2 J1 J3 J1 J2 y y μ μ2 → J5 J6 ∞ →  → ˜ ˜ ˜ ↑ %. I↑ %. J2 = J1 J3 J3 3 ∩ y y yI xI x˜ x˜ y y J˜ J 1 2 J1 J3 J1 J2 5 6 μ2 μ3 → → → I↑ %. ↑ .x 3 J˜2 = J˜1 J˜3 J↑ %. J 3 7 J5=J˜5 J7 ∩ ∪ y y y y x x y y J˜ J I1 I2 J˜ J˜ J1 J2 5 6 μ 4 → 1 3 → 3 y↑ . → I↑ %. ˜ ˜ ˜ J↑ %. J x 3 J2 = J1 J3 3 7 J5=J˜5 J7 ∩1 ∪ ∗The transition shown actually occurs at μ = 1 3 .

37 1.2.2 The Tent Map, Topological Conjugacy and Universality

This section will briefly introduce the Tent map, which along with the Logistic map are the two main systems we use for chaos based algorithm development. The Tent map is a piecewise linear map that also can exhibit chaos. Instead of presenting in detail the chaos in the Tent map, as we have done in the previous section for the Logistic map, we will introduce the concept of Topological Conjugacy, which relates one map to the other, guaranteeing the properties and behaviours in one map to exist in the other. In closing the section we will move further than Topological Conjugacy, into the promised property of chaos known as Universality. Universality plays a very important role in computation with chaotic systems since it releases us from confinement to any single physical realization; as long as a system is chaotic the algorithms we develop can be implemented by the system, regardless of whether the nature of the system is electrical, optical, chemical, or even a realization in some other physical realm.

The Tent map. Just as the Logistic map, the Tent map is discrete time, but instead of a “continuous” polynomial, it is a piecewise linear “discontinuous” mapping of the form:

1 μ xn, for xn < , n+1 ∙ 2 Tμ (xn) = xn+1 =  (1–3)  1 μ (1 xn), for xn, ∙ − 2 ≤  18 where x [0, 1](I) , 0 < n, and for our purposes we set μ = 2 . In this domain and ∈ μ value, the Tent map, except for the curvature from the polynomial terms, looks and ˜ 1 ˜ 1 2 behaves like the Logistic map, we can even mark sub-intervals, I1 = [0, 3 ], I2 = [ 3 , 3 ] ˜ 2 and I3 = [ 3 , 1] with similar behaviour as the Ii of F , see Figure 1-9. In addition, the bifurcation diagram for the Tent map is practically the same as for the Logistic map, see

18 As we have done for the Logistic map, for the Tent map in the case of n n + 1 and μ = 2 we will not use the super- and sub-scripts and refer to the function simply→ as T , unless emphasis is required.

38 Figure 1-10, even though it looks more “compressed” and the bifurcations for the period doubling are closely packed, basically everything that we showed for the Logistic map applies to the Tent map as well. We can actually show this “identity” mathematically through the Topological conjugacy of the two maps.

Figure 1-9. Plot of the function T : I I. From the graph we can see how → T : I1 I1 I2, T : I3 I1 I2, and T : I2L I3 and T : I2R I3; and of → ∪1 → ∪ → → course how 2 is a pre-image of 0.

Topological Conjugacy. Topological conjugacy has its roots in set theory, consider two distinct sets, X and Y, and for each set there is a “relation”, say and P respectively, between its individual elements. For each set consider the “larger” set of Q the union of the elements that are in the domain and converse domain of the “relation”,

39 Figure 1-10. Bifurcation diagram for the Tent map. Eventual evolution of an arbitrary initial state x0. Inset: Magnification of the region μ [1, 1.1] and x [0.495, 0.5], showing more clearly the period-doubling.∈ ∈

the “field” of the set, for example given x X : (x) =x ´, then x andx ´ are in the “field” ∈ P of X and , analogously for Y and . If there is a “relation” , which is one-to-one and P Q S has as domain the field X, : X and converse domain Y, : Y , then the “relations” { P } { Q } and are “similar”, that means and have identical properties, in other words P Q P Q their effects on their respective sets are indistinguishable. When the sets considered are

topological spaces, the case of the Logistic and Tent map on I, and the “relations” actual functions, then the functions are said to be Topologically conjugate to each other. To make this concept more concrete, and show as promised that there is no difference

between the Logistic and Tent maps, consider the two functions F : I I and T : I I; → → the fact that I acts as the domain and converse domain for both functions should not worry us for two reasons, first of the sets (or spaces) considered are actually arbitrary

40 Table 1-2. Topological conjugacy. Given that the Tent map takes every x I on ax ´ I, ∈ ∈ and the Logistic map takes every y I on ay ´ I, they are Topologically ∈ ∈ conjugate when a function, G, exists that takes every x I on a y I and ∈ ∈ everyx ´ I on ay ´ I; so that G T (x) = F G(x) =y ´. Specifically G(x) =∈ sin2 π x ∈. ◦ ◦ 2 ∙ T
I I (x)−→ (´x) G G ↓F ↓ I I (y)−→ (´y)

and in reality unchanged19 it is the “relations” (or functions) that we will actually manipulate, and second, the actual set of points in I for each function is different, both functions are applied on the same “interval”, which contains the same points, but the points themselves are different for each function, except 0, 1 , 1 . So following { 2 } Table 1-2 we have the following relationships: T : I I, F : I I, G : I I → → → and of course for any x, y, x,´ y´ I,T (x) =x, ´ F (y) =y, ´ G(x) = y and G(´x) =y ´, ∈ which imply G T (x) = G(´x) =y ´ or F G(x) = F (y) =y ´. For the Logistic ◦ ◦ and Tent map the conjugating function is given by G(x) = sin2 π x , specifically: 2 ∙ y1 = F G(x0) y1 = G T (
x0) ◦ ◦

1 1 y1 = 4 y0 (1 y0) x0 < x0 ∙ ∙ − 2 2 ≤ 2 π 2 π y1 = 4 sin x0 (1 sin x0 x1 = 2 x0 x1 = 2 (1 x0) ∙ 2 ∙ ∙ − 2 ∙ ∙ ∙ − 2 π
2 π

2 π 2 y1 = 4 sin x0 cos x0 y1 = sin x1 y1 = sin (π π x0) ∙ 2 ∙ ∙ 2 ∙ 2 ∙ − ∙ 2

2
2 y1 = sin (π x0) y1 = sin (π x0) y1 = sin (π x0) ∙ ∙ ∙

19 You can actually consider each domain and converse domain as fixed points in a set and the “relations” as just a “virtual” link between these points in each set, not actually affecting the points.

41 Of course by induction, if not by simply considering function composition, this result applies for any, and at any, n, F n G = G T n, see Figure 1-11 for an illustration. ◦ ◦ When we consider the “spatial” correlation between functions given by Topological conjugacy, in addition to the “temporal” correlation between a function and its future

iterates (e.g. F and F n) given by renormalization, we have a universal correlation between any “functions” that exhibit similar behaviours.

Figure 1-11. Topological conjugacy between evolved states, up to n = 5. The dashed line follows the evolution of G(x0) under the Logistic map (F ), while the n dash-dotted line follows the evolution of T (x0). The action of G(x) both as G G the first step (x0 y0) and last step (x5 y5) is shown along the triangular → → arrows. (f(x) is F (parabola), T (piecewise linear), G (sigmoid), I (identity) ). { }

42 Universality. The theory of Universality is, as the name implies, extremely wide and intricate. It correlates many branches of Physics, from the theory of critical phenomena to Hamiltonian mechanics, and many branches of Mathematics, from statistics to vector spaces. The aim of this section is to present some important qualitative results, both as a hint of proof of Universality, and its main consequence, the independence between “behaviours” and actual physical system or mathematical interpretation. The most formal origins of Universality come from Mitchell Feigenbaum [29, 51, 52], circa 1975 when he discovered the universal constant δ = 4.669.... Originally δ was observed within a system as the “rate” of onset of period doubling as:

μ n+1 μ2n 2 − δ, as n , (1–4) μ n+2 μ n+1 ≡ → ∞ 2 − 2 kth where μ2k is the μ value of onset of the 2 cycle, in terms of Figure 1-12 it is the limit of the sequence b1 , b2 , b3 , ... δ. Almost immediately though with this initial observation, b2 b3 b4 came both the theoretical and experimental confirmation (see Table 1-3) that extended the existence of δ from within a single system to every system that undergoes period doubling. More importantly though, and the actual basis of Universality, out of the theoretical treatment emerges a “convergent function” that encapsulates all such systems. A very rudimentary approach to describe this function is the following, consider any function that relates “points” in a set in the manner we have been concerned with in this chapter. Further, consider each such function as itself a “point” in a set of functions, these

“function-points” converge to a single “function-point”, in some sense just as the bi bi+1 converge to δ. We have seen in the previous section how x, n, μ of F , can all be varied in some way to create a combination of F and an interval from which chaos emerges, now we see that the functional form of F itself can also vary. 1.2.3 Threshold Control and Excess Overflow Propagation

In this short section we will introduce two processes which have extended the breadth of influence of chaos theory, and are widely used in algorithms of chaos based

43 Figure 1-12. Logistic map bifurcation diagram for some values within 3 < μ < μ . “//” ∞ on the axis indicate discontinuity in the displayed points, b1 and b2 are actually longer than shown. For the Logistic map the sequence of the ratios b1 b2 b3 , , ... converges to δ. In other systems the bi can be different from the b2 b3 b4 Logistic map, but the convergent value of the ratios is the same. Inset: Figure 1-1.

Table 1-3. Experimental measurements of Feigenbaum‘s constant (δ) in different systems based on their period doubling. (Adapted from [53].) Hydrodynamic Electronic Optic Acoustic Systems Water Mercury Diode Transistor Laser Helium Observed δ 4.3 0.8 4.4 0.1 4.3 0.1 4.7 0.3 4.3 0.3 4.8 0.6 ± ± ± ± ± ±

computation. We will not go in details for either process, as both are now quite extensive in their own right. Control and synchronisation of chaotic systems [13–15, 54–64] have been studied for almost 20 years now with numerous results; we simply demonstrate the threshold control mechanism, as already mentioned in relation to the second part of the Sharkovsky theorem. The other process we introduce, the excess overflow propagation [65–68], also

44 has a long history and connections to a wide variety of other fields as critical phenomena, phase transitions, cooperative behaviours, and more.

Threshold Control Mechanism. Following Sharkovsky’s theorem and specifically the second part, we know that a map exists for every periodicity and more, that once periodicity q is established periodicity p is guaranteed, where p C q. Therefore if we start with a map which has period 3, the Logistic map F4 for example, this same map can be of some other “maximal”20 period simply by confining the domain of the map, even more this period will seem attractive21 . Basically instead of looking for a whole new different map of “maximal” periodicity q, we take F4 : I I and consider the map which is the → part of F4 : J J, where J = (0, x∗] and x∗ is the maximal point value of the sequence of → points that are the desired orbit, hence the name of the method “threshold control”. The mechanism can be defined as:

F (x), for F (x) x∗, F˜(x) =  ≤ (1–5)   x∗, for x∗ < F (x),

 the map, F (x), is evolved as usual unless it exceeds x∗, in which case it is limited back to x∗ and normal evolution continues, F˜(x) is the resulting part of F (x) which is the map of desired periodicity, see Figure 1-13 for a specific example of period 4 selection, anda demonstration of the process.

3 Specifically for the logistic map, the interval ( 4 , 1) provides thresholds for periodic 3 orbits of all orders greater than 2, for example < x∗ 0.905 forces the map in a cycle of 4 ≤ order 2, 0.9505 x∗ 0.9698 of order 3, 0.905 < x∗ < 0.925 of order 4 and so forth, see ≤ ≤ 3 Figure 1-14 for a more extensive list. Obviously thresholds from [0, 4 ] produce fixed points (period 1).

20 Maximal in the sense of the Sharkovsky ordering. 21 Only because all other periods are repelling.

45 Figure 1-13. Threshold Control Mechanism. F4 the dotted parabola, is confined on the interval J = (0, x∗] and as a result the map F˜4 is produced, the solid parabola. In this case x∗ = 0.915. Empty circles, , mark three different initial conditions and their paths, in dotted lines,# showing how they are “pushed” onto the period 4 sequence of points x1, x2, x3, x4, marked with full circles, .

Excess Overflow Propagation. Up to now we have been considering, in some form or other, just a single chaotic system, and even thought the potential of even a single chaotic system is immense, two is always more than one. Since we are considering more than one system we need to define some way in which the two systems to interact. Currently the method used in chaotic computation is the excess overflow propagation

1 method. We define a “monitoring value”, ∗x, for the state of the “emitting” system, f( x);

46 Figure 1-14. Threshold values for confining the logistic map on orbits of periodicity 2to 50.

1 once this “monitoring value” is crossed over by the actual state of the system, f( x) >∗ x, the difference between the actual state of the system and the “monitoring value”, the

1 2 excess overflow (f( x) ∗ x = E), is propagated to the receiving system(s) f( x + E). − Note that the “emitting” system is not affected in any way, unlike with the threshold control method where we confine the state of the system, also at the “receiving” system the incorporation of the excess overflow can happen both before and after an iteration of the system f(2x) + E. The method is very similar to the threshold control method, but independent, in the sense that the “monitoring value” can act as a threshold control, but is not necessary, the two values can be different, providing us more flexibility. 1.3 Conclusion

We have presented, as briefly as possible, parts of Chaos Theory that are thebasis for Chaotic Computation. Even though this exposition was made in the spirit of Set Theory and Logic, the exact connections were not actually laid out since our forthcoming

47 presentation of algorithms, developed for Chaotic Computation, do not actually reach such lengths. The possibility is there through, and we plan to explore further this view in our future work.

48 CHAPTER 2 INTRODUCTION TO CHAOTIC COMPUTATION Chaotic computation is the exploitation of the richness in behaviours of chaotic systems to perform computational tasks. This chapter deals with different manipulations of state variables that lead to selection of a specific behaviour, without relinquishing access to other behaviours, demonstrating the inherent versatility. Natural parallelism emerges both through cooperative processes of independent chaotic systems and through exploitation of multi-dimensional systems. A very good abstraction of chaotic computation is the translation of functions and data operators from the software realm onto the hardware; direct implementation of the objective task in hardware. Here specifically we almost exclusively present, the theoretical development of algorithms and methods of chaotic computaion and use the Logistic map to illustrate specific examples. For actual physical realizations, and verifications, of these results we direct the reader to resultsof electronic implementations, using Chua‘s circuit [69], and Logistic map circuit [70, 71], and to simulation results [26] of a IR NH3 laser using the Lorentz system [72, 73]. In addition we can direct the reader to the very recent physical realization results for chaotic computation using synchronization [74] and stochastic resonance [75]. 2.1 Number Encoding

The primary requirement for any computational system is representation of data. There need to be methods through which data can be recognized, stored, and reproduced, obviously only these three processes do not make a very capable computer. The absolute universal language is mathematics and so numbers is the most generic form of data representation. Therefore we begin with the different methods available in chaotic computation for number representation. 2.1.1 Excess Overflow as a Number

This is the simplest method for encoding numbers using a chaotic computer and follows from the excess overflow propagation mechanism. For any given threshold (x∗), the

49 n amount by which the system variable monitored exceeds the threshold, E = f (x∗) x∗, − th n for some n iteration such that f (x∗) > x∗, is named “excess overflow” and is used to represent an integer. More specifically, we utilize an interval, K1, for which the dynamical system that drives our chaotic computer contains fixed points for a single iteration, n = 1,

under threshold control, i.e. f(x∗) > x∗, x K1. Given a requirement as of representing ∀ ∈ the set of integer numbers 1,N , we can find from the interval K1 the x∗ which produces { } the largest value of excess Emax and “equate” this with the largest integer we wish to encode, N. As a result, we can define 1 Emax/N δ, where δ is called “unit overflow” ≡ ≡ and every integer in the set 1,N can be represented by an excess overflow of z δ, where { } ∙ z 1,N . Obviously the integer 0 is represented by zero amount of excess, obtained by ∈ { } setting the system at a natural fixed point, i.e. f n(x) = x, n. ∀ As an illustrative example we use the Logistic map, Equation 1–1. For the Logistic map the interval [0, 0.75] produces fixed points under threshold control, note that 0 and 0.75 are the “natural” fixed points of this system and either can be usedto represent the integer 0. By taking the derivative of F (x∗) x∗ we find the threshold − that produces the maximum excess overflow at x∗ = 3/8, with an emitted excess of

Emax = 9/16, see Figure 2-1. Given the set of integers [0, 100] we can encode them using threshold controlled logistic map elements following the steps described above, i.e.

2 −3+√3 4 (−4) (−z δ) − ∙ ∙ ∙ 100 Emax δ = (9/16)/100 and x∗ = 2 ( 4) , z [0, 100], see Figure 2-2. ≡ ⇒ ∙ − ∀ ∈ The extensive versatility of this method will be demonstrated in following sections. We will show implementations of this method not only for other number representation methods, but also in algorithms for decimal and binary arithmetic, as well as for boolean logic operations. 2.1.2 Periodic Orbits for Number Representation

The most immediate extension to the excess overflow encoding method is to consider the behaviour of the dynamical system outside the region already being utilized, i.e.

x / K1. We can utilize the effect of the threshold control mechanism on this interval, K2, ∈

50 Figure 2-1. Emitted excess by thresholding the logistic map in the interval [0, 0.75]. The threshold x∗ = 3/8 produces the largest excess of Emax = 9/16. to stabilize the system to a periodic orbit. Following Sharkovsky’s theorem, see Theorem 1.1 on page 30, since we have a period three orbit, orbits with periods of all other integer values, and more, are guaranteed to exist. Therefore it is an obvious extension to utilize the appropriate orbit to represent its respective integer number. More specifically, for each

n m n > 1 we find a x∗ K2, for which f (x∗) > x∗ and f (x∗) < x∗, m < n. So for every ∈ ∀ integer we have a x∗ that forces the system to emit excess at periodicity n 2, , in ∈ { ∞} some sense the converse of what is shown in Figure 1-14. Clearly any of the thresholds for n = 1 fixed points found in the previous method can be utilized for representation of integer 1 and as before a natural fixed point can be used for integer 0. We will show implementations of this method for integer multiplication, calculating the least common multiple of any set of integers, and in conjunction with the excess

51 Figure 2-2. Encoding the set of integers 0, 1, ..., 100 . We use 100 Emax = 9/16 to { } ≡ obtain each excess z δ and the threshold x∗ that produces it. ∙ overflow method in a third method for representing numbers more in the spirit ofbinary representation. 2.1.3 Representation of Numbers in Binary

In this final example of number encoding methods we will combine the previous two methods, we use both periodic orbits and excess overflow. We can represent a number in binary format by coupling together elements which have their periodicity determined

#digits position by their position away from the radix point as 2 − . The farther away from the radix point an element is the shorter periodicity we give it, with the bit farthest away being on period one. The elements are joined together serially, so that the “generic overflow” generated by each element cascades through the array until it reaches theopen end at the most significant digit, where we have the readout, see Figure 2-3. To be more

specific a binary number aN . . . a1 will be represented by N elements. Each element, j, as in the encoding based on periodicity, will be set at a threshold so that to produce a

52 N j “generic overflow” at periodicity of2 − , if the binary digit aj = 1, and at a threshold

N 1 of 0, if aj = 0, thus generating no overflow. The resulting array is updated 2 − times

N j 1 resulting in a multiple of the unit overflow j=1 aj2 − at the readout. An illustrative example using the LogisticP map is shown in Figure 2-3, where we are using four elements, N = 4, to represent the binary number 1111. Following the example,

a4, the farthest element from the radix point, is set to a threshold that emits excess on

every update, i.e. periodicity one (x∗ (0, 0.75)); a3 is given a threshold x∗ (0.75, 0.905), ∈ ∈ to produce excess on every second update; a2 from x∗ (0.905, 0.925) for an excess every ∈ fourth update; and a1 from x∗ (0.925, 0.926) for an excess every eighth update. As a ∈ result we will collect emitted excess 8 + 4 + 2 + 1 = 15, respectively from each element, as is desired for encoding 1111 in binary. This method allows us to encode large numbers without the need of thresholds that produce proportionally large periodicities, or of thresholds that produce excess in steps of very small δ. This is a good example of the flexibility of chaotic computation. In the specific example we “sacrificed” efficiency, in the number of elements per encoded integer, to allow short periodicities to encode larger numbers. Obviously, the method can be easily modified to encode numbers in any other base representation. 2.2 Arithmetic Operations

We have established not one, but three methods for representing numbers, we should see them in action. The most obvious starting point is the simplest arithmetic operation: addition, which we extend into multiplication, which we extend to the least common multiple problem. 2.2.1 Decimal Addition

There are multiple chaotic computation algorithms for decimal addition. We will present three such algorithms, each one building on its predecessor, from serial addition to parallel to branching.

53 Figure 2-3. Number encoding in binary format. The excess from each element cascades to the one “above” it until the readout is reached. The element closest to the readout, a4, emits on every update, and as we move “down” the chain the elements emit in increasing powers of 2. The overall result is after 8 updates, we have 8 units of excess from a4, 4 from a3, 2 from a2 and 1 from a1, giving us a total of 15 units. Any binary number can be represented with this method. (Adapted from [26].)

Serial. The most straightforward algorithm for addition utilizes the excess overflow encoding method in a serial manner. In reality it is a natural extension of the encoding method, since each number is encoded as a proportional excess we can chain-link the elements and cascade the emitted excess from each element into its neighbour all the way to the end of the chain. Each excess “builds up” on the one after it, all “naturally” summing up at the edge of the chain, see Figure 2-4. As we have seen in Section 2.1.1 the “unit overflow” (δ) works as the proportionality constant between integers and emitted excess, therefore the addition of integers i, j, k, l is simply replicated by

54 iδ + jδ + kδ + lδ = (i + j + k + l) δ, using the avalanching of the excesses. For ∙ this algorithm the computational time is dependent on the adaptive process and the number of terms in the sum. Specifically after a single chaotic update of all the elements, it takes as many adaptation steps as there are terms in the sum to complete the operation.

Figure 2-4. Serial Addition. We recruit as many elements as terms in a given sum. Each element is assigned a number from the sum, encoded using the excess overflow method. The elements are coupled together in a chain such that the excess can flow down the chain. The result is at the open end we collect the sumofall the emitted excesses as a multiple of δ. (Adapted from [25].)

Parallel and Branching. Following from the previous example, consider the sum of four terms, i, j, k, l using the serial addition algorithm. If we take a closer look at the dynamics of the “last” element in the chain (in this case the one encoding l), we see that this element will receive the excess of all preceding elements “simultaneously”. To visualize this consider the “local” dynamics of three elements, once the chaotic update is complete. The excess of the first element is avalanched to the second, where it “builds up” with the local excess at the second element, so the third element will receive the combined excess of the two previous to it as a single excess. This can also be achieved by introducing the two excesses independently of each other, but simultaneously, i.e. arrive

55 at the same time and “build up” on each other locally at the third element instead of at the second. Of course this can be extended to any number of elements preceding the “last” element in the chain. Turning back to our example of adding i, j, k, l, the topology of the connectivity instead of being a chain is now a tree diagram, see Figure 2-5. By turning the chain into a tree we have collapsed the serial addition algorithm to a two step serial addition, regardless of the number of terms in the sum. The first step is to sum all, but one of the terms in parallel and then serially combine the parallel sum to the remaining term, before reading the result at the open end. We need to note that we can not fully parallelize the operation by connecting all terms directly to the open end, since we are not attributing any dynamical properties to the open end, i.e. if the open end had the ability to correctly “build up” excesses it would be identical to all the other dynamical systems used in the sum, making it the “last” element1 . Obviously for a sum of N terms the shortest computational time for the operation is to connect N 1 terms in parallel to the “last” term and perform the operation in two − steps, this would be analogous to increasing the number of “branches” in the network. In case though of connectivity and/or spatial restrictions we could also extend the algorithm by increasing the number of “trunks” in the network, as is shown in Figure 2-6; note that now our computational time is dependant on the number of “trunks”. This is another case of chaotic computation exhibiting its flexibility, we can “sacrifice” temporal performance to satisfy spatial constraints. 2.2.2 Binary Addition

Extending the above addition algorithms for the binary number encoding method is straightforward. The serial addition of binary numbers is realized by connecting the “end bit” of one number with the “end bit” of the following number in a chain manner all the way to the last number and then to the open end, see Figure 2-7. Similarly for

1 This could actually be overcome by inserting to any sum, 0 as the last term.

56 Figure 2-5. Decimal parallel addition. The excess from the elements encoding i, j, k is simultaneously propagated to the element encoding l on a first avalanching step, and on the second avalanching step the collective sum is read at the open edge. (Adapted from [26].)

Figure 2-6. The branching algorithm can be extended to a larger treelike structure. The computational time in this case is proportional to the number of branches in the longest path that terminates at the “last” element. The above network will sum 15 terms in four avalanching steps. (Adapted from [26].)

57 implementing the parallel addition method for binary numbers we connect the “end bit” of all terms in the sum, but one, to the “end bit” of the single term chosen to act as the collection hub for all the excesses, before sending the result to the open end, see Figure 2-8.

Figure 2-7. Schematic representation of the serial addition method for binary numbers. The numbers 7, 5, 2, 1 are encoded as explained above in Section 2.1.2 and the elements are serially connected. The excess overflow builds up as it moves trough the network and 15 units of excess are collected at the OUTPUT. (Adapted from [26].)

58 Figure 2-8. Schematic representation of the parallel addition method for binary numbers. In this case a branching topology is used for the network, where one of the systems acts as a collection hub for simultaneous build up of excess. (Adapted from [26].)

2.2.3 Decimal Multiplication and Least Common Multiple

We can extend any addition method to perform multiplication in the usual way, the product of two numbers m n is a sum of n terms of the number m (and of course × visa versa m terms of the number n). With our two above addition methods we have two obvious ways of implementing multiplication the serial and parallel summation of the terms. Furthermore using the excess overflow encoding of numbers we have a third method; we can use a single dynamical system that emits the appropriate excess on every update, specifically the amount that represents one of the numbers, m δ for instance, and ∙ we update the system n times collecting a total of excess equal to (m n) δ, the product × ∙

59 of the multiplication. This third method utilizes time as a computational quantity, and leads to a fourth multiplication method, which we will also use as a stepping stone for the method of least common multiple of many numbers. Following the periodic orbit encoding method, we can represent each number, of a two number product, using for each number a chaotic element set to emit excess at the appropriate periodicity. Therefore given a product m n we utilize two dynamical × systems, one set to emit every m updates and the other to emit every n updates, the product of the two numbers is given by the number of updates on which the two elements emit simultaneously; more specifically the element emitting every m updates will have its nth emission on the (m n)th update and the same applies for the other element. The × simple extension of this method to a larger number of terms results in an algorithm for the least common multiple of all the terms in consideration2 . 2.3 Binary Operations

The power of conventional computers lies in their ability to perform boolean algebra. Even so their building blocks are restricted by manufacture to one of the two fundamental gates, NOR or NAND, which with suitable combinations, of either, the other logic gates can be reproduced, for example AND(X,Y) NOR(NOR(X,X),NOR(Y,Y)). This clearly ≡ is not the most efficient method for boolean algebra; a much more efficient alternative is for each gate to require only one building block, conversely each building block to be able to perform all gates. We show how a single chaotic element can represent each of all logic gates through simple state manipulations, removing the need for combining elements [32, 76]. Furthermore we show how multi-dimensionality can lead to natural parallelism, and we go even further, exiting the capabilities of conventional algorithms and addressing a problem designed for quantum computation.

2 The case of three numbers in a product is handled serially, find the product of two of the terms, and multiply it by the third term.

60 Figure 2-9. Schematic representation of the method for computing the Least Common Multiple of four numbers. Each element emits excess to the collecting element at the appropriate periodicity for its encoded number. The result at the OUTPUT is excess of the collecting element with magnitude equal to the number of term elements that emitted on the current update. The number of updates that causes the collecting element to emit excess equal to the number of terms (i.e. all term elements emitting simultaneously) is the Least Common Multiple of the terms. (Adapted from [26].)

2.3.1 Logic Gates

As we have shown an important characteristic of chaotic computation is the versatility we have in implementing the same algorithm with different methods. This of course extends to implementations for representation of logic gates, i.e. there are multiple ways we can achieve this representation. In this section we focus on the most

61 straightforward method developed and in order to present the method more clearly we specify it to the logistic map3 , Equation 1–1. The method we use consists of three steps: (a) initialization, (b) chaotic update, and (c) threshold control and excess overflow. Compared to the methods presented in previous sections, the new concept for this method is initialization; i.e. the setting of the initial state of the system, (X0) just before the first chaotic update, based on specific rules. This initial condition of an element is used to define which logic operation it performs andon what set of inputs. Specifically, we initialize a logistic map element by setting its initial value x0 according to:

prog I1 I2 x0 = x + x + x , for gates that operate on two inputs, • prog I x0 = x + x , for gates that operate on one input, • xprog can be thought of as “programming” the gate and xIi as the input values. For an input of logical 1, xIi = δ, and for an input of logical 0, xIi = 0. As before a chaotic update implies the application of the logistic map: x0 F (x0). The control and overflow → mechanism remains the same as well: E = 0 if F (x0) x∗, and E = F (x0) x∗ ≤ − if F (x0) > x∗. Where x∗ is the threshold for the element and E the excess overflow generated. Here in the context of binary algebra where the set of integers contains only 0 and 1, E and δ are actually equivalent4 . Turning to the specific logic gates to be implemented, the following Table 2-1, summarizes the input - output relationships we are to represent.

prog Ii The task is to identify initial conditions, x + x , and threshold values, x∗, for which a chaotic update will result in F (x0) x∗ for where the outputs in the above table ≤

3 Universality of chaotic systems allows us to assume demonstrations on the logistic map can be carried over to any other chaotic system. 4 Equivalence of inputs and outputs is actually a “soft” requirement.

62 Table 2-1. Truth-table for the logic operations AND, OR, XOR, NOR, NAND, NOT, and the identity operation (WIRE).

I1 I2 AND OR XOR NOR NAND I NOT WIRE 0 0 0 0 0 1 1 0 1 0 0 1 0 1 1 0 1 1 0 1 1 0 0 1 1 0 1 1 1 1 1 0 0 0

are 0, and F (x0) > x∗ where the output is 1. As a specific example, for the OR gate we have the following requirements:

I1 I2 prog 1. I1 = I2 = 0, which implies x = x = 0, i.e. x0 = x .The required output is 0, prog which implies F (x ) x∗. ≤

I1 I2 prog 2. I1 = 0 and I2 = 1, which implies x = 0 and x = δ, i.e. x0 = x +δ. The required prog output is 1, which implies F (x + δ) x∗ = δ. This requirement is symmetric to − I1 = 1 and I2 = 0, so the conditions for satisfying both requirements are identical.

I1 I2 0 prog 3. I1 = I2 = 1, which implies x = x = δ, i.e. x = x + 2 δ. The required output prog ∙ is 1, which implies F (x + 2δ) x∗ = δ. − prog All of the above requirements need to be satisfied by the same values for x and x∗, such that all three conditions hold true regardless of inputs. In a similar manner we can provide required conditions for a chaotic element to represent every gate, see Table 2-2.

Table 2-2. Necessary and sufficient conditions for a chaotic element to satisfy the logic operations AND, OR, XOR, NOR, NAND, NOT, and the identity operation (WIRE). Input xI1 + xI2 AND OR XOR prog prog prog 0 F (x ) x∗ F (x ) x∗ F (x ) x∗ prog ≤ prog ≤ prog ≤ δ F (x + δ) x∗ F (x + δ) x∗ δ F (x + δ) x∗ δ prog ≤ prog − ' prog − ' 2δ F (x + 2δ) x∗ δ F (x + 2δ) x∗ δ F (x + 2δ) x∗ − ' − ' ≤ Input xI1 + xI2 NOR NAND prog prog 0 F (x ) x∗ δ F (x ) x∗ δ prog − ' prog − ' δ F (x + δ) x∗ F (x + δ) x∗ δ prog ≤ prog − ' 2δ F (x + 2δ) x∗ F (x + 2δ) x∗ ≤ ≤ Input xI NOT WIRE prog prog 0 F (x ) x∗ δ F (x ) x∗ prog − ' prog ≤ δ F (x + δ) x∗ F (x + δ) x∗ δ ≤ − '

63 Values that simultaneously satisfy the above conditions are easily found. Specifically by choosing δ = 0.25, the OR gate can be realized by choosing values for xprog = 1/8 and for x∗ = 11/16:

prog 1. F (x ) = F (1/8) = 7/16 x∗(= 11/16), ≤ 2. F (xprog + δ) = F (3/8) 11/16 = 15/16 11/16 = 1/4(= δ), − − 3. F (xprog + 2δ) = F (5/8) 11/16 = 15/16 11/16 = 1/4(= δ). − − In fact values that satisfy the conditions for all the gates and δ = 0.25, have been identified and are summarized in Table 2-3.

prog Table 2-3. Initial values, x , and threshold values, x∗, required to implement the logic gates AND, OR, XOR, NOR, NAND, NOT, and the identity operation (WIRE), with δ = 0.25. Value AND OR XOR NOR NAND NOT WIRE xprog 0 1/8 1/4 1/2 3/8 1/2 1/4 x∗ 3/4 11/16 3/4 3/4 11/16 3/4 3/4

The fact that we have a method to implement all logic gates is not impressive, what is impressive is the fact that we can switch from one gate to another from one computational step to the next, and even more, switch very easily and “fast’,’ in a relative timescale. This leads to the concept of on-the-fly hardware re-programming; an architecture based on conventional computation, so all other components can be easily imported, but with the efficiency chaotic computation offers. 2.3.2 Parallel Logic and the Half Adder

In this section we present what is probably the most important extension of performing boolean logic with chaotic systems [32]. The same procedure we illustrated above for the logistic map is implemented on the 2-dimensional neuron model [77]:

2 xn = (xn 1) exp(yn 1 xn 1) + k (2–1) − ∙ − − −

yn = a yn 1 b xn 1 + c (2–2) ∙ − − ∙ −

64 where a = 0.89, b = 0.18, c = 0.28, k = 0.03; these parameter values keep the model dynamics completely chaotic. Two distinct cases are investigated:

1. The possibility of performing XOR gate and AND gate in parallel (Half Adder)

2. Performing two AND gates independently. The first case specifically, is the application of the two gates on the same set ofinputs, each gate being performed in a different dimension. The second case involves operating the two AND gates on different sets of inputs, again each dimension performing one operation. As with the case of the logistic map, the first step is to define the necessary conditions needed to satisfy the truth table of each case, specifically following the truth tables (Tables 2-4, 2-5) we convert them to the conditional Tables 2-6, 2-7. Following the process of the previous section we determine values for the “programming” state shift and threshold value that satisfy these conditions, see Tables 2-8, 2-9.

Table 2-4. Truth table for XOR and AND logic gates on the same set of inputs. (Case 1)

I1 I2 XOR AND 0 0 0 0 0 1 1 0 1 0 1 0 1 1 0 1

We should mention that each case is investigated independently, i.e. there is no requirement that the number of iterations (n) is identical for both cases, or that the value representing a logical 1 (δ) is the same. We have already seen some type of parallel operations with chaotic computing in Section 2.2.1 with the parallel addition, here though we see a clear demonstration of parallelism. The dynamical system performs two completely different operations simultaneously. In fact the next section builds further on the parallel capabilities of chaotic computation, by tackling the complex Deutsch-Josza problem.

65 Table 2-5. Truth table for two AND gates operating on independent inputs. (Case 2) 1 1 2 2 1 1 2 2 I1 I2 I1 I2 AND(I1 ,I2 ) AND(I1 ,I2 ) 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0 1 1 0 1 0 1 0 0 0 0 0 1 0 1 0 0 0 1 1 0 0 0 0 1 1 1 0 1 1 0 0 0 0 0 1 0 0 1 0 0 1 0 1 0 0 0 1 0 1 1 0 1 1 1 0 0 1 0 1 1 0 1 1 0 1 1 1 0 1 0 1 1 1 1 1 1

Table 2-6. Required conditions to satisfied parallel implementation of the XOR and AND gate. (Case 1) Initial conditions XOR AND prog prog x , y xn x∗ yn y∗ prog prog ≤ ≤ x + δ1, y + δ2 xn x∗ δ1 yn y∗ prog prog − ' ≤ x + 2 δ1, y + 2 δ2 xn x∗ yn y∗ δ2 ∙ ∙ ≤ − '

Table 2-7. Required conditions for implementing two AND gates on independent sets of inputs. (Case 2) 1 1 2 2 Initial conditions AND(I1 ,I2 ) AND(I1 ,I2 ) prog prog x , y xn x∗ yn y∗ prog prog ≤ ≤ x , y + δ2 xn x∗ yn y∗ prog prog ≤ ≤ x , y + 2 δ2 xn x∗ yn y∗ δ2 prog prog∙ ≤ − ' x + δ1, y xn x∗ yn y∗ prog prog ≤ ≤ x + δ1, y + δ2 xn x∗ yn y∗ prog prog ≤ ≤ x + δ1, y + 2 δ2 xn x∗ yn y∗ δ2 prog prog∙ ≤ − ' x + 2 δ1, y xn x∗ δ1 yn y∗ prog ∙ prog − ' ≤ x + 2 δ1, y + δ2 xn x∗ δ1 yn y∗ prog ∙ prog − ' ≤ x + 2 δ1, y + 2 δ2 xn x∗ δ1 yn y∗ δ2 ∙ ∙ − ' − '

66 prog prog Table 2-8. Examples of initial values, x , y , and thresholds x∗, y∗, that satisfy the conditions presented in Table 2-6, yielding the parallel operation of XOR and AND gates. In this range δ 0.7, and the number of required iterations to yield the correct result is n =' 10. (Case 1) prog prog x y x∗ y∗ 1.8 1.7 0.44 1.11 1.4 1.65 0.38 1.07

Table 2-9. Examples of initial values xprog, yprog, and thresholds , that satisfy the conditions presented in Table 2-7, yielding operation of two AND gates on independent inputs. For these values, δ 0.0115 and the number of iterations required n = 20; note that the high number' of required iterations and the need for more decimal precision are a direct consequence of the large number of required conditions to be satisfied simultaneously. (Case 2) prog prog x y x∗ y∗ 0.5375 1.0835 0.231 1.454

An important difference from the previous section on single binary gates is thatin this method the number of iterations that are necessary for the conditions to be satisfied is greater than 1, actually in Chapter 5 we report extensive progress in involving the time dimension with chaotic computation. 2.3.3 The Deutsch-Jozsa Problem

The Deutsch-Jozsa problem and its solution algorithm [78] is a celebrated result of quantum computing, since it is the first example of quantum computing outperforming classical algorithms. In addition it has been the stepping stone for the other two major results of quantum computing, Shor’s factoring algorithm [79] and Grover’s search algorithm [80] (details of the chaotic computation search algorithm are given in Chapter 3). In this section we will demonstrate the chaotic computing algorithm for solving the Deutsch-Jozsa problem, which is just as efficient as its quantum counterpart, provides the answer in a single computational step, but in contrast is much more realizable and provides a more apparent result.

67 The problem can be stated as follows: Given a binary domain, i.e. a discrete domain of 2k states, and given an arbitrary binary function f : 0, 1 k 0, 1 , i.e. a function { } → { } that maps every point in the binary domain to a 0 or a 1, determine whether the function is constant, maps the whole domain on either 0 or 1 exclusively, or whether the function is balanced, maps the whole domain in equal terms on 0 and 15 . In other words, evaluate f for every given point6 , and count the number of resulting 0s and 1s for all points, which basically is the conventional approach in solving the problem. As a result the conventional algorithm in the best case requires evaluating the function for only 2 points, i.e. the function results in a 0 (or 1) for the first point considered, and the opposite for the second point thus the function is balanced. In the worst case though, i.e. when the first N/2 points give the same result, it would take

k 1 2 − + 1 evaluations to conclude whether the function is constant or balanced, i.e. the number of computational steps needed grows exponentially with the number of bits that define the domain. In the context of chaotic computation the situation is much simpler since we can perform the given function on all domain points simultaneously, furthermore we can read the result in one step. To make the demonstration of the algorithm clearer we explicitly use the tent map, Equation 1–3 and work in a state space that applies to this map. Also we partition the explanation of the algorithm in three steps: (a) definition of the domain, (b) definition of the function space, and (c) implementation of the function alongwith reading the result. As is typical of chaotic computation there are at least three different

5 We are guaranteed by the problem that the function will be either constant or balanced, exclusively. 6 The number of points in a given domain is given as: N = 2k, where k is the number of binary digits considered.

68 implementations of the algorithm [26], below we present, with some modifications from [27], the most elaborate, but clear realization. (a) Consider a binary domain space ( k) defined by points of length k binary digits, B therefore the number of points in k is equal to N = 2k. Since we are to work with the B tent map we will translate every point in k, onto a point in the domain of the tent map B k [0, 1]. Take any of the 2 points as a1a2a3...ak, where each ai 0, 1 , we map each such ∈ { } point onto [0, 1] using7 :

k −(k+1) −i x = 2 + ai2 (2–3) i=1 X We encode the whole domain on an array of N elements, denoted by XN , each element(j) having a state x(j) given by Equation 2–3. To illustrate this, consider the case where k = 3, the binary domain ( 3) has eight points 000, 001, 010, ..., 111 which B { } translate onto the tent map domain as 1 , 3 , 5 , 7 , 9 , 11 , 13 , 15 , each of these points { 16 16 16 16 16 16 16 16 } is used as the state value of a dynamical element in an eight element array. So the whole domain in consideration is encoded by a single array as:

1 3 5 7 9 11 13 15 X8 = x(1) = x(2) = x(3) = x(4) = x(5) = x(6) = x(7) = x(8) = . { 16 16 16 16 16 16 16 16 } (b) The function space of the functions: f : k 0, 1 consists of 22k functions, B → { } that is for each of the N = 2k points there are two output possibilities. Regardless of the total number of possible functions only two are constant, the function that has outputs all 1 and the function that has outputs all 0, for all input points. The number of balanced functions though, is dependant on k and is given by simple combinatorics as

L L! CM = (L M)!M! , where for a set of L items, made up of only two distinct objects, C is the − number of combinations where there are M items of one of the two objects. In the case of balanced functions for a binary domain of N points the above relationship becomes

7 The 2−(k+1) term is added so that points 0 and 0.5 are not used for encoding.

69 N N! CN/2 = (N (N/2))!(N/2)! . More specifically for our example of k = 3, we have a total number − 23 8 of possible functions 2 = 256, out of which two are constant and (C4 =)70 are balanced. In our case, of the function domain being represented by an array of tent maps as described above, the function space will be populated with functions which are constructed out of combinations of the following two basis functions:

1 xn+1 = T (xn) = 1 2 xn (2–4) − ∙ − 2

1 xn+1 = T (xn) = 1 T (x n) = 2 xn (2–5) − ∙ − 2

e where T is the tent map and T the inverted tent map, and n is used to denote the time

2k step. Each one of the 2 functionse is constructed as one of the different combinations of the two basis functions of length 2k, specifically the function space looks like: T (1)T (2)...T (j),T (1)T (2)...T (j), ... { ..., T (1)T (2)...T (j 1)T (j), ... e − ..., T (1)T (2)...T (j) , e e } k where j = 2 , and each sequence of length j of the two basis functions, ise onee of the e possible functions (F) to be applied on the domain space8 . Referring back to our concrete example of k = 3, we have 256 combinations of T and T in the function space, ranging from the single sequence of eight consecutive T , to 70 combinations of four f : 0, 1 k 0 and four f : 0, 1 k 1 , to the single case of e { } → { } { } → { } eight consecutive T ; and of course the other 184 combinations9 .

e

8 In our notation here we use j instead of N to indicate the relation between the functions and the array of dynamical elements encoding the domain space. 9 Care not to confuse the 70 functions of four T and four T with the 70 balanced functions! A function that is balanced in T and T is not necessarily balanced in its output. e e

70 Figure 2-10. Basis function T - Tent Map.

(c) Turning to the specific processes involved in solving the problem, we will focus on the example of k = 3 to make this section more illustrative. As we have seen for the 3-bit case there are 256 functions, in the context of the problem, we are given one of these functions (F) (i.e. a specific sequence of 8 T and/or T ) and are assured that it

is either constant or balanced, the task is to find which onee it is. In a straightforward

manner we setup the array of dynamical elements, X8, to encode the binary domain

n as explained in (a) and then we apply the given function on the whole array F(X8 ) =

n+1 X8 , where n denotes the time step. Down to the scale of the individual elements we have T (xn(j)) = xn+1(j) and T (xn(j)) = xn+1(j), where j = 1, ..., 8, whether

e

71 Figure 2-11. Basis function T - Inverted Tent Map.

e it is T or T applied on the jth element is based on which function (F) we are given.

Once the functione is applied we threshold the elements at x∗ = 0.5 and collect the excess the usual way. Basically we are using 0.5 as a separatrix of the state space, i.e.

n+1 n+1 n+1 x (j) > 0.5 1 x (j) 0.5 = Ej and x (j) 0.5 0,Ej = 0 (where E is ≡ ⇒ − ≤ ≡ emitted excess), this is a standard application of symbolic dynamics. Now we have the collected excess in a single step we also have the answer: If the collected excess is 0, then the function we have is the constant function F : k 0 k, B → { }

72 i.e. the function that has output all 0; If the collected excess10 is 2k/4 = 8/4 = 2, which is the maximum possible excess for 3, then we have the constant function F : k 1 k, B B → { } i.e. the function that has output all 1. Obviously if our collected excess is some other value we have a balanced function. This is as much as we are required by the original problem, but we can do even better, using the excess collected we can determine in which of five classes the given balanced function belongs to. The importance of this algorithm is not in the actual task it accomplishes, which is of little practical use. The primary importance is, like for the quantum analogue, in demonstrating an extremely higher efficiency than conventional computation in solving problems of this class. Further though for chaotic computation it is a milestone against quantum computation as well. It shows how chaotic computation can handle problems as well as quantum computation, if not better. 2.4 Conclusion

This chapter almost exclusively dealt with theoretical developments during the first four years of Chaotic Computation (1998-2002), while we relegated the experimental realizations to the references, so that to maintain a uniform global tone to this dissertation. In closing and without wishing to undermine the other results, once again we draw attention to the important result of the solution to the Deutsch-Jozsa problem, and ask the reader to consider the connections between this problem, Set Theory, Logic, and to be more specific to the structure of a chaotic function at the Feigenbaum point.

10 1 The factor of comes from the fact that the average excess emitted Ej is 0.25. 4 h i

73 Figure 2-12. Four realizations of the chaotic Deutsch-Jozsa algorithm for the case k = 3. There are eight binary inputs 000, 001, 010, 011, 100, 101, 110, 111 and 1 3 5 7 9 11 13 15 through Equation 2–3 we obtain the state values 16 , 16 , 16 , 16 , 16 , 16 , 16 , 16 each given to an element of an array. The vertical lines mark the state value of each element x(j). The horizontal line is the separatrix at 0.5. (a) Given the constant function F : k 1 k, this is the function: T TTTTT T T , the application of the functionB brings→ { } every element over 0.5, the maximum excess of 2 is emitted from the array. (b) Given the constante e functione e F : k 0 k, TT T T T TTT , all array elements remain under 0.5 resulting in B → { } zero excess. (c) Given the balanced function of eight T functions, produces an excess of 1. (d) A randomlye e e e chosen balanced function, four elements are over 0.5 and four under, the excess is 1.25. e

74 Figure 2-13. The total excess emitted from each of the 72 functions. The two squares indicate the two constant functions, referring back to Figures 2-12(a,b). As you can see the balanced functions (circles) separate in five groups. The points (c) and (d) refer to the respective graphs of Figure 2-12.

75 CHAPTER 3 SEARCHING AN UNSORTED DATABASE In this chapter we present a chaotic computation algorithm for searching an unsorted database for a match between a queried for item and the contents of the database [28]. The dynamical system we use for the demonstration of the algorithm is the Tent map, given in Equation 1–3. In general most commonly used devices for storing and processing information are based on the binary encoding of information, i.e. upon bits. Larger chunks of information are encoded by combining consecutive bits into bytes and words. Here we show a different approach for information encoding and storage, based on the wide variety of patterns that can be extracted from nonlinear dynamical systems. We specifically demonstrate the use of arrays of nonlinear dynamical systems (or elements) to stably encode and store information (such as patterns and strings). Furthermore we demonstrate how this storage method enables the efficient and rapid search for specified items of information in the data store. It is the nonlinear dynamics of each array element that provides flexible capacity storage, as well as the means to preprocess data for exact and inexact pattern matching. In particular, we choose chaotic systems to store and process data through the natural evolution of their dynamics. More importantly perhaps we note that, our method involves just a single procedural step, it is naturally set up for parallel implementation and can be realized with hardware currently employed for chaos-based computing architectures. We first show a slightly modified storing and encoding scheme, based on theExcess overflow scheme of Section 2.1.1, in which we use the actual fixed point state of the system, instead of the excess generated. Specifically we demonstrate this scheme as applied to the Tent map for storing, and associate the storage with more than just numbers, different encodings. We then show the actual “search process” and howthe results are a direct consequence of the non-linear nature of the Tent map. Finally with

76 specific examples we show implementations of the method for not only exact matches, but for also “inexact”, approximate matches, to a given target. 3.1 Encoding and Storing Information

Consider a list of N data storage elements (labeled as j = 1, 2, ..., N) in an array, where each element stores and encodes of one of M distinct items. N can be arbitrarily large and M is determined by the kind of data being stored. For instance when storing English text we can consider the letters of the alphabet to be each a naturally distinct item, so M = 26. For the case of data stored in decimal representation M = 10, and for work in bioinformatics (manipulating the symbols A, T, C, and G) we have M = 4. We can also consider strings and patterns as the items. For instance for manipulating English text we can use a large set of keywords as the basis, necessitating very large M. We store this list of N elements by N dynamically evolving chaotic elements. The state of the

j elements at discrete time n is given by Xn[m], where (j = 1, 2, ..., N) indexes each element of our list and (m = 1, 2, ..., M) indexes an item in our ”alphabet” (namely one of the M distinct items). To reliably store information one must confine each dynamical system to a fixed point behaviour, i.e. a state that is stable and constant throughout the dynamical evolution of the system over time n, basically so that the list remains unchanged. We therefore employ a threshold control mechanism, see Section 1.2.3, to flexibly control the dynamical elements onto the large set of period 1 fixed points.

2 Specifically for the tent map, thresholds in the range [0, 3 ] yield fixed points, namely 2 Xn = T, for all time, where T is a threshold from 0 T . See Figure 3-1 for a ≤ ≤ 3 schematic of the tent map under the threshold mechanism, which is effectively described

2 by a “be-headed map”. It is clear from Figure 3-1 that in the range [0, 3 ] the value of Xn

lies above Xn implying that the system with state Xn at threshold T will be mapped to a state higher than T in the subsequent iterate and thus will be clipped back to T. Another way of graphically rationalizing this is to note that fixed point solutions are obtained

where the Xn+1 = Xn line intersects the “beheaded” tent map. The value of X at the

77 intersection yields the value of the fixed point, and the slope at the intersection naturally

2 gives the stability of the fixed point. It is clear from Figure 3-1 that in the range [0, 3 ] this intersection is on the “plateau”, namely the fixed point solution is equal to the threshold value. Further the solution of the fixed point for this map is super stable as the slopeis exactly zero on the “plateau”. This makes the thresholded state very robust and quite insensitive to noise.

Figure 3-1. The Tent map under the threshold mechanism. Shown two cases of threshold 1 1 control at 4 and 2 . Effectively each threshold value is on a “plateau” yielding a fixed point. The symbol M indicates the action of the Tent map on the thresholded value Xn < Xn+1, while H indicates the effect of the control 2 Xn+1 T, for X values in the range [0, ]. → 3

78 Returning to our data of a given “alphabet” we can take a large set of thresholds T[1], T[2], ..., T[M] from the fixed point range, setting up a one-to-one correspondence { } of these M thresholds with the M distinct items of our data. This allows each item m to be uniquely encoded by a specific threshold T[m], with (m = 1, 2, ...M). So the number of distinct items that can be stored in a single dynamical element is typically large, as the size of M is limited only by the precision and resolution of the threshold setting and the noise characteristics of the physical system being employed. Therefore given an unsorted list of N data, chosen out of the “alphabet” of M items, as described above, we setup an array of N elements (labeled as j = 1, 2, ..., N), each a Tent map, each with a threshold Tj[m] reliably storing and encoding the appropriate item of the list. That is, if element j holds item m in the unsorted list, the threshold value of element j is set to Tj[m], without changing, or in any way affecting any parameter of the list. So by denoting the threshold of element j by Tj[m] we have the following:

j j if the state of element j of the system, Xn[m], exceeds its prescribed threshold T [m]

j j j j (i.e. when Xn[m] > T [m]) the state Xn[m] is reset to T [m]. Since the thresholds lie in the range yielding fixed points of period 1, this enables each element to hold its stateat

j j value Xn[m] = T [m] for all times n. In our encoding for a reason that will become apparent in the next section, the

1 thresholds are chosen from the interval (0, 2 ), namely a subset of the fixed point window 2 [0, 3 ]. For specific illustration, without loss of generality consider each itemtobe represented by an integer m, in the range [1,M]. Defining a resolution r between each integer as:

1 1 r = , (3–1) 2 ∙ (M + 1) gives us a lookup table, mapping the encoded item to the threshold, specifically relating the integers m in the range [1,M] to the thresholds Tj[m] in the range [r, 1 r] by: 2 −

79 Tj[m] = m r. (3–2) ∙ Therefore we obtain a direct correspondence between the set of integers 1 to M, where each integer can represent any item, and a set of M threshold values of a dynamical system. Even more we can store N list elements by setting appropriate thresholds, via Equation 3–2, on N dynamical elements. As mentioned before, the thresholded states encoding different items are very robust to noise since they are superstable fixedpoints. Finally this correspondence, or representation, is important for the process of encoding information in an M-level representation and, as we shall see below, it is primarily important for the process of searching the list for certain bits of information, by utilizing a specific property of the system. 3.2 Searching for Information

Once we have a given list stored by setting appropriate thresholds on N dynamical elements, we can query for the existence of a specific item in the list. Here we show how the manner in which the information is encoded helps us preprocess the data such that the effort required in the pattern matching searches is reduced. Specifically wewill demonstrate how we can use one global operational step to map the state of elements with the matching item to an unique maximal state that can be easily detected. Note that such an operation enables us to detect matches to strings/patterns (of length equivalent to log2 M binary bits) in one step. It would take typically log2 M steps to do the same for the case of binary encoded data. When searching for a specific item in the list, we globally shift the state ofall elements of the list up by the amount that represents the queried item. Specifically the

j j state Xn[m] of all the elements (j = 1, ..., N) is raised to Xn[m] + Q[k], where Q[k] is a search key given by:

1 Q[k] = T[k], (3–3) 2 −

80 where k is the item being searched for, and T[k] its respective threshold. This addition shifts the interval that the list elements can span, from [r, 1 r] to [r + Q[k], 1 r + Q[k]], 2 − 2 − where Q[k] is the globally applied shift. Note that what we are searching for is the representation of the item, not the item itself. For example, we can encode each letter of the alphabet by a number, such that the lowest threshold Tj[1] represents the letter A, the next highest Tj[2] represents B, etc. When we search for A, we are really searching for the element with a state with threshold Tj[1]. The item being searched for is encoded in a manner “complimentary” to the encoding of the items in the list (much like a key that fits a particular lock); i.e. Q[k]+T[k] adds up

1 to 2 . This guarantees that only the element(s) matching the item being searched for will 1 1 have its state shifted to 2 . The value of 2 is special in that it is the only state value that on the subsequent update of the system will reach the value of 1.0, which is the maximum state value for the Tent map. So only the elements holding an item matching the queried item will reach the extremal value 1.0 on the dynamical update following a search query. Note that the important feature here is the nonlinear dynamics mapping uniquely the

1 1 state 2 to 1, while all other states (both higher and lower than 2 ) get mapped to values lower than 1. See Figure 3-2 for a schematic of this process.

1 The salient characteristic of the point 2 is the fact that it is the unique critical point, and so it acts as “pivot” point for the nonlinear dynamical folding that will occur on the interval [r + Q[k], 1 r + Q[k]] during the next update. This provides us with a single 2 − global monitoring operation to push the state of all the elements matching the queried item to the unique maximal point in parallel. The crucial ingredient is the use of the existing critical point in the dynamical mapping to implement selection. Chaos is not strictly necessary here. It is evident that for unimodal maps higher nonlinearities allow larger operational ranges for the search operation and also enhance the resolution of the encoding. For the Tent map specifically, it can be shown that the minimal nonlinearity necessary for the above search operation to work is operation in the chaotic region.

81 Figure 3-2. Schematic representation of the changes in the state of an element for (i) a matching queried item, (ii) an item “higher” than the queried item, and (iii) 1 an item “lower” than the queried item. The “key” value used is Q = 4 , so the matched item as a value of 0.25. The behaviour of three values is shown 0.1, 0.25 and 0.3. It is clear that the application of the “key” does not seem to relatively affect the three values, a simple linear translation. The application of the map though, indicated by M, clearly maps both 0.1 and 0.3 to lower states than the maximal state, acquired solely by 0.25.

82 Another specific feature of the tent map is that its piecewise linearity allows the encoding and search operation to be very simple indeed. Of course to complete the search we must now detect the maximal state located at 1. This can be accomplished in a variety of ways. For example, one can simply employ a level detector to register all elements at the maximal state. This will directly give the total number of matches, if any. So the total search process is rendered simpler as the state with the matching pattern is selected out and mapped to the maximal value, allowing easy detection. Even more, by relaxing the detection level by a prescribed “tolerance”, we can check for the existence within our list of numbers or patterns that are close to the item or pattern being searched for. In this case “close to” means “having a representation that is close to the representation of the item for which we are searching for”. Using the earlier example of English letters of the alphabet encoded using the lowest threshold Tj[1] for A, the next higher threshold for B, etc., relaxing the detection threshold a small amount allows us to find mistyped words, where L or N were substituted for Morwhere X or Z were substituted for Y. However, if we had chosen our representation such that the ordering put T and U before and after Y (as is the case on a standard QWERTY keyboard), then our relaxed search would find spellings of “bot” or “bou” when “boy” was intended. Thus “nearness” is defined by the choice of the representation and can be chosen advantageously depending on the intended use. Figure 3-5 gives an illustrative example of detecting such inexact matches. So nonlinear dynamics works as a powerful “preprocessing” tool, reducing the determination of matching patterns to the detection of maximal states, an operation that can conceivably be accomplished by simple addition and in parallel. 3.3 Encoding, Storing and Searching: An Example

Consider the case where our data is English language text, encoded as described above by an array of tent maps. In this case the distinct items are the letters of the

1 English alphabet. As a result M = 26 and we obtain r = 54 = 0.0185185... from Equation

83 3–1, and the appropriate threshold level for each item is obtained from Equation 3–2. More specifically, consider as our list the sentence “strawberry fields”. Each letter1 in this sentence is an element of the list with a value selected from our 26 possible values and can be encoded using the appropriate threshold, as in Figure 3-3(a). Now the list, as encoded above, can be searched for specific items. Figure 3-3 presents the example of searching for the letter “l”. To do so the search key value corresponding

15 to letter “l” (from Equation 3–3, Q[l] = 54 ) is added globally to the state of all elements. Then through their natural evolution, at the next time step the state of the element(s) containing the letter “l” is maximized. In Figure 3-4 we performed an analogous query for the letter “e”, which is present twice in our list, to show that multiple occurrences of the same item can be detected. Finally in Figure 3-5 we search for an item that is not part of our given list, the letter “x”. As expected Figure 3-5(c) shows that none of the elements are maximized. By lowering the detection level to the value 1 (2 r), we have detected − ∙ whether adjacent items to the queried one are present. Specifically we have detected that the letters “w” and “y” are contained in the list. This demonstrates that inexact matches can also be found by this scheme. 3.4 Discussion

A significant feature of the presented search method is that it employs asingle simple global shift operation and does not entail accessing each item separately at any stage. It achieves this through the use of nonlinear folding to select out the matched item, and this nonlinear operation is the result of the natural dynamical evolution of the elements. So the search effort is considerably simplified because it uses the native responses of the nonlinear dynamical elements. We can then think of this as a natural application, at the machine level, in a computing machine consisting of chaotic modules [25–27, 32, 69, 70, 76, 81–85]. It is also equally potent as a special-applications “search

1 The space between the words is ignored.

84 Figure 3-3. Searching for “l”. (a) Threshold levels encoding the sentence “strawberry fields”, bars marked as ; (b) the search key value for letter “l” is added to all elements, bars marked as ; (c) the elements update to the next time step, bars marked as . For clarity we marked solid black any elements that reach the detection level.  chip”, which can be added on to regular circuitry and should prove especially useful in machines, which are repeatedly employed for selection/search operations. In terms of the processor timescale, the search operation requires one dynamical step, namely one unit of the processor‘s intrinsic update time. The principal point here is the scope for parallelism that exists in our scheme. This is due to the selection process occurring through one global shift, which implies that there is no scale-up (in principle) with size N. Additionally conventional search algorithms work with ordered lists, and the time required for ordering generically scales with N as O(N log N). Here in contrast, there is no need for ordering, and this further reduces the search time. Regarding information storage capacity, note that we employ an M-state encoding, where M can be very large in principle. This offers much gain in encoding capacity. As

85 Figure 3-4. Searching for “e”. (a) Threshold levels encoding the sentence “strawberry fields”, bars marked as ; (b) the search key value for letter “e” is added to all elements, bars marked as ; (c) the elements update to the next time step, bars marked as . For clarity we marked solid black any elements that reach the detection level.  in the example we present above, the letters of the alphabet are encoded by one element each; binary coding would require much more hardware to do the same. Specifically, consider the illustrative example of encoding a list of names, and then searching the list for the existence of a certain name. In the current ASCII encoding technique, each ASCII letter is encoded into two hexadecimal numbers or 8 bits. Assuming a maximum name length of k letters, this implies that one has to use 8 k binary bits per name. So typically ∙ the search operation scales as O(8kN). Consider in comparison what our scheme offers: if base 26 (“alphabetical” representation) is used, each letter is encoded into one dynamical system (an “alphabit”). As mentioned before, the system is capable of this dense encoding as it can be controlled on to 26 distinct fixed points, each corresponding to a letter. Again assuming a maximum length of k letters per name, one needs to use k “alphabits” per

86 Figure 3-5. Searching for “x”. (a) Threshold levels encoding the sentence “strawberry fields”, bars marked as ; (b) the search key value for letter “x” is added to all elements, bars marked as ; (c) the elements update to the next time step, bars marked as . It is clear that no elements reach the detection level at 1.0; (d) By lowering the detection level we can detect whether items “adjacent” to “x” are present. For clarity we marked solid black any elements that reach the detection level (“w” and “y”). name. So the search effort scales as kN. Namely, the storage is 8 times more efficient and the search can be done roughly 8 times faster as well! In general if base S encoding is employed, for example where S is the set of all possible names (size(S) N), then ≤ each name is encoded into one dynamical system with S fixed points (a “superbit”). So one needs to use just 1 “superbit” per name, implying that the search effort scales simply as N, i.e. 8k times faster than the binary encoded case. Even more ,in practice the final step of detecting the maximal values can conceivably be performed in parallel. This would reduce the search effort to two time steps (one to map the matching item to themaximal value and another step to detect the maximal value simultaneously). In that case the search effort would be 8kN times faster than the binary benchmark.

87 Alternate ideas to implement the increasingly important problem of search have included the use of quantum computers [80]. However, our nonlinear dynamical scheme has the distinct advantage that the enabling technology for practical implementation need not be very different from conventional silicon devices. Namely, the physical design of a dynamical search chip should be realizable through conventional CMOS circuitry. Implemented at the machine level, this scheme can perform unsorted searches efficiently. CMOS circuit realizations of chaotic systems, like the tent map, already operate beyond the region of 1 MHz [86, 87]. Thus a complete search for an item comprising of search key addition, update, threshold detection, and list restoration can be performed at 250 kHz, regardless of the length of the list. Even more though, nonlinear systems are abundant in nature, and so embodiments of this concept can be conceived in many different physical systems ranging from fluids to electronics to optics. Potentially good candidates for physical realization of the scheme include nonlinear electronic circuits and optical devices [88]. Also systems such as single electron tunneling junctions [89], which are naturally piecewise linear maps, can conceivably be employed to make such search devices. In summary we have presented a method to efficiently and flexibly store information using nonlinear dynamical elements. We demonstrate how a single element can store M distinct items, where M can be large and can vary to best suit the nature of the data being stored and the application at hand. Namely, we have information storage elements of flexible capacity, capable of naturally storing data in different bases or indifferent alphabets or with multilevel logic. This cuts down space requirements by log2 M in relation to elements storing via binary bits. Further we have shown how this method of storing information can be naturally exploited for searching of information. In particular, we demonstrated a method to determine the existence of an item in an unsorted list. The method involves a single global shift operation applied simultaneously to all the elements comprising the list, such that the next dynamical step “pushes” the element(s) storing the matching item (and only those) to a unique, maximal state. This extremal state can then

88 be detected by a simple level detector, directly giving the number of matches. Even more the maxima state can be treated as a maximal range, in which case approximate matches are identified as well.

89 CHAPTER 4 A SIMPLE ELECTRONIC IMPLEMENTATION OF CHAOTIC COMPUTATION This chapter is a short exposition of the results of our publication [34] concerning an iterated map with a very simple (i.e. minimal) electronic implementation. We first propose and characterize the map and then provide the circuit to implement the map. We proceed to determine control thresholds for flexibly representing the five fundamental logic gates and demonstrate how this map (and circuit) can be used to implement the search algorithm introduced in Chapter 3. 4.1 An Iterated Nonlinear Map

We begin by considering an iterated map governed by the following equation:

α xn xn+1 = ∙ β , (4–1) 1 + xn where α and β are system parameters. Figure 4-1 shows the bifurcation diagrams for different values of α and β. It is evident that this map yields dynamics ranging from fixed points through chaos. It is also clear that the map follows the period-doubling route to chaos with respect to α, and it does so as well with respect to β. In the following sections we will consider the map in the chaotic regime, with α = 2 and β = 10, namely the chaotic map given by:

2 x x n , n+1 = ∙ 10 (4–2) 1 + xn This operating point is indicated by the dotted line in the bottom right panel of Figure 4-1 and the graphical form of this map is presented in Figure 4-2. 4.2 Threshold Control Chaos into Different Periods

Using the map given by Equation 4–2, we wish to construct a system that can represent M distinct states, where M can be large. The size of M will be limited only by our ability to distinguish one state from the next in the presence of noise. To do this we

90 Figure 4-1. Bifurcation diagram of the iterated map in Equation 4–1 for various values of α and β. The dotted line in the bottom right panel, indicates the chosen operating point as prescribed by Equation 4–2. Here x is the value taken by the map after initial transients have died out. ∞

use the simple and easily implementable threshold control mechanism described in Section 1.2.3. Specifically we place under control the state variable x as:

2 xn ∙ 10 , for xn+1 x∗, 1+xn xn+1 =  ≤ (4–3)   x∗, for x∗ < xn+1, x  where ∗ is the imposed threshold value. The effect of this control is to limit the available phase space by clipping the state variable. In this method no parameters are adjusted, and only one state variable is occasionally reset. Note that this scheme is computationally simple and requires no costly run-time computations. Figure 4-3 illustrates the behaviour of the system under varying thresholds.

Depending on the value of the threshold x∗, this control method produces a wide variety of nonlinear dynamical behaviours ranging from fixed points to periodic behaviour of various

91 Figure 4-2. Graphical form of the map to be implemented by an electronic circuit. The parameters for this form are set at α = 2 and β = 10. periodicities to chaos. As indicated in the figure, the system is controlled to fixed points for thresholds for x∗ < 1. When the threshold is above unity, many different periodic (as well as chaotic) orbits become available. 4.3 Electronic Analog Circuit: Experimental Results

The realization of the discrete map of Equation 4–2 in circuitry is depicted in

Figure 4-4. In the circuit Vin and Vo denote input and output voltages and in terms of the equation xn and xn+1, respectively. A simple nonlinear device is constructed by coupling two complementary (n-channel and p-channel) (Q1, Q2) junction field-effect transistors

2x (JFETs) [90] mimicking the nonlinear characteristic curve f(x) = 1+x10 . The voltage across resistor R1 is amplified by a factor of 5 using the operational amplifier U1 inorder to scale the output voltage back into the range of the input voltage, a necessary condition for a circuit based on a map. The resulting voltage characteristics of the nonlinear device are depicted in Figure 4-5, compare with the mathematical form of the map in Figure 4-2.

92 Figure 4-3. Effect of threshold value x∗ on the dynamics of the system given by Equation 4–3.

In order though to realize the map of Equation 4–3, we require two more sample and hold circuits, in addition to a threshold controller circuit, see Figure 4-6. The first sample and hold (S/H) circuit holds the input signal (xn) in response to a clock signal CK1. The output from this sample and hold circuit is fed as input to the nonlinear device for the subsequent mapping, that is Equation 4–2. A second sample and hold (S/H) circuit takes the output from the nonlinear device in response to a clock signal CK2. In lieu of control,

nd st the output from the 2 sample-and-hold circuit (xn+1) closes the loop as the input to 1 sample-and-hold circuit, through the threshold control circuit. The main purpose of the two sample-and-hold circuits is to introduce discreteness into the system and additionally

93 Figure 4-4. Circuit diagram of the nonlinear device of Equation 4–3. (Left) Intrinsic (resistorless), complementary device made of two (n-type and p-type) JFETs. Q1: 2N5457, Q2: 2N5460. (Right) Amplifier circuitry to scale the output voltage back into the range of the input voltage. R1: 535 Ω, U1: AD712 op-amp, R2: 100 kΩ and R3: 450 kΩ. Here Vin = xn and Vo = xn+1. to set the iteration speed. To implement the control for nonlinear dynamical computing, the output from the 2nd sample and hold circuit is input to the threshold controller, as that described by Equation 4–3. The output from this threshold controller then becomes

the input to the 1st sample-and-hold circuit. In Figure 4-6, the sample and hold circuits are realized with National Semiconductors sample and hold IC LF398, triggered by delayed timing clock pulses CK1 and CK2 [70]. Here a clock rate of either 10 kHz or 20 kHz may be used. The threshold controller circuit is shown in Figure 4-7 is realized with an AD712 operational amplifier, a 1N4148 diode, a

1 kΩ series resistor and the threshold control voltage, x∗(= Vcon). The Figure 4-8(a) displays the uncontrolled chaotic waveform and the Figures 4-8(b-d) show representative results of the chaotic system under different threshold values

x∗(= Vcon). It is clear that adjusting the threshold yields cycles of varying periodicities. Also, note that simply setting the threshold beyond the bounds of the attractor (5 V)

94 Figure 4-5. Voltage response characteristics of the nonlinear device, based on Equation 4–2 and circuit of Figure 4-4.

gives back the original dynamics, and so the controller is easily switched on and off. A detailed comparison shows complete agreement between experimental observations and analytical results. For instance, the threshold that needs to be set in order to obtain a certain periodicity and the trajectory of the controlled orbit can be worked out exactly through symbolic dynamics techniques. Further, the control transience is very short here

3 (typically of the order of 10− times the controlled cycle length) and the perturbation involved in threshold control is usually small. This method is then especially useful in the situation where one wishes to design controllable components that can switch flexibly between different behaviours. Calibrating the systems characteristics at the outset with

95 Figure 4-6. Schematic diagram for implementing the threshold controlled nonlinear map. CK1 and CK2 are clock timing signals, while the modules designated S/H are sample-and-hold circuits.

Figure 4-7. Circuit diagram of the threshold controller. Vin and Vo are the input and output, D is a 1N4148 diode, R = 1 kΩ, and U2 is an AD712 op-amp. Vcon = x∗ (controller input voltage).

96 respect to threshold gives one a look-up table directly and simply to extract widely varying temporal patterns.

Figure 4-8. PSPICE simulation results of the experimental circuit. The ordinate is xn and the abscissa is the discrete time n measured in ms. (a) Uncontrolled chaos: x∗ = 6 V, (b) period 5 cycle: x∗ = 4 V, (c) period 2 cycle: x∗ = 3.7 V and (d) period 1 cycle: x∗ = 3.5 V.

4.4 Fundamental Logic Gates with a Chaotic Circuit

Here we explicitly show how, by using the threshold controlled map of Equation 4–3, we obtains the clearly defined logic gate operations NOR, NAND, AND, OR,and XOR. The state of the system is represented by the state value of x. The initial

state of the system is represented as x0. In our method all five basic gate operations involve the following steps: specification of x0 based on the operation and the inputs through threshold control, nonlinear update (evolution of the circuit dynamics), and output interpretation through threshold “monitoring” in the spirit of Excess Overflow Propagation, from Section 1.2.3. Specifically:

prog I1 I2 prog 1. Inputs and Programming; x0 = x + x + x . Here x is a programming shift that fixes the initial state x0 of the system, based on the gate to be operated. Letting a finite voltage δ denote a logical 1, we set xIi = δ for an input of logical 1 and xIi = 0 for an input of logical 0.

97 2. Nonlinear update; i.e., x0 f(x0), where f(x) is the nonlinear function, given by Equation 4–2. →

3. Thresholding to obtain the output Z defined as:

0, for f(x) x∗, Z = ≤ ( f(x) x∗, for x∗ < f(x), − where x∗ = 1 is the threshold reference signal; which is set the same for all gates unlike previously in the case of the Logistic map in Section 2.3.1. The output is interpreted as logic output 0 if Z = 0 and logic output 1 if 1 δ < Z. Since the system is nonlinear (and 2 ∙ may be chaotic), in order to specify the initial x0 accurately in hardware experiments, one needs a controlling mechanism. Here we will employ a threshold controller to set the initial x0. So in this example we will use the clipping action of the threshold controller to achieve the initialization. A comparator is used to recover the output from the state of Z. The logical operations are defined by the input to output mappings depicted in thetruth table of Table 4-1.

Table 4-1. Truth-table for the five fundamental logic gates NOR, NAND, AND, ORand XOR.

I1 I2 NOR NAND AND OR XOR 0 0 1 1 0 0 0 0 1 0 1 0 1 1 1 0 0 1 0 1 1 1 1 0 0 1 1 0

From the definition of f(x), x∗ and the above truth table, we obtain a set of inequality conditions that need to be satisfied simultaneously, shown in Table 4-2. Note that the symmetry with respect to the inputs reduces the four conditions in the truth table of Table 4-1 to three distinct conditions, with rows 2 and 3 of Table 4-1 leading to the single condition of row 2 in Table 4-2. The above inequalities have many possible solutions depending on the size of δ. By setting δ = 0.3 we can easily solve the equations for the different programming shifts that

each gate requires. The specific xprog values for the five different logical operations are listed in Table 4-3.

98 Table 4-2. Necessary and sufficient conditions to be satisfied by a chaotic element inorder to implement the logical operations NOR, NAND, AND, OR and XOR. Where f(x) is given by Equation 4–2 and 1 is the monitoring threshold (x∗) of interpretation of the logic output.

I1 I2 NOR NAND AND 0 0 1 < f(xprog) 1 < f(xprog) f(xprog) 1 0/1 1/0 f(xprog + δ) 1 1 < f(xprog + δ) f(xprog + δ≤) 1 1 1 f(xprog + 2 δ)≤ 1 f(xprog + 2 δ) 1 1 < f(xprog +≤ 2 δ) ∙ ≤ ∙ ≤ ∙ I1 I2 OR XOR 0 0 f(xprog) 1 f(xprog) 1 0/1 1/0 1 < f(xprog≤+ δ) 1 < f(xprog≤+ δ) 1 1 1 < f(xprog + 2 δ) f(xprog + 2 δ) 1 ∙ ∙ ≤ Table 4-3. Numerical values of xprog for implementing logical operations NOR, NAND, AND, OR and XOR, based on Equation 4–2. Operation NOR NAND AND OR XOR xprog 0.9138 0.6602 0.0602 0.3602 0.45

By setting up a map the with initial condition x0, as defined above, we allow the map to update to a new value x1 = f(x0) and compare this value to the monitoring threshold x∗ = 1, if the new state of the map is greater than the threshold a logical 1 is the output, if the new state is less than the threshold a logical 0 is the output. The updated states of a chaotic element, following Equation 4–2, satisfying the conditions of Table 4-2, with xprog values given in Table 4-3 are shown in Table 4-4. The circuitry described in the section above has been tested to do the logic operations described here and shows complete agreement with the simulation results.

Table 4-4. Updated state values, x1 = f(x0), of a chaotic element satisfying the conditions in Table 4-2 in order to implement the logical operations NOR, NAND, AND, OR and XOR. Operation NOR NAND AND OR XOR f(xprog) 1.3 1.3 0.1204 0.7204 0.8997 f(xprog + δ) 0.3057 1.1525 0.7204 1.3 1.42 f(xprog + 2 δ) 0.0472 0.2270 1.3 1.1525 0.7988 ∙

We have presented a proof of principle device that demonstrates the capability of this nonlinear map to implement the five fundamental logic gates; it does this by exploiting the nonlinear response of the system. The main benefit is its ability to exploit a single chaotic

99 element to reconfigure into different logic gates through a threshold based morphing mechanism. Contrast this to a conventional field programmable gate array element, where reconfiguration is achieved through switching between multiple single purpose gates. 4.5 Encoding and Searching a Database Using Chaotic Elements

In the spirit of Chapter 3 we apply that method to the map, and circuit, given by Equation 4–3. Specifically we show how this map can be utilized to stably encode and store various items of information (such as patterns and strings) to create a “database”. Further we demonstrate how this storage method allows to efficiently determine the number of matches (if any) to some specified item [28]. Consider an array of elements each of which evolves according to Equation 4–3. The nonlinear dynamics of the array elements will be utilized for flexible capacity storage, as well as for pre-processing data for exact (and inexact) pattern matching tasks.

Encoding information. We consider a “database” of length N and each member of the database is encoded in an element obeying Equation 4–3, we index these elements with j = 1, 2, 3, ...N , so the state of the whole array, at a time n, can be represented { } j as Xn. At the same time the database is made up of items from an “alphabet” of total number of unique items M, indexed with m = 1, 2, 3, ...M . We correlate each item m { } with a threshold, x∗, for Equation 4–3, such that the element is confined on a fixed point of period 1, we define T j[m] as the threshold for the jth element encoding the mth item. For this map, thresholds ranging from 0 to 1 yield fixed points, as depicted in Figure 4–3.

j j j Namely Xn = T [m], for all time n, when the threshold is chosen as 0 < T [m] < 1. This can be obtained exactly from the fact that x < f(x), x (0, 1), implying that the ∀ ∈ subsequent iteration of a state at T j[m] will always exceed T j[m] and thus be reset to T j[m].

100 1 In our encoding, the thresholds are chosen from the interval (0, 2 ), namely a sub-set of the fixed-point window (0, 1)1 . Without loss of generality, consider each item to be represented by an integer z from the range [1,M]. Defining a resolution r between each threshold as: 1 1 r = , (4–4) 2 ∙ M gives a lookup map from the encoded integer to the threshold, relating the integers z in

j 1 the set [1,M] to thresholds T [m] in the range [r, 2 ], by:

T j[m] = z r. (4–5) ∙

Therefore we obtain a direct correspondence between a set of integers ranging from 1 to M, where each integer represents an item, and a set of M threshold values. So we can store N database elements by setting appropriate thresholds (via Equation 4–5) on N dynamical elements. Clearly from Equation 4–5, if the threshold setting has better resolution (smaller r), then a larger range of values can be encoded. Note however that precision is not a restrictive issue here, as different data representations can always be chosen in order to suit a given precision of the threshold mechanism.

Processing Information. Once we have a given database stored by setting appropriate thresholds on N dynamical elements, we can query for the existence of a specific item in the database in one global operational step. This is achieved by globally shifting the state variable of all elements of the database up by an amount that represents the item being searched for.

1 Actually we can use as much as the interval (0, 0.8027), since 0.8027 is the pre-image of the maximum.

101 Noting that the maximal state variable value for this system is 1.4449, one raises the

j j state Xn of each element j to Xn + Q[k], where Q[k] is a search key given by:

Q[k] = 0.8027 T [k], (4–6) −

where k is the index of the integer(item) being queried for, k z. 0.8027 is the unique ∈ value of this system that evolves to the maximal value 1.4449 on an iteration of the system. So the value of the search key is simply 0.8027 (the pre-image of the maximal state variable value) minus the threshold value corresponding to the item being searched for, given k z we have T [k] = k r. ∈ ∙ The addition of the “search key”, Q[k] shifts the interval that the database elements

1 1 can span, from [r, 2 ] to [r + Q[k], 2 + Q[k]]. Since Q[k] + T [k] adds up to 0.8027, it is guaranteed that only the element(s) matching the item being queried for will have its(their) state shifted to 0.8027, which is the only state which after the subsequent iteration will maximize to the value of 1.44492 . So the total search process is rendered simple as the state with the matching pattern is selected out and mapped to the maximal value, allowing easy detection. Further, by relaxing the detection level by a prescribed “tolerance”, we can check for the existence within our database of numbers or patterns that are “close to”3

Representative Example. Consider the case where our data is English language text, encoded as described above on a letter by letter basis by an array of maps, following Equation 4–5. In this case the distinct items are the letters of the English alphabet and we have M = 26. We obtain r = 1 0.0192 from Equation 4–4 and the appropriate 52 ≈ threshold level for each item is obtained via Equation 4–5. More concretely, consider as

2 Note all other states (both higher and lower than 0.8027) get mapped to values lower than 1.4449. 3 Where “close to” is defined by the designer of the database.

102 our database the phrase “the quick brown fox”; each letter in this phrase is an element of the database and can be encoded using the appropriate threshold, as in Figure 4-9(a).

Figure 4-9. Searching for “b”. (a) Threshold levels encoding the phrase “the quick brown fox”, bars marked as ; (b) the search key value for the letter “b” is added to all elements, bars marked as ; (c) the elements update to the next time step, bars marked as . For clarity we mark black the elements that reached the detection level.  Now we query the database regarding the existence of specific items. Figure 4-9 presents the example of querying for the letter “b”. To do so the search key value

2 corresponding to letter “b”( 52 ) is added globally to the states of all elements, Figure 4-9(b). Then through their natural evolution, upon the next time step, the state(s) of the element(s) containing the letter “b” is(are) maximized, Figure 4-9(c). In Figure 4-10 we perform an analogous query for the letter “o”, which happens to be present twice in our database to show that multiple occurrences of the same item can be detected. Finally in Figure 4-11 we query for an item that is not part of our given database, the letter “d”. As expected Figure 4-9(c) shows that none of the elements are maximized. By lowering the

103 Figure 4-10. Searching for “o”. (a) Threshold levels encoding the phrase “the quick brown fox”, bars marked as ; (b) the search key value for the letter “o” is added to all elements, bars marked as ; (c) the elements update to the next time step, bars marked as . For clarity we mark black any elements that reached the detection level.  detection level to the value 1.4449 f(0.8027 r) = 1.4411, just “one step” down from the − − maximal, we detect whether items “adjacent” to the desired one are present. Specifically we detect that the letters “c” and “e” are contained in our database. This demonstrates that inexact matches can also be found, just as easily. 4.6 Conclusion

In summary, we introduced a simple map having rich nonlinear dynamics, and a simple electronic circuit realization. Then we demonstrated the direct and flexible implementation of the five basic logic gates using this simple nonlinear map (circuit). Further, we showed how the dynamics of this map can be utilized to provide an efficient database search method.We have experimentally implemented the electronic circuit analog

104 Figure 4-11. Searching for “d”. (a) Threshold levels encoding the phrase “the quick brown fox”, bars marked as ; (b) the search key value for the letter “d” is added to all elements, bars marked as ; (c) the elements update to the next time step, bars marked as . It is clear that no elements reach the detection level at 1.4449; (d) By lowering the detection level we can detect whether items “adjacent” to “d” are present (“c” and “e”). of this nonlinear map and have demonstrated the efficacy of the threshold controller in yielding different controlled responses from this map circuit.

105 CHAPTER 5 LOGIC OPERATIONS FROM EVOLUTION OF DYNAMICAL SYSTEMS In this chapter we propose the direct and flexible implementation of logic operations using the dynamical evolution of a nonlinear system [33]. The concept involves the observation of the state of the system at different time instances to obtain different logic outputs. We explicitly implement the basic NAND, AND, NOR, OR and XOR logic gates, as well as multiple-input XOR and XNOR logic gates. Further we demonstrate how the single dynamical system can do more complex operations such as bit by bit addition in just two iterations. The concept uses the nonlinear characteristics of the time dependence of the state of the dynamical system to extract different responses from the system. The highlight of this method is that a single nonlinear system is capable of yielding a time sequence of different logic operations. Further we explicitly demonstrate, through the three examples, how results from this method can be obtained by varying any of the “defining

init variables” (x0, x , δ, n). 5.1 Generation of a Sequence of (2-input) Logic Gate Operations

We outline a method for obtaining the five basic logic gates using different dynamical iterates of a single nonlinear system. In particular consider a chaotic system whose state is represented by a value x. The state of the system evolves according to some dynamical rule. For instance, the updates of the state of the element from time n to n + 1 may be well described by a map, i.e., xn+1 = f(xn), where f(x) is a nonlinear function. Now this element receives inputs before the first iteration (i.e., at n = 0) and outputs a “signal” after evolving for a (short) specified time or number of iterations. The method canbe applied for any sequence of the gates, for illustrative purposes we chose the sequence NAND, AND, NOR, XOR and OR (see Table 5-1 for the truth table). In general the method involves the following steps:

init I1 I2 init 1. Input definition (for a 2 input operation): x0 = x + x + x , where x is the initial state of the system. (Comparable to xprog in previous chapters, but now it is not defining a single gate but a sequence of gates, or more general, operations).

106 Table 5-1. The truth table of the five basic logic operations NAND, AND, NOR, XOR, OR.

I1 I2 NAND AND NOR XOR OR 0 0 1 0 1 0 0 0 1 1 0 0 1 1 1 0 1 0 0 1 1 1 1 0 1 0 0 1

before data inputs are introduced, x0 is the actual initial state of the system that includes the data to be operated on. As previously we set xIi = δ for the logical Ii input Ii = 1, and x = 0 for the logical input Ii = 0. So we need to consider the following three cases:

(a) Both I1, and I2 are 0 (row 1 in Table I), i.e. the initial state of the system is: init init x0 = x + 0 + 0 = x .

(b) Either I1 = 0 and I2 = 1, or I1 = 1 and I2 = 0 (row 2 or 3 in Table I), i.e. the init initial state is: x0 = x + 0 + δ.

(c) Both I1 and I2 are 1 (row 4 in Table I), i.e. the initial state is: init x0 = x + δ + δ = xinit + 2 δ. ∙

2. Chaotic evolution over some prescribed number of steps, i.e. fn(x0) xn, for 1 < n. →

3. The evolved states fn(x0) yield the logic output, at each n, as follows: Logic Output = 0, if fn(x0) xn∗ , Logic Output = 1, if fn(x0) > xn∗ , where xn∗ is a reference monitoring value, at≤ time instance n.

Since the system is chaotic, in order to specify the initial x0 accurately we employ the threshold control mechanism, see Section 1.2.3, we note that this mechanism can be invoked at any subsequent iteration as well. For logic recovery, the updated or evolved value of f(x) is compared with xn∗ value using a comparator action, in the spirit of Excess Overflow Propagation, again as in Section 1.2.3. In order to obtain all the desired input-output responses of the different gates, as displayed in Table 5-1, we need to satisfy the conditions enumerated in Table 5-2. Note that the symmetry of inputs to outputs reduces the four conditions in the truth Table 5-1 to three distinct conditions, with 2textnd and 3rd row of Table 5-1 leading to the 2nd

condition of Table 5-2.

107 Table 5-2. Necessary and sufficient conditions to be satisfied by a chaotic element inorder to implement the logic operations NAND, AND, NOR, XOR and OR on subsequent iterations. Here xinit = 0.325 and δ = 0.25, for all n. While x∗ = 0.75, for n = 1, 2, 3, 4 , that is for NAND, AND, NOR, XOR logic n { } operations, and x5∗ = 0.4 for OR logic operation. Logic gate NAND AND NOR XOR OR Iteration (n) 1 2 3 4 5 Condition 1: x1=f(x0) > x∗ f(x1) x∗ f(x2) > x∗ f(x3) x∗ f(x4) x∗ Logic input 1 ≤ 2 3 ≤ 4 ≤ 5 (0,0) x1 = 0.88 x2 = 0.43 x3 = 0.98 x4 = 0.08 x5 = 0.28 x0 = 0.325 Condition 2: x1=f(x0) > x∗ f(x1) x∗ f(x2) x∗ f(x3) > x∗ f(x4) > x∗ Logic input 1 ≤ 2 ≤ 3 4 5 (0,1) or (1,0) x1 = 0.9775 x2 = 0.088 x3 = 0.33 x4 = 0.872 x5 = 0.45 x0 = 0.325 + δ x0 = 0.575 Condition 3: text x1=f(x0) x∗ f(x1) > x∗ f(x2) x∗ f(x3) x∗ f(x4) > x∗ Logic input ≤ 1 2 ≤ 3 ≤ 4 5 (1,1) x1 = 0.58 x2 = 0.98 x3 = 0.1 x4 = 0.34 x5 = 0.9 x0 = 0.325 + 2 δ ∙ x0 = 0.825

init So given dynamics f(x), we must find values of threshold(s) xn∗ , initial state(s) x and δ satisfying the conditions derived from the specific truth table to be implemented. Using, as usual, the Logistic map, Equation 1–1, we incorporated in Table 5-2 actual

init values of x and xn∗ for n = 1, 2, 3, 4, 5, which satisfy the conditions imposed by the truth table considered. For illustrative purposes, the graphical representation of five iterations of the Logistic map is shown in Figure 5-1, displaying the results in Table 5-2.

In summary, the inputs setup the initial state xinit +xI1 +xI2 . Then the system evolves over n iterative time steps to each updated state xn. The evolved state is compared to a monitoring threshold xn∗ , at every n. If the state at iteration n, is greater than the threshold a logical 1 is the output and if the state is less than the threshold a logical 0 is the output. This process is repeated for each subsequent iteration.

Note that in the above example we present results for which xinit is not varied and xn∗ is varied; the emphasis is on the specific behaviour that represents a computational

108 Figure 5-1. Graphical representation of five iterations of the Logistic map. Three different initial conditions are considered, each representing one of the three cases of two logic inputs, (0,0) by N, (0,1)/(1,0) by N, (1,1) by M. At each iteration comparison with a monitoring threshold is performed, 0.75 for n = 1, 2, 3, 4 , 0.4 for n = 5. The results from two gates are also shown, the circles{ mark } n = 1 (the NAND gate) compare with 0.75 and the colouring corresponds to the appropriate case of data; the squares mark n = 5 (the OR gate) compare with 0.4, and again the colouring is appropriate to the data.

109 task and not the “actual” values as such. The case of δ is slightly more complicated since it actually represents a constant “entity”, nevertheless varying its “actual” value is also possible, but more care is needed, hence in this example we simply set δ = 0.25. Therefore in a more general context, we are relating inputs and required outputs with specific behavioural patterns, at the same time we can say the reverse, we findthe behavioural pattern that can represent a needed operation. Hence the “actual” state values are not of great importance, and we can generate “templates” like Figure 5-2.

To generate this specific template, we set xn∗ = 0.65 and varied x0, in contrast to the example above; once again δ was kept constant at 0.25. Basically the behaviour of each combination of variables is then interpreted as one of the eight possible symmetric binary operations. It is clear from this figure that we are not confined to n < 5, so the “length” of the sequence of operations can be extended, and also the actual order of the sequence of operations can be changed. Theoretically all operations and all sequences of operations exist, under some combination of “actual” values of variables. We should note that, as is clear from Figure 5-2, the range of each operation is decreasing in size with increasing iterations, and since the dynamics are chaotic at some point we will lose definition, as we mentioned above though the threshold control mechanism can be invoked to “re-initialize” the system. 5.2 The Full Adder and 3-Input XOR and NXOR

This section is a direct extension of the previous section, hence it is comprised of simply two demonstrations. We extend the above method for sequential logic operations in two ways, first to more than just two data inputs, and second to more than justlogic gate operations. Specifically we show the implementation of the binary full adder andthe implementations of 3-input XOR and NXOR gates. For these implementations we employ the usual three steps, but modified for 3 inputs as follows:

110 Figure 5-2. Patterns of binary two input symmetric operations. We use the Logistic map init with δ = 0.25, x∗ = 0.65, n and vary x . n ∀

init I1 I2 I3 1. Input definition (for a 3 input operation): x0 = x + x + x + x , as usual with the addition of one “extra” input. In the context of the full adder, I1 corresponds to the input binary number A, I2 corresponds to the input binary number B, and I3 corresponds to the carry input Cin (the carry from the “previous” positional digit addition), as in Table 5-3. So we need to consider the following four cases:

(a) If all inputs are 0 (1st row in Table 5-3), i.e. the initial state of the system is: init x0 = x + 0 + 0 + 0 = x0. (b) If any one of the input equals 1 (2nd, 3rd and 5th row in Table 5-3), i.e. the init init init init initial state is: x0 = x +0+0+δ = x +0+δ +0 = x +δ +0+0 = x +δ. (c) If any two inputs equal to 1 (4th, 6th and 7th row in Table 5-3), i.e. the initial init init init init state is: x0 = x + 0 + δ + δ = x + δ + 0 + δ = x + δ + δ + 0 = x + 2 δ. ∙

111 (d) If all inputs equal to 1 (8th row in Table 5-3), i.e. the initial state is: init init x0 = x + δ + δ + δ = x + 3 δ. ∙ 2. Chaotic evolution (for two time steps), of the initial state given from the data f2(x0) x2. We confine the time steps to only two to make clear the example ofthe full adder.→ In general of course, further time steps can be also considered providing results for other 3 input operations, as in Figure 5-2.

3. The evolved state fn(x0) yields the logic output as follows: Logic Output = 0, if fn(x0) x∗ , ≤ n Logic Output = 1, if fn(x0) > xn∗ , where xn∗ is a monitoring threshold, with n = 1, 2 in the full adder example.

Table 5-3. The truth table of full adder, and necessary conditions to be satisfied. Using the Logistic map, state values x1 (iteration n = 1) and x2 (iteration n = 2) are init used to obtain Cout and S respectively. Here x1∗ = 0.8, x2∗ = 0.4, x = 0.0 and δ = 0.23. Input bit of Input bit of Input bit of Output Condition for Output Number (A) Number (B) Carry (Cin) Cout SCout (x1) S (x2) 0 0 0 0 0 f(x0) x∗ f(x1) x∗ ≤ 1 ≤ 2 0 0 1 0 1 f(x0 +δ) x∗ f(x1) > x∗ ≤ 1 2 0 1 0 0 1 f(x0 +δ) x∗ f(x1) > x∗ ≤ 1 2 0 1 1 1 0 f(x0 + 2 δ) > x∗ f(x1) x∗ ∙ 1 ≤ 2 1 0 0 0 1 f(x0 +δ) x∗ f(x1) > x∗ ≤ 1 2 1 0 1 1 0 f(x0 + 2 δ) > x∗ f(x1) x∗ ∙ 1 ≤ 2 1 1 0 1 0 f(x0 + 2 δ) > x∗ f(x1) x∗ ∙ 1 ≤ 2 1 1 1 1 1 f(x0 + 3 δ) > x∗ f(x1) > x∗ ∙ 1 2

The Full Adder. We now demonstrate how one can obtain the ubiquitous bit by bit arithmetic addition, involving three logic inputs and two outputs, in consecutive iterations, with a single one-dimensional chaotic element. In general the simple 1-bit binary arithmetic addition requires a full adder logic which adds three individual bits together (two bits being the digit inputs and the third bit assumed to be carry from the addition of the next least-significant bit addition operation, known as“Cin”). A typical full-adder requires two half-adder circuits and an extra XOR gate. In total, the implementation of a full-adder requires five different gates (three XOR gates andtwo AND gates). However in the present direct implementation by utilizing the dynamical evolution of a single map, the Logistic map, we need only two iterations to implement a

112 full adder. The truth table and necessary conditions to be satisfied for the parallelized

full adder operation are given in Table 5-3. The Carry bit output Cout and the Sum bit output S are respectively recovered from the first and second iterations of a Logistic map,

init init with δ = 0.23, x = 0.0 x1∗ = 0.8 and x2∗ = 0.4. Note in this example we have kept x

constant and varied the xn∗ . So basically each system takes the three inputs, the value of the two bits to be added and the carry from the previous bit addition, and produces the carry for the next addition on the very first update. This new carry can of course be immediately supplied to thenext system ready to perform the addition of the next bit, while the sum (S) is “calculated” on the second update.

3-Input XOR and NXOR. Three or more input logic gates are advantageous because they require less complexity in actual experimental circuit realization than that of coupling conventional 2-input logic gates [91, 92]. Specifically the truth table for 3-input XOR and XNOR logic gate operations, and the necessary conditions to be satisfied by a map to perform these operations, are shown in Table 5-4. In this representative case, we consider a different scenario, for both operations we use the2nd iteration of the map, so both operations can not be accomplished by the same system. We set the threshold

init (x2∗) at the fixed value of 0.5 and so we accomplish the XOR gate with x = 0.0 and the NXOR gate with xinit = 0.25, most importanly though we have an exact fixed valued δ 0.25. ≈ 5.3 Conclusion

Our previous results have shown that a single nonlinear dynamical system, with proper tuning of parameters and control inputs, can flexibly become any logic gate. We did confine ourselves though, to gates of only two inputs and to patterns createdbya single iteration of a system. The main result of this chapter is that the various temporal patterns embedded in the dynamical evolution of nonlinear systems are capable of performing sequences of

113 Table 5-4. The truth table of the 3-input XOR and NXOR logic operations, necessary and sufficient conditions to be satisfied by the map. State value x2 (iteration n = 2) is used for logic operation recovery. Here x∗ = 0.5 and δ 0.25. init ≈ init I1 I2 I3 XOR(I1,I2,I3) XNOR(I1,I2,I3) XOR (x = 0.0) NXOR (x = 0.25) 0 0 0 0 1 x2 x∗ x2 x∗ δ ≤ − ≈ 0 0 1 1 0 x2 x∗ δ x2 x∗ − ≈ ≤ 0 1 0 1 0 x2 x∗ δ x2 x∗ − ≈ ≤ 1 0 0 1 0 x2 x∗ δ x2 x∗ − ≈ ≤ 0 1 1 0 1 x2 x∗ x2 x∗ δ ≤ − ≈ 1 0 1 0 1 x2 x∗ x2 x∗ δ ≤ − ≈ 1 1 0 0 1 x2 x∗ x2 x∗ δ ≤ − ≈ 1 1 1 1 0 x2 x∗ δ x2 x∗ − ≈ ≤ logic operations in time (or iterates), and minimize the control that is needed, we only invoke control mechanism on initialization, from there on we just monitor the state. The implementation of a sequence of logic functions in time, as described above, is another mechanism through which computer architectures based upon the chaos computing approach can be optimized for better performance. Even further in this chapter we introduced mechanisms for multi-input logic, and showed how in the context of chaotic computation behaviours are more important than exact and specific values of variables, providing even more flexibility in a physical realization. With these fundamental ingredients in hand it becomes clear that exploiting not just the pattern formation of nonlinear dynamical systems, but the formation of sequences of such patterns, produced naturally by such systems, may prove a key ingredient towards making nonlinear dynamical computational architectures a real alternative to conventional static logic computer architectures.

114 CHAPTER 6 MANIPULATING TIME FOR COMPUTATION We have shown in Chapter 5 how time can be involved in the computational process, specifically we have shown how specific “time instances” can be used for programming a system to perform a specific computational operation. In the present chapter, in some sense we extend the utility of time; we show how the dimension of time can be used for a complete definition of a computational operation, that is we use time instances not only for programming a system, but also for introducing to the system the “data” to be manipulated. To achieve this we predefine the behaviour of the state of the system within a “clock cycle” and work with the fact that this behaviour is not constant within the given cycle and so at different time instances different results can be obtained. In a simplified mathematical context we are just interchanging x with t in the already presented algorithms. 6.1 Introduction

We first present, in a general context, the time based algorithm for performing the five fundamental 2-input logic gates and then the algorithm for performing the search of an unsorted database. Further we support the method with specific illustrative examples using neural systems [93, 94], and electronic circuits. We should note that in both examples the general method is adapted to best suit the context of each the actual implementation. 6.1.1 Flexible Logic Gates

For any computational system to be able to perform flexible (2-input) logic three inputs need to be given to the system. The first input is the “Programming Instruction” that defines which logic operation will be performed. The other two inputs arethetwo logic variables, “Input1” (I1) and “Input2” (I2), see Figure 6-1. In the present method we utilize a time instance, ttotal, as a single adjustable variable that can be used for the representation of all three inputs. We define one time instance to represent the

115 Figure 6-1. Schematic representation of a flexible 2-input logic gate. For an Output three parameters need to be defined, the two data streams, Input1 and Input2, and one Programming Instruction, which is to manipulate the data.

programming, tprog, and two more time instances to represent the two inputs, tI1 , tI2

respectively, with the relationship for a total time defined as: ttotal = tprog + tI1 + tI2 . We represent a logical 1 by a finite time length, tin (relate to δ in the case of algorithms based on state manipulation), and a logical 0 by a time length of zero length (tin = 0). It is important to note that:

Since a logical 0 at the inputs is set to be of zero time length, the set of inputs (0,0) • is the same time interval as programming the gate (tprog).

The time interval for representing an input of logical 1 must be the same irrespective • of whether the logical 1 is at Input1 or at Input2, as the two are indistinguishable by a logic gate; the sets (1,0) and (0,1) are thus represented by the same time length;

The total time interval for the set of inputs (1,1) is twice that of the sets (1,0) and • (0,1).

116 The way we use the total time, ttotal, is to define a time instance within the clock cycle of a “generic signal” constructed in such a way so as to encompass the results of different logic operations at different time instances. Of course because of the “nature” of time this time instance needs to be in reference to a fixed “zeroth” time instance, which can be defined by the initialization of the clock cycle. In fact in the general context demonstration we consider a single cycle initiated at the zeroth instance and use ttotal to define a “moment of observation”, at which the result is obtained. There is thoughthe converse scenario as well, which we show in a following illustrative example in Section 6.2, we can consider the “final” time instance as fixed and use ttotal as a “negative” shift in instances from this final instance, basically changing the “moment of generation” ofthe generic signal. To construct the “generic signal” we consider the behaviour of the five fundamental logic gates (AND, NAND, OR, NOR, XOR). From the truth table for these logic gates (Table 6-1) we can create for each gate a graph of the input-output relationships for the three pair sets of logical inputs, (0, 0) , (1, 0)/(0, 1) , (1, 1) , Figure 6-2(a)-(e). Since { } { } { } we are to consider time as the input quantity, we can encapsulate all five graphs into a single graph where the input dimension is identified with time instances, as explained in Figure 6-2(f). The combination of the truth tables into this single non-linear function results in the form of : f(u, t) > 0, for t Δt > t > t + Δ t, else f(u, t) = 0; which provides the required − behaviour for a system to be utilized as a flexible logic gate. This function is very similar to an action potential generated by a neuron; resting at a low voltage (f(u, t) = 0), and for a brief length of time rising very rapidly to a higher voltage (f(u, t) > 0), and then dropping very rapidly back to its original level. We are considering this function in neural context, not only because of the form of the function, but also because it is clear that in some way neurons are capable of logic operations [95]. Note though, that we do not

117 actually propose that the methods we present here are the method neurons in fact use; we use neurons primarily for demonstration purposes. If we consider the inputs (and programming) to be represented as a time instance we can see from Figure 6-2(f) that at different moments in time this signal is “high” or “low”, we can interpret these two states as a logical 1 or 0 at the output. Therefore it is only a matter of “when” we sample such a signal to obtain a different response, i.e. depending on which time instance we chose to sample the signal we obtain a 1 or 0. Therefore the method utilizes this behaviour to provide a flexible logic gate system by varying the exact “moment of observation” (ttotal) of the signal depending on which gate will be performed on what data. The “flexible” part should be evident, as wecan easily change the tprog from one cycle to the next, enabling the system to perform any two different computational operations. Table 6-1. The truth table of each of the five fundamental logic gates, AND, NAND, OR, NOR, XOR.

I1 I2 AND NAND OR NOR XOR 0 0 0 1 0 1 0 1/0 0/1 0 1 1 0 1 1 1 1 0 1 0 0

Purely as a reference example, based on the signal of Figure 6-2(f) we give in Table 6-2 values for the different times to perform each of the five logic gates, on each ofthe three pair sets of inputs. In each of the three cases of inputs, the time length representing a logical 1 is two time “units” long (tin = 2). We utilize a signal as given in Figure 6-2(f), with clock cycle defined to be 9 time units long. Depending on which logic operation we wish to perform and also on what the inputs are, a sampling instance within the clock cycle is defined, as explained above and given by ttotal. This sampling instance is in time units with reference to the initialization from the clock cycle. We take the NOR gate as an illustrative example. For the NOR gate we have tprog = 5 time units. For the set of inputs (0,0) we sample the signal at the 5th (tprog + 0 + 0) time unit obtaining a 1 at the output, see Figure 6-2(f). In the case where our inputs are (1,0) or (0,1) we have a sampling

118 Figure 6-2. (a) Input-Output relationship of AND gate; (b) Input-Output relationship of NAND gate; (c) Input-Output relationship of OR gate; (d) Input-Output relationship of NOR gate; (d) Input-Output relationship of XOR gate; (f) Combined figure of the Input-Output relationships of the five fundamental gates. Different sections of this graph represent different instances of Input-Output operations. The 15 different cases, of the gates and inputs considered, are marked; with marking an Output of 1 and marking an Output 0. #

Table 6-2. Time values, in arbitrary time “units” that produce each of the five gates considered, based on the signal of Figure 6-2(f).

I1 I2 AND NAND OR NOR XOR 0 0 0 4 2 5 3 1/0 0/1 2 6 4 7 5 1 1 4 8 6 9 7 instance of 7 time units, given by (tprog + tin + 0), obtaining a logical 0 at the output. Similarly for the case NOR(1,1) we have (tprog + tin + tin) giving a sampling time at the 9th time unit again obtaining a logical 0 at the output. Thus we have a correct representation of the NOR gate. Using a similar scheme, and by simply changing the programming time

length, tprog we can switch to performing a different gate.

119 6.1.2 Search Algorithm

Promoting this idea further, one can now consider each moment in time as an opportunity to represent any specific computational instruction or data. In order to demonstrate more practical applications of using time for computational commands (data and programming), we will demonstrate a variation on our algorithm for performing a search on a given unsorted database [28]. The construction of a search engine requires the following steps:

1. A representation of data in the database;

2. The matching of data with searched for items;

3. Localization of the matched data. Any given database is constructed out of M unique items, we already referred to this set of items as the “alphabet”, and further each item can be represented by a unique natural number, see Section 3.1. We have shown above that a finite time interval can represent a logical 1, we can extend this idea to time lengths of multiple sizes, each one representing a different number. Thus we utilize different time lengths to represent each one of the unique items that constitute a given alphabet. These time lengths can be marked by a delta function (a spike in neural context) occurring at a specific moment in time with respect to the initialization of a clock cycle; the interval between initialization and the delta function is the time length for a specific item. So given any database we can represent it by a series of delta function generators, Rx, all receiving initialization from the clock cycle, and each providing a spike at the specific delayed moment in time providing a time interval which represents the data item it stores, see Figure 6-3. As shown in Figure

6-3 we can connect these function generators, Rx, each independently to a Coincidence Detector Unit (CDU). The function of the CDUs is to produce a “confirmation”, a spike, at the moment when it receives two simultaneous spikes. The “search element” is another delta function generator, Rsearch,k set to produce a spike at the moment that represents the data item we wish to search for. As a result the coincidence detector(s) produce a

120 spike only if the searched for item is within the given database since only then will a CDU receive two simultaneous spikes (from a database element and the searched for item). Note that the terminology we use is such as to relate this abstract presentation of the method directly to the neural implementation to follow, specifically R will be a neural synapse strength coefficient, while the TDU and CDU will be specific neural circuits. For the final step of localization of positive results of a search we connect allthe coincidence detectors to an accumulator, Integrating Unit (IU), each weighted by a different weight following G¨odelnumbering [31], i.e. the first CDU weighted by20, the second by 21, the third by 22 and so on, providing us a way to uniquely identify the positions of all matched items, by measuring the “height” of the weighted sum at each moment in time. In addition we can perform multiple searches within the same clock cycle; i.e. we can connect to the CDUs multiple search elements, each encoding a different item and search for all the items, technically, at the “same time”. Performing multiple searches does not create a problem at the accumulator since the confirmation spikes from the CDUs will arrive at different times, for different items, and thus spikes of the same itemwillbe accumulated separately from spikes of a different item. Given that our implementation of logic and application to database searching is based on sampling signals at an appropriate time the main limitation of our methods is: “how finely can we ‘slice’ time?” For the implementation of logic gates, sampling rate willdefine how fast we can perform a logic operation, with a higher sampling rate the smaller the width of the square pulse can be and thus the shorter the time it takes to complete a clock cycle (perform an operation). In practice we are limited by the technology used to implement our method. For the search algorithm, the higher the sampling rate the more items we can store within a given clock cycle. There is one more limitation to the search algorithm, the fact that with increasing size of the database the weights for each

121 Figure 6-3. Schematic representation of the time based search method. location increase exponentially, this would create a problem based on the technology used to implement the method, but can be addressed by a multi-stage addressing system. 6.2 Neural Implementation

Our demonstrations with neural circuits are mainly inspired by the work of Abarbanel et. al. [93, 94]. Specifically we take a neural circuit created by Abarbanel et. al. designed for interspike interval recognition, and we extract from this circuit two compartments, the Time Delay Unit (TDU) and the Detection Unit (DU) (which we rename to Coincidence Detectino Unit (CDU)). We utilize these sub-circuits to manipulate the timing of action potentials, both for their creation and detection. 6.2.1 Neural Models

This short section is a slight deviation from our main theme to very briefly introduce two classic neuron models and three models of synaptic connections, which we use for our algorithm implementations. This is purely done to place the reader in, at least, some

122 context of neuroscience; for further expositions we direct to work by Izhikevich [96–98] and Latham et. al. [99, 100], among many others.

Type I neuron. This is a model in the framework of Hodgkin Huxley, a single compartment model with fast sodium channel, a delayed rectifier potassium channel and a leak channel [96, 98, 101] defined by the following equation:

dV (t) 3 4 C = IDC + gNam (t)h(t)(ENa V (t)) + gK n (t)(EK V (t)) dt − −

+gL(EL V (t)) + IS(t), (6–1) −

where V (t) is the membrane potential, IDC the external DC current, IS(t) the synaptic current, and by defining r = Na, K, L to refer to the sodium, potassium and leak

channels, we have Er as the reversal potentials and gr as the conductance of each channel.

2 In our simulations we have used the following parameter values: C = 1 μFcm− ,

2 2 ENa = 50 mV, EK = -95 mV, EL = -64 mV, gNa = 215 mScm− , gK = 43 mScm− ,

2 gL = 0.813 mScm− . Finally the gating variables X = m, h, n satisfy the following { } dX(t) equation: = αX (V (t))(1 X(t)) βX (V (t))X(t), where αX and βX are given by: dt − − .32(13 (V (t) V th)) 0.28((V (t) V th) 40) αm = (13 (−V (t) V− th)) , βm = ((V (t) V− th) 40)− , e − 4.0− 1 e − 5 − 1 17 (V (t) V− th) − − 18− 4 αh = .128e , βh = 40 (V (t) V th) , e − 5 − +1 0.032(15 (V (t) V th)) 0.5 αn = (15 (V−(t) V th−)) , βn = (V (t) V th) 10 , e − 5 − 1 e −40 − − where Vth = 65 mV. In this model the frequency of spiking as function, which in the − context of our algorithm defines a clock cycle, is given by: f = C√IDC I0, where I0 is − the threshold for spiking and C is a constant based on the model parameters.

Bistable neuron. Again this model derives from Hodgkin Huxley as a single compartment model with a leak current, but the sodium channel is persistent and the potassium channel fast, the defining equation is given by:

dV (t) C = IDC + gNam (V (t))(ENa V (t)) + gK n(t)(EK V (t)) dt ∞ − −

+gL(EL V (t)) + +IS(t), (6–2) −

123 dn(t) n (V (t)) n(t) where the gating variable n(t) is given by: = ∞ − and m (V ), n (V ) = X (V ) dt τn { ∞ ∞ } ∞ by the activation function: X (V ) = 1/(1 + exp((VX V )/kX )); the remaining variables ∞ − are the same as in the type I neuron model, see Equation 6–1. In our simulations we

2 have used the following parameter values: C = 1 μFcm− , ENa = 60 mV, EK = -90 mV,

2 2 2 EL = -80 mV, gNa =20 mScm− , gK = 10 mScm− , gL = 8 mScm− , Vm = -20 mV,

Vn = -25 mV, km = 15 mV, kn = 5 mV, τn = 0.16 mV. This model mainly differs from the type I neuron, as its resting state and oscillating state can both be stable and as a result the system can switch from one state to the other simply by a variation in its state variable (V (t)), unlike type I for which, for behavioural changes a system parameter needs to be varied (IDC )[96, 98]; actually we utilize this difference in behaviour to construct the TDU.

Synapses. We utilize in our neural circuits three types of synaptic connections and their currents are given by:

E Excitatory AMPA: IS(t) = gAS (t)(EE V (t)), where gA is the synaptic • − E conductance, EE = 0 mV is the reversal potential and S (t) the gating variable, that is biologically related to the fraction of bound glutamate receptors;

E Excitatory NMDA: IS(t) = gN B(V (t))S (t)(EE V (t)), where gN is the synaptic • − E conductance and B(V ) = 1.0/(1 + 0.288 exp( 0.062V )). EE and S (t) are the same quantities as for the AMPA synapse; −

I Inhibitory GABAA: IS(t) = gGS (t)(EI V (t)), where gG is the synaptic • − I conductance, EI = 75 mV is the reversal potential and S (t) the gating variable, related to the fraction− of bound gaba receptors.

X X S0(θ(t)) S (t) X E I The gating variables are given by: S˙ (t) = − , where S (t) = S (t),S (t) , τˆ(S1 S0(θ(t))) − { } S0(φ) = 0.5(1+tanh(120(φ 0.1))) is a sigmoidal function, and given the heaviside function − Θ(X) = 1 if X > 0, else Θ(X) = 0 we have θ(t) = Θ(t ti) Θ((ti + τR) t) Θ(X), here i − ∙ − ∙ th ti is the time of the i presynaptic neuronal spike.P Finally we have two time constants

τR =τ ˆ(S1 1) and τD =τS ˆ 1, in biological sense the time for the neurotransmitter to bind − and release, respectively; for the AMPA synapse τR = 0.1 ms and τD = 1.5 ms, for NMDA

τR = 2.5 ms and τD = 70 ms, and for GABAA τR = 1.12 ms and τD = 5.5 ms.

124 Time Delay Unit (TDU) and Coincidence Detection Unit (CDU). The Time Delay Unit is constructed with one of each of the two neuron models we presented, a type I neuron and a bistable neuron, our construction follows directions given in [93]. A schematic of the circuit is given in the inset (a) of Figure 6-4. Neuron α is type I neuron and neuron β is a bistable neuron and both neurons are initiated at a resting potential state. The input connections to the neurons are a AMPA synapse to neuron β and a NMDA synapse to neuron α. The neurons are also connected to each other through a GABAA synapse, α β, and a NMDA synapse, β ¨ α, it is this synapse that we −−

modulate to create a delay. As we have the circuit configured an incoming action potential

arriving at t0 is directed into both neurons. For neuron α, since it is a type I neuron, a single incoming action potential has no effect other than to increase the membrane potential slightly, but there will be no behavioural change. For neuron β though the situation is different, as it is a bistable neuron an incoming action potential “pushes” the neuron to its oscillating state, so neuron β starts generating action potentials. These action potentials drive, through the connecting NMDA synapse, neuron α and eventually will “push” the membrane potential of neuron α enough for it to fire its own “output” action potential. The timing of this output potential depends on how long it takes for neuron β to push neuron α over its firing threshold, and of course this time delay canbe modulated by a “stronger” or “weaker” connection between the two neurons. Specifically the strength of the aforementioned NMDA synapse, β ¨ α, can be varied using a weight

2 parameter R as gN = R gN , with gN = 1 mScm− . Therefore an incoming action R ∙ potential at time t0 will be technically “absorbed” by the system and an output action potential will be generated at a later time t0 + τ(R); in essence though, since there is no difference between the two action potentials, in effect what we achieve is the delayin the propagation of a single action potential. Through simulations we have quantified the effect of R on τ(R) and their relationship is presented in Figure 6-4. Of course we need to make sure that only a single output potential is created, this is achieved through the

125 Figure 6-4. Time Delay Unit (TDU). The relationship between the parameter R, which defines synaptic strength, and the time delay created τ. Inset (a): A schematic representation of the TDU neural circuit. Inset (b): Domain and range where R vs. τ(R) is linear, and hence the range of R values used for flexible logic gate implementation. (Adapted from [93].)

inhibitory GABAA synapse between the two neurons, α β, which takes the “output” −−

spike of neuron α and directs it, besides to the output, back into neuron β inhibiting it and pushing it away from its oscillating state back to its resting state, ceasing further generation of action potentials from neuron β, and as a direct consequence prevents the generation of further action potentials from neuron α itself. The Coincidence Detection Unit is a much simpler case, it is simply a single type I neuron that acts as a classic integrator receiving inputs from two sources through two AMPA synapses. The strength of the conductance of the two synapses is equated so that both input sources have equal effect on the neuron. In addition the strength isset high enough for the neuron not to cross its firing threshold, unless it receives two spikes

126 simultaneously (or within tolerance of a small time interval, defined by the conductance of the synapses). 6.2.2 Algorithm Implementations

The first neural based implementation of logic emulation is simply based ontheidea of sampling the membrane voltage signal of a single neuron at different moments in time. The second method utilizes a circuit of two bi-directionally coupled Hodgkin Huxley neurons, the TDU circuit. Logic gates are emulated by varying the synaptic strength R that determines the time delay τ(R) and then observing the output at a predetermined fixed time instant. The search algorithm is straightforwardly implemented alongthe directions given in the more general context presentation of Section 6.1.2.

Varying the “moment of observation”. This is truly a very simple implementation. We just consider a single neuron producing an action potential, see inset of Figure 6-5. Since this signal is a product of simulation we consider each point generated by the simulation as an instance in time and with the zeroth simulation point as a reference point we define ttotal according to the computational operation to be performed, exactly as in Section 6.1.1. Take the NOR gate as a specific example. In Figure 6-5 we show the results of sampling our signal at the time instances of 2555 for the operation NOR(0,0), 2605 for NOR(0,1)/(1,0) and at 2655 for NOR(1,1). In terms of programming (tprog) and input time lengths (tin) we have tprog = 2555 time steps, specific to the NOR gate, and tin = 50 time steps. A small deviation from the general case is that the action potential actually fluctuates between 48 mV and -92 mV, it is not simply 0 or 1. Therefore we set avoltage threshold, at -45 mV, over which we define the logical 1 and under which the logical 0,at the output. As is clear from Figure 6-5 the appropriate Output values are obtained, that is the voltage is over -45 mV for the (0,0) case and below for the other two cases. The whole procedure can be interpreted as follows: to perform the logic gate NOR we require to

127 Figure 6-5. Demonstration of operating a NOR gate, with a single neuron using different sampling times. Depending on the inputs a different sampling moment is defined beyond the 2555th moment that defines the NOR gate. Each possible case is shown with a circle, black circle ( ) indicating a 1 at the Output, empty circle ( ) a 0. The Output is defined by a voltage threshold at -45 mV, shown. Inset:# The whole action potential considered.

Table 6-3. Appropriate time sample instances, based on simulation points, to perform each of the five gates considered. In brackets the voltage of the action potential, at each point, is given.

I1 I2 AND NAND OR NOR XOR 0 0 2405(-57 mV) 2505 (-9 mV) 2455(-56 mV) 2555 (1 mV) 2485(-52 mV) 1/0 0/1 2455(-56 mV) 2555 (1 mV) 2505 (-9 mV) 2605(-86 mV) 2535 (34 mV) 1 1 2505 (-9 mV) 2605(-86 mV) 2555 (1 mV) 2655(-86 mV) 2585(-50 mV) wait 2555 time steps as a programming time length and then another 50 time steps for each occurrence of a single logical input of 1. Using the same system, that is a single Hodgkin Huxley neuron producing an action potential at the same rate, time instances for reading an Output, with respect to the zeroth simulation point, along with their resulting respective voltage value, for accomplishing the five fundamental gates are given in Table 6-3. For confirmation we refer back to the gate truth table, Table 6-1.

128 Varying the moment of generation. Converse to the previous implementation, in this implementation we fix the moment of observation, in the specific example at

t∗ = 425 ms from an initiating action potential at t0, and we vary the moment of actual generation of the signal. We use “actual time” instead of simulation points to emphasise the independence between the moment of observation and the computational operation. To have the ability to generate an action potential at different times, with respect to t∗, we use a TDU as part of our computational system. Following Figure 6-4, the TDU

receives an input action potential at t0 and produces a delayed output action potential at t0 + τ(R), it is the moment of generation of this output action potential we use for computation. Exactly as explained before, with respect to time instances, we utilize R

values taken from the relationship τ(R), and with Rtotal = Rprog + RI1 + RI2 we represent a desired computational operation. Note that we use the domain 1.68 < R < 1.72 where the curve of Figure 6-4 is linear, see inset (b) of same figure, so that the changes in R are linearly proportional to the changes in τ(R) (actual time) and hence changes for a logical 1 at the input are independent of whether it is at Input1 or Input2; also note that R and τ(R) are inversely proportional, that is higher R means earlier action potential generation. Finally again as in the above implementation we define a voltage threshold at -45 mV for distinction of the Output between 1 and 0. Obtaining the result at the “moment of observation”, and actually implementing neurally a sampling scheme, can be realized with a Coincidence Detection Unit, that is a neuron that itself will fire when two neurons connected to it fire simultaneously, sooneof these two neurons can be the one “acting” as our gate, having its settings (R) changed according to the programming and inputs, and the other neuron be fixed to fire solely ata specific moment, “acting” as the “reader”. As an explicit example, see Figure 6-6, consider the universal gate NOR with Rprog = 1.705. For the NOR gate and the case of inputs (0,0) Rtotal = 1.705 + 0 + 0; with this R value the action potential is generated with its peak at 424.5 ms after the initiating

129 Figure 6-6. Demonstration of operating a NOR gate, with a neural circuit using different delay times. All three cases of inputs are shown, in actuality only one action potential is generated by the system. The time of generation of the action potential is inversely proportional to R. Given the “moment of observation” at 425 ms, the Output is obtained using the voltage threshold at -45 mV. Once again is 1 at the Output and 0. # action potential, which results in a voltage of -16 mV at the moment of observation of 425 ms, which is over the threshold of -45 mV, thus the output is 1. For the cases (1,0)

and (0,1), which are degenerate, we have Rtotal = 1.705 + 0.005 + 0. The action potential is generated with a peak at 424 ms, so by the time of the moment of observation the voltage drops, below the threshold, to -89 mV, producing a 0 at the output. We have a similar result for the third and final case of (1,1) at the inputs, Rtotal = 1.705 + 0.005 + 0.005, the peak of the action potential happens at 423.5 ms, so again the voltage drops below the threshold by the time of the moment of observation. It is clear that our system works perfectly to reproduce the logic gate NOR, similar results have been obtained for the other of the four fundamental gates, the R values, and the respective voltages at the moment of observation, are given in Table 6-4; which once again can be compared with Table 6-1.

130 Table 6-4. Appropriate R values so time shift an action potential in order to perform each of the five gates considered. The action potential is sampled always at thesame moment in time, 425 ms after an initiating action potential. In brackets the voltage of the action potential, at 425 ms, is given.

I1 I2 AND NAND OR NOR XOR 0 0 1.690(-58 mV) 1.700 (48 mV) 1.695(-56 mV) 1.705(-16 mV) 1.697(-53 mV) 1/0 1/0 1.695(-56 mV) 1.705(-16 mV) 1.700 (48 mV) 1.710(-89 mV) 1.702 (35 mV) 1 1 1.700 (48 mV) 1.710(-89 mV) 1.705(-16 mV) 1.715(-85 mV) 1.707(-53 mV)

Neural search algorithm. The search method can also be implemented with the same neuronal circuits, TDU and CDU along with an Integrating Unit (IU), simply a single neuron with its threshold for firing an action potential set very high. The first component of our search engine is encoding by each unique moment intimea database element. As we have shown above, items from any given alphabet can be represented by time intervals. We do the same, but instead of using actual time intervals for representation of items we use unique R values. So a database of size N, regardless of size, can be constructed using N TDUs each one representing one of the M distinct items of the alphabet associated with a unique R value and as a result a unique instance in time for generating an action potential. The second component, just as in the abstracted explanation above, is a Coincidence Detector Unit (CDU), this is a single neuron at resting potential. The property of this neuron that makes our search engine work is that it requires two action potentials to arrive at its membrane simultaneously for the neuron to produce a response action potential. If and when, two action potentials coincide the CDU will generate its own action potential, otherwise it will not fire, providing us a way to confirm the existence or not of a searched for item in the database. So now we can incorporate in our model the Search Engine elements, TDUs set at the R values for the items we wish to search for. We couple each CDU with each TDU of the database and to all the Search Engine elements, referring back to Figure 6-3, so that when the Search Engine elements fire, the CDUs that are attached to the searched for items will

131 receive two simultaneous action potentials and fire themselves. If a searched for item does not exist in the database no CDU will receive two simultaneous spikes so there will be no action potential from the CDU. Finally we connect all CDUs to an Intergrading Unit (IU) to combine all search results. Each CDU is connected to the IU with a synapse of different conductance weight (g), following G¨odelnumbering; the first CDU is connected with a weight 20g the second with weight 21g, third unit with 22g and so on. This provides us with a way to uniquely mark each position of each item in the database. At the output we read the voltage of the IU and interpret the searched for results. There are two parameters to consider at the reading of the result, first is the time instance at which a voltage increase occurs, which represents the searched for item, and second is the amount by which the voltage of the IU is increased. Basically, each moment in time represents an information item, as defined from the different R values we use to encode them, and each increase in voltage represents a position in the database given by the G¨odelnumbering. 6.3 Electronic Implementation

As it turns out the electronic implementation of this method can be performed with simple multipliers instead of a complicated sampling circuit of variable delays. Basically we generate two signals, one representing the “generic signal”, as in Section 6.1.1, and the other signal is generated at the time instance that represents the computational operation to be performed, whether logic operation or search operation. The multiplier simply multiplies the two signals and the product of course will be 1 only if the two signals coincide in time, in essence we are emulating the neural Coincidence Detection Unit.

Logic gates algorithm. As shown in Figure 6-7 we connect the reference signal square pulse generator (Vr) to a multiplier (U1) along with the varying “sampling signal” generator (Vs). The varying sampling signal generator emulates the moment of observation by producing a short square pulse signal with varying delay compared to

132 Figure 6-7. Schematic representation of an electronic circuit for logic using time. Vr provides the reference signal while Vs the “moment of observation”. The two signals switch from 1 V to 0 V, therefore U1 through simple multiplication provides the Output, given straightforwardly by the voltage of resistor R1 (1 V logical 1, 0 V logical 0).

the reference square pulse of Vr. The function of the multiplier U1 is to simply multiply the two signals and therefore the resulting signal is the same as if sampling the reference square pulse Vr at the moment of observation. For our implementation we used a reference signal of frequency 10 Hz and pulse width 30 ms, shown in Figure 6-8(a), as a result our time length representing a logical 1 is of 30 ms(=tin). The “sampling signal” from Vs is again a pulse of frequency 10 Hz, but its generation instance can be varied creating a variable delay signal, see Figures 6-8(b)(d)(f). The time delay of generation of the “sampling signal” is our programming time length plus the appropriate delay for each of the three pairs of inputs, values for programming the five fundamental gates and each case of inputs are given in Table 6-5.

133 Figure 6-8. Demonstration of operating a NOR gate using an electronic circuit utilizing time dependant computation. (a) Reference signal, to be sampled at different times. Refer back to Figure 6-2(f). (b) Sampling signal for the case of inputs (0,0) to a NOR gate, equivalent to tprog + 0 + 0 = 60 ms. (c) Output signal for performing NOR(0,0), output is a logical 1, signal is “high”. (d) Sampling signal for the cases of inputs (1,0)/(0,1) to a NOR gate, equivalent to tprog + tin + 0 = 90 ms. (e) Output signal for performing NOR(1,0) or NOR(0,1), output is a logical 0, signal is “low”. (f) Sampling signal for the case of inputs (1,1) to a NOR gate, equivalent to tprog + tin + tin = 120 ms. (g) Output signal for performing NOR(1,1), output is a logical 0.

Table 6-5. Delay times for Vs, the moment of the “sampling signal” with respect to the start of the clock cycle, to implement each of the five gates considered.

I1 I2 AND NAND OR NOR XOR 0 0 0 ms 40 ms 10 ms 60 ms 30 ms 1/0 0/1 30 ms 70 ms 40 ms 90 ms 60 ms 1 1 60 ms 100 ms 70 ms 120 ms 90 ms

134 As a specific illustrative example take the NOR gate. As indicated in Table 6-5 the programming time length for the NOR gate is 60 ms, and as stated above the time length representing a logical 1 at the inputs is 30 ms. Therefore for the set of inputs (0,0), tprog + 0 + 0, we generate a “sampling signal” with simply a delay of 60 ms, see Figure 6-8(b). For the sets of inputs (1,0) and (0,1), tprog +tin +0, we generate a “sampling signal” with delay 90 ms, see Figure 6-8(d); and finally for the set (1,1), tprog + tin + tin, we generate a “sampling signal” with delay 120 ms, see Figure 6-8(f). The output is read at the resistor R1 and is shown in Figures 6-8(c)(e)(g), as VOut. From Figures 6-8(c)(e)(g) we can see that the correct Output is generated for each case of inputs for reproducing the truth table of a NOR gate.

Search algorithm. For the search method, again our electronic implementation is greatly simplified with the use of multipliers, U1, U2, U3, simply in place of Coincidence Detection Units, see Figure 6-9 for the specific schematic. As a specific example consider

encoding the numbers 1, 2, 3 in three database elements Ri,Rj,Rk, using m Δt, where { } ∙ Δt = 10 ms and m N, representing any abstract item. In the search elements RS1, ∈ RS2, RS3, we have encoded our search for numbers, specifically all three 1, 2, 3 . The { } the accumulated signal from the search elements is fed independently to each multiplier, along with the signal of each multiplier‘s associated database element. The result from each multiplier is a signal of a square pulse if the searched for item exists or a zero signal if it does not exist. For the final step each multiplier is connected to an accumulator sub-circuit through resistors, R7, R8, R9 whose actual resistance follows G¨odelnumbering. The readout is set at the output of the accumulator that collects the signals from the multipliers, at U5. As seen in Figure 6-10(b) all three items are identified as existing in the given database, the timely spikes at the output, and their location in the database is given by the voltage height of their respective spike. Specifically we see in Figure 6-10(b) three spikes, the first spike at 10 ms represents the occurrence of item given the number 1in

135 Figure 6-9. Schematic representation of an electronic circuit for the time dependant search method. The R-units are signal generators, Rx being the generators used for encoding the database, RSi being the generators used for encoding searched for items. The U-units are multipliers, replacing the CDUs of Figure 6-3. R7, R8, R9 are the resistors associated with each database element defining its location through their resistance value. The dash-boxed sub-circuits are accumulator circuits (relating to IUs) “putting togther” both a “search signal”, in the case of multiple simultaneous searches, and the result signal from the multipliers.

the database and the height of 1 V represents the fact that this item is stored in the first location of the database. Analogously for the other two spikes representing items numbered 2 and 3, at time instances of 20 ms and 30 ms respectively, again the height of each spike signifies the location of the item; 2 Vfor2nd position and 4 V for the 3rd position in the database. 6.4 Discussion

We have presented two algorithms that use time intervals for computation; the first for performing flexible logic and the second for searching a unsorted database. The central

136 Figure 6-10. Demonstration of performing a search using an electronic circuit utilizing time dependant computation. Refereing to the schematic of Figure 6-9: (a) The output of the search key accumulator at U6, this is the “search signal” from all the search elements that is fed to all multipliers; (b) The output of the result accumulator U5, confirming the existence of items items 1, 2, 3 with pulses at 10 ms, 20 ms, and 30 ms, respectively; also providing{ the } location of each item in the database, through the voltage of each pulse, 1 V, 2 V, 4 V, respectively. idea of the proposed algorithms is that each, and any, moment in time can be utilized to represent a computational operation and/or data. With the use of a defined clock cycle length we segment time into manageable pieces, within which we define moments in time that will produce different responses based on a variable quantity. In our proposed algorithms the responses are of two types, either the crossing of a threshold or the coincidence of events, both of which can be interpreted as computational operations. With careful construction of a signal, which encapsulates the five fundamental logic gates, we have shown how time intervals can be used to represent the programming of a universal flexible logic gate. In addition we have shown how a zero length time interval can represent a logical 0 and a finite length time interval a logical 1. A straightforward

137 combination (summation) of the programming time length and the logical inputs time length can provide a moment in time at which sampling the signal will produce the desired output. We can thus visualize an array of many computational elements each receiving a common signal. Each of these elements is programmed to sample the signal at different times producing, in parallel, the results of different logical operations. Actually there is no reason to confine ourselves to the “simple” signal presented here, Figure 6-2(f). The method can be generalized to sampling more “complicated” signals, reproducing even more complicated operations at different times; in the spirit of Chapter 5. Furthermore, we have shown how many different time intervals can be utilized inan algorithm to perform searches within a given database. We have shown how each natural number can be represented by a time interval, defined between the “tick” of a universal clock (clock cycle) and a delta function. Within one clock cycle we search the database by looking for coincidences between our search for elements and the elements encoded in the database. It is significant to note that our method completes a search within one single clock cycle regardless of the size of the database; and it is not confined to searching for single items sequentially, but multiple searches can be performed within the same clock cycle. For both algorithms presented we provided realizations, both with neural and electronic circuits. As we have shown, the implementations are straightforward, especially in the case of electronic circuits. The presented methods are a direct consequence of considering delayed time responses as computational commands. For both implementations the main limitation is the smallest timescale of the system; a consequence of considering time as computational data and programming, the smaller the time interval we can utilize, the more data and operations can be represented.

138 CHAPTER 7 CONCLUSION Chaotic Computation has come a long way since its first step in 1998 [25]. Both within the “original” group of Ditto et. al. [102, 103], and even now days within the more the more extended group of collaborators [104, 105], theoretical and experimental advancements have established Chaotic Computation as a realistic possibility for the future of computation. We are glad to report that the field has also attracted the attention of researchers beyond our group [106, 107]. Even further, the aim of bringing research out of the laboratory and into the public domain is actively pursued; Chaologix Inc. (www.chaologix.com) is an active company with the sole aim to realize the technology and present the solution to the public. In accomplishing this goal the main attempts circle around conventional CMOS based VLSI circuitry [108], but as we emphasised Universality allows us to consider even more esoteric approaches such as magneto-based circuitry [109], or even high speed chaotic photonic integrated circuits [110]. The aims of this dissertation were threefold. Obviously foremost, expose the reader to the whole field of Chaotic Computation; in addition to present the material withits natural multidisciplinary nature, from the underlining mathematics, to the physics that governs our systems, the inspiration from biology and the engineered implementations; note our attempt was to expose the reader to details from each field, but at the same time without deviating from the main aim. Finally and of equal importance, we aimed at establishing where the “cutting edge” of research of the field is, prepare the groundwork for our future work. In closing we wish to point out projections of the future paths emanating from each chapter. Developing further the connections implied in chapter one is essential in determining the ultimate limitations of our research and “bootstraps” directly to where chapter two left of; specifically the further extension of the Deutsch-Jozsa

139 algorithm, in the spirit of how quantum computing progressed, into algorithms for more specific functions, like Fourier Transform and integer factorization, and even further. The importance of searching, matching, pattern recognition in our current general technological aims, drives us to consider promoting further our method for searching; a sorting algorithm should probably be the most immediate target, with issues of multidimensional data, and perhaps extrapolated action, or decision, taking not the too far of future. While all the aforementioned could be considered mainly theoretical milestones, the implementation side can not be ignored. Faster, more efficient, smaller, we share all goals that current established computer hardware technology has, simpler electronic, or even alternative, realizations will be pursued. Finally, and in some sense coming full circle back to the fundamental mathematics, through chapters five and six we have setup the field for investigating further how to obtain independence from actual exact state values, and/or even the utilization of only specific physical quantities, an attempt perhaps to make more tangible the abstract concept of “computation”.

140 REFERENCES [1] G. E. Moore, Electronics 38, 114 (1965). [2] R. Landauer, IBM Journal of Research and Development 5, 183 (1961). [3] W. Arden, et al., International technology roadmap for semiconductors, 2008 update, overview, Tech. rep., International Roadmap Committee (2008). [4] M. D. Austin, et al., Nanotechnology 16, 1058 (2005). [5] R. P. Feynman, R. W. Allen, T. Hey, Feynman Lectures on Computation (Perseus Books Group, 2000). [6] M. A. Nielsen, I. L. Chuang, Quantum Computation and Quantum Information (Cambridge University Press, 2000). [7] R. Edwards, H. T. Siegelmann, K. Aziza, L. Glass, Chaos: An Interdisciplinary Journal of Nonlinear Science 11, 160 (2001). [8] M. N. Stojanovic, D. Stefanovic, Nature Biotechnology 21, 1069 (2003). [9] M. N. Stojanovic, D. Stefanovic, Journal of American Chemical Society 125, 6673 (2003). [10] J. Hasty, F. Isaacs, M. Dolnik, D. McMillen, J. J. J. Collins, Chaos: An Interdisci- plinary Journal of Nonlinear Science 11, 207 (2001). [11] S. H. Strogatz, Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry and Engineering (Perseus Books Group, 2001). [12] W. Ditto, T. Munakata, Communications of the ACM 38, 96 (1995). [13] E. Ott, C. Grebogi, J. A. Yorke, Physical Review Letters 64, 1196 (1990). [14] M. Ding, et al., Physical Review E (Statistical Physics, Plasmas, Fluids, and Related Interdisciplinary Topics) 53, 4334 (1996). [15] M. Ding, et al., Chaos: An Interdisciplinary Journal of Nonlinear Science 7, 644 (1997). [16] N. Margolus, Physica D: Nonlinear Phenomena 10, 81 (1984). [17] T. Toffoli, N. Margolus, Cellular automata machines: a new environment for modeling (MIT press, 1987). [18] J. P. Crutchfield, K. Young, Physical Review Letters 63, 105 (1989). [19] T. Toffoli, N. Margolus, Physica D: Nonlinear Phenomena 47, 263 (1991). [20] A. V. Holden, J. V. Tucker, H. Zhang, M. J. Poole, Chaos 2, p367 (1992 (Jul)).

141 [21] J. P. Crutchfield, Physica D: Nonlinear Phenomena 75, 11 (1994). [22] A. Krikelis, C. Weems, Computer 27, 12 (1994). [23] Agota´ T´oth,K. Showalter, The Journal of Chemical Physics 103, 2058 (1995). [24] C. Moore, Physical Review Letters 64, 2354 (1990). [25] S. Sinha, W. L. Ditto, Physical Review Letters 81, 2156 (1998). [26] S. Sinha, W. L. Ditto, Physical Review E (Statistical Physics, Plasmas, Fluids, and Related Interdisciplinary Topics) 60, 363 (1999). [27] S. Sinha, T. Munakata, W. L. Ditto, Physical Review E (Statistical, Nonlinear, and Soft Matter Physics) 65, 036214 (2002). [28] A. Miliotis, S. Sinha, W. L. Ditto, International Journal of Bifurcation and Chaos 18, 1551 (2008). [29] M. J. Feigenbaum, Los Alamos Theoretical Division Annual Report pp. 98–102 (July 1975 - 1977 Sep.). [30] A. N. Whitehead, B. Russell, Principia Mathematica to *56 (1910-1913) (Cambridge University Press, 1962). [31] K. G¨odel, On formally undecidable propositions of principia mathematica and related systems (1931) (Courier Dover Publications, 1992). [32] S. Sinha, T. Munakata, W. L. Ditto, Physical Review E (Statistical, Nonlinear, and Soft Matter Physics) 65, 036216 (2002). [33] K. Murali, A. Miliotis, W. L. Ditto, S. Sinha, Physics Letters A In Press, Cor- rected Proof, (2009). [34] A. Miliotis, K. Murali, S. Sinha, W. L. Ditto, M. L. Spano, A simple nonlinear dynamical computing device (2009). To appear (Chaos, Solitons and Fractals, 2009). [35] J. Gleick, Chaos: Making a New Science (Penguin (Non-Classics), 1988). [36] R. L. Devaney, An Introduction to Chaotic Dynamical Systems, 2nd Edition (Westview Pr (Short Disc), 2003). [37] H. O. Peitgen, H. J¨urgens,D. Saupe, Chaos and Fractals: New Frontiers of Science (Springer, 1993). [38] E. Ott, Chaos in Dynamical Systems (Cambridge University Press, Cambridge, 2002). [39] J. Moehlis, K. Josic, E. T. Shea-Brown, Scholarpedia 1, 1358 (2006). [40] B. B. Mandelbrot, The Fractal Geometry of Nature (W. H. Freeman, 1982).

142 [41] K. G. Wilson, Physical Review B 4, 3174+ (1971). [42] K. G. Wilson, Scientific American (1979). [43] J. Banks, J. Brooks, G. Cairns, G. Davis, P. Stacey, American Mathematical Monthly 99, 332 (1992). [44] T. Li, J. A. Yorke, American Mathematical Monthly 82, 985 (1975). [45] N. A. Sharkovsky, Scholarpedia 3, 1680 (2008). [46] N. A. Sharkovsky, International Journal of Bifurcation and Chaos 5, 1263 (1995). [47] P. Stefan, Communications in Mathematical Physics 54, 237 (1977). [48] P. Saha, S. H. Strogatz, American Mathematical Monthly 68, 42 (1995). [49] J. Bechhoefer, American Mathematical Monthly 69, 115 (1996). [50] W. B. Gordon, American Mathematical Monthly 69, 118 (1996). [51] M. J. Feigenbaum, Journal of Statistical Physics 19, 25 (1978). [52] M. J. Feigenbaum, Physica D: Nonlinear Phenomena 7, 16 (1983). [53] P. Cvitanovi´c, Universality in chaos (Bristol, Adam Hilger, Ltd., 1984). [54] W. L. Ditto, S. N. Rauseo, M. L. Spano, Physical Review Letters 65, 3211 (1990). [55] W. L. Ditto, K. Showalter, Chaos: An Interdisciplinary Journal of Nonlinear Science 7, 509 (1997). [56] V. In, W. L. Ditto, M. L. Spano, Physical Review E (Statistical Physics, Plasmas, Fluids, and Related Interdisciplinary Topics) 51, R2689 (1995). [57] K. Murali, S. Sinha, Physical Review E (Statistical, Nonlinear, and Soft Matter Physics) 68, 016210 (2003). [58] T. Shinbrot, et al., Physical Review Letters 68, 2863 (1992). [59] L. Glass, W. Zeng, International Journal of Bifurcation and Chaos 4, 1061 (1994). [60] D. J. Christini, V. In, M. L. Spano, W. L. Ditto, J. J. Collins, Physical Review E (Statistical Physics, Plasmas, Fluids, and Related Interdisciplinary Topics) 56, R3749 (1997). [61] V. In, et al., Chaos: An Interdisciplinary Journal of Nonlinear Science 7, 605 (1997). [62] C. Wagner, R. Stoop, Physical Review E (Statistical, Nonlinear, and Soft Matter Physics) 63, 017201 (2000).

143 [63] S. Sinha, W. L. Ditto, Physical Review E (Statistical, Nonlinear, and Soft Matter Physics) 63, 056209 (2001). [64] S. Sinha, Physical Review E (Statistical, Nonlinear, and Soft Matter Physics) 63, 036212 (2001). [65] S. Sinha, Physical Review E (Statistical, Nonlinear, and Soft Matter Physics) 66, 016209 (2002). [66] S. Sinha, Physical Review E (Statistical Physics, Plasmas, Fluids, and Related Interdisciplinary Topics) 49, 4832 (1994). [67] P. Bak, C. Tang, K. Wiesenfeld, Physical Review A 38, 364 (1988). [68] P. Bak, C. Tang, K. Wiesenfeld, Physical Review Letters 59, 381 (1987). [69] K. Murali, S. Sinha, W. L. Ditto, International Journal of Bifurcation and Chaos 13, 2669 (2002). [70] K. Murali, S. Sinha, W. L. Ditto, Physical Review E (Statistical, Nonlinear, and Soft Matter Physics) 68, 016205 (2003). [71] K. Murali, S. Sinha, W. L. Ditto, Pramana 64, 433 (2005). [72] U. H¨ubner,N. B. Abraham, C. O. Weiss, Physical Review A 40, 6354 (1989). [73] C. O. Weiss, et al., Applied Physics B: Lasers and Optics 61, 223 (1995). [74] K. Murali, S. Sinha, Physical Review E (Rapid Communications) 75, 025201 (4) (2007). [75] K. Murali, S. Sinha, W. L. Ditto, A. R. Bulsara, Reliable logic circuit elements that exploit nonlinearity in the presence of a noise floor (2009). To appear (Physical Review Letters, 2009). [76] T. Munakata, S. Sinha, W. L. Ditto, IEEE Transactions in Circuits and Systems 49, 1629 (2002). [77] D. R. Chialvo, A. V. Apkarian, Journal of Statistical Physics 79, 375 (1993). [78] D. Deutsch, R. Jozsa, Proceedings: Mathematical and Physical Sciences 439, 553 (1992). [79] A. Ekert, R. Jozsa, Reviews of Modern Physics 68, 733 (1996). [80] L. K. Grover, Physical Review Letters 79, 325 (1997). [81] B. S. Prusha, J. F. Lindner, Physics Letters A 263, 105 (1999). [82] N. J. Corron, S. D. Pethel, B. A. Hopper, Physical Review Letters 84, 3835 (2000).

144 [83] E. R. Hunt, Physical Review Letters 67, 1953 (1991). [84] K. Myneni, T. Barr, N. Corron, S. Pethel, Physical Review Letters 83, 2175 (1999). [85] K. Chlouverakis, M. Adams, Electronics Letters 41, 359 (2005). [86] T. Morie, S. Sakabayashi, M. Nagata, A. Iwata, Circuits and Systems I: Fundamen- tal Theory and Applications, IEEE Transactions on 47, 1652 (2000). [87] K. Murakoshi, T. Morie, M. Nagata, A. Iwata, Design Automation Conference, 2000. Proceedings of the ASP-DAC 2000. Asia and South Pacific pp. 23–24 (2000). [88] J. Garcia-Ojalvo, R. Roy, IEEE Transactions on Circuits and Systems I: Fundamen- tal Theory and Applications 48, 1491 (2001). [89] T. Yang, L. O. Chua, International Journal of Bifurcation and Chaos 10, p1091 (20000501). [90] L. Chua, J. Yu, Y. Yu, Circuits and Systems, IEEE Transactions on 32, 46 (1985). [91] M. M. Mano, Computer system architecture (Prentice-Hall, Inc. Upper Saddle River, NJ, USA, 1993). [92] T. C. Bartee, Computer Architecture and Logic Design (McGraw-Hill, 1991). [93] S. S. Talathi, H. D. I. Abarbanel, W. L. Ditto, Physical Review E (Statistical, Nonlinear, and Soft Matter Physics) 78, 031918 (17) (2008). [94] H. Abarbanel, S. Talathi, Physical Review Letters 96, 148104 (4) (2006). [95] C. N. Zhang, M. Zhao, M. Wang, IEEE Transactions on neural networks 11, 739 (2000). [96] E. M. Izhikevich, International Journal of Bifurcation and Chaos 10, 1171 (2000). [97] E. Izhikevich, Neural Networks, IEEE Transactions on 15, 1063 (2004). [98] E. Izhikevich, Dynamical systems in neuroscience: The geometry of excitability and bursting (MIT press, 2007). [99] P. E. Latham, B. J. Richmond, P. G. Nelson, S. Nirenberg, Journal of Neurophysiol- ogy 83, 808 (2000). [100] P. E. Latham, B. J. Richmond, S. Nirenberg, P. G. Nelson, Journal of Neurophysiol- ogy 83, 828 (2000). [101] B. Ermentrout, Neural Computation 8, 979 (1996). [102] K. Murali, S. Sinha, I. R. Mohamed, Physics Letters A 339, 39 (2005).

145 [103] S. Sinha, W. L. Ditto, Philosophical Transactions of the Royal Society A: Mathemat- ical, Physical and Engineering Sciences 364, 2483 (2006). [104] M. R. Jahed-Motlagh, B. Kia, W. Ditto, S. Sinha, International Journal of Bifurca- tion and Chaos 17, 1955 (2007). [105] H. R. Pourshaghaghi, B. Kia, W. Ditto, M. R. Jahed-Motlagh, Chaos, Solitons and Fractals In Press, Corrected Proof, (2008). [106] H. Leung, H. Yu, K. Murali, Physical Review E (Statistical, Nonlinear, and Soft Matter Physics) 66, 036203 (2002). [107] D. Cafagna, G. Grassi, International Journal of Bifurcation & Chaos 16, p1521 (2006 (May)). [108] W. L. Ditto, K. Murali, S. Sinha, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 366, 653 (2008). [109] R. Koch, Scientific American 293, p56 (2005 (Aug)). [110] M. Yousefi, et al., Physical Review Letters 98, 044101 (2007).

146 BIOGRAPHICAL SKETCH Abraham Miliotis was born in Annapolis, Maryland, USA, from parents of Greek-Cypriot descent. He was raised in Cyprus where he attended The English School and gained admission to Imperial College, London in 1997. In 2000 he graduated from Imperial College with a Bachelor of Science in Physics with Theoretical Physics. Under the supervision of professor Dr. Dimitri Vvedensky, he completed his final year thesis, “Stock market crashes treated as critical phenomena”. Dr. D. Vvedensky also introduced Abraham to the work of professor Dr. William Ditto on using dynamical systems for computation, whom he joined at Georgia Institute of Technology, Atlanta, Georgia in 2000, as a graduate student in the Physics department. Between 2001 and 2003 though, Abraham paused his studies to pursue some entrepreneurial opportunities. He moved back to Cyprus where, among other projects, he was involved in the creation of Megaland Computers LTD, an IT retail sales and small-office/home-office support company, and also the setting up of Game-Spot Internet/Gaming Caf´es. In 2003 he rejoined Dr. William Ditto at the Biomedical Engineering department of University of Florida, where he completed his Ph.D. in May of 2009 on “Chaotic Computation”.

147