Phase-Locking and Arnold Coding in Prototypical Network Topologies

Home , Arnold tongue

DISCRETE AND CONTINUOUS Website: http://aimSciences.org DYNAMICAL SYSTEMS SERIES B Volume 9, Number 1, January 2008 pp. 145–162

PHASE-LOCKING AND ARNOLD CODING IN PROTOTYPICAL NETWORK TOPOLOGIES

Stefan Martignoli and Ruedi Stoop

Institute for Neuroinformatics UZH / ETHZ Winterthurerstrasse 190 8057 Z¨urich, Switzerland

Abstract. Phase- and frequency-locking phenomena among coupled biological oscillators are a topic of current interest, in particular to neuroscience. In the case of mono-directionally pulse-coupled oscillators, phase-locking is well understood, where the phenomenon is globally described by Arnold tongues. Here, we develop the tools that allow corresponding investigations to be made for more general pulse-coupled networks. For two bi-directionally coupled oscillators, we prove the existence of three-dimensional Arnold tongues that mediate from the mono- to the bi-directional coupling topology. Under this transformation, the coupling strength at which the onset of chaos is observed is invariant. The developed framework also allows us to compare information transfer in feedforward versus recurrent networks. We find that distinct laws govern the propagation of phase-locked spike-time information, indicating a qualitative difference between classical artificial vs. biological computation.

1. Introduction. Stable limit-cycle oscillations arise as the solutions of a variety of autonomous nonlinear differential equations. Primarily, they are generated when systems pass either through saddle-node or through Hopf bifurcations. Corre- spondingly, stable limit-cycles are widely observed in physics, chemistry and biology. As an example, the most salient neuron models, such as the Hodgkin-Huxley, the Fitzhugh-Nagumo, the Morris-Lecar, or the spatially detailed compartmental cable models of neurons yield stable limit-cycle solutions in biophysically relevant parameter regimes. Whereas for low-dimensional model systems the limit cycle property can be verified from the equations, in the case of the high-dimensional cable model or for biological neurons, the limit-cycle property is exhibited by the phenomenon of phase- and frequency locking. In biology, the limit-cycles interact by means of sharp voltage pulses, the spikes. For mono-directionally periodically pulse-perturbed limit-cycle oscillators, it is simple to derive one-dimensional maps that describe this interaction. These maps are mathematically well understood: The emerging phenomena of phase- and frequency- locking are globally organized along Arnold tongues. For more general networks, a corresponding theory is still missing. Here, we develop the tools that are applicable, in principle, to arbitrary network topologies. Using this approach, we first generalize mono-directional phase-locking to bi-directional coupling, where a smooth change

2000 Mathematics Subject Classiﬁcation. Primary: 37E10; Secondary: 37E45. Key words and phrases. Arnold tongues, phase-locked oscillators, recurrent topology. This research was supported by the SNF-grant 65293.

145 146 STEFAN MARTIGNOLI AND RUEDI STOOP of the involved interaction strengths, characterizes the transition from the feedforward to the recurrent topology. Our first finding will be that during this transition, the Arnold tongues undergo a continuous transformation as well. The developed description also allows to investigate and compare general feedforward versus recurrent networks. Here our main finding is that the properties of how spike-time information is processed in these networks, differ in a salient way, which may explain the observed increased efficacy of biological neural networks, which predominantly are of recurrent type. We start with a review of phase-locking phenomena and their relation to the circle-map universality class. Our present study is motivated by recent neurobi- ological experiments, where experimentally measured response functions provide insights into the dynamics of coupled neurons, and point out a potential role of phase-locking as a mechanism of information encoding. In Section 2, we develop a novel discrete phase-response map that can be used to describe arbitrary networks of pulse-coupled limit-cycle oscillators. Using this tool, we investigate in Sections 3-4 bi-directional pair coupling, where we prove the existence of three-dimensional Arnold tongues that connect the feedforward and recurrent topologies of coupled pairs of neurons in a continuous way. Using coupled neurons as elements, we will construct and analyze more general networks in Sections 5. In particular, the efficacy of information processing in feedforward and in recurrent networks will be compared.

1.1. Circle-map universality. Locking was first described in 1665 by Christiaan Huygens [1], who discovered that two pendulum clocks, attached on opposite sides of a wall, tend to synchronize their frequencies in anti-phase. In order to describe the physics behind this phenomenon, he developed a theory, which today still provides the most salient insights into the phenomenon. More recently, in a different context, Arnold [2] provided a refined mathematical description of locking. He studied the return map of a periodically forced oscillator by means of the equation K x = f (x )= x +Ω sin(2πx ) mod1, (1) n+1 Ω,K n n − 2π n the so-called sine-, or Arnold circle-map, where Ω = TS/T0 is the frequency ratio of the two oscillators, and K denotes the mono-directional perturbation strength. The connection between limit-cycle oscillators and the sine-map can be derived from the so-called kicked rotator model [3]. For this system, the equations of motion can be analytically reduced to the one-dimensional discrete sine-map, in the limit of very strong damping. One essential insight added by Arnold is that the sine-map is characteristic for the universality class of circle-maps of the form fΩ,K : [0, 1] [0, 1], → x = f (x )= x +Ω g(K, x ) mod1, (2) n+1 Ω,K n n − n where the phase response function (PRF) g(K, xn), defined by

g(K, x) := T (K, x)/T0, g(K, 0) = g(K, 1)=1, g(0, x)=1, (3) measures the effect T (K, x) of a perturbation of strength K, arriving at phase x of an oscillator of intrinsic period T0. This discrete map formulation can be seen as the Poincarésection of a phase model of the motion along the stable limit-cycle [4]. In the latter picture, a perturbed limit-cycle oscillator is represented by a differential PHASE-LOCKING IN PROTOTYPICAL NETWORK TOPOLOGIES 147 equation of the form N φ˙ = ω + K P (φj )R(φ), (4) j=1 X where φ(t) is the phase, ω the intrinsic constant angular velocity, R(φ) the sensitivity function and K the coupling strength. P (φj ) is called the influence function and can be interpreted as a stimulating waveform, originating from a source of perturbation labelled by j. For weak coupling, the sensitivity function is equivalent to an ’infinitesimal’ PRF 1 g(K = ǫ, φ). We will use the discrete-time model introduced by Arnold. − As an example, the PRF of a sinusoidal limit-cycle perturbed by spikes is given by K g(K, x)=1 sin(2πx), (5) ± 2π · where the plus sign refers to in-phase, and the minus sign to anti-phase 1/1- synchronization. For specific physical or biological oscillators, the PRF can be determined by simple experiments [5, 6, 7]. Eq. (5) may serve as a guiding line for many experimental pulse-perturbed limit-cycles in the following sense. An experimentally obtained return map fΩ,K belongs to the circle-map universality class if the following requirements are met [8]: a) fΩ,K (x) is a diffeomorphism for K KC, fΩ,K(x) is non-invertible.

The salient properties of the circle-map universality class (Eq. (2)) in the Ω,K - parameter space are the following [9, 8] { }

Arnold tongues Tp/q of stable phase-locking emerge at every rational winding • number ω = p/q from the Ω-axis. The intervals of locking expand with increasing interaction strength K. At a critical interaction strength KC [2], the tongues start to overlap. For subcritical interaction strength, the widths of the Arnold tongues are • described by sequences of rational Farey-tree numbers Q [0, 1] (global scaling law) [10]. ∈ As a function of Ω, at K = KC , the winding numbers ω form a devil’s stair- • case [2], where the union of locked intervals has full measure [11]. Local scaling laws govern the transition from quasiperiodicity to chaos [12]. •

1.2. Phase-locking in neurobiology. Experimentally measured PRFs g(K, x) can be used to study phase-locking in neurobiology. Provided that a modeled or real neuron has a suﬃciently stable limit-cycle behavior (which is generally the case), experimental results of spiking neurons perturbed by strong spikes can be modeled by circle-maps (Eq. (2)). In the last decade, PRFs have been measured for a variety of biological and model neurons [13, 14, 6, 7, 15, 16, 17, 18, 19]. Stoop et al. [14] measured the PRFs of both inhibitory and excitatory stimulated pyramidal neurons in rat neocortex and determined experimentally the corresponding Arnold tongues (see Fig. 1). 148 STEFAN MARTIGNOLI AND RUEDI STOOP

In particular, it was proven that only strong inhibitory stimulations could lead to chaotic response, on a small, though nonzero set of the Ω,K -parameter space [14]. More recently, Netoff et al. [16] induced 1/1-locking{ in hybrid} circuits, in which neurons from the hippocampal formation of rats were coupled to virtual neurons, by taking advantage of previously measured PRFs. Similar dynamic-clamp techniques were used by Pervouchine et al. [17] in three-cell networks of the entorhinal cortex, resulting in the claim that the measured PRFs offer a possible explanation for the generation of β-frequency oscillations. A slightly stronger notion of weak coupling in the Winfree model sense (see Eq. (4)) was verified in neurons of the Aplysia Californica [18], implying that the “infinitesimal” PRFs are independent of the stimulating waveform. Our contribution focuses on type I oscillators (oscillation appears via a saddle-node bifurcation). For type II oscillators (oscillation appears via a Hopf bifurcation), the findings hold on a somewhat reduced parameter range. In the Morris-Lecar neuron model [19], both bifurcations are observed, depending on the chosen parameters.

V 1.4

1 φ 0 1.2 t

1 s 0.8 K 1 2 0.6 34 0.4

0.2

0 0 0.2 0.4 0.6 0.8 1 Ω

Figure 1. Arnold tongues Tp/q of an inhibitory stimulated pyramidal neuron, calculated from experimentally measured PRFs, from [7]. Numbers denote the periodicity q associated with a rational winding number ω = p/q. Inset: Experimental veriﬁcation along the line s (sweeping through Ω). Upper panel: Measured membrane potential as a function of time. Lower panel: Associ- ated phases of arriving stimulations. Repeated periodicities of 2, 3 and 4 can be read oﬀ (followed by an unsettled high period and period 1, not shown). PHASE-LOCKING IN PROTOTYPICAL NETWORK TOPOLOGIES 149

The code that the neurons use to transport and process information is still unknown. From a computational viewpoint, the ability of a periodically stimulated neuron to respond with any desired periodicity p is remarkable. Here, p is the periodicity of the phases x where the stimulations arrive, drawn from the rational winding number ω = p/q. As has been first pointed out in [6], Arnold tongues offer a coding scheme, able to represent both presently discussed neuronal coding mechanisms, that is rate-coding and temporal coding. The code can be represented as a mapping Coding: F , F q , (6) { s 0}→{ } where Fs is the stimulating frequency and F0 the intrinsic spiking frequency of the target neuron. Arnold coding has several appealing properties [6], in particular a Huffman-property, generated by the Farey-tree ordering of tongue widths. The shorter the corresponding sequences are (periodicities q), the larger and the more stable the corresponding partitions in parameter space are. An encoding of the system’s trajectory in the Ω,K -parameter space may then look as the inset of Fig. 1 that was taken along{ the} line s in the large figure: a logbook that reports the periodicities along the trajectory. Such a coarse-graining of the behavior can be associated with a precise notion of computation [20]. Biological, in particular cortical, neurons are generally embedded in large and complex networks, with predominantly recurrent local connectivity. It is generally believed that the network’s computational properties depend on the underlying network topology, and that it is the recurrent connectivity that endows the brain with its computational power (that seems to outscore the feedforward structures of traditional computers by far). For identical oscillators and interaction strengths, the return maps for bi-directionally coupled pairs, rings and lattices have been derived [21]. Here, we are interested in general p/q- phase-locking. To this end, tools allowing the investigation of arbitrary complex network architectures, valid for all possible intrinsic frequencies and interaction strengths will be developed.

2. Discrete phase-return maps for arbitrary network architectures. Large networks of pulse-coupled limit-cycle oscillators are often studied in terms of the Winfree model (4), see, e.g. [22]. Discrete-time dynamical systems are, however, simpler to handle, both numerically and analytically. Here, we derive a discrete algorithm that provides the parallel evolution for arbitrary network size and archi- tecture, as well as a convenient extraction of the phase at which a stimulation arrives • the interspike interval (ISI) • the spike-time • the (one-dimensional) Lyapunov exponents. • j Assume there are N oscillators with intrinsic ISIs T0 , j = 1, 2, ..., N connected by a connectivity matrix Kij , Kii = 0. For simplicity of presentation we assume that j only one type of PRFs g(Kij , x ) will be present. This assumption can be easily relaxed if needed. Let g(K, x) satisfy g(K, 0) = g(K, 1) = 1, g(0, x) = 1, and additionally g(K, x) x 0 x [0, 1], (7) − ≥ ∀ ∈ which sets an upper bound to interaction strengths K [0,Kmax.]. At time t0 = j ∈ j 0, the oscillators have the phases φ0. The oscillators’ next spiking times t0 are 150 STEFAN MARTIGNOLI AND RUEDI STOOP

j j j scheduled to be t0 = T0 (1 φ0). Neuron number i and time t of the ﬁrst spike in the network, are given by −

j j t,i : t = min [T0 (1 φ0)]. (8) { } j∈1,2,...,N − At time t, the oscillators are at phases

i x0 = 0, j i j t0 t j T0 i x0 = 1 −j = φ0 + j (1 φ0), (9) − T0 T0 −

j where j =1, 2, ..., n, j = i. Note that phase x0 in Eq. (9) refers to the time when a stimulation arrives at oscillator6 j, similarly to the circle-maps, Eq. (2). The general relationship between φ and x, for a spike from oscillator i to oscillator j, is given by i j j T0 i xn = φn + j (1 φn). (10) T0 − j By the spike, phase x0 is reset to φj =1 g(K , xj ). (11) △ − ij 0 In Ref. [21], this relationship is used to deﬁne the phase-resetting curve (PRC). The reset phases are then given by

φj = xj +1 g(K , xj ). (12) 1 0 − ij 0 For the nth spike, we obtain

j j t,i : t = min [T0 (1 φn)], (13) { } j∈1,2,...,N − i i j j T0 i j T0 i φn+1 = (1 δij ) (φn + j (1 φn)+1 g(Kij , φn + j (1 φn))). (14) − · T0 − − T0 − For the sine-map PRF (Eq. (5)), the corresponding expression would be

i i j j T0 i Kij j T0 i φn+1 = (1 δij ) (φn + j (1 φn) sin2π(φn + j (1 φn))). (15) − · T0 − ∓ 2π T0 −

j Note that this algorithm yields the phases φn after phase resetting, whereas the j phase return maps yield the phases xn at perturbation arrival. The relation between the two quantities is via Eq. (10). The above algorithm could alternatively be written by using the nth scheduled j spiking time tn of neuron j as the variable. With a derivation similar to the one for the phases, this approach yields

j i j j j j tn tn t,i : t = min (tn), tn+1 = tn + T0 [δij 1+ g(Kij , 1 − )]. (16) j=1,2,...,N j { } · − − T0 For a fast numerical evaluation, both algorithms can be cast in vector and matrix notation. PHASE-LOCKING IN PROTOTYPICAL NETWORK TOPOLOGIES 151

3. Phase-locking for bi-directional pair coupling. We now extend the previ- ous work [21] studying the stability of 1/1-locking of recurrently connected identical pairs of oscillators. By making use of the return map derived from Eq. (14), we derive a model of bi-directional pair-coupling valid for all possible frequency ratios and interaction strengths. This allows us to address the following questions: How does the transition from the feedforward topology described by circle- • maps (Eq. (2)) to the recurrent topology (bi-directional pair coupling) take place? How are the Arnold tongues organized in the general, bi-directional case? • At which interaction strengths is the onset of possible chaotic behavior, com- • pared to the corresponding circle-maps? As a first step towards such a model, we represent the system in the three-dimensional Ω,K1,K2 -parameter space. Without loss of generality we may choose Ω := {T 1/T 2 1,} and define g (x) := g(K , x), K := K and g (x) := g(K , x), 0 0 ≤ 1 12 1 12 2 21 K2 := K21, respectively. If one interaction strength is chosen to be zero, we thus obtain the traditional circle-maps 1 K = 0 : x1 = x1 + g (x1 ) mod1, 1 n+1 n Ω − 2 n K = 0 : x2 = x2 +Ω g (x2 ) mod1. 2 n+1 n − 1 n In 3D, the corresponding two Arnold-tongue sheets of mono-directional coupling (henceforth termed ’basic’) lie in the Ω,K1,K2 -parameter space on two orthogonal faces, with the tongues emerging{ from the common} Ω-axis (see Fig. 2f). For K2 = 0, we obtain the traditional Arnold-tongues picture (Fig. 2a), whereas for K1 = 0, the Arnold tongues are repeated on the intervals Ω [1, 1/2], Ω [1/2, 1/3], Ω [1/3, 1/4], ... in a harmonic series (Fig. 2e). This{ corresponds∈ to∈ ∈ } ′ TS the fact that the same Arnold-tongues structure results for Ω = ( T0 mod1) = 1 1 2 ( Ω mod 1). If, as requested, Ω = T0 /T0 1, then the second oscillator’s winding number ω = p/q [0, 1], whereas the≤ first oscillator’s winding numbers ω = 2 ∈ 1 q/p [1, ), where p q. ω2 and ω1 are connected via the isomorphism π : Q [0, 1]∈ Q∞ [1, ) by ≤ ∈ → ∈ ∞ p q π ( )= , (17) q p treating 1/0 and as identical. π associates to every tongue in the Ω, 0,K - ∞ { 2} parameter plane exactly one tongue in the Ω,K1, 0 -plane. For the derivation of the phase-return map of this model, we start{ from the} general algorithm (14), where we need to distinguish three cases. We first assume that at the nth step, oscillator 1 spikes, whereas at the (n + 1)th step, oscillator 2 spikes. At step n, according to Eq. (12) the phases are reset to

φ2 = x2 +1 g1(x2 ), φ1 =0. (18) n n − n n At step n + 1, from the same equation it is obtained that the phase at which the spike arrives at oscillator 1 is

2 1 1 T0 2 xn+1 = φn + 1 (1 φn) T0 − 1 := h (x2 )= (g (x2 ) x2 ). (19) 1 n Ω · 1 n − n 152 STEFAN MARTIGNOLI AND RUEDI STOOP

a) K2 = 0 b) K2 = 0.5 K1 4 4

3.5 3.5

3 3

2.5 2.5

K 2 2 1 K1

1.5 1.5

1 1

0.5 0.5

0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1 Ω Ω

e) K1 = 0 f) 4 0 Ω 3.5 1 4 3 a b c 2.5 3

K 2 2 d 2 1.5 K1

1 1 e 0.5

0 1 2 3 4 0.2 0.4 0.6 0.8 1 K Ω 2

Figure 2. Two bi-directionally coupled anti-phase sine-map PRFs: The continuous transformation of the Arnold tongues Tp/q from the Ω,K1, 0 basic structure (a), into the Ω, 0,K2 basic structure{ (e) is} shown by ﬁve slices (see right bottom{ pan} el). Quasiperiodicity: black, Arnold tongues of stable phase-locking: white. Potentially chaotic behavior: red/gray (positive Lyapunov exponents λ (Eq. (25)). PHASE-LOCKING IN PROTOTYPICAL NETWORK TOPOLOGIES 153

If oscillator 1 and 2 are interchanged, we analogously obtain (case 2): x2 := h (x1 )=Ω (g (x1 ) x1 ). (20) n+1 2 n · 2 n − n For the third case, we assume that oscillator 1 spikes at step n and at step n + 1, i.e., two times in a row. At step n, the spike arrives at the second oscillator’s phase 2 xn. The phases then are reset according to Eq. (18). At step n +1, Eq. (10) yields 2 the phase xn+1 to be 1 2 2 T0 1 xn+1 = φn+1 + 2 (1 φn) T0 − := f 1(x2 )= x2 +Ω g (x2 )+1. (21) n n − 1 n The desired general phase-return map is now obtained by the composition of the three maps h1 (Eq. (19)), h2 (Eq. (20)) and f1 (Eq. (21)). Their composition returns the phases x := x1 of stimulations arriving at the oscillator with the smaller 1 2 intrinsic ISI (T0 /T0 1), labeled by index 1. Formally, this phase-return map can be written as ≤ f : [0, 1] [0, 1] : x = f (x ). (22) Ω,K1,K2 → k+1 Ω,K1,K2 k

Generally, fΩ,K1,K2 will be a piecewise deﬁned map. The pieces are from individual branches associated with numbers s, deﬁned by f(x, s):[0, 1] R : f(x, s)= h (f (s)(h (x))), (23) → 1 1 2 (s) where f1 denotes the s-fold iterate of f1. The augmented iteration index s +1 counts how often oscillator 1 spikes before oscillator 2 does. Consequently, f(x, s) > f(x, s + 1) x [0, 1]. The explicit form of f(x, s = 0) is ∀ ∈ 1 f(x, 0) = x + g (Ω (g (x) x)) g (x). (24) Ω · 1 · 2 − − 2 For Ω = 1, Eq. (24) corresponds to the bi-directional map derived in [21]. The branches f(x, s) that compose the return map fΩ,K1,K2 (Eq. (22)) are found by means of a recursive procedure: Start with f(x, 0). If there is a set A0 [0, 1] where f(x A , 0) 1, then f (x) = f(x, 0) for x A . For the remaining⊂ ∈ 0 ≤ Ω,K1,K2 ∈ 0 set [0, 1] A0, take f(x, 1). If there is a set A1 ([0, 1] A0) where f(x A1, 1) 1, then f \ (x)= f(x, 1) for x A . For the⊂ remaining\ set [0, 1] (A∈ A ),≤ take Ω,K1,K2 ∈ 1 \ 0 ∪ 1 f(x, 2) etc. In this way, the bi-directional return-map fΩ,K1,K2 is fully described by Eq. (22) and Eq. (23). From this observation, the mathematical properties of fΩ,K1,K2 can be determined; the proofs are simple exercises of induction.

Lemma 1: fΩ,K1,K2 has the following properties:

1. For Kj < Kj,C (j = 1, 2), fΩ,K1,K2 is a homeomorphism of the circle. For ﬁxed Kj

fΩS ,K1,K2 (x)= f(x, s).

As an illustration and proof, for the determination of Ω0, we evaluate Eq. (24) for x = 1 for Ω and ﬁnd that Ω=Ω0 = 1. Similarly, we obtain from f(1, 1) = 1/Ω (g (Ω ) Ω )=1 the equation g (Ω ) = 2Ω , to be solved for Ω , which 1 · 1 1 − 1 1 1 1 1 yields, under the assumption on g1, exactly one value Ω1. Successive values Ωi, 154 STEFAN MARTIGNOLI AND RUEDI STOOP

a) b) 1 4

s=0

f Ω,K ,K 1 2 f‘Ω,K1,K2

s=1

0 x 1 0 x 1 x 0 x 0

Figure 3. Bi-directional return map fΩ,K1,K2 for two anti-phase sine-map PRFs (Ω = 0.625 and K1 = K2 =0.8) a), and its deriv- ′ ative fΩ,K1,K2 b). In a), the general construction of fΩ,K1,K2 by means of the two branches f(x, 0) and f(x, 1) is illustrated (Prop-

erty 1 of fΩ,K1,K2 ). The rectangle shows an asymptotically stable orbit of periodicity p = 2. Note the jump in the ﬁrst derivative ′ fΩ,K1,K2 at x0. i > 1, are computed analogously. For an intermediate value Ω (Ω , Ω ), there ∈ s+1 s is a certain xs given by f(xs,s)=1 (and also by f(xs,s +1)=0), such that

f(x, s) 0 x xs, fΩ,K1,K2 (x)= ≤ ≤ ( f(x, s + 1) xs < x 1. ≤ For the anti-phase sine-map PRF Eq. (5), the phase return map fΩ,K1,K2 (x) with Ω (Ω1, Ω0) is shown together with its derivative in Fig. 3, where the two defining branches∈ f(x, 0) and f(x, 1) are indicated. Note the jump in the first derivative at x0. The map yields an asymptotically stable orbit of periodicity p =2, leading to the winding number ω =2/3, since the point on the branch f(x, 1) contributes twofold to the periodicity q =3. 2. For fixed K1,K2 and Ω1 < Ω2, we have f(Ω1,x,s) > f(Ω2,x,s), for all x [0, 1] and s N0. The latter property induces an ordering on the family ∈ ∈ fΩ,K1,K2 of circle homeomorphisms, as is illustrated in Fig. 4. 3. The lift f¯ : R R defined on x [0, 1] by f¯(x, s)= f(x, s)+ s can be Ω,K1,K2 → ∈ continued to R by putting f¯(x+m,s)= f(x, s)+m+s; m Z. In Fig. 4b, such lifts are shown for the anti-phase sine-map PRF. Note that the lift∈ is continuous, but that it has two repeated points of non-differentiability for Ω =Ω , one at (x mod 1)= 0 6 s and another at (x mod 1) = xs.

3.1. Arnold tongues. The construction of fΩ,K1,K2 (Eq. (22)) by means of diﬀer- ent s-branches f(x, s) (Eq. (23)), can be seen as the result of an interaction of two basic Arnold structures in the Ω,K ,K -parameter space, justifying our three- { 1 2} dimensional representation model. The behavior of the system for K1,K2 > 0 can be cast in the following proposition:

Proposition 1. Let Kj

a) b) 1 2 Ω=0,6

Ω=0,7 -

f Ω,K1,K2 fΩ,K1,K2 1

0 x 1 0 x 1

Figure 4. a) Ordering of the maps deﬁned by Eq. (22) with two anti-phase sine-map PRFs (Eq. (5)) along the line K1 = K2 =0.8

(Property 2 of fΩ,K1,K2 ). Ω is decreasing in steps of 0.25 from ¯ Ω=0.7 (lowest map) to Ω = 0.6 (highest map). b) Lifts fΩ,K1,K2 corresponding to the maps of Fig. (a), where Ω = 0.7, 0.65, 0.6

(Property 3 of fΩ,K1,K2 ). The lifts are continuous, but have two repeated points of non-diﬀerentiability at (x mod1) = 0 and (x mod 1) = x0 (compare with Fig. 3b).

(i) for every rational winding number ω = p/q, every initial state approaches a periodic orbit

(ii) fΩ,K1,K2 has generically stable periodic orbits of all rational winding numbers (iii) a stable periodic behavior with winding number ω = p/q extends over a whole single Ω-interval. b) The set T = (Ω,K ,K ) ω(f )= p/q is closed and connected. p/q { 1 2 | Ω,K1,K2 } c) Nonchaotic orbits of fΩ,K1,K2 are guaranteed for K1 K1,C and K2 K2,C. d) The Lyapunov exponent λ can be written as ≤ ≤

N−1 1 ′ λ = lim log fΩ,K1,K2 (xk) N→∞ N | | Xk=0 N−1 M−1 1 ′ 1 ′ 2 = lim log g2(xl ) 1 + log g1(xm) 1 , (25) N→∞ N | − | | − | m=0 Xl=0 X where the index l denotes the stimulations arriving at the second oscillator and m at the first oscillator, respectively. Note that to the second summation term, a visit on the s-branch of f(x, s) contributes s +1 summands. Sketch of a proof of Proposition 1: (For a detailed proof see [23]). ¯ ¯ a) For K fixed and Ω1 < Ω2, we have fΩ1,K1,K2 (x) > fΩ2,K1,K2 (x), x R, i.e., the ordering properties are preserved by the lift. Properties (i)-(iii) are∀ then∈ direct consequences of the Properties 2 and 3 of fΩ,K1,K2 , taking into account the strong theorems that hold [9] for orientation-preserving circle homeomorphisms. b) The orbits of fΩ,K1,K2 depend in a differentiable way on K1 and K2. Therefore, T = (Ω,K ,K ) ω(f )= p/q is a closed connected set. p/q { 1 2 | Ω,K1,K2 } c) fΩ,K1,K2 (Eq. (22)) is composed of different s-branches f(x, s) that obey f(x, s) > 156 STEFAN MARTIGNOLI AND RUEDI STOOP f(x, s + 1) for all x [0, 1] and s N 0 . The functions f(x, s) are compositions ∈ ∈ { } of the functions h1 (Eq. (19)), h2 (Eq. (20)) and f1 (Eq. (21)). h1 and f1 are S invertible iff K1

maps. If any of the two interaction strengths turns supercritical, fΩ,K1,K2 is rendered non-invertible, which is a necessary condition for chaos. From Property (iii) it follows that at subcritical values of K1,K2 , for every • rational winding number ω = p/q there exists a single Ω-interval{ } of stable phase-locking. By Proposition b), these Ω-intervals are continuously connected in the two Kj-directions, so that three-dimensional Arnold tongues Tp/q are obtained, that continuously connect the two π-associated basic Arnold tongues. By inspection of Fig. 2, the continuity of the transformation from one basic Arnold structure into the other can be inferred. Thus, Property b) leads in Ω,K1,K2 -space in a continuous way from the feedforward to the recurrent{ network structure.} Because of condition (7), the anti-phase sine-map PRF (5) was chosen for • Fig. 2. This map allows to study the tongues also in the supercritical range (for the in-phase sine-map, Eq. (7) is violated, which prevents journeys into this area), where in the numerical simulations the unfolding of the overlapping tongues can be observed. Open questions deal with the exact form of the global and local scaling laws for fΩ,K1,K2 for the two sine-map PRFs. Since there are two points of non-differentiability in fΩ,K1,K2 (x), the strong circle diffeomorphism theorems (see e.g. [9]) do not hold, which impedes a renormalization group analysis [12]. More- over, a simple geometrical self-similarity between the stable fixed-point maps of (q) fΩ,K1,K2 , as provided in the case of the mono-directional sine-map (1) by a linear transformation [10], seems not to exist. With respect to the global scaling law, we have the problem of two distinct interacting Farey-orderings, a regular one, and PHASE-LOCKING IN PROTOTYPICAL NETWORK TOPOLOGIES 157 a repeated “inverse” ordering corresponding to Ω′ = 1/Ω mod 1. Our numerical simulations indicate that the widths of Tp/q follow a yet unknown, “mixed” Farey ordering that take both periodicities p, q into account. The smaller p and q are, the larger and more stable Tp/q is. As a consequence, the symmetry of the tongues over the Ω-axis is lost, e.g., T1/3 is larger and more stable than T2/3.

4. Information transport and processing. Analogous to the code provided by the mono-directional interaction, see Eq. (6), the three-dimensional Arnold tongues provide an encoding of the three-dimensional system state. A single tongue Tp/q now implies an identical encoding for both oscillators. If one oscillator is considered as the sender and the other as the receiver, an encoding of the system state can be schematically expressed as Coding: Fs, Fr q where Fs, Fr are the intrinsic frequencies, and q is the periodicity{ of the}→{ receiver.} If the system changes its state in time, this corresponds to a trajectory in Ω,Ks,Kr -parameter space. As various periodic spiking patterns are emitted along{ this trajectory,} this provides an encoding of the trajectory. If the sender changes its state in time (e.g., by a change from p p′), this change is observable at the receiver (by a change q q′, since the two→ periodicities are entangled via ω = p/q, ω′ = p′/q′), and a communication→ between the two oscillators is established. The transport of information from sender to receiver can be schematically expressed by the mapping π p q , (26) { } −→ { } where the transfer function between the two codes is provided by means of the isomorphism π (17), the inverse of the winding number. Thus, under mono-directional interaction, the target oscillator is able to respond to the sender with any desired periodicity. For the bi-directionally coupled system, Eq. (26) expresses that the information, encoded at the sender in terms of a periodicity drawn from its winding number, will be transferred into a periodicity at the target oscillator, drawn from the inverse winding number. In extended networks (N pulse-coupled oscillators), the system’s evolution is described by the algorithm Eq. (14). According to the parameter space representation obtained for pair-coupling, for N oscillators we obtain maximally (N 2 1)- dimensional Arnold tongues that are embedded in a (N 1)-dimensional Ω-space.− Additionally, there is the possibility of sets of m-fold partially− locked oscillators, where 2

5. Feedforward versus recurrent information processing. In the simplest manageable realization, the feedforward / recurrent topologies can be represented by feedforward / bi-directionally coupled chains, respectively. In our first numerical experiment, we demonstrate that both configurations exhibit information transport in the sense of Eq. (26). In order to generate a trajectory similar to that obtained j from proceeding along line s in Fig. 1, we fix the intrinsic ISIs to T0 = 1, j = 1, 2, 3, ..., N and then force a sender at position j = a to sweep its intrinsic ISIs a across the interval T0 [1, 2]. In our experiments, we choose N = 21 and a = 1 for the feedforward chain,∈ and N = 21, a = 11 for the recurrent chain, using for the interaction between the oscillators the anti-phase sine-map PRF (5), with K =0.6. a a For every T0 [1, 2], the 1/1-locked network at T0 = 1 is taken as the initial ∈ I condition. In Fig. 5 for both networks the first two Lyapunov exponents λ1 and 158 STEFAN MARTIGNOLI AND RUEDI STOOP

a) ... c) . . .

0 0

I I λ 1 λ 1

-0.01 -0.01

1 2 1 s j 2 Ts / Tj T 0 / T 0 0 0 b) d) 0 0

I I λ 2 λ 2

-0.04 -0.01

1 s j 2 1 s j 2 T 0 / T 0 T 0 / T 0

Figure 5. I I First two Lyapunov exponents λ1, λ2 of a chain of length N = 21 with feedforward (a,b), respectively recurrent (c,d), connections. A sender at position j = a sweeps its intrinsic ISIs a j across the interval T0 /T0 [1, 2], j = 1, 2, ..., N, j = a. Displayed are the results for a feedforward∈ chain (a = 1), and6 a recurrent chain (a = 11), respectively.

I a j λ2 are evaluated, as a function of the ratio T0 /T0 , where Eqs. (14) and (16) and Eqs. (28), (32) and (33) provide the tools for these evaluations. For both networks, I the largest Lyapunov exponents λ1 < 0 indicate complete locking of the network, similar to a cut through the binary interaction Arnold tongues, i.e., all oscillators respond with periodic spiking patterns (see Figs. 5a,c). As a consequence, the information, as defined by Eq. (6) and Eq. (26), can be seen to propagate along the chain. This implies that the state of the sender can be inferred from any element down the chain. I If λ1 = 0, the motion of the sender would be incommensurate with its nearest neighbor. The weight of this case in the Ω-space, however, is strongly reduced if K is increased or if the network works under dynamic conditions. Moreover, due to our preparation starting at identical frequencies, the oscillators with numbers j = a remain partially phase-locked, which is indicated by the second, negative, Lyapunov6 exponent (see Fig. 5b,d). This suggests that the information encoded in the periodic spike patterns could be transported across arbitrarily long networks, and that, from this angle, there wouldn’t emerge any difference between recurrent and feedforward networks. In our second numerical experiment, we show, however, that such a conclusion misses important aspects of the problem. In the second experiment, we start again from a chain with the oscillators initial- ized as previously, where the individual oscillators are in states of 1:1 locked periodic spiking. After perturbation, due to the negativity of the Lyapunov exponents, the differences between the ISIs of neighboring patterns, however, decay, as a function PHASE-LOCKING IN PROTOTYPICAL NETWORK TOPOLOGIES 159

a) ... b) 0.2 -1

-2

-3

-4 B(j) 0.1 log(B(j)) -5

-6

-7 0 2 4 6 8 10 1 1.5 2 2.5 3 j log(j) c) . . . d) 0.2 -2

-3

-4

B(j) 0.1 log(B(j)) -5

-6

-7

0 -8 2 4 6 8 10 2 3 4 5 6 j-50 (j-50)0.75

Figure 6. Measured bandwidth B(j) (27) of a quasiperiodic golden mean signal, injected at position j = a, into a 1/1-locked network, as a function of the position j (chain length N = 101, −4 resolution B(j)cut =5 10 ). Coupling of oscillators by means of the anti-phase sine-map· Eq. (5) at K = 0.6. Panels a),b) for the feedforward network (using a = 1) demonstrate a power-law decay of B(j), whereas panels c),d) for the recurrent network (a = 51) show an exponential decay for recurrent chains. of the topological distance d from the sender. Although local periodicities in principle continue to exist, ever increasing resolutions would be required to detect these periodicities further down the chain. To compare how fast this input sensitivity decay takes place, we implemented a quasiperiodic signal (using the most locking- a j resistant golden mean ratio T0 /T0 mod 1 = (√5 1)/2) through our 1/1-locked j − network (T0 = 1, j =1, 2, 3, ..., N, j = s, N = 101, and where the sender’s position was at a = 1 in the feedforward, and6 at a = 51 in the recurrent network). After relaxation of the system, the signal bandwidth j j TMax TMin B(j) := −j , j =1, 2, 3, ..., N, (27) T0 j was measured as a function of the position j along the chain, where ISIs Tn emitted at step n were determined with the help of the algorithm (16). The inspection of the results displayed in Fig. 6a,c clarifies that B(j), as a function of the distance to the sender, is rapidly decreasing in both cases, but according to different laws. In Fig. 6b,d, the laws for the two topologies are determined using a resolution at −4 B(j)cut = 5 10 . In the feedforward chain, the decay of B(j) can be fitted by · −α a power law B(d) = B0 d (α 2.49), see Fig. 6c. In the recurrent chain, the · ∼ β decay is best fitted by an exponential of the form B(d) = B0 exp( αd ) (here: β = 0.75 and α 1.27), see Fig. 6d. Thus, the topology expresses· − itself in the ∼ 160 STEFAN MARTIGNOLI AND RUEDI STOOP bandwidth B(j) (27) available for the propagation of the local network information at the sender. It is natural to discuss the distinct decay laws in the framework of measures of computation [20], with the result that the computations performed by recurrent networks significantly exceed those of feedforward networks.

6. Discussion. Using a discrete-time framework, we have shown that by means of a continuous change in the interaction strengths, phase-locking in feedforward topologies is continuously transformed into phase-locking in bi-directional topologies. We suggest that this finding holds true for any network size and connectivity. This, in particular, would imply that a modification of stimulating connections af- fect the response in a continuous way. Furthermore, we have shown that for binary coupled oscillators, the onset of chaos remains to be given by the critical interaction strength of the corresponding circle-maps. The developed framework was used to study phase-locking in prototypical extended networks, where we focused on the comparison of the information transport in feedforward versus recurrent network topologies. Our main finding is that different propagation laws emerge for the two topologies. If information coding is by means of Arnold tongues, this result signi- fies that the network’s underlying topology governs how information is transported through the network. With the tools developed in this contribution, topology- dependent network behavior and the network’s information-theoretic features can be investigated.

Appendix: Lyapunov exponents for arbitrary network topologies. Usual techniques are the calculation of the Lyapunov exponents of single connections or the network-averaged Lyapunov exponent λ¯, derived from a replica network with an initial small perturbation. While these approaches may yield important insight, here we demonstrate the numerical evaluation of all N 1 Lyapunov exponents for − arbitrary connectivity matrices Kij . Let the dynamical system of Eq. (14) be given. Formally, it can be written as ~ ~ j 1 2 i N a vector φn+1 = f(φn) with components φn+1 = fj (φn, φn, ..., φn, ..., φn ), where at every time step n, index i refers to the spiking oscillator. From this vector, we derive the system of variational equations [24]

y = Df(φ~ ) y , (28) n+1 n · n ~ k where (Df(φn))jk = dfj /dφn denotes the Jacobi-matrix. In general, this system is derived via [24] d yj = f (φ~ + ǫy ) . (29) n+1 dǫ j n n |ǫ=0 For the component yj , j = i, Eq. (14) therefore yields n+1 6 i j d ~ ′ j j T0 ′ j i yn+1 = fj(φn + ǫyn) ǫ=0 = (1 g (Kij , xn))yn j (1 g (Kij , xn))yn, (30) dǫ | − − T0 − i where we used Eq. (10) to simplify the notation. It remains to calculate dfi/dφn j and dfi/dφn for j = i: The oscillator i that spikes at step n does not receive a perturbation - its motion6 is along the stable limit-cycle. We have d d yi = f (φ~ + ǫy ) = (0 + ǫyi mod 1) = 1 yi , (31) n+1 dǫ i n n |ǫ=0 dǫ n · n PHASE-LOCKING IN PROTOTYPICAL NETWORK TOPOLOGIES 161

j and therefore dfi/dφn = δij . As a consequence, at every step n, the spiking oscillator is evaluated by min [T j (1 φj )] (Eq. (13)) and labeled by index i. For this j=1,2,...,N 0 − n perturbation, the Jacobi-matrix Df(φ~n) reads

dfi j = δij , dφn

dfj ′ j j = i : j = 1 g (Kij , xn), 6 dφn − i dfj T0 ′ j j = i : i = j (1 g (Kij , xn)), 6 dφn −T0 − dfj k = j, k = i : k = 0, (32) 6 6 dφn j where xn is given by Eq. (10). If there is no connection from oscillator i to oscillator j (K = 0), Eq. (32) correctly yields df /dφj = 1 and df /dφi = T i/T j for j = i. ij j n j n − 0 0 6 The nonzero elements of Df(φ~n) lie thus on the diagonal and the ith column. It is easy to see that the eigenvalues of this matrix can be read off from the diagonal. For j = i, the eigenvectors are e (the unit vector with the jth entry equal to 1). 6 j The eigenvector that corresponds to the eigenvalue µi = 1 is linearly independent of the set ej , j = i if and only if µ = 1 is not degenerate. Generally,{ } the 6 N Lyapunov exponents are calculated as follows [24]: Let N orthonormal initial vectors y0, v0, ... be given that satisfy Eq. (28). The maximal Lyapunov exponent is then defined as λI = lim 1 log y . For an evaluation 1 n→∞ n | n| of the second Lyapunov exponent, the vector vn is orthogonalized in every step ′ n with respect to yn, using the Gram-Schmidt orthogonalization method vn = hvn,yni v 2 y n |yn| n, where , denotes the scalar product of two vectors. The Lyapunov − I· h· ·i I 1 ′ exponent λ2 is then obtained as λ2 = limn→∞ n log vn . Iterative application of this procedure yields all N Lyapunov exponents. | | In the present context (Eqs. (14) and (32)), the situation is slightly different. By means of a Poincarésection, the system of N differential equations, obtained for the N coupled oscillators, can in every time step be reduced to a (N 1)-dimensional discrete dynamical system. Consequently, there are only (N 1)− Lyapunov exponents to be evaluated. This implies that the dimension of the− space spanned by the Jacobi-matrix (32) is reduced by one. Fortunately, the eigenvectors ej corresponding to the eigenvalues µj (j = i in both cases) are all orthogonal. Thus, the remaining space of dimension one,6 orthogonal to the set e , is spanned by the { j} vector ei. Therefore, the vectors yn, vn, ... have first to be orthogonalized with respect to ei: y′ = y y , e e ; n n − h n ii · i I 1 ′ λ1 = lim log yn . (33) n→∞ n | |

Intuitively, the orthogonalization of Eq. (33) with respect to ei can be understood in the sense that the Poincar´esection is taken along the direction of the oscillator i that is spiking at step n. The orthogonalization of the remaining N 2 vectors − vn, ., performed as in the general case, yields all Lyapunov exponents. 162 STEFAN MARTIGNOLI AND RUEDI STOOP

REFERENCES

[1] Ch. Huygens, “Horologium Oscillatiorium,” Muguet, Paris, 1673. [2] V. I. Arnold, “Geometrical Methods in the Theory of Ordinary Differential Equations,” Springer, New York, 1983. [3] H. G. Schuster, “Deterministic Chaos,” Physik-Verlag, Weinheim, 1984. [4] A. T. Winfree, “The Geometry of Biological Time,” Springer, New York, 1980. [5] L. Glass and M. C. Mackey, “From Clocks to Chaos. The Rhythms of Life,” Princeton Uni- versity Press, Princeton, 1988. [6] R. Stoop, K. Schindler and L. A. Bunimovich, Noise-driven neocortical interaction: a simple generation mechanism for complex neuron spiking, Acta Biotheoretica, 48 (2000), 149–171. [7] R. Stoop, K. Schindler and L. A. Bunimovich, When pyramidal neurons lock, when they respond chaotically, and when they like to synchronize, Neuroscience Research, 36 (2000), 81–91. [8] A. Argyris, G. Faust and M. Haase, “Die Erforschung des Chaos,” Friedr. Vieweg Verlag GmbH, Braunschweig/Wiesbaden, 1994. [9] A. Katok and B. Hasselblatt, “Introduction to the Modern Theory of Dynamical Systems,” Cambridge University Press, London, 1995. [10] P. Cvitanović, M. H. Jensen, L. P. Kadanoff and I. Procaccia, Renormalization, unstable manifolds, and the fractal structure of mode locking, Phys. Rev. Lett., 55 (1985), 343–346. [11] M. H. Jensen, P. Bak and T. Bohr, Transition to chaos by interaction of resonances in dissipative systems, Phys. Rev. A., 30 (1984), 1960–1969. [12] M. J. Feigenbaum, L. P. Kadanoff and S. J. Shenker, Quasiperiodicity in dissipative systems: a renormalization group analysis, Physica D., 5 (1982), 370–386. [13] A. D. Reyes and E. E. Fetz, Two modes of interspike interval shortening by brief depolariza- tions in cat neocortical neurons, J. Neurophysiol, 69 (1993), 1661–1672. [14] R. Stoop, K. Schindler and L. A. Bunimovich, Neocortical networks of pyramidal neurons: from local locking and chaos to macroscopic chaos and synchronization, Nonlinearity, 13 (2000), 1515–1529. [15] S. A Oprisan and C. C. Canavier, Phase response curve via multiple time scale analysis of limit-cycle behavior of type I and type II excitability, Biophys. J., 78 (2000), 218–230. [16] T. I. Netoff, M. I. Banks, A. D. Dorval, C. D. Acker, J. S. Haas, N. Kopell and J. A. White, Synchronization in hybrid neuronal networks of the Hippocampal formation, J. Neurophysiol, 93 (2005), 1197–1208. [17] D. D. Pervouchine, T. I. Netoff, H. G. Rotstein, J. A. White, M. O. Cunningham, M. A. Whit- tington and N. J. Kopell, Low-dimensional maps encoding dynamics in Entorhinal Cortex and Hippocampus, Neural Computation, 18 (2006), 2617–2650. [18] A. J. Preyer and R. J. Butera, Neuronal oscillators in Aplysia Californica that demonstrate weak coupling in vitro, Phys. Rev. Lett., 95 (2005). [19] R. F. Galan, G. B. Ermentrout and N. N. Urban, Efficient estimation of phase-resetting curves in real neurons and its significance for neural-network modeling, Phys. Rev. Lett., 94 (2005). [20] R. Stoop and N. Stoop, Natural computation measured as reduction of complexity, Chaos, 14 (2004), 675–679. [21] P. Goël and G. B. Ermentrout, Synchrony, stability, and firing patterns in pulse-coupled oscillators, Physica D., 163 (2002), 191–216. [22] J. T. Ariaratnam and S. H. Strogatz, Phase diagram for the Winfree model of coupled nonlinear oscillators, Phys. Rev. Lett., 86 (2001), 4278–4281. [23] S. Martignoli, “Information Transport in Neural Networks Defined by Phase Response Func- tions,” Diplomarbeit der Universität Bern, 2005. [24] R. Stoop and W.-H. Steeb, “Berechenbares Chaos in Dynamischen Systemen,” Birkhäuser Verlag, Basel, 2006. Received August 2006; revised July 2007. E-mail address: [email protected] E-mail address: [email protected]