Dynamical Architectures for Controlling Feeding in Aplysia Californica

Home , Isolated system

DYNAMICAL ARCHITECTURES FOR CONTROLLING FEEDING IN APLYSIA CALIFORNICA

by KENDRICK M. SHAW

Submitted in partial fulﬁllment of the requirements for the degree of Doctor of Philosophy

Dissertation Adviser: Dr. Hillel J. Chiel Dissertation Co-Adviser: Dr. Peter J. Thomas

Department of Biology CASE WESTERN RESERVE UNIVERSITY

January 2014 CASE WESTERN RESERVE UNIVERSITY SCHOOL OF GRADUATE STUDIES

We hereby approve the dissertation of Kendrick Matthew Shaw Candidate for the Doctor of Philosophy degree1.

Robbin E. Snyder Hillel J. Chiel Peter J. Thomas Dominique M. Durand Scott E. Cooper

October 16th, 2013

1We also certify that written approval has been obtained for any proprietary information contained within. Contents

Acknowledgements...... 9

Abstract...... 11

1 Introduction...... 13 1.1 Background and history...... 15 1.1.1 Chain reﬂexes and central pattern generators...... 15 1.1.2 Mathematical background...... 17 1.1.3 Mathematical models of neurons...... 25 1.2 The choice of Aplysia californica as a model system...... 26 1.3 Mathematical framework and central hypotheses...... 28 1.4 Outline of the remainder of the dissertation...... 31

2 Signiﬁcance of Dynamical Architecture...... 33 2.1 Introduction...... 34 2.2 Mathematical Framework...... 37 2.2.1 Limit cycles...... 38 2.2.2 Destabilization of ﬁxed points...... 39 2.2.3 Stable heteroclinic channels...... 39 2.3 Model Description...... 41

3 CONTENTS 4

2.3.1 Neural model...... 41 2.3.2 Model of the periphery and load...... 44 2.3.3 Proprioceptive input...... 52 2.3.4 Noise...... 52 2.3.5 Parameter changes used for the limit cycle simulations...... 54 2.3.6 Connection to mathematical framework...... 55 2.4 Materials and Methods...... 55 2.4.1 Intact animals...... 56 2.4.2 Suspended buccal mass preparation...... 56 2.4.3 Isolated buccal ganglion...... 57 2.4.4 Data analysis...... 57 2.4.5 Numerical methods...... 58 2.5 Results...... 59 2.5.1 Tuning the limit cycle...... 59 2.5.2 Mechanisms of adaptation to load...... 65 2.6 Discussion...... 75 2.6.1 Limitation of the model and results...... 78 2.6.2 Larger implications for pattern generators...... 80

3 Phase Resetting in a Phaseless System...... 84 3.1 Introduction...... 85 3.2 The piecewise linear iris system...... 95 3.3 Limit Cycles in the Iris System...... 102 3.3.1 Dynamics Within A Linear Region...... 103 3.3.2 Dynamics Across Regions...... 105 CONTENTS 5

3.4 Effects of a small instantaneous perturbation...... 109 3.4.1 Initial effects of a small perturbation...... 109 3.4.2 Subsequent effects of a perturbation...... 113 3.4.3 Inﬁnitesimal phase response curve...... 117 3.5 Asymptotic phase resetting behavior as a 0...... 119 → 3.6 Isochrons...... 124 3.7 Smooth System...... 125 3.8 Discussion...... 127 3.8.1 Sensitivity and control...... 129 3.8.2 Comparison to the PRC near a homoclinic bifurcation...... 131 3.8.3 Qualitative comparison with a biological model: iPRCs for the Morris–Lecar system...... 134 3.8.4 Stability of synchronous solutions for two iris systems with diffusive coupling...... 140 3.8.5 Generalization to higher dimensional systems...... 146 3.8.6 Phase resetting in the absence of an asymptotic phase...... 148

4 Conclusion and Future Directions...... 151 4.1 Review of previous chapters...... 152 4.2 Future directions...... 156 4.3 Conclusion...... 163

Bibliography...... 165 List of Tables

2.1 Model parameters...... 47 2.2 State variables...... 47 2.3 Parameters used for the limit cycle simulations...... 53

3.1 Comparison of iPRCs of homoclinic, QIF, and Iris systems...... 134

6 List of Figures

2.1 Isolated trajectories of the SHC and limit cycle...... 43 2.2 Phases of swallowing behavior in the model...... 45 2.3 Model schematic...... 46 2.4 Timing dependence of the limit cycle...... 60 2.5 Improved efﬁcacy with increased maximum muscle activation...... 62 2.6 Metabolic cost of increased muscle activation...... 63 2.7 Mechanical efﬁciency of the limit cycle...... 64 2.8 Effects of mechanical load on timing...... 67 2.9 Effects of mechanical load on trajectory...... 68 2.10 Simulation of held seaweed...... 70 2.11 Effects of held seaweed in vivo ...... 71 2.12 Simulation of reduced proprioception...... 72 2.13 Behavior of reduced preparations...... 74 2.14 Skewness of retraction duration in simulations...... 76 2.15 Skewness of retraction duration in vivo ...... 77

3.1 Smooth system phase plot...... 88 3.2 Smooth system time plot...... 90

7 LIST OF FIGURES 8

3.3 Iris system schematic...... 95 3.4 Iris system phase plots...... 96 3.5 Iris system time plots...... 101 3.6 Iris square entry to exit map...... 108 3.7 Iris bifurcation diagram...... 110 3.8 Iris iPRCs...... 120 3.9 Iris isochrons...... 126 3.10 Smooth system iPRCs...... 128 3.11 Morris–Lecar time plots...... 137 3.12 Morris–Lecar iPRC...... 139 3.13 Coupled Iris systems...... 142

4.1 PRC for the limit cycle neuromechanical model...... 157 4.2 PRC for the heteroclinic channel neuromechanical model...... 158 4.3 Model with biological noise...... 160 4.4 Trade off between proprioception and noise...... 162 4.5 Saddle bypass diagram...... 163 Acknowledgments

I would like to thank my research advisor, Dr. Hillel Chiel, for his patience, friendship, and guidance throughout my time in the laboratory. His constant encouragement and relentless striving for perfection has helped me and many others accomplish what we did not think was possible. Without him this research would not have happened, and his wisdom will shape my future work. I would also like to thank my co-advisor, Dr. Peter Thomas, for his enthusiasm and guidance in introducing me to the world of mathematical biology, and his guidance on how to bound a problem well enough that it can be solved with rigor. I would like to thank Dr. Robbin Snyder, for her friendship and advice in navigating academia, Dr. Scott Cooper, for his careful clinical tutorship and advice on blending clinical work and basic research, and Dr. Dominique Durand, whose occasional skepticism has helped drive me to strengthen this work. I would like to thank my lab mates, Hui Lu, Jeffrey McManus, Miranda Cullins, Catherine Kehl, and Jeffery Gill for their friendship and useful feedback. Miranda Cullins, Jeffrey McManus, and Hui Lu also provided the in vivo and in vitro behavior and burst onset and offset times used in chapter2, which I am grateful for. I would also like to thank Andrew Horchler for his friendship, advice, and many good conversations on the behavior of heteroclinic channels and stochastic simulations. I am also indebted to my friends Barry Rountree, Eric Herman, and Stephanie Medlock. Their enthusiasm in tackling difﬁcult problems together for the fun of it carried over into my work in the lab, and their encouragement and patience with me during difﬁcult times Acknowledgments 10

in my research helped me have faith that each obstacle could be overcome. I would like to thank Dr. Cliff Harding, Dr. George Dubyak, and the other members of the MSTP steering committee for their advice and guidance of my training of a clinician– scientist, and how to develop this work in this larger context. I would also like to thank my parents, family, and close friends in Seattle for supporting me in pursing this work. Finally I would like to thank the various sources of funding that have supported this work, including the Case MSTP (NIH T32-GM007250), NIH NS047073, and the Case Innovation Achievement Award Fellowship. Dynamical Architectures for Controlling Feeding in Aplysia californica

Abstract by KENDRICK M. SHAW

For behaviors such as swallowing, walking, and swimming, the nervous system must reliably generate sequences of motor behavior. Two competing models have been proposed for how this task is accomplished - chain reflex theory and central pattern generator theory. Chain reflex theory posits that the nervous system contains a sequence of reflexes, so that the action of one reflex creates the sensory input required to trigger the next. In contrast, central pattern generator theory posits that the nervous system is capable, in the absence of sensory input, of generating motor patterns that closely resemble the motor patterns during behavior. When modeling these behaviors with systems of differential equations, these two ideas correspond to a collection of stable nodes, in the case of the chain reflex theory, and a stable limit cycle, in the case of central pattern generator theory. Many systems can exhibit motor patterns in the absence of sensory input, violating the predictions of chain reflex theory, but those patterns are very distorted compared to in vivo behaviors, violating the assumptions of central pattern generator theory. This dissertation explores a third hypothesis, known as a stable heteroclinic channel, where a trajectory slows dramatically Abstract 12 in small regions as it passes near saddle points, creating local regions of sensitivity. We explore the implications of these dynamics by building a neuromechanical model of swallowing in Aplysia californica which can be changed from a stable heteroclinic channel to a limit cycle by changing a single parameter, and then compare the behavior within these two regimes to the behavior seen in vivo. We find that the stable heteroclinic channel provides a better match for what is seen in vivo, due to its timing sensitivity. We then analytically explore the basis for this sensitivity by studying a tractable heteroclinic channel and deriving a closed-form expression for its infinitesimal phase response curve. We show that the qualitative behavior of the tractable model is present in more complex models, including the Morris-Leccar neuron model as it approaches the homoclinic bifurcation. We then discuss the implications of this research for future work in motor pattern generation. Chapter 1

Introduction

13 CHAPTER 1. INTRODUCTION 14

To generate many forms of adaptive behavior, the nervous system needs to reliably generate sequences of motor activity. For example, as we walk, the nervous system must reliably sequence swing and stance behavior for each leg or risk stumbling (Winter, 1991). Similarly, a failure to generate the proper sequence of motor activity in swallowing, for example, failure to close the larynx before peristalsis carries the food bolus past, can lead to aspiration of food and serious sequellae such as aspiration pneumonia (Denaro et al., 2013). Indeed, aspiration pneumonia is a leading cause of death after a stroke (Henon´ et al., 1995). Generating the correct sequence of activity is often not enough, however; the system must also be sensitive enough to adjust the duration of components of the pattern in response to incoming sensory information. If a cat’s leg encounters an obstacle during its swing, the nervous system will increase the ﬂexion of the leg and extend the length of the swing phase to lift the foot over the obstacle, especially if contact occurs late in the swing phase (Forssberg, 1979). Similarly, the animal may extend the touch-down phase of the walking cycle if the animal steps into a hole, allowing more time for the foot to reach the ground (Halbertsma, 1983; Hiebert and Pearson, 1999). In human swallowing, the oral and pharyngeal stages of swallowing nearly double in duration when the bolus consistency is changed from liquid to semisolid (McHorney et al., 2006). It is not clear how the nervous system achieves this balance between robustness and sensitivity. Changes that make the system more sensitive to incoming sensory information often also allow irrelevant sensory information and noise to corrupt the pattern, and changes that make the pattern more robust to irrelevant sensory inputs and noise often make it less sensitive to relevant sensory information. The challenge of balancing these seemingly contradictory goals will be a recurring theme in this dissertation. CHAPTER 1. INTRODUCTION 15

1.1 Background and history

1.1.1 Chain reﬂexes and central pattern generators

Much of the early research in motor control focused on the question of sensitivity, specifically the ability of the nervous system to adapt the nature and timing of behavior to incoming sensory stimuli. Loeb(1899) suggested that feeding, reproductive, and other behaviors in the frog and other “lower” animals could be thought of as examples of a “kettenreflex”, or chain reflex, where the behavior was composed of very simple reflexes such that the action of one reflex triggered the next reflex. For example, the sight of a fly in the periphery might cause the animal to turn its head towards the fly, the sight of the fly in front of the head might trigger an opening of the jaws and extension of the tongue, and so on throughout the entire behavior. Sherrington later applied the chain reflex theory to “higher” vertebrates in his studies of walking in cats and dogs after a spinal transection (Sherrington, 1910, 1913). As support for this theory, he noted that lifting and dropping the animal’s hind paw was sufficient to trigger a series of stepping movements, and catching the paw part way through the walking cycle would immediately stop the walking cycle. Although often viewed as a champion of chain reflex theory, however, even Sherrington was quick to note its limitations, noting that electrical stimulation of the spinal cord would still sometimes trigger a stepping motion in deafferented limbs and isolated muscles, and thus conceded that this principle of chain reflexes “is of itself not the sole rhythm-producing factor in the reflex,” (Sherrington, 1910). Chain reflex theory enjoyed many proponents throughout the early part of the 20th century, although some still argued for the existence of intrinsic oscillators in the nervous system generating the timing of behavior (Marder and Bucher, 2001). The consensus CHAPTER 1. INTRODUCTION 16

of the scientific community began to shift in the 1960’s, however, to theories involving central oscillators, due to accumulating examples of systems that could generate patterns of activity in the absence of sensory input that resembled the motor activity seen in the intact animal. One of the early examples of such a system was flight in the locust (Wilson, 1961). Wilson was able to selectively cut all nerves containing sensory afferents from the wings, yet he was still able to produce flight-like activity in the wing muscles when air was blown on the head of the animal. In particular, many began to argue not only for intrinsic neural oscillators playing a role in oscillatory behavior, but that those oscillators could produce physiologic motor patterns in the absence of sensory input carrying timing information. Such oscillators were given the name central pattern generators (CPGs) (Marder and Bucher, 2001). In its simplest form, this theory would posit that the central nervous system can generate an “ideal” motor pattern in the absence of sensory input, and that sensory input merely tunes this pattern in a mild way. In this pure form, the central pattern generator provides tremendous robustness, in that it can produce useful patterns under a large range of sensory inputs, and even do so when sensory input is entirely lost. While the patterns observed in vitro did share much of the structure and sequence of in vivo motor patterns they were not the near copies of in vivo activity that the idealized central pattern generator theory would predict. Instead, the in vitro patterns were often distorted and slow compared to the patterns seen in vivo; for example in Wallen´ and Williams(1984) the swimming frequencies observed in an in vitro lamprey preparation were about half of those observed in vivo, and in Chrachri and Clarac(1990) the backward walking frequencies in a reduced crayfish preparation were 10 to 30 times slower than those seen in vivo. These differences in in vitro and in vivo patterns led even some of the early proponents of central pattern generators to sound a note of caution. Robertson CHAPTER 1. INTRODUCTION 17

and Pearson, two of the early researchers of the nervous system controlling flight in the locust, noted “Although now abundantly clear that a central rhythm generator can produce powerful oscillations in the activity of flight motor neurons and interneurons, it is equally clear that the properties of this central oscillator cannot fully account for the normal flight pattern” (Selverston, 1985). Thus the nervous system seems to have neither the dependence on sensory input for rhythmic behavior predicted by the chain reflex theory nor the ability to generate physiological patterns in the absence of sensory input predicted by the idealized central pattern generator theory. What lies between these two extremes? Before exploring this question, we will first introduce a mathematical framework within which we can represent and compare different classes of motor control models.

1.1.2 Mathematical background

In this section we will define what we mean by a dynamical system, and then provide definitions of some structures that a dynamical system may have that we will use later in the paper. We also briefly discuss stochastic dynamical systems.

1.1.2.1 Dynamical systems

A dynamical system is a formalization of the idea of a deterministic process. Formally, it

can be deﬁned as a triple (T,X,φ(x,t)) where T is a number set representing time, X is a state space, and φ : X T X is an evolution operator with the properties φ(x,0) = x and × → φ(x,t +s) = φ(φ(x,t),s) for all x X and t,s T (Kuznetsov, 2004). This can be thought ∈ ∈ of as a set of rules, φ, for calculating the new state of the system φ(x,t) based on an initial state of the system x X and an amount of time that has elapsed t T. The two properties ∈ ∈ of φ capture the intuitions that the system should require a non-zero amount of time to CHAPTER 1. INTRODUCTION 18

evolve, and that the effects of advancing the system by a ﬁxed amount of time should be the same whether the system is advanced in a single step or multiple steps. Although this second rule may at ﬁrst seem to exclude systems that depend on an absolute time, such as a system forced by an external input, these “non-autonomous” systems can be represented by augmenting the state with a new variable t T containing the absolute time that evolves ∈ at the appropriate rate, i.e. φ((...,t),s) = (...,t + s) for all t,s T. ∈ For this dissertation, we will mostly restrict our discussion to the subset of dynamical

systems where time is a real number (i.e. T = R), the state of the system can be represented n + as a vector of real numbers (i.e. X = R ,n Z ), and the evolution operator φ(x0,t) ∈ represents integration of the ordinary differential equation

dx = f (x), (1.1) dt

with the initial condition x0 over a time t, where f : X X. We will also assume we have → chosen a suitable distance metric on X (for example the L2, or Euclidean, norm).

1.1.2.2 Orbits, limit cycles, and ﬁxed points

In the context of dynamical systems, an orbit is, intuitively, the path traced out by a point as it evolves forward and backward through time. More formally, it is a set of points A X ∈ such that for any two points x0,x1 A, there exists a time t T such that x1 = φ(x0,t). ∈ ∈ Note that this deﬁnition implicitly provides a unique mapping from every point x X to ∈ an orbit containing x. Note also that this deﬁnition is different than the colloquial use of the word “orbit”, in that an orbit need not form a closed loop. The colloquial use of the word orbit maps a bit more closely onto a different structure, known as a limit cycle. A limit cycle is an isolated periodic orbit. Here periodic is taken in its normal sense, CHAPTER 1. INTRODUCTION 19

namely that there exists a period τ > 0,τ T such that for any point x in the limit cycle ∈ x = φ(x,τ), and τ is the smallest positive value for which this equality holds1. By isolated, we mean that there is some (possibly small) open neighborhood around the orbit that does not contain other periodic orbits; this restriction is present to exclude, for example, the inﬁnite set of concentric circles in a uniformly rotating disc. We can state this restriction

more precisely by saying that there exists a distance ε > 0 such that for every point x in

the limit cycle, for all points x′ within a distance ε of x that are not part of the limit cycle,

the orbit containing x′ is not a periodic orbit. A ﬁxed point is a state of the system that does not change over time. It can be equivalently deﬁned either as an orbit containing a single point, as a point x such that

φ(x,t) = x for all t T, or as a point x such that f (x) = 0 (where f is the function used in ∈ (1.1)). Note that although formally a point is not the same as a set containing a single point, in practice these deﬁnitions of a ﬁxed point are used interchangeably with the obvious conversion between a point and a set containing only that point implied.

1.1.2.3 Saddle points, heteroclinic cycles, and heteroclinic channels

There are many different types of ﬁxed points, but we will be particularly interested in a type of ﬁxed point known as a standard hyperbolic saddle point. Assume that f is

differentiable in an open neighborhood around a fixed point. That is, if f (x) = 0, then d f /dx is a well defined continuous function. The fixed point is then a standard hyperbolic saddle iff the set of eigenvalues of the Jacobian of f evaluated at this fixed point contains one or more eigenvalues with a positive real component, one or more eigenvalues with a negative real component, and does not contain any eigenvalues whose real component is zero. Intuitively, this means that the point will attract trajectories along some directions

1The equality will obviously hold for integer multiples of τ due to the previously mentioned constraints on φ. CHAPTER 1. INTRODUCTION 20

and repel them along other directions. There will be a set of points Γ−, known as the stable

manifold, that asymptotically approach the ﬁxed point over time, i.e., if the saddle is xs,

then the stable manifold is the set x X limt ∞ φ(x,t) = xs . There will also be a set { ∈ | → } of points that asymptotically approach the saddle backwards in time, Γ+, known as the

unstable manifold, i.e. the set x X limt ∞ φ(x,t) = xs . { ∈ | →− } + Almost all of the orbits near the saddle will not be contained in either Γ or Γ−, + however. Generically, Γ− and Γ will be curves or (hyper) surfaces and thus have zero “volume” (more formally zero measure); thus almost all of the orbits near the saddle simply pass by the saddle rather than originating or terminating at the saddle. These paths that pass near the saddle will be of interest to us in later sections. If we assume that f is smooth

(inﬁnitely differentiable) within a neighborhood of the saddle xs, then the magnitude of

f will become arbitrarily small in a neighborhood around xs. Intuitively, this means that almost all of the trajectories near the saddle will slow down as they approach the saddle, travel slowly through the neighborhood of the saddle, and accelerate as they leave the saddle. In some systems, an orbit may be contained in both the unstable manifold of one saddle and the stable manifold of a second saddle, such that it intuitively originates at the ﬁrst saddle and terminates at the second. Such an orbit is known as a heteroclinic orbit. If there

is a sequence of saddle points xs,0,xs,1,...,xs,n and a corresponding set of heteroclinic orbits connecting them into a cycle such that the unstable manifold of one node intersects the stable manifold of the next node in the sequence (with the ﬁrst node following the last), we refer to the union of the saddles and their connecting orbits as a heteroclinic cycle. For simplicity, we will restrict our attention to the case where each of the saddles has a 1-dimensional unstable manifold. A heteroclinic cycle is structurally unstable, meaning that an arbitrarily small change to CHAPTER 1. INTRODUCTION 21

the vector ﬁeld described by f can cause the unstable manifold of one saddle to “miss” the stable manifold of the next saddle. Because of this brittle dependence on the structure of f , in the absence of certain symmetries (Guckenheimer and Holmes, 1988) one is unlikely to see heteroclinic cycles in real-world systems. When a heteroclinic cycle is destroyed, however, a “ghost” of sorts remains; for sufﬁciently small perturbations when f is smooth, there will still be trajectories that pass by each saddle in sequence (Afraimovich et al., 2004a,b). Furthermore, certain categories of perturbations will produce a limit cycle which passes near each saddle. An example of such a category is perturbations that cause the unstable manifold of each saddle to pass “inside” the stable manifold of the next in a planar system (Reyn, 1980). A family of limit cycles such as this is structurally stable, and we will refer to these limit cycles and their associated heteroclinic cycle as a stable heteroclinic channel (Rabinovich et al., 2008b; Afraimovich et al., 2011) 2. To be more precise, we

need to restrict the set of limit cycles to those that pass within some minimum distance ε of each of the saddles; in this case we could say that a heteroclinic channel exists within

the neighborhood ε of the saddles. As with many terms like “short” and “tall”, however, it is often convenient to use these terms in a qualitative sense without defining artificially sharp boundaries. Thus for most of this dissertation, we will use intuitive definition of “close” based on the slow passage near the saddle rather than formally choosing a specific

distance for ε; a corresponding value of ε could easily be found by taking the maximum of the closest distance between the limit cycle and each of the saddles.

1.1.2.4 Attractors and stability

In a dynamical systems context, and invariant set is a set of points where the entire set does not change when evolved through time (even though the individual points might). More

2More generally, a stable heteroclinic channel may also refer to a connected chain of saddles that create a non-cyclic channel. We will only consider the cyclic case in this dissertation, however. CHAPTER 1. INTRODUCTION 22

formally, a set of points A X is an invariant set if and only if φ(a,t) a A,t T = A. ⊂ { | ∈ ∈ } Fixed points, limit cycles, and heteroclinic cycles are all examples of invariant sets. Because ﬁxed points, limit cycles, and heteroclinic cycles are points and curves of zero volume (measure) in space, it is almost certain that an experimental preparation will not begin with initial conditions that happen to lie on a particular example of these invariant sets. Thus it is important to consider the behavior of orbits that lie near these structures, because we are much more likely to see these nearby orbits when observing the system than the ﬁxed point, limit cycle, or heteroclinic cycle itself. In many of these cases, the nearby orbits will approach the invariant set in question and begin to behave like the elements of this set.

We first consider a fixed point, then a limit cycle. If there is some ε > 0 neighborhood B of a fixed point x such that all points in the neighborhood B asymptotically approach

x, i.e. limt ∞ φ(b,t) b B = x , then we consider x to be a stable ﬁxed point. The { → | ∈ } { } same approach can be used to deﬁne a stable limit cycle γ, i.e. there exists an ε > 0

neighborhood B of the limit cycle γ such that limt ∞ φ(b,t) b B = γ. { → | ∈ } For a heteroclinic cycle Γ, the definition of stability becomes a bit more delicate. In the planar case, for example, it is clear that any point on the “outside” of the cycle will follow a trajectory leading away from heteroclinic cycle, so in the generic case every neighborhood around the cycle includes points that do not approach the cycle. In some cases, however, almost all of the points on the inside of the cycle may approach the heteroclinic cycle, so the concept of stability still seems useful. To work around this difficulty, we replace the neighborhood B with a set C of finite volume (measure) such

that Γ C and limt ∞ φ(c,t) c C = Γ. If such a set C exists, we refer to Γ as a stable ⊂ { → | ∈ } heteroclinic cycle. We next consider a sufﬁcient condition for the stability of a heteroclinic cycle. A CHAPTER 1. INTRODUCTION 23

hyperbolic saddle with a one dimensional manifold is called dissipative if the minimum of the absolute value of the stable eigenvalues is greater than the unstable eigenvalue (Afraimovich et al., 2011). Intuitively, this means that the saddle compresses incoming trajectories more than it stretches them. If all of the saddles in the heteroclinic cycle are dissipative, the heteroclinic cycle is a stable heteroclinic cycle (Afraimovich et al., 2011), and we will call the associated heteroclinic channel a stable heteroclinic channel.

1.1.2.5 Stochastic systems

In a few cases, we simulate a stochastic system with additive noise. We can represent such a system by combining a probability triple that captures the likelihood of various outcomes with a dynamical system whose state encodes one set of outcomes.

A probability triple (Ω,F ,P) is composed of a set of all possible outcomes Ω, a σ- algebra3 of Ω representing, intuitively, all “reasonable” collections of possible outcomes,

and a function P : F [0,1] representing the probability of observing a given set of → outcomes that obeys basic intuitions of probability4. We will use a version of n-dimensional Brownian motion (also known as a Wiener process) to drive the “noise” of our system. An n-dimensional version of Brownian motion

is a stochastic process (i.e. a family of functions parameterized by ω Ω) where the ∈ change in position from time t to a later time s is independent of any previous changes in position and follows an n-dimensional Gaussian distribution with a mean of 0 and covariance of t s I (where I is the n-dimensional identity matrix). More formally, | − | if we let p be the probability density function of a zero-mean n-dimensional Gaussian

3 i.e. a set F that contains the empty set (i.e. ∅ F ), the complement of each of its elements (i.e. C ∈ S∞ A = (Ω A) F for all A F ), and any finite or countably infinite union of its elements (i.e. Ai F ∖ ∈ ∈ i=0 ∈ for all A0,A1,... F ). ∈ 4i.e. P(∅) = 0, P(Ω) = 1, and the probability of a set of mutually exclusive outcomes is equal to the S∞ sum of their probabilities, i.e. for any countably infinite family of disjoint sets A0,A1,... F , P( i=0 Ai) = ∞ ∈ ∑i=0 P(Ai) CHAPTER 1. INTRODUCTION 24

distribution parameterized by a scalar variance σ 2,

1 x 2 p(x,σ 2) = exp ‖ ‖ , (1.2) (2πσ 2)n/2 − 2σ 2

then a stochastic process Wt is an n-dimensional version of Brownian motion if the

probability of the process passing through a set of intervals F1,...,Fn F at times ∈ t1,...,tn T, t1 < t2 < < tn is ∈ ···

Z P(Wt1 F1,...,Wtn Fn) = p(x1,t1) p(xn xn 1,tn tn 1) dx1 dxn. (1.3) ∈ ∈ F1 Fn ··· − − − − ··· ×···×

We combine a probability triple (Ω,F ,P) with a dynamical system (T,X,φ) to form n a stochastic dynamical system, where the state (u,ω,t) X = R Ω T contains both ∈ × × which outcomes ω Ω that will occur and the absolute time t T. Because the outcomes ∈ ∈ that will occur do not change with time and absolute time should advance in a sensible

manner, we also require φ((...,ω,t),s) = (...,ω,t + s). The evolution of u is then driven by the stochastic differential equation

du = f (u) dt + η dWt(ω), (1.4)

where η is a diagonal matrix representing the magnitude of the noise and Wt(ω) is a version of n-dimensional Brownian motion. The formal deﬁnition of a stochastic differential equation is somewhat involved; for brevity, we will limit ourselves to the case of additive

noise and “well behaved” functions f (u) (i.e. f is Lipshitz continuous within the region of integration5). Under these conditions, the solution to the stochastic differential equation

n 5meaning that there exists a ﬁnite constant K such that for all u,v R within the region of integration, f (u) f (v) < K u v ∈ ‖ − ‖ ‖ − ‖ CHAPTER 1. INTRODUCTION 25

(1.4) can be thought of as the result of taking many small Euler steps and adding noise at

each step. More formally, if we deﬁne Euler integration from the initial condition uN,0 over

the time interval (0,t), t T, in N steps as the ﬁnal term uN N of the recurrence relation ∈ ,

uN,n = uN,n 1 + f (uN,n 1)(t/N) + η(Wnt/N(ω) W(n 1)t/N(ω)), (1.5) − − − −

then the solution u(t) of (1.4) is the (mean square) limit of uN,N as the number of steps N taken between 0 and t goes to inﬁnity (Kloeden and Platen, 1992). For a more formal and general deﬁnition of stochastic differential equations, we refer the interested reader to Øksendal(2003).

1.1.3 Mathematical models of neurons

The membrane potential of a neuron can be modeled as an electrical circuit containing,

in parallel, a capacitor Cm representing the charge on the membrane and several ion

channels, each represented as a conductance, gi, in series with a battery, Ei, representing the electrochemical potential of the ions to which the channel is permeable. In general,

a conductance gi may be dependent on the membrane potential and one or more gating variables ni, j. Under these assumptions, the change in the membrane potential over time can be described by dV = I + ∑gi(V,ni, )(Ei V), (1.6) dt i · − where I represents currents in the cell from extrinsic sources, such as synaptic currents or the injection of a current by an experimenter. The dynamics of the gating variables, in turn,

can be described by a membrane-potential-dependent steady state value, ni, j,∞(V), which is approached at a rate determined by a membrane-potential-dependent time constant, CHAPTER 1. INTRODUCTION 26

τi, j(V). The change in the gating variables can then be described by

dni j ni j ∞(V) ni j , = , , − , . (1.7) dt τi, j(V)

The dynamics of this system of equations describe the behavior of the neuron; for example, tonic firing would appear as a simple limit cycle, tonic bursting as a limit cycle containing a coiled region with many spikes connected to a less convoluted region for the inter burst interval, and a stable fixed point would represent a quiescent neuron (we refer the interested reader to Izhikevich(2010) for an in-depth exploration of the dynamics of this family of neural models). In many cases, it is useful to represent the firing rate of the neuron, meaning the number of spikes per unit time, rather than representing the membrane potential directly. A neuron in such a firing rate model can often be represented by the one-dimensional differential equation da = F(a,I). (1.8) dt

There are multiple methods that can be used to make this transformation; for a review we would refer the interested reader to Ermentrout and Terman(2010a). Firing rate models are often used as a lower dimensional representation of the neurons in a neural circuit, and we will use these for the model in chapter2.

1.2 The choice of Aplysia californica as a model system

To ground our exploration of possible architectures in biological pattern generators, it is important that we choose one or more biological systems with structured behavior to compare our theories and models against. Ideally, such a model organism would have CHAPTER 1. INTRODUCTION 27

extensive existing research available to draw from both on the biomechanics of the behavior and on the neural circuit driving the behavior. The behavior in question should be readily observable, and should also have well defined and physiologically meaningful ways of perturbing the behavior so that the response to the perturbation can be studied. Furthermore, experimental techniques should be available for recording behavior in the intact animal and the corresponding fictive behavior in the isolated nervous system. It should be possible to manipulate individual neurons within the nervous system, and ideally the number of neurons that need to be manipulated to influence behavior should be small. For all of these reasons, we have chosen to use swallowing in the marine mollusk Aplysia californica as the model system whose dynamics we will study. We will briefly describe the behavior, then explain how this behavior meets the criteria above. The feeding apparatus in Aplysia consumes food by protracting a grasper, known as the radula-odontophore, through the jaws, closing the grasper on the food, generally seaweed such as Gracilaria spp., and then withdrawing the grasper with the food into the buccal cavity and towards the esophagus. Ingestive behaviors are divided into bites, where the animal attempts to grasp food and fails, bite-swallows, where the animal does not initially have the food in its mouth but successfully grabs the food and pulls it partially into the buccal cavity, swallows, where the food starts in the buccal cavity and the animal pulls it further in, and rejections, where the animal pushes food out of the buccal cavity by closing on the food and protracting the grasper (Morton and Chiel, 1993a). Of these behaviors, we have chosen to focus on swallows because they are easy to elicit from the animal and have a mechanical component (force on the seaweed) that is comparatively easy to manipulate. Swallowing in Aplysia meets all of the criteria we outlined for an ideal model system above. There is an extensive experimental literature on the feeding system of Aplysia for us to build on, including mapping of relevant motor pools (Church and Lloyd, 1994), CHAPTER 1. INTRODUCTION 28

biomechanics (Sutton et al., 2004a,b; Novakovic et al., 2006), sensory inputs and integration (Chiel et al., 1986; Proekt et al., 2007) neural plasticity (Schwarz and Susswein, 1986; Katzoff et al., 2006), and key neurons involved in pattern generation (Hurwitz et al., 1996; Hurwitz and Susswein, 1996; Susswein et al., 2002; Hurwitz et al., 2003; Nargeot et al., 2002). The feeding behavior is easily visible from outside the animal (Morton and Chiel, 1993a) (unlike, for example, the gastric mill in the crustacean stomatogastric ganglion), and changing forces on the seaweed during swallowing provides a behaviorally relevant perturbation (unlike biting, where the animal does not grasp the food). Techniques exist for recording from the key nerves controlling feeding in vivo (Cullins and Chiel, 2010), in reduced preparations (McManus et al., 2012), and in the isolated nervous system (Lu et al., 2013), which is currently not possible in systems such as C. elegans. The nervous system has comparatively few cells (an estimated 20,000 neurons (Kandel, 2000), compared to an estimated 75,000,000 neurons in a mouse (Williams, 2000)) and small neural pools, so that electrical manipulation of a single cell can cause signiﬁcant muscle activation (Church and Lloyd, 1994) or changes in timing (Hurwitz and Susswein, 1996; Zhurov and Brezina, 2006). In addition, the somata of many of the motor and interneurons are comparatively large, electrically compact with the synaptic region (unlike most arthropod neurons), occur in stereotyped locations within the ganglia, and can often be identiﬁed by their location and electrophysiological “personalities” (Kandel, 1979), making them tractable to experimental measurement and manipulation of individual cells (McManus et al., 2012; Lu et al., 2013).

1.3 Mathematical framework and central hypotheses

We now have reviewed the background that we need to discuss the central hypothesis. Consider the motor behavior of swallowing in Aplysia californica. If we make the (weak) CHAPTER 1. INTRODUCTION 29

assumption that the dynamics of the behavior can be described in a deterministic manner, we can describe the behavior in the context of dynamical systems theory. We assume that the state of the system can be decomposed so that the state is the concatenation of a vector, a, containing the state of the components of the central nervous system, and a second vector, x, containing the state of the periphery; this is reasonable to the extent that we can clearly delineate the boundary between the central nervous system and the periphery. The differential equations for the behavior in the intact animal can then be written in the form

da = f (a) + g(a,x), (1.9) dt dx = h(a,x). (1.10) dt

Here f represents the dynamics of the central nervous system that do not depend on sensory input from the periphery, g represents all of the dynamics of the central nervous system that do depend on sensory input from the periphery, and h contains the dynamics of the periphery including the motor system and its response to activation coming from the central nervous circuit. Note that the additive model of (1.9) arises naturally when, for example, the state a contains the membrane potential of cells and the sensory input takes the form of synaptic currents, as in (1.6). We further make the assumption that, in a reduced preparation, the dynamics of the nervous system can be described as

da = f (a). (1.11) dt

Unlike equations (1.9- 1.10), this is a somewhat strong assumption that in the reduced preparation the nervous system behaves in the way it would in the intact animal if sensory input were removed. We add to this the assumption that dynamics producing the sponta- CHAPTER 1. INTRODUCTION 30

neous activity in the reduced preparation (1.11) are related to those producing the behavior in the intact animal. While this assumption is a source of controversy in experimental neurobiology (Marder and Bucher, 2001), this is the common assumption behind the utility of studying reduced preparations that produce motor output that resembles that seen in vivo and is thus not unreasonable to make. The standard assumption when modeling motor pattern generators is that the isolated nervous system contains one or more intrinsic oscillators driving the pattern (Kopell, 1986), which is generally represented as the isolated dynamics (1.11) having a stable limit cycle. In this dissertation, we will explore the following hypothesis: The dynamics of the pattern generator controlling swallowing in Aplysia californica are better described by a model where the intrinsic neural dynamics6 contain a stable heteroclinic channel than one where the intrinsic dynamics contain a more homogeneous limit cycle.

To test this hypothesis, we will use a number of models of swallowing behavior and intrinsic neural dynamics and compare them with the observed behavior of the animal. We have attempted to use models that are as simple as possible while still reproducing the qualitative aspects of the behavior we wish to examine. Although this type of model can not offer the quantitative predictions that a more detailed model might, there are a number of reasons to favor these less complex models during the early stages of a research program. Striving for simplicity forces one to examine the necessity of each component of the model, and it can be very instructive when a seemingly minor component of the model turns out to critical for the behavior. Simpler models are also generally more tractable to analysis, which we will take advantage of in chapter3, and the phenomena seen are more likely to generalize to other systems. Once the phenomena in a simple model are understood,

6Or, alternatively, the slow subsystem in an appropriate fast slow decomposition of the neural dynamics (e.g. Butera et al.(1996); Nowotny and Rabinovich(2007)) CHAPTER 1. INTRODUCTION 31

additional complexity can always be added. Such an approach has been successful both in our own lab (where simple kinematic models such as Drushel et al.(2002) have led to the development of more quantitative models such as Novakovic et al.(2006)) and in others (e.g. Leon Glass, whose studies of very simple phase oscillator models of the heart, e.g. Guevara and Glass(1982) have provided insights into phenomena seen in vivo, such as entrainment of reentrant rhythms (Glass et al., 2002)).

1.4 Outline of the remainder of the dissertation

In chapter2, in order to ﬁnd differences between the behavior produced by neural dynamics containing a stable heteroclinic channel and neural dynamics containing a limit cycle, we develop a neuromechanical model of swallowing in Aplysia californica, such that the dynamics of the isolated nervous system can be changed between a stable heteroclinic channel and a similar limit cycle with a simple change in parameters. We then examine the behavior of this model with both parameter sets when subjected to load, and we compare the responses of the model to those seen in the animal, both in in vivo behavior and in ﬁctive patterns in isolated preparations. We hypothesized that the behavior of the animal would more closely match the behavior of the model in the heteroclinic channel regime than in the limit cycle regime. In the three qualitative differences between the models we examined, we found that the comparable data from the animal were much more similar to the heteroclinic channel’s behavior and thus consistent with this hypothesis. In chapter3, we hypothesized that the differences seen in the previous chapter were the result of regions of localized sensitivity. We thus set out to characterize the sensitivity of the system and develop quantitative tools that could be used to help distinguish a pattern generator built on a limit cycle from one built on a heteroclinic channel. We explore the question of sensitivity in heteroclinic systems by constructing an analytically tractable CHAPTER 1. INTRODUCTION 32

planar heteroclinic system. We then show that the tools of phase response curve analysis, which at ﬁrst might not seem suitable for the study of a non-periodic structure such as a stable heteroclinic cycle, can be applied by taking appropriate limits as a stable limit cycle bifurcates into a heteroclinic cycle. Consistent with our hypothesis, the phase response curve shows small regions where the system is much more sensitive than it is outside of these regions. This analysis shows, somewhat surprisingly, that the point of maximal sensitivity is not near the saddle itself, but instead lies on the approach to the saddle where the orbits start to diverge. We show that the pattern of sensitivity in the tractable planar model appears to generalize to other homoclinic and heteroclinic systems, such as a Morris–Lecar neuron in the homoclinic limit. In the ﬁnal chapter, we review the results and discuss some potential future directions for research in this area. Chapter 2

The Signiﬁcance of Dynamical Architecture for Adaptive Responses to Mechanical Loads During Rhythmic Behavior

33 CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 34

2.1 Introduction

Motor behaviors, such as cat running, crayfish swimming, and dog lapping all require the nervous system to reliably generate a sequence of motor outputs. To be efficient, however, a fixed sequence of activity is not enough: a cat that fails to step over an obstacle may lose its footing and fall (Forssberg et al., 1975; Forssberg, 1979) and a crayfish that wanders into a current of cold water must control muscles that may suddenly have become stronger but relax more slowly (Harri and Florey, 1977). Sensory feedback plays a key role in allowing an animal to adapt its behavioral pattern to the circumstances in which it finds itself. The way that this sensory information is integrated into pattern generation to produce adaptive behavior, however, can be difficult to ascertain. As discussed in chapter1, two competing theories, chain reflexes and central pattern generators, have been proposed, with central pattern generators enjoying the greater support in the current literature. It should be noted, however, that patterns generated by the isolated nervous system often are very distorted compared to those seen in vivo. In particular, phases of the motor pattern are often significantly longer than those observed in the intact animal. This has led many investigators to question the descriptive power of central pattern generator theory. In the words of Robertson and Pearson, “Although now abundantly clear that a central rhythm generator can produce powerful oscillations in the activity of flight motor neurons and interneurons, it is equally clear that the properties of this central oscillator cannot fully account for the normal flight pattern” (Selverston, 1985). There is some evidence that slowing of isolated neural patterns may be due to the absence of sensory feedback. In Pearson et al.(1983), cycle-by-cycle stimulation of the appropriate sensory afferents was able to restore wing-beat frequency in fictive flight in the locust. Restoration of the normal pattern by sensory input suggests that biological pattern CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 35

generators may occupy a middle ground between pure central pattern generators and chain reflexes. In some cases, endogenous neural input may control where a systems lies on this continuum. AsB assler¨ (1986) noted when considering a relaxation-oscillator like model of a central pattern generator “Hence, one and the same system can behave either like a CPG or like a chain reflex, depending only on the amount of endogenous input.” These investigators thus warned about the dangers of inferring the mechanism used by a pattern generator in vivo based only on the behavior of a pattern generator in vitro. Despite these hesitations, the empirical data supporting central pattern generator hypothesis led to a focus on providing a mathematical formulation for this theory using the qualitative analysis of dynamical systems, which was becoming an important tool for theoretical neuroscience by the middle of the twentieth century. The behavior of an ideal central pattern generator naturally corresponds to a system of nonlinear ordinary differential equations whose solutions contain stable limit cycle (an attracting isolated periodic orbit). As a result, this structure has played a central role in the mathematical description of central pattern generators (Ijspeert, 2008). In contrast, there have been fewer attempts to model chain reflexes with systems of differential equations. Instead, much of the work modeling these types of sensory- dependent systems use different tools, such as finite state machines (Lewinger et al., 2006). While these models can capture individual phases of the behavior well, they generally do not describe the transitions between the phases, which may be important in understanding some forms of behavior. In contrast, one could view the state of a chain reflex system in terms of a series of stable fixed points. In each phase of the motion, the trajectory would be captured by one of the fixed points until the appropriate (external) sensory input pushed the system out of the neighborhood attracted to that fixed point and into the basin of attraction of the next. CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 36

Between these two extremes of models of central pattern generators and chain reflexes, one may consider systems in which the progress of a periodic orbit is slowed, but not stopped, by passage near one or more fixed points. This behavior arises naturally in a structure known as a “stable heteroclinic channel” (Rabinovich et al., 2008b), where multiple saddle points (fixed points that attract in some directions while repelling in others) are connected in a cycle, so that the unstable manifold of each saddle point brings the system near the stable manifold of the next fixed point. This structure has been used to describe motor behavior such a predatory swimming behavior in Clione (Levi et al., 2004; Varona et al., 2004). To our knowledge, however, these models of pattern generation have not been directly compared to those built with a more “pure” limit cycle that does not pass near fixed points. A potential advantage of a dynamical system that allows trajectories to move close to equilibrium points is that it may spend longer or shorter times in that vicinity, rather than proceeding through the cycle with a relatively fixed phase velocity. In turn, this could allow an animal greater flexibility in responding to unexpected changes in the environment, such as increases or decreases in mechanical load as it attempts to manipulate an object. To examine this range of dynamics, we have created a neuromechanical model based on the feeding apparatus of the marine mollusk Aplysia californica. We examine the model in two parameter regimes which produce similar output under small loads. In the first parameter regime, the isolated neural dynamics form a homogeneous limit cycle, as would be expected for an idealized central pattern generator. In the second parameter regime, the isolated neural dynamics form a stable heteroclinic cycle, moving it closer along the continuum to a chain reflex. We show that in this second regime, the behavior of the model falls between that of an idealized chain reflex and an idealized central pattern generator. We then compare the behavior of the two models to the observed behavior of the animal, CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 37 and show that several of the features of the animal’s behavior are better described by the model with the stable heteroclinic channel than the model with the limit cycle. At the end of the chapter, we reflect on possible general principles suggested by this work.

2.2 Mathematical Framework

In this section, we describe a general mathematical framework we will use for modeling the behavior of a motor pattern generator. We model a central pattern generator receiving sensory input from the body as a system of differential equations specifying the evolution

n m of a vector of n neural state variables, a R , and a vector of m state variables, x R , ∈ ∈ representing the mechanics and periphery (e.g. muscle activation). We assume that an applied load interacts only with the mechanical state variables, so that the differential equations can be naturally written in the following form:

da = f (a, µ) + εg(a,x), (2.1) dt dx = h(a,x) + κl(x). (2.2) dt

Here µ is a vector of parameters which can encode states such as arousal of the animal, f (a, µ) represents the intrinsic dynamics of a motor pattern pattern generator, h(a,x) represent the dynamics of the periphery with the given central input, g(a,x) represents the effects of sensory feedback from the periphery, l(x) represents the effects of an external load or perturbation, and ε,κ R+ are scaling constants, not necessarily small. Note that ∈ here we have added to the framework introduced in chapter1 by adding the forces due to the load as a separate term. We further assume that all of these functions are smooth, inﬁnitely differentiable, and have bounded ranges over the domain of interest. CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 38

2.2.1 Limit cycles

We ﬁrst consider the case of an idealized central pattern generator, where a part of the nervous system can produce sequences of motor activity that closely resemble those seen in vivo, even when it is not attached to the periphery. Thus we assume that, for some range

of the parameter µ, the dynamics of the isolated neural circuit, da/dt = f (a, µ), contain an attracting limit cycle γ(t) which represents the observed motor pattern. We make the further important assumption that there exists a behaviorally relevant mechanical load

κl(x) with which the complete system contains a corresponding limit cycle ξ(t), such that the neural component of ξ(t) closely matches that of the isolated system. By this we mean that, with appropriately chosen initial conditions,

Z T 2 γ(t) Paξ(t) dt 1, (2.3) 0 || − || ≪

m+n n where Pa : R R is the projection operator onto the a subspace, T is the period of → γ(t), and is the usual L2 (Euclidean) norm. Note that it is possible that these very || · || similar patterns will produce very different behavior in the periphery. Therefore, we will

also assume that this isolated pattern γ(t) produces similar behavior in the periphery, that is that the system dx/dt = h(γ(t),x)+κl(x) contains an attracting periodic orbit ζ(t) such that with an appropriate choice of initial conditions

Z T 2 ζ(t) Pxξ(t) dt 1, (2.4) 0 || − || ≪

m+n m where Px : R R is the projection operator onto the x subspace. Finally, we assume → that gradually adding back sensory input does not distort the pattern or the behavior,

meaning that both γ(t) and ζ(t) persist for small values of ε and remain similar to CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 39

projections of ξ(t). Because the system is smooth, the integral in (2.3) can be made to be small by insuring

that the magnitude of the sensory feedback, εg(Paξ(t),Pxξ(t)) , is small for all t (0,T). || || ∈ Although a careful choice of g could be small along ξ but not elsewhere, we will focus || || on the more generic case where ε 1. ≪ 2.2.2 Destabilization of ﬁxed points

We next consider the chain reﬂex. In this case, the dynamics of the isolated nervous system,

da/dt = f (a, µ), will contain a set of stable nodes, A, where each node represents a “stage” of the chain reﬂex that can be destabilized by sensory input. Note that in this case, unlike

the central pattern generator, ε may need to be of O(1) to destabilize a node. The combined dynamics of the nervous system and the periphery, however, would still be expected to

contain a stable limit cycle ξ(t) rather than a series of fixed points. Similar dynamics have been seen in models of other biological oscillators; for example in Novak et al.(1998), the authors created a model of the cell cycle where fixed points in the biochemical dynamics (analogous to the isolated neural dynamics) can be destabilized by changes in cell size (analogous to the periphery) so that the coupled system contains a limit cycle. Because a chain reflex can not explain the fictive motor patterns produced by the isolated nervous system, we will not explore this alternative in this chapter.

2.2.3 Stable heteroclinic channels

We now consider a system that is intermediate between the two extremes of an idealized central pattern generator and a chain reﬂex. We can construct such a system from a set of n- dimensional hyperbolic saddle points, each with a one-dimensional unstable manifold and an n 1 dimensional stable manifold, arranged in a cycle such that the unstable manifold − CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 40

of one saddle point intersects the stable manifold of the next, forming a heteroclinic orbit. We refer to these saddle points and their connecting heteroclinic orbits as a heteroclinic cycle (Guckenheimer and Holmes, 1988). Under appropriate conditions, this heteroclinic cycle attracts nearby orbits (and thus can be called a stable heteroclinic cycle). In particular, if we deﬁne the (positive) ratio of

the least negative stable eigenvalue λi,s and the unstable eigenvalue λi,u of the ith saddle

as the saddle index νi = λi s/λi u (Shilnikov et al., 2002), then the heteroclinic cycle − , , will attract nearby orbits if ∏i νi > 1 (Afraimovich et al., 2004a). This type of dynamics can arise naturally from neural models involving symmetric mutually inhibitory pools of neurons; for example see Nowotny and Rabinovich(2007). An unperturbed trajectory on the heteroclinic cycle will, like the chain reflex model in section 2.2.2, asymptotically approach a fixed point. Unlike the chain reflex model, however, a very small perturbation in the unstable direction will push the trajectory out of the stable manifold, allowing the trajectory to leave the neighborhood of the fixed point (and potentially travel to the neighborhood of the next fixed point). Arbitrarily small amounts of noise can thus insure that the system will almost certainly not remain stuck at a given fixed point (Stone and Holmes, 1990). Thus, rather than the stability of states seen in the chain reflex model, the heteroclinic cycle exhibits metastability (Afraimovich et al., 2011), where the trajectory spends long but finite periods of time near each fixed point (Bakhtin, 2011). Thus, like the chain reflex, the system can spend short or long periods of time in one particular state depending on sensory input, but, like the limit cycle, the system will eventually transition to the next state even in the absence of sensory input. While stable heteroclinic cycles are structurally unstable (i.e. a small change in the vector field will generally break the cycle), small perturbations can result in the creation of a stable limit cycle that passes very close to the saddles. For example, in the planar case, CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 41

any sufﬁciently small perturbation that pushes the unstable manifold of the saddles towards the inside of the unperturbed stable heteroclinic cycle will result in a stable limit cycle (Reyn, 1980). Similar conditions can be found for higher dimensional stable heteroclinic cycles (Afraimovich et al., 2004a). These families of limit cycles that pass close to the original saddles, known as stable heteroclinic channels (Rabinovich et al., 2008b), are structurally stable, and exhibit many of the same properties of sensitivity and metastability as the original stable heteroclinic cycles. As we will see, this extreme sensitivity can be advantageous for generating adaptive behaviors.

In the next section, we provide an example of model dynamics f (a, µ) exhibiting a limit cycle for µ > 0 and a bifurcation to a heteroclinic cycle at µ = 0. We then investigate the behavior of the full (a,x) system in the “limit cycle” and “heteroclinic cycle” parameter regimes.

2.3 Model Description

2.3.1 Neural model

We wish to explore the effects of different types of neural dynamics on the behavior of the animal. Although detailed, multi-cellular and multi-conductance models of neurons and circuits underlying feeding pattern generation in Aplysia have been described (Baxter and Byrne, 2006; Cataldo et al., 2006; Susswein et al., 2002), the complexity of these models makes it difficult to use them for mathematical analysis. As a consequence, we choose to represent motor pools (which contain neurons that are electrically coupled to one another or have mutual synaptic excitation) using nominal firing-rate models. As discussed in section 2.2, we define the neural dynamics as a combination of

an intrinsic component, f (a, µ), that does not depend on the periphery, and a sensory CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 42

(coupling) component, g(x), which does depend on the periphery. For mathematical tractability, we assume that the intrinsic and sensory drive combine linearly, thus giving the evolution equation of the neural activity.

da = f (a, µ) + εg(x ), (2.5) dt r where a is a vector of the activity of each of the N neurons, ε is a parameter scaling the

strength of sensory input, xr is a biomechanical state variable which we will define in more detail in section 2.3.2, and µ is a parameter that can shape the intrinsic dynamics. Specifically we will consider the following modified Lotka–Volterra model which captures the dynamics of N neural pools:

! ! 1 fi(a, µ) = 1 ∑ρi ja j ai + µ , (2.6) τn − j

for 0 i < N. Here µ is a scalar parameter representing intrinsic excitation, τn is a time ≤ constant, and ρ is the coupling matrix

  i j  1 =  ρi j = 0 i = j 1 (mod N) (2.7)  −   γ otherwise, where γ is a coupling constant representing inhibition between neural pools.

When N > 2 and γ > 2 this system contains a stable heteroclinic cycle when µ = 0. In contrast, as shown in ﬁgure 2.1, it contains a stable limit cycle for small positive values of

µ, with the distance between the limit cycle and saddles increasing with increasing values CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 43

a_I3

a_I2 a_h

Figure 2.1: When µ = 0, in the absence of sensory input (ε = 0), the intrinsic neural dynamics contain a stable heteroclinic cycle (black line) connecting saddles at (1,0,0), (0,1,0), and (0,0,1). When µ is a small positive number and ε = 0, the heteroclinic cycle is broken and a stable limit cycle arises (shown in light blue for the value of µ used for the limit cycle in this chapter). CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 44

of µ. With the goal of parsimony, we use N = 3 and thus (2.6) can be expanded to

1 f0(a, µ) = (a0(1 a0 γa1) + µ), (2.8) τn − − 1 f1(a, µ) = (a1(1 a1 γa2) + µ), (2.9) τn − − 1 f2(a, µ) = (a2(1 a2 γa0) + µ). (2.10) τn − −

We explain the correspondence of these three neural pools to neural pools in Aplysia in the next section.

2.3.2 Model of the periphery and load

We next couple the neural dynamics to a nominal mechanical model of feeding in Aplysia. During ingestive behaviors in Aplysia, a grasper, known as the radula-odontophore, is protracted through the jaws by a muscle referred to as I2. The grasper closes on food, and then is retracted by a muscle called I3, and then opens again, completing the cycle (see figure 2.2). The timing of closing is often not precisely aligned with the switch from protraction to retraction. Instead, closing usually occurs before the end of protraction, although the amount of overlap varies by behavior, from very little overlap in swallows to a significant overlap in rejection. A general model for biting and swallowing could thus contain four components: protraction while open, protraction while closed, retraction while closed, and retraction while open. For simplicity, we reduce these to three components, each of which corresponds to one of the three neural pools in the neural model: protraction open, protraction closing, and retraction open, as shown in figure 2.3. The protraction

open motor pool (a0) corresponds to the electrically coupled group of neurons B31, B32, B61, B62, and B63, which activate the I2 muscle and are all active during protraction (Hurwitz et al., 1996, 1997; Susswein et al., 2002). The protraction closing neuron pool CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 45

Figure 2.2: The model breaks swallowing into three phases. First, the odontophore protracts while open (lower right). Near the end of protraction, the odontophore begins closing (left) and protracts a small distance while closed. In the last phase, the odontophore retracts while closed (upper right). The protraction muscle (I2) is shown in blue, the grasper (the radula-odontophore) is shown in red, and the ring-like retraction muscle (I3) is shown in yellow, with a section cut away to show the grasper. The green strand is seaweed, with the arrows showing how the seaweed moves within a single cycle. CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 46

Figure 2.3: Schematic of the neuromechanical model of the feeding apparatus in Aplysia. The three neural pools (a0, a1, and a2) control three phases of the behavior shown in ﬁgure 2.2: protraction open, protraction closing, and retraction closed. The solid lines and triangles indicate excitatory synaptic coupling with a neuromuscular transform represented by a low pass ﬁlter. The dashed line and Θ symbol represent a simple summation and thresholding that control closing in the model. The a0 neural pool represents the B31, B32, and B63 neurons, the a1 motor pool represents these same neurons with the addition of B8 (which experiences slow excitation from B34), and the a2 motor pool represents B64, B3, B6, B9, and B8 (which is excited by B64) activity. CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 47

parameter value description γ 2.4 inhibition strength from next pool ε 2 10 3 sensory feedback strength · − κ 3√3/2 length-tension curve normalization constant µ 0 neural pool intrinsic excitation σ0 -1 proprioceptive direction for protraction open neural pool σ1 1 proprioceptive direction for protraction closing neural pool σ2 1 proprioceptive direction for retraction closed neural pool τn 0.05 neural pool time constant τm 0.05 muscle activation time constant br 0.10 grasper damping constant bsw 0.10 seaweed damping constant c0 1 position of shortest length for I2 c1 1.1 position of center of I3 Fsw 0.01 force on the seaweed resisting ingestion k0 1 I2 muscle strength and direction − k1 1 I3 muscle strength and direction S0 0.5 proprioceptive neutral position for protraction open neural pool S1 0.5 proprioceptive neutral position for protraction closing neural pool S2 0.25 proprioceptive neutral position for retraction closed neural pool umax 1 maximum muscle activation w0 2 maximal effective length of I2 w1 1.1 maximal effective length of I3 Table 2.1: Model parameters

state initial description variable value a0 1.0 activity of I2 motor pool (non-negative) 9 a1 10− activity of hinge motor pool (non-negative) 9 a2 10− activity of I3 motor pool (non-negative) u0 0.0 activity of I2 muscle u1 0.0 activity of I3 muscle xr 0.5 grasper position (0 is retracted, 1 is protracted) xsw 0.0 seaweed position (positive is away from the animal) Table 2.2: State variables CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 48

(a1) corresponds to these same I2 motor neurons with the addition of the B8 motor neurons, which activate the I4 muscle used in closing(Morton and Chiel, 1993b). The retraction

closed pool (a2) contains B8 with the addition of the I3 motor neurons B3, B6, and B9 which are simultaneously active during retraction (Church and Lloyd, 1994). Thus the I2

muscle will be driven by both protraction-open (a0) and protraction-closing (a1) motor

pools, whereas the I3 muscle is driven by a single motor pool (a2). The I2 and I3 muscles are known to respond slowly to neural inputs (Yu et al., 1999); we thus model their activation as a low-pass ﬁlter of the neural inputs using the time

constants from the model of the I2 muscle described by Yu et al.(1999). Using ui for the

activation of the ith muscle, τm for the ﬁlter’s time constant, we use

du0 1 = ((a0 + a1)umax u0), (2.11) dt τm − du1 1 = (a2umax u1). (2.12) dt τm −

In general, the force a muscle can exert will vary with the length to which it is extended (Zajac, 1989; Fox and Lloyd, 1997). The shape of this curve is typically explained by the sliding filament theory as follows: for some maximal length, the actin and myosin fibers will not overlap and the muscle will be limited to passive forces, but below that length, the force will first rise with the increasing overlap of the actin and myosin fibers, reach a maximum, and then decline as the overlapping fibers start to exert steric effects (Gordon et al., 1966). More recently, changes in lattice spacing between the fibers has also been shown to have a role in the force-length dependence (Williams et al., 2013). We model this length/tension curve using the following simple cubic polynomial:

φ(x) = κx(x 1)(x + 1) (2.13) − − CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 49 where κ is a scaling constant. This equation crosses through zero force at zero length and

again reaches zero at the nominal maximal length of 1. We let κ = 3√3/2 to normalize the maximum force between these two points to 1 (which occurs at a length of 1/√3). Although mechanical advantage plays an important role in swallowing (Sutton et al., 2004b; Novakovic et al., 2006), when combined with the length tension curve, the resulting force resembles a shifted and rescaled version of the original length tension curve over the range of motion used in swallowing. We thus choose position and scaling constants for the length-tension curve to approximate the resultant force curve in the biomechanics, rather than the length-tension curve of the isolated muscle. We assume the tension on each muscle is linearly proportional to its activation, and sum all of the muscle forces giving

xr ci Fmusc = ∑kiφ − ui. (2.14) i wi

Here xr (0,1) is the position of the grasper, ki is a parameter representing the strength ∈ and direction of each muscle, ci the position of the grasper where the ith muscle is at its

minimum effective length, and wi the difference between the maximum and minimum

effective lengths for the ith muscle. The sign of ki determines the direction of force of the

muscle; when ki is negative (as it is for I2) the muscle will pull towards its position of shortest length, and when it is positive (as it is for I3) it will push away from this position (in the case of I3, squeezing the radula-odontophore out of the ring of the jaws. We model opening and closing of the odontophore (and thus holding and releasing the seaweed) as a simple binary function, where the odontophore is closed when certain motor pools are active and open otherwise. Speciﬁcally, the radula was considered to be

closed when a1 + a2 0.5, and open when a1 + a2 < 0.5. This threshold can be viewed ≥ CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 50

as a plane dividing phase space into two regions with different mechanics (holding the seaweed and not holding the seaweed). In our experience, the teeth on the radula tend to hold the seaweed very ﬁrmly, and the animal tends to let go before the seaweed slips from its grasp. Thus the seaweed and the odontophore are considered to be “locked together” when the odontophore is closed and we do not attempt to model slip. The seaweed is assumed to be pulling back with a constant

force Fsw, which is included in the net force on the odontophore when the odontophore is closed

The seaweed and odontophore are assigned viscous damping constants bsw and br, respectively; thus the full equations of motion are

dx r = v (2.15) dt r dx sw = v (2.16) dt sw dvr Fmusc brvr = − (2.17) dt mr dvsw Fsw bswvsw = − (2.18) dt msw when the odontophore is open, and

dx dx r = sw = v (2.19) dt dt r dvr Fmusc + Fsw (br + bsw)vr = − (2.20) dt mr + msw

vsw = vr (2.21) when the odontophore is closed. Note that we are assuming that the momentum of the seaweed is negligible. CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 51

We have observed that when seaweed is abruptly pulled, animals respond with rapid movements of the radula/odontophore without oscillations. This suggests that the system is at least critically damped under these conditions, if not over damped. Furthermore, since the mass of the buccal mass is very small (a few grams), and the accelerations during movement are typically small (based on MRI measurements, they may be close to zero during most of the motion (Neustadter et al., 2002, 2007)), we choose to use equations of motion that assume a viscous limit. Thus, instead of directly simulating equations 2.15-2.21, we use the following reduced system:

dx F r = musc (2.22) dt br dx F sw = sw (2.23) dt bsw when the odontophore is open, and

dx dx F + F sw = r = musc sw (2.24) dt dt br + bsw when the odontophore is closed. For simulations without a mechanical load, where Fsw = 0

and bsw = 0, we replace this equation with

dx sw = 0, (2.25) dt which leaves the seaweed stationary when the radula-odontophore is open. It is entirely possible that the system is effectively quasi-static, and that positional forces dominate over viscous forces, but this formulation does not assume that from the outset. CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 52

In some of the simulations we wish to simulate seaweed that is held or ﬁxed in place rather than experiencing a constant force. For these simulations we replace the constant

Fsw, with a function modeling the force as a stiff spring using Hooke’s law, i.e.

Fsw(xsw) = (xspring xr)kspring. (2.26) −

2.3.3 Proprioceptive input

Proprioceptive neurons detect the position of and forces within the animal’s body. These mechanoreceptors can take many forms, from the muscle spindles and golgi tendon organs of vertebrates to the muscle organs seen in crustaceans to the S-channel expressing neurons seen in mollusks(Vandorpe et al., 1994). Rather than model these in detail, we have assumed that, as a function of the position of the grasper, the proprioceptive sensory neurons will create a net excitation or inhibition of each neural pool. For simplicity we have used a linear relation for this proprioceptive input as a function of position,

g(xr) = (xr Si)σi, (2.27) −

where xr (0,1) is the position of the grasper, Si is the position where the net proprioceptive ∈ input to the ith neural pool is zero, and σi 1,1 is the direction of proprioceptive ∈ {− } feedback for the ith motor pool.

2.3.4 Noise

All biological systems are subject to noise, and as we will show, this can have important effects on the dynamics. Typical examples of noise in a neural context would include the small ﬂuctuations caused by opening and closing of ion channels (known as channel CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 53

parameter value description β 0.20405 neural pool global time constant µ 1 10 3 neural pool intrinsic excitation · − α0 0.6101 neural pool local time scaling near protraction open α1 0.9201 neural pool local time scaling near protraction closing − α2 0.276 neural pool local time scaling near retraction closed umax 2.9 maximum muscle activation Table 2.3: Parameters used for the limit cycle simulations

noise (White et al., 2000; Goldwyn and Shea-Brown, 2011)), the variable release of neural transmitter vesicles, and stochastic effects from small numbers of molecules in second messenger systems. One can also treat parts of the system that we are not including in the model as “noise” (Schiff, 2012), such as small variations in sensory input from the environment with a mean of zero.

We model this noise as a 3-dimensional Weiner process of magnitude η (i.e. white noise). This form of noise arises naturally when the noise is created by many small identical independent events with ﬁnite variance, such as channels opening and closing. Although most biological noise is bandwidth limited, the higher frequencies of the noise are ﬁltered out by the dynamics of the model and can thus be ignored. Noise is added to the

neural state variables ai, but assumed to be negligible for the mechanical state variable xr. For simulations in which noise is used, we thus replace the ordinary differential equation (2.5) with the stochastic differential equation

da = f (a, µ) + εg(xr) dt + η dWt, (2.28)

where Wt is a three-dimensional Weiner process. CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 54

2.3.5 Parameter changes used for the limit cycle simulations

As mentioned in section 2.3.1, the isolated neural dynamics (i.e. when ε = 0) exhibits a stable heteroclinic cycle when µ = 0, and a stable limit cycle for small positive values of µ1. In this chapter, we will be exploring the effects of these two different dynamical regimes on the ingestive behavior of the model. We will refer to them as the stable heteroclinic

channel (where µ is zero) and the limit cycle (where µ > 0). Note that when the neural dynamics is coupled to the dynamics of the periphery, for both models the combined system exhibits a stable limit cycle. For the remainder of the chapter, however, we will use the phrase “the limit cycle” to refer to the model whose isolated neural dynamics exhibit a limit cycle, and “the stable heteroclinic channel” to refer to the model whose isolated neural dynamics exhibit a stable heteroclinic channel. As we will describe in section 2.5.1, without additional tuning of the parameters, the limit cycle performs much more poorly than the stable heteroclinic channel. For the limit cycle models, we thus change a small number of parameters as shown in table 2.3. We also perform a phase dependent adjustment of timing by replacing the neural time constant

τn with the following activity-dependent time scaling function

τn(a) = (1 + α a)β. (2.29) ·

Here β is a scalar parameter representing a uniform adjustment in the speed of the trajecto-

ries (analogous to the previous constant τn), and α is a vector parameter representing an activity-dependent scaling of the speed. Note that this change affects the timing but not

1For sufﬁciently large values of µ, the intrinsic excitation overwhelms the mutual inhibition between the pools and all of the pools become tonically active via a supercritical Hopf bifurcation. This tonic activity does not produce ingestive behavior in our model, so we will not examine it further in this chapter. CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 55

the location of the trajectories in space in the isolated neural dynamics.

2.3.6 Connection to mathematical framework

This system can be understood within the mathematical framework presented in section 2.2. In particular, the neural state vector a, the neural dynamics f , and the sensory feedback g correspond to the variables and functions of the same name in (2.1). In the case of the full dynamics (2.15-2.21), the peripheral state vector x can be seen as the concatenation of u,

xr, and vr. If we assume the mass of the seaweed to be negligible compared to the mass of the odontophore, we can then model l as a vector function with all components equal to 0

except for the vr component of l when the odontophore is closed. We set this component equal to the change in force with the addition of the seaweed, i.e.

Fsw bswvr lvr = − . (2.30) mr

2.4 Materials and Methods

Predictions of the model were tested using data from intact animals, from animals in which all but feeding proprioceptive input had been removed (the suspended buccal mass), and preparations from which all sensory input had been removed (the isolated cerebral and buccal ganglia). Adult Aplysia californica were obtained from Marinus Scientiﬁc,

Long-Beach CA, USA. The animals were housed in aerated 50 gallon aquariums at 16∘C with a 12 hour light/dark cycle, and fed 0.5 g of dried laver every other day. Animals were presented with seaweed to test feeding behavior before use, and all animals used generated bites at 3 to 5 second intervals when tested. CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 56

2.4.1 Intact animals

Details of the recording methods for intact animals are described in Cullins and Chiel (2010). Briefly, animals from 350 g to 450 g were anesthetized by injecting 30% of the animal’s mass of isotonic (0.333 molar) magnesium chloride solution into the hemocoel. Hook electrodes were then surgically implanted and attached to the I2 muscle, the radular nerve (RN), buccal nerve 2 (BN2), and buccal nerve 3 (BN3). The animals were allowed to recover, and then presented with 5 mm seaweed strips to elicit swallowing patterns. Video and EMG/ENG were recorded simultaneously to capture the behavior corresponding to the feeding motor patterns. Electrical recordings were made using an A-M Systems model 1700 amplifier with a 10-1000 Hz band-pass filter for EMG and a 100-1000 Hz bandpass filter for the ENG recordings, and captured using a Digidata 1300 digitizer and AxoScope software (Molecular Devices).

2.4.2 Suspended buccal mass preparation

The methods used for the suspended buccal mass are described in McManus et al.(2012). Brieﬂy, animals from 250 g to 350 g in weight were anesthetized by injecting 50% of the animal’s mass of isotonic magnesium chloride into the hemocoel. The buccal mass and attached buccal and cerebral ganglia were then dissected out and placed in Aplysia saline

(460 mM NaCl, 10 mM KCl, 22 mM MgCl2, 33 mM MgSO4, 10 mM CaCl2, 10 mM glucose, 10 mM MOPS, pH 7.4–7.5). Hook electrodes were attached to the I2 muscle, RN, BN2, BN3, and branch a of BN2 (BN2a). The buccal mass was then suspended via sutures through the soft tissue at the rostral edge and the two ganglia pinned out behind it, with the cerebral ganglia placed in a separate chamber isolated from the main chamber using vacuum grease. To elicit ingestive patterns, the Aplysia saline in the chamber containing CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 57

the cerebral ganglion was changed to a solution of 10 mM carbachol (Acros Organics) in Aplysia saline. Electrical recordings were made using an A-M Systems model 1700 amplifier with a 10-500 Hz band-pass filter for EMG and a 300-500 Hz bandpass filter for the ENG recordings, and captured using a Digidata 1300 digitizer and AxoGraph software (Axon Instruments).

2.4.3 Isolated buccal ganglion

The methods used for the isolated ganglia are described in Lu et al.(2013). Briefly, the animal was euthanized and the buccal mass and buccal and cerebral ganglia dissected out as described for the suspended buccal mass. The ganglia were then dissected away from the buccal mass along with a small strip of I2 attached to the I2 nerve, and the ganglia were pinned out in a two-chambered dish lined with Sylgard 184 (Dow Corning), with a vacuum grease seal separating the solution in the chamber with the cerebral ganglion from that in the chamber with the buccal ganglion. Suction electrodes were attached to BN2, BN3, RN, and the excised strip of the I2 muscle. For ingestive patterns, a 10 mM carbachol solution was applied to the chamber containing the cerebral ganglion. Electrical recordings were made using an A-M Systems model 1700 amplifier with a 10-500 Hz band-pass filter for EMG and a 300-500 Hz bandpass filter for the ENG recordings, and captured using a Digidata 1300 digitizer and AxoGraph software (Axon Instruments).

2.4.4 Data analysis

Selection of patterns for analysis varied by preparation. For the intact animal, patterns were considered swallows if the video showed the animal grasping the seaweed throughout the pattern and the net movement of the seaweed was inward; other behaviors such as bites and rejections were not studied for this chapter. For the suspended buccal mass and isolated CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 58

ganglia, patterns were used near the middle of the carbachol application, as the patterns tend to be more distorted when carbachol is first added and late into the application as the behavior slows. Onsets and offsets of activity in the I2 muscle EMG were identified based on the onset and offset of high frequency firing. Activity of I3 was identified based on the activity of the three largest units on the buccal nerve 2 ENG, which have previously been identified by our lab as B3, B6, and B9 (Lu et al., 2013). A subset of the burst onset and offset timings were independently identified by a second researcher to verify inter-rater reliability.

2.4.5 Numerical methods

The stochastic differential equations were simulated in C++ using an explicit order two weak scheme with additive noise, described in Kloeden and Platen(1992). If the stochastic differential equation is expressed in vector form as

dyt = A(yt) dt + B(yt) dWt, (2.31)

this scheme is described by the following recurrence relationship:

y˜n+1 = yn + A(yn)h + B(yn)∆Wn, (2.32)

1 yn+1 = yn + 2 (A(y˜n+1) + A(yn))h + B(yn)∆Wn. (2.33)

Here h is the length of a time step and ∆Wn is a Weiner increment (a vector of pseudo- random numbers from a Gaussian distribution with mean zero and variance h). Note that in the deterministic case this reduces to the Heun method (Kloeden and Platen, 1992).

At each time step, any negative values of the ai state variables were replaced with their CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 59

absolute value to prevent noise from pushing the system into a non-physiologic range. Random numbers for the Wiener increments were generated using a Mersenne twister with a period of 219937 1 (Matsumoto and Nishimura, 1998). A time step of size 10 3 − − was used. This was veriﬁed to be sufﬁciently small by simulating the model with default parameters and seeing a change in period of less than 30 parts per million (from 4.02704 to

3 4 4.02693) when the step size was changed from 10− to 10− . Onsets and offsets of bursts were determined by when the activity of the next motor pool rose above the previous one

(i.e. ai+1 > ai), and were linearly interpolated between time steps to improve accuracy.

2.5 Results

2.5.1 Tuning the limit cycle to match the stable heteroclinic channel

Animals in the wild need to adapt their behavior to a changing environment. For example, the marine mollusk Aplysia must adapt to the changing forces imposed on it by the stipe of seaweed it is attempting to consume. These forces can vary considerably during the behavior: a stipe of seaweed might initially present very little resistance, but accumulated elastic forces in the seaweed will grow as the animal pulls against the holdfast and tidal forces can present a sudden load with little warning. In addition, these forces can pull the seaweed back out of the animal’s mouth during protraction when the grasper is open. This raises the question of which dynamical architecture - the limit cycle or the stable heteroclinic channel - can better adapt to these changing requirements. We therefore explore the efﬁcacy of the limit cycle and the stable heteroclinic channel in the ingestion of seaweed over a range of resisting forces on the seaweed. Although the stable heteroclinic channel model can be changed to a limit cycle by

increasing the intrinsic excitability µ, as shown in ﬁgure 2.4 (top line: stable heteroclinic CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 60 0.10 0.05 0.00 Average seaweed velocity seaweed Average −0.05 −0.10 0.00 0.05 0.10 0.15 0.20 Force on seaweed Figure 2.4: The limit cycle produced by changing intrinsic neural excitability (µ) alone performs much more poorly than the stable heteroclinic channel, but the limit cycle’s performance can be improved by adjustments in timing. Top black line: stable heteroclinic channel. Lower red line: limit cycle produced by only changing µ. Orange line, second from bottom: limit cycle produced by changing µ and τn, which controls the overall cycle duration. Green line, third from bottom: limit cycle produced by changing µ and replacing the constant τn with the function τn(a) = (1 + α a)β, thus allowing the limit cycle to spend similar times to the stable heteroclinic channel· at different phases of the motor pattern. CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 61

channel (µ = 0), bottom line: limit cycle (µ > 0, all other parameters ﬁxed), the resulting model is unable to effectively ingest seaweed, i.e. the net velocity of seaweed is negative for all values of force on the seaweed. We thus attempt to tune the parameters for the limit cycle to make it more effective and more comparable to the stable heteroclinic channel. We

use the behavior of the stable heteroclinic channel under a light seaweed load (Fsw =0.01,

bsw =0.10) to guide our parameter changes. Under these conditions, the stable heteroclinic channel ingests seaweed at a rate of 0.090/s, but the limit cycle with changes only to µ egests seaweed at a rate of 0.087/s (i.e. the seaweed is pulled away more quickly than it can be ingested). There are a number of reasons why the limit cycle is less effective at ingesting seaweed.

The increase in µ dramatically decreases the time spent near the saddles without increasing the time spent moving between saddles; as a result, the period of the neural pattern

decreases from 4.03s to 0.98s. To compensate for this change, we increased the time

scaling constant τn to match the periods. This adjustment increases the efficacy of the limit cycle to ingest at a rate of 0.019/s; a similar improvement is seen across a range of loads as shown in figure 2.4, second line from the bottom. The next obvious cause of the lower efficiency is the length of time each neural pool

is active; with the changes to µ and the constant τn, each pool is active for nearly equal periods of time (1.35, 1.32, and 1.36 seconds for protraction open, protraction closing, and retraction closed, respectively), whereas in the stable heteroclinic channel, protraction

closing (0.49s) is much shorter than protraction open and retraction closed (1.92s and 1.61s). These differences in how long each neural pool is active are likely to be due to differences in sensory responsiveness, which we will explore in section 2.5.2. In general, a limit cycle could spend different amounts of time in each region of the pattern without requiring dependence on sensory input. To illustrate this point, we adjust the timing of the CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 62 0.10 0.05 0.00 Average seaweed velocity seaweed Average −0.05 −0.10 0.00 0.05 0.10 0.15 0.20 Force on seaweed Figure 2.5: Increasing the maximum muscle activation allows the limit cycle to perform as well as the stable heteroclinic channel over a range of forces. Black line: stable heteroclinic channel. Bottom green line: limit cycle with timing changes only (matches the green line in ﬁgure 2.4). Dark blue line, second from bottom: limit cycle with timing changes and twice the maximum muscle activation. Violet line, third from bottom: limit cycle with timing changes and three times the maximum muscle activation. Cyan line, fourth from bottom: limit cycle with timing changes and four times the maximum muscle activation.

limit cycle by making τn activity dependent, as described in (2.29), setting β equal to our

previous constant τn and adjusting the parameter α to make the duration of activity match that seen in the stable heteroclinic channel with the test seaweed load. This increases the

efficacy of the limit cycle to 0.067/s, and again improves the performance of the limit cycle across a range of loads as shown in figure 2.4, third line from the bottom. Despite these changes to the intrinsic neural dynamics, the limit cycle is still less effective than the stable heteroclinic channel. One reason for this remaining deficit is that CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 63 30 20 10 Integrated muscle activity per unit length of seaweed muscle Integrated 0

0.00 0.05 0.10 0.15 0.20 Force on seaweed Figure 2.6: Increased muscle activation in the limit cycle comes at a metabolic cost. Black line: stable heteroclinic channel. Red line, yellow line, green line, and blue line: limit cycle with timing changes and 1, 2, 3, or 4 times the maximum muscle activation respectively.

the sharp transitions in the stable heteroclinic channel may provide faster activation of the muscles than the more gradual onset and offset of activity in the limit cycle. As shown in ﬁgure 2.5, this slower activation and deactivation can be compensated for by increasing the maximum activation of the muscle (or, equivalently, the cross section of the muscle)

umax. Increasing umax by a factor of 2.9 results in a rate of ingestion of 0.090, which is comparable to the efﬁcacy of the stable heteroclinic channel. Note that, as shown in the

ﬁgure, even with relatively high values of umax, the stable heteroclinic channel is more effective than the limit cycle when the load on the seaweed is very small or very large. Although increasing the maximum muscle activation allows the limit cycle to match CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 64 3 2 1 Work done by muscles per unit length of seaweed muscles done by Work 0

0.00 0.05 0.10 0.15 0.20 Force on seaweed Figure 2.7: With higher loads, the limit cycle is less efﬁcient than the stable heteroclinic channel, and does more mechanical work for a given amount of seaweed ingested. Black line: stable heteroclinic channel. Red line, yellow line, green line, and blue line: limit cycle with timing changes and 1, 2, 3, or 4 times the maximum muscle activation respectively. CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 65

or even exceed the efficacy of the stable heteroclinic channel over a range of loads, this change has a metabolic cost for the animal. To a first approximation, the energetic cost of contraction is proportional to the force generated by the muscle (Sacco et al., 1994). Thus, under the model’s assumption that we are in the linear regime of the force-activation curve, the energetic cost of contraction is also proportional to the activation of the muscle. In figure 2.6 we show the energetic cost, in the form of integrated muscle activation over time, per length of seaweed ingested. Assuming the system has reached steady-state, this is

R T ∑ ui(t) dt 0 i , (2.34) xsw(0) xsw(T) − where T is the period of the behavior. Note that even at low loads, the limit cycle pays a higher metabolic cost per unit length of seaweed ingested. The limit cycle’s behavior is also mechanically less efﬁcient at higher loads. In ﬁgure 2.7, we show the mechanical work done by the muscles per unit length of seaweed ingested; i.e.

R T F s dxr ds 0 musc( ) dt t=s . (2.35) xsw(0) xsw(T) − Note that the limit cycles using more strength are able to remain mechanically efﬁcient over a larger range of forces, but the stable heteroclinic channel is still more mechanically efﬁcient at higher loads than a limit cycle with muscles that are four times stronger. We will explore the differences in behavior that lead to these effects in the next section.

2.5.2 Mechanisms of adaptation to load

How do the two architectures adapt to these changes? In ﬁgure 2.8, we can see the changes between low and high seaweed forces. In the limit cycle, the time course of neural CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 66

activation is very similar under both high and low load conditions. As a result, the forces in the high-load condition dramatically reduce the distance inward that the seaweed is pulled before the odontophore releases the seaweed (thick green line). Note that once the seaweed is released, the retraction force on the odontophore is no longer opposed, and causes a rapid retraction. In the stable heteroclinic channel, by comparison, we can see that the neurons involved in retraction (red) increase their average duration of activity. The resulting long retraction allows the animal to draw in more seaweed by allowing the muscles to exert a greater peak force. The mechanisms of these changes in timing can be seen in figure 2.9. In both the stable heteroclinic channel and the limit cycle, the trajectory is moved only a small distance by sensory input. In the case of the limit cycle, the new trajectory passes through a very similar region of phase space as the unperturbed trajectory, and thus the timing of the limit cycle does not change very much. In contrast, in the stable heteroclinic channel, the small perturbation moves the trajectory near the saddle point where the flow changes rapidly even over these short distances. During retraction, the trajectory passes closer to the saddle where the flow is very small; thus it spends longer in this region. Similarly, during protraction, the proprioception of the more protracted radula pushes the trajectory further away from the slow flow near the saddle, thus reducing the amount of time spent in protraction. It is natural to ask whether the intact behaving animal employs similar strategies. Because it is difficult in the intact animal to assess the dynamic forces generated by seaweed bunching up in the buccal cavity as seaweed is ingested, we consider a simplified situation where a stiff elastic force is encountered during a swallow that prevents the seaweed from moving inward, such as the holdfast of the seaweed. In the model, this resistance can be simulated by attaching a stiff spring to the seaweed. We can create the CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 67

Stable heteroclinic channel Limit cycle 1.0 1.0 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0.0 0.0

270 275 280 285 290 295 300 270 275 280 285 290 295 300 load 1.0 1.0 Without mechanical 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0.0 0.0

270 275 280 285 290 295 300 270 275 280 285 290 295 300 1.0 1.0 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0.0 0.0

270 275 280 285 290 295 300 270 275 280 285 290 295 300 load With 1.0 1.0 mechanical 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0.0 0.0

270 275 280 285 290 295 300 270 275 280 285 290 295 300

Figure 2.8: Forces on seaweed can selectively prolong the retraction phase of the stable heteroclinic channel, but have little effect on the limit cycle. Black and green lines show the position of the radula-odontophore, with the thick green sections showing the positions when the odontopore is closed on the seaweed and the black sections showing the positions when the odontophore is open. The blue, gold and red lines show the activity of the protraction open, protraction closing, and retraction closed motor pools, respectively. For the mechanical load, Fsw was increased to 0.1 and bsw was increased to 1.0. The positions of the odontophore are similar for both the stable heteroclinic channel and the limit cycle when there is little load. Note that the duration of retraction closed (gold) increases substantially in the stable heteroclinic channel under load, resulting in a greater retraction while holding the seaweed, but not in the limit cycle under load. CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 68

Stable Heteroclinic Channel Limit cycle

● ●●●● ● ● ● ●● 1.0 1.0 ● ● ● ● ● ● ● ● ● ● a_I3 0.5 a_I3 0.5 ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● 0.0 ● ●●● 0.0 ● ● ● ●●●●●● 0.0 0.0 0.0 0.0

0.5 0.5 0.5 0.5

a_I2 a_I2 1.0 1.0 a_h 1.0 1.0 a_h

1.00010 1.00010 1.00 1.00

● ● ●● 0.95 ● 0.95 ● ●●● ● ● ● ● ●●● ● ● ●●● ● ● ● ●● ● ● a_I3 1.00005 a_I3 1.00005 ● a_I3 0.90 a_I3 0.90 ● ● ● ● ● 0.85 0.85

● ● 1.00000 1.00000 0.80 0.80 0e+00 0e+00 0.00 0.00 0e+00 0e+00 0.00 0.00 0.05 0.05 0.05 0.05 5e−05 5e−05 5e−05 5e−05 0.10 0.10 0.10 0.10 0.15 0.15 0.15 0.15 a_I2 a_I2 a_I2 a_I2 1e−041e−04 a_h 1e−041e−04 a_h 0.20 0.20 a_h 0.20 0.20 a_h

Figure 2.9: A small change in the trajectory caused by sensory input can have a large effect on the timing of the stable heteroclinic channel by pulling the trajectory closer to the saddle; the same magnitude of change has very little effect on the limit cycle. The upper row shows the trajectory of the neural variables in phase space with no load present for both the stable heteroclinic channel and the limit cycle. The circles represent points equally spaced in time (by 100 ms) to provide a sense of the velocity of the trajectory. When the force exerted by the seaweed is increased (Fsw = 0.1, bsw = 1.0), the position of the trajectory changes only a small amount. To show this, the lower row contains a magnification of the top corner of the trajectory (where the retraction-closed motor pool is most active) for the stable heteroclinic channel with the light (0.01) and heavy (0.1) mechanical load, and the limit cycle with the same two loads. The stable heteroclinic channel plots are magnified by 10000 times, whereas the limit cycle plots are magnified by 10 times. CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 69 analagous situation in the animal by feeding the animal a thin strip of seaweed and then holding the seaweed during a swallow to present a resisting force. The resulting change to the pattern, shown in figure 2.10, is very similar to what was seen with a constant force applied to the seaweed: again retraction duration lengthens and protraction duration shortens. How do these strategies compare to those used by the animal itself? As shown in figure 2.11, the duration of retraction increases, although the duration of protraction does not appear to increase or decrease. It is possible that the additional strategies used by the animal, such as moving the head, allowed the animal to fully retract and thus negated the need for shorter protractions. It is not surprising that an animal would behave in an adaptive manner to the behaviorally relevant task of consuming seaweed. If the dynamics of the central nervous system are stable heteroclinic channel-like, would this create any changes that would not be expected from a purely adaptive standpoint? There are two we will discuss here: the response to removal of proprioception and the shape of the distribution of durations of components of the pattern. The effects of removing sensory input are different for the limit cycle and the stable heteroclinic channel as shown in figure 2.12. In a limit cycle, the dynamics of the pattern generator produce a well formed pattern even in the absence of sensory input. In a stable heteroclinic channel, in contrast, normal patterns are only generated when sensory input pushes the trajectories away from the saddles. When sensory input is reduced, the trajectory passes much closer to the saddles and all parts of the pattern that are responsive to sensory input will be lengthened. In a deterministic system with a precisely tuned stable heteroclinic channel (so that the cycle is a true heteroclinic cycle that includes the saddle points), the duration of patterns will grow without bound. In a more realistic scenario, CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 70

Stable Heteroclinic Channel Limit cycle 1.0 1.0 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0.0 0.0

270 275 280 285 290 295 300 270 275 280 285 290 295 300 1.0 1.0 Without spring force 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0.0 0.0

270 275 280 285 290 295 300 270 275 280 285 290 295 300 1.0 1.0 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0.0 0.0

270 275 280 285 290 295 300 270 275 280 285 290 295 300 With 1.0 1.0 spring force 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0.0 0.0

270 275 280 285 290 295 300 270 275 280 285 290 295 300

Figure 2.10: A spring resisting retraction can selectively prolong the retraction phase of the stable heteroclinic channel, but has little effect on the limit cycle. For the “with spring force” plots, Fsw is deﬁned by (2.26) with kspring = 2.0 and xspring = 0.9; parameters are otherwise as described in table 2.1. Line colors are the same as those in ﬁgure 2.8. CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 71

Protraction Retraction 4

3 +

+ + + + + + +

2 + + + + + + + + + + + + + + Duration (s) Duration + + + + + + + + + + + + + 1 + + + + + + + + + + + + 0 held held not held not held

Figure 2.11: When a force is applied to the seaweed in vivo (by holding the seaweed), the activity of the neurons involved in retraction (corresponding to retraction closed) is prolonged (right), while the activity of the protractor muscle (corresponding to the start of protraction open to the end of protraction closing) is not (left). Medians differ (Mann– Whitney test, p = 0.013), 30 unheld swallows and 7 held swallows were used from the same two animals. Results are similar if unheld swallows from all 6 animals are used (not shown, p = 0.003). CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 72

Stable Heteroclinic Channel Limit cycle 1.0 1.0 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0.0 0.0

270 275 280 285 290 295 300 270 275 280 285 290 295 300 With 1.0 1.0 0.8 0.8 0.6 0.6 sensory feedback 0.4 0.4 0.2 0.2 0.0 0.0

270 275 280 285 290 295 300 270 275 280 285 290 295 300 1.0 1.0 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0.0 0.0

270 275 280 285 290 295 300 270 275 280 285 290 295 300 1.0 1.0 Without 0.8 0.8 0.6 0.6 sensory feedback 0.4 0.4 0.2 0.2 0.0 0.0

270 275 280 285 290 295 300 270 275 280 285 290 295 300

Figure 2.12: Removing sensory feedback slows both protraction and retraction in the stable heteroclinic channel, but has little effect on the limit cycle. For plots without sensory 30 feedback, ε = 0 and η = 10− ; parameters are otherwise as described in table 2.1. however, small amounts of noise and small imperfections in tuning of the cycle will lead

30 to longer, but still finite, cycle times. In figure 2.12, we thus set the noise η to 10− to show that even a very small amount of noise is sufficient to prevent the stable heteroclinic channel from becoming “stuck” in one of the phases. When sensory input is removed from the animal, does the duration of protraction and CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 73

retraction increase as is seen in the stable heteroclinic channel, or remain about the same as is seen in the limit cycle? To investigate this, we examine two preparations of the animal with reduced sensory input and compare them to the intact animal. In the first preparation, the suspended buccal mass(McManus et al., 2012), the feeding apparatus and the ganglia controlling feeding are dissected out of the animal and suspended in a saline solution. This preparation thus removes sensory input the animal would have gotten from the lips, anterior tentacles, and other parts of the body, but not the proprioceptive feedback from the feeding apparatus itself. In the second preparation, the isolated ganglia, the feeding apparatus is also dissected away, leaving just the ganglia controlling feeding. As shown in figure 2.13, protraction (containing both the protraction open and protraction closing phases) and retraction (closed) both increase in duration from the intact animal to the suspended buccal mass, and increase further in duration from the suspended buccal mass to the fictive patterns of the isolated ganglia. Note that this increase in both protraction and retraction differs from the selective increase in retraction when the seaweed was held in figure 2.11, but matches the increase in both phases seen in the stable heteroclinic channel. When subject to small amounts of noise, the stable heteroclinic channel and the limit cycle show different forms of variability in timing. In the limit cycle, perturbations from the noise have very similar effects regardless of where they occur in the cycle, so, by the central limit theorem, their cumulative effect is approximately Gaussian in the limit of small noise2. In contrast, as described in Shaw et al.(2012), perturbations that occur while approaching the saddle can have much larger effects than perturbations that occur while leaving the saddle, so the central limit theorem does not apply. As predicted by Stone and Holmes(1990), this results in a distribution that is skewed to the left. The

2In the limit cycle, the time spent passing through one part of the cycle can be approximated as the ﬁrst passage time of a Brownian particle with drift, so the small noise assumption is important; the distribution will become skewed as the noise becomes large relative to the drift. CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 74

Protraction Retraction

15 + +

+ + + + +

10 + + + +

●+

Duration (s) Duration ●+

5 ●+

+ + + + + + + + + ●+ + + + + ●+ + + + + + + + + + + + + + + + + + + + 0 SBM SBM in vivo in vivo ganglia ganglia

Figure 2.13: Protraction and retraction intervals are longer in the suspended buccal mass than in the intact animal, and longer in the isolated ganglia than in either the suspended buccal mass or the intact animal. Bites were used (rather than swallows) because there is no clear analog of a swallow in the isolated ganglia. Medians differ signiﬁcantly by preparation type for both protraction (Kruskal–Wallis, p < 0.001) and retraction (Kruskal–Wallis, p < 0.001). Results are similar when swallows from the in vivo and suspended buccal mass preparations are used instead of bites (not shown, p < 0.001 for both protraction and retraction). Recordings in vivo: 146 bites from 6 animals. Suspended buccal mass: 8 bites from 2 animals. Isolated ganglia: 13 motor patterns from 2 animals. CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 75

result of simulations with noise, shown in figure 2.14, is that the distribution for the stable heteroclinic channel is significantly skewed compared to the more symmetric distribution for the limit cycle. When we measure the duration of retraction of swallows in the intact animal, shown in figure 2.15, we see that the distribution is significantly skewed, more closely resem-

bling that seen in the stable heteroclinic channel domain of the model (skewness = 1.4,

D’Agostino test for skewness (D’Agostino et al., 1990): √b1 = 4.56, p < 0.001). Shown is the kernel density estimator of the total duration of B6/B9 and B3 activity from 84 swallowing patterns in 6 animals. Because the D’Agustino test is sensitive to outliers, we verified the skewness using the more robust medcouple measure of skewness (Brys et al., 2004) and bootstrapping to establish significance (1000000 samples of 84 points from a standard normal distribution). The medcouple confirms that the distribution is skewed (medcouple: 0.2968, p=0.012). As expected, the medcouple changes very little when the smallest and largest data points are removed, confirming that the measured skewness is not due to the outlier on the right (medcouple of trimmed data: 0.2966, p=0.013, using 82 instead of 84 points per bootstrap sample to match the reduced number of patterns).

2.6 Discussion

In this chapter, we have examined a neuromechanical model of feeding in Aplysia in two parameter regimes. In the first parameter regime, the pattern generator acts like an idealized central pattern generator, generating a physiologically efficient pattern in the absence of sensory input. In the second parameter regime, which has dynamics more similar to those of a chain reflex, passage near saddle points leads to greater sensitivity to sensory inputs, and the system generates very distorted patterns in the absence of sensory input. CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 76 4 3 2 density 1 0 0.6 0.8 1.0 1.2 1.4 SHC retraction duration (s) 80 60 density 40 20 0 1.600 1.605 1.610 1.615 1.620 1.625 1.630 LC retraction duration (s) Figure 2.14: In the presence of small amounts of noise, retractions are significantly more skewed for the stable heteroclinic channel than for the limit cycle (skewness = 0.91 vs 0.03, for the stable heteroclinic channel and limit cycle respectively. D’Agostino test for skewness(D’Agostino et al., 1990): √b1 = 32 vs 1.1, p < 0.001 vs p = 0.27). Shown is the kernel density estimator for the last aI3 duration in each of 10000 simulations with 4 noise η = 10− . CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 77 0.6 0.5 0.4 0.3 density 0.2 0.1 0.0

−1 0 1 2 3 4 5 6

Retraction duration (s)

Figure 2.15: Retraction durations are signiﬁcantly skewed during swallowing patterns in intact Aplysia californica (skewness = 1.4, D’Agostino test for skewness (D’Agostino et al., 1990): √b1 = 4.56, p < 0.001). Shown is the kernel density estimator of the total duration of B6/B9 and B3 activity from 84 swallowing patterns in 6 animals. Because the D’Agostino test is sensitive to outliers, we veriﬁed the skewness using the more robust medcouple measure of skewness; see text for details. CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 78

We have shown that the model based on an idealized central pattern generator does not adapt as well to changing loads as the model based on the stable heteroclinic channel does. We showed that part of this change is due to a prolongation of retraction allowing greater activation of the slow retractor muscles. We then showed that the animal itself appears to use the same strategy of prolonging retraction when faced with loads in vivo. We showed that the stable heteroclinic channel provided a better match to the biological data than the idealized central pattern generator, even for aspects of the behavior that do not convey an obvious evolutionary advantage. The ﬁrst example, removal of sensory feedback, showed increased slowing in the animal and the stable heteroclinic channel, rather than the near-constant timing predicted by the idealized central pattern generator. The second example, distribution of burst durations, showed a very skewed distribution both in vivo and in vitro, compared to the much less skewed distribution predicted by the model with the idealized central pattern generator.

2.6.1 Limitation of the model and results

We have intentionally created a very nominal model of feeding behavior which does not capture many of the details known about feeding in Aplysia. As previous work from our lab and others has shown, there are many degrees of biomechanical freedom beyond protraction and retraction that inﬂuence the efﬁcacy of feeding (Sutton et al., 2004a,b; Novakovic et al., 2006), the muscles involved have many properties which we do not include in our model (Yu et al., 1999; Zajac, 1989), and the mechanics of seaweed are much more complex than we have represented in the model (Denny and Gaylord, 2002; Harder et al., 2006). Similarly, the dynamics of proprioception are much more complex than the linear model we have used (Evans and Cropper, 1998), and there are more than three pools of neurons involved in feeding behavior (Hurwitz et al., CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 79

1997), with dynamics that are much more complex than the ﬁring rate model we have used (Susswein et al., 2002). In addition, neuromodulation and learning may alter the dynamics of the network slowly over time (Nargeot and Simmers, 2012; Susswein and Chiel, 2012). Thus we expect at best a qualitative match to the in vivo behavior, and can not compare the results against other models as rigorously as could be done with a model capable of quantitative predictions. A nominal model also has advantages, however. As complexity is added to a model, it can become more difﬁcult to interpret the mechanics and, as a result, less clear what details of the dynamics are responsible for an observed aspect of the behavior. In addition, as the parameter space grows, it becomes less obvious how dependent the results are on the particular choice of parameters (Foster et al., 1993). Thus the nominal model we have used makes it clearer that the effects we see are caused by the effects of sensory perturbations on the passage near a saddle point, and the role of the parameters in creating these dynamics can be easily understood in an intuitive manner. In addition, the dynamics we have included in the model (e.g. mutually inhibitory motor pools, slow muscle antagonistic muscles, and a slow muscle transfer function) are all dynamics that are common to many other systems. Thus our results about the qualitative behavior of the model clearly may be applicable to these other systems, which would not be as clear with a more specialized model. It is possible that some of these omitted details are critical for producing the observed behavior and that the simpler dynamics we are using may not represent the behavior of the actual system. For example, in our model the passage near a saddle, where the state variables are changing slowly, corresponds to a burst in the actual behavior where some state variables (e.g. membrane potential and gating variables) are changing quickly. Many such systems, however, can be decomposed into fast and slow subsystems, (Butera et al., CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 80

1996; Krupa et al., 2008; Sherwood and Guckenheimer, 2010), and these slow passages near saddles may occur in the dynamics of the slow state variables, as has been seen by Nowotny and Rabinovich(2007). Ideally, one would want to create a more detailed model of feeding in Aplysia, and then use a principled reduction to ﬁnd the slower dynamics. The work done in this chapter may be useful for guiding such a reduction.

2.6.2 Larger implications for pattern generators

2.6.2.1 Biological aspects

Many previous authors have noted the difference between patterns seen in vitro and those seen in vivo, but the field has not yet reached a consensus about the source of these differences. In this chapter, we propose that passage of trajectories near a fixed point provides a model that can explain some of the distortions in timing seen in vitro. Furthermore, this dynamical structure may help a pattern generator better use sensory input to adapt to a changing environment, and thus this structure may be selected for by evolutionary pressures. Although stable heteroclinic cycles are not structurally stable and thus are unlikely to be seen in a biological context, stable heteroclinic channels are structurally stable and thus robust to parameter variations and noise (Afraimovich et al., 2004b), and thus are plausible dynamics for a biological system. This emergence of passage near fixed points controlling timing has been seen in other models, for example Spardy et al.(2011a,b). Many other pattern generators that have been previously identified may lie between the two extremes on this continuum between ideal central pattern generators and chain reflexes. Slower patterns in the absence of innervation have been seen in lamprey swimming (Wallen´ and Williams, 1984), crayfish walking (Chrachri and Clarac, 1990), and locust flight (Pearson et al., 1983). CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 81

These models of pattern generation may also be relevant in clinical contexts. In mammals, ﬁctive respiration can be observed in the isolated central nervous system, and is hypothesized to arise from the interaction of two pattern generators in the medulla - an inspiratory pattern generator in the pre-Botzinger complex, and an expiratory pattern generator in the retrotrapezoidal-parafacial area (Tomori et al., 2010). It has been known for some time, however, that vagotomy (cutting the vagus nerve, which contains sensory afferents involved in respiration) causes a dramatic slowing, but not cessation of respiration. Qualitatively this behavior is much closer to what we have shown in the stable heteroclinic channel model, and not that of the idealized limit cycle. This would suggest that small perturbations may be enough to cause the changes seen in central sleep apnea and possibly sudden infant death syndrome, but also suggests that the system may remain quite sensitive to certain perturbations even in the pathological state. In the case of central sleep apnea, good models of the dynamics and sensitivity might allow for new treatment modalities such as transcranial direct current stimulation during episodes of apnea or hypopnea.

2.6.2.2 Mathematical implications

Many of the behaviors we have observed in the stable heteroclinic channel may depend primarily on localized regions of the dynamics where the intrinsic dynamics, f (a, µ), are comparable in magnitude to εg(a,x). When this is not the case, the sensory input will have little effect on the speed of the trajectory, and thus can only cause large changes in timing by dramatically changing the length of the trajectory. Although we have used a stable heteroclinic channel in our model to create localized regions of slowing, several related dynamics architectures may produce similar effects. For example, in a saddle-node bifurcation on an invariant cycle, the ﬂow around a limit cycle slows near a point as one approaches the bifurcation. This slowing may create qualitatively CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 82

similar behavior3. Other examples may include relaxation oscillators where some parts of the trajectory are much slower than others, e.g. van der Pol(1926), which can create similar regions of sensitivity (Bassler¨ , 1986). These localized regions of slowing may not always be apparent in the model as written. For example, many dynamical models, such as bursting cells, may not have localized regions of slowing in the form that they are written, but can be decomposed using fast-slow analysis into state variables that change on different time scales. In these systems, saddle points may exist in the slower state variables that were not apparent in the complete system. Feldman(1966) introduced the hypothesis that motor trajectories could be understood as the result of a control process that sets up one or a sequence of biomechanical equilibrium points. Typically, the control is set by an unspeciﬁed central mechanism that may take into account high-level sensory (visual, auditory) or goal-related information. Our framework is consistent with the Equilibrium Point Hypothesis (EPH) when

the system (2.1- 2.2) has ε set to zero, the autonomous central dynamics has a ﬁxed

point f (atarget, µ) = 0 for which the target conﬁguration, xtarget, is a ﬁxed point of the

biomechanics, i.e. h(atarget,xtarget) = 0, with a suitable adjustment in the case of a nonzero load. The incorporation of sensory feedback from the motor apparatus is not explicitly included in the EPH, although it is implicit in the setting of the neural equilibrium point. Because of the key role of sensory sensitivity demonstrated in this chapter, it is natural to ask what can be said more generally for systems of the form (2.1-2.2). For weak proprioceptive input and/or weak mechanical forcing, the phase response curves of

the isolated neural dynamics, (2.1) with ε = 0, and the neural dynamics coupled to the periphery, (2.1-2.2), could be expected to play a role. It is not clear how the two phase

3The similarity actually goes deeper than this; if one adds a new state variable representing the bifurcation parameter µ and sets dµ/dt = 0, the limit cycle in the augmented system now passes near a degenerate saddle at µ = 0. CHAPTER 2. SIGNIFICANCE OF DYNAMICAL ARCHITECTURE 83

response curves are related, particularly in the case of the stable heteroclinic channel for which the uncoupled system does not have a finite period limit cycle and therefore does not have a well defined phase. However, as we have shown elsewhere (Shaw et al., 2012) one can analyze the infinitesimal phase response curve for the limit cycle obtained in the limit

of small µ, and the inﬁnitesimal phase response curve diverges in a systematic fashion for phases intermediate between successive saddle points. This large sensitivity could play a role in making stable heteroclinic channels a more sensitive dynamical architecture for incorporating guidance of motor systems via modest sensory input signals – as in the boundary between the typical limit cycle and the chain reﬂex. Chapter 3

Phase Resetting in an Asymptotically Phase- less System

84 CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 85

In the previous chapter, we found that a nominal model of feeding in Aplysia with intrinsic neural dynamics containing a stable heteroclinic channel was better able to capture several qualitative aspects of the animals behavior than a very similar model with a stable limit cycle was able to do. Many of the differences appeared to be differences in the way that incoming sensory information modiﬁed the timing of the motor behavior. In this chapter, we therefore take a closer look at how the timing of a stable heteroclinic channel can be modulated by perturbations such as incoming sensory information, and compare this sensitivity with that of a limit cycle. In chapter4, we will then discuss how the analytical results we derive in this chapter could be used to quantitatively test the ability of a stable heteroclinic channel to describe motor pattern generation in Aplysia and perhaps have applications in the control of motor pattern generators.

This chapter ﬁrst appeared as the following journal article, and is used with permission: Shaw, K. M., Park, Y.-M., Chiel, H. J., and Thomas, P. J. (2012). Phase resetting in an asymptotically phaseless system: On the phase response of limit cycles verging on a heteroclinic orbit. SIAM Journal on Applied Dynamical Systems, 11:350–391.

3.1 Introduction

Animals often generate speciﬁc sequences of behavior, for example the movements of the limbs during walking, the feeding apparatus while chewing and swallowing, or body undulations in swimming. When a repeated sequence of motions can be produced reliably, the pattern generator circuit controlling the behavior is typically modeled as an autonomous system of ordinary differential equations admitting a stable isolated periodic orbit, i.e. a CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 86

limit-cycle oscillator (Ijspeert, 2008; Wilson, 1999). Limit cycle oscillators have played a fundamental role in understanding the generation and control of repetitive motions underlying swimming in the lamprey (Buchanan and Cohen, 1982; Cohen et al., 1992), neural activity and bursting (Coombes and Bressloff, 2005; Izhikevich, 2007), the gaits of quadrupeds (Buono and Golubitsky, 2001; Golubitsky et al., 1999) bipeds (Pinto and Golubitsky, 2006) and monopeds (Beer et al., 1999; Chiel et al., 1999), as well as cardiac and respiratory activity (De Schutter et al., 1993; Rubin et al., 2009; Sammon, 1994). Another class of systems generating reproducible sequences of activity has been proposed under the rubric of stable heteroclinic sequences (Afraimovich et al., 2004b) or stable heteroclinic channels (Rabinovich et al., 2008b). A dynamical system possesses a heteroclinic sequence if there exists a chain of hyperbolic saddle ﬁxed points for which the unstable manifold of each saddle intersects the stable manifold of the next (Afraimovich et al., 2008); they generalize the notion of a stable heteroclinic cycle, an attractor comprising a ﬁnite collection of saddle points with heteroclinic connections linking them in a repeating chain (Armbruster et al., 2003; Guckenheimer and Holmes, 1990; Stone and Armbruster, 1999; Stone and Holmes, 1990). A heteroclinic cycle is stable or attracting when the product around the cycle of the saddle values Vi is strictly greater than unity { } (Afraimovich et al., 2004a); the saddle value for a hyperbolic saddle point with eigenvalues

1 2 n 1 1 λu > 0 > λ λ λ is the ratio λ /λu. We let Vi denote the saddle value for s ≥ s ≥ ··· ≥ s − − s the ith saddle. Stable heteroclinic sequences and cycles have been proposed to provide a framework within nonlinear dynamics for understanding a range of phenomena, including olfactory processing in insects (Rabinovich et al., 2008a), search behavior in the marine mollusk Clione limacina (Varona et al., 2002), “winnerless competition” in neural circuits (Afraimovich et al., 2002, 2004a) and ecological models (Afraimovich et al., 2008), genesis of network-dependent bursting activity (Nowotny and Rabinovich, 2007), and the balance CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 87

of emotion and cognition in behavioral control (Afraimovich et al., 2011) as well as other areas (Rabinovich et al., 2006). Because they require the intersection of one dimensional unstable and codimension one stable manifolds, the saddle connections comprising a heteroclinic cycle are structurally unstable (Andronov and Pontryagin, 1937). For planar systems, Reyn showed that a phase portrait containing a separatrix polygon generically forms a limit cycle when subject to a perturbation satisfying a net inflow condition (Reyn, 1980), provided the unperturbed heteroclinic cycle is attracting. Trajectories near an unperturbed attracting heteroclinic cycle traverse the cycle with longer and longer return times; the trajectory along the heteroclinic cycle itself has ”infinite period.” Similarly, for limit cycles arising from perturbations satisfying the net inflow condition, the smaller the size of the perturbation, the longer the period of the limit cycle.

As an example, consider the system deﬁned on the 2-torus 0 yi 2π, i 1,2 : ≤ ≤ ∈ { }

dy 1 = f (y ,y ) = cos(y )sin(y ) + α sin(2y ), (3.1a) dt 1 2 1 2 1 dy2 = g(y1,y2) = sin(y1)cos(y2) + α sin(2y2). (3.1b) dt −

As shown in Figure 3.1a, this system has four saddle points connected by heteroclinic

connections. The saddles have identical eigenvalues λu = 1 2α, λs = 1 2α, so the − − − 4 1+2α 4 product of the saddle values, V = ∏i=1 Vi, is V = 1 2α . If α (0,1/2) then V > 1, − ∈ indicating that the heteroclinic 4-cycle is attracting. Although the heteroclinic cycle connecting the four saddle points is structurally unstable, any perturbation of the vector ﬁeld that pushes the unstable manifold of each saddle towards the interior of the cycle relative to the stable manifold of the next saddle will generically lead to the formation of a stable limit cycle. For example, consider the effects CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 88

π π

π π 2 2

0 0

π π − 2 − 2

π π − π π 0 π π − π π 0 π π − − 2 2 − − 2 2 (a) (b)

Figure 3.1: Parametric perturbation of an attracting heteroclinic cycle to a stable limit cycle. Note solid line is x-nullcline and dashed is y-nullcline. (a) A trajectory of the smooth toroidal system given by Equations 3.1, passing near four distinct saddles each with eigenvalues λs = 1 2α and λu = 1 2α. The trajectory shown begins near the unstable spiral point at− location− (π,π) near− the center of the plot. The trajectory was integrated for a total time of 200 units using α = 7/30 and µ = 0 . It passes closer to each successive saddle, slowing progressively. (b) The perturbed system, Equations 3.2, when α = 7/30 and µ = 0.01, forms a stable limit cycle passing close to the four saddle points. Please see corresponding movie (ﬁle smooth.mpg); this animation cycles through phase portraits for the smooth system for values of µ varying from 0 to 1/2 and back again. At µ = 2α the limit cycle collapses via a Hopf bifurcation to a single stable ﬁxed point located between the four surrounding saddles. CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 89

of “rotation” of the ﬂow of Equations 3.1 parametrized by 1 > µ > 0:

dy 1 = f (y ,y ) + µg(y ,y ), (3.2a) dt 1 2 1 2 dy2 = g(y1,y2) µ f (y1,y2). (3.2b) dt −

We will refer to this system as the smooth system, in contrast to the piecewise linear

system to be introduced subsequently. As shown in Figure 3.1b, for µ > 0 the heteroclinic connections are broken, but a limit cycle has been created inside of the saddles that passes near each of the saddle points.

For any small positive value of µ, the system 3.2 will have a stable limit cycle,

γ(t) = γ(t + T), with period T(µ) ∞ as µ 0+. For µ > 0 we may identify a phase → → θ [0,θmax) with each point on the limit cycle by choosing an arbitrary point γ0 to have ∈ phase θ(γ0) = 0 and requiring dθ/dt = θmax/T(µ). Typically one chooses θmax to equal either 2π or unity. Because of the fourfold symmetry of the systems considered here it will be convenient to set θmax = 4 throughout, so that traversal of each quarter of the limit cycle will correspond to a unit increment in phase. Figure 3.2 illustrates the time course of

trajectories of Equations 3.2 for α = 7/30 and µ 10 3,0.1,0.3,0.45 . ∈ { − } To each point ξ0 = (x0,y0) in the basin of attraction of the limit cycle we may assign an

asymptotic phase φ(ξ0) [0,4) satisfying ξ(t) γ(t θ(ξ0)T/4) 0 as t ∞, where ∈ || − − || → → ξ(t) and γ(t) are solutions of 3.2 satisfying initial conditions ξ(0) = ξ0 and γ(0) = γ0, respectively, and (x,y) = x + y . The level curves of φ, called isochrons, foliate the || || | | | | basin of attraction; see (Izhikevich, 2007) for examples. An important question for understanding the dynamics of control in central pattern generators is how such systems combine robustness to perturbation with ﬂexibility to adapt CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 90

1.5 1.0 0.5 0.0 0.5 −1.0 −1.5 − 0 20 40 60 80 100 120 140

1.5 1.0 0.5 0.0 0.5 −1.0 −1.5 − 0 10 20 30 40 50

1.5 1.0 0.5 0.0 0.5 −1.0 −1.5 − 0 5 10 15 20 25

1.5 1.0 0.5 0.0 0.5 −1.0 −1.5 − 0 5 10 15

Figure 3.2: Time plots of limit cycle trajectories of the smooth system with various values of µ. When µ 1 the time plots exhibit prolonged dwell times due to slow transits past the saddle points≪ and relatively rapid transits between them. As µ increases, the limit cycle moves away from the saddle points and the speed becomes more uniform (note the change in horizontal scale). As a result, the time plots become more sinusoidal resembling an Andronov–Hopf oscillator. The ﬁrst and second ordinates are shown in black and gray, 3 respectively, with α = 7/30. From top to bottom µ = 10− ,0.1,0.3,0.45. Compare Figure 3.5. CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 91

behavior to variable environmental or physiological conditions.1 Many biological systems combining repetitive behavior and behavioral or metabolic control exhibit limit cycle behavior that is strongly influenced by passage of trajectories near one or more unstable fixed points or quasiequilibria. A recent model for the generation of multiple rhythmic states in a respiratory CPG (Rubin et al., 2009) provides an example. This model examined dynamic reconfiguration of a CPG network by external inputs as a mechanism for adaptive alteration of patterned rhythmic output. During normal rhythmic activity, trajectories of the system decomposed into a series of slowly varying components separated by rapid switching, for instance, from an expiratory to an inspiratory phase of activity. Control of the rhythm through changes in the period as well as the duration of different functional phases could be effected by modulatory signals making small changes to the dynamics of escape and release from inhibition, thereby changing the paths of trajectories in the vicinity of quasiequilibria. Similarly, control of the net speed of motion produced by a model locomotory CPG coupled to an explicit musculoskeletal system resulted from the adjustment of CPG trajectories in proximity to unstable fixed points of the model (Markin et al., 2010; Spardy et al., 2011a,b). Patterns of biting, swallowing and rejection in Aplysia may be understood in terms of sequential traversals between neuromechanical equilibrium points (Sutton et al., 2004b,a). During normal cell growth and proliferation a living cell passes repeatedly through several phases (including cell division), yet the cell “cycle” is typically described not as a standard limit cycle but as a sequence of traversals between quasiequilibria that act as checkpoints (Tyson and Novak, 2001).2 In the vicinity of each

1Whether originating endogenously (neural noise, internal control mechanisms), or exogenously (environmental fluctuations), perturbations of the system’s dynamics occur on a variety of time scales. For clarity we will limit discussion to two extremes of fast and slow perturbations. Perturbations occurring on slow time scales relative to other system dynamics will be referred to below as static or parametric perturbations, for example fixing a value µ > 0 in Equations 3.2. Fast perturbations will be approximated as instantaneous trajectory dislocations. 2Strictly speaking, the system may be viewed as a limit cycle if the dynamics are embedded in a larger CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 92 quasiequilibrium, the cell cycle dynamics slow for an indefinite period of time, until a regulating condition is met (Novak´ and Tyson, 2008). In each of these examples, although the flow strictly speaking forms a deterministic limit cycle, the behavior may be actively managed through the introduction of variable dwell times that function as control points along trajectories. Families of limit cycles verging on a heteroclinic (or homoclinic) cycle such as the limit cycles of the one parameter family of systems given by Equations 3.2 provide an opportunity for studying the role of saddle points in the control of timing of rhythmic behaviors. As a first step, it is natural to consider the structure of the infinitesimal phase resetting curves which reflect the sensitivity of the return time along the limit cycle to small instantaneous perturbations. When µ = 0 the flow of Equations 3.2 does not admit a periodic solution with finite period; consequently the asymptotic phase and hence the phase response is not well defined. However, for any µ > 0 we can define the phase response, and we can study its behavior as the family of limit cycles approaches the heteroclinic cycle.

To ﬁx terminology, for 1 µ > 0, consider a trajectory ξ(t) following a stable ≫ n limit cycle γ : t [0,T) γ(t) R . Suppose the trajectory is perturbed by a small ∈ → ∈ + instantaneous displacement, taking x(t−) = γ(t + θ0T/4) to x(t ) = γ(t + θ0T/4) +~r. Provided the new initial condition x(t+) remains within the basin of attraction of the limit cycle, the orbit will approach the limit cycle with x(t) γ((t + θ1T/4) mod T) 0 || − || → as t ∞, for some new phase θ1 [0,4), resulting in a shift in asymptotic phase equal → ∈ to (θ1 θ0) mod 4. The limit cycle’s sensitivity to weak instantaneous perturbations − typically varies both with the phase at which the perturbation occurs, θ0(t) = 4((t/T) space encompassing the control variables as well; the point remains that the periodic behavior is strongly inﬂuenced by passage near unstable equilibria or near-equilibria, which function to regulate the timing of the system. CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 93

n mod 1), and with the direction in which it occurs, η = ~r/ ~r R . Differences in || || ∈ sensitivity at different phases, which are important for understanding possible control mechanisms, are captured by the inﬁnitesimal Phase Response Curve or iPRC, deﬁned as

(φ(γ(θT/4) + εη) θ) mod 4 Z(θ, µ,η) = lim − . (3.3) ε 0 ε →

As above, φ(ξ) is the asymptotic phase associated with a point ξ in the basin of attraction for the limit cycle, θ is the phase at which the instantaneous perturbation is applied, µ is the parameter controlling proximity to the heteroclinic, η is the direction of the fast perturbation, γ is the limit cycle trajectory and T is its period; the latter two entities are functions of µ. In order to study the behavior of the phase response as the system approaches the heteroclinic conﬁguration, it is important to emphasize that the limit

deﬁning the iPRC in Equation 3.3 is taken before the subsequent limit µ 0. → Analytic calculation of the iPRC is typically accomplished via an adjoint equation method, and exact solutions are known in very few cases (Ermentrout and Terman, 2010b; Izhikevich, 2007). In order to perform the analysis required to gain qualitative insight into

the behavior of the phase response under the sequential limits ε 0, µ 0, we construct → → a piecewise linear approximation to the smooth system. The iris system, described below, is topologically equivalent on an open set including the family of limit cycles and the saddle points, and qualitatively captures the behavior of the smooth system. For the iris

system, we obtain exact results including the form of the limit cycle for positive µ and an explicit formula for the inﬁnitesimal phase response curve. Our main result, stated in Theorem1 below, shows that for the iris system, the sensitivity to small displacements parallel to the direction of the stable manifold has two regions with distinct sensitivity behavior. In the limit, as the family of orbits approaches the heteroclinic cycle, there is CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 94

an interval of phases ϕ [0,ϕc] for which the infinitesimal phase response goes to zero. ∈ This interval is separated by a critical phase ϕc from a second interval ϕ (ϕc,1] for ∈ which the iPRC diverges to +∞. The junction φ 0 (mod 1) corresponds to the boundary ≡ between regions on which we define a piecewise linear dynamics. We show numerically that qualitatively similar results hold for the smooth system.3 Limit cycles in piecewise linear (PWL) dynamical system have been studied previously in several contexts. For instance, in the context of Glass networks (Edwards and Glass, 2000; Glass and Pasternack, 1978a,b) PWL systems have been used to represent the dynamics of idealized genetic regulatory systems. In this case, the structure is somewhat different from that considered here, in that the fixed point driving the flow in each piecewise linear region is strictly attracting and lies outside the region, rather than having one unstable eigendirection and lying inside the corresponding flow region. Consequently, families of limit cycles verging on a heteroclinic cycle do not appear in Glass networks. PWL planar dynamical systems in which a fixed point and a limit cycle coexist do occur in models approximating the Fitzhugh–Nagumo equations; such systems were used to study traveling wave phenomena (McKean, 1970) and period adding bifurcations under periodic forcing (Coombes and Osbaldestin, 2000). Neither homoclinic nor heteroclinic cycles appear in these systems, however. Recently, Coombes investigated phase response curves for limit cycles in both the PWL McKean–Nagumo model and a new model related to the Morris–Lecar system (Coombes, 2008); this paper exploited the existence of exact solutions for the iPRC to study synchronization in gap-junction coupled networks, in both the strong and weak coupling limits. The PWL Morris–Lecar system does contain a homoclinic bifurcation; to the best of our knowledge, however, the analysis presented here

3See also section 3.8.3 for numerically computed iPRCs for the Morris–Lecar system near a bifurcation from a family of limit cycles to a homoclinic orbit. CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 95

is the ﬁrst to obtain exact results for the scaling of the inﬁnitesimal phase response curve, as a system of limit cycles approaches a saddle-homoclinic or heteroclinic orbit.

3.2 The piecewise linear iris system

(a) (b)

Figure 3.3: Construction of the iris system from the smooth system. (a) The flow near each saddle point of the smooth system (3.2) is approximately linear in a square region aligned with the (orthogonal) eigenvectors. By virtue of the fourfold symmetry of the system we can extend each square until it has a side of length 2b, remaining centered on the saddle. Depending on the extent of the rotational (parametric) perturbation of the vector field, there will be square of side a forming a gap around the unstable spiral point. (b) We extend the linearized flow for each saddle throughout the corresponding square of side 2b, creating a piecewise linear vector field defined on the plane with the squares th th of side a removed. The i square is centered on the i saddle, xi (i = 1,2,3,4). In each s u square the inward arrows indicate Wloc and the outward arrows indicate Wloc, parallel to the stable and unstable eigenvector directions, respectively.

As µ 0+, the period of the limit cycle in system 3.2 diverges and the asymptotic → phase may no longer be deﬁned. Nevertheless, the response of the system to small, transient perturbations remains of interest. To explore behavior analogous to phase resetting in the

µ 0+ limit, we introduce a piecewise linear planar dynamical system analogous to the → smooth system in Equations 3.1. Figure 3.3a illustrates the construction: we tile the plane CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 96

θ = 0 θ = 0

(a) (b)

θ = 0

Figure 3.4: The behavior of the iris system depends on the offset a. The system is shown 1 rotated through an angle of tan− (a/b) for easier visual comparison with the smooth system. (a)[ a = 0] When the− offset a is 0, the unstable manifold of each saddle connects to the stable manifold of the next, forming a stable heteroclinic orbit. (Thin blue line: sample trajectory. All trajectories proceed clockwise.) (b)[ a = 0.05] When 0 < a 1, a stable limit cycle exists that passes close to the saddles. Its basin of attraction extends≪ inward to a small, unstable limit cycle around the center (red curve and arrow). (c)[ a = 0.2] As a increases, the limit cycle becomes more rounded and the ﬂow along the limit cycle more regular. (d)[ a = 0.255] If a continues to grow, the stable and unstable limit cycles disappear in a fold bifurcation. Please see corresponding movie (ﬁle: iris.mpg); this animation cycles through phase portraits of the iris system for values of a ranging from 0 to 0.255 and back again. CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 97 with large squares of size 2b centered on each saddle and smaller squares of size a between

them. When the rotation parameter µ is zero, the large squares align and the square of side a vanishes. As the vector ﬁeld “rotates” in the vicinity of each saddle, the squares rotate

and a increases from zero. Within each square, we introduce coordinates ξ = (s,u). With respect to these coordinates, the local ﬂow obeys the linear equations

ds/dt = λs, (3.4) − du/dt = u. (3.5)

Here λ > 0 is the eigenvalue corresponding to the eigenvector (1,0) tangent to the stable manifold W s = (s,0) b < s < b . The unstable manifold W u = (0,u) b < u < b loc { | − } { | − } is tangent to the eigenvector (0,1) corresponding to the unstable eigenvalue, which without loss of generality is set to one. The stable and unstable eigenvectors are arranged so that when a > 0 the ﬂow on the inner quadrants of each square moves clockwise; the ﬂow leaving the square around the ith saddle enters the square around the i + 1st saddle (mod 4), as shown in Figure 3.3.

th Formally, for k 1,2,3,4 let the center of the k square, (xk,yk), be given by the ∈ { } k iπ/4 iπ/4 real and imaginary parts, respectively, of zk = √2( i) (be + (a/2)e ), where − − th 0 a < b. Define the k square to be Sk = (x,y) : x xk b & y yk b . In the ≤ { | − | ≤ | − | ≤ } interior of the first square, the flow satisfies x˙ = λ(x x1) and y˙ = y y1. The flow in the − − − interior of squares two through four is defined so that the vector field is equivariant with respect to fourfold rotation about the origin. Adjacent squares form a boundary of length

(2b a). At the boundary between Sk and Sk 1, the flow leaves Sk and enters Sk 1, by − + + construction. The vector field is discontinuous across these boundaries. For definiteness, we take the flow at points on the mutual boundary Sk Sk 1 of adjacent squares to be ∩ + CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 98

defined so that the vector field is continuous when approached from square Sk+1, the square into which the trajectories enter.4 The four offset squares may be repeated to form a partial tiling of the plane, leaving a complementary set composed of smaller squares of size a. We will view the entire system as confined to the 2-torus, however. We leave the flow unspecified in the interiors of the small squares, and we take any egress points from a large square into a small square to be absorbing, i.e. the flow is set to zero at the boundary of the small squares. Since we are interested in the phase response of limit cycles whose basins of attraction are entirely contained within the larger squares, the flow in the smaller squares is of no consequence. Figure 3.4 illustrates the system, and shows sample trajectories for different values of a. We will refer to the piecewise linear system as the iris system because it resembles the iris of a camera, opening and closing as a increases or decreases.

It is clear that the iris system will form a stable heteroclinic orbit when a = 0 (corresponding to µ = 0 in the smooth system); we will show in 3.3.2 that it also exhibits S stable limit cycles passing near the saddles for small values of a > 0 (corresponding to µ > 0) as shown in Figure 3.4. When the limit cycle exists, as above, we assign a phase

θ to each point γ in the cycle as the fraction of the cycle’s period (scaled by θmax = 4)

required to reach that point from a deﬁned starting point γ(0) = γ0 on the cycle, i.e.

4 θ(ν) = min t > 0 γ(t) = ν , (3.6) T { | }

where T is the period, i.e. T = min t > 0 γ(t) = γ0 . It is clear that with this transfor- { | } 4Formally, the piecewise linear flow we define here is an example of a differential inclusion (Boukal and Krivanˇ , 1999; Filippov, 1988;K rivanˇ and Vrkocˇ, 2007). In our case, the component of the flow transverse to a domain boundary does not change sign across the mutual edge of adjacent boxes; therefore the technical machinery of differential inclusion theory is not needed to specify the flow unambiguously. CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 99

mation dθ/dt is constant and equal to 4/T. We deﬁne the point where the limit cycle ﬁrst

enters the bottom left square (surrounding saddle S1) as γ0. Because the ﬂow is piecewise linear, we may derive analytically the form of the phase

response curve for any values of λ > 1 > a > 0 for which a stable limit cycle exists.

Theorem 1. Let λ > 1 > a > 0, let the iris system be deﬁned as in 3.2, and deﬁne the S function

ρ(u) = uλ u + a. (3.7) −

1. If the function ρ has two isolated positive real roots then the iris system has a stable limit cycle. Let u denote the smallest positive real root of 3.7. The limit cycle

trajectory enters each square at local coordinates (1,u) and exits each square at local coordinates (s,1) where s = uλ . The period of the limit cycle is T = 4log(1/u).

2. Let θ [0,4) be the phase of an instantaneous perturbation in direction η and let ∈ θ = k + ϕ where ϕ [0,1) and k 0,1,2,3 . If k = 0 then the inﬁnitesimal phase ∈ ∈ { } response of the limit cycle is

η β(ϕ) Z(η,ϕ,a) = · , (3.8) log(1/u)(u λs) −

(1 ϕ) ϕ where β(ϕ) = s − ,u and η = (ηs,ηu) is a unit vector in the L1 norm. If k 1,2,3 then the inﬁnitesimal phase response is given by Z(η ,ϕ,a) where η = ∈ { }   ′ ′ 0 1 k  −  R η and R =  . The magnitudes of the phase responses to perturbations 1 0 parallel to the stable and unstable eigenvector directions is greatest at phases corresponding to boundaries between piecewise linear regions.

3. As a 0 for ﬁxed λ, the entry coordinate scales as u = a + o(a), the inﬁnites- → CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 100

imal phase response to perturbations parallel to the unstable eigendirection in a given square diverges, and the response to perturbations parallel to the stable

eigendirection diverges when ϕ (1 1/λ,1); otherwise, it converges to zero. ∈ −

Figure 3.5 illustrates the time course of trajectories of the iris system for λ = 2 and a 10 3,0.1,0.2,0.24 . ∈ { − } While a general consideration of phase resetting in the vicinity of a heteroclinic bifurcation on an invariant circle (or, similarly, near a homoclinic bifurcation) is beyond the scope of this paper, it is natural to conjecture that the limiting behavior of phase response curves near such bifurcations may be similar to that observed here. We discuss the form analogous results might take in a higher dimensional setting in 3.8.5. S Lemmas3 and4 in 3.3 provide necessary and sufficient conditions to guarantee the S existence of a stable limit cycle. Lemmas6,7 and8 in 3.4 develop the response of a S limit cycle trajectory to a small transient perturbation, leading to direct calculation of the infinitesimal phase response curve. Lemmas 10 and 11 in 3.5 describe the asymptotic S behavior of the iris system in the heteroclinic limit, i.e. as a 0. In 3.5 we also compare → S analytic and numerical results for the phase response curves of the iris system. In 3.6 we S numerically explore the isochrons of the iris system. In 3.7 we numerically obtain phase S response curves for the smooth system given by Equations 3.1 and compare their structure with those of the iris system. In order to make a qualitative comparison between our results and a system enjoying direct biological motivation, in 3.8.3 we show iPRCs obtained S numerically for the planar Morris–Lecar system near a bifurcation from a family of limit cycles to a saddle homoclinic orbit. In 3.8.4 we confirm numerically and analytically S that the iPRCs obtained for our PWL iris system successfully predict the stability of the synchronous solution for pairs of identical iris systems with diffusive coupling. CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 101

1.0 0.5 0.0 0.5 − 1.0 − 0 10 20 30 40 50 60 70 80

1.0 0.5 0.0 0.5 − 1.0 − 0 5 10 15 20 25

1.0 0.5 0.0 0.5 − 1.0 − 0 2 4 6 8 10 12 14

1.0 0.5 0.0 0.5 − 1.0 − 0 2 4 6 8 10

Figure 3.5: Time plots of limit cycle trajectories of the iris system with various values of a. When a 1 the trajectories slow dramatically when passing near the saddles and travel more quickly≪ between them, resulting in time plots with prolonged dwell times. As a increases, the limit cycle moves away from the saddle points and the limit cycle trajectory becomes faster and more uniform in speed, resulting in time plots with a more sinusoidal character. Note the change in horizontal scale. The horizontal and vertical ordinates are shown in black and gray, respectively, with λ = 2. From top to bottom 3 a = 10− ,0.1,0.2,0.24. Compare Figure 3.2. CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 102

In Brown et al.(2004) Brown et al. studied phase response curves for limit cycles near the four codimension one bifurcations leading to periodic ﬁring in standard neuronal models (saddle-node bifurcation of ﬁxed points on a periodic orbit, supercritical Hopf bifurcation, saddle-node bifurcation of limit cycles, and homoclinic bifurcation). Their analysis of the homoclinic bifurcation corresponds to our analysis of the iris system in a certain limit; for a detailed comparison see 3.8.2. S 3.3 Limit Cycles in the Iris System

We now prove several results about the piecewise linear iris system that we will need to prove Theorem1. We start by studying the trajectory within one of the square regions to construct a map from the time and position of entry into the region to the time and position of egress out of the region. Next, we connect four of the linearized regions together. We

then prove the existence of a limit cycle for sufﬁciently small values of a > 0. We also examine the effects of a perturbation of the trajectory within the neighborhood on the exit time and exit position. We use the maps thus derived to show that a stable limit cycle exists, and use the perturbation results to determine the asymptotic effect of a small perturbation on the phase of the oscillator (i.e. the phase response curve). First, we will examine the dynamics within a single linear region around a saddle of the iris system.

Remark 2 (Nondimensionalization). Assume each saddle of the iris system has two real orthogonal eigenvectors, one stable and one unstable. Assume the region around each saddle is a square aligned with these vectors and centered on the saddle with a length of

2L along each side (so that the saddle is a distance of L away from each edge). We will

refer to the unstable eigenvalue of the saddle as λu > 0 and the stable eigenvalue as λs < 0. CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 103

The trajectories within the region have the dynamics

ds du = λ s, = λ u. (3.9) dt s dt u

Assume that when a trajectory leaves the edge of one region at position x f = (s f ,u f ) =

(s f ,L), it enters the next region with an offset of a, i.e. xi = (si,ui) = (L,s f + a). (See construction in Figure 3.3.) We assume that there is a cycle of four regions the trajectory may traverse this way.

We can nondimensionalize the system by deﬁning the new state variables s* = s/L

and u* = u/L, by rescaling time as t* = λut and the offset as a* = a/L. We may deﬁne

λ = λs/λu as the saddle ratio, which will play a critical role in stability ( 3.3.2). Using − S these deﬁnitions, the governing equations become

ds* du* = λs*, = u*. (3.10) dt* − dt*

We will use the non-dimensionalized version of the system for the remainder of the paper. For simplicity of notation, we will omit the asterisks in the sequel.

3.3.1 Dynamics Within A Linear Region

Within a single square region surrounding a given saddle point, we will use the local coor-

dinates (s,u) to represent the displacement parallel to the stable and unstable eigenvectors,

respectively. The line segments (s,0) R2 0 s 1 and (0,u) R2 0 u 1 are { ∈ | ≤ ≤ } { ∈ | ≤ ≤ } part of the stable and unstable manifolds of the saddle, respectively, forming separatrices of the ﬂow in the square. We will restrict attention to ﬂow entering the square along the

edge (1,u) R2 0 < u < 1 . It is easy to see that a trajectory entering at an initial point { ∈ | } λ xi (si,ui) = (1,ui) at time ti with ui > 0 will exit at point (s f ,u f ) = (s f ,1) = (u ,1) at ≡ i CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 104

time log(1/ui). We deﬁne the map fl : R R from an entry position along the s = 1 edge → to an exit position along the u = 1 edge of the region of linear ﬂow, and a function T1(ui) representing the transit time from an entry position (1,ui):

λ fl(ui) = ui , T1(ui) = log(1/ui). (3.11)

Note that fl is a monotonically increasing function of ui and that T1 is a monotonically

decreasing function of ui. The closest approach of a trajectory to the saddle point occurs when the position

x = (s,u) is perpendicular to the velocity v = ( λs,u), or 0 = λ exp[ 2λt]+u2 exp[2t]. − − − i Similarly, one may calculate the time at which the speed of the trajectory is minimal. These times are

logλ 2logui 3logλ 2logui t = − , t = − . (3.12) closest 2(λ + 1) slowest 2(λ + 1)

We can examine the position of closest approach by integrating the system from the entry

position for a time tclosest, which gives the location

1 2λ 2 tclosest 2 /( + ) uclosest = uie = ui λ/ui , λ 2λ 2 λtclosest 2 /( + ) sclosest = e− = ui /λ .

Taking the ratio of the two coordinates we ﬁnd

1/2 uclosest = λ sclosest, (3.13)

and thus the point of closest approach of each trajectory in this quadrant lies along a line CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 105

passing through the saddle. A similar calculation using tslowest shows that the slowest points lie along the line

3/2 uslowest = λ sslowest, (3.14) which also passes through the saddle point at the origin of the local coordinate system.

3.3.2 Dynamics Across Regions

We will next explore the conditions under which the iris system contains a limit cycle.

Lemma 3. The iris system described in 3.2 contains a limit cycle iff the function S

ρ(u) = uλ u + a −

has isolated real roots u†,u‡ (0,1), with u† u‡. These roots exist iff λ > 1 and ∈ ≤

λ/(1 λ) 1/(1 λ) λ − λ − + a 0. − ≤

Proof. To track crossings between regions, we will add a subscript to the variable names

indicating how many region crossings have occurred since time t = 0, so that ui,3 is the

ingress location (ui) immediately after the third crossing.

When a > 0, trajectories leaving one region enter the next with an offset a, i.e. (si,n+1,ui,n+1) =

(1,s f ,n + a). We therefore have a second monotonically increasing map from the point of egress along the edge of one square to the entry point along the edge of the next square,

fe : s f n ui n 1 = s f n + a. , → , + , We can now form a map from the entry position along the edge of one linear region to CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 106 the entry position along the edge of the next region via the composition

λ h(ui) = ( fe fl)(ui) = (ui) + a. (3.15) ∘

As the composition of two monotonically increasing maps, h is also monotonically increasing. Noting that the entry edges form a transversal section of the flow, it is clear that h4, the fourfold composition of h, forms a Poincare´ map for any cycles crossing this edge. Limit cycles will form isolated fixed points in this map, so to find potential limit cycles we look

4 for points where ui = h (ui). Because h is composed of four identical monotonic maps, the limit cycles will also be ﬁxed points in these component maps and thus we have a ﬁxed

λ λ point at ui = u if and only if u = h(u) = u + a, or equivalently ρ(u) u u + a = 0, ≡ − (cf. Equation 3.7).

We now consider the potential values of λ, which by deﬁnition must be positive. First,

consider 0 < λ < 1. Rewriting ρ(u) as uλ (1 u1 λ ) + a and remembering that u (0,1), − − ∈ we can see that ρ(u) is now the sum of a positive product and a non-negative parameter, and thus cannot be zero. Next, consider λ = 1. In this case ρ(u) becomes just a, and thus ρ = 0 implies a = 0. Note that ρ is independent of u and thus every u (0,1) is a solution, ∈ corresponding to a one parameter family of neutrally stable, non-isolated orbits. Therefore

λ must be greater than one if a limit cycle exists.

Differentiating ρ twice with respect to u, we ﬁnd that ρ is convex for all u > 0 when λ > 1, and differentiating it once we ﬁnd that ρ has a minimum at

1/(1 λ) (ui)min = λ − . (3.16)

Because ρ(u) is continuous, and equal to a when u = 0 or u = 1, by the midpoint theorem CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 107 it will have a root between 0 and (ui)min and a root between (ui)min and 1 iff

λ λ/(1 λ) 1/(1 λ) 0 ρ((ui)min) = (ui) (ui)min + a = λ − λ − + a. (3.17) ≥ min − −

We will call the smaller root u† and the larger (possibly identical) root u‡.

Figure 3.6 illustrates the map h = fe fl when λ = 2 and a = 0.2. ∘ We now examine the stability of the roots u† and u‡.

Lemma 4. If the roots u† and u‡ of Equation 3.7 exist and are distinct, u† gives the entry

position of a stable limit cycle along the s = 1 edge of a region and u‡ gives the entry position of an unstable limit cycle along the same edge.

Proof. We can determine the stability of the two points by examining the derivative of

h at these points; if dh/dui > 1 the ﬁxed point will be unstable, and if dh/dui < 1 | | | | the point will be stable. Because the existence of the roots implies λ > 1, we know

λ 1 2 2 dh/dui = (ui) − λ will be positive. Differentiating h a second time we see that d h/dui = λ 2 (ui) (λ 1)λ is always greater than zero, and thus dh/dui is monotonically increasing. − − Since a ﬁxed point in a map is stable iff the magnitude of the derivative at that point is

greater than one, we ﬁnd the point where dh/dui passes through 1:

λ 1 1 = dh/dui = λ(ui) −

1/(1 λ) ui = λ − .

This, however, is just (ui)min from Equation 3.16. Because this lies between the two † roots, dh/dui † < dh/dui = 1, and thus u is a stable ﬁxed point of h, and 1 = |u |(ui)min CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 108

1.0

0.8

0.6

0.4

0.2

0.0 0.0 0.2 0.4 0.6 0.8 1.0

Figure 3.6: The map h = fe fl from the entry position of a trajectory along the edge a square to its entry position along∘ the edge the next square forms a map analogous to a return map (blue curve). This map is continuous and monotonically increasing, so a limit cycle can only exist where a trajectory enters the next square at the same position it entered the current square (green line indicates the identity map). In this example there are two intersections corresponding to the stable and unstable limit cycles. Here a = 0.2 and λ = 2. The blue curve meets the vertical axis at the offset between squares a, and changing the offset raises or lowers the blue curve without changing its shape. Varying a allows for 0, 2, or 1 limit cycles (corresponding to Figure 3.4d, Figure 3.4c, and the fold bifurcation that occurs between them). CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 109

‡ dh/dui < dh/dui ‡ and thus u is an unstable ﬁxed point of h. Because h is a |(ui)min |u Poincare´ map at the entry plane of the region, these ﬁxed points correspond to the entry points of a stable and unstable limit cycle respectively.

Remark 5. Lemmas3 and4 together establish Theorem1, Part 1.

As just shown, for small positive a two limit cycles exist — one stable and one unstable. As a 0, the stable limit cycle is destroyed in a heteroclinic bifurcation and the unstable → limit cycle collapses to a point at the center of the system. As a increases, the two limit cycles collide in a cycle-cycle fold bifurcation. The collision occurs when the inequality

(3.17) becomes an equality. The two roots converge to (ui)min as the two limit cycles that cross the section at the two roots merge. Figure 3.7 shows the bifurcation diagram.

3.4 Effects of a small instantaneous perturbation

We now examine the effects of a small perturbation of a trajectory starting on the limit cycle. We will ﬁrst explore the effects on the exit time and exit position from the square, and then extend our analysis to the asymptotic effect of the perturbation on phase over many cycles (the phase response curve or PRC).

3.4.1 Initial effects of a small perturbation

† Assume a trajectory begins on the stable limit cycle at position x0 = (1,u ) at time t = 0,

and receives a small perturbation of size r in the direction of a unit vector η = (ηs,ηu)

at time t′. We will consider only perturbations that occur before the trajectory leaves the † † square (i.e. 0 t < T1(u ) = T ); earlier or later perturbations can be reduced to this case ≤ ′ 1 by an appropriate time shift and discrete rotation. We will also require that the perturbation not push the trajectory out of the basin of attraction of the limit cycle. The inﬁnitesimal CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 110

1.0

trajectories spiral into center 0.8

fold bifurcation 0.6 a

0.4 stable limit cycle

0.2

neutrally stable orbits heteroclinic orbit 0.0 0 1 20 λ

Figure 3.7: Two-parameter bifurcation diagram for the iris system. When the stable-to- unstable eigenvalue ratio λ is sufficiently large relative to a, a stable and an unstable limit cycle coexist (region labeled stable limit cycle). As a increases the two limit cycles are destroyed in a fold bifurcation (line labeled fold bifurcation). As a approaches 0, the system approaches a heteroclinic orbit (line labeled heteroclinic orbit). When a is sufficiently large or λ < 1, trajectories spiral into the center as shown in Figure 3.4d (area labeled trajectories spirals into center). When λ = 1 and a = 0, the system becomes a square filled with neutrally stable orbits (point labeled ‘neutrally stable orbits’). CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 111

PRC we calculate below is obtained in the limit of small perturbations; for finite a > 0, the basin of attraction has finite width, and trajectories that escape the limit cycle’s attracting set are not assigned an asymptotic phase. It is straightforward to calculate the effects of the perturbation on the time and position at which the trajectory leaves the square. We first consider the typical case where the perturbation does not push the trajectory across the border of the square.

Lemma 6. Consider a trajectory initially on the stable limit cycle γ of the iris system

such that γ(0) = (1,u†), and a perturbation ∆x of size ∆x = r in the direction of unit || || † vector η = (ηs,ηu) = ∆x/r at time 0 < t′ < T1 . Assume the perturbation does not push the trajectory out of the basin of attraction or into another square. Then the perturbation will result in a change in the position of entry to the next square,

† t′ λ † λ 1 t′ ∆u1′ = ηs(u e ) + ηuλ(u ) − e− r + o(r), (3.18) and a change in transit time of the square,

t † ∆T ′ = (ηue− ′ /u )r + o(r), (3.19) 1 − as r 0. →

Proof. Immediately before the perturbation occurs, the trajectory will be at the point

λt † t lim x(t) = (e− ′ ,u e ′ ). t t → ′− CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 112

The perturbation shifts the position to

λt † t lim x(t) = (e− ′ + rηs,u e ′ + rηu). t t + → ′

The trajectory will exit the square once xu grows to 1 at time

† t T ′ = t′ + log(u e ′ + rηu) 1 −

and enters the next square along the s = 1 edge at position

λt′ λ(T1′ t′) λt′ † t′ λ u1′ = a + (e− + rηs)e− − = a + (e− + rηs)(u e + rηu) .

For r 1, we may write ≪

† t t † T ′ = t′ log(u e ′ ) (ηue− ′ /u )r + o(r), 1 − − † λ † t′ λ † λ 1 t′ u1′ = a + (u ) + ηs(u e ) + ηuλ(u ) − e− r + o(r)

† t † † λ † as r 0. Noting that t log(u e ′ ) = log(ui) = T and a+(u ) = u , these expressions → ′ − − 1 may be viewed as the exit time and position for the limit cycle with a linear correction and additional higher order terms in r. That is,

† t † T ′ = T (ηue− ′ /u )r + o(r), 1 1 − † † t′ λ † λ 1 t′ u1′ = u + ηs(u e ) + ηuλ(u ) − e− r + o(r).

† † Subtracting T1 and u respectively then gives our result.

Perturbations that push the trajectory across the boundary between two squares can be CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 113

handled without much additional difﬁculty. For example, we can treat a perturbation that crosses into the next square as an advancement of the trajectory to the point that it enters the next square followed by a perturbation to the new location in the new square. Perturbations to the previous square can be handled in a similar fashion, as can perturbations to the

diagonal square. However, for any finite a > 0 and any perturbation time other than t′ = 0 † or t′ = T1 , the perturbed point will remain in the same square as the unperturbed point for sufficiently small r > 0. Therefore the additional cases do not alter the calculation of the PRC except possibly at a finite set of points, and they will be omitted.

3.4.2 Subsequent effects of a perturbation

In the absence of perturbation, the limit cycle passing through γ(0) = (1,u†) will exhibit

† a (constant) sequence of edge crossing locations ui n = u ,n = 0,1,2,... at a sequence , { } of crossing times T † = nT † at constant intervals ∆T † T †. A single perturbation ∆x = n 1 n ≡ 1 r(ηs,ηu) at time t will lead to a new sequence of edge crossings u and crossing times ′ { i′,n} T . Following the perturbation, the time spent between the nth and n + 1st crossings is { n′} offset from the unperturbed dwell time: T T = T † + ∆T . Next we calculate the n′+1 − n′ 1 n′+1 change in entry position for subsequent edge crossings (∆u = u u†) and the dwell n′ i′,n −

time offsets for subsequent squares (∆Tn′+1).

Lemma 7. Under the conditions of Lemma6, the entry position after the nth crossing

(n 1) will be offset by ≥

n 1 † λ 1 − † t′ λ † λ 1 t′ ∆un′ = λ(u ) − ηs(u e ) + ηuλ(u ) − e− r + o(r), (3.20) CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 114 and the time spent in that square will be offset by

n 1 † 1 † λ 1 − † t λ † λ 1 t ∆T ′ = (u )− λ(u ) − ηs(u e ′ ) + ηuλ(u ) − e− ′ r + o(r) (3.21) n+1 − as r 0. → th Proof. A perturbed trajectory will enter the n edge with an offset ∆un′ . This situation is equivalent to a trajectory entering along the limit cycle that experiences a perturbation in

the u direction (η = (0,1)) of magnitude r = ∆un′ immediately upon entering the square. st We can thus use Equation 3.18 with t′ = 0 to ﬁnd the entry position along the next (n+1 ) edge:

† λ 1 ∆un′ +1 = λ(u ) − ∆un′ + o(∆un′ ).

Therefore to leading order the sequence of offsets follows a geometric series, with closed form

n 1 † λ 1 − ∆un′ = λ(u ) − ∆u1′ + o(∆u1′ ) n 1 † λ 1 − † t λ † λ 1 t = λ(u ) − ηs(u e ′ ) + ηuλ(u ) − e− ′ r + o(r)

as r 0. Substitution into Equation 3.19 provides the deviation in the time of passage → through each square, compared with that for the limit cycle trajectory. Again setting

η = (0,1), r = ∆un′ , and t′ = 0 gives

† 1 ∆T ′ = (u )− ∆u′ + o(u′ ) n+1 − n n n 1 † 1 † λ 1 − † t λ † λ 1 t = (u )− λ(u ) − ηs(u e ′ ) + ηuλ(u ) − e− ′ r + o(r). − CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 115

We now examine the asymptotic effect of the perturbation at long times. Because

u† is less than one, both ∆u and ∆t will approach zero for large n. This is equivalent to saying that the orbit will asymptotically return to the stable limit cycle, as expected. It may, however, return with a different phase than the unperturbed trajectory.

Lemma 8. Under the conditions of Lemma6, the cumulative change in crossing times after n crossings is

n+1 n+2 † t λ † λ 1 t † λ 1 ηs(u e ′ ) 1 λ(u ) + ηue ′ 1 λ(u ) − − − − − ∆C′ = r + o(r), n − u† λ(u†)λ − (3.22) which converges to † t λ t ηs(u e ′ ) + ηue− ′ ∆C′ = r + o(r) (3.23) ∞ − u† λ(u†)λ − as n ∞. →

Proof. The cumulative change in crossing times is the sum of the dwell time offsets in the previous squares: n n ∆Cn′ = ∑ ∆Tk′ = ∆T1′ + ∑ ∆Tk′ k=1 k=2 CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 116

Substituting into Equations 3.19 and 3.21 yields the result:

η e t′ n r k 2 u − † λ 1 − † t′ λ † λ 1 t′ ∆Cn′ = † r + ∑ −† λ(u ) − ηs(u e ) + ηuλ(u ) − e− + o(r) − u k=2 u ! 1 n 2 k t′ † t′ λ † λ 1 t′ − † λ 1 = −† ηue− + ηs(u e ) + ηuλ(u ) − e− ∑ λ(u ) − r + o(r) u k=0  n 1  † λ 1 − 1 1 λ(u ) − t′ † t′ λ † λ 1 t′ − = − ηue− + ηs(u e ) + ηuλ(u ) − e− r + o(r) u†  1 λ(u†)λ 1  − − n 1 n † t λ † λ 1 − t † λ 1 ηs(u e ′ ) 1 λ(u ) + ηue ′ 1 λ(u ) − − − − − = r + o(r). − u† λ(u†)λ −

Recalling that 0 < u† < 1 and λ > 1, taking the limit as n ∞ gives →

† t λ t ηs(u e ′ ) + ηue− ′ ∆C′ = r + o(r) ∞ − u† λ(u†)λ −

as required.

The limit cycle enters each square at location (1,u†) and exits at ((u†)λ ,1), so it is natural to deﬁne the exit coordinate along the stable eigenvector axis as s† = (u†)λ . Recalling that u† = s† + a, Equation 3.23 may be simpliﬁed as

† λt t ηs(s e ′ ) + ηue− ′ η β ∆C′ = r + o(r) = · r + o(r). (3.24) ∞ − u† λs† −u† λs† − −

† λt t The vector β = s e ′ ,e− ′ may be thought of as a solution to the same differential equation as a trajectory within the ﬁrst square, ξ = (s,u), but with time reversed and initial CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 117 conditions set to the far end of the square. That is,

    dξ λ 0 1  −    =  , ξ(0) =  , dt 0 1 u†     d λ 0 s† β     =  , β(0) =  . dt 0 1 1 −

3.4.3 Inﬁnitesimal phase response curve

We now derive the infinitesimal phase response curve. Initially, we restrict attention to perturbations occurring within the first quarter cycle, i.e. within the first square. The PRC for perturbations occurring after k additional border crossings is obtained via a discrete rotation operation. We define the phase of a point on the limit cycle as in Equation 3.6, i.e. as the smallest amount of time required to reach that point from the point where the limit cycle enters the first square. Therefore the asymptotic shift in crossing times translates into an asymptotic shift in phase as

4 ∆C∞′ ∆ϕ = ∆C′ = − , −T ∞ log(1/u†)

† † t † ϕ recalling that T = 4T1 = 4log(1/u ). Since e− = (u ) we may write β in the form

† λ(1 ϕ) † ϕ † (1 ϕ) † ϕ β(ϕ) = (βs(ϕ),βu(ϕ)) = (u ) − ,(u ) = (s ) − ,(u ) , (3.25)

simplifying the subsequent analysis. Thus the asymptotic change in phase for a perturbation

at phase ϕ [0,1) is ∈ η β(ϕ) ∆ϕ = · r + o(r). log(1/u†)(u† λs†) − CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 118

The inﬁnitesimal phase response curve, obtained by taking the limit of ∆ϕ/r as r 0, is → therefore η β(ϕ) Z(η,ϕ,a) = · , log(1/u†)(u† λs†) − which is Equation 3.8.

For a perturbation at phase ϕ 1, let k be the number of boundary crossings preceding ≥ the perturbation, i.e. the positive integer satisfying ϕ [k,k + 1). Deﬁne a k-fold quarter ∈ rotation operation on the phase variable and the direction of the perturbation by

ϕ ϕ′ = ϕ k, (3.26) → − k η η′ = R η, (3.27) →

  0 1  −  where R =  . Then the inﬁnitesimal phase response due to a perturbation in 1 0

direction η at phase θ > 0 is given by Z(η′,ϕ′,a). The dependence of the PRC on the phase is entirely contained in the term η β. · Consider the components of the response to perturbation in the horizontal and vertical

directions. When ϕ [0,1), the component βs along the direction of the stable eigenvector, ∈ corresponds to βx, the component in the horizontal direction. Similarly the components

along the unstable direction βu and the vertical component βy are identical. Moving clockwise around one full orbit we observe that:

when ϕ [0,1), βx = βs and βy = βu; ∈ when ϕ [1,2), βx = βu and βy = βs; ∈ − when ϕ [2,3), βx = βs and βy = βu; ∈ − − and when ϕ [3,4), βx = βu and βy = βs. ∈ − CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 119

For each component the resulting phase response curve consists of two continuous seg-

ments separated by a jump proportional to (u† +s†), with antisymmetry β(ϕ +2) = β(ϕ). − The horizontal response component, βx, peaks at βx(1) = 1 and is strictly positive for

ϕ [0,2). Similarly, βy peaks at βy(0) = 1 and is strictly positive for ϕ [0,1) [3,4). ∈ ∈ ∪ Hence the extrema of both components occur at phase values corresponding to the boundaries between adjacent squares.

Remark 9. This concludes the proof of Theorem1, Part 2.

Figure 3.8 shows examples of the full iPRCs as a function of θ [0,4) for λ = 2 and ∈ a 10 3,0.1,0.2,0.24 . ∈ { − } 3.5 Asymptotic phase resetting behavior as a 0 → We wish to study the asymptotic behavior of the phase resetting curve at the heteroclinic limit, that is, as the parameter a 0. This limit corresponds, at least by analogy, with the → limit µ 0 in the smooth vector ﬁeld system of Equations 3.2. → Lemma 10. Under the conditions of Lemma6, the limit cycle entry value scales as

u† = a + o(a), as a 0. → Proof. Recall that u† is the smaller root of ρ(u), so u† = a + (u†)λ . Since λ > 1, clearly u† = a + o(u†), as u† 0, and u† > a whenever u† > 0. For sufﬁciently small a > 0 we → have u† as an implicitly deﬁned function of a. It is straightforward to see that du†/da = 1

† † † when u = 0. We wish to show that u = a + o(a) as well, i.e. that lima 0(u a)/a = 0. → − Since u† = a+o(a), for any ε > 0 we can ﬁnd a δ > 0 such that 1 a/u† < min[ε/2,1/2] − whenever 0 < u† < δ. If a < δ then (u† a)/a = 1/(1 (1 a/u†)) 1 = 1/(1 x) − − − − − − 1 = x/(1 x) where x = 1 a/u†. Since x < min[ε/2,1/2], we are guaranteed that − − x/(1 x) < ε, as required. − CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 120

200 150 100 50 0 50 −100 −150 −200 − 0 1 2 3 4

6 4 2 0 2 −4 −6 − 0 1 2 3 4

8 6 4 2 0 2 −4 −6 −8 − 0 1 2 3 4

15 10 5 0 5 −10 −15 − 0 1 2 3 4

Figure 3.8: Infinitesimal phase response curves (iPRCs) for the iris system with various values of a. As a becomes small and the limit cycle approaches the heteroclinic orbit, the phase response curve becomes dominated by peaks at the edges between squares where flow changes from compressing trajectories outward to expanding them inward. As a grows and the flow along the limit cycle becomes more uniform, the iPRC becomes less sharply peaked. Line (dark blue): analytically obtained iPRC for a perturbation in the positive x direction. Points (dark blue): numerical iPRC, calculated using an 4 instantaneous perturbation of 10− in the horizontal direction. From top to bottom, 3 a = 10− ,0.1,0.2,0.24. The light gray curve represents the corresponding iPRC for perturbations in the vertical (positive y) direction. The discontinuities between the positive and negative portions of each curve are steps of height (u† + s†) (see 3.4.3 for further details). S CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 121

As a decreases, the peaks of the iPRC grow, while their width shrinks. The integral under each strictly positive region is

Z 2 1 + λ λu† (u†)λ V = βx(ϕ)dϕ = − −† , (3.28) 0 λ log(1/u ) which goes to zero as u† 0 or equivalently as a 0. Lemma 11 characterizes the → → behavior of the “normalized” iPRC α(ϕ) = β(ϕ)/V as a 0. →

Lemma 11. For the iris system the vector α(ϕ) = β(ϕ)/V has the following properties as a 0: →

1. For ϕ = 1 or 3, αx ∞, respectively. For ϕ 1,3 , αx 0. → ± ̸∈ { } →

2. For ϕ = 0 or 2, αy ∞, respectively. For ϕ 0,2 , αy 0. → ± ̸∈ { } → R 4 3. α(ϕ) dϕ = 4 (in the L1 norm). 0 || ||

† Proof. As a 0 both V 0 and u 0. At ϕ = 1, βx 1 for all a > 0, so as a 0 → → → ≡ → the ratio βx(1)/V +∞. On the other hand, for ﬁxed ϕ [0,2) 1 , βx(ϕ)/V 0 as → ∈ ∖{ } → † u 0. The behavior of αx or αy within the other intervals follows from the symmetry → relations discussed in the preceding paragraph. Thus 1 and 2 are established. For 3, note that by construction each interval of length two centered on a peak of one component makes a unit contribution to the integral. That is,

Z Z Z Z 1 = αx(ϕ)dϕ = αx(ϕ)dϕ = αy(ϕ)dϕ = αy(ϕ)dϕ. [0,2) − [2,4) − [1,3) [3,4) [0,1) ∪

The integral of the L1 norm α(φ) is the sum of these terms. || ||

Remark 12. Lemma 11 and the symmetry of α together imply that each component of α CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 122 converges weakly to a sum of delta function distributions on the circle:

αx(ϕ) δ(ϕ 1) δ(ϕ 3), (3.29) → − − −

αy(ϕ) δ(ϕ) δ(ϕ 2) (3.30) → − −

as a 0. → Rewriting the full PRC in terms of α gives

η α(ϕ)V Z(η,ϕ,a) = · = (η α(ϕ))M(u†), (3.31) log(1/u†)(u† λs†) · −

† 1+λ λu† (u†)λ where M(u ) = − 2 − . The function M represents the magnitude of the λ(log(1/u†)) (u† λ(u†)λ ) − 2 1 PRC and diverges as u† log (u†) − as u† 0. → The asymptotic phase response to an arbitrary instantaneous perturbation in direction

η is a sum of the response to the perturbation component along the stable eigenvector

direction, ηs and along the unstable eigenvector direction, ηu, within whichever square the perturbation occurs. The asymptotic behavior of the phase response in the heteroclinic limit for each component is distinct. We rewrite the PRC to emphasize these contributions

thus (restricted, for convenience, to the lower left square, or ϕ [0,1)): ∈

† (1 ϕ) † ϕ ηs (s ) − + ηu (u ) Z(η,ϕ,a) = . (3.32) log(1/u†)(u† λs†) −

First consider the response to perturbation along the unstable direction. Recall that

for any p > 0, up logu 0 as u 0+. It follows after a brief calculation that for any → → ϕ [0,1) ∈ ϕ u lim = +∞. u 0+ log(1/u)(u λuλ ) → − CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 123

This result means that regardless of the phase of the perturbation, the sensitivity of the asymptotic phase to small displacements parallel to the unstable eigenvector diverges as the system approaches the heteroclinic bifurcation. The result is intuitively appealing because when the system is close to the heteroclinic bifurcation the long period leads to a larger “compounding” effect enhancing the cumulative result of a small perturbation within a given square. However, as is often the case when dealing with nested limits, intuition can be misleading. In contrast to the preceding situation, consider the response to perturbation

along the stable direction. The behavior of the limit now depends on the parameter λ:



uλ(1 ϕ)  0, ϕ [0,1 1/λ]; lim − = ∈ − u 0+ log(1/u)(u λuλ ) →  +∞, ϕ (1 1/λ,1). − ∈ −

Thus, in the heteroclinic limit, small perturbations parallel to the stable eigenvector become inconsequential if they occur at an early enough phase. At a particular phase,

1 ϕ = 1 , (3.33) * − λ the response becomes highly sensitive to arbitrarily small perturbation.

Remark 13. This concludes the proof of Theorem1, Part 3.

For comparison we can obtain from Equations 3.12 the fractional phases at which the trajectory is closest to the saddle point (in the ﬁrst square) or moving at the slowest speed:

1 logλ/(2logu†) 1 3logλ/(2logu†) ϕ = − , ϕ = − . closest λ + 1 slowest λ + 1

† As u 0 both phases converge to ϕ0 = 1/(λ + 1), which is distinct from the phase at → which the PRC’s asymptotic sensitivity to perturbations in the stable direction changes. CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 124

Curiously, the asymptotic value ϕ0 of the phases of the slowest and closest point along the trajectory coincides with ϕ , the critical phase for sensitivity, precisely when λ is equal * to the golden ratio, (1 + √5)/2 1.618..., because under those conditions 1 1/λ = ≈ − 1/(λ + 1).

3.6 Isochrons

Provided a > 0 in the iris system, we may deﬁne isochrons as the level sets of the asymptotic phase function θ(x) for any point x in the basin of attraction of the limit cycle. As described in Brown et al.(2004); Guckenheimer and Holmes(1990); Izhikevich(2007) and elsewhere, the points on a given isochron will converge over time to a single point on the limit cycle with a particular phase. Thus, in a sense, they represent points with the

same asymptotic “time”. As discussed above, given a limit cycle γ(t) Ta the asymptotic { }t=0 phase of a point x0 is deﬁned as the unique value in θ [0,θmax) such that the limit ∈

lim x(t) γ(t + θ(x0)) = 0, (3.34) t ∞ → | − | where x(0) = x0 is the initial condition for the trajectory. The usual approach for finding isochrons is based on an adjoint equation method, cf. (Brown et al.(2004), Appendix A; see also Ermentrout and Terman(2010b), Chapter 11). From the chain rule, the phase field must satisfy d h i dx θ θ(x(t)) = ~∇θ (x(t)) = max , (3.35) dt · dt T(a) where T(a) is the period of the limit cycle for a given value of a > 0. Let (0 s 1,0 ≤ ≤ ≤ u 1) be local coordinates relative to any saddle of the iris system, i.e. such that the flow ≤ CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 125 satisfiess ˙ = λs, u˙ = u. Writing θv for ∂θ/∂v, Equation 3.35 reads −

λsθs + uθu = θmax/T(a). (3.36) −

By virtue of the fourfold rotational symmetry of the iris system, the following boundary

conditions are required in addition to the PDE 3.35: for 0 < s < 1 a, −

θ(s,1) = θ(1,s + a) + 1. (3.37)

The boundary condition renders the PDE nonlinear, and a simple closed form solution is not available. Solving the PDE numerically, however, is equivalent to ﬁnding the isochrons via direct simulation of trajectories for a suitable grid of initial conditions. We used a 400 400 square mesh of initial conditions within each square, tracking solutions until × either they left the system of large squares (initial conditions on the interior of the unstable limit cycle) or converged with a suitable tolerance to a small neighborhood of the limit cycle, at which point the asymptotic phase could be assigned. Figure 3.9 shows an example of the isochrons generated with this technique.

3.7 Smooth System

In order to compare the family of inﬁnitesimal phase response curves obtained for the iris system with that of a family of continuous vector ﬁelds experiencing a similar heteroclinic bifurcation with fourfold symmetry, we numerically evaluated the phase response for small instantaneous perturbations in the horizontal and vertical directions for limit cycles given

by the system 3.1 for positive values of the twist parameter µ. Figure 3.10 shows good qualitative agreement with the concentration of phase response sensitivity at points along CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 126

Figure 3.9: Isochrons for the iris system obtained via numerical iteration of the map from initial conditions to a sequence of boundary crossing. The iris system on the torus has two distinct stable limit cycles for a > 0, illustrated here in two color schemes (green-blue and fuschia-violet). The tiling of the plane by periodic extension of the basic torus system is shown. Solid lines indicate the stable limit cycles, dashed lines indicate the unstable limit cycles. All limit cycles rotate clockwise. The flow inside the small squares is not defined, and points inside the unstable limit cycles are not part of the basin of attraction of the stable limit cycles. The stable manifolds of each saddle comprise the separatrices between the basins of attraction of distinct stable limit cycles. The system shown has a = .2 and λ = 2. Colors indicate the phase. Note the compression of the level curves near the boundaries between adjacent squares, which correspond to peaks in the phase response curve. Note also that the isochrons are not strictly parallel to the stable eigenvalue direction (c.f. the discussion in 3.8.2). Please see corresponding movie (file: iris isochrons.mpg); this animation illustratesS the flow together with the isochron structure, by evolving points of equal phase forward together in time. After five seconds the animation changes to showing isochron structures for different values of the unstable/stable manifold offset parameter a, from a = 0 to a = 0.247 and back again. CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 127 the trajectory near integer values of the phase ϕ, cf. Figure 3.8.

3.8 Discussion

The iris system introduced here is one of only a handful of nonlinear dynamical systems for which an explicit form for the limit cycle or the phase response curve is known. For this system we show analytically that the sensitivity of the response to small instantaneous perturbations depends (as one would expect) on the distance to the heteroclinic bifurcation, on the phase at which the perturbation occurs, and the direction of the perturbation in the phase plane. In addition, we find the intuitively appealing result that regardless of phase, as the static perturbation away from the heteroclinic vector field diminishes, the phase response as measured by the iPRC becomes hypersensitive to any small perturbation parallel to the unstable eigenvector direction, in that the iPRC diverges as the bifurcation parameter of the iris system a 0. Unexpectedly, however, we also find that for perturbations along → the stable eigenvector direction, the response to some will diverge while the response to others will have no effect in the limit as a 0. The system appears ultrasensitive in → this case to the phase at which the perturbation occurs, with perturbations sufficiently far in advance of the approach towards the saddle point effectively absorbed by the flow, in

contrast to perturbations beyond a critical phase, ϕ = 1 1/λ, which diverge as a 0. * − → From a biological point of view, this dual sensitivity to the timing and direction of the transient perturbation is significant if flows structured by fixed points are to be exploited in nature as control points for rhythmic behaviors. In neural systems, for instance, it is commonplace to consider perturbations restricted to a single dimension in a multidimen- sional flow, namely perturbations along the dimension of membrane potential (Brown et al., 2004). When a neuron’s membrane potential lies in the linear regime, its dynamics are typically dissipative due to membrane conductances, tending to align the voltage direction CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 128

150 100 50 0 50 −100 −150 − 0 π/2 π 3π/2 2π 4 3 2 1 0 1 −2 −3 −4 − 0 π/2 π 3π/2 2π 4 3 2 1 0 1 −2 −3 −4 − 0 π/2 π 3π/2 2π 20 −22 −24 −26 −28 −30 −32 −34 −36 −38 − 0 π/2 π 3π/2 2π

Figure 3.10: Infinitesimal phase response curves for the smooth system with various values of µ. As µ becomes small and the limit cycle approaches the heteroclinic orbit, the phase response curve becomes dominated by peaks much like the iris system shown in Figure 3.8. As µ grows, the phase response curve becomes sinusoidal as the system approaches an Andronov–Hopf bifurcation. Points connected by dashed line (dark blue): numerical infinitesimal phase response curve, calculated using an instantaneous perturbation of 4 10− in the horizontal direction. Dots (dark blue): infinitesimal phase response curve for perturbations in the horizontal direction. Light gray: infinitesimal phase response curve 3 for perturbations in the vertical direction. From top to bottom, µ = 10− ,0.1,0.3,0.45. CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 129 with the stable direction for a subthreshold fixed point, at least for some portion of the flow. In general, if slowly adaptive modulatory processes within an organism’s control circuitry can impose quasistatic perturbations of an existing vector field to move trajectories closer to or farther away from a homo- or heteroclinic bifurcation point, the modulatory processes can filter which perturbations the system becomes sensitive to at different phases of the ongoing oscillation. Investigating these possibilities in specific systems such as the feeding central pattern generator of the marine mollusk Aplysia will make it possible to test this prediction empirically. In chapter2, we showed that several qualitative aspects of the behavior matched what we would predict from these regions of sensitivity. In chapter4, we will briefly discuss additional experiments that could be performed in Aplysia to better characterize and quantify these regions of sensitivity, providing an even more rigorous test of the ability of these dynamics to describe the behavior of biological pattern generators.

3.8.1 Sensitivity and control

Our results suggest that for systems in which a control parameter can change the approach of trajectories towards a hyperbolic saddle ﬁxed point with a one dimensional unstable manifold, the sensitivity to speciﬁc kinds of perturbations could be actively managed by

manipulating the critical phase ϕ . This is a novel kind of control for rhythmic behaviors; * additional mechanisms for control include manipulating the period of a typical trajectory by controlling the time spent near a saddle point, and regulating the variable dwell time in different states for a limit cycle trajectory passing near multiple unstable ﬁxed points. Oscillations arising from co-dimension one bifurcations other than a saddle node- homoclinic have phase response curves near the bifurcation point that are smooth and sinusoidal. By contrast, many phase response curves observed in real systems show sharp transitions between small and large responses, even in systems generating highly regular CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 130

behaviors such as swimming in the lamprey. Lamprey motor units participating in the CPG underlying smooth swimming showed broad regions of near zero phase response combined with steep changes in sensitivity in other regions, going from zero to peak phase response in an interval corresponding to about 10% of the limit cycle duration (c.f. Figure 6 of (Varkonyi´ et al., 2008)). Preliminary data obtained from the marine mollusk Aplysia californica during feeding behavior show extended, variable dwell times in preferred regions of phase space separated by rapid transitions from region to region (Shaw et al., 2010), suggesting the possibility that this central pattern generator’s dynamics may resemble that of a limit cycle making close encounters with a series of unstable fixed points. The qualitative match of the timing changes and distributions with the behavior of the animal with the model described in chapter2 in the heteroclinic cycle regime but not the limit cycle regime provide further support for this possibility. The infinitesimal PRC is equal to the gradient of the asymptotic phase function evaluated at the limit cycle. For the iris system, the level curves appear to pinch together at the boundaries where they abruptly change direction, and this pinching means the density of the level curves is highest at the square boundaries. One might naively expect that the phase response should be most sensitive to perturbations immediately adjacent to the saddle points; instead the peak sensitivity occurs at boundaries between regions dominated by one or another saddle. This effect is qualitatively present both in the piecewise linear iris system and in the smooth system considered here. Within the piecewise linear regions of the iris system the flow is homogeneous; it is the boundaries that make the system nonlinear and it should not be surprising that that is where the greatest sensitivity occurs. In addition, the boundaries between piecewise linear regions are the points at which the stable and unstable manifolds of adjacent saddles make their closest approach. In the sine system, it appears that near the saddles the flow is governed by the local linear approximation, CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 131 while in the region between saddles the flow is “more nonlinear” coinciding with greater sensitivity to perturbation.

3.8.2 Comparison to the PRC near a homoclinic bifurcation

While the structure and asymptotic behavior of the phase response curve described in Theorem1 were derived speciﬁcally for the iris system, we expect the qualitative picture to hold for generic one-parameter families of limit cycles verging on a heteroclinic or a homoclinic bifurcation of C1 vector ﬁelds as well. Treating the general case lies beyond the scope of this paper (but see 3.8.5). However, it will be useful to compare our results S with the analysis of phase resetting near a homoclinic bifurcation given in ((Brown et al., 2004), 3.1.3). In 3.8.3 we investigate the iPRC for the Morris–Lecar system near its S S homoclinic bifurcation. In the present section, we consider two analytic approximations of the iPRC for a “generic” near-homoclinic orbit appearing in the literature (Brown et al., 2004; Izhikevich, 2007).

n In Brown et al.(2004) the authors consider a vector ﬁeld on R with a hyperbolic

saddle ﬁxed point at the origin with a single unstable eigenvalue λu and stable eigenvalues

λs j with λu < λs = min j λs j . For positive values of a bifurcation parameter µ, the system , | , | is assumed to have a limit cycle that spends the overwhelming majority of its time in a box

B = [0,∆]n. Trajectories exit the box when the coordinate along the unstable eigendirection x = ∆ and are instantaneously reinjected with x = ε. When n = 2 the geometry is similar to that of the iris system (compare Figure 3 of Brown et al.(2004) with our Figure 3.3b). The main difference is that in the homoclinic system, the value of the injection point along the unstable limit coordinate is assumed to be independent of the egress point, whereas in

the iris system its dependence is speciﬁed by the map h = fe fl (Figure 3.6). Equivalently, ∘ it is as if the derivative of the function h were zero at the ﬁxed point u† rather than having CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 132

finite slope. This situation would occur for the homoclinic system if the compression of the flow taking trajectories from the egress boundary to the ingress boundary were sufficiently great. However, in analyzing the homoclinic system, one must also assume that the time spent along the portion of the limit cycle outside the box B is vanishingly small. Reconciling these two limiting processes for the general homoclinic case remains an interesting problem.

For purposes of comparison, note that ε/∆ in Brown et al.(2004) corresponds to u† for the iris system, and the phase θ [0,2π) in Brown et al.(2004) corresponds to 2πϕ in the ∈ iris system. Table 3.1 compares the inﬁnitesimal PRCs for the two systems. In Brown et al. (2004) the perturbation is assumed to be in a particular direction corresponding to a voltage

deflection in a neural model; the factor νx arises in the change of coordinates from the direction of voltage perturbations relative to the unstable eigenvector direction. The flow is assumed to be infinitely compressive during the time between egress and reinjection, and consequently it is assumed that only components of the perturbation along the unstable eigendirection contribute to the iPRC. This assumption holds for the iris system in the

limit as the stable-to-unstable eigenvalue ratio λs/λu = λ ∞, but not for ﬁnite values − → of λ. The λ < ∞ case includes additional corrections; see Table 3.1. Finally, assuming that the phase response is independent of displacement along the stable eigenvector direction is equivalent to assuming that the isochrons run parallel to the stable eigenvector throughout the box B. For the iris system the slopes of the isochrons do

approach zero as λ ∞, becoming more and more parallel to the stable eigenvector, but → for λ = ∞ the asymptotic phase and the isochrons are no longer well defined. For finite values of λ the isochrons are well defined, but are not parallel to the stable eigenvector. Compare the isochrons in the lower left quadrant of the iris system shown in Figure 3.9. Another example of a system with a saddle homoclinic bifurcation is the quadratic CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 133

integrate and ﬁre (QIF) model neuron (Izhikevich, 2007). For example, consider the QIF deﬁned by the voltage equation

v˙ = v2 1 (3.38) − with reset to v0 = 1 + µ whenever v ∞, which happens in ﬁnite time. For µ > 0, this → model produces a sequence of action potentials, or voltage resets, at times tk = t0 + kT(µ) with period 1 µ + 2 T(µ) = ln ; (3.39) 2 µ

the period diverges logarithmically as µ 0+. The inﬁnitesimal phase response curve may → be derived by taking the limit of small instantaneous perturbations of the voltage at times

between resets. Deﬁning the phase of the perturbation at time t as ϕ = (t t0)/T(µ) * * − mod 1, one can show that the iPRC is

1 ϕ ϕ 1 µ+2 − µ+2 − µ + µ 2 Z(ϕ, µ) = − . (3.40) µ+2 2ln µ

This quantity corresponds to the iPRC in the “unstable” direction for the planar homoclinic and heteroclinic iris systems. Because the QIF is a scalar model, there is no analog of

perturbations in the “stable” direction. If we evaluate the expression (3.40) at ϕ 1, ≡ corresponding to the “top” of the spike, we ﬁnd for all µ > 0 that Z(0, µ) = 0. For all other phases 0 ϕ < 1, the iPRC diverges to +∞ as ≤

(2/µ)1 ϕ Z(ϕ, µ) − (as µ 0+), (3.41) ∼ 2ln(2/µ) → consistent with the calculation for the iris system and the standard planar homoclinic. CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 134

System Zu(ϕ) Zs(ϕ)

ϕ (νx/∆)u Saddle Homoclinic (Brown et al., 2004) ulog(1/u) 0

ϕ (1 ϕ) u s − Iris Heteroclinic (u λs)log(1/u) (u λs)log(1/u) − − 1 ϕ ϕ 1 µ+2 − + µ+2 − 2 µ µ − QIF “Saddle” Homoclinic (Izhikevich, 2007) µ+2 - 2ln µ

Table 3.1: Phase response curve for the standard homoclinic system (as in Brown et al. (2004)), the quadratic integrate-and-ﬁre neuron, and the iris system, compared. For clarity, we write u,0 < u < 1, for u† = ε/∆, and the phase 0 ϕ 1 for both the iris and the standard homoclinic systems. Here s = uλ ; in the λ ≤∞ limit≤ s 0, and the systems → → coincide (up to a scaling factor νx/∆ related to the choice of perturbation direction). The iPRC for perturbations parallel to the unstable (resp. stable) eigenvector direction is given by Zu (resp. Zs). The perturbation in the “stable” direction is not deﬁned for the QIF model. See text for interpretation.

3.8.3 Qualitative comparison with a biological model: iPRCs for the

Morris–Lecar system

Preparations are underway in our laboratory to test the sensitivity of rhythmic feeding behavior in the marine mollusk Aplysia californica, in both intact animals and in a semi-intact suspended buccal mass preparation, to perturbations in the form of variable mechanical loading of the buccal mass during feeding. By using a servomotor to apply controlled forces to seaweed strands while the sea slug is ingesting them, it is possible to examine changes in the timing of the feeding rhythm in response to variable operating conditions. While the suspended buccal mass preparation includes the mouthparts and the ganglion comprising the feeding central pattern generator, much of the proprioceptive feedback from the periphery to the central circuit is removed. Preliminary data show that the CPG CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 135

of the reduced preparation produces qualitatively similar rhythms, but with longer dwell times at a sequence of quasiequilibria (Shaw et al., 2010), suggesting the hypothesis that the underlying CPG lies closer to a homoclinic or heteroclinic structure in the absence of peripheral feedback. How well can we expect the differential sensitivity of the simple model systems considered here to compare with phase response curves in real biological systems operating near heteroclinic, or, more generically, homoclinic orbits? As a ﬁrst step towards answering this question, we studied the inﬁnitesimal phase response curves for a family of limit cycles verging on a homoclinic orbit in the classical Morris–Lecar equations (Morris and Lecar, 1981; Rinzel and Ermentrout, 1989). These equations describe the evolution of the voltage v and potassium gate w, and include a fast calcium current with gating variable m as well:

dv 1 = (Iapp gCam∞(v)(v vCa) gKw(v vK) gL(v vL)), (3.42) dt C − − − − − − dw = (w∞(v) w)/τ(v). (3.43) dt −

The terms m∞,w∞ and τ satisfy (for convenience we deﬁne ξ = (v vc)/vd) − 1 v va m∞ = 1 + tanh − , (3.44) 2 vb

w∞(v) = (1 + tanhξ)/2, (3.45)

τ(v) = 1/(φ cosh(ξ/2)). (3.46) CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 136

We used standard parameters from (Ermentrout and Terman, 2010b), near the bifurcation of a stable limit cycle to a saddle homoclinic orbit:

vK = 84,vL = 60,vCa = 120, (3.47) − −

gK = 8,gL = 2,C = 20, (3.48)

va = 1.2,vb = 18, (3.49) −

vc = 12,vd = 17.4,φ = 0.23,gCa = 4, (3.50)

and studied the inﬁnitesimal phase response curve numerically for various applied current values the applied current, Iapp. Figure 3.11 shows the numerically computed limit cycle tra-

jectory for the Morris–Lecar system, for applied currents Iapp 40,36.2,35.05,35.009 . ∈ { } For a modest injected current (Iapp = 40) the model produces spikes with a period of T = 24.0 time units, and the iPRC is smooth and small, with values ranging from

0.6 to 0.4 (Figure 3.12, bottom). As Iapp decreases, the interval between spikes in- − creases, with T = 39.0 for Iapp = 36.2, T = 75.25 for Iapp = 35.05, and T = 110.25 for

Iapp = 35.009. The limit cycle undergoes a saddle-homoclinic bifurcation at approximately

Iapp = I 35.0067, at which point T ∞. * ≈ → We obtained numerical estimates of the iPRC by applying small perturbations at different phases, with customized codes using open source tools (the scipy/numpy numerical libraries(Oliphant, 2006; Jones et al., 2001)). As the applied current is reduced towards this

point, the iPRC grows in magnitude, reaching a range from -100 to +75 for Iapp = 35.009. This dramatic increase in sensitivity is qualitatively consistent with our results, as well as with previous analyses of phase response curves near a homoclinic bifurcation ((Brown et al., 2004), and (Izhikevich, 2007), online 10.4.3 and Figure 8.3). However, we hasten S CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 137

20 0.45 15 0.40 10 0.35 5 0.30 0 0.25 5 0.20 −10 0.15 −15 0.10 −20 0.05 −25 0.00 − 0 50 100 150 200 250 300

20 0.45 15 0.40 10 0.35 5 0.30 0 0.25 5 0.20 −10 0.15 −15 0.10 −20 0.05 −25 0.00 − 0 50 100 150 200

20 0.45 15 0.40 10 0.35 5 0.30 0 0.25 5 0.20 −10 0.15 −15 0.10 −20 0.05 −25 0.00 − 0 20 40 60 80 100

20 0.45 15 0.40 10 0.35 5 0.30 0 0.25 5 0.20 −10 0.15 −15 0.10 −20 0.05 −25 0.00 − 0 10 20 30 40 50 60 70

Figure 3.11: Time plots of limit cycle trajectories of the Morris–Lecar system with various values of Iapp. Left: voltage v and recovery variable w as functions of time (v: black trace, left scale. w: gray trace, right scale). Right: limit cycle trajectories in the (v,w) phase plane. Small circles indicate unstable ﬁxed points (unstable nodes inside the limit cycles, and hyperbolic saddle ﬁxed points outside). When Iapp & I 35.0067 the time plots exhibit prolonged dwell times near the hyperbolic saddle (small* ≈ circle near the bottom left of the limit cycle in the phase plane plot). As Iapp increases, the limit cycle moves away from the saddle point and the speed becomes more uniform. From top to bottom Iapp = 35.009,35.05,36.2,40. Compare Figures 3.2 and 3.5. CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 138 to point out that the extrema of the iPRCs plotted in Figure 3.12 appear to occur at phases

ϕ lying outside a small neighborhood of the saddle point. For this system we deﬁne the zero phase as the phase corresponding to the peak of the voltage spike, hence the portion of the trajectory near the saddle point corresponds roughly to the broad ﬂat region e.g.

0.2 . ϕ . 0.9 in the top panel of Figure 3.12. If we define a small neighborhood U enclosing the saddle point, it may be that the phase response diverges for phases ϕ such that (v(t(ϕ)),w(t(ϕ))) U, but our numerical results do not give strong evidence for or ∈ against this claim. It is interesting to compare this situation with the iPRC for the quadratic IF (integrate- and-fire) model ( 3.8.2). Referring to expression (3.40), we see that for any fixed value of S µ > 0, the greatest sensitivity to perturbation in the QIF model occurs immediately following the reset. Indeed, it is easy to show, for 0 ϕ < 1 and µ > 0, that ∂Z(ϕ, µ)/∂ϕ < 0. In ≤ 10 of (Izhikevich, 2007) (the online portion of the text), Figure 10.39 shows that the iPRC S predicted by the QIF model diverges from that obtained numerically for a planar conductance based model, with the greatest discrepancy between the two occurring immediately following the peak of the spike. Clearly, there is an opportunity for additional analysis of the asymptotic behavior of the sensitivity to perturbations in saddle-node homoclinic systems such as occur in the Morris–Lecar equations. CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 139

150 100 50 0 50 − 100 − 0.0 0.2 0.4 0.6 0.8 1.0

10 8 6 4 2 0 2 −4 −6 − 0.0 0.2 0.4 0.6 0.8 1.0

0.8 0.6 0.4 0.2 0.0 0.2 −0.4 −0.6 − 0.0 0.2 0.4 0.6 0.8 1.0

0.6 0.4 0.2 0.0 0.2 − 0.4 − 0.0 0.2 0.4 0.6 0.8 1.0

Figure 3.12: Infinitesimal phase response curves (iPRC) for the Morris–Lecar system (Equations 3.42-3.50) with various values of the applied current Iapp. Left: iPRC for voltage perturbations. The phase is normalized to the scale 0 φ 1, with φ = 0 set to the maximum of the voltage trajectory. Right: trajectories in the≤ (v,w)≤ plane, with unstable fixed points denoted by open circles, as in Figure 3.11. Trajectories move counterclockwise about the interior fixed point. From bottom to top: Iapp = 40.0,36.2,35.05, and 35.009. The homoclinic bifurcation occurs at approximately Iapp = 35.0067 (equations integrated numerically with a Runge-Kutta scheme in XPP). As the applied current lessens, the limit cycle trajectory passes closer to the saddle point, and the iPRC grows larger. However, the most rapid growth in the iPRC appears to occur for phases away from the saddle (phases near φ = 0) even as the system spends a growing fraction of time in small neighborhoods of the saddle. The saddle point is absent for Iapp = 40. CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 140

3.8.4 Stability of synchronous solutions for two iris systems with diffusive

coupling

The inﬁnitesimal phase response curve is of interest for several reasons. One major application is analysis of synchronization behavior of coupled oscillators (Ermentrout and Kopell, 1984). Dissipative or diffusive coupling – for instance, coupling two oscillatory model neurons through a resistive linkage such as a gap junction – can lead to counter- intuitive effects. Sherman and Rinzel (Sherman and Rinzel, 1992) showed that while strong electrical coupling of bursting cells leads, as expected, to synchronous spiking within bursts, weak electrical coupling leads to antiphase synchrony of spikes within bursts. Moreover, this “microscopic” effect on spike timing leads to “macroscopic” effects such as extended burst duration, because appearance of the homoclinic trajectory in the slow variable, providing the mechanism for burst termination, was delayed by the shift in the trajectories caused by electrotonic coupling (De Vries et al., 1998; Sherman and Rinzel, 1992). The effect of weak coupling on convergence or divergence of nearby trajectories in pairs of cells typically varies around the limit cycle representing the oscillating uncoupled cell (Kopell and Ermentrout, 2002); the coupling function (Equation 3.60, below) obtained by averaging around the limit cycle accounts for the net effect. A similar situation can arise when coupling two systems each of which follows a limit cycle that passes near a homoclinic orbit ((Kopell and Ermentrout, 2002), Figure 2.8). Consequently, it is interesting to investigate the stability of the synchronized solution for two iris systems coupled with diffusive coupling in one of the coordinates, for instance the horizontal coordinate, x. Consider an uncoupled iris system with a stable limit cycle CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 141 described by a pair of ordinary differential equations

x˙ = F(x,y;a), (3.51)

y˙ = G(x,y;a), (3.52) where a > 0 is the bifurcation parameter (the gap between the unstable and stable manifolds, W u and W s for successive saddles). As derived above, the limit cycle (LC) corresponds to

an ingress point u relative to any given square, and egress point s = uλ , where λ > 1 is the saddle value. The ingress point for the LC satisﬁes

uλ u + a = 0. (3.53) −

For instance, when λ = 2 and a = 0.01 we have u 0.0102... and s 1.0205 10 4. ≈ ≈ × − The period of the LC is T = 4ln(1/u), and we deﬁne the phase as φ(t) = 4t/T, with zero phase chosen to be the point of ingress to the lower left square. Hence we have φ [0,1) ∈ for the ﬁrst (lower left) square, φ [1,2) for the second square, φ [2,3) for the third ∈ ∈ square, and φ [3,4) for the last square. The iPRC for perturbations in the horizontal ∈ direction is given by   Ks1 φ , φ [0,1),  −  ∈  φ 1  Ku − , φ [1,2), Z(φ) = ∈ (3.54) 3 φ  Ks − , φ [2,3),  − ∈   Kuφ 3, φ [3,4), − − ∈ where K > 0 is a constant that is common to all terms, namely K = [ln(1/u)(u λs)] 1 > 0. − − CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 142

1 0.5 0

iPRC -0.5 -1 0 0.5 1 1.5 2 2.5 3 3.5 4 1 0.5 ) φ 0 x( -0.5 -1 0 0.5 1 1.5 2 2.5 3 3.5 4 3 2 φ 1 0

dx/d -1 -2 -3 0 0.5 1 1.5 2 2.5 3 3.5 4 Phase φ ∈ [0, 4)

Figure 3.13: Inﬁnitesimal phase response curve and trajectory for the iris system (x- component) with twist parameter a = 0.02. Top: the x-component of the iPRC, Zx(φ) (neglecting a positive constant K). Middle: the x-component of the trajectory, x(φ) (blue) and the same trajectory lagged by ∆φ = 0.1 (red). Bottom: the derivative dx/dφ(φ). CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 143

A pair of iris systems with diffusive coupling is now described by four equations:

x˙1 = F(x1,y1;a) + k(x2 x1), (3.55) −

x˙2 = F(x2,y2;a) + k(x1 x2), (3.56) −

y˙1 = G(x1,y1;a), (3.57)

y˙2 = G(x2,y2;a). (3.58)

If x = u(t),y = v(t) is a stable LC solution for an uncoupled iris system (for a > 0), we wish

to know whether or not for k > 0 the synchronous solution x1 = x2 = u(t),y1 = y2 = v(t) is stable. Following standard weak coupling arguments (Ermentrout and Terman, 2010b) we can represent the solutions to the coupled systems in terms of their phases as deﬁned for

the uncoupled system, x(φ) and y(φ). The two coupled systems with a (small) phase

difference χ = φ2 φ1 evolve according to −

χ˙ = H( χ) H(χ) + o(χ), (3.59) − − where the H-function is deﬁned as

k Z 4 H(χ) = Z(φ)(x(φ + χ) x(φ))dφ. (3.60) 4 φ=0 −

The roots of the expression H( χ) H(χ) = 0 are ﬁxed points for the evolution of χ, and − − χ = 0 corresponds to the synchronous solution. The stability of the synchronous solution depends on the sign of d/dχ(H( χ) H(χ)) = 2H (0) at χ = 0. If this quantity is − − − ′ negative, we have stable synchrony. If it is positive, synchrony is unstable. CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 144

For the piecewise linear iris system, we have explicit expressions for the trajectory

x(φ), using s = uλ :

   u s φ  −2 (1 s ), φ [0,1);  − − ∈    s u 2 φ  −2 (1 u − ), φ [1,2); x(φ) = − − ∈ (3.61)   s u φ 2  −2 + (1 s − ), φ [2,3);  − ∈    u s 4 φ  − + (1 u ), φ [3,4).  2 − − ∈

Figure 3.13 shows the trajectory x(φ), iPRC (x component) Zx(φ), and derivative of the trajectory dx/dφ, for a = 0.02. It is straightforward to obtain the necessary derivative at χ = 0, namely

k Z 4 k Z 4 dx H′(0) = Z(φ) x′(φ + χ) = Z(φ) (φ)dφ. (3.62) 4 φ=0 χ=0 4 0 dφ

The derivatives of the trajectory x with respect to the phase φ are:

  sφ lns, φ [0,1);   ∈  2 φ dx  u − lnu, φ [1,2); = − ∈ (3.63) dφ φ 2  s − lns, φ [2,3);  − ∈   u4 φ lnu, φ [3,4). − ∈

Remarkably, the φ dependence drops out of the integrands, giving the following (recalling CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 145 that K = [ln(1/u)(u λs)] 1 > 0, and s = uλ ): − − φ IZ(φ) dx Z(φ) dx R Z(φ) dx (φ) dφ ∈ dφ dφ I dφ 1 φ φ I = [0,1) Ks − s lns Kslns Kslns I = [1,2) Kuφ 1 u2 φ lnu Kulnu Kulnu − − − − − I = [2,3) Ks3 φ sφ 2 lns Kslns Kslns − − − − I = [3,4) Kuφ 3 u4 φ lnu Kulnu Kulnu. − − − − −

Therefore, k k λslnu ulnu k H′(0) = K (2slns 2ulnu) = − = . (3.64) 4 − 2 ln(1/u)(u λs) 2 − Thus for k > 0 this result indicates stabilization of the synchronous solution, for this system, under weak diffusive coupling along the x-direction. This result was conﬁrmed by numerical simulations.

Remark 14. While the calculation above holds at a ﬁnite distance from the heteroclinic

bifurcation (i.e. setting the twist parameter a > 0), the results do not appear to carry over to the heteroclinic bifurcation point (a = 0). Indeed, for the piecewise linear iris system, in this case the unperturbed trajectories coincide with the stable/unstable manifolds of each saddle. Because of the system geometry, diffusive coupling in the x coordinate alone will cause nearby trajectories to approach each other if they are separated in phase, but they will never leave the orbit connecting two adjacent saddles, and hence remain

“synchronized”. On the other hand, if one trajectory is perturbed off the uncoupled orbit by a small amount towards the interior of the heteroclinic cycle, that trajectory will pass the next saddle point in ﬁnite time. If it only exerts a perturbation parallel to the horizontal direction, say, on the other trajectory, it will not help the other get around the corner. Therefore one trajectory will orbit near the heteroclinic path while the other trajectory CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 146

oscillates back and forth on the line connecting two horizontally displaced saddles. This situation will hold for any transverse initial perturbation (near the zero phase point, for example) no matter how small, and hence the diffusive coupling does not stabilize the “synchronous” solution in this case.

This situation is dependent on the particular geometry of the iris system, and would not necessarily obtain in systems with different geometry. The coupling could, for instance, perturb trajectories along a different direction. For the Morris–Lecar system near the homoclinic bifurcation, one can apply a change of coordinates at the saddle point with respect to which the stable and unstable eigenvectors become orthogonal. In the new coordinates, “diffusive” coupling mediated by the voltage difference acts along a direction

at some angle with respect to the stable and unstable eigenvectors, ξs and ξu. Whether diffusive coupling stabilizes or destabilizes the synchronous solution then depends on the detailed geometry of the trajectories near the homoclinic. If we consider diffusive-like coupling in an arbitrary direction for a pair of identical iris systems, when they are located at the heteroclinic point we have an evolution given by 24 sets of linear maps in subsets of

R4. Intuitively, many complicated and interesting trajectories both periodic and aperiodic, could occur.

3.8.5 Generalization to higher dimensional systems

n Consider a smooth dynamical system dynamical system in R , x˙ = F(x, µ). Suppose there

are constants µ < 0 < µ+ and an interval I = µ µ < µ < µ+ for which the following − { | − } assumptions hold:

For all µ I, there is a hyperbolic saddle ﬁxed point at the origin, F(0, µ) = 0, with ∙ ∈ one dimensional unstable manifold W u and (n 1) dimensional stable manifold W s. − CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 147

For µ > 0 and µ I, there is a smooth family of orbitally stable limit cycles colliding ∙ ∈ transversely with a saddle homoclinic loop at the origin, at µ = 0.

For all µ 0, µ I, the single unstable eigenvalue λu(µ) > 0 is smaller than λs(µ), ∙ ≥ ∈ − where λs is the stable eigenvalue with negative real part closest to zero.

n For all µ I, there is an open neighborhood U R of the origin within which the ∙ ∈ ⊂ flow is “sufficiently close” (in some appropriate sense) to the linear flow given by

x˙ = DF(0, µ) x. ·

Suppose we deﬁne, for this system, “zero phase” to be the phase at which the limit cycle enters the (ﬁxed, small) neighborhood U enclosing the origin. Under the hypotheses above, it would not be unreasonable to anticipate the following:

The inﬁnitesimal phase response curve Zu(φ; µ) for perturbations parallel to the un- ∙ stable eigenvector direction diverges as µ 0+ for all phases φ with corresponding → trajectory location x(φ) U. ∈

n There will be a subspace S1 of R tangent to x = 0, and a phase φ1, such that ∙ the inﬁnitesimal phase response curve Z1(φ; µ) for perturbations restricted to S1

+ will converge to zero as µ 0 for phases φ (0,φ1]. For phases φ > φ1 we → ∈ would anticipate the existence of perturbations within S1 that lead to divergent phase responses.

There may be additional subspaces Sk 1 Sk and phases φk 1 > φk with the property ∙ + ⊂ + that the inﬁnitesimal phase response curve Zk(φ, µ) for perturbations restricted to

+ the subspace Sk will converge to zero for phases φ (0,φk], as µ 0 . For phases ∈ → + φ > φk we expect that perturbations within Sk Sk 1 will diverge as µ 0 . ∖ + → CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 148

If this intuition should be borne out, it would indicate the existence of a nested hierarchy of subspaces at the saddle point, corresponding to greater and greater ability to “absorb” perturbation applied in particular directions. The reasoning behind this conjecture is that

near a saddle point, the (n 1) dimensional stable manifold will be converging towards − x = 0 at different exponential rates along different eigendirections of the linearization DF(0, µ). Perturbations in directions corresponding to more rapidly decaying components, i.e. those with larger negative eigenvalues, will be damped more quickly, and therefore will be less destabilizing, than perturbations in other more unstable directions.

3.8.6 Phase resetting in the absence of an asymptotic phase

The ability to produce a wide variety of stable phase relationships (intermediate phase locked states, cf. (Urban and Ermentrout, 2011)) is an important aspect of central pattern generators, and may be important for their adaptive control. As an example, Szucs¨ and colleagues (Szucs¨ et al., 2009) observed synchronization of pairs of central pattern generators in separate animals coupled artificially through a variety of configurations using extended dynamic clamp (Pinto et al., 2001). Simulated diffusive (electrical) coupling led to synchronization only for sufficiently large simulated conductances, while simulated contralateral inhibition from lateral pyloric (LP) to pyloric dilator (PD) pacemaker neurons was more effective at inducing 1:1 synchronized phase locking, as was mutual inhibition between the PD neurons. Mutually inhibitory reciprocal synaptic connections between bursting cells can be one mechanism for generating stable heteroclinic sequences (Nowotny and Rabinovich, 2007). Prolonged intervals separating switches between distinct phase- locked states was also studied in terms of a heteroclinic loop in a coupled oscillator system in (Kori and Kuramoto, 2001). Although many applications of phase response curve calculations focus on coupling, CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 149

the iPRC is a helpful conceptual tool in other settings as well. For instance, Nadim et al. showed that inhibitory feedback within a CPG served to reduce the variability in duration of a specific burst phase by flattening the iPRC, providing a mechanism for control and phase stabilization of a central pattern generator (Nadim et al., 2011). Deterministic limit cycles are not the only way to represent (approximately) periodic biological rhythms. Alternatives include heteroclinic or homoclinic attractors perturbed by small amplitude noise (Bakhtin, 2011; Stone and Holmes, 1990) as well as fixed points of spiral sink type subject to small noisy perturbations (Boland et al., 2008; Izhikevich, 2007). A system comprising a determinstic flow with a stable spiral fixed point, when perturbed by small to modest amounts of noise, will show noisy oscillatory trajectories that can be difficult to distinguish from a small amplitude limit cycle (Boland et al., 2008). For example, the small oscillations in membrane potential of a nerve cell brought near to firing by a steady depolarizing current of modest size may be due either to “noisy spiral sink” dynamics or to “noisy small limit cycle” dynamics (Stiefel et al., 2010). The classical definition of asymptotic phase breaks down for noisy systems, for spiral sinks, and for heteroclinic or homoclinic orbits. Nevertheless “phase resetting” is actively studied in such systems both experimentally (Ermentrout and Saunders, 2006; Ermentrout et al., 2008; Stiefel et al., 2010, 2008) and theoretically (Ichinose et al., 1998; Rabinovitch and Rogachevskii, 1999; Rabinovitch et al., 1994). For the Bonhoeffer–van der Pol equations, for example, Rabinovitch and colleagues showed how to define an analog of the classical isochron, for points converging to a spiral sink along a distinguished trajectory (a “T-attractor”) (Rabinovitch and Rogachevskii, 1999) (see also (Ichinose et al., 1998)). Generally speaking, noise effects can expose the structure of bifurcations in neural systems through stochastic resonance (Gai et al., 2010) or spike time bifurcations (Thomas et al., 2003; Toups et al., 2011), and it would be useful to have a broader theory of “phase CHAPTER 3. PHASE RESETTING IN A PHASELESS SYSTEM 150 response” going beyond the case of deterministic limit cycles. Chapter 4

Conclusion and Future Directions

151 CHAPTER 4. CONCLUSION AND FUTURE DIRECTIONS 152

In this chapter, we will brieﬂy review the results of the previous chapters and then discuss possible future directions for this work.

4.1 Review of previous chapters

In this dissertation, we have tested the hypothesis that swallowing behavior in Aplysia californica can be better modeled by a system whose neural dynamics contain a stable heteroclinic channel than by one whose dynamics contain a more homogeneous limit cycle. In chapter1, we provided the background material needed to understand the hypothesis and its context. We briefly reviewed the history of central pattern generator and chain reflex theory. We then provided a brief review of parts of dynamical systems theory that we used in later chapters, in particular limit cycles, which are often used to model central pattern generators, and stable heteroclinic channels, whose properties may lie between the predictions of a central pattern generator and a chain reflex. In chapter2, in order to explore phenomena that could be used to distinguish a pattern generator with a stable heteroclinic channel from one with a limit cycle, we developed a model of swallowing in Aplysia californica that could be simulated in both a parameter regime where it behaved as a stable limit cycle, resembling an ideal central pattern generator, and in a regime where it behaved as a stable heteroclinic channel, which is somewhat closer to a chain reflex. We showed that, when the strength of sensory input is small, the heteroclinic channel performs much better than the limit cycle over a range of mechanical loads. While tuning the limit cycle’s timing to closely match that of the stable heteroclinic channel improved the limit cycle’s performance, only by increasing the muscle strength/activation could the limit cycle perform as well as the stable heteroclinic channel. The increase in muscle strength, however, resulted in a greater metabolic cost CHAPTER 4. CONCLUSION AND FUTURE DIRECTIONS 153

and lower mechanical efficiency for the limit cycle for an equivalent degree of behavioral efficacy than the stable heteroclinic channel. In order to find which model better matched the behavior of the animal, we next looked for measurable differences between the model in the limit cycle regime and the model in the stable heteroclinic regime. Noting the greater efficacy of the stable heteroclinic channel to adapt to increased loads, we found that retraction duration was significantly increased with increasing load in the stable heteroclinic channel, but not the limit cycle. Because it is difficult to present a uniform load to the animal while it is actively consuming seaweed, we then showed a similar prolongation of retraction when the seaweed was “held” by adding a spring to the model. We then examined recordings from the intact animal and saw that, when the seaweed was held, the duration of retraction increased, matching what was seen with the stable heteroclinic channel but not with the limit cycle. Because this prolongation of retraction might be expected as a result of adaptive pressure, we next examined one of the assumptions of an ideal central pattern generator more directly: that the isolated nervous system is capable of generating the timing of the behavior without any sensory input bearing timing information from the periphery. Thus, as expected, the duration of protraction and retraction in the limit cycle changed very little when sensory input was removed. The stable heteroclinic channel, however, produced much longer protraction and retraction phases when sensory input was removed. In the animal, increasingly reduced preparations showed significant increases in both protraction and retraction duration, thus matching the behavior of the stable heteroclinic channel but not the limit cycle. In addition, the lengthening of both protraction and retraction differed from the selective lengthening of retraction when the seaweed was held, suggesting that this is a different phenomenon and not just another trigger for the same response seen with the held seaweed. CHAPTER 4. CONCLUSION AND FUTURE DIRECTIONS 154

Finally, we examined the shape of distribution of retraction times expected in the presence of small amounts of noise. In the limit cycle with small amounts of noise, retraction durations had a relatively Gaussian distribution, with little skew. By comparison, the retraction times of the stable heteroclinic channel showed a much more skewed distribution, as one would expect from Stone and Holmes(1990). When examined in the intact animal during swallowing, retraction times showed a skewed distribution, again more closely matching the predictions of the stable heteroclinic channel than the stable limit cycle. In chapter3, we focused on the selective sensitivity we had seen with the stable heteroclinic channel in the previous chapter and attempted to sharpen our understanding of this phenomenon in order to find additional signatures of a heteroclinic channel that we might be able test experimentally. Many of the tools used to study the sensitivity of oscillators make assumptions that initially do not seem compatible with a stable heteroclinic cycle, such as the assumption of a well defined phase. By instead studying the heteroclinic channel in the form of a family of limit cycles near a heteroclinic bifurcation, however, we were able to apply these tools in a novel way to understand the dynamics. We first developed a smooth planar system with four saddles forming a heteroclinic cycle, and then demonstrated that a twist of the vector field could break the cycle and produce a family of stable limit cycles. We reviewed the behavior of the limit cycles as the bifurcation parameter was changed, showing that they spent increasing amounts of time near the saddles as they approached the heteroclinic bifurcation. To develop a more analytically tractable system, we then linearized the region around each saddle and showed that by changing an offset between the linearized regions, we could create a family of limit cycles whose behavior was qualitatively similar to the behavior seen in the smooth system as the system approached the heteroclinic limit. In CHAPTER 4. CONCLUSION AND FUTURE DIRECTIONS 155 the piecewise linear system, however, we were able to find a closed-form solution for the trajectories within each region, which we were not able to do in the smooth system. Using this solution, we produced a map from the position and time where the trajectory entered a square to the position and time that it exited a square. We then used this map to prove the existence of a stable limit cycle for sufficiently small positive values of the offset parameter and dissipative saddle values. In addition, we generated the two dimensional bifurcation diagram showing the qualitative behavior of the system with any combination of offset and saddle values. We then examined the sensitivity of this stable limit cycle to perturbations at different times during the cycle by finding the infinitesimal phase response curve. By computing the effects of a perturbation on the exit position and time, we were then able to use the maps from entry and exit position to generate a recurrence relation for the entry position in later passages through squares. We then found a closed form for this recurrence relationship, and used this to calculate the time spent by the perturbed trajectory in each passage through a square relative to the time spent by the limit cycle trajectory. Finally, we were able to calculate the asymptotic advancement or delay caused by the perturbation by computing the (infinite) summation of all of these differences in time spent traversing the squares.1 Now that we had a closed form solution for the infinitesimal phase response curve for a limit cycle with an arbitrary offset, we then examined how the phase response curve changes as the system approaches the heteroclinic limit. We showed that the phase response curve developed very large peaks in sensitivity at the transitions between squares, and not near the saddle itself. We further showed that there was a direction where the system was insensitive to perturbations until it reached a critical phase. Except for this

1 Subsequently, our lab discovered a means of calculating the iPRC directly by solving an adjoint equation with a matched boundary condition (Park, 2013). CHAPTER 4. CONCLUSION AND FUTURE DIRECTIONS 156

region of sensitivity, however, the phase response curve diverged everywhere, reflecting the divergence of the period as the heteroclinic bifurcation was approached. We numerically found the infinitesimal phase response curve for the original smooth system, and we found that it showed a similar formation of sharp peaks in the transit between saddles, suggesting that this result was not an artifact of the discontinuous flow between regions in the piecewise linear system. We also numerically found the isochrons for the piecewise linear system, which as expected showed increased density near the boundaries between squares, so that a small perturbation could push this system across many isochrons.

4.2 Future directions

The results of chapter3 suggest another signature of a stable heteroclinic channel as a pattern generator that could be verified in the laboratory. It is possible to experimentally measure the phase response curve, for example using the approach in Phoka et al.(2010). Thus a natural next step would be to measure the phase response curve of the B63 group to small depolarizations of B64 (i.e. perturbations in the direction of the next saddle). If the results from chapter3 hold, we would expect to see a sharp peak in the phase response curve near the onset of B63 activity. This same phenomenon can be tested in the model from chapter2. In figures 4.1 and 4.2, we show the (numerically derived) phase response curve for the model in the stable limit cycle and stable heteroclinic channel parameter regimes. Note that, as seen in chapter3, the peaks are earlier, larger, and sharper in the stable heteroclinic channel regime. In these figures, note that there are hints of a relation between the phase response curves of the neural pools and those of the mechanical system. This makes intuitive sense: CHAPTER 4. CONCLUSION AND FUTURE DIRECTIONS 157 0.0e+00 0 −1.0e−05 a −2.0e−05

0.0 0.2 0.4 0.6 0.8 1.0 0.0e+00 1 −1.0e−05 a −2.0e−05

0.0 0.2 0.4 0.6 0.8 1.0 0e+00 −2e−06 2 a −4e−06 −6e−06

0.0 0.2 0.4 0.6 0.8 1.0 3e−04 2e−04 r x 1e−04 0e+00 −1e−04 0.0 0.2 0.4 0.6 0.8 1.0 Phase of perturbation

Figure 4.1: Phase response curve for the model in chapter2 in the limit cycle regime. The horizontal axis shows the phase at which the perturbation was applied, and the vertical axis shows the resulting change in asymptotic phase. The perturbation consisted of an 6 instantaneous increase in the state variable by 10− for a0, a1, and a2, and 0.05 for xr. CHAPTER 4. CONCLUSION AND FUTURE DIRECTIONS 158 −0.001 0 a −0.003 −0.005

0.0 0.2 0.4 0.6 0.8 1.0 0.000 −0.002 1 −0.004 a −0.006 −0.008

0.0 0.2 0.4 0.6 0.8 1.0 −0.00228 2 a −0.00232 −0.00236 0.0 0.2 0.4 0.6 0.8 1.0 0.02 0.01 0.00 r x −0.01 −0.03

0.0 0.2 0.4 0.6 0.8 1.0 Phase of perturbation

Figure 4.2: Phase response curve for the model in in chapter2 in the heteroclinic channel regime. The horizontal axis shows the phase at which the perturbation was applied, and the vertical axis shows the resulting change in asymptotic phase. The perturbation consisted 6 of an instantaneous increase in the state variable by 10− for a0, a1, and a2, and 0.05 for xr. Note the difference in vertical scale compared to figure 4.1. CHAPTER 4. CONCLUSION AND FUTURE DIRECTIONS 159 in the absence of compensation by the pattern generator, we might expect the effects of an instantaneous perturbation of position to decay away exponentially through time. This physical displacement results in a perturbation of sensory input to the neurons, and thus the overall mechanical phase response curve could be seen as a convolution of a weighted sum of the phase response curves of the individual neurons and an exponential decay. It may be possible to formalize this relation and provide a rigorous analysis of this relationship in the limit of weak sensory feedback. Note also that the phase response curves shown are for the coupled system. It is natural to ask whether the phase response curves in the coupled system can be estimated in the presence of weak sensory feedback for the behavior of the isolated system. In the case of stable heteroclinic channels, the lack of a clearly defined period introduces challenges, but the approaches of chapter3 provide tools that may be able to address these. A rigorous analysis of this relationship could be a fruitful area of research. Again, this relationship could be experimentally verified by measuring the phase response curves of neurons in the isolated system and the impulse response function in a reduced preparation, then verifying that the predicted phase response curve for physical perturbations matched the measured phase response curve. Noise is an ever present influence in biological systems, and we have only briefly examined its effects in this dissertation. Although the effects of noise on timing in heteroclinic cycles has been detailed by Stone and Holmes(1990), the change in this distribution as one moves away from the heteroclinic bifurcation in the heteroclinic channel to our knowledge has not been closely examined. The phase response curve from chapter3 may be useful in developing an expression for the shape of this distribution. Although very small amounts of noise are sufficient to prevent a heteroclinic cycle from becoming “stuck”, as seen in chapter2, heteroclinic channels can maintain robust CHAPTER 4. CONCLUSION AND FUTURE DIRECTIONS 160 1.0 0.8 0.6 0.4 0.2 0.0

270 275 280 285 290 295 300 Figure 4.3: Biological levels of noise do not corrupt the sequencing of the phases in the heteroclinic model from chapter2: protraction open (blue) followed by protraction closing (red) followed by retraction closed (gold). The noise η = 0.02; other parameters are as described in chapter2.

sequence of activity even in the presence of much larger amounts of noise. In ﬁgure 4.3, we show the model from chapter2 in the heteroclinic channel regime with a noise level of

η = 0.02; this is comparable to 0.5 mV RMS of noise2 in a single neuron whose average membrane potential changes over a 25 mV range during the behavior. Note that, although the period of the behavior has been shortened, the sequence of the activation of the motor pools is reliably maintained. Although the frequency of the behavior has been increased, it is probably possible to compensate for this change through parameter adjustments (for example, the strength of sensory feedback would need to be increased because at its current

strength of η = 0.001, its effects are likely to be drowned out by the noise). In the animal itself, the average noise over the entire pool of neurons corresponding to a saddle is likely to have a much smaller effect than in this single-cell example. The trade off between noise and proprioception in driving the behavior of a biological system is also potentially a question of interest. As can be seen in ﬁgure 4.4, the frequency of a behavior driven by a stable heteroclinic channel may be determined by sensory input, noise, or a combination of the two. In Aplysia, one might expect that the behavior of the

2comparable to the sub-threshold noise described in Purkinje neurons in Jacobson et al.(2005) CHAPTER 4. CONCLUSION AND FUTURE DIRECTIONS 161

intact animal is largely driven by sensory input, but that the isolated ganglia has a greater sensitivity to noise. This could be explored more thoroughly by applying a weak noisy electrical ﬁeld to the ganglia and seeing its effects on the timing behavior in both the intact animal and the isolated ganglia. The heteroclinic network we have explored suggests a surprisingly simple architecture for building heteroclinic channels - neuronal pools connected by mutually excitatory connections, and with inhibitory connections to other pools except for the next pool in the sequence (which may have weaker inhibition or no inhibition). In Aplysia, the B63 and B64 neuronal groups used in the model in chapter2 have these properties, and as we have seen their behavior is consistent with that of a heteroclinic channel. In the animal, however, there is often not a direct transition from B63 to B64 activity; instead the B64 burst appears to be terminated by another pool containing B52 (Nargeot et al., 2002) with inhibitory connections to B63. This could represent the next “saddle” in the sequence of behavior. It would be natural to test whether this model of the circuit is robust enough to be used to modify the dynamical structure of the pattern generator, as one might want to do in a clinical application. A potential experiment for doing so, shown schematically in ﬁgure 4.5, would be to use dynamic clamp to create an inhibitory connection between the neurons in the B52 motor group and those in the B63 motor group while simultaneously blocking the existing inhibition of the B52 motor group in an attempt to create a new heteroclinic connection that bypasses the retraction phase, essentially rewiring the pattern generator so that it skips over one phase of the pattern. If this experiment is successful, it suggests a very general method for adding or removing components of a neural pattern. If this experiment failed, the form of the failure would also be instructive. For example, if B64 activated despite the activation of B52, it could suggest that slow excitation rather CHAPTER 4. CONCLUSION AND FUTURE DIRECTIONS 162

10 0.250

0.225

0.200 20

0.175

0.150 ) ) η (

0.125 Log Noise (log 50

0.100

7010 9 8 7 6 5 4 3 2

Log proprioception strength (log(ε))

Figure 4.4: Trade off between proprioception and noise in a variant of the model from chapter2(Shaw et al., 2010). Shades of gray and contour lines show the frequency of bites in Hz. Note that there are three domains in the ﬁgure: in the upper left noise dominates, and changes in proprioception strength have very little effect. In the lower right, proprioception dominates, and changes in noise magnitude have very little effect. Along the diagonal, both noise and proprioception play a role in determining the frequency of the behavior and changes to either have an effect on the behavior. CHAPTER 4. CONCLUSION AND FUTURE DIRECTIONS 163

Figure 4.5: Schematic of saddle bypass experiment. Arrows show heteroclinic orbits between saddles. By using dynamic clamp to create artiﬁcial inhibitory synapses between neurons in the B52 group and neurons in the B63 group, it may be possible to create a new heteroclinic connection (red) such that the retraction phase of a carbachol pattern is bypassed.

than release from inhibition was responsible for the transition from protraction to retraction. Instead of a saddle point with maximal sensitivity near the beginning of a burst as seen in a stable heteroclinic channel, such dynamics might have greater sensitivity near the end of burst as one might see with a relaxation oscillator. This could be veriﬁed in the phase response curves from previous experiments, and if consistent with previous experiments it could provide a new direction for future work.

4.3 Conclusion

As the statistician George E. P. Box noted, “essentially, all models are wrong, but some are useful” (Box and Draper, 1987). In this dissertation, we have compared the properties of a central pattern generator designed around a limit cycle, as is common in the literature, to one designed around a stable heteroclinic channel. We have shown that the localized regions of sensitivity present in heteroclinic channels may provide adaptive properties for a pattern generator, and that a model based on a heteroclinic channel provides better predictions for swallowing behavior in Aplysia than a model based on a limit cycle. There is still much work to be done to develop the tools and experimental methodologies CHAPTER 4. CONCLUSION AND FUTURE DIRECTIONS 164 to explore this hypothesis in Aplysia and other systems, and potential applications to clinical interventions are still merely thought experiments. We hope, however, that we have convinced the reader that stable heteroclinic channels provide a useful alternative to limit cycles for understanding motor pattern generation, and that the next time the reader examines a motor behavior, he or she will see it in a different way. Bibliography

Afraimovich, V., Ordaz, F. C., and Urias, J. (2002). A class of cellular automata modeling winnerless competition. Chaos, 12(2):279–288. 86

Afraimovich, V., Tristan, I., Huerta, R., and Rabinovich, M. I. (2008). Winnerless competition principle and prediction of the transient dynamics in a Lotka-Volterra model. Chaos, 18(4):043103. 86

Afraimovich, V., Young, T., Muezzinoglu, M. K., and Rabinovich, M. I. (2011). Nonlinear dynamics of emotion-cognition interaction: When emotion does not destroy cognition? Bulletin of Mathematical Biology, 73(2):266–284. 21, 23, 40, 87

Afraimovich, V. S., Rabinovich, M. I., and Varona, P. (2004a). Heteroclinic contours in neural ensembles and the winnerless competition principle. International Journal of Bifurcation and Chaos, 14(04):1195–1208. 21, 40, 41, 86

Afraimovich, V. S., Zhigulin, V. P., and Rabinovich, M. I. (2004b). On the origin of reproducible sequential activity in neural circuits. Chaos: An Interdisciplinary Journal of Nonlinear Science, 14(4):1123. 21, 80, 86

Andronov, A. and Pontryagin, L. (1937). Systemes` grossiers. Dokl. Akad. Nauk. SSSR, 14:247–251. 87

165 BIBLIOGRAPHY 166

Armbruster, D., Stone, E., and Kirk, V. (2003). Noisy heteroclinic networks. Chaos, 13(1):71–79. 86

Bakhtin, Y. (2011). Noisy heteroclinic networks. Probability Theory and Related Fields, 150(1-2):1–42. 40, 149

Baxter, D. A. and Byrne, J. H. (2006). Feeding behavior of Aplysia: A model system for comparing cellular mechanisms of classical and operant conditioning. Learn. Mem., 13(6):669–680. 41

Beer, R. D., Chiel, H. J., and Gallagher, J. C. (1999). Evolution and analysis of model CPGs for walking: II. general principles and individual variability. J Comput Neurosci, 7(2):119–47. 86

Boland, R. P., Galla, T., and McKane, A. J. (2008). How limit cycles and quasi-cycles are related in systems with intrinsic noise. Journal of Statistical Mechanics: Theory and Experiment, 2008:P09001. 149

Boukal, D. S. and Krivan,ˇ V. (1999). Lyapunov functions for Lotka-Volterra predator-prey models with optimal foraging behavior. J. Math. Biology, 39:493–517. 98

Box, G. E. P. and Draper, N. R. (1987). Empirical model-building and response surfaces. Wiley, New York. 163

Brown, E., Moehlis, J., and Holmes, P. (2004). On the phase reduction and response dynamics of neural oscillator populations. Neural Comput, 16(4):673–715. 102, 124, 127, 131, 132, 134, 136

Brys, G., Hubert, M., and Struyf, A. (2004). A robust measure of skewness. Journal of Computational and Graphical Statistics, 13(4):996–1017. 75 BIBLIOGRAPHY 167

Buchanan, J. T. and Cohen, A. H. (1982). Activities of identified interneurons, mo- toneurons, and muscle fibers during fictive swimming in the lamprey and effects of reticulospinal and dorsal cell stimulation. J Neurophysiol, 47(5):948–60. 86

Buono, P. L. and Golubitsky, M. (2001). Models of central pattern generators for quadruped locomotion. I. primary gaits. J Math Biol, 42(4):291–326. 86

Butera, R. J., Clark, J. W., Byrne, J. H., and Rinzel, J. (1996). Dissection and reduction of a modeled bursting neuron. Journal of Computational Neuroscience, 3(3):199–223. 30, 79

Bassler,¨ U. (1986). On the deﬁnition of central pattern generator and its sensory control. Biological Cybernetics, 54(1):65–69. 35, 82

Cataldo, E., Byrne, J., and Baxter, D. (2006). Computational model of a central pattern generator. In Computational Methods in Systems Biology, volume 4210 of Lecture Notes in Computer Science, pages 242–256. Springer Berlin / Heidelberg. 41

Chiel, H. J., Beer, R. D., and Gallagher, J. C. (1999). Evolution and analysis of model CPGs for walking: I. dynamical modules. J Comput Neurosci, 7(2):99–118. 86

Chiel, H. J., Weiss, K. R., and Kupfermann, I. (1986). An identiﬁed histaminergic neuron modulates feeding motor circuitry in Aplysia. The Journal of neuroscience : the ofﬁcial journal of the Society for Neuroscience, 6(8):2427–50. 28

Chrachri, A. and Clarac, F. (1990). Fictive locomotion in the fourth thoracic ganglion of the crayﬁsh, Procambarus clarkii. The Journal of Neuroscience, 10(3):707–719. 16, 80

Church, P. J. and Lloyd, P. E. (1994). Activity of multiple identiﬁed motor neurons recorded BIBLIOGRAPHY 168

intracellularly during evoked feedinglike motor programs in Aplysia. J Neurophysiol, 72(4):1794–1809. 27, 28, 48

Cohen, A. H., Ermentrout, G. B., Kiemel, T., Kopell, N., Sigvardt, K. A., and Williams, T. L. (1992). Modelling of intersegmental coordination in the lamprey central pattern generator for locomotion. Trends Neurosci, 15(11):434–8. 86

Coombes, S. (2008). Neuronal networks with gap junctions: A study of piecewise linear planar neuron models. SIAM Journal on Applied Dynamical Systems, 7:1101–1129. 94

Coombes, S. and Bressloff, P. C., editors (2005). Bursting: The Genesis of Rhythm in the Nervous System. World Scientiﬁc. 86

Coombes, S. and Osbaldestin, A. H. (2000). Period-adding bifurcations and chaos in a periodically stimulated excitable neural relaxation oscillator. Phys Rev E Stat Phys Plasmas Fluids Relat Interdiscip Topics, 62(3 Pt B):4057–66. 94

Cullins, M. J. and Chiel, H. J. (2010). Electrode fabrication and implantation in Aplysia californica for multi-channel neural and muscular recordings in intact, freely behaving animals. Journal of Visualized Experiments, (40). 28, 56

D’Agostino, R. B., Belanger, A., and D’Agostino, R. B. (1990). A suggestion for using powerful and informative tests of normality. The American Statistician, 44(4):316–321. 75, 76, 77

De Schutter, E., Angstadt, J. D., and Calabrese, R. L. (1993). A model of graded synaptic transmission for use in dynamic network simulations. J Neurophysiol, 69(4):1225–35. 86 BIBLIOGRAPHY 169

De Vries, G., Sherman, A., and Zhu, H. R. (1998). Diffusively coupled bursters: effects of cell heterogeneity. Bull Math Biol, 60(6):1167–200. 140

Denaro, N., Merlano, M. C., and Russi, E. G. (2013). Dysphagia in head and neck cancer patients: Pretreatment evaluation, predictive factors, and assessment during radio-chemotherapy, recommendations. Clinical and experimental otorhinolaryngology, 6(3):117–126. 14

Denny, M. and Gaylord, B. (2002). The mechanics of wave-swept algae. Journal of Experimental Biology, 205(10):1355–1362. 78

Drushel, R. F., Sutton, G. P., Neustadter, D. M., Mangan, E. V., Adams, B. W., Crago, P. E., and Chiel, H. J. (2002). Radula-centric and odontophore-centric kinematic models of swallowing in Aplysia californica. J Exp Biol, 205(14):2029–2051. 31

Edwards, R. and Glass, L. (2000). Combinatorial explosion in model gene networks. Chaos, 10(3):691–704. 94

Ermentrout, B. and Saunders, D. (2006). Phase resetting and coupling of noisy neural oscillators. J Comput Neurosci, 20(2):179–90. 149

Ermentrout, B. and Terman, D. H. (2010a). Mathematical foundations of neuroscience. Springer, New York. 26

Ermentrout, G. B., Galan,´ R. F., and Urban, N. N. (2008). Reliability, synchrony and noise. Trends Neurosci, 31(8):428–34. 149

Ermentrout, G. B. and Kopell, N. (1984). Frequency plateaus in a chain of weakly coupled oscillators, I. SIAM J. on Mathematical Analysis, 15(2):215–237. 140 BIBLIOGRAPHY 170

Ermentrout, G. B. and Terman, D. H. (2010b). Foundations Of Mathematical Neuroscience. Springer. 93, 124, 136, 143

Evans, C. G. and Cropper, E. C. (1998). Proprioceptive input to feeding motor programs in Aplysia. J. Neurosci., 18(19):8016–8031. 78

Feldman, A. G. (1966). Functional tuning of the nervous system with control of movement or maintenance of a steady posture. II. Controllable parameters of the muscle. Biophysics, 11(3):565–578. 82

Filippov, A. F. (1988). Differential Equations with Discontinuous Righthand Sides. Mathe- matics and Its Applications. Kluwer Academic Publishers, Dordrecht, The Netherlands. 98

Forssberg, H. (1979). Stumbling corrective reaction: a phase-dependent compensatory reaction during locomotion. Journal of Neurophysiology, 42(4):936–953. 14, 34

Forssberg, H., Grillner, S., and Rossignol, S. (1975). Phase dependent reﬂex reversal during walking in chronic spinal cats. Brain Research, 85(1):103–107. 34

Foster, W. R., Ungar, L. H., and Schwaber, J. S. (1993). Signiﬁcance of conductances in Hodgkin-Huxley models. Journal of Neurophysiology, 70(6):2502–2518. 79

Fox, L. E. and Lloyd, P. E. (1997). Serotonin and the small cardioactive peptides differen- tially modulate two motor neurons that innervate the same muscle ﬁbers in Aplysia. The Journal of Neuroscience, 17(16):6064–6074. 48

Gai, Y., Doiron, B., and Rinzel, J. (2010). Slope-based stochastic resonance: how noise enables phasic neurons to encode slow signals. PLoS Comput Biol, 6(6):e1000825. 149 BIBLIOGRAPHY 171

Glass, L., Nagai, Y., Hall, K., Talajic, M., and Nattel, S. (2002). Predicting the entrainment of reentrant cardiac waves using phase resetting curves. Physical Review E, 65(2):021908. 31

Glass, L. and Pasternack, J. S. (1978a). Prediction of limit cycles in mathematical models of biological oscillations. Bull Math Biol, 40:27–44. 94

Glass, L. and Pasternack, J. S. (1978b). Stable oscillations in mathematical models of biological control systems. J. Math. Biology, 6:207–223. 94

Goldwyn, J. H. and Shea-Brown, E. (2011). The what and where of adding channel noise to the Hodgkin-Huxley equations. PLoS Comput Biol, 7(11):e1002247. 53

Golubitsky, M., Stewart, I., Buono, P. L., and Collins, J. J. (1999). Symmetry in locomotor central pattern generators and animal gaits. Nature, 401(6754):693–5. 86

Gordon, A. M., Huxley, A. F., and Julian, F. J. (1966). The variation in isometric tension with sarcomere length in vertebrate muscle ﬁbres. The Journal of Physiology, 184(1):170–192. 48

Guckenheimer, J. and Holmes, P. (1988). Structurally stable heteroclinic cycles. Math- ematical Proceedings of the Cambridge Philosophical Society, 103(01):189–192. 21, 40

Guckenheimer, J. and Holmes, P. (1990). Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields, volume 42 of Applied Mathematical Sciences. Springer- Verlag, third edition. 86, 124

Guevara, M. R. and Glass, L. (1982). Phase locking, period doubling bifurcations and chaos in a mathematical model of a periodically driven oscillator: A theory for the BIBLIOGRAPHY 172

entrainment of biological oscillators and the generation of cardiac dysrhythmias. Journal of Mathematical Biology, 14(1):1–23. 31

Halbertsma, J. M. (1983). The stride cycle of the cat: the modelling of locomotion by computerized analysis of automatic recordings. Acta physiologica Scandinavica. Supplementum, 521:1–75. 14

Harder, D. L., Hurd, C. L., and Speck, T. (2006). Comparison of mechanical properties of four large, wave-exposed seaweeds. American Journal of Botany, 93(10):1426–1432. 78

Harri, M. and Florey, E. (1977). The effects of temperature on a neuromuscular system of the crayﬁsh, Astacus leptodactylus. Journal of comparative physiology, 117(1):47–61. 34

Hiebert, G. W. and Pearson, K. G. (1999). Contribution of sensory feedback to the generation of extensor activity during walking in the decerebrate cat. Journal of Neuro- physiology, 81(2):758–770. 14

Hurwitz, I., Kupfermann, I., and Susswein, A. J. (1997). Different roles of neurons B63 and B34 that are active during the protraction phase of buccal motor programs in Aplysia californica. Journal of Neurophysiology, 78(3):1305 –1319. 44, 78

Hurwitz, I., Kupfermann, I., and Weiss, K. R. (2003). Fast synaptic connections from CBIs to pattern-generating neurons in Aplysia: Initiation and modiﬁcation of motor programs. Journal of Neurophysiology, 89(4):2120 –2136. 28

Hurwitz, I., Neustadter, D., Morton, D. W., Chiel, H. J., and Susswein, A. J. (1996). Activity patterns of the B31/B32 pattern initiators innervating the I2 muscle of the BIBLIOGRAPHY 173

buccal mass during normal feeding movements in Aplysia californica. Journal of Neurophysiology, 75(4):1309–26. 28, 44

Hurwitz, I. and Susswein, A. J. (1996). B64, a newly identiﬁed central pattern generator el- ement producing a phase switch from protraction to retraction in buccal motor programs of Aplysia californica. Journal of Neurophysiology, 75(4):1327 –1344. 28

Henon,´ H., Godefroy, O., Leys, D., Mounier-Vehier, F., Lucas, C., Rondepierre, P., Duhamel, A., and Pruvo, J. P. (1995). Early predictors of death and disability after acute cerebral ischemic event. Stroke, 26(3):392–398. 14

Ichinose, N., Aihara, K., and Judd, K. (1998). Extending the concept of isochrons from oscillatory to excitable systems for modeling an excitable neuron. International Journal of Bifurcation and Chaos, 8(12):2375–2385. 149

Ijspeert, A. J. (2008). Central pattern generators for locomotion control in animals and robots: A review. Neural Networks, 21(4):642–653. 35, 86

Izhikevich, E. M. (2007). Dynamical Systems in Neuroscience: The Geometry of Excitabil- ity and Bursting. Computational Neuroscience. MIT Press, Cambridge, Massachusetts. 86, 89, 93, 124, 131, 133, 134, 136, 138, 149

Izhikevich, E. M. (2010). Dynamical systems in neuroscience: the geometry of excitability and bursting. MIT Press, Cambridge, Mass.; London. 26

Jacobson, G. A., Diba, K., Yaron-Jakoubovitch, A., Oz, Y., Koch, C., Segev, I., and Yarom, Y. (2005). Subthreshold voltage noise of rat neocortical pyramidal neurones. The Journal of Physiology, 564(Pt 1):145–160. 160 BIBLIOGRAPHY 174

Jones, E., Oliphant, T., Peterson, P., et al. (2001). SciPy: Open source scientiﬁc tools for Python. 136

Kandel, E. R. (1979). Behavioral biology of Aplysia: a contribution to the comparative study of opisthobranch molluscs. W.H. Freeman, San Francisco. 28

Kandel, E. R. (2000). Principles of neural science. McGraw-Hill, Health professions division, New York; St. Louis; San Francisco [etc.]. 28

Katzoff, A., Ben-Gedalya, T., Hurwitz, I., Miller, N., Susswein, Y. Z., and Susswein, A. J. (2006). Nitric oxide signals that Aplysia have attempted to eat, a necessary component of memory formation after learning that food is inedible. J Neurophysiol, 96(3):1247–1257. 28

Kloeden, P. E. and Platen, E. (1992). Numerical solution of stochastic differential equations. Springer-Verlag, Berlin. 25, 58

Kopell, N. (1986). Modeling CPGs: a robust approach. In Neurobiology of Vertebrate Locomotion. 30

Kopell, N. and Ermentrout, G. (2002). Handbook of Dynamical Systems, volume 2, chapter 1. Mechanisms of Phase-Locking and Frequency Control in Pairs of Coupled Neural Oscillators. North-Holland. 140

Kori, H. and Kuramoto, Y. (2001). Slow switching in globally coupled oscillators: robustness and occurrence through delayed coupling. Phys Rev E Stat Nonlin Soft Matter Phys, 63(4 Pt 2):046214. 148

Krupa, M., Popovic,´ N., Kopell, N., and Rotstein, H. G. (2008). Mixed-mode oscillations BIBLIOGRAPHY 175

in a three time-scale model for the dopaminergic neuron. Chaos: An Interdisciplinary Journal of Nonlinear Science, 18(1):015106–015106–19. 80

Kuznetsov, Y. A. (2004). Elements of applied bifurcation theory. Springer, New York. 17

Krivan,ˇ V. and Vrkoc,ˇ I. (2007). A Lyapunov function for piecewise-independent differential equations: stability of the ideal free distribution in two patch environments. J. Math. Biology, 54:465–488. 98

Levi, R., Varona, P., Arshavsky, Y. I., Rabinovich, M. I., and Selverston, A. I. (2004). Dual sensory-motor function for a molluskan statocyst network. J Neurophysiol, 91(1):336– 345. 36

Lewinger, W. A., Rutter, B. L., Blumel,¨ M., Buschges,¨ A., and Quinn, R. D. (2006). Sensory coupled action switching modules (SCASM) generate robust, adaptive stepping in legged robots. In Proceedings of the 9th International Conference on Climbing and Walking Robots (CLAWAR’06), Brussels, Belgium, September 12-14, page 661–71. 35

Loeb, J. (1899). Einleitung in die vergleichende Gehirnphysiologie und vergleichende psychologie, mit besonderer berucksichtigung¨ der wirbellosen thiere. Leipzig, J. A. Barth. 15

Lu, H., McManus, J. M., and Chiel, H. J. (2013). Extracellularly identifying motor neurons for a muscle motor pool in Aplysia californica. Journal of Visualized Experiments, (73). 28, 57, 58

Marder, E. and Bucher, D. (2001). Central pattern generators and the control of rhythmic movements. Current Biology, 11(23):R986–R996. 15, 16, 30 BIBLIOGRAPHY 176

Markin, S. N., Klishko, A. N., Shevtsova, N. A., Lemay, M. A., Prilutsky, B. I., and Rybak, I. A. (2010). Afferent control of locomotor CPG: insights from a simple neuromechanical model. Ann N Y Acad Sci, 1198:21–34. 91

Matsumoto, M. and Nishimura, T. (1998). Mersenne twister: a 623-dimensionally equidis- tributed uniform pseudo-random number generator. ACM Trans. Model. Comput. Simul., 8(1):3–30. 59

McHorney, C. A., Martin-Harris, B., Robbins, J., and Rosenbek, J. (2006). Clinical validity of the SWAL-QOL and SWAL-CARE outcome tools with respect to bolus ﬂow measures. Dysphagia, 21(3):141–148. 14

McKean, H. (1970). Nagumo’s equation. Advances in Mathematics, 4:209–223. 94

McManus, J. M., Lu, H., and Chiel, H. J. (2012). An in vitro preparation for eliciting and recording feeding motor programs with physiological movements in Aplysia californica. Journal of Visualized Experiments, (70). 28, 56, 73

Morris, C. and Lecar, H. (1981). Voltage oscillations in the barnacle giant muscle ﬁber. Biophysical Journal, 35(1):193 – 213. 135

Morton, D. W. and Chiel, H. J. (1993a). In vivo buccal nerve activity that distinguishes ingestion from rejection can be used to predict behavioral transitions in Aplysia. Jour- nal of Comparative Physiology A: Neuroethology, Sensory, Neural, and Behavioral

Physiology, 172(1):17–32. 27, 28

Morton, D. W. and Chiel, H. J. (1993b). The timing of activity in motor neurons that produce radula movements distinguishes ingestion from rejection in Aplysia. Journal of BIBLIOGRAPHY 177

Comparative Physiology A: Neuroethology, Sensory, Neural, and Behavioral Physiology, 173(5):519–536. 48

Nadim, F., Zhao, S., Zhou, L., and Bose, A. (2011). Inhibitory feedback promotes stability in an oscillatory network. Journal of Neural Engineering, 8(6):065001. 149

Nargeot, R., Baxter, D. A., and Byrne, J. H. (2002). Correlation between activity in neuron b52 and two features of ﬁctive feeding in Aplysia. Neuroscience Letters, 328(2):85–88. 28, 161

Nargeot, R. and Simmers, J. (2012). Functional organization and adaptability of a decision- making network in Aplysia. Frontiers in Neuroscience, 6. 79

Neustadter, D. M., Drushel, R. F., and Chiel, H. J. (2002). Kinematics of the buccal mass during swallowing based on magnetic resonance imaging in intact, behaving Aplysia californica. Journal of Experimental Biology, 205(7):939–958. 51

Neustadter, D. M., Herman, R. L., Drushel, R. F., Chestek, D. W., and Chiel, H. J. (2007). The kinematics of multifunctionality: comparisons of biting and swallowing in Aplysia californica. Journal of Experimental Biology, 210(2):238–260. 51

Novak, B., Csikasz-Nagy, A., Gyorffy, B., Nasmyth, K., and Tyson, J. J. (1998). Model scenarios for evolution of the eukaryotic cell cycle. Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences, 353(1378):2063–2076. 39

Novak,´ B. and Tyson, J. J. (2008). Design principles of biochemical oscillators. Nat Rev Mol Cell Biol, 9(12):981–91. 92

Novakovic, V. A., Sutton, G. P., Neustadter, D. M., Beer, R. D., and Chiel, H. J. (2006). Mechanical reconﬁguration mediates swallowing and rejection in Aplysia californica. BIBLIOGRAPHY 178

Journal of Comparative Physiology A: Neuroethology, Sensory, Neural, and Behavioral

Physiology, 192(8):857–870. 28, 31, 49, 78

Nowotny, T. and Rabinovich, M. I. (2007). Dynamical origin of independent spiking and bursting activity in neural microcircuits. Physical Review Letters, 98(12):128106–4. 30, 40, 80, 86, 148

Øksendal, B. K. (2003). Stochastic differential equations: an introduction with applications. Springer, Berlin; New York. 25

Oliphant, T. E. (2006). Guide to NumPy. Provo, UT. 136

Park, Y. (2013). Inﬁnitesimal Phase Response Curves for Piecewise Smooth Dynamical Systems. Case Western Reserve University. 155

Pearson, K. G., Reye, D. N., and Robertson, R. M. (1983). Phase-dependent inﬂuences of wing stretch receptors on ﬂight rhythm in the locust. J Neurophysiol, 49(5):1168–1181. 34, 80

Phoka, E., Cuntz, H., Roth, A., and Hausser,¨ M. (2010). A new approach for determining phase response curves reveals that Purkinje cells can act as perfect integrators. PLoS Comput Biol, 6(4):e1000768. 156

Pinto, C. M. A. and Golubitsky, M. (2006). Central pattern generators for bipedal locomotion. J Math Biol, 53(3):474–89. 86

Pinto, R. D., Elson, R. C., Szucs,¨ A., Rabinovich, M. I., Selverston, A. I., and Abarbanel, H. D. (2001). Extended dynamic clamp: controlling up to four neurons using a single desktop computer and interface. J Neurosci Methods, 108(1):39–48. 148 BIBLIOGRAPHY 179

Proekt, A., Jing, J., and Weiss, K. R. (2007). Multiple contributions of an input-representing neuron to the dynamics of the Aplysia feeding network. Journal of Neurophysiology, 97(4):3046–3056. 28

Rabinovich, M., Huerta, R., and Laurent, G. (2008a). Transient dynamics for neural processing. Science, 321(5885):48–50. 86

Rabinovich, M. I., Huerta, R., Varona, P., and Afraimovich, V. S. (2008b). Transient cog- nitive dynamics, metastability, and decision making. PLoS Comput Biol, 4(5):e1000072. 21, 36, 41, 86

Rabinovich, M. I., Varona, P., Selverston, A. I., and Abarbanel, H. D. (2006). Dynamical principles in neuroscience. Reviews of Modern Physics, 78(4):1213–1265. 87

Rabinovitch, A. and Rogachevskii, I. (1999). Threshold, excitability and isochrones in the Bonhoeffer–van der Pol system. Chaos, 9(4):880–886. 149

Rabinovitch, A., Thieberger, R., and Friedman, M. (1994). Forced Bonhoeffer–van der Pol oscillator in its excited mode. Phys Rev E Stat Phys Plasmas Fluids Relat Interdiscip Topics, 50(2):1572–1578. 149

Reyn, J. W. (1980). Generation of limit cycles from separatrix polygons in the phase plane. In Martini, R., editor, Geometrical Approaches to Differential Equations, number 810 in Lecture Notes in Mathematics, pages 264–289. Springer Berlin Heidelberg. 21, 41, 87

Rinzel, J. and Ermentrout, G. (1989). Analysis of neural excitability and oscillations. In Koch, C. and Segev, I., editors, Methods in Neuronal Modeling. MIT Press, second edition. 135 BIBLIOGRAPHY 180

Rubin, J. E., Shevtsova, N. A., Ermentrout, G. B., Smith, J. C., and Rybak, I. A. (2009). Multiple rhythmic states in a model of the respiratory central pattern generator. J Neurophysiol, 101(4):2146–65. 86, 91

Sacco, P., McIntyre, D. B., and Jones, D. A. (1994). Effects of length and stimulation frequency on fatigue of the human tibialis anterior muscle. Journal of Applied Physiology, 77(3):1148–1154. 65

Sammon, M. (1994). Geometry of respiratory phase switching. J Appl Physiol, 77(5):2468– 80. 86

Schiff, S. J. (2012). Neural control engineering: the emerging intersection between control theory and neuroscience. MIT Press, Cambridge, MA. 53

Schwarz, M. and Susswein, A. (1986). Identiﬁcation of the neural pathway for reinforce- ment of feeding when Aplysia learn that food is inedible. J. Neurosci., 6(5):1528–1536. 28

Selverston, A. I. (1985). Model neural networks and behavior. Plenum Press, New York. 17, 34

Shaw, K., Cullins, M., Lu, H., McManus, J., Chiel, H., and Thomas, P. (2010). Evidence for a stable heteroclinic channel underlying a central pattern generator. In 2010 Neuroscience Meeting Planner. San Diego, CA: Society for Neuroscience. 130, 135, 162

Shaw, K. M., Park, Y.-M., Chiel, H. J., and Thomas, P. J. (2012). Phase resetting in an asymptotically phaseless system: On the phase response of limit cycles verging on a heteroclinic orbit. SIAM Journal on Applied Dynamical Systems, 11:350–391. 73, 83 BIBLIOGRAPHY 181

Sherman, A. and Rinzel, J. (1992). Rhythmogenic effects of weak electrotonic coupling in neuronal models. Proc Natl Acad Sci U S A, 89(6):2471–4. 140

Sherrington, C. S. (1910). Flexion-reflex of the limb, crossed extension-reflex, and reflex stepping and standing. The Journal of Physiology, 40(1-2):28–121. 15

Sherrington, C. S. (1913). Further observations on the production of reflex stepping by combination of reflex excitation with reflex inhibition. The Journal of Physiology, 47(3):196–214. 15

Sherwood, W. E. and Guckenheimer, J. (2010). Dissecting the phase response of a model bursting neuron. SIAM Journal on Applied Dynamical Systems, 9(3):659–703. 80

Shilnikov, L. P., Shilnikov, A. L., Turaev, D. V., and Chua, L. O. (2002). Methods of Qualitative Theory in Nonlinear Dynamics, Part II. World Scientiﬁc. 40

Spardy, L. E., Markin, S. N., Shevtsova, N. A., Prilutsky, B. I., Rybak, I. A., and Rubin, J. E. (2011a). A dynamical systems analysis of afferent control in a neuromechanical model of locomotion. I. Rhythm generation. Journal of Neural Engineering, 8(6):065003. 80, 91

Spardy, L. E., Markin, S. N., Shevtsova, N. A., Prilutsky, B. I., Rybak, I. A., and Rubin, J. E. (2011b). A dynamical systems analysis of afferent control in a neuromechanical model of locomotion. II. Phase asymmetry. Journal of Neural Engineering, 8(6):065004. 80, 91

Stiefel, K. M., Fellous, J.-M., Thomas, P. J., and Sejnowski, T. J. (2010). Intrinsic subthreshold oscillations extend the inﬂuence of inhibitory synaptic inputs on cortical pyramidal neurons. Eur J Neurosci, 31(6):1019–26. 149 BIBLIOGRAPHY 182

Stiefel, K. M., Gutkin, B. S., and Sejnowski, T. J. (2008). Cholinergic neuromodulation changes phase response curve shape and type in cortical pyramidal neurons. PLoS One, 3(12):e3947. 149

Stone, E. and Armbruster, D. (1999). Noise and O(1) amplitude effects on heteroclinic cycles. Chaos, 9(2):499–506. 86

Stone, E. and Holmes, P. (1990). Random perturbations of heteroclinic attractors. SIAM Journal on Applied Mathematics, 50(3):726–743. 40, 73, 86, 149, 154, 159

Susswein, A. J. and Chiel, H. J. (2012). Nitric oxide as a regulator of behavior: New ideas from Aplysia feeding. Progress in Neurobiology, 97(3):304–317. 79

Susswein, A. J., Hurwitz, I., Thorne, R., Byrne, J. H., and Baxter, D. A. (2002). Mech- anisms underlying ﬁctive feeding in Aplysia: Coupling between a large neuron with plateau potentials activity and a spiking neuron. Journal of Neurophysiology, 87(5):2307– 2323. 28, 41, 44, 79

Sutton, G. P., Macknin, J. B., Gartman, S. S., Sunny, G. P., Beer, R. D., Crago, P. E., Neustadter, D. M., and Chiel, H. J. (2004a). Passive hinge forces in the feeding apparatus of Aplysia aid retraction during biting but not during swallowing. Journal of Comparative Physiology A, 190(6):501–514. 28, 78, 91

Sutton, G. P., Mangan, E. V., Neustadter, D. M., Beer, R. D., Crago, P. E., and Chiel, H. J. (2004b). Neural control exploits changing mechanical advantage and context dependence to generate different feeding responses in Aplysia. Biological Cybernetics, 91(5):333–345. 28, 49, 78, 91 BIBLIOGRAPHY 183

Szucs,¨ A., Huerta, R., Rabinovich, M. I., and Selverston, A. I. (2009). Robust microcircuit synchronization by inhibitory connections. Neuron, 61(3):439–53. 148

Thomas, P. J., Tiesinga, P. H. E., Fellous, J.-M., and Sejnowski, T. J. (2003). Reliability and Bifurcation in Neurons Driven by Multiple Sinusoids. Neurocomputing, 52-54:955–961. 149

Tomori, Z., Poliacek, I., Jakus, J., Widdicombe, J., Donic, V., Benacka, R., and Gresova, S. (2010). Distinct generators for aspiration and expiration reﬂexes: localization, mechanisms and effects. Journal of physiology and pharmacology: an ofﬁcial journal of the Polish Physiological Society, 61(1):5–12. 81

Toups, J. V., Fellous, J.-M., Thomas, P. J., Sejnowski, T. J., and Tiesinga, P. H. (2011). Finding the event structure of neuronal spike trains. Neural Computation, 23(9):2169– 2208. 149

Tyson, J. J. and Novak, B. (2001). Regulation of the eukaryotic cell cycle: molecular antagonism, hysteresis, and irreversible transitions. J Theor Biol, 210(2):249–63. 91

Urban, A. and Ermentrout, B. (2011). Sequentially ﬁring neurons confer ﬂexible timing in neural pattern generators. Phys Rev E Stat Nonlin Soft Matter Phys, 83(5 Pt 1):051914. 148 van der Pol, B. (1926). On “relaxation-oscillations”. Philosophical Magazine Series 7, 2(11):978–992. 82

Vandorpe, D. H., Small, D. L., Dabrowski, A. R., and Morris, C. E. (1994). FMRFamide and membrane stretch as activators of the Aplysia S-channel. Biophysical Journal, 66(1):46–58. 52 BIBLIOGRAPHY 184

Varkonyi,´ P. L., Kiemel, T., Hoffman, K., Cohen, A. H., and Holmes, P. (2008). On the derivation and tuning of phase oscillator models for lamprey central pattern generators. J Comput Neurosci, 25(2):245–61. 130

Varona, P., Levi, R., Arshavsky, Y. I., Rabinovich, M. I., and Selverston, A. I. (2004). Competing sensory neurons and motor rhythm coordination. Neurocomputing, 58- 60:549–554. 36

Varona, P., Rabinovich, M. I., Selverston, A. I., and Arshavsky, Y. I. (2002). Winnerless competition between sensory neurons generates chaos: A possible mechanism for molluscan hunting behavior. Chaos, 12(3):672–677. 86

Wallen,´ P. and Williams, T. L. (1984). Fictive locomotion in the lamprey spinal cord in vitro compared with swimming in the intact and spinal animal. The Journal of Physiology, 347(1):225–239. 16, 80

White, J., Rubinstein, J., and Kay, A. (2000). Channel noise in neurons. Trends Neurosci., 23:131–137. 53

Williams, C. D., Salcedo, M. K., Irving, T. C., Regnier, M., and Daniel, T. L. (2013). The length–tension curve in muscle depends on lattice spacing. Proceedings of the Royal Society B: Biological Sciences, 280(1766). 48

Williams, R. W. (2000). Mapping genes that modulate mouse brain development: A quantitative genetic approach. In Gofﬁnet, A. M. and Rakic, P., editors, Mouse Brain Development, number 30 in Results and Problems in Cell Differentiation, pages 21–49. Springer Berlin Heidelberg. 28 BIBLIOGRAPHY 185

Wilson, D. M. (1961). The central nervous control of ﬂight in a locust. Journal of Experimental Biology, 38(2):471–490. 16

Wilson, H. R. (1999). Spikes, Decisions and Actions. Oxford University Press. 86

Winter, D. A. (1991). The biomechanics and motor control of human gait: normal, elderly and pathological. University of Waterloo Press. 14

Yu, S. N., Crago, P. E., and Chiel, H. J. (1999). Biomechanical properties and a kinetic simulation model of the smooth muscle I2 in the buccal mass of Aplysia. Biological Cybernetics, 81(5-6):505–13. 48, 78

Zajac, F. E. (1989). Muscle and tendon: properties, models, scaling, and application to biomechanics and motor control. Critical reviews in biomedical engineering, 17(4):359– 411. 48, 78

Zhurov, Y. and Brezina, V. (2006). Variability of motor neuron spike timing maintains and shapes contractions of the accessory radula closer muscle of Aplysia. The Journal of Neuroscience, 26(26):7056–7070. 28