Algorithmic Analysis of Irrational Rotations in a Single Neuron Model$
Total Page:16
File Type:pdf, Size:1020Kb
Journal of Complexity 19 (2003) 132–152 http://www.elsevier.com/locate/jco Algorithmic analysis of irrational rotations in a single neuron model$ Hayato Takahashia,* and Kazuyuki Aiharab,c a Department of Statistical Science, The Graduate University for Advanced Studies, The Institute of Statistical Mathematics, 4-6-7 Minami-Azabu, Minato-ku, Tokyo 106-8569, Japan b Department of Mathematical Engineering and Information Physics, Graduate School of Engineering, University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-8656, Japan c CREST, Japan Science and Technology Corporation (JST), 4-1-8 Hon-Cho, Kawaguchi, Saitama 332-0012, Japan Received 1 June 2002; accepted 11 September 2002 Abstract We study computability of real-valued functions and the information needed for simulation of dynamical systems. In particular, we describe application of Kolmogorov complexity theory to computer simulation of irrational rotations in a single neuron model. We deduce the information needed for a parameter in simulating the dynamics by showing a difference in Kolmogorov complexity between a computable parameter and a non-computable parameter. Finally, we show that all trajectories generated by irrational rotations are non-computable iff its parameter is non-computable. r 2002 Elsevier Science (USA). All rights reserved. Keywords: Kolmogorov complexity; Algorithmic information theory; Computability; Bayes code; MDL code; Nagumo–Sato model; Single neuron model; Irrational rotation; Sturmian sequence; Symbolic dynamics 1. Introduction Although continuous analysis has been greatly progressing in mathematics and physics, there is just a paucity of studies (for example, see [25,30]) on computability $Revised from ‘‘Asymptotic evaluation of Kolmogorov complexity of pulse sequences generated by a single neuron model’’. *Corresponding author. The present address: The Institute of Statistical Mathematics, 4-6-7 Minami- Azabu, Minato-ku, Tokyo, 106-8569, Japan. Tel.: 81-3-3446-1501; fax: 81-3-5421-8750. E-mail addresses: [email protected] (H. Takahashi), [email protected] (K. Aihara). 0885-064X/02/$ - see front matter r 2002 Elsevier Science (USA). All rights reserved. PII: S 0 8 8 5 - 064X(02)00017-1 H. Takahashi, K. Aihara / Journal of Complexity 19 (2003) 132–152 133 Nomenclature N the set of natural numbers Q the set of rational numbers R the set of real numbers n f0; 1g the set of finite binary strings f0; 1gn the set of binary strings of length n N f0; 1g the set of infinite binary sequences o1:n ¼ o1o2?on; oiAf0; 1g; 1pipn: N oAf0; 1g ; o ¼ o1o2? KðxÞ the prefix Kolmogorov complexity of a string x KðxjyÞ the prefix Kolmogorov complexity of a string x with a given string y ðX; F; mÞ a probability measure space X ¼½0; 1Þ; F is Borel s algebra and m is a measure on ðX; FÞ r 0prp1 r r A0 ¼½0; 1 À rÞ; A1 ¼½1 À r; 1Þ Tr : X-X TrðxÞ¼x þ r mod 1 Sr ¼ Ar T À1Ar ? T Ànþ1Ar o1:n o1 - r o2 - - r on P ðo Þ¼ mðSr Þ r 1:n o1:n D ¼fðr; xÞjxASr ; 0 r 1g o1:n o1:n p p Bo1:n ¼fr j Prðo1:nÞ40g Lo1:n ¼ sup Bo1:n À inf Bo1:n of real-valued functions and information required to simulate dynamical systems [27]. These subjects are important when we solve analytical and physical problems with computers. In this paper, we explore a particular problem on irrational rotations and a single neuron model. Irrational rotations are a classical subject in dynamical systems theory, and the single neuron model examined in this paper is called the Nagumo–Sato model [23], which can potentially generate aperiodic behavior described with irrational rotations. We study the information on a parameter for simulation of irrational rotations by means of algorithms. The theory of Kolmogorov complexity, the complexity of a string measured in terms of a minimum length program that generates the string [8,15,18,28], plays an especially important role in our analysis. From Kolmogorov complexity of an observed sequence of a dynamical system, we know how much information is necessary to simulate the dynamics because Kolmogorov complexity is a minimum length program that generates the sequence. Therefore, if the asymptotic order of Kolmogorov complexity of an observed sequence of length n is f ðnÞ; where f ðnÞ is some increasing function of n; then the 134 H. Takahashi, K. Aihara / Journal of Complexity 19 (2003) 132–152 total bits for the initial value and the parameter values i.e., the total bits that are needed to simulate the dynamics are on the order of at least f ðnÞ (see also [10]). Although the dynamics of the Nagumo–Sato model [23] is almost always periodic (for details see [11,12] and the remark in Section 2), we discuss mainly its aperiodic behavior because in periodic behavior, the spike sequence is computable and the asymptotic value of Kolmogorov complexity with a given sequence length is constant. Moreover, aperiodic behavior has been observed in an electrophysiological experiment with squid giant axons under a condition corresponding to the Nagumo– Sato model (see [21] and Fig. 1). Although such aperiodic dynamics can be well described by a chaotic neuron model derived by replacing the Heaviside output function of the Nagumo–Sato model with a sigmoidal one, which can be extended to a chaotic neural network model [1,9], a kind of aperiodic structure of solutions is immanent in the Nagumo–Sato model itself as Sturmian sequences that are studied in this paper. The present paper is our first step toward analyzing more general dynamics. The prefix Kolmogorov complexity (see [8,18]), which is a variant of the originally defined Kolmogorov complexity [15,28], is represented by string x with a given length n as KðxjnÞ: The precise definition of the prefix Kolmogorov complexity is given in Section 5. We relate a measure on infinite sequences to the dynamics of the Nagumo–Sato model to evaluate Kolmogorov complexity. Aperiodic spike sequences of the Nagumo–Sato model are considered observed data of a trajectory of an irrational rotation. Such sequences generated by irrational rotations are called Sturmian sequences [3,4,22,31], which is an example of symbolic dynamics and described with a probability measure on infinite sequences. There are several known relations between Kolmogorov complexity and measures. For example, if a measure P on infinite sequences is ergodic [5], then limn-NKðxjnÞ=n ¼ h; PÀa.e. (almost everywhere with respect to P), where x is a string of length n and h is the entropy of P [6,32]. In this case we cannot derive the oðnÞ term of Kolmogorov complexity, and this cannot be applied to our case because Fig. 1. A complicated response (the lower waveform) experimentally observed in a squid giant axon where the nerve membrane in the resting state is stimulated by periodic pulses (the upper waveform). H. Takahashi, K. Aihara / Journal of Complexity 19 (2003) 132–152 135 the order of Kolmogorov complexity of the Nagumo–Sato model is oðnÞ: The probability model we use is parametric, and MDL coding is generally used for parametric models [26]. However, MDL coding is not applicable to our model. The main reason is that the likelihood function of our model is not differentiable at maximum-likelihood estimate (see also Remark 6.2). If a probability measure P is computable (for the computability of measure, see [19] and Section 4), then Kolmogorov complexity KðxjnÞ is bounded above by Àlog PðxÞþc where PðxÞ is the probability of sequences with the initial segment x and constant c independent of P and x [19]. But if the measure is not computable, we cannot apply this property. In this paper we use Bayes coding [2] to give an asymptotic evaluation of the oðnÞ term of Kolmogorov complexity, particularly when the measure is not computable. First, we explore computability of the measure derived from the Nagumo–Sato model. Specifically, we show that computability of the model parameter is equivalent to computability of the derived measure. Second, we deals with the Kolmogorov complexity of spike sequences of the Nagumo–Sato model when the derived measure is not computable. This shows a difference in Kolmogorov complexity between computable parameters and non-computable parameters. Finally, we show that the parameter is non-computable iff all sequences generated by our model are non- computable. All technical proofs are given in the appendix. 2. The Nagumo–Sato model, Hata’s result, and Sturmian sequences In this section, we briefly describe the Nagumo–Sato model and its characteristics. The Nagumo–Sato model is a special version of Caianiello’s neuronic equation [7]. This model deals with a single neuron stimulated by a periodic constant-strength input (see the upper waveform of Fig. 1). The refractory effect due to past firing of the neuron is assumed to decay exponentially with time. The dynamics of the model can be expressed as follows [23]: "# Xn r xnþ1 ¼ 1 A À a xnÀr=b À y ; ð1Þ r¼0 where 1½x is the Heaviside function, A is the strength of the input stimulus, a40; b41; and y is the threshold value. xn is the output of the neuron at time n: xn ¼ 0 and 1 represent non-firingP and firing of a output spike, respectively. Letting y ¼ 1 þðA À yÞ=ab À n x =br; (1) is rewritten as follows [23,11,12]: n ( r¼0 nÀr bðyn À cÞþ1ifynoc; ynþ1 ¼ f ðynÞ ð2Þ bðyn À cÞ if ynXc; xnþ1 ¼ 1½yn À c; AÀy 1 where b ¼ 1=b and c ¼ 1 À a ð1 À bÞ: We assume that 0oco1 because the neuron keeps firing if cp0 or resting if cX1: We also assume that f is a map from I ¼½0; 1Þ to itself because the dynamics with any initial value must fall into I after some 136 H.