Robotics and Autonomous Systems 60 (2012) 707–713 Contents lists available at SciVerse ScienceDirect Robotics and Autonomous Systems journal homepage: www.elsevier.com/locate/robot A neuromodulation model of behavior selection in the fighting behavior of male crickets Kuniaki Kawabata a,∗, Takashi Fujii b, Hitoshi Aonuma c, Tsuyoshi Suzuki b, Masatoshi Ashikaga d, Jun Ota e, Hajime Asama f a RIKEN, Saitama, 351-0198, Japan b Tokyo Denki University, Tokyo, Japan c RIES, Hokkaido University, Hokkaido, Japan d Uchida Yoko Co., Japan e RACE, The University of Tokyo, Chiba, Japan f The University of Tokyo, Tokyo, Japan article info a b s t r a c t Article history: The results of neurophysiological research have indicated the important role of neuromodulation in Available online 20 November 2011 behavior decisions during fighting between male crickets. To understand such a dynamic behavior generation mechanism, we attempt to construct a neuromodulation model of behavior selection for the Keywords: fighting behavior of a cricket. In this paper, we propose a dynamic system model of neuromodulation Internal state with the efficacy based on neurophysiological knowledge, facts, and hypotheses from the viewpoint Social interaction Nitric oxide of synthesis. The efficacy of the sensory processing center is introduced to our previous model and Biogenic amine its dynamics is influenced by the frequency of the interactions with the other cricket. Utilizing an Efficacy extended neuromodulation model, computer simulations related to a multiple-individual environment Synthetic neuroethology were carried out, and the results are discussed. In particular, the properties related to the standard deviation of the value of octopamine in several population density cases are discussed. ' 2011 Elsevier B.V. All rights reserved. 1. Introduction Fig. 1 shows an example of the sequence of fighting behavior between male crickets. Usually, when it perceives cuticular Biologists have often performed neurophysiological and behav- pheromones on the body surface of the opponent cricket, the ioral experiments to elucidate the neuronal mechanisms underly- male cricket shows aggressive behavior. However, a cricket does ing adaptive behavior in animals. However, usually, such experi- not show aggressive behavior after it becomes a loser in a mental approaches accumulate plenty of spacio-temporally inde- previous fight. Thus, if the defeated cricket senses the same pendent pieces of data. In order to understand the mechanisms un- pheromone within several minutes, it shows avoidance behavior. derlying adaptive behavior, we need to reconstruct our knowledge, The experience of defeat thus makes the cricket select a different and we propose a synthetic approach to bridge the gap between behavior for the same stimulus. The nitric oxide (NO)/cyclic guanosine monophosphate (cGMP) cascade has been particularly physiological aspects and behavioral aspects by constructing a dy- considered to be closely related to fighting behavior selection by namic system model [1]. crickets [3,4]. Cricket fighting provides us with a good model system The motivation and aim of our current study is to clarify the to understand the neuronal mechanisms underlying adaptive adaptive behavior selection mechanism by modeling the role of behavior selection [2]. When a male cricket comes across another neuromodulation in the central nervous system by focus on fight- male cricket, it first senses the cuticular substances called cuticle ing behavior between crickets. In our previous work [5], based pheromone using its antennae and then shows fighting behavior. on several neuroethological issues, we modeled the NO/cGMP cascade function in cricket fighting behavior with regard to the relationship between octopamine (OA) concentration and behav- ∗ ior selection. We also discussed our proposed neuromodulation Correspondence to: 2-1, Hirosawa, Wako, Saitama, 351-0198, Japan. E-mail addresses: [email protected] (K. Kawabata), [email protected] model as an NO/cGMP-OA system being considered an appropri- (H. Aonuma), [email protected] (T. Suzuki), [email protected] (J. Ota), ate one for an individual cricket's response in a fight between [email protected] (H. Asama). crickets. 0921-8890/$ – see front matter ' 2011 Elsevier B.V. All rights reserved. doi:10.1016/j.robot.2011.11.003 708 K. Kawabata et al. / Robotics and Autonomous Systems 60 (2012) 707–713 Fig. 2. A basic neuromodulation model for behavior selection based on the NO/cGMP-OA system [5]. points, and its dimension is n (mm). The concentration state of NO in the AL is expressed as a vector N of .n C 1/ dimensions and that x at discrete position x .0 ≤ x ≤ n; x 2 Z/ as N . NO is a free radical that reacts with metal ions and oxygen immediately after diffusion throughout the brain, after which it disappears, presumably being Fig. 1. Snapshots of fighting behavior between male crickets. present for about 10 s. To express this effect, the equation of NO diffusion is given as follows. However, the model could not explain the behavior modifi- @N @ 2N cation to the size of the environment in the multiple-individual D D − γ N C N − N ; (1) 2 N in out cricket situation [6]. As a certain factor for the differences, we con- @t @x sidered it was important to focus on the interaction effect among where Nin and Nout represent the state vector for NO generation the crickets [7]. There also have been reports [8–10] that the infor- and consumption at each position in the AL. The increased NO mation processing mechanism of the insect is modulated by the concentration in the AL is certain, so we assume that the amount x biogenic amine system. Therefore, we put forward a hypothesis of NO generation Nin at position x is determined for position set I that the interactions among the crickets affects the efficacy in the at the source: neuronal system for behavior selection. (0:0 if x 62 I, In this paper, we modify our proposed neuromodulation model x D 2 ^ Nin 1:5 if (x I fighting), (2) based on the NO/cGMP-OA system and also extend the model, 1:0 otherwise. introducing the efficacy of the sensory processing center. Multiple- x individual simulations are also carried out, and we discuss Modeling property Nout as a smooth function such as Eq. (3), we simulated behaviors depending on the size of the field by using obtain cricket agents depending on the effect of the size of the space. We a × .1 C tanh.50 × .N x − 0:4///=2 if x 2 , also discuss what effect the efficacy has on the internal properties N x D O (3) out 0 otherwise. and behavior selection. The amount of cGMP is expressed as a scalar value C. The 2. A neuromodulation model for behavior selection of an behavior of cGMP is expressed in Eq. (4) using the constant γC individual cricket in a fight based on the idea that excessive amounts of cGMP are decomposed in the body: As we described above, the NO/cGMP cascade has been considered to be related to fighting behavior of crickets. To dC D −γC C C Cin − Cout ; (4) demonstrate this, fighting behavior was observed when an NO dt synthesis inhibitor is injected into the heads of the crickets. In D x where Cin ΣxNout and Cout represent amounts of cGMP this case, even the defeated cricket may exhibit fighting behavior. generation and consumption. Based on physiological knowledge, This indicates that the appropriate behavior is not selected if cGMP suppresses OA production, so cGMP consumption is the NO/cGMP cascade does not function normally. Even under modeled as a smooth function. these circumstances, the pheromone is identified and NO is closely related to the pheromone behavior modified by past Cout D b × .1 C tanh.10 × .C − 0:64//=2/: (5) experience [11]. The NO/cGMP cascade mediates the efficacy of The cricket selects its behavior based on the amount of OA. various neural pathways. One of these is to change OA levels in the The scalar A indicates the OA value, and OA is generated by cGMP brain [12,13]. Therefore, the NO/cGMP cascade is considered to be and used to select the behavior. Likewise, the amount of OA is deeply related to behavior selection through the OA system. expressed in Eq. (6) by using the constant γA: In our previous work, a basic neuromodulation model [5], namely the mimetic NO/cGMP-OA system, was proposed (Fig. 2), dA D −γAA C Ain − Aout ; (6) and it was confirmed that the model was an adequate one to be an dt individual cricket's internal dynamics model [5]. Subsequently, we where Ain and Aout represent the amount of OA generation and gave some modifications to our previous neuromodulation model. consumption. OA is used in the cricket to obtain energy from fat. Here, we describe the formulations [5] once more. Here, Ain is By noting the internal state of the cricket based on the amounts of NO, cGMP, and OA, we modeled a basic NO/cGMP-OA Ain D b − 0:6 × Cout : (7) model in fighting behavior, according to sensual information from Also, using the constant c, Aout is antennae: the pheromones from another individual increase the NO concentration in the antennal lobe (AL). For simplification, the c if fighting, A D (8) AL is considered as a row of discrete NO generation/consumption out 0 otherwise. K. Kawabata et al. / Robotics and Autonomous Systems 60 (2012) 707–713 709 Fig. 4. A snapshot of multiple-individual simulation. populations often fight, while those raised in environments with high-density populations tend to show much lower incidences of this behavior [14]. Populations in environments with intermediate densities, between high and low (medium), have been observed and reported to contain one cricket that continues to attack other males, while the rest of the males attack less and less often.