Neurocomputing 152 (2015) 27–35

Contents lists available at ScienceDirect

Neurocomputing

journal homepage: www.elsevier.com/locate/neucom

Biological context of Hebb learning in artificial neural networks, a review

Eduard Kuriscak a,n, Petr Marsalek b,c, Julius Stroffek b, Peter G. Toth b a Institute of Physiology, First Medical Faculty, Charles University in Prague, Albertov 5, CZ-128 00 Praha 2, Czech Republic b Institute of Pathological Physiology, First Medical Faculty, Charles University in Prague, U Nemocnice 5, CZ-128 53 Praha 2, Czech Republic c Czech Technical University in Prague, Zikova 4, CZ-166 36 Praha 6, Czech Republic article info abstract

Article history: In 1949 Donald Olding Hebb formulated a hypothesis describing how excite each other and how Received 7 July 2014 the efficiency of this excitation subsequently changes with time. In this paper we present a review of this Received in revised form idea. We evaluate its influences on the development of artificial neural networks and the way we 26 September 2014 describe biological neural networks. We explain how Hebb's hypothesis fits into the research both of Accepted 9 November 2014 that time and of present. We highlight how it has gone on to inspire many researchers working on Communicated by W.L. Dunin-Barkowski fi Available online 27 November 2014 arti cial neural networks. The underlying biological principles that corroborate this hypothesis, that were discovered much later, are also discussed in addition to recent results in the field and further Keywords: possible directions of synaptic learning research. Artificial neural networks & 2014 Elsevier B.V. All rights reserved. Biological neural networks Hebb learning Hebb rule Hebb

1. Introduction binary (all-or-none) representation and processing of information in the brain [8]. However this idea was not entirely new and its roots can In 2014 we commemorate 110 years since the birth of Donald be traced back to 1926, to a paper by Adrian and Zotterman [9].The Olding Hebb and 65 years since the first publication of his influential mid-twentieth century saw many researchers developing sophisti- book The Organization of Behavior [1].Inthefirst half of the twentieth cated hypotheses of how information might be processed by neuronal century, one of the most tantalizing questions in the field of mind and circuits in the brain. In 1948, Wiener's book [10] brought the per- brain research was the problem of how the physiology of the brain spectives of information theory and signal processing to the field. correlates to high level behavior of mammals and especially humans. During this time, Hebb was working on a theory that would Pavlov proposed conditioned reflexes [2] as an explanation of how the explain complex psychological behaviors within the framework of neural excitation line connects the triggered target muscle to some neural physiology. The approach he adopted stemmed from the best external excitation along the way. In 1943, McCulloch and Pitts [3] practices used by behavioral psychologists in North America of the used ideas coined almost a decade earlier by Turing [4] to formulate time, dating back some 40 years to William James [11]. To explain logical calculus as a framework of neural computation. A few examples behavior using hypothetical neural computations, Hebb had to make of similar hypotheses include that of Jeffress, who during his sabbatical a few novel assumptions, one of which has become the most cited at California Institute of Technology in 1948 proposed a neural circuit sentence of his 1949 book [1]. It is the formulation of the general rule for localization [5]. The proposed circuit was found in birds 40 describing how changes in synaptic weights (also strengths, or years later [6].Similarly,in1947Laufberger[7] proposed the idea of efficiencies) control the way how neurons excite each other: “When an of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic Abbreviation: ANN, Artificial neural networks; AMPA, α-Amino-3-hydroxy-5- change takes place in one or both cells such that A'sefficiency, as one of Methyl-4-isoxazole-Propionic Acid; BNN, Biological neural networks; BCM, Bienen- the cells firing B, is increased.” stock, Cooper, and Munro; CA3, Cornu Ammonis no. 3 (area in the ); Hebb's simplified formulation of the assumption was deliberate. LTD, Long-term depression; LTP, Long-term potentiation; NMDA, N-Methyl- D- Whilst the idea of increasing synaptic efficiency had been pre- Aspartate; NO, Nitric oxide; STDP, Spike timing dependent plasticity n Corresponding author. Tel.: þ420 224 96 8413. viously presented and the possible underlying chemical and biolo- E-mail address: Eduard.kuriscak@lfl.cuni.cz (E. Kuriscak). gical processes already studied [12], their biological was not http://dx.doi.org/10.1016/j.neucom.2014.11.022 0925-2312/& 2014 Elsevier B.V. All rights reserved. 28 E. Kuriscak et al. / Neurocomputing 152 (2015) 27–35 yet known in detail [13]. Using the above postulate allowed Hebb to Hebb rule, therefore we have incorporated them into this review in develop his theory further without discussing the processes inv- particular emphasis on those that have a biological counterpart. olved in changing synaptic transmission. A further simplified and In addition to the term supervised learning, the more general term popular version of describing this assumption is “Neurons that fire of reinforcement learning is often used. From a psychological, or together, wire together.” behavioral point of view, this is any learning whereby its process is Today, the phenomena of neural adaptation, training, learning facilitated (reinforced) either by (positively emotionally charged) and working memory seems to be almost a trivial fact present in reward or by (negatively emotionally charged) punishment [2]. our everyday lives as we improve our cognitive skills. By applying Originating in the behaviorism, the term reinforcement was intro- this simple rule in the study of artificial neural networks (ANN) we duced into the ANN theories by several researchers between the can obtain powerful models of neural computation that might be years 1980 and 1986 [14,18,19]. Reinforcement in ANN means close to the function of structures found in neural systems of many learning with the use of feedback input. This feedback is usually diverse species. binary, signaling only that output is to be accepted or rejected. There This review paper is divided into the following parts: after this is no information about desired output as in supervised learning. initial introduction, the application of the Hebb rule in selected The neural systems of all higher animals are hierarchically neural networks is outlined. This is succeeded by a brief review organized with many levels of connections between the neurons. discussing how biological , neurons and networks func- Lower level circuits are typical for local, mostly inhibitory inter- tion, prior to the closing section, consisting of a summary and neurons. Unsupervised learning is believed to be more likely future directions. A commented list of relevant web links can be present in these lower level biological cellular circuits and their found in the appendix. mechanisms of learning since fewer conditions are imposed on its mechanism. Hebb's view on changes in synaptic efficiency con- siders only local factors acting on corresponding neurons and synapses. To implement the Hebb rule, there is no need for any 2. Models of neural networks supervision in learning. However, in biological neural networks (BNN) there exist multiple forms of reinforced feedback that affect The above rule that Hebb proposed describes only the way in learning and employ both local mechanisms in the circuit and which synaptic efficiencies are changed in a dynamic system. global, longer range mechanisms. Many examples of biological Most artificial neural networks are characterized by two phases feedback connections that change their are found of synaptic change, learning and recall. First, in the learning in the visual pathway, specifically in the retina and visual cortex, phase there are outputs to train the network to achieve a desired which contain well described hierarchies of feedback connections. response to a given input. Secondly, in the recall phase the Global hierarchical reinforcement can be described as an abstract synaptic efficiencies do not usually change and the network is form. This higher level of description also bridges psychology with only used to calculate the response to a given input, based on the biology. From both a psychological and a biological perspective, we synaptic efficiencies calculated previously. can look at emotions as an example of tagging and reinforcing Let us illustrate the two phases using the example of the Hopfield memories [20]. network [14–16]. In this network, the Hebb rule is used in the iterative learning phase to set up the weights of input patterns 2.1. Formulations of Hebb rule successively stored in the network. This is achieved by mixing the input and required output of all the neurons in the network with the One of the simpler formulations of the Hebb rule can be fi rst pattern. After this the next pattern is learned. The activity of the written as neurons approach the desired values by repeating the learning procedure. The Hebb rule is repeated in the network to set synaptic τ dwi ¼ ð Þ out in; ð Þ w f wi r ri 1 efficiencies accordingly, while the update algorithms may vary. dt

Finally in the recall phase the weights are no longer adjusted and where weight wi of the i-th input changes with time t and time the memory retrieval of the network consist of the completed recall constant τw. This constant includes both the change rate and the of partial inputs. pre-set strength factor. According to the terminology of [21,22], τ 1 The type of learning whereby the required output is used w is a constant in the term correlating post- and pre-synaptic instead of the actual neuron output to change the synaptic weights rates. The right side of this ordinary differential equation contains fi ð Þ out in is often called the supervised learning rule. In some cases, instead of an unspeci ed function of weight f wi . r and ri are the using the neuron output directly, only the difference between the respective output and input rates. Other equivalent formulations original and the required output is used during learning. This eli- of the Hebb rule can be found in [21]. minates a change of weights if the neuron already yields the Several properties of the Hebb rule are important for its imp- required output or if it is close to the output value. In contrast, lementation in ANN. Six properties are summarized by Gerstner and applying only the changes in synaptic weights based on the actual Kistler [22]: (1) locality, meaning restricting the rule to input and neuron activity is called the unsupervised learning rule. In a strict output neurons of the given synapse (however some supervised sense, the Hebb rule in its original formulation did not include learning rule variants are not local);(2)cooperativity,therequirement supervised learning as only one synapse from neuron A to neuron B of simultaneous activity of both neurons; (3) boundedness of the is increasing its efficiency. In supervised learning there is a third weight values thereby preventing their divergence; (4) competition, factor C, which represents the additional input from another source that some synapses are strengthened at the expense of other of information, or the output of B. The efficiency here is the function synapses that are weakened; (5) long term stability, which is a of the activity of A, B and C (there are also supervised learning rules natural requirement for the dynamic stability of the neural network where the activity of B is ignored). system (this is however only one side of the dilemma of stability However, the Hebb rule can be utilized by supervised learning. For versus plasticity); (6) synaptic depression and facilitation, or weight example, the idea behind contrastive Hebbian learning is to clamp decrease and increase, which is perhaps the most important property. output neurons to desired values and then use Hebbian learning rule Synapses must be able to change in both directions, or, as is the to set the weights across the network [17]. Several other supervised case when extremal values occur in biological neurons, when new learning rules use mathematical formulations similar to that of the connections grow or existing links are disconnected. Obviously, E. Kuriscak et al. / Neurocomputing 152 (2015) 27–35 29 the purpose of decreasing efficiency is to assure the convergence properties that are analogous to holography, an imaging method of of the network to a mean net activity value [23,24]. physical optics [28].Theterm“non-holographic” indicates concepts

In Eq. (1), when f ðwiÞ is positive, because the firing rates can be developed beyond the holographic metaphor. This model has two only positive numbers, the weight only grows and diverges. layers, where each neuron of the input layer is connected with all Simply limiting the weights (according to point 3. above) would neurons in the output layer. This model can be trained to associate not eliminate the divergence problem, because at some point all the given input with the required output. When we denote the k-th ð k ; k … k Þ weights in the network will reach this limit. In any practical pattern input as x1 x2 xn and the corresponding required output fi ð k ; k … k Þ application of neural networks, a decrease of the ef ciency is as as y1 y2 yn , the synaptic weight from input neuron i to output important as its increase (point 6 above). In order to obtain a neuron j is set using the equation stable solution, we can equivalently rewrite the weight changes as ! P differences of firing rates, which can converge to a stable point: ¼ ∑ p p ; ð Þ wij H xi yj 5 p ¼ 1 dw τ i ¼ f ðw Þðrout routÞðrinðtÞrin Þ; ð2Þ w dt i 0 i i;0 where P denotes the number of patterns stored to the network and out in HðxÞ is a hard limiter function, where HðxÞ¼1forx40andHðxÞ¼0 where r0 and ri;0 are thresholds for output and input rates respectively. This variant is called the Hebb rule with post-synaptic otherwise. This is a straightforward example of the Hebb learning and pre-synaptic gating [22]. Next, we will look into how the learning rule on the synapse between neurons with forced outputs. Since the process using the Hebb rule is accomplished in selected ANNs. weights and neural states attain only 0 or 1, simultaneous activity of neurons in one pattern causes synapse to have maximum strength, or we can say, neurons that fired together once, wired together. 2.2. Feed-forward networks Further analysis of the Willshaw model can be found in [29].More biologically plausible modifications of the network with only partial The first published model of a neural network with learning connectivity between layers and probabilistic synaptic transmission capability was the perceptron [25]. The network was inspired by are compared to hippocampal cells in the work of Graham and the putative biological neural circuits involved in visual percep- Willshaw [30]. tion. Although the change of synaptic efficiency proposed by Hebb Learning rules based on back-propagation of errors are often was not used in the perceptron, Hebb's theory was discussed in criticized as biologically implausible as they assume the fast the paper. Hebb's theory was however criticized, mainly as it did propagation of information about errors along in the opposite not offer a model of behavior and only attempted to build a bridge direction. Moreover, this information must be very precise and between biophysics and psychology. The simplified model of the specific for each neuron [31]. Signaling in the BNN using various perceptron consists of three layers: sensory cell layer, association chemical compounds to alter synaptic efficiencies is relatively slow unit layer and response unit layer. The proposed learning rule only and its target areas are spread out. An example of a biologically adjusts values of the synapses from the first and second layers. more plausible learning rule for multi-layer feed-forward networks The first concept of neuronal learning in networks was demon- was introduced by Barto and Anandan [32].Theirassociative strated in the ADALINE (Adaptive Linear Neuron or later Adaptive reward-penalty rule utilizes the Hebb learning rule together with Linear Element) network, an example of perceptron, a single layer the global reinforcement signal: network [26]. The authors, Widrow and Hoff, used the steepest ( gradient descent technique, to search for the extreme function ρðxi 〈xi〉Þxj if r ¼þ1 ðrewardÞ; Δwij ¼ 0 ð6Þ values. This procedure is similar to the back-propagation learning ρ ðxi 〈xi〉Þxj if r ¼1 ðpenaltyÞ; algorithm for multi-layer feed-forward networks, developed in 0 1986 [19]. Widrow and Hoff formulated the error function as the where ρ4ρ 40 are constants and 〈xi〉 is expected (mean value) difference between the obtained neuron output and the desired output of neuron i in the network based on stochastic neural units. neuron output. They then minimized the square power of the A network incorporating a variant of this rule was successfully used defined error function as it is used in the least squares optimiza- to model visual space representation in area 7a, in the posterior tion method. Simplified formula for only one neuron output y and parietal cortex of monkeys [31]. only one pattern x ¼ðx1; x2…xnÞ can be written as follows: Another example of the learning rule in feed-forward networks based on Hebb's postulate is Sanger's rule [33] (also known as the yðxÞ¼a þa x þa x þ⋯þanxn; 0 1 1 2 2 generalized Hebbian algorithm), mainly used in principal compo- 2 ¼ ðÞð Þ ð Þ 2; ð Þ E d x y x 3 nent analysis. It is an extension of the Oja learning rule [34] for networks with multiple outputs. The Oja learning rule for one where dðxÞ denotes the desired output d1ðxÞ; d2ðxÞ…dnðxÞ. When we calculate the derivatives of square error E2 according to the neuron acting as a principal component analyzer is written as weights a and use the gradient method, we get the learning rule i Δwij ¼ αxjðxi xjwiÞ: ð7Þ formula: AtermintheHebbruleαx x was replaced by αx x0 where j i j i t ¼ t 1 þαðÞð Þ ð Þ ; ð Þ 0 ai ai d x y x xi 4 ¼ xi xi xjwi is considered as the effective input to the unit. This used by Widrow and Hoff, for the t-th iteration of the rule. This is additional feedback controls the growth of the weights and the iterative form of the supervised learning rule, with learning prevents divergence. rate α. Instead of directly assigning weights to the corresponding target value, they approach their target values with each iteration. 2.3. Recurrent networks The weight change in the next iteration is calculated based on not one but several patterns. This can potentially result in a clash, Hopfield was inspired by the spin glass theory in physics when one pattern requires an increasing weight while the other [35,36] and published a similar neural network model [14,15]. requires a decrease. There are n neurons and the output of each neuron is connected as Willshaw [27] described a two layer model of an associative an input to all other neurons. The neuron states are again binary, neural network with only binary valued neural outputs and synaptic valued originally 0 and 1, or equivalently 1 and þ1, [16]. weights (valued using only 0 and 1). The name of this model, non- This model works as an auto-associative memory and the input holographic associative memory, stems from the biological memory to the network is presented by setting up all the neuron states 30 E. Kuriscak et al. / Neurocomputing 152 (2015) 27–35

k q ; k q …k q according to the network input. The neuron states are then x1 x2 xn denotes the activities of individual neurons within updated in iterations either synchronously (with all the neuron iteration q of the cycle in pattern k. The learning rule itself is the states updated in one iteration) or asynchronously (only one same as in the Willshaw model except that it is rewritten in a form neuron is picked up and its state is updated in one iteration). that include neural activities within cycles. Further details can be The synaptic weights are assigned the values obtained by the found in [39,40]. Synaptic facilitation and depression can addi- following version of the Hebb rule: tionally enlarge the network capacity, as demonstrated in [41–43].

p ¼ ∑ k k; ð Þ wij xi xj 8 2.4. Networks of self-organizing maps and other unsupervised t ¼ 1 networks where wij denotes the synaptic weight from neuron i to neuron j, k k xi and xj denote the neural states of neurons i and j respectively in After 1980, statistics, optimization, dynamic differential equations, pattern k, and p denotes the number of patterns to be stored to the and corresponding numerical methods were rapidly developing due network. to the advances in scientific computing. From 2000 onwards, any An application of Hebb's hypothesis here would not assume reference to applications of ANN algorithms was regarded as a symmetrical synaptic weights, however, the formulation of the standard computational method, as it can be seen in a vast number learning in Eq. (8) results in symmetry. This symmetry is the of examples in technical and scientific computing [44]. essential property of the Hopfield network. Without this symme- Observations made regarding the spatial organization of neural try the network would not converge to its stable state. Further systems led to the development of other ANN learning theories. analysis of the auto-associative memories of the Hopfield type can Kohonen proposed another unsupervised neural network model, be found in [37]. that was motivated by sensory neural circuits, especially by vision. An extension of this model to include continuous neuron states This model is referred to as the self-organizing map [45]. Kohonen as a function of time was published in 1984 [15] in close succession proposed a model capable of forming a topographic map corre- to the first paper [14]. In this second paper, Hopfield proposed that sponding to the inner (spatial) organization of inputs. Such a the model could be implemented directly using electrical circuits. spatial relationship imposes a metric of the input as a metric space The learning rule is the same as in the previous paper. to synaptic weights. Consider for example the topology of a two One generalization of the Hebb rule was aimed at explaining dimensional space. In this case, neurons are viewed as nodes on a particular phenomena of the visual cortex. The BCM theory grid with lateral connections to neighboring neurons. All the (named after Bienenstock, Cooper, and Munro [23]) was proposed neurons have input connections from the same source. The lateral to account for experiments measuring the selectivity of neurons in connections are excitatory to close neurons, inhibitory to those the primary sensory cortex and selectivity changes dependent on further away and there are no lateral connections between the neuronal input. The linear neuron output y is obtained simply by neurons distant from each other. The model calculates a discrimi- summing the input xi multiplied with weight wi: nant function (an analogy for neuronal output) for every neuron y ¼ ∑wixi ð9Þ and picks up the one with the highest function value, referred to as the winner neuron. The weights of the neurons are then updated The weight is calculated by a rule expressing synaptic change as according to the equation a product of the presynaptic activity and a nonlinear function of ð Þþαð Þ ð Þ fi wi t t x t postsynaptic activity. The weight depends on modi cation thresh- w ðt þ1Þ¼ ; ð12Þ i Jw ðtÞþαðtÞxðtÞJ old θM: i where t denotes time and the network is updated in discrete time dwi ¼ ð θ Þ α ; ð Þ y y M xi wi 10 ð Þ ð Þ dt steps. Value wi t denotes the weight vector of i-th neuron, x t is the vector input presented to the network and JyJ is the metric of and decays with the factor α. The function θ ¼ Ep½ðy=y Þ sets the M 0 the Euclidean norm of the vector y. As in the equations above, modification (sliding) threshold according to the mean activity parameter αðtÞ is the learning rate, here it is a monotonously taken over all the p patterns. Synapses are facilitated or depressed decreasing function of time t. This function differs for the winner according to this equation. neuron, the nearest neighboring neurons, those further away and It has been shown that if the cycles of sparse activity are stored to the neurons most distant from the winner neuron. amodelwithaHopfield network topology and Willshaw-model-like Similar ideas about the lateral excitation and inhibition as well neurons and synapses, where both neuron outputs and synapses are as choosing the neuron with the highest response have been only binary 0 or 1, the capacity of the model can be higher than that previously discussed by Rosenblatt in [25], where reinforcement of the Hopfield network. In the Hopfield network, capacity scales learning was used instead to adjust weights. There is no visible with the number of neurons n and is proportional to 0.15n.The direct evidence of any Hebbian idea behind Eq. (12). However, the properties of sparse networks were studied under conditions where following discriminant function used in neurons is the dot product the percentage of the active neurons in the whole cycle approached of weight and input vectors: those observed in biological networks and their proportion was maintained at around 1.5% [38]. wi:x ¼ Jwi JJxJ cos ϕ ¼ JxJ cos ϕ; ð13Þ We have used another variant of the Hebb learning rule to store where ϕ is the angle between vectors wi and x and due to the the patterns in the network in a similar way to that of the normalization in Eq. (12), the vector weight norm is unity Willshaw model. When many indexes in the rule are omitted, JwiðtÞJ ¼ 1. If the weight vector is of a different direction than the form of the Hebb rule can be clearly seen. We use weight the presented pattern, the angle ϕ will be decreased by the modification in cyclic activation sequences: !! learning Eq. (12), which increases the cosine value in Eq. (13). r lðkÞ1 This observation can be rephrased as follows: the response of ¼ ∑ k lðiÞ k 1 þ ∑ k q k ðq þ 1Þ ; ð Þ wij H xi xj xi xj 11 the winner neuron and the nearest neighboring neurons to the k ¼ 1 q ¼ 1 presented input is increased by the learning rule. From this point where r is the number of patterns stored, kx denotes the k-th of view the self-organized map is close to the Hebb rule and pattern, l(k) denotes the k-th pattern's cycle length, kx1 ; kx2 …kxlðkÞ increases the weights in a way that the target neuron output is denotes the network activities within the cycle iterations and higher. However, this is done simultaneously for all input weights E. Kuriscak et al. / Neurocomputing 152 (2015) 27–35 31

Table 1 Neural networks with biological correlates. Table 1 shows the hypothetical biological neural mechanisms used as paradigms for the construction of artificial neural networks.

Name Artificial neural network (ANN) Biological (BNN) correlates

Hopfield ANN It is an auto-associative network by Hopfield [14]. It uses CA3 region in hippocampus is one of several brain regions resembling Hebb rule in iterative learning and update the Hopfield ANN [46] Perceptron It consists of layers of neurons with converging All sensory pathways, including the visual pathway, contain pathways of connections [25] converging axons [46] ADALINE ADAptive LInear NEuron uses an optimization algorithm No biological correlate to our knowledge in learning [26] Holographic model It was proposed by Pribram [28] and Willshaw [27] Image storage in hologram is a metaphor of biological memory. At present it is mostly discussed metaphorically BCM learning rule It is named after Bienenstock, Cooper and Munro [23] It was originally described as a mechanism of the visual cortex and can be used in ANN Kohonen ANN It is also called self-organizing map, it is an ANN with It was originally proposed by Kohonen to model neocortical columns [45] a geometrical organization Cyclic ANN It is an example of a more elaborated Hebb rule [39] Wilson [47] proposes such cyclic activity in the model of the hippocampus. Many other neural circuits function with the use of periodic activation.

and subsequently the normalization can decrease the weight even associative neural network [56], similar to that of Hopfield's [46]. though it would be increased by the regular form of the Hebb rule. Its function in mammals is related to memory, emotions, motiva- Biological neural networks discussed in this section are sum- tion and space navigation, via connections through the limbic marized in Table 1. system circuits to the rest of the brain [20]. Shepherd's book The Synaptic Organization of the Brain, from which we highlight the chapter on hippocampus and its CA3 area [46], serves as a good 3. Comparison of artificial and biological neuronal networks literary source for mathematical and engineering readers. Biological neurons are functional units signaling their activity by In this section we discuss how artificial neural networks,whichare spikes (action potentials), that evoke intricate sub-cellular machinery, frequently parts of computer software applications, are influenced by as they pass the synaptic connections between neurons. At the studies of biological neural networks, made of real living neurons. synapse, an electrical signal of the pre-synaptic membrane is con- Artificial neural networks were first developed in the mid-to-late verted into a chemical signal transmitted by molecules called med- 1940s [3], inspired by neural theories formulated from the 1920s iators. These have either excitatory or inhibitory effects on the post- onwards, that explain signal processing in real nervous systems synaptic membrane. A common example of an excitatory mediator is [9,48]. The time lag between these observations and their computa- Glutamic Acid (GLU), whilst a common example of an inhibitory tional implementations was due to the lack of practically usable mediator is γ-Amino-Butyric Acid (GABA). Inhibition is frequently computational hardware and an absence of sufficiently elaborated realized by local interneurons [46,38]. Two examples of excitatory mathematical and information theories. These did not appear until postsynaptic membrane receptors, named after their drug activators, thelate1940s[10,49].Sincethenbothfields of research mutually are α-Amino-3-hydroxy-5-Methyl-4-isoxazole-Propionic Acid (AMPA) inspired each other, but retained separation. The advancement of and N-Methyl-D-Aspartate (NMDA). Calcium (Ca2þ )ionsactasimpor- artificial neural networks was motivated by the desire to mimic and tant intra-cellular messengers of processes started at the neural replicate fundamental biological, neurophysiological and psychologi- membrane [57,58]. The mediator molecules themselves in turn act cal phenomena (e.g. adaptation, memory and cognition). Gradually on the post-synaptic membrane, giving rise to the post-synaptic this became an applied science oversimplifying some processes and potential which then contributes to the generation of the spike in omitting redundant and detailed biological features, including the the post-synaptic neuron. The propagation of chemical signals in the Hebb rule. In spite of this approach and the core aspects of the opposite direction, from the post-synaptic neuron back to the pre- artificial neural networks that use Hebb rule variants, the simplifica- synaptic neuron, and the coincident activation of both these neurons tions proved fruitful and successfully mimicked many attributes and within the short (millisecond) time frame are two putative mechan- functions of biological neural circuits. isms enabling the realization of the Hebb rule in biological neurons. Studies of BNN have shown that many discovered mechanisms Two examples of messengers of retrograde neuronal signals are intra- have not yet been implemented in any artificial neural network. cellular Ca2þ ions [59] and nitric oxide (NO, both intra- and extra- Examples include the signal processing on dendrites and dendritic cellular) [60]. Spikes converging from two neurons are grouped spines changing with time [50,51], the nonlinear summation of together on virtually all possible time ranges, but not all of them elicit postsynaptic currents and shunting effects [52], and the variable synaptic changes [5,61,62]. signal propagation delays and effects of spike timing jitter [53–55]. There are numerous experimental protocols stimulating neurons Such ideas later entered the field of computational neuroscience, in either intact nervous system (in vivo), or in brain slices and the branch of research exploring, modeling and analyzing biolo- neuronal cultures maintained in a dish (in vitro), or alternatively in gical neural circuits by means of software and hardware computa- detailed models reproducing biologicalneuralmachineryincompu- tional resources, numerical and information processing. ter hardware (in silico). In experimental protocols, synaptic weight The motivation stemming from biological neurons and neuro- changes are referred to as potentiation, depression, plasticity, and nal circuits continues to inspire ANN design. Many of the observa- also Hebbian changes. Short term potentiation (STP) and long term tions are based on mammalian brain studies. Hierarchical layers of potentiation (LTP) are two phenomena elicited in hippocampal slices functional units and auto-associative connections are present in and other preparations [46]. The spike timing dependent plasticity biological neural nets. The hierarchy of the visual pathway was (STDP), and changes following the BCM rule are assumed to take used by Hebb and his numerous followers as an example of a place also in intact brain functioning in vivo [23,24,63–66]. layered structure. The structure of the CA3 area in the hippocam- There are numerous examples of reproducible behaviors in biolo- pus (Cornu Ammonis area 3) is believed to contain an auto- gical neurons. In behavioral, psychological and biological experiments, 32 E. Kuriscak et al. / Neurocomputing 152 (2015) 27–35 one can manipulate newly perceived patterns of a visual scene [53], Even before the advent of the ANN, their solutions had already been complex of speech-like utterances [67] and other sensory devised, for example (1) the classification of data was solved using stimuli. Even perceptual illusions are used in experimental manipula- statistical factor analysis, cluster analysis or Bayesian classification; tions [68] and their neuronal mechanisms established. Numerous (2) the recognition of patterns could be achieved by covariance or attempts to bridge biology and psychology have not yet grasped the correlation analysis as well as their successors like principal com- exactness of studied phenomena and are far from complete. Our ponent analysis; (3) curves could be fitted by regression methods or perceptions retain some of their emotional and sensory qualities even their generalized variants, such as the generalized least squares as recollections from our memory. How are these seamlessly executed method, and also by polynomials and splines fitting; and (4) time and experienced phenomena realized? How are our inner thoughts series forecasts could be performed by (nonlinear) auto-regressive and memories represented by biological neurons? These and many moving average methods. These examples are mostly statistical other similar questions are left as part of a wide array of open pro- methods with overlapping types of tasks and solutions. The ANN blems in neurobiology. There remains a long way to closing the gap solution to these classical problems thereby is sometimes more between the multitude of elementary phenomena of experimental robust, or it is used as a method of abstraction of the original biology mentioned above to psychological and behavioral phenomena, problem yielding a more concise solution. which were the subject of Hebb's highly celebrated book [69]. As a practical example, consider a task to classify vehicles on a highway. There is a toll gate equipped with several standard industrial laser scanners, which have a given angular resolution 4. Beyond Hebb's assumptions and future directions and frame rate. Motor vehicles pass under the gate at a certain cruising speed. The task of the software is to classify vehicles into The processes of learning executed by the ANN models mentioned several categories: personal cars, busses and trucks of a particular here are inspired by Hebb's original idea. But learning rules are very size and weight. This task includes both pattern recognition and often not exactly the rule originally proposed by Hebb. The funda- classification. The task is accomplished by a three layer feed- mental concept maintained is that changes in the synaptic efficiency forward ANN. The use of the ANN enables to combine both vehicle are based on the local properties and activity of the corresponding shape recognition and classification into one algorithm [44]. neurons. Various simulations and approaches have been used in Before some of the data mining techniques were made possible models as well as in biological experimentation on neurons. It was by computation numerics, it was known that some problems would originally hypothesized that a straight line existed between the be harder to solve than others. This led to the concept of ill posed and learning and recall processes, where a change in synaptic efficiency well posed problems, and the subsequent development of regular- is part of the learning process, while other neural activities caused by ization methods to convert the former to the latter. Regularization synaptic transmission are part of the recall process. and optimization methods are therefore useful alternatives of ANN The detailed models of signal transmission in biological synapses application. One example of a generally considered harder problem is known today describe highly complex dynamics of underlying synap- the task of inferring a 3D structure based on 2D information. This tic changes [70,54].Thesechangesinfluence the parameters that task can be alternatively modified to an intermediary problem, describe synaptic dynamics [24]. New variables and more degrees of creating a 2þ1/2 D object sketch based onprojectioninformation freedom in single synapse descriptions have been introduced. The of a 2D object [18]. Another example of a harder problem is the natural question that comes to mind is what exactly should be con- production of meaningful sentences based on grammatical rules. sidered as part of the learning or the recall process. This is especially When attempts are made to cross beyond the mathematical for- relevant when designing novel ANN mechanisms. It can be seen from mulation of such hard problems using ANN, we find that these experiments and some models that synaptic efficiencies change problems relate to questions of artificial intelligence. Yet it seems that dynamically and employ time ranges that are fast enough to compare the decision as to what problems are contained in the field of to duration of interspike intervals and could also surely affect memory artificial intelligence is rather arbitrary and has changed more fre- recall processes under certain circumstances [70]. It has been shown quently over the years than problems solved by classical ANN. that short term potentiation can improve the capacity of auto-ass- Another future challenge for the ANN is the demand for higher ociative memory. modularity and scalability. There is a need for modular interfaces, Synapses in a neural network can be strengthened either via a which would make it possible to connect the output from one supervised, or an unsupervised weight change process. Hebb pro- neural network to the input of another. In addition to this not posed a rule of weight changes in time. The application of this rule many ANNs to date contain modules within modules, or even hasbeenfruitfulinthedesignofartificial neural networks. There is several levels of nesting modules. The major difference between growing evidence to show that, in biological neural networks, ANN and BNN is that in the latter modularity of both interconnec- variations of this rule are employed. This justifies the reference to tions (in neural pathways) and nesting of modules (in neural theHebbruleeveninthedescriptionofhigherlevelmemory nuclei, with specific reference to the columnar organization of the processes and inspires studies of synaptic weight change mechanisms cerebral cortex) are ubiquitous. in both artificial and biological neural networks. 4.2. Future perspectives of BNN 4.1. Future perspectives of ANN The worm, Caenorhabditis elegans, is commonly used animal model In order to discuss the future perspectives of ANN computations, in biology, with a nervous system consisting of exactly 302 cells. Their we must first divide problems solved by the ANN into two groups. connectivity is well described and function of individual neurons has The first of these are problems with known solutions. These are been identified to some extent. A connectivity map of this system prototypical tasks for the ANN and their solutions are well guides us in understanding its functioning. To date, the connectivity described. The second ones are open problems, where the solutions, and functional maps of higher organisms are only partially described. or even heuristic approaches are not known. Some examples of Examples of them are invertebrates such as the fruit fly, Drosophila problems successfully solved using an ANN are as follows: (1) the melanogaster, or vertebrates such as the rhesus macaque, Macaca classification of data into several categories; (2) the recognition of mulatta, and also humans. New methods of tracing connections and input patterns, typically 2D images; (3) curve fitting; and (4) the probing neuronal function are now available. Most findings extend the forecasting of future data, based on a time series of historical data. observation of the modularity and universality of neurons and E. Kuriscak et al. / Neurocomputing 152 (2015) 27–35 33 synapses as functional units, which make up individual specialized 2. MATLAB (RM) like matrix computation numerical libraries are neural circuits of the animal brain. in free open source programs: (1) Octave, http://www.gnu.org/ Most synapse strengthening biological protocols have been software/octave/ and (2) SciLab, https://www.scilab.org/. described in simple, isolated preparations. Definition of the STDP is 3. MATLAB (RM) sources to the book of Wilson [47] are at his a generalization of the LTP and LTD protocols, variations of these homepage: http://cvr.yorku.ca/webpages/wilson.htm#book. protocols have resulted in description of the STDP. The neuromuscular 4. Open source book by Gerstner and Kistler, Neuron Models [21] synapse of the African clawed frog Xenopus laevis embryo cells has is available at http://icwww.epfl.ch/gerstner/SPNM/node72. been used as in vivo preparation demonstrating the STDP [71]. html. Gerstner et al. [72] presented a theoretical argument that in order 5. Elsevier Freedom collection (for academic institutions) con- to explain the fine tuning of the circuit encompassing first binaural tains many open source papers [57,67,74] at https://info. neuron in higher animals, we need learning of fine tuned spike timing myelsevier.com/?q=products/collections/freedom-collection. triggered neural response. A natural way of uncovering the biological 6. JSTOR (Journal STORage) available from academic subscrip- mechanisms is a progress from embryonic, or in vitro isolated tissues, tions contains many papers of historical reference and primary or from lower animal preparations to more complex systems, and sources at http://www.jstor.org. later the phenomena are found in a more subtle form in neurons of 7. Neurocomputing journal offers open access to older papers later developmental stages, then in vivo and finally in higher animals. [40,41,43,61,62] at http://www.journals.elsevier.com/neuro The succession of experimental studies of the LTP, LTD and STDP computing/. follows this chronological order. In this succession, the back- 8. Proc. Natl. Acad. Sci. USA journal has an open archive, where propagating action potential was one of the missing links conveying [6,14,15,31,53,56,59,60] and other papers can be found at the synaptic information in anti-dromic direction [57].Itwaslater http://www.pnas.org/. shown that this effect was required for the LTP as well [73].Recently, 9. Some papers of our group are related to both ANN and BNN. They the STDP has been demonstrated not only in synapses from excitatory are [44,53–55,57],the newer are available on request, the older are to excitatory neurons, but also in excitatory to inhibitory and mostly open source now and are at (1) http://nemo.lf1.cuni.cz/ inhibitory to excitatory synapses. Therefore it is not surprising that mlab/marsalek-HOME/;or(2)http://scholar.google.cz/citations? the STDP is present in both hippocampus and neocortex in the view_op=search_authors&mauthors=PetrþMarsalek. excitatory to excitatory synapses and a variant of the LTP can be 10. Scholarpedia contains many papers related to Hebb learning: demonstrated in where most of the circuitry uses (1) BCM theory; (2) Hebb [75]; (3) Hebb rule; (4) Hopfield inhibitory synapses. The effect of dopamine of the “reward” pathway network [16]; (5) Kohonen network; (6) LTD; (7) LTP; (8) mem- in neocortex is a conceptual link of supervised learning, or reinforce- ory; (9) models of synaptic plasticity; (10) reinforcement ment, between the synapses and behavior [65]. learning; (11) STDP [66] and (12) supervised learning; the However, many fundamental findings, whether structural or pages are at http://www.scholarpedia.org/. functional, still wait for plausible reasoning, explanation or imple- mentation into ANN models. Although our understanding of the nervous system contains many conceptual elements adopted from References and elaborated by ANN, an abundance of new findings in funda- mental research have shown that biological neural networks use a [1] D.O. Hebb, The Organization of Behavior, Wiley, New York. Reprinted in 2002 multitude of computationally diverse processes. Our explanation of by: Lawrence Erlbaum Associates, NJ, 1949. them is made possible by the Hebb rule and other principles [2] I.P. Pavlov, Conditioned Reflexes, an Investigation of the Physiological Activity fi of the Cerebral Cortex, Oxford University Press, London, UK, 1927. implemented already into arti cial neural networks. There is no [3] W.S. McCulloch, W. Pitts, A logical calculus of the ideas immanent in nervous doubt that some of these findings that are yet to be elaborated by activity, Bull. Math. Biophys. 5 (4) (1943) 115–133. molecular biology, quantum chemistry, biological physics, or any [4] A.M. Turing, On computable numbers, with an application to the Entschei- – quantitative branch of natural sciences will profoundly influence dungsproblem, Proc. Lond. Math. Soc. 42 (1936) 230 265. [5] L.A. Jeffress, A place theory of , J. Comput. Physiol. Psychol. our understanding of both artificial and biological neural networks. 41 (1) (1948) 35–39. In order to limit the number of literary references to a reasonable [6] C.E. Carr, M. Konishi, Axonal delay lines for time measurement in the 's – scale, we have deliberately not cited many further papers or books brainstem, Proc. Natl. Acad. Sci. USA 85 (21) (1988) 8311 8315. [7] V. Laufberger, Vzruchova theorie, (The Impulse Theory), Czech Medical in this last section. We advise the reader to direct her attention to Association, Prague, 1947 (in Czech). the world wide web pages listed in Appendix A. [8] L.J. Kohout, A Perspective on Intelligent Systems: A Framework for Analysis and Design, Chapman and Hall, London, New York, 1990. [9] E.D. Adrian, Y. Zotterman, The impulses produced by sensory nerve-endings: Part II. The response of a single end-organ, J. Physiol. Lond. 61 (2) (1926) Acknowledgments 151–171. [10] N. Wiener, Cybernetics: Or Control and Communication in the Animal and the Machine, Hermann and Cie, 1948 (2nd revised ed., Wiley, New York, 1961). This work was supported by graduate students research pro- [11] W. James, Pragmatism: A New Name for Some Old Ways of Thinking, Longman gram SVV 2014 no. 260 033 to P.M., J.S. and P.G.T. and PRVOUK Green and Company, New York, NY, 1907. program no. 205024 to P.M.,both programs at Charles University in [12] C.U.A. Kappers, G.C. Huber, E.C. Crosby, The Comparative Anatomy of the Nervous System of Vertebrates, Including Man, Macmillan, New York, NY, Prague. Thanks to Elisa Brann, Libor Husnik, Jiri Kofranek and 1936. Martin Vokurka. [13] R.E. Brown, P.M. Milner, The legacy of Donald O. Hebb: more than the Hebb synapse, Nat. Rev. Neurosci. 4 (12) (2003) 1013–1019. [14] J.J. Hopfield, Neural networks and physical systems with emergent collective computational abilities, Proc. Natl. Acad. Sci. USA 79 (8) (1982) 2554–2558. Appendix A [15] J.J. Hopfield, Neurons with graded response have collective computational properties like those of two-state neurons, Proc. Natl. Acad. Sci. USA 81 (10) – We gathered several web reference sources of mostly computer (1984) 3088 3092. [16] J.J. Hopfield, Hopfield network, Scholarpedia 2 (5) (2007) 1977. source codes and open source books and papers. Most of the [17] J.R. Movellan, Contrastive Hebbian learning in the continuous Hopfield model, sources listed below were visited on July 22, 2014: in: Proceedings of the 1990 Connectionist Models Summer School, 1990, pp. 10–17. [18] D. Marr, Vision: A Computational Investigation into the Human Representation 1. MATLAB (RM) library sources of ANN simulations are at http:// and Processing of Visual Information, Henry Holt and Company, New York, NY, www.mathworks.com/products/neural-network/. 1982. 34 E. Kuriscak et al. / Neurocomputing 152 (2015) 27–35

[19] D.E. Rumelhart, G.E. Hinton, R.J. Williams, Learning representations by back- [56] S.R. Kelso, A.H. Ganong, T.H. Brown, Hebbian synapses in hippocampus, Proc. propagating errors, Nature 323 (6088) (1986) 533–536. Natl. Acad. Sci. USA 83 (14) (1986) 5326–5330. [20] E.R. Kandel, In Search of Memory: The Emergence of a New Science of Mind, [57] P. Marsalek, F. Santamaria, Investigating spike backpropagation induced Ca2þ W.W. Norton and Company, New York, NY, 2007. influx in models of hippocampal and cortical pyramidal neurons, Biosystems [21] W. Gerstner, W.M. Kistler, Neuron Models: Single Neurons, Populations, 48 (1–3) (1998) 147–156. Plasticity, Cambridge University Press, Cambridge, MA, 2002. [58] M. Mlcek, J. Neumann, O. Kittnar, V. Novak, Mathematical model of the [22] W. Gerstner, W.M. Kistler, Mathematical formulations of Hebbian learning, electromechanical heart contractile system—regulatory subsystem physiolo- Biol. Cybern. 87 (5–6) (2002) 404–415. gical considerations, Physiol. Res. 50 (4) (2001) 425–432. [23] E.L. Bienenstock, L.N. Cooper, P.W. Munro, Theory for the development of [59] A. Zador, C. Koch, T.H. Brown, Biophysical model of a Hebbian synapse, Proc. neuron selectivity: orientation specificity and binocular interaction in visual Natl. Acad. Sci. USA 87 (17) (1990) 6718–6722. cortex, J. Neurosci. 2 (1) (1982) 32–48. [60] T.J. O'Dell, R.D. Hawkins, E.R. Kandel, O. Arancio, Tests of the roles of two [24] L. Benuskova, W.C. Abraham, STDP rule endowed with the BCM sliding diffusible substances in long-term potentiation: evidence for nitric oxide as a threshold accounts for hippocampal heterosynaptic plasticity, J. Comput. possible early retrograde messenger, Proc. Natl. Acad. Sci. USA 88 (24) (1991) Neurosci. 22 (2) (2007) 129–133. 11285–11289. [25] F. Rosenblatt, The perceptron, a probabilistic model for information storage [61] P. Marsalek, Neural code for sound localization at low frequencies, Neuro- and organization in the brain, Psychol. Rev. 65 (6) (1958) 386–408. computing 38–40 (2001) 1443–1452. [26] B. Widrow, M.E. Hoff Jr., Adaptive switching circuits, IRE WESCON Conv. Rec. 4 [62] P. Marsalek, J. Kofranek, Sound localization at high frequencies and across the (1960) 96–104. frequency range, Neurocomputing 58–60 (2004) 999–1006. [27] D.J. Willshaw, O.P. Buneman, H.C. Longuet-Higgins, Non-holographic associa- [63] Y. Dan, M.M. Poo, Spike timing-dependent plasticity of neural circuits, Neuron tive memory, Nature 222 (5197) (1969) 960–962. 44 (1) (2004) 23–30. [28] K.H. Pribram, The neurophysiology of remembering, Sci. Am. 220 (1) (1969) 73–86. [64] S. Song, Hebbian learning and spike-timing dependent plasticity, in: J. Feng [29] D. Golomb, N. Rubin, H. Sompolinsky, Willshaw model: associative memory (Ed.), Computational Neuroscience: A Comprehensive Approach, CRC Press, with sparse coding and low firing rates, Phys. Rev. A 41 (4) (1990) 1843–1854. Boca Raton, FL, USA, 2004, pp. 324–358. [30] B. Graham, D. Willshaw, Probabilistic synaptic transmission in the associative [65] N. Caporale, Y. Dan, Spike timing-dependent plasticity: a Hebbian learning net, Neural Comput. 11 (1) (1999) 117–137. rule, Annu. Rev. Neurosci. 31 (2008) 25–46. [31] P. Mazzoni, R.A. Andersen, M.I. Jordan, A more biologically plausible learning [66] J. Sjöström, W. Gerstner, Spike-timing dependent plasticity, Scholarpedia 5 (2) rule for neural networks, Proc. Natl. Acad. Sci. USA 88 (10) (1991) 4433–4437. (2010) 1362. [32] A.G. Barto, P. Anandan, Pattern-recognizing stochastic learning automata, IEEE [67] J.L. Eriksson, A.E.P. Villa, Learning of auditory equivalence classes for vowels by Trans. Syst. Man Cybern. 15 (3) (1985) 360–375. rats, Behav. Process. 73 (3) (2006) 348–359. [33] T.D. Sanger, Optimal unsupervised learning in a single-layer linear feedfor- [68] R. von der Heydt, E. Peterhans, G. Baumgartner, Illusory contours and cortical ward neural network, Neural Netw. 2 (6) (1989) 459–473. neuron responses, Science 224 (4654) (1984) 1260–1262. [34] E. Oja, Simplified neuron model as a principal component analyzer, J. Math. [69] T.J. Sejnowski, The book of Hebb, Neuron 24 (4) (1999) 773–776. Biol. 15 (3) (1982) 267–273. [70] M. Tsodyks, H. Markram, The neural code between neocortical pyramidal [35] E. Ising, Beitrag zur Theorie des Ferromagnetismus, Z. F. Phys. A 31 (1) (1925) neurons depends on neurotransmitter release probability, Proc. Natl. Acad. Sci. 253–258 (in German). USA 94 (2) (1997) 719–723. [36] D. Sherrington, S. Kirkpatrick, Solvable model of a spin-glass, Phys. Rev. Lett. [71] T. Morimoto, X.H. Wang, M.M. Poo, Overexpression of synaptotagmin mod- 35 (26) (1975) 1792–1796. ulates short-term synaptic plasticity at developing neuromuscular junctions, [37] D.J. Amit, Modeling Brain Function. The World of Attractor Neural Networks, Neuroscience 82 (4) (1998) 969–978. Cambridge University Press, Cambridge, MA, 1989. [72] W. Gerstner, R. Kempter, J.L. van Hemmen, H. Wagner, A neuronal learning [38] E.T. Rolls, A. Treves, Neural Networks and Brain Function, Oxford University rule for sub-millisecond temporal coding, Nature 383 (1996) 76–78. Press, New York, NY, 1998. [73] L.N. Cooper, STDP: spiking, timing, rates and beyond, Front. Synaptic Neurosci. [39] J. Stroffek, E. Kuriscak, P. Marsalek, Pattern storage in a sparsely coded neural 2 (14) (2010) 00014-1–3. network with cyclic activation, Biosystems 89 (1–3) (2007) 257–263. [74] G.E. Hinton, Connectionist learning procedures, Artif. Intell. 40 (1) (1989) [40] J. Stroffek, P. Marsalek, Short-term potentiation effect on pattern recall in 185–234. sparsely coded neural network, Neurocomputing 77 (1) (2012) 108–113. [75] R.M. Klein, Donald Olding Hebb, Scholarpedia 6 (4) (2011) 3719. [41] J. Torres, J. Cortes, J. Marro, H. Kappen, Attractor neural networks with activity- dependent synapses: the role of synaptic facilitation, Neurocomputing 70 (10–12) (2007) 2022–2025. [42] D. Bibitchkov, J.M. Herrmann, T. Geisel, Pattern storage and processing in Eduard Kuriscak received his M.D. in the field of attractor networks with short-time synaptic dynamics, Netw. Comput. Neural General Medicine from Palacky University, Olomouc, Syst. 13 (1) (2002) 115–129. in 1998, and his Ph.D. in Normal and Pathological [43] D. Bibitchkov, J.M. Herrmann, T. Geisel, Effects of short-time plasticity on the Human Physiology from Charles University, Prague, in associative memory, Neurocomputing 44-46 (2002) 329–335. 2002. He is also trained as an applied physicist. He was [44] J. Stroffek, E. Kuriscak, P. Marsalek, Highway toll enforcement, IEEE Veh. awarded the Hlavka scholarship for young scientists of Technol. Mag. 5 (4) (2010) 56–65. the Czech Republic. He was a coordinator of several [45] T. Kohonen, Self-organized formation of topologically correct feature maps, system administration and software development pro- Biol. Cybern. 43 (1) (1982) 59–69. jects, and is currently an assistant professor at the [46] T.H. Brown, A.M. Zador, Hippocampus, in: G.M. Shepherd (Ed.), The Synaptic Department of Physiology, First Medical Faculty, Organization of the Brain, Oxford University Press, New York, 1990, Charles University, Prague, since 1998. pp. 346–388. [47] H.R. Wilson, Lyapunov functions and memory, in: Spikes, Decisions and Actions, Oxford University Press, New York, 1999, pp. 223–250. [48] J.C. Eccles, Some aspects of Sherrington's contribution to neurophysiology, Notes Rec. R. Soc. Lond. (1957) 216–225. [49] C.E. Shannon, A mathematical theory of communication, Bell Syst. Tech. J. 28 (4) (1949) 656–715. Petr Marsalek is a professor of Biophysics and Com- [50] F. Santamaria, S. Wils, E.D. Schutter, G.J. Augustine, The diffusional properties puter Science in Medicine at Charles University in of dendrites depend on the density of dendritic spines, Eur. J. Neurosci. 34 (4) Prague. There he received his M.D., in 1990, his B.S. in (2011) 561–568. Mathematics and Computer Science, in 1992, and his [51] C. Sala, M. Segal, Dendritic spines: the locus of structural and functional Ph.D. in Biophysics, in 1999. His postdoctoral stays have plasticity, Physiol. Rev. 94 (1) (2014) 141–188. included California Institute of Technology and Johns [52] S. Cushing, T. Bui, P.K. Rose, Effect of nonlinear summation of synaptic currents Hopkins University in the USA, and the Max Planck on the input-output properties of spinal motoneurons, J. Neurophysiol. 94 (5) Institute for the Physics of Complex Systems in Dres- (2005) 3465–3478. den, Germany. He is affiliated with the Department of [53] P. Marsalek, C. Koch, J. Maunsell, On the relationship between synaptic input Pathological Physiology at the First Medical Faculty of and spike output jitter in individual neurons, Proc. Natl. Acad. Sci. USA 94 (2) Charles University in Prague and part time at the Czech (1997) 735–740. Technical University in Prague. [54] E. Kuriscak, P. Marsalek, Model of neural circuit comparing static and adaptive synapses, Prague Med. Rep. 105 (4) (2004) 369–380. [55] E. Kuriscak, P. Marsalek, J. Stroffek, Z. Wunsch, The effect of neural noise on spike time precision in a detailed CA3 neuron model, Comput. Math. Methods Med. 2012 (595398) (2012) 1–16. E. Kuriscak et al. / Neurocomputing 152 (2015) 27–35 35

Julius Stroffek is currently working towards his Ph.D. Peter G. Toth is currently working towards his Ph.D. at at the Department of Pathological Physiology, First the Department of Pathological Physiology, First Med- Medical Faculty, Charles University in Prague. He ical Faculty, Charles University in Prague. He received received his master's degree in Computer Science, in his master's degree in Computer Science, in 2011, at the 2004, at the Faculty of Mathematics and Physics, Faculty of Mathematics and Physics, Charles University Charles University in Prague, under the supervision of in Prague, under the supervision of Petr Marsalek. Petr Marsalek. After his graduation he has been work- ing as a software engineer at multinational software companies.