AN ABSTRACT MODEL OF A CORTICAL HYPERCOLUMN

Baran Çürüklü1, Anders Lansner2

1Department of Computer Engineering, Mälardalen University, S-72123 Västerås, Sweden 2Department of Numerical Analysis and Computing Science, Royal Institute of Technology, S-100 44 Stockholm, Sweden

ABSTRACT preferred stimulus was reported to be lower than to preferred stimulus. An abstract model of a cortical hypercolumn is presented. According to the findings by Hubel and Wiesel [16] This model could replicate experimental findings relating the primary has a modular structure. It is to the orientation tuning mechanism in the primary visual composed of orientation minicolumns each one cortex. Properties of the orientation selective cells in the comprising some hundreds of pyramidal cells and a primary visual cortex like, contrast-invariance and smaller number of inhibitory interneurons of different response saturation were demonstrated in simulations. We kinds. Contrast edge orientation is coded such that the hypothesize that broadly tuned inhibition and local cells in each orientation minicolumn respond selectively excitatory connections are sufficient for achieving this to a quite broad interval of orientations. Further, the behavior. We have shown that the local intracortical orientation hypercolumn contains orientation minicolumns connectivity of the model is to some extent biologically with response properties distributed over all angles, and plausible. thus represents the local edge orientation pertinent to a given point in visual space. A similar modular arrangement is found in many other cortical areas, e.g. rodent whisker barrels [15]. 1. INTRODUCTION The Bayesian Confidence Propagation Neural Network model () has been developed in analogy Most in the primary visual cortex (V1) with this possibly generic cortical structure [14]. This is respond to specific orientations even though relay cells in an abstract neural network model in which each unit the lateral geniculate nucleus (LGN), that carries the corresponds to a cortical minicolumn. The network is information from retina to V1, does not show evidence of partitioned into hypercolumn-like modules and the orientation selectivity. It is not known in detail how summed activity within each hypercolumn is normalized orientation selectivity of the cells in V1 emerges and the to one. issue is hotly debated (for a recent review see Ferster et The above network model relates to the so-called al., [9]). Hubel and Wiesel [10] proposed that orientation normalization models of V1 proposed by Albrecht et al. selectivity of simple cells in V1 was a consequence of [20] and Heeger [21] that address properties of simple synaptic input from LGN. Still today the Hubel and cells mentioned above. These assume that input from the Wiesel feedforward model serves as a model of thalamic LGN grows linearly with contrast stimulus. This input is input to cortex. However many of the properties of divided by a linearly growing inhibitory input. The effect orientation selective cells in V1 cannot be predicted by is division of the input from the LGN and that the such a feedforward model. Contrast-invariance of summed activity of the cells in a hypercolumn is orientation tuning seen by simple and complex cells is normalized. This would correspond to saturation of a cells perhaps the most striking example. As contrast increases activity. Later Carandini et al. [22,23] proposed that a the height of the response curve increases while the width pool of cells with different preferred orientations and remains almost constant [11,12,13]. spatial frequencies drives the shunting inhibition. It was also shown that response to contrast stimulus Cross-orientation inhibition is yet another feature of increases over approximately 50-60% of the response cortical simple and complex cells. Response to range and this behavior is followed by a rapid saturation superposition of two gratings is less than sum of each and normalization of the cells activity [13]. The saturation response alone [8,25]. Morrone et al. [8] suggested that level seems to be determined by stimulus property this inhibition arises from a pool of cells with different (orientation, spatial frequency) and not by electrical orientations. The cross-inhibition effect could be properties of the cells. Maximum response to non- A B 180°

90°

0° 900 µm

C Output from the minicolumn D

2 chandelier 100 % Percent of cells maximum Output response layer Network of 12 excitatory cells One 600 µm basket Network cell of 11 excitatory Input cells layer 40 % 3 chandelier cells

56 µm

Contrast 100 % Input from cortex and LGN

Figure 1. A, The hypothesized repetitive layout of the cat V1 D, A scheme showing the activity of the excitatory cells demonstrated by 7 hypercolumns. B, The partial in two minicolumns with preferred orientation (bottom hypercolumn model used during the simulations consisted solid lines) and non-preferred orientation (bottom dashed of 17 minicolumns. C, A scheme showing one of the lines). Input layer excitatory cells (exponential curves) are subsampled orientation minicolumns and the basket cell driving the basket cell (thick top curve). The output layer representing the pool of inhibitory cells. Excitatory cells excitatory cells (sigmoidal curves) are normalized when in the input layer are connected to the basket cell, and the the basket cell’s activity increases linearly. basket cell inhibits the excitatory cells in the output layer. explained by a shunting inhibition proposed by the two excitatory neurons inside a minicolumn layer was a normalization models [22,23]. function of the distance between them [6]. Here we present an abstract model of a cortical The implication of this scheme for the output layer is hypercolumn derived from the above-mentioned BCPNN that, the distribution (in terms of orientation) of the architecture. Our main intention has been to address inhibitory input to an excitatory cell is broader than the response saturation and contrast-invariance of orientation excitatory input. This is because the basket cell represents tuning behaviors of cortical cells. Initial tests were a pool of orientation specific neurons inhibiting a pool of showing that cross-orientation inhibition was also excitatory neurons with all possible preferred orientation. prominent. All these behaviors could be achieved by a Even though the connectivity pattern seen in the output very simple network architecture (Fig. 1). layer is very simple, it is still biologically plausible. Kisvárday et al. [27] reported that, in an area of the size of 2. NETWORK MODEL cat V1 hypercolumn, 56 % of the excitatory and 47 % of the inhibitory connections were at iso-orientation, while The foundation of our network model is the columnar cross-inhibition was shown by 14 % of excitatory and 20 organization seen in V1 and elsewhere in the cortex [16]. % of inhibitory connections respectively. This indicates We assume that V1 is composed of repetitive structures, that, the inhibitory network is less orientation specific so-called minicolumns, and that these minicolumns rotate than the excitatory network. A study based on ferret around hypothetical centers [18], and form hypercolumns prefrontal microcircuits is also pointing in the direction of (Fig. 1a). In our model a hypercolumn is represented by a the basket cells as responsible for gain control of the local finite number of minicolumns, each representing a cortical network [26]. particular orientation (Fig. 1b). For sake of simplicity we Besides the basket cell, there were other inhibitory decided to use a partial hypercolumn model composed of cells in the model. These cells were the local inhibitory 17 subsampled orientation minicolumns, ranging from 0° chandelier cells [28] located inside the minicolumns, and to 180°, with the angular distance of 11.25° between two hence inhibiting excitatory cells with the same orientation successive ones (Fig. 1b). The diameter of the cylinder preference. In the input layer three chandelier cells shaped minicolumns was 56 µm, as a consequence of the inhibited an excitatory cell, while two chandelier cells study done by Peters et al. [17] on cat V1. In that study, it inhibited the excitatory cells in the output layer. was reported that apical dendrites of layer V pyramid cells Cortical neurons are known for their irregular spiking formed clusters with a center-to-center spacing of about activity [3,4], and were thus modeled as Poisson 56 µm, which provides an estimate of the mean distance processes. As the kernel of the Poisson process we used a between two successive minicolumns. The circular leaky integrate-and-fire model [1,2]. The role of the leaky arrangement of the minicolumns gives the biologically integrate-and-fire model was to sum the presynaptic plausible hypercolumn diameter of 900 µm for cat V1. inputs to generate the membrane potential of the cells. The minicolumns has a height of 600 µm, and are Maximum frequency of the excitatory and the inhibitory abstractions of the layer II-IV of cat V1 [17]. Each of the neurons were 100 Hz and 300 Hz respectively. Half- subsampled minicolumns is composed of 28 neurons (Fig. height of the IPSPs were 10 ms, and 15 ms for the EPSPs. 1c) and the population is heterogeneous with all Mean amplitudes of the EPSPs inside the minicolumns values sampled from a uniform distribution with a were 0.94 mV for the input layer, and 9.4 mV for the standard deviation of 10%. output layer. The strength of the synaptic connection The hypercolumn model consists of two separate between the input layer excitatory cells and the basket cell layers with two different tasks in order to achieve was set to give an EPSP of 3.5 mV. IPSPs generated by normalization behavior proposed by the BCPNN model the chandelier cells had a mean of -3.2 mV, and that of the (Fig. 1c). It will be shown later that the excitatory neuron basket cell was -55.1 mV. Observe that the values are in the output layer replicates experimental findings exaggerated, specially the IPSP generated by the basket relating to the orientation tuning mechanism in V1. cell, for compensating the small number of cells used in There were no connections between these two layers, the network model. It is assumed that in cortex some 20 % and thus the interaction between them was through one of the cells are various inhibitory cells [31]. The PSP basket cell, representing a pool of inhibitory cells (Fig. 1b values and number of connections, especially inhibitory and 1c). This layout resulted in feedforward connectivity ones, were calculated to preserve this ratio between the inside the hypercolumn model. Every input layer inhibitory and the excitatory populations in V1. The PSP excitatory cell was connected to the basket cell, and thus values were sampled from a uniform distribution with a could drive it. The basket cell was connected to every standard deviation of 10%. excitatory cell in the output layer. There were no An axonal diameter of 0.3 µm [29] resulted in a spike connections between excitatory cells situated in different propagation velocity of 0.85 m/s [30]. orientation minicolumns. Connection probability between Contrast response function components, both constant currents. The first was a

12 function of the contrast stimulus increase, and the second was, besides of contrast, also a function of the orientation 10 of the postsynaptic cell. Tuning of this second component was 40° half-width at half-height [19] both for the 8 excitatory and the chandelier cells. Observe that the basket cell did not receive any input from the LGN. The 6 orientation dependent LGN input at 100 % contrast, for

Response (Hz) Response 4 cells with the preferred orientation was 2.7 nA for excitatory cells, and 0.9 nA for chandelier cells. 2 Orientation independent part of the LGN input was 0.9 nA for all excitatory cells, and 0.32 nA for all chandelier 0 0 20 40 60 80 100 cells. Observe that input to cells having non-preferred Contrast orientation during high contrast might exceed input to cells having preferred orientation during low contrast as a Figure 2. Contrast response function curves corresponding to results of the orientation independent part of the LGN mean activity of excitatory cells situated in the output layer. Top curve corresponds to cells situated in the input. As the background activity all excitatory cells minicolumn having the same orientation preference (90°) received additional current input. Excitatory cells in the as the input to the hypercolumn. The thick curve in the output layer received 2 nA, while input layer excitatory middle corresponds to mean activity of cells in all 17 cells received 1 nA. To guarantee that the inhibitory cells minicolumns. The bottom curve corresponds to activity of were active in their logarithmic range these cells received cells having a 45° orientation preference. 3.5 nA throughout the simulation. The current values were sampled from a uniform distribution with a standard deviation of 10%. Orientation tuning Experimental findings related to the orientation tuning 12 15 % contrast mechanism in V1, and thus normalization in the BCPNN 30 % contrast framework [14] is possible to achieve by assuming that 10 60 % contrast excitatory and inhibitory cells are active in specific

8 regions of their gain functions, and that these regions define their ranges. The sigmoidal gain function of the 6 Poisson neuron could be divided roughly into two regions; the low activity region (<50 %) would

Response (Hz) Response 4 correspond to the exponential function, and the high activity region (≥ 50%) to the logarithmic function. We 2 assume here that the excitatory cells are in their low activity region and the inhibitory cells are in their high 0 -80 -60 -40 -20 0 20 40 60 80 activity region. Orientation (degrees) In order to analyze the network behavior we start with the input layer, and later continue with the output layer. Figure 3. Contrast-invariance demonstrated by the network. Excitatory cells in the input layer of the hypercolumn Selectivity remains constant while the peak increases as a function of increasing contrast. model behaved like cells in a Hubel and Wiesel feedforward model [10]. This was not a surprise because 3. SIMULATION RESULTS orientation tuning of these cells was a function of the LGN input. Remember that the excitatory cells One important assumption made was the linear response approximated the exponential function, and hence of the LGN cells to the contrast stimulus increase. This amplified their input. This resulted in the narrowing of the assumption was in line with the normalization models orientation tuning. Carandini et al. [19] reported that the mentioned above. Results by Movshon et al. [24] indicate half-width at half-height of the tuning of the spike ° that the majority of LGN cells (namely P cells) have linear responses was approx. 23 while membrane responses response functions and shows very little or no sign of were approx. 38°. Their finding could motivate the saturation as a function of contrast stimulus increase. Our narrowing of the half-width at half-height of the model does not have a LGN component, hence the LGN orientation tuning. At the same time, activity of the cells input is modeled as a constant current. During the having non-preferred orientation was increased above simulations we defined the input to the modeled cells in resting activity levels as an effect of the increased the following way. The simulated LGN had two contrast. This resulted in widening of the orientation from the basket cell to the excitatory cells in the output tuning curve. layer. Remember that linear increase of the contrast However, activity shown by the excitatory cells in the results in exponentially increased activity of the input output layer (Fig. 2) corresponded well to the reported layer excitatory cells, and that these cells are driving the results of the nonlinear behavior of simple and complex basket cell. The basket cell linearizes the input from these cells [13]. The first region corresponded to the dynamic cells, because the response function of the basket cell is response range of the cortical cells. During this phase the logarithmic. As a result, the basket cell responds to activity of the cells increased monotonically as a function contrast in a linear fashion. Output from the basket cell is of increased contrast. This phase was followed by a rapid then fed into the output layer excitatory cells. The main saturation. During the last phase the cells were normalized part of the input received by these excitatory cells is from i.e. their activity was constant even though the contrast the LGN input and this intracortical inhibition. Observe was increasing. It should be stressed that saturation of that, in theory, both these inputs increase linearly with activity was evident in all cells independent of their contrast. This means that these two inputs have constant orientation preferences, and that, this level was not a and positive slopes. Consequently, the relative difference function of the cells electrical properties as reported by between them corresponds to a constant value, and hence [13]. defines a cells activity during the normalized phase. It was shown by Albrecht et al. [13] that the contrast Within the dynamic response range (contrast < 50%), response function could be approximated by a hyperbolic net input to excitatory cells is increasing. The reason for function this is that the excitatory cells in the input layer cannot drive the basket cell. When a certain threshold (contrast ≈ n n n Response(C) = Rmax·(C / (C + C50 )) 40 %) is reached the input to the basket cell is strong enough (Fig. 2) to drive it. n where C50 defined contrast value that was required to The cross-inhibition effect was also tested during the produce 50% of the cell’s maximum response. Rmax was simulations (not shown here). In the presence of one the cell’s maximum response rate. It was also reported in additional line stimulus the basket cell’s activity increased that study that contrast response curves were shifted resulting in a stronger inhibition of the excitatory cells vertically downward as the stimulus orientation diverged than in case with a single line stimulus. It is also from the preferred orientation. This would mean that Rmax straightforward to see that activity of the basket cell is changed while C50 and n remained relatively constant. dependent of the contrast of the additional line stimulus. This behavior was believed to be important for preserving the relative frequency response function independent of 4. DISCUSSION the contrast [12,13]. The excitatory cells in the output layer of the model hypercolumn had all these properties We have presented an abstract model of a cortical (Fig. 2). The Rmax levels, <12 Hz, were below maximum hypercolumn derived from the BCPNN architecture. This frequency levels (≈100 Hz) governed by the electrical model could replicate important experimental findings properties of the cells (Fig. 2). C50 levels (approx. 24%) of relating to the orientation tuning mechanism in the modeled cells were biologically plausible (Fig. 2). primary visual cortex. Properties of the orientation Contrast dependent inhibition was reported by Sclar et selective cells in the primary visual cortex like, contrast- al. [11]. According to their results, as the contrast invariance and response saturation were demonstrated. increased activity of the cells having orientation One important assumption made was the linear response preference that differed significantly from the stimulus of the LGN cells to the contrast stimulus increase. As a orientation decreased below their spontaneous activity result of this assumption, we showed that the levels. This behavior was also demonstrated in our normalization of the cells in the output layer could be simulation, as seen when comparing the low and high explained by the local connections inside the contrast curves (Fig. 3). Cells having orientation hypercolumn. preference that differed more than approximately 50° Narrowing of the orientation tuning was possible from the stimulus orientation were inhibited below their through the reinforcement of the LGN input by the spontaneous activity levels. Contrast-invariance of excitatory cells. As a side effect, cells having non- orientation tuning in simple and complex cells could be preferred orientation were excited above their resting seen as an effect of this contrast dependent inhibition. activity levels, and this affected their orientation tuning Both contrast response function and contrast- negatively. The divisive inhibition of the excitatory cells invariance of orientation tuning could be explained by our in the output layer by the basket cell resulted in network architecture. In order to explain interactions in sharpening of the orientation tuning curves and detail we would like to focus on the connections from the normalization of their activity. The basket cell represented excitatory cells in the input layer to the basket cell, and a pool of inhibitory cells with a mixture of preferred orientations. The activity of the basket cell was a function spatial frequency discrimination: a comparison of single of the excitatory cells in the input layer, and thus cells and behaviour”, J. of Neurophysiology, 57:773–86, represented the total activity inside the hypercolumn. This 1987. network configuration is supported by studies made on cat [13] D.G. Albrecht, D.B. Hamilton, “Striate cortex of monkey and cat: contrast response function”, J. of Neurophysiology, V1. 48:217–37, 1982. It is well known that the long-range horizontal [14] A. Sandberg, A. Lansner, F.M. Petersson, and Ö. Ekeberg, intracortical connections play an important role in V1. “A Bayesian attractor network with incremental learning” The impact of such connections to the orientation tuning Network: Computing in Neural Systems, in press, 2002. mechanism of the cortical cells will be addressed in the [15] D. Purves, D.R. Riddle, A-S LaMantia, “Iterated patterns of near future. One experiment will be to simulate cortical circuitry (or how the cortex gets its spots)”, TINS, 15 plasticity in the framework of the BCPNN incremental 362–8, 1992. learning algorithm. In these experiments, stimuli defined [16] D. Hubel, T.N. Wiesel, “The functional architecture of the as lines with random orientations moving across the macaque visual cortex. The Ferrier lecture.” Proc. Royal. model cortex will provide the activity patterns required Soc. B 198: 1-59, 1977. [17] A. Peters, E. Yilmaz, “Neuronal organization of area 17 of for the learning algorithm to form assemblies of cat cortex”, , 3:49-68, 1993. connected minicolumns. Our intention is to look into how [18] V. Braitenberg, C. Braitenberg, “Geometry of the well these resemble cortical connectivity patterns seen in orientation columns in the visual cortex”, Biol. Cyber., V1 and how they influence the response dynamics of the 33:179-186, 1979. network. [19] M. Carandini, D. Ferster, “Membrane potential and firing rate in cat primary visual cortex”, J. of Neurosci., 20(1):470- 5. REFERENCES 484, 2000. [20] D.G. Albrecht, W.S. Geisler, “Motion selectivity and the [1] W. Gerstner, Pulsed Neural Networks, The MIT Press, contrast-response function of simple cells in the visual Chapter 1, 1998. cortex”, Visual Neuroscience, 7:531–46, 1991. [2] W. Kistler, W. Gerstner, and J.L. van Hemmen, “Reduction [21] D.J. Heeger, “Normalization of cell responses in cat striate of Hodgkin-Huxley equations to a threshold model”, Neural cortex”, Visual Neuroscience, 9:181–97, 1992. Comput., 9:1069-1100, 1997. [22] M. Carandini, D.J. Heeger, “Summation and division by [3] C. Koch, Biophysics of Computation: Information neurons in visual cortex”, Science, 264:1333–36, Processing in Single Neurons, Oxford University Press, 1994. Chapter 15, 1999. [23] M. Carandini, D.J. Heeger, and J.A. Movshon, “Linearity [4] M.N. Shadlen, W.T. Newsome, “The Variable Discharge of and normalization in simple cells of the macaque primary Cortical Neurons: Implications for Connectivity, visual cortex”, J. of Neurosci., 17:8621–44, 1997. Computation, and Information Coding”, J. of Neurosci., [24] J.A. Movshon, M.J. Hawken, L. Kiorpes, A.M. Skoczenski, 18(10):3870–3896, 1998. C. Tang, and L.P. O’Keefe, “Visual noise masking in [5] V. Braitenberg, A. Schüz, CORTEX: Statistics and Geometry macaque LGN neurons”, Invest. Ophthalmol. Vis. Sci. of Neuronal Connectivity, Chapter 36, Springer, 1998. [Suppl] 35:1662, 1994. [6] B. Hellwig, “A quantitative analysis of the local connectivity [25] A.B. Bonds, “The role of inhibition in the specification of between pyramidal neurons in layer 2/3 of the rat visual orientation selectivity of cells in the cat striate cortex”, cortex”, Biol. Cybern., 82:111-121, 2000. Visual Neuroscience, 2:41–55, 1989. [7] T.W. Troyer, A.E. Krukowski, N.J. Priebe, and K.D. Miller, [26] L.S. Krimer, P.S. Goldman-Rakic, “Prefrontal “Contrast-invariant orientation tuning in cat visual cortex: Microcircuits: Membrane properties and excitatory input of thalamocortical input tuning and correlation-based local, medium, wide arbour interneurons”, J. of Neurosci., intracortical connectivity”, J. of Neurosci., 18:5908–27, 21(11):3788-3796, 2001. 1998. [27] Z.F. Kisvárday, É. Tóth, M. Rausch, and U.T. Eysel, [8] M.C. Morrone, D.C. Burr, and L Maffei, “Functional “Orientation-specific relationship between population of implications of cross-orientation inhibition of cortical visual excitatory and inhibitory lateral connections in the visual cells. I. Neurophysiological evidence”, Proc. R. Soc London cortex of the cat”, Cerebral Cortex, 7:605-618, 1997. Ser. B 216:335–54, 1982. [28] I. Farinas, J. DeFelipe, “Patterns of synaptic input on [9] D. Ferster, K.D. Miller, “Neural Mechanisms of Orientation corticortical and cortithalamic cells in the cat visual cortex. Selectivity in the Visual Cortex”, Annual Reviews of II The initial segment”, J. Comp. Neurol., 304, 70-77, Neuroscience, 23:441-471, 2000. 1991. [10] D.H. Hubel, T.N. Wiesel, “Receptive fields, binocular [29] V. Braitenberg, A. Schüz, CORTEX: Statistics and interaction and functional architecture in the cat’s visual Geometry of Neuronal Connectivity, Springer, 1998. cortex”, J. Physiol., 160:106-154, 1962. [30] C. Koch, Ö. Bernander, Axonal Modeling, in: M.A. Arbib [11] G. Sclar, R.D. Freeman, “Orientation selectivity in the cat’s (Ed.), The Handbook of Brain Theory and Neural Networks, striate cortex is invariant with stimulus contrast”, Exp. Brain 129-134, The MIT Press, 1998. Res., 46:457–61, 1982. [31] C.D. Gilbert, J.A. Hirsch, and T.N. Wiesel, “Lateral [12] B.C. Skottun, A. Bradley, G. Sclar, I. Ohzawa, and R. interactions in cat visual cortex Cold Spring Harbor Symp. Freeman, “The effects of contrast on visual orientation and On Quantitative Biology vol LV”, Cold Spring Harbor Press, 663-76, 1990.