Mo delling Bino cular Neurons in the Primary Visual Cortex David J Fleet David J Heeger Hermann Wagner Intro duction Neurons sensitive to bino cular disparityhave b een found in the visual cortex of many mammals and in the visual wulst of the owl and are thought to play a signicant role in stereopsis Barlow et al Nikara et al Hub el and Wiesel Clarke et al Pettigrew and Konishi Poggio and Fischer Fischer and Kruger Ferster Poggio and Talb ot OhzawaandFreeman a Ohzawa and Freeman b LeVayandVoigt Ohzawa et al DeAngelis et al Wagner and Frost A number of physiologists have suggested that disparity might b e enco ded by a shift of receptiveeld p osition Hub el and Wiesel Pettigrew et al Pettigrew Maske et al Poggio et al Wagner and Frost According to this p ositionshift mo del disparity selective cells combine the outputs of similarly shap ed mono cular receptive elds from dierent retinal p ositions in the left and righteyes More recently Ohzawa et al and DeAngelis et al have suggested that disparity sensitivity might instead b e a result of intero cular phase shifts In this phaseshift mo del the centers of the left and righteye receptive elds coincide but the arrangement of receptive eld subregions is dierent This chapter presents a formal description and analysis of a bino cular energy mo del of disparity selectivity According to this mo del disparity selectivity results from a combination of p ositionshifts andor phaseshifts Our theoretical analysis suggests how one might p erform an exp eriment to estimate the relative contributions of phase and p osition shifts to the disparity selectivity of bino cular neurons based on their resp onses to drifting sinusoidal grating stimuli of dierent spatial frequencies and disparities We also show that for drifting grating stimuli the bino cular energy re sp onse with phase andor p osition shifts is a sinusoidal function of dispar ity consistent with the physiology of neurons in primary visual cortex area of the cat Freeman and Ohzawa However Freeman and Ohzawa also found that the depth of mo dulation in the sinusoidal disparity tuning curves was remarkably invariant to intero cular contrast dierences This is inconsistent with the bino cular energy mo del As a consequence we prop ose a mo died bino cular energy mo del that incorp orates two stages of divisive normalization The rst normalization stage is mono cular preceding the combination of signals from the twoeyes The second normalization stage is bino cular Our simulation results demon strate that the normalized bino cular energy mo del provides the required stability of the depth of resp onse mo dulation Simulations also demonstrate that the mo dels mono cular and bino cular contrast resp onse curves are con sistent with those of neurons in primary visual cortex Bino cular Interaction and Disparity Selectivity There are two ma jor classes of neurons in primary visual cortex V of the monkey or A of the cat namely simple cells and complex cells Hub el and Wiesel Both typ es are selective for stimulus p osition and orientation They resp ond vigorously to stimuli of a preferred orientation but less well or not at all to stimuli of other orientations Many neurons are also disparity selective Disparitysensitive cells are often divided into four typ es tunedexcitatory tunedinhibitory near and far Poggio and Fischer Disparity selec tivity in these dierenttyp es might arise from dierentmechanisms Poggio and Fischer Ferster but see Nomura et al for the opp osite p oint of view Tunedinhibitory near and far cells usually receive a strong excitatory input from one eye and an inhibitory input from the other eye ie the mono cular inputs are unbalanced and most of them do not show bino cular facilita tion Tunedexcitatory cells show a sharp resp onse peak due to bino cular facilitation the resp onses at disparities anking the peak are often inhib ited and they have balanced mono cular inputs This chapter is primarily concerned with tunedexcitatory complex cells although all of the analysis and conclusions could b e applied to tunedexcitatory simple cells as well Linear Neurons and Energy Neurons There is a long tradition of mo deling simple cells as linear neurons Hub el and Wiesel Campb ell et al Campb ell et al Movshon et al a OhzawaandFreeman a Hamilton et al This mo del is attractive b ecause a linear neuron can be characterized with a relatively small numb er of measurements Model ling Binocular Neurons in the Primary Visual Cortex left right monocular A linear neuron binocular + B linear neuron monocular C energy + neuron + binocular D energy + neuron + Fig Receptive elds of mo del neurons A A mono cular orientation selective linear neuron Its resp onse dep ends on a weighted sum of the stimulus intensi ties within its receptive eld Shaded ellipses depict inhibitory subregions of the weighting function and the unshaded ellipse depicts an excitatory subregion B A bino cular linear neurons resp onse dep ends on aweighted sum of the stimulus intensities presented to both eyes The reference points black dots b elow the weighting functions indicate that the twoweighting functions are in exact bino cu lar corresp ondence C A mono cular energy neuron sums the squared resp onses of two mono cular linear neurons The weighting functions of the two linear neurons are identical except for a phase shift D A bino cular energy neuron sums the squared resp onses of two bino cular linear neurons All four linear weighting functions are centered in exact mono cular and bino cular retinal corresp ondence Figure A shows a schematic diagram of a mono cular linear neuron A linear neurons resp onse is a weighted sum of stimulus intensities within a small region of the entire visual eld called the neurons receptive eld In the illustration the three ellipses depict subregions of the receptive eld one with p ositiveweights the unshaded ellipse and two with negativeweights the shaded ellipses The neuron is excited when a bright light is ashed in the positive subregion and inhibited when a bright light is ashed in a negative subregion Bright lights ashed simultaneously in b oth p ositive and negative subregions tend to cancel The p ositive and negative weights are balanced so the neuron do es not resp ond to blank stimuli Rather its resp onse is prop ortional to stimulus contrast for patterned stimuli that vary in intensityover space and time Figure B depicts a bino cular linear neuron This neurons resp onse dep ends on a weighted sum of the stimulus intensities presented to both eyes The left and righteye receptive elds are identical for the neuron depicted in the gure but this need not be the case in general Also the left and righteye receptive elds of this particular linear neuron are in exact bino cular corresp ondence as indicated by the small reference points below the weighting functions One problem with the linear mo del of simple cells is that linear neurons can have negative resp onses b ecause they sum input intensities using b oth p ositive and negativeweights However extracellular resp onses ring rates of real neurons are by denition p ositive Neurons with a high maintained ring rate could enco de p ositive and negative values by resp onding either more or less than the maintained rate But simple cells havevery little main tained discharge Instead p ositive and negative values may b e enco ded by two neurons one resp onsible for the p ositive part and one for the negative part The two neurons are complements of one another an excitatory subre gion of one neurons receptive eld is aligned with an inhibitory subregion of the other neurons receptiveeld The resp onse of each neuron is halfwave rectied so that only one of the two neurons has a nonzero resp onse at any given time Simple cells are often characterized as halfwaverectied linear neurons eg Movshon et al a Heeger b not have discrete ON and OFF receptive eld subre Complex cells do gions and have b een mo deled as energy neurons Adelson and Bergen Emerson et al Heeger b Pollen and Ronner An energy neuron sums the squared resp onses of a quadrature pair of linear neurons that are out of phase but with otherwise identical tuning prop erties Fig C Equivalently an energy neuron could sum the squared resp onses of four halfwaverectied linear neurons Model ling Binocular Neurons in the Primary Visual Cortex The mono cular energy neuron depicted in Fig C has one linear sub unit that is evensymmetric even phase and another that is o ddsymmetric o dd phase but this is not necessary The critical prop erty is that the two subunits must be in quadrature phase phase shift Although simple cell weighting functions are not necessarily even or o ddsymmetric Field and Tolhurst Heggelund Jones and Palmer a the re ceptive elds of adjacent simple cells tend to exhibit or phase relationships Foster et al Liu et al Palmer and Davis Pollen and Ronner A lo cal pool of simple cells thus provides the right combination of signals for an ideal energy neuron Approximately the same behavior may be obtained by summing the squared resp onses of many linear neurons or halfwaverectied linear neurons regardless of their phase but with receptive elds distributed over a lo cal spatial region A bino cular energy neuron Ohzawa et al is depicted in Fig D This neuron sums the squared resp onses of a quadrature pair of bino cular linear neurons This chapter is primarily concerned with the behavior of bino cular energy neurons Disparity