1. Introduction: Global Capture of a Moving

2570 J. Opt. Soc.Am. A/Vol. 14,No. 10/0ctober 1997 Cheyet at.

JonathanChef. StephenGrossberg. and Ennio Mingolla Departmentof Cognitiveand Neural Systemsand Centerfor Adaptive Systems,Boston University, 677Beacon Street,Boston, Massachusetts 02215

Received December 20, 1995; revised manuscript received March 12, 1997; accepted March 18, 1997 A neural network model of visual motion perception and speed discrimination is developed to simulate data concerning the conditions under which components of moving stimuli cohere or not into a global direction of motion, as in barberpole and plaid patterns (both type 1 and type 2). The model also simulates how the perceived speed of lines moving in a prescribed direction depends on their orientation, length, duration, and contrast. Motion direction and speed both emerge as part of an interactive motion grouping or segmentation process. The model proposes a solution to the global aperture problem by showing how information from feature tracking points, namely, locations from which unambiguous motion directions can be computed, can propagate to ambiguous motion direction points and capture the motion signals there. The model does this without computing intersections of constraints or parallel Fourier and non-Fourier pathways. Instead, the model uses orientationally unselective cell responses to activate directionally tuned transient cells. These transient cells, in turn, activate spatially short-range filters and competitive mechanisms over multiple spatial scales to generate speed-tuned and directionally tuned cells. Spatially long-range filters and top-down feedback from grouping cells are then used to track motion of featural points and to select and propagate correct motion directions to ambiguous motion points. Top-down grouping can also prime the system to attend a particular motion direction. The model hereby links low-level automatic motion processing with attention- based motion processing. Homologs of model mechanisms have been used in models of other brain systems to simulate data about visual grouping, figure-ground separation, and speech perception. Earlier versions of the model have simulated data about short-range and long-range apparent motion, second-order motion, and the effects of parvocellular and magnocellular lateral geniculate nucleus lesions on motion perception. @ 1997 Optical Society of America [S0740-3232(97)00810-7] Key words: Vision, motion perception, directional grouping, speed perception, aperture problem, attention, visual cortex, neural network.

1. INTRODUCTION: GLOBAL CAPTURE the ambiguous motion signals near the line middle. The OF A MOVING OBJECT'S DIRECTION preferred ambiguous motion direction and speed are nor- AND SPEED mal to the line's orientation. For a vertical line, the When an object moves,aperture ambiguity and image or speed of the feature tracking signals at the line ends detector noise often prevent all but a small subsetof its equals that of the preferred ambiguous speed near the image features, suchas its bounding contours,from gen- line middle. For a tilted line, the preferred ambiguous erating unambiguousmotion direction cues. Despitethis speed is less than that of the feature tracking speed. If fact, object percepts often seem to pop out with a well- the speed of the line is judged by using a weighted aver- defined motion direction and speed. The present paper age of feature signals and ambiguous signals, then the developsa neural modelof how this feat is accomplished tilted line will be perceived to move slower than the ver- by the brain. tical line, as found by Castet et ai.1; see Fig. 2(a). These Five general designprinciples motivate the model'sde- data also show that ambiguous speeds have a greater ef- velopment. The first is that unambiguousfeature track- fect as line length increases when the line is briefly ing signals capture and transform ambiguousmotion sig- viewed. Feature tracking signals at the line ends thus nals into coherent representations of object direction. propagate inward along the line to capture ambiguous Two classic examplesof feature tracking are shown in motion speeds and directions. Since capture takes longer Fig. 1. A secondprinciple is that object direction and to complete when lines are longer, ambiguous motion sig- speedare both emergentproperties of this process. nals have a larger effect on longer lines. Theseconsiderations lead to the model'scentral design Our model simulates data of Castet et ai.1 It also problem: What type of feature tracking processescan se- simulates how the barberpole illusion is produced,2 how it lect unambiguous direction and accurate speed signals is affected by configurational changes, and how plaid pat- from ambiguousmotion signals? For example,consider terns move both coherently and incoherently. In particu- the horizontal motion of both a vertical and a tilted line lar, the model provides explanations of when moving that move at the same speed. Supposethat the unam- plaid patterns cohere or do not,3-5 how contrast affects biguous feature tracking points at the line ends capture their perceived speed and direction,6 and why movement

0740-3232/97/1002570-25$10.00 @ 1997 Optical Society of America Cheyet at. Vol. 14,No. 10/0ctober 1997/J. Opt. Soc.Am. A 2571

of type 2 patterns differs from that of type 1 patterns. 7-9 tor sum direction because of temporal delay between Fou- BownslO has provided experimental evidence against the rier and non-Fourier information and argues in favor of a Yo-Wilson9 hypothesis that type 2 plaids move in the vec- feature tracking explanation. These simulations build upon earlier ones that employ a subset of model mechanisms that do not include its mo- <" tion capture process.l1 The earlier model incorporates a third design principle: Perceived motion direction and speed are a collective property of multiple populations of cells with different receptive field sizes, or scales. This multiscale network models the short-range motion processof Braddick,12 including the fact that the short-range motion limit Dmaxdepends on the spatial frequency con- tent of the image.13-17 In the model, larger scales pref- erentially process higher speeds. A key problem in de- signing this multiscale short-range motion filter is to keep the largest scale from winning at all motion speeds, because it has a larger receptive field that can attain a higher activation level. Once this problem is solved, the model simulates how visual speed perception, discrimination, and reaction time are affected by stimulus contrast, duration, dot density, and spatial frequency.l,7.18-29 -- A fourth design principle concerns how nearby contours -local signals percept of different orientation and contrast polarity that are moving in the same direction cooperate to generate a (b) pooled motion direction signal. This process is modeled Fig. 1. Feature tracking and motion capture: Often only a by a long-range motion filter that has been used to simu- small subset of image features (e.g.,line ends)generate unam- late data about long-range apparent motion (Kolers30), in- biguous motion direction signals,as a result of aperture ambigu- cluding beta, gamma, delta, reverse, split, Ternus, and ity and image or detectornoise. We modelhow these unambigu- reverse-contrast Ternus motion and Korte's laws.31-33 ous locationsgenerate feature tracking signals that are used to capture ambiguoussignals: (a) moving line, (b) barberpoleillu- A fifth design principle concerns motion capture. The sion. key problem here is how to capture the correct motion di-

(a) (b)

~ Q) Q) 0.. (.f) >Q) -~ ~Q)

Relative Orientation (deg) Fig. 2. Effects of line length and orientation on perceivedspeed of horizontally moving lines. Relative perceivedspeed for three different line orientationsand lengths are shownas percentages of the perceivedspeed of a vertical line of the samelength. Part (a) shows data from Castetet at. (Ref. 1,p. 1925). Eachdata line correspondsto a different line length (0.21,0.88, and 1.76deg). The horizontal axis showsthe ratio of the speednormal to the line's orientation relative to the actual translation speed. The three data points from left to right for eachline length correspondto line angles of 60, 45, and 30 deg from vertical, respectively. The horizontal dotted line indi- catesa veridical speedperception; results belowthis line indicate a bias toward the perceptionof slowerspeeds. Part (b) showssimu- lation results, also for three lengths and orientations. In both casesperceived relative speeddecreases with line length and angle from vertical. Simulated lines use slightly different orientations from those in the experimentsso that the simulated input conformsto the Cartesiangrid.

---"" 2572 J. Opt. Soc.Am. A/Vol. 14,No. 10/0ctober 1997 Chey et at.

rection without distorting its speed estimate. Surpris- through experiments showing that the direction of motion ingly, motion capture can be achieved by a long-range of the individual components and the plaid as a whole re- grouping network that also allows attention to prime a spond differently to stimulus paramet.er changes. For desired motion direction. In other words, motion cap- example, Adelson and Movshon3 showed that contrast ture, which seemsto be an automatic and preattentive thresholds for the detection of plaid components alone process,is carried out by the same circuit that permits were lower than contrast thresholds for the detection of top-down attention to focus selectively on a desired coherently moving plaids. Ferrera and Wilson 7 showed direction.34-36 Cavanagh37has described an attention- elevations in detection thresholds for plaid patterns fol- based motion process,in addition to low-level or auto- lowing masking by plaids with different component matic motion processes,and has shown that it provides angles, suggesting that masking reduced the response of accurate velocity judgments. The present work models mechanisms tuned to the plaid motion direction but not to how the attentive processand the motioncapture process the component directions. Movshon et at. 52 identified are linked and therebyexplains howthe attentive process motion-sensitive cells in visual area MT that were tuned yields accurate velocityjudgments. either to the direction of individual components or to the This capture-and-priming circuit obeysthe samerules direction of the plaid. as attentive priming circuits that have been used to modelother brain processes,such as early form vision,visual object recognition, auditory streaming, and speech 3. HOW IS AMBIGUOUS MOTION perception.38 Line motion illusion,39,40motion induct- RESOLVED? ion,41-43and transformational apparent motion44,45have Given that a long-range motion stage operates on the out- also beensimulated by using these model motion mecha- puts of short-range motion detectors, how are the two nisms as they interact with visual forms!6.47 stages combined to generate a global motion signal for a Unlike modelsof Yo and Wilson9 or Castet et al.,1 the moving form? Adelson and Movshon3 suggested that plaid present modeldoes not postulate the existenceof distinct motion is determined by the intersection of constraint channels for processingFourier and non-Fourier signals lines of each component. The intersection-of-constraints or for processingfeature tracking and locally ambiguous (IOC) solution does not, however, always correctly de- motion signals. Instead, a single hierarchically orga- scribe motion data. Ferrera and Wilson 7,8 constructed nized processingstream reacts appropriatelyto both fea- several exceptions to the IOC solution by using type 2 ture tracking and ambiguousmotion signals in a context- plaids for which the motion vector given by the IOC lies sensitivefashion. This processingstream is incorporated outside the arc formed by the motion vectors perpendicu- into larger multistream model systems to explain data lar to the two components [Fig. 3(a)]. Such plaids more about form-motion interactions and visual search.31,47-5o rigorously test the IOC solution than do type 1 plaids, Multiple streams are not, however,needed to explain the whose IOC solution direction lies within the two compo- data that are consideredhere. The stages that are nent directions. Ferrera and Wilson showed that, in cer- needed.'~ll first be functionally motivated beforetechni- tain situations, the motion of type 2 plaids is biased to- cal details are addressed. ward directions normal to the component orientations and that their perceived speed is slower than that pre- dicted by the lOCo Rubin and Hochstein53 showed that 2. SHORT-RANGE AND LONG-RANGE when viewing moving lines through an aperture, observ- MOTION PROCESSING ers misjudged direction away from the IOC solution and Several types of data suggestthat both spatially short- toward the vector sum of directions perpendicular to the range and long-rangemechanisms are involved in motion component orientations. processing. One classical source is data about short- Perceived type 1 plaid directions can also deviate from range and long-range apparent motion. Short-rangeap- the IOC solution. Stone et al.6 showed that when the two parent motion typically operates over distancesunder 15 components are of unequal contrast, the plaid direction is min of visual angle and at interstimulus intervals under biased toward the direction normal to the component with 100 ms.12,51Long-range apparent motion operatesover higher contrast. They modified the IOC rule to account distancesup to severaldegrees and overinterstimulus infor this bias by using an additional processing stage that tervals up to 500 ms (Kolers30). Following the proposal of Grossbergand Rudd,32,33the presentmodel further de- velops the hypothesis that both short-range and long- (a) (b) range motion is mediated by a hierarchically organized , processwithin a single processingstream. Various other perceptsthan apparent motion also sug- gest that initial localized motion measurementsare fol- lowed by a long-range motion pooling stage. Plaid patterns3 typically consistof two overlappingsinusoidally or square-wavemodulated luminance gratings. Such a pattern has two possiblevisual interpretations: as a pair Fig. 3. (a) In type 2 plaids, the IOC solution lies outside the arc formed by the directions normal to the components. In (b) the of independently moving componentsor a single coher- IOC solution predicts motion upward, whereas the vector sum of ently moving plaid. Evidence for a motion pooling pro- the component directions lies between the directions of the two cessunderlying coherentplaid motion has been obtained components.

" Chey et at. Vol. 14,No. 10/0ctober 1997/J. Opt. Soc.Am. A 2573

makes a speedestimate that dependson componentcon- grated easily when features signaled different motion di- trast. Stone and Thompson26showed that the speed of rections in eachaperture but is integrated when only am- high-contrast gratings is perceivedas faster than that of biguous motion signals filled the apertures. Thus the gratings of lower contrast. Therefore it is reasonableto two sorts of motion signals appear to be averaged,with assume that a higher-contrast component will have a feature signals given more importance than ambiguous faster perceivedspeed, biasing the IOC calculated direc- signals. tion toward that component'smotion vector, Given that the IOC solution cannot easily explain all the data, notably the observedbiases in type 2 plaid mo- 4. MOTION COHERENCE AND tion, what alternatives are there? Plaid direction could INCOHERENCE be calculated as the vector sum of componentdirections; that is, as the sum of motionvectors orthogonalto the ori- In certain situations, the visual systemdoes not combine entation of the components. However,vector summation motion signalsacross space, as whenplaid componentsdo is also insufficient. For example,the perceived motion not cohere. Adelsonand Movshon3proposed that coher- direction of type 2 plaids tends toward the IOC direction enceis determined by the relative spatial frequenciesof after a suitable duration (seebelow). the components. HJmand VVilson4argued that the dif- An alternative explanation of perceivedplaid motion ference in orientation or direction of motion of two grat- can be given in terms of feature tracking. In plaid dis- ings determines their coherence,with gratings having plays, the intersections of the componentgratings are closerorientations more likely to cohere. VVilliams and trackable features that provide unambiguousmotion in- Sekuler57earlier showedthat coherentmotion can be ob- formation that is the same as the IOC specifiedmotion. servedin fields of dots moving in a range of directions if Such a feature tracking explanation is, however,hard to that range is sufficiently small. Stoner et at.58 empha- distinguish from the IOC explanation, since it, by itself, sizedrelative depth cuesderived from the luminance val- doesnot explain deviations from laC-computeddirections ues where the gratings intersected. In their experi- any better than lOCo ments, if the luminancessignaled that the gratings were The importanceof feature tracking points in motion in- transparent, then they tended not to cohere. Other static tegration has been supportedby studies showingthat fea- depth cues also affect the probability of grating ture signals can be differentiated from ambiguousmotion coherence.59 signals by their different responsesto changesin stimu- Grating coherencecan thus dependon many cues. Al- lus parameters. For example, Lorenceauet at. 54mea- though depth cues are not included here, Grossberg49,60 sured discrimination performance in the perception of has modeled how depthful surfaces,including transparent surfaces,may separateoccluding and occludedforms. translating lines of different orientations, contrasts,and Baloch and Grossberg46.47and Francis and Grossberg31 lengths. Discrimination worsened at low contrast, long have modeledhow theseforms may modulate motion per- length, ~d short duration. They postulated higher contrast thresholds and longer integration times for the cepts. These form-motion interactions are not invoked here. Instead,we simulatedata whereinthe relative ori- mechanismsthat respond to feature signals. Similarly, entations of componentsdetermines perceived coherence. Yo and Wilson9 showedthat the bias in perceivedmotion direction of type 2 patterns toward the vector sum of the componentdirections was reduced over time, which they interpreted as suggestinglonger integration times for the 5. MODEL OVERVIEW: SHORT-RANGE feature motion detectors. SPATIAL PROCESSING Mingolla et at.55studied the integration of directional The model stagesare functionally motivated in this sec- information by using a display with small apertures,each tion and the next. of which contained a single moving bar. By varying bar length, they controlled whether feature information was A. Multiscale Short-Range Spatial Filter present in each aperture. Bars whose end points were Figure 4 summarizes the model's five main processing visible had correctlyperceived motions; otherwise,motion stages. Figure 5 plots how the first three stages respond was ambiguous,as in the classicaperture problem. Min- to one-dimensional inputs as a function of their speed.l1 golla et at.55found that features substantially affectedthe First, transient cells react to changes in visual inputs. perceiveddirection of motion. Theseresults suggestthat Cells sensitive to image transients occur at several stages feature information is critical in plaid perceptionand that of brain motion processing, including the y cells of the IOC constraints are not calculated from ambiguousmo- retina.61-63 The hypothesis that transient cells input to tion signals without feature motion information. the motion system is consistent with data showing that The results of Mingolla et at. 55and Castet et at.1 sug- lesions of magnocellular lateral geniculate nucleus layers gest that perceived directions combine direction signals greatly disrupt motion processing but that lesions of the normal to componentorientations and feature tracking parvocellular layers do not.64 Transient cells input to a signals. Although these results show that feature infor- spatially short-range multiscale filter that models the mation can overrule ambiguousmotion signals, in addi- short-range motion process of Braddick.12,51 Each curve tion, the perceivedmotions of type 2 plaids showthat fea- of Fig. 5 (level 2, short-range filters) plots maximum ac- ture signals do not always completely dominate tivity of one filter size as a function of input speed. The ambiguous motion signals. In addition, Lorenceauand largest scale wins at every speed because each filter re- Shiffrar56 showed that motion information is not inte- sponds as soon as it receives input. To achieve a more 2574 J. Opt. Soc.Am. A/Vol. 14,No. 10/0ctober 1997 Chey et al.

self-similar across scale. Self-similar thresholds have Directional Grouping also been used to model multiscale groupings of visual65 and Priming and speech66signals. They seem to be .a rather general principle of neural design. Figure 5 (level 2, thresholded output) shows the maximal thresholded output of each scale as a function of input speed. Now each scale wins Long-Range Filters within a different speed range that increases with scale size. These monotonically increasing curves are not, however, well enough separated to form usable tuning curves. Competition Competition (across position within each scale and across scale at each position) achieves this goal [see Fig. 5 (level 3)]. A weighted average of the outputs of these tuning curves was used in Chey et al.ll to simulate data about Short-Range Filters how speed perception, discrimination, and reaction time are affected by stimulus contrast, duration, dot density, and spatial frequency. In summary, the first few stages of the model use tran- Transient Cells sient cells that feed a multiscale short-range motion filter whose larger scales selectively process higher speeds as a result of the combined action of self-similar thresholds (at the low end of their tuning curves) and competition (at the high end). Input

Fig. 4. Model processingstages. Seethe text for details. B. From Undirectional to Directional Processing In the simulation of Fig. 5, the moving target was one dimensional. In the full two-dimensional model that is de- 0.741 veloped here, a central problem is how to convert undirectional transient cell responses into directionally sensitive responses. An early stage in this transformation uses an inhibitory veto mechanism.67-70 Barlow and Levick71 first showed that inhibition led to directionally selective ganglion cells in the rabbit retina. These inhibitory connections veto responses in nearby cells by using a logical NOToperation. Gamma-aminobutyric acid (GABA) medi- ates inhibition in directional rabbit retina cells, and gamma-aminobutyric acid antagonists eliminate directional selectivity. 72 These ganglion cells respond to single light flashes with much the same threshold as that Level 2: Short-range filters Level 2: Thresholded output of paired flashes presented in the direction that was not 1.4 0.5 vetoed by inhibition. Evidence for inhibitory processes in directional selectivity has also been found in cat cortex. Hubel and Wiese173,74suggested that directional selectivity of simple cells could be explained by summing responses at adja- 0 cent On and Off regions of the cell. Moreover, On and Off retinal ganglion cells 75converge at cortical simple cells.76 Level 3: Intra-scale competition Level 3: Inter-scale competition However, later studies rejected this hypothesis.67-7o For Maximal t example, Goodwin et at.69 studied simple cells in cat stri- Activity I ~ ate cortex that showed On and Off receptive field regions I 100 for both stationary flashed stimuli and moving edges. Speed(log scale) (receptors/unittime) The majority of the cells' directional selectivity did not correlate with the spatial arrangement of their receptive Fig. 5. Maximal response of one-dimensional model cells to a range of simulated speeds. For levels where there are multiple fields and was independent of the width of the moving bar spatial scales at each position, activities from different scales are used as a stimulus, invalidating the spatial summation shown as different curves superimposed on the same plot. The hypothesis. Like Barlow and Levick,71 they concluded smaller scales tend to respond less vigorously to fast speeds. that inhibition in the nonpreferred direction was prima- See the text for details. rily responsible for directional selectivity. selective response,larger scalesshould require more in- Both Barlow and Levick71 and Goodwin et at.69 found put in order to fire. This problemis correctedas follows. directional selectivity within small subunits of observed First, each filter is given a positive output threshold cell receptive fields. For example, Goodwin et at. re- that increaseswith filter size; that is, output threshold is ported that one cell was divided into 22 subunits, each of Cheyet at. Vol. 14,No. 10/0ctober 1997/J. Opt. Soc.Am. A 2575

which demonstrated the same directional selectivity of interactions do not, however, sufficiently boost the ampli- the cell as a whole. In fact, Goodwinet al. were unableto tude of feature tracking signals relative to that of ambigu- find nondirectionally selective subregionswithin the re- ous signals. ceptive field down to a displacementthreshold of 1 arc- What is needed is a form of competition that enhances min. In summary, early directional selectivity appears the activities of directional cells that have few directional to be basedon inhibitory veto processes,which operateat competitors at a given position, attenuates activities of di- a small scalecompared with receptivefield sizesof direc- rectional cells with many directional competitors, and tionally selectivecells in either rabbit retina or cat corti- does not disrupt speed estimates. A divisive, or shunt- cal cells. ing, competition across direction and scale accomplishes At what processingstage does such a directional veto this by computing the ratio of competing activities. 77,78 mechanismoperate? Consistent with the above data, we Competition occurs across direction and scale because suggestthat it occursas part of transient cell processing, many speeds are represented, and thus scales activated, prior to the short-range filter. This hypothesis mini- at each ambiguous point (Fig. 1). Directional competi- mizes the sensitivity of the veto mechanismto stimulus tion also increases with the directional difference. Its form. Sincethe vetoing signal is spatially offset from the net effects are twofold: Unambiguous feature tracking cell that it is vetoing, a cell can be erroneouslyactivated signals are boosted relative to ambiguous signals, and (not vetoed)if it occurs at line ends or corners when the ambiguous signals are biased toward a direction of mo- veto mechanisms lie beyond the end of the stimulus. tion that is perpendicular to a line's orientation. Such problemsare reducedif vetoing occursbefore short- range filtering, since the impact of a small number of false directional signals is reduced by subsequentspatial 6. MODEL OVERVIEW: LONG-RANGE averaging. SPATIAL PROCESSING As noted below,the veto mechanismis designedso that A. Long-Range Filter directional transient cells respond just as well at fast The long-range motion filter pools signals from multiple speedsas at slow speeds. Responsesof directional tran- orientations and contrast polarities in a prescribed direc- sient cells to a tilted line moving to the right are shownin tion of motion.33 It is the model processing stage that Fig. 6(a). Theseresponses are ambiguouswith respectto generates cells that are truly directionally selective and is the direction of motion of the line. They constrainit only proposed to occur in cortical area MT, where similar cells within 180 deg. The aperture problemis clearly visible occur.21.79-82 This processing stage also pools signals here, and there is, as yet, no representationof stimulus from both eyes, as do MT cells,22.83and explains how long- speed. range apparent motion occurs with dichoptically presented stimuli.84.85 Long-range spatial averaging further boosts the relative advantage of feature tracking signals, C. Short-RangeFilter and Feature 1racking especially at object corners (Fig. 7). It also reduces direc- Figure 6(a) shows that the very short spatial range over tional and speed biases that are due to incomplete activa- which vetoing operatesis insufficient to generatea reli- tion of the inhibitory kernels of the intrascale competi- able feature tracking signal. Vetoing eliminates the tion, while preserving speed estimates elsewhere. wrong direction, but it does not boost processingin the right direction. Such a boostis neededbecause a small set of feature tracking signals needsto overwhelmthe ef- B. Long-RangeDirectional Grouping and Atlentional fects of a much larger set of ambiguoussignals (Fig. 1). Priming The short-range filter is assumedto be spatially aniso- The grouping network enablesthe small set of stronger tropic to accumulateevidence that a feature is moving in feature tracking signals to select consistent directional a given direction and thereby boost its feature tracking and speedsignals from the large set of weakerambiguous signals. motion signals. The groupingcells interact through feed- Figures 6(b) and 6(c) show how the anisotropic short- back with the long-rangefilter cells (Fig. 1). In this way range filter and its self-similar threshold transform the the small advantageof feature tracking signals in a given directional transient cell outputs of Fig. 6(a) in response spatial region can be amplified at the grouping cells and to a tilted line moving to the right. The thresholdedout- fed back to the long-rangefilter cells,where consistentdi- puts of the horizontally oriented filters at the line endsre- rectional and speedsignals are selectedand inconsistent spond best, and they do so only to the correct direction of onessuppressed. This selectionprocess expands the re- motion at these feature tracking points. All other posi- gion of consistent signals. Another pass through the tions still experiencethe aperture problem. feedbackloop expands it further, and so on. A feedbacksystem requires more processingtime than a feedforward system. Various data show that motion D. Competition capture does take time. These include data of Castet As in the one-dimensionalmodel, competition converts et al.! on the effects of a tilted line's length on its per- cell responsesinto true tuning curves. Intrascale compe- ceived speed(Fig. 2), and data of Ferrera and Wilson; tition again occurs across space within each scale. It simulated in Section 7, about how the perceived motion doesso only in the direction of its short-rangefilter; cross- directions of type 2 plaids changetoward the feature mo- directional inhibition could severely impair speed esti- tion direction over 60-150 ms. Wilson et al.86 inter- mates. Interscale competitionagain occursat eachposi- preted this effectas evidencefor a longer integration time tion within each direction. These competitive in a pathway specializedfor processingfeature tracking 2576 J. Opt. Soc.Am. A/Vol. 14,No. 10/0ctober 1997 Cheyet al.

Fig. 6. Simulated responsesto moving tilted line. Line length in eachdirection codesthe activity level of the maximally active directionally tuned scale at that location. Ambiguousfilter activation occursin the line interior, and unambiguousresponses occur at the line ends after thresholding in (c). signals. Data of BownslO contradict this interpretation. Such cells receive excitatory input over a wide spatial ex- The present model proposes that this amount of time tent from all scales of long-range filter cells with similar naturally arises within a single processing stream as a re- directional preferences. Thus grouping cells pool over sult of long-range grouping and feedback. both space and direction. Such cooperative influences The feedback mechanism is implemented through a have been described in experiments containing elements layer of long-range directional grouping cells (Fig. 8). moving in a range of different directions.57 A winning di- Chey et at. Vol. 14,No. 10/0ctober 1997/J. Opt. Soc.Am. A 2577

(a)

"" ""~ , , , '- """"""'" \""""""'\ \""""""\\ \ """"""" """"""" """"""" """""'" "

_:l.1S -./.-

(b) Fig. 7. Simulations of long-rangefiltering: Part (a) showsmaximal activities from eachscale after long-rangefiltering. Filtering enhancesactivity at feature points relative to ambiguousmotion signals. The ambiguousmotion signals are relatively uniform along the length of the line. Part (b) showsthe sameactivities interpreted as motion vectors.

f ~-- ~ rection is then selected by mutual competition between L?ng-~ange grouping cells tuned to different directions. fJ -11 g=~~;a1 Inhibitory feedbackfrom the winning grouping cells is Cells deliveredto all long-rangefilter cells that are tuned to different directions. Through this suppression grouping cells choosea direction but not a speed. The speedchoice

+ JJ JJ 11 is implicit in the direction choice,being whateverspeed is

JJ' JJ representedby the surviving long-range filter cells. The

JJJJJJ / / i}i}i} 'i}i}/ original speedestimates of the winning direction are un- Long-Range affected by this operation. Filter Cells The grouping feedbackcan be implemented in at least two ways. One way distributes inhibition to all long- Fig. 8. Long-rangedirectional grouping and feedbackin which range filter cells that are tuned to other directions. In each grouping cell nonspecmcallyinhibits all long-rangefilters the other way, inhibitory feedbacknonspecifically inhibits while exciting cells with the samedirectional preference. all directions but is supplementedby specific excitatory

'Ii},~ 2578 J. Opt. Soc.Am. A/Vol. 14, No. 10/0ctober 1997 Cheyet ai.

feedback from grouping cells to long-range filter cells of survive the inhibition. All other signals, here all other the same direction (see Fig. 8). The specific excitation directions,are inhibited. This processof top-down prim- balances the nonspecific inhibition at that direction. The ing realizes a type of attention. Higher cognitive pro- net effect is again off-surround inhibition. Here, though, cessescan use this priming mechanism'to track objects there is no need to grow inhibitory connections selectively that are moving in attended directions. to all other directions. All inhibitory feedback is nonspe- The directional grouping circuit is proposedto occur in cific, and all excitatory feedback is specific and reciprocal. the ventral part of MST, which has large directionally This network realizes a matching rule that is familiar in tuned receptive fields that are specialized for detecting adaptive resonance theory (ART).3S.S7 By the ART moving objects.88 In this interpretation, MSTvcan atten- matching rule, top-down matching signals (here from tionally modulate MT, which is proposedto include long- grouping to filter cells) can prime a given direction while range filter cells. Consistent with this proposal,Treue inhibiting all other directions. During matching of and Maunse1l89have shown that attention can modulate bottom-up and top-down signals, only bottom-up signals motion processingin cortical areas MT and MST in be- that are confirmed by an excitatory top-down prime can having macaquemonkeys. O'Cravenet ai.9ohave shown

Fig. 9. Maximal activities acrossall scalesof the long-rangefilters during the rightward motion of a 45-degline at three successive times. Ambiguousmotion signals are eliminated by feedbackinhibition from the long-rangedirectional grouping cells.

~ Chey et at. Vol. 14,No. 10/0ctober 1997/J. Opt. Soc.Am. A 2579

by using magnetic resonance imaging that attention can modulate the MT/MST complex in humans. This interpretation predicts that MST u cells make a directional choice whose feedback to MT overcomes aperture ambigu- ities and selects an object's true direction of motion. c: ',=0 ~(J 7. DATA SIMULAnONS a The modelis used to simulate increasinglycomplex properties of psychophysicaldata, namely, how feature tracking and ambiguous signals are combined,how feature tracking signals interact with each other, and how feature tracking and ambiguoussignals may not combine.

A. Line Motion Time Castet et al.l had observerscompare the speedof tilted lines with that of a vertical comparisonline undergoing horizontal movement. They varied both the orientation and the length of the lines. Their data showtwo major effects[see Fig. 2(a)]. First, the perceivedspeeds of tilted lines are slower, and the degreeof speedbias increases with line tilt from vertical. Second,the magnitude of this bias increases with line length. Our simulations show how unambiguousfeature tracking signals and ambiguous motion signals from line interiors are combined. In eachsimulation a line length and orientationwere chosen,and the motion of the line with those characteristics was simulated. Speeddata were collectedat a fixed time after the simulated motion starts. A ratio was computed between the spatially averaged speed signal obtained from the simulation and that obtained from a simulation Time with a vertical line of the same length and speed. These Fig. 10. Motion direction and speed of a moving line derived from long-range filter activity over time (the same filter activities ratios measurethe perceivedrelative speedof eachline to are shown in Fig. 9). The top plot shows how the perceived di- the vertically oriented line. Figure 2(b)shows these ra- rection of a moving line gradually converges to its actual direc- tios for three orientations and three line lengths. tion of motion (0 deg from horizontal) after starting at a direction In Castet et al.,l each line was displayed for only a almost perpendicular to the line's orientation (45 deg). The bot- short duration (167 ms). The length of this presentation tom plot shows how the perceived speed of the line gradually asymptotes over time. is compatible with the hypothesis that feature tracking information has not yet fully propagatedalong the line and that speedbiases are due to residual ambiguousmo- of the aperture to determine line direction. See tion signals present along the line length. This hypoth- Hildreth91and Nakayamaand Silverman16for supportive esis suggeststhat the biases are due to incompletepro- data. cessingand are thereforetransient. Under certain conditions the barberpoledisplay can si- Figure 9 shows the evolution over time of long-range multaneouslysupport at least two distinct perceivedmo- filter cell activities during motion of a line tilted at 45 deg tion directions: motion coincident with the long axis of from vertical. As the line moves,grouping cells become the aperture and motion orthogonalto the line. For ex- active and propagatethe feature tracking signals along ample,in a horizontal aperture with a diagonallyoriented the line. Although the interior line signals originally in- moving line, the line appearsto move diagonally in the dicate lower speedsperpendicular to the line's orienta- comer regionsand horizontally in the central region [Fig. tion, over time, the perceiveddirection and speedat each l(b)]. This perceptis most prevalent when one line is in point becomeconsistent along the wholeline length. Fig- the aperture at any time. Multiple moving lines tend to ure 10 plots how perceived direction and speedchange lock the perceptinto motion along the length of the aper- gradually over time. ture. There are at least two possibleexplanations for the per- B. Barberpole Motion ceiveddiagonal motion in the comerof the barberpoledis- When a moving oriented line is viewed through a circular play: The diagonal motion could be an average of the aperture,its motionis ambiguous. However,when a line horizontal and vertical feature signals at eachend of the is viewed through a rectangular aperture,the classicbar- line. Alternatively, the competingfeature signals could berpole illusion is produced,whereby the line is perceived cancel each other out, leaving ambiguousmotion signals to move with the direction of the aperture's long axis.2 in the direction normal to line orientation. This suggeststhat the visual systemutilizes the feature One way to distinguish betweenthese explanations is tracking signals derived from the line endingsat the edge to alter the orientation of the line. Accordingto the first 2580 J. Opt. Soc.Am. A/Vol. 14,No. 10/0ctober 1997 Chey et at.

explanation,this should result in at mosta small shift to- tions from the different aperturescould indicate different ward the direction normal to the new orientation. Ac- directionsof motion, although the diamond moved rigidly cording to the secondexplanation, sucha changein orien- as a whole. Observershad difficulty perceivingrigid dia- tation should also have little effect until it becomeslarge mond motion, suggestingthat feature information could enoughto favor one of the feature directions. Then the not be integrated acrossthe apertures. Integration be- motion may be entirely captured by that direction. Infor- came more likely after stimulus manipulations reduced mal observationssuggest that the secondexplanation is the influence of terminator motions, such as adding correct: The perceptis usually in the dir~ction of the ap- jagged aperture edgesor using low-contrast terminators erture edgewhose orientation is most nearly perpendicu- relative to contour contrast. lar to the line. This result suggeststhat diagonal motion Interference betweenmultiple feature tracking signals in the cornerof barberpole displays results from inconclu- is attributed in the modelto grouping cell kernels that are sive competitionbetween different feature motion signals, sufficiently large to overlap severalfeature tracking sig- in the sensethat no feature gains dominanceand propa- nals. When small grouping cell kernels are used, there gates to acrossthe moving line. will be somelocations near the line end that cohere with The difficulty of integrating feature signals in different just the nearestfeature tracking signal. directions of motion was also demonstratedby Lorenceau Figure 11 illustrates time slicesof motion signals from and Shiffrar ,56who studied the integration of motion in- a simulation of a moving line behind a rectangular aper- formation between multiple apertures that revealed a ture. Figure 12 showshow a small (5-deg)change in line portion of a translating diamond. The terminator mo- orientation causesmore rapid convergenceto the feature

- - -

\ ... '" ... "

~=2.3' -z2.0000

Fig. 11. Time slices from the movementof a line behind a rectangularaperture illustrating the two phasesof motion in the barberpole illusion. Eachgraphic showslong-range filter cell activity interpreted as motionvectors. In the first time slice,the line movesthrough the corner region,and signals in the line interior indicate motionperpendicular to the line's orientation. Feature signals dominatethe output near the aperture edgesbut do not dominatethe ambiguousmotion signals along the rest of the line. In the later time slice, motion is captured by feature motion signals in the direction coincidentwith the long axis of the aperture.

~ Chey et at. Vol. 14,No. 10/0ctober 1997/J. Opt. Soc.Am. A 2581

t: 0 ",0 () ~ 0

Time Fig. 12. Perceiveddirection of lines moving behind a rectangularaperture (barberpoleillusion) overtime. Results are shownfrom two independentsimulations utilizing two different line orientations. Whenthe orientation of the line is tilted away from 45 deg,the perceived direction of the line convergesmore rapidly to the direction coincidentwith the long axis of the aperture.

direction closestto the vector perpendicularto the line's herent motion could be observed at all. Lindsey and orientation. The larger the bias provided by the line ori- Todd also found that increasing orientational differences entation, the more rapidly this effect is felt, until the ex- results in an increased probability of observing incoher- treme case is reached,in which the line moves perven- ent motion after adaptation. dicularly to an aperture edge. The resilience of coherent motion percepts is not sur- Another way for one feature tracking direction to win prising when one considersthat a plaid display contains in a comet region is to prime the grouping cells to a par- only consistent feature tracking motion signals at the ticular motion direction and to influence the long-range componentintersections. The modelassumes that inco- filter cells through feedback. This situation can occur herent motionresults from failure of feature tracking sig- when multiple lines movebehind the aperture. Initially, nals to dominatethe grouping cell competition,freeing in- the motion of the line is ambiguous,but over time a win- dividual componentmotions to expressdirections normal ning direction emerges at the grouping cell level. If a to their orientations. Unlike the barberpole display, in- secondline enters the aperture before residual grouping completecompetition cannot arise, since there are no con- cell activity decays,the previous winning direction can flicting feature tracking signals. The only competition continue to win the competition. This priming effectsug- comesfrom the ambiguoussignals of the individual com- gests why comer diagonal movementsare not observed ponents. when multiple lines occurin a barberpoledisplay. If this is true, then why are two-dimensionalplaid percepts ever observedto moveincoherently? How can plaid C. Plaid Motion displays exhibit incoherentmotion when simple lines of a Kim and Wilson4 argued that relative line orientation is single orientation are always coherent with feature sig- the prime factor in determining plaid coherence. There nals derived from their end points? Two elementsof the is, however, still much dispute regarding the importance model help to explain this. First, the feature tracking of other stimulus parameters and the probabilities of ob- signals derived from plaid displays differ from those observing coherent motion.5,58,59,92Part of this dispute is tained from moving lines with visible end points. The in- due to the many stimulus configurations employed in tersection points of plaid components,although forming plaid motion designs. trackable features, may be less salient than line ends. Kim and Wilson4 reported that coherent motion is al- For plaids formed from sine-wavegratings, the intersec- most always observed for component orientations within tion points form amorphous blobs that have no clear 45 deg of the plaid direction and almost never for larger edgesor other feature points to track. This may explain orientational differences. These data were collected by why Kim and Wilson4and others who use sine-wavegrat- using sine-wave gratings of different spatial frequencies. ings so often observe incoherent motion. Square-wave Lindsey and Todd5 reported that square-wave gratings plaids have more sharply defined intersection points. moved coherently at all relative orientations and that it When the luminance values at the intersections are was necessary to undergo prolonged viewing before inco- added,the intersections form moving diamonds that are 2582 J. Opt. Soc.Am. AlVol. 14,No. 10/0ctober 1997 Chey et at. easily tracked. Luminancediscontinuities are not neces- The simulations consider square-wave plaids with uni- sary to observecoherent motion, as when the luminance form luminance values. At each intersection of such a of the moving diamond equals that of the components. display [Fig. 13(a)], there are four feature points, one at each comer of the diamond, that provide trackable features. To track both leading and trailing edges at an intersection point, the model uses both On and Off cells. On cell transient responses are generated along the leading edge of both contours [Fig. 13(b)], Off responses along the trailing edges [Fig. 13(c)]. Grossberg and Rudd32.33proposed that the short-range filters process On and Off transient responses independently and that these channels are combined at the long- range filter stage. Until now we have considered only a single set of transient responses without specifying whether they are On, Off, or a combination of both. Both channels are now used and are processed independently until combined at the long-range filter. The resulting feature signals are smaller than those derived from line ends, because two orientations join at a comer. At such a comer, both long-range and short-range filters sum over both sides of the contour, resulting in a less specific feature signal. This helps to explain why plaid patterns are more likely to move incoherently than lines. Baloch et al.93 have shown how these On and Off mechanisms simulate first-order and second-order motion percepts. The second major factor controlling plaid incoherence, particularly for the square-wave gratings studied by Lindsey and Todd,5 is that the viewer is exposed to the motion for a long time. Plaid pattern motion is highly re- petitive and can thus fatigue motion detectors. Likewise, after prolonged viewing of barberpole displays, percepts can fluctuate from one feature direction to the other, also presumably as a result of adaptation to the prevailing di- ::::::;:::::::::::::::::::::::::::::::::::::::. rection. ;."..;. 1. CoherentPlaid Motion Ferrera and Wilson94classified plaids into three groups: Type 1 plaid direction lies in the arc between the direc- (b) tions normal to the components. Type 2 plaid direction lies outside this arc (Fig. 3). Type 1 patterns are called symmetric or asymmetric, depending on whether the anglesformed betweenthe componentsand the direction of motion are the same for both components(symmetric) or not (asymmetric). The simplest pattern, type 1 symmetric, is simulated first. Figure 13 shows the stimulus configuration and the On and Off transient cell responsesalong its leading and trailing edges. Figure 14 shows long-range filter outputs generated from the combinedOn and Off channels. Figure 15(a) shows coherent plaid motion. The feature motion signals derived from the corner points have propagatedacross the whole plaid pattern, captur- ing motioninto a commonhorizontal direction that correspondsto the motion direction of the plaid as a whole. _.~ 2. IncoherentPlaid Motion Coherent plaid motion occurs when a single direction (c) wins at the grouping cells. Incoherent motion occurs when no suchwinner is established. Under the assump- Fig. 13. (a) Input luminance, (b) On transient cell, and (c) Off transient cell responses for a type 1 symmetric plaid whose com- tion that adaptationto the perceiveddirection during co- ponents are oriented at 45 deg from vertical during rightward herent motion contributes to incoherent motion percepts, movement. incoherentplaid motioncan be simulated by reducing fea- Cheyet at. Vol. 14,No. 10/0ctober 1997/J. Opt. Soc.Am. A 2583

~k~~~~~"~~~kW~~~~~"_L.'-L-"""k-""""""---"-"""~"""",,,,~, " :.'" ~~~~~~~~r~-~~~~~~~~r--. ..~.~..~~.~~~..~""U. ~~. ...~..., "F~F~~~~~~~"F~~~-~' ...~~~~~~~~~~.~~~ij.~~~~~..~ ~.~~~~...~..~-**- ""~~T#~..~~F~~~rr~~". . ~~~~~~~~~~~~~~~~~~'.~r'rr~~~~~~w.rrr~~~r.." ".;

~~~~~~~~.~~~~~~~~~~ F~~*~~~~~~~~~*~~~.~"'..

~~,,~~~~~.~~~,~~~~~::~~~t'~~~r~rrr~~*r~.".. ..

~~~~~,*,~~~.~~~~~~~~~:~~~~~~~r~rr~~r~..'.'..'., ~~¥..~.&~~~~~~~~~..ou ~~ ~~~~~~ r~~~,#'r~." ". ...o.~.-~~~"¥,,.. ~.. .. ~F~r~~~~~~rr~.".".". \...~~..~-oc,,~... ~.,..,..,..,.~.,.r".' I ~~. ." .. .. a * ~ " ~ .."-'""",-. "".#f""~~""""'.'."..'....,,".,.r*rr~."..'.'."..

.." '..~.. ~~~",,~ ~< #-~~---~~~~~~"'~*--~~'E'E'E~~~'-'- ~**~**~ y~ --"'I""~~*"~~~~~'.--.\o.¥~~~'E'E~~.. ~~~******~ ,.,. "I'C""~"""." ~.

~"'..r~~*~~~~~'~~~~~_.~~~~~~~~~~~",*,~"...'.'.~ *"~"r,.~~~~~. ~..r~*~#.~'~~~~~~~~~~~~~~~~~'*"'~~.'"~ ~*" ",.r~f"#"~.',..rr*#"# frli'fl"f'fO.."..~~~~... p.' ~ ,.~~~~~~~~~,,~~~.~~~~'*'~~ """""~~"'" '""" ~~""~~L. ..,~~~"~~~~'*'" ~-""'~"..'..' ..

...rrrr~~~~~~~~~~~r*~#"'~~\.~~~*'~~~~~~~~~*'~~~~..".''r~~r*~**~~~~r~~~~~~-~~~~"~~~~~"~~~~~"..".."rr~~~'~~~'~'r~~*tr~..\..~~~'*'~~~~~.~~"~~~".~'... I.

rrrrr~"tf"""',""rr~",.f'."..' ~..~~k*,i-"~~~~"~""""'~.~", '~~~FF~~~~~~~FFFF~~~~"'.' ,~~~~.,~~.,~~~~~~~~~~~~V-~

~..-r-r-~"""' ~ r-r-~",,"'...r' ' "~"""""""~~o-~~-"""""""""~---I

_.1.9Z -.3.0000

Fig. 14. Long-rangefilter responsescombining On and Off channelsduring simulated motion of a type 1 symmetricplaid. Components are aligned at 45 deg from vertical. Filter responseshave beenmagnified to showambiguous signals clearly. The large feature signals do not thereforeappear at their true magnitude. Suchfeature signals are presentat the four comers,where the componentsintersect. ture signal strengths to a level where they can no longer tation is required to produce incoherent motion for win the grouping cell competition. smaller differences between the component orientations Several possible mechanisms could becomefatigued (see Fig. 16). during coherentplaid motion. We assumethat this ad- Type 1 asymmetric plaids exhibit much the same be- aptation occurs at only a single processingstage, the syn- havior with respect to coherence and incoherence as do apses connecting the long-range filters to the grouping type 1 symmetric plaids. The critical variables are again cells. A/homologoustype of habituation has elsewhere the relative difference between the component orienta- beenused to explain data aboutvisual persistence,form- tions and the actual direction of motion. The model pre- motion interactions, and aftereffects.31,48,95,96In the dicts that the relative orientation difference between the present application, after adaptation, eachgrouping cell components is less important than the relative differences receivesa smaller input signal from the long-rangefilter between the components and the feature tracking direc- cells that have been active. During coherent motion tion. Asymmetric plaid patterns could be used to test grouping cells suppressthe activity of all long-rangefil- this hypothesis, since it is possible to vary these param- ters but those whose directional preferencesmatch the eters independently. Such manipulations have not been chosen direction of movement (Fig. 8). Therefore only reported in the literature. these filter cells will becomeadapted, while all other filters will remain in an unadapted state. 3. Type 2 Plaid Motion Figure 15(b)simulates long-range filter activity with In type 1 plaids, coherence is aided by the fact that the the same type 1 symmetric plaid, but with adaptedhori- plaid motion direction lies between the two component di- zontal filter cell responses. Now there is no winner at rections. In type 2 plaids, this is not the case, and this the grouping cell level, so ambiguouslong-range filter re- difference presumably underlies the fact that biases have sponsesremain intact alongthe components. Thesecom- been reported in the perceived direction of type 2 plaids, ponent responsesare hypothesizedto correspondto inco- but not type 1.7 A key feature of these biases is that herent motion. their magnitude is duration dependent [Fig. 17(a)]. Wil- As discussedabove, it is difficult to modelqualitatively son et aI.56 modeled this bias as the result of a delay in a data regarding plaid motion becauseof the variety of non-Fourier motion pathway, so that initial percepts are stimulus configurationsand viewing conditionsused and based only on the responses of Fourier motion pathways, the variability in results obtained under different experi- which respond to the component orientations only. An mental regimes. However, other things being equal, explicit delay was required because model output is com- simulations should show that incoherent motion is corre- puted simultaneously with motion integration. Such a lated with the difference between componentorienta- postulate of differential response time is unnecessary in tions. Simulations were run in which plaid patterns the present model. Instead, the change in bias derives were tested with different levels of long-range filter cell from the integration time needed for the grouping cells to adaptation. These simulations show that greater adap- become active and influence the long-range filter cells. 2584 J. Opt. Soc.Am. A/Vol. 14, No. 10/0ctober 1997 Chey et at.

The critical model property is that long-rangefilter cells ased toward the component motion signals. Figure 17(b) becomeactive before the feedoackgrouping mechanism plots the time course of the present model's perceived mo- selectsa winning direction. During this time, modelre- tion direction in response to a simulated type 2 plaid. sponseis based on feedforward motion signals and is bi- Such directional biases have not been reported for type

[- r.- '--'--',~"~ ' ,, ~ ' ~ ,

(a)

, , ; -- -~/~~~ ~/ , , , , , , , , , -, , , , , , , , , , , , ; ; ; ~///~~~ /, , , , , , , , , , , , , , , , , , , , , ,"""',/...//"", I .- " " " , , , , , , , , , , , , , , , , , , , , , .- " " " '/""",//.../",,/ -""""""""", , " " " '//""'///;;;/;,~ ~ , , , , , , , , , , , , , , , , , , , , " " '//;;/",~,;~~~~~~ -, , , , , , , , , , , , , , , , , , " ."""",//~~~~--- -~""""""'" ."/",/,,,,/~~ -~~"""""'" , , / , , , , , , , , , ~ ~ -~"""""" """""" """""" "//;/,/,,, """"'" /~~~~~~//, """"~, """"'"""""" ,~""~~~,~~~~~~~ ~ ~~~-~~ ,~~~~~"",/".../,"",,'/'/'// ~

"""""" ""'/,/"" , , , , , , , , , , , , ~ ~ ~ / , , , , , , , , , , , , , , , , , , , , , , , , ~ ~ ~ , , , , , , , , , , , / , """" ""'-~-~--,~ ;/"" /'" // , , ."""""""" ~ ~ ~" / /, / /'" /'" / / / , , , ."""""""" ,~/'" ,/'" /," "///"."""""""" , , , , "/,,, """,' "/,///,."""""""" , , , , , """"/""""//" .""""""", '/"'/'//...~...,/"...///~ ."",].5000 ", ~// ~ ; ~~ rime= 2..14-

(b) Fig. 15. (a) Long-rangefilter outputs interpreted as motion vectorsduring the coherentplaid motion with componentsoriented at 45 deg from vertical. Motion signals are almostentirely uniform acrossthe pattern, indicating rigid motion. (b) Long-rangefilter outputs interpreted as motion vectors for a plaid pattern during incoherentmotion. The two componentsmove independently. The feature points still move horizontally,a commonperceptual phenomenon when viewing incoherentlymoving plaid patterns. Chey et at. Vol. 14,No. 10/0ctober 1997/J. Opt. Soc.Am. A 2585

~ ~> c 0 O;; ~ Q. ~ "0 oct:

ComponentAngles

.. Direction of Motion

I Component I Angle I (from vertical) Fig. 16. Adaptationlevels at which simulatedtype 1 symmetric plaids are perceivedto move incoherentlyfor a range of component angles from horizontal for a horizontally moving plaid. Each adaptation level is the ratio of the adapted to the non- adaptedlong-range filter outputs. Only the rightward selective filters are adapted. When the componentangles are near horizontal, greater adaptationis required for incoherentmotion to be produced. This result is consistentwith the smaller probability of seeing!;oherentmotion with these components.

1 plaid patterns. They would presumablybe small be- Duration (msec) causeof the similarity betweenthe vectoraverage of component directions and the plaid direction. The model (b) --- ,!I/) 45 suggests,however, that there may be an additional rea- v son why biases are not reported for type 1 asymmetric ~ 40 plaids: When grouping cells selecta winning direction, 35 grouping cell activity eliminates competing directional 30 signals, leading to a gradual changein overall perceived 25 direction toward the winning grouping cell's preferred di- 20 rection. For type 2 plaids, activity along both components convergesto the actual plaid direction from the 15 same side. For type 1 plaids, each componentconverges 10 '" to the plaid direction from a different side. Although it was not modeled,it is reasonableto assumethat compo- ""--~ nent motions lead to a perceivedcoherent motion only if 'i 1:522:533:5' they are sufficiently similar. Thus it may be that biases Duration are reported in type 2 motion for situations in which type Fig. 17. The perceiveddirection of type 2 patterns is duration dependent. At small exposuredurations, patterns are perceived 1 plaids move incoherently. to move in the vector sum direction. For longer durations the Informal observationssuggest that type 2 plaids tend lac directiondominates. Data in (a) are replotted from Yo and to move incoherently. Even when they move coherently, Wilson,9obtained by using componentswhose normals are at the blobs at the intersection point of the componentsare 48.2 and 70.5deg from the direction of motion (vector sum 55.5 often segmentedfrom the componentsthemselves. How- deg). Simulated results are shown in (b), obtained by using componentsoriented at 45 and 67.5 deg (vector sum 56.25deg). ever, Kim and Wilson4reported that for a wide range of The simulations use slightly different orientations to suit the componentdirections, ranging from 18.4to 71.6deg away Cartesian grid underlying the simulated input. In both cases from the plaid direction, tYPe 2 patterns moved coher- componentspeeds are adjusted so that the lac prescribeddirec- ently when different spatial frequencycomponents were tion of motion of the plaid is at 0 deg.

'" 2586 J. Opt. Soc.Am. A/Vol. 14,No. 10/0ctober 1997 Chey et al. (a) temporarybias occursto one component'sdirection of motion. Thesebiases can explainthe data reportedin Stone et al.6 because,as in the Ferrera-Wilson7 study, their data were gathered with short-duration viewing. Each plaid was viewed for 300ms but was only at full contrast 'to for 200 ms. The authors did not report results using ~ longer viewing periods, possibly becausethe biases be- ~t/J came less evident or nonexistent after prolonged expo- ~ sure. The data of Stone et al.6 were simulated by changing the input magnitudesused for eachcomponent grating in plaid input patterns, as was doneby Cheyet al.ll to simulate stimulus contrast changes. These input magnitudes were varied in the same way as they were in the experi- 0 1 2 3 4 ments of Stone et al.6; namely, by choosinga base input contrast/magnitudeand then varying the componentcon- trast ratios while keeping the total input contrast/ magnitude constant. Figure 18 shows that the simulations provide a goodqualitative fit to these data. These results used 1.5 simulation time steps,a slightly longer duration than was used in the type 2 simulations. This also reflects the data: Ferrera and Wilson7 employed , bO shorter durations than did Stone et al.6 G) '0 '-''" tU 8. DISCUSSION i:Q The speed-sensitive motion boundary contour system that is further developed herein has, by now, been shown to simulate a wide range of behavioral and neural data about motion perception. The earliest version of the model was used to simulate data about short-range and long-range apparent motion, including data about beta, Log2 contrastratio gamma, delta, reverse, split, Ternus, and reverse-contrast Fig. 18. The perceived direction of type 1 symmetric plaid pat- Ternus motion and Korte's laws.31-33 It also suggested terns is dependent on component contrasts. (a) Data from Stone how long-range apparent motion mechanisms could be et al.6 show bias in perceived direction as a function of the con- used to generate continuous attentive tracking signals in trast ratio between the two components for four different base response to moving targets that are intermittently oc- contrasts. (b) Simulation results show the same qualitative biases for two different base contrasts. In both casescomponents cluded by intervening objects.48,97 were oriented at 67.5 deg from vertical. The model was then adapted in Nogueira et ai.98 to use only transient cell inputs to explain additional data about second-order motion, including the perceived reversal of suIts are extendedherein to include the effects of group- motion direction with distance from the stimulus, and ing on contrast-sensitive plaid motion. As discussed Why monkeys with lesions of the parvocellular but not above,Stone et al.6 usedthe fact that perceivedspeed in- magnocellular layers of the lateral geniculate nucleus are creaseswith contrast to provide an IOC-basedexplana- capable of detecting the correct direction of second-order tion for the reported biasesin plaid motion with unequal motion.99 Baloch and Grossberg47extend these results. contrast components. The modelprovides an alternative These studies did not, however, fully exploit the multiple accountof their data, as follows. Increasesin stimulus spatial scales of the model. Chey et ai.ll showed how contrast increase the energy or activation levels of the multiscale filtering and competition could be used to short-rangefilters. Velocityestimates are determinedby simulate further data concerning how visual speed per- the energyproduced by the short-rangefilters. If the fil- ception and discrimination are affected by stimulus con- ters that detectthe motion of the higher-contrastgrating trast, duration, dot density, and spatial frequency. are more highly activated, then their velocity estimates The present refinement of the model builds upon these will dominate the directional percept,leading to the ob- results and those of Grossberg and MingollalOO to show servedbiases. how the global aperture problem may be solved; namely, As was demonstratedfor type 2 patterns above,direc- how long-range filtering and grouping mechanisms can tional perceptsin the modelconverge to the feature track- transform the outputs of the speed-sensitive multiscale ing direction overtime. When symmetric plaid patterns filter into a coherent representation of object speed and are employed,such convergence is not noticeablebecause direction. The model shows how the motion capture pro- the energycontributed by eachcomponent is balancedby cess that carries out this transformation can also act like the other. However,when one componenthas a higher a top-down attentive mechanism for priming motion di- contrast than the other, this is no longer the case,and a rections without disrupting speed estimates. The model Chey et al. Vol. 14,No. 10/0ctober 1997/J. Opt. Soc.Am. A 2587

hereby clarifies how low-level automatic motion detectors fying assumptionsfor the sake of computationaltractabil- and attention-modulated motion grouping processes may ity, there are severalmore probableexplanations for this work together to generate accurate velocity judgments.37 discrepancy, including that Stone et al.6 used slower This capture-and-priming circuit is familiar in Adap- speed stimuli (2 deg/s) than did Yo and Wilson tive Resonance Theory ART, in which it has been used to (8/9.6 deg/s)and lower luminances (100 cd/m2)than did simulate data about early vision, visual object recogni- Castet et al. (131 cd/m2). Both effects could be compen- tion, auditory streaming, and speech percep- sated by a somewhat longer presentation time. A sys- tion.38,87,101-103This connection with ART indicates how tematic parametric study that included all the stimuli of key model parameters may self-organize during develop- the three experimentswould be most helpful. For defi- ment. More recently, the model has been incorporated niteness,a simulation time of 3 units was set equal to 160 into explanations of the line motion illusion and motion ms in Table 1, but this conversionfactor must remain un- induction experiments.46,47 In all these explanations, certain until more systematic parametric data become similar mechanisms of transient cell detection, short- available. range filtering, competition, and long-range filtering have The perceived motion of coherentlymoving plaid pat- been used. Thus the total explanatory power of the pro- terns has a natural explanationusing the motion of fea- posed mechanisms, and the experimental support for ture tracking signals. Incoherent plaid motion was at- them, goes far beyond the data simulated herein. tributed to the fact that feature signals are too weak to To generate a globally consistent representation of ob- influence nearby ambiguoussignals. Most reports of in- ject motion, the motion boundary contour system model coherentplaid motion use sine-wavegratings (e.g.,Adel- takes account of both unambiguous feature tracking sig- son and Movshon3and Kim and Wilson4),whose feature nals and ambiguous signals that are due to the aperture blobs at the intersectionpoints result in less powerful fea- problem. The model differs in several ways from others ture signals than do the clearly defined diamondslocated that have postulated that both feature and nonfeature at the intersectionpoints of square-wavegratings, which motion signals are involved in motion perception.1,4,86 Lindsey and Todd5have shownto move coherently until First, the model does not process the two types of signals after a substantial period of adaptation. by using different mechanisms. Instead, each signal Long-rangegrouping in the motion boundary contour type is processed by the same mechanisms operating at systemmodel operates by using large isotropic input ker- different image locations. Observations that feature nels. This assumptionis a simplification. Processingof tracking signal processing has different characteristics more complex image scenes requires consideration of from those of ambiguous motion processing9,56are ex- more complexinput kernels, including inputs from seg- plained as a result of the long-range process rather than mentations performed by the static form system. Motion of a separate processing channel. integration is affected by the form relationships between Feature tracking signals are amplified in the model by locations where feature tracking and ambiguous motion anisotropic filtering of transient motion signals and com- signals are generated. For example, Shiffrar et al.l07 petitive/processes. Supportive evidence for such mecha- have shown that the perceivedmotion of barberpole dis- nisms has been described for barberpole-type motion by plays was determined primarily by the motion of the line Power and Moulden,104who showed that aperture widths terminators, even in the presenceof superimposedmov- influenced perceived motion of translating lines. Addi- ing dots whose movementis also unambiguous. Assum- tional evidence for the importance of anisotropic filtering ing that the motion signals generatedby the moving dots comes from studies indicating that multiple dot flashes and line terminators are of similar strength, this suggests contribute to apparent motion percepts105and that the lu- that the terminators exert a greater influence on the per- minance of, and distance between, dot flashes influence ceived direction of motion of the lines becausethey are perceived apparent motion between those flashes.1OG connectedto the line, whereas the dots are spatially iso- The model explains various data by assuming that fea- lated from the lines. Similarly, Nakayama and ture tracking motion signals dominate motion percepts Silverman16showed that terminators attached to a mov- and that deviations from feature tracking directions1,9are ing curve were more dominant in determining perceived the result of incomplete feature signal integration. rigidity and that the influence of disconnectedtermina- These explanations are compatible with the short dura- tors was dependent on their distance from the curve. tions used in experimental paradigms that show per- Baloch and Grossberg46.47and Francis and Grossberg31 ceived directions of motion deviating from the feature mo- have modeled how such form-motion interactions may tion directions. The model is also able to account for data occur. showing that integration processes are affected by non- speed parameters, such as stimulus contrast.6 The relative durations used in the various simulations Table 1. Experimental Durations and Simulation are roughly in accord with the relative times utilized in Durations for Three Sets of Experiments the modeled experiments, as seen in Table 1. The experiment of Stone et al.6 is somewhat discordant with the Stimulus Simulation Simulation other data. To reproduce similar magnitude biases, the Study Duration (ms) Time Steps Duration (ms) simulations were run for only 1.5 time units, although Castet et al.l 167 3 160 this experiment utilized longer exposure durations than Yo and Wilson9 40-160 3 160 did the experiments of Castet et al.1 or Yo and Wilson.9 Stone et al.6 200 1.5 80 Despite the fact that the model made a number of simpli- 2588 J. Opt. Soc.Am. A/Vol. 14,No. 10/0ctober 1997 Chey et at.

APPENDIX A: NETWORK EQUATIONS AND PARAMETERS levelS: DirectionalGrouping l @)H E> H_@ " and Priming The network is defined by differential equations that specify the time-varying activity or membrane potential ' of individual neurons or populations of neurons. These 1<:D4k~10.t.(~.t.0 Level 4: Long-RangeFilters cells are typically controlled by a membraneor shunting @ .t. l,oJ.t. E> 1 equation,lOB-llOwhich performs leaky integration of its inputs:

dx Interdirectional Competition ""cfi = a[ -x + (/3 -yx)E -(8 + EX)!] (Al)

In Eq. (A1) the activity x of a cell is driven by excitatory v input E and inhibitory input I. The integration rate of l

~-:> Chey et at. Vol. 14,No. la/October 1997/J. Opt. Soc.Am. A 2589

4. Level 3: Competition Network axy, (A4) Intrascale competition. The activity of an intrascale competitive cell hts is determined by a feedforward center- where set B ij consists of ten adjacent change-sensitive re- surround competitionwith nearby short-rangefilters: ceptor positions. Sets B ij are defined so that each transient cell draws from a nonoverlapping set of receptors. L gfj L g1j d Transient cell activity is shunted: Its response rate in- (I, J)eEij (I, J)Erdv creases with total receptor activity in set Bij, but its am- ~=101 plitude is bounded by 1. dt Directional interneurons. Directional interneuron ac- (A9) tivity c~ time-averages transient cell output (Fig. 19). SetEt definesthe spatial locations that form the excita- Each interneuron helps to create directional transient tory kernel of the cell. It includesfive thresholdedshort- signals and so is assigned a direction d: range filters offset from position(i , j) by -2 to 2 in direction d. Set It defines the inhibitory kernel, which includes six filters offset by distancesranging from -5 to (A5) -3 and 3 to 5 in direction d. Interscalecompetition. The interscale competitionac- Each interneuron receives excitatory input bij from a tivity kts refines speedtuning by using a shunting equa- transient cell and inhibitory input cfJ from an adjacent tion. Excitatory input at scales is given by the rectified inhibitory interneuron. The coordinates (J, J) are offset activity [htS]+ of an intrascale competitive cell of the by 1 unit from (i,j) in direction d. Mutual competition same direction and scale raised to a power. Inhibitory between interneurons prevents a directional interneuron input sums the rectified activities of all other scales of opposite direction from inhibiting directional transient raised to the samepower: cells of the correct direction at high speeds. The inhibitory input to an interneuron is therefore given by cfJ, dkt!.s -if- = -kts + (1 -ktS)([htt]+)3 where D is the opposite direction from d, given by (d + 8 )mod 16. Inhibition is set to be stronger than excitation in Eq. (A5). L ([h~t]+)3 Directional transient cells. Directional transient cell d. t... activities et with direction d receive excitatory input bij -(1 + kij) II{t * s}II' (AI0) from transient cells, which is vetoed by directional interneuron activity cfJ : The power function sharpensthe tuning of eachcell. lnterdirectional competition. The interdirectional competitionactivity l~. occurs at each position across di- (A6) rection. Excitatory input comesfrom the like-directional interscale competitive cell, and inhibitory input comes from all other directions across all scales weighted by 3. Level 2: Short-RangeFilters their mutual distance in directional space;namely, Short-range spatial filter activity fif performsspace and time averaging of directional transient cell responses. d[d..OJ = 10( -l~. + 10[k~.]+ Each filter has a scales, which determinesreceptive field - size, and a direction d, which determinesreceptive field dt orientation and the directionally selectivetransient cells that activate it:

ad The shunting of inhibitory input to these cells ensures :L eIJ' ] (A7) (1. J) e~d that relative activities acrossscales are maintained}7 IJ Set Fijd includes 28 + 1 receptors aligned in the orienta- 5. Level 4: Long-RangeFilter tion coincident with direction d and centered on position Long-range filter activity mts space-and time-averages (i, j). Each short-range filter output is thresholded by interdirectional competitive cell outputs within an elon- an amount that increases linearly with filter size before gated, directional receptivefield. It also receivesinhibi- being spatially blurred. The output is given by tory feedbackfrom long-rangedirectional grouping cells that chooseamong the competingfilter directions: gts = L {exp[ -(i -1)2 -(j -J)2]}[f1j]3s/2 d (I, J)eGii L [Z:;]+ (A8) dm,t!.B (x,y)eMf; de IJ = -mijdB + IIMil1 Set at contains five filters offset by distances ranging from -2 to 2 around position (i, j) in direction d. This short-range Gaussianfilter reducesincomplete activation of spatial competitivekernels in the next stage.

~"ijJtii 2590 J. Opt. Soc.Am. A/Vol. 14,No. 10/0ctober 1997 Chey et at.

Set Mt contains 11 cells offset by distances from -5 to 5 To project a continuous line onto a two-dimensional in direction d from position (i, j). In related applica- grid of pixels, certain sampling problems must be faced, tions the long-range filter is assumed to have a Gaussian since in a regular two-dimensional Cartesian grid verti- profile,33 which is omitted here for computational simplic- cally and horizontally oriented short-rangefilters are not ity. the samelength as that of diagonallyoriented filters. To overcomethis problem, simulations calculated transient 6. Level 5: Directional Grouping and Priming cell responsesat offsets from short-range filter positions Each grouping cell activity nt sums activity from long- to ensure that all filters of the same scale were of the range filters and competeswith other groupingcells to se- samesize. Since this results in a large number of tran- lect a winning direction, which is then usedto selectcon- sient cells,simulations were simplified to reduceprocess- sistent long-range filter activities. Each such cell sums ing time. filter activities over a wide spatial range,a small range of Transient cell responseswere precalculated for each directions, and all scales: simulation, since their responses can be determined ahead of time from input speed. These responsesare dn d. l' , 'J used to determine transient cell responsesbased on the -=- time sincethe leading edge(for On responses)or the trail- dt 5 J' ing edge(for Off responses)of an input passeda given im- (A13) age location. Thesetimes are easily calculated from the where the excitatory input is given by lines' starting position, speed, and orientation. Line simulations utilized only On responsesand continued for 4 time units. Line directions were sampled at time step ~ ~ ~ X(ld -DI)'I'(IDI)([mfj]+)2 (I,J)eOijD . 3 as describedbelow. Four line orientations were used: d - N ij -lIoijll ~. 22.5, 45, 67.5,and 90 deg from horizontal; and three line lengths were used: 5, 13, and 26 units. (AI4) Function X selects the range of directions over which the 8. Lines behind Aperture Inputs grouping cell summates. It is set equal to 1 for Id -DI Barberpolesimulations used a line moving behind an ap- = 0, 1/2 for Id -DI = 1, and 0 otherwise. Set Oij de- erture. Line terminators were never present in the im- termines the domain of spatial averaging. In most simu- age. The line started in the top left-hand corner of the lations it is set large enough to cover the entire image. image and moved until it exited the bottom right-hand The activity m fJ of each long-range filter cell that excites corner. To simulate diagonal barberpole motion, the a grouping cell is rectified and raised to the power 2. Ev- grouping cells spannedthe length of the moving line. In ery grouping cell competes with every other grouping cell any simulation in which grouping cells span the entire at the s~e spatial position. Shunting ensures that each image,the activity of a single grouping cell was calculated cell's activity can never grow above 1. This limits the to- for each directional preferenceand used to reflect net- tal possible feedback strength and ensures that feedback work activity. strength cannot become excessively large. Term 'l'(D) controls the adaptation level of the long- 9. Plaid Inputs range filter cell that signals motion in direction. Adap- Plaid inputs were formed from two overlapping compo- tation occurs from sustained activation of long-range fil- nent lines. Line speedswere chosenso that, regardless ters with directional preference over a long time period. of componentorientations, the plaid moved horizontally It reduces 'l'(D), where D is the winning grouping cell di- at a given speed. A completeplaid simulation input is rection. Term 'l'(D) is initially set to 1 for all directions. shownin Fig. 13,including responsesof On and Off direc- To simulate adaptation, 'l'(D) is set to a fractional value. tional transient cells calculated in the same way as that To find the value at which incoherent motion is first ob- for line inputs. The quantitative plaid simulations used served, 'l'(D) is initially set to 1, then reduced by decre- only the On plaid responsesto the leading edgeof a plaid ments of 0.05 until incoherent motion is observed; that is, to reduceprocessing time. until no winning direction is established at the grouping Three different relative componentorientations were cells. utilized when simulating coherentand incoherent plaid motion: 22.5, 45, and 67.5 deg from horizontal. For 7. Line Inputs each componentorientation, a series of simulations was Inputs consist of a temporal sequenceof luminance pat- run in which 'l'(D) was altered, where D designates terns. Any changeof luminance in these patterns from rightward motion. Thesesimulations started with 'l'(D) one time unit to the next triggers change-sensitiverecep- at 1, at which all plaids movedcoherently, and reduced it tor activity. Three types of inputs were simulated: by units of 0.05 until incoherentmotion was observed. lines, lines moving behind apertures, and plaids. Simu- Changesin plaid componentcontrasts were simulated lations of moving lines specify a line length, orientation, by altering receptor responsemagnitudes in the same and speed. Lines move horizontally with speeds ad- way as was done for the one-dimensionalsimulations. justed so that the horizontal componentof line motion is These simulations assumeda base level of luminance, the same. The line movesfor a fixed time. An image ar- correspondingto receptorresponse magnitude, and then ray sufficiently large to contain the line at its start and calculated a set of response magnitudes that corre- end position is chosen. spondedto the simulated seriesof contrastratios. These Chey et at. Vol. 14,No. 10/0ctober 1997/J. Opt. Soc.Am. A 2591 ratios ranged from 2°.5 to 23.5. For each contrast ratio, a are derived as emergentproperties of network interac- simulation was run and an image direction sampled at tions. Finally, one needsto assesshow many data may time unit 1.5. be rationalized by a single set of proc~sses. This being said, in Eqs. (A2) and (A3), one parameter scales input 10. Outputs amplitude. In the dynamical Eqs. (A4)-(A14), most Energy is defined at each location within each direction baselineprocessing rates were chosenequal to 1 for sim- by summing the long-rangefilter outputs from Eq. (A12) plicity. All fast rates were chosenequal to 10, again for as follows: simplicity, since our goal herein was qualitative rather than quantitative data fits. The slower rate of direc- (A15) tional grouping and priming in Eq. (A13)was set to 1/5. There are a total of 11 nonunity parameters in these and then summing these responsesacross all directions: equations. In addition, sevenparameters determine the sizes of the receptive fields, and there were 16 directions and four scales. These parameters are robust just so Yij = ~£oJ Yij. d (A16) long as reasonablerelative sizes are observed;e.g., long- d range filters have larger scales than short-range filters. The speedmeasure st is derived from the long-rangefil- Most of the directions were not critical in fitting the data ter outputs within eachdirection and position: curves. They illustrate how the model solvesthe aperture problem. With the use of these parameters,33 data ~ m4.ss points were fit in Figs. 2 and 16-18. Figures 5-7 and Lot IJ seS 9-15 simulated manyhundreds of data points to describe st = d . (Al7) the speed-sensitivetuning curves at the various model Yij stagesand the temporal evolution of the motion capture To interpret motion signals as vectors,speed measures processover entire vector fields of motion direction and are combined over different directions. For each direc- speedvectors in responseto visual forms moving through tion a motion vector is defined as the vector extended in time under various experimental conditions. The same that direction with a magnitude equal to the speedmea- processeshave also beenused to explain many other mo- sure: tion data sets,as reviewed in Section8. v~ = (s~ cos d, s~ sin d). (AI8)

The sum of these vectors is the perceived motion at that location: ACKNOWLEDGMENTS The authors thank Cynthia Bradford and Diana Meyers for their valuable assistance in the preparation of the (A19) manuscript. To determine perceived direction and/or speed of a The researchof J. Cheywas supported in part by the moving object,a weighted averageof motion vectors was AdvancedResearch Projects Agency (ONR NOOO14-92-J- taken acrossthe image. Outputs were eliminated from 4015), the U.s. Air Force Office of Scientific Research positions whose total energy across all scalesand direc- (AFOSR F49620-92-J-0225),the National ScienceFoun- tions was less than somethreshold (set to 1 in all simu- dation (NSF IRI-90-00530),and the U.S. Office of Naval lations); Research(ONR NOOO14-91-J-4100).The research of S. Grossbergwas supported in part by the Advanced Re- w..IJ = { (O,O) ifYijvisual system,"Vision 101,470-489 (1994). Res.24, 33-39 (1984). 51. O. J. Braddick, "Low-leveland high-level processesin ap- 25. G. A. Orban, H. Kennedy,and J. Bulier, "Velocity sensi- parent motion," Philos. Trans. R. Soc.London Ser. B 290, tivity and direction selectivity of neuronsin areasVI and 137-151(1980). V2 of the monkey: influence of eccentricity,"J. Neuro- 52. J. A. Movshon,E. H. Adelson,M. S. Gizzi, and W. T. New- physiol. 56, 462-480 (1986). some,"The analysis of movingvisual patterns," in Patte~n 26. L. S. Stoneand P. Thompson,"Human speed perception is RecognitionMechanisms, Vol. 11 of Experimental BraIn contrast dependent,"Vision Res. 32, 1535-1549(1992). ResearchSupplementum, C. Chagas,R. Gattass, and C. 27. P. Thompson,"Perceived rate of movementdepends on Gross,eds. (Springer-Verlag,Berlin, 1985),pp. 117-151. contrast," Vision Res. 22, 377-380 (1982). 53. N. Rubin and S. Hochstein,"Isolating the effect of one- 28. P. D. Tynan and R. Sekuler,"Motion processingin periph- dimensionalmotion signals on the perceiveddirection of eral vision: reactiontime and perceivedvelocity," Vision moving two-dimensionalobjects," Vision Res. 33, 1385- Res. 22, 61-68 (1982). 1396(1993). 29. S. N. J. Watamaniuk, N. M. Grzywacz,and A. L. Yuille, 54. J. Lorenceau,M. Shiffrar, N. Wells, and E. Castet, "Dif- "Dependenceof speedand direction perception on cin- ferent motion sensitive units are involved in recovering Chey et al. Vol. 14,No. 10/0ctober 1997/J. Opt. Soc.Am. A 2593

the direction of moving lines," Vision Res.33, 1207-1217 organizationof directionally sensitivecells in visual area (1993). MT of the macaque,"J. Neurophysiol.51, 16-31 (1984). 55. E. Mingolla, J. T. Todd, and J. F. Norman,"The percep- 80. J. Allman, F. Miezin, and E. McGuinness,"Direction and tion of globally coherentmotion," Vision Res. 32, 1015- velocity-specificresponses from beyondthe classicalrecep- 1031(1992). tive field in the middle temporal visual area (MT)," Per- 56. J. Lorenceau and M. Shiffi-ar, "The influence of termina- ception14, 105-126 (1985). tors on motion integration across space," Vision Res. 32, 81. W. T. Newsome,M. S. Gizzi, and J. A. Movshon,"Spatial 263-273 (1992). and temporal properties of neurons in macaqueMT," In- 57. D. W. Williams and R. Sekuler, "Coherent global motion vest. Ophthalmol. Visual Sci. 24, 106 (1983). percepts from stochastic local motions," Vision Res. 24, 82. S. M. Zeki, "Functionalorganization of a visual area in the 55-62 (1984). posterior bank of the superior temporal sulcus of the 58. G. R. Stoner, T. D. Albright, and V. S. Ramachandran, rhesus monkey," J. Physiol. (London) 236, 546-573 "Transparencyand coherencein human motion percep- (1974). tion," Nature (London)344, 153-155 (1990). 83. D. C. Bradley,N. Qian, and R. A. Anderson,"Integration 59. J. C. Trueswell and M. M. Hayhoe, "Surface segmentation of motion and stereopsisin middle temporal cortical area mechanisms and motion perception," Vision Res. 33, 313- of macaques,"Nature (London)373, 609-611 (1995). 328 (1993). 84. J. A. Gengerelli,"Apparent movementin relation to hom- 60. S. Grossberg,"Cortical dynamics of 3-D figure-ground onymousand heteronymousstimulation of the cerebral perception of 2-D pictures," Psychol.Rev. 104, 618-658 hemispheres,"J. Exp. Psychol.38, 592-599 (1948). (1997). 85. I. M. Spigel,"Problems in the study of visually perceived 61. C. Enroth-Cugelland J. Robson,"The contrast sensitivity movement: an introduction," in ContemporaryTheory of retinal ganglion cells of the cat," J. Physiol. (London) and Researchin Visual Perception,R. H. Haber,ed. (Holt, 187,517-552 (1966). Rinehart & Winston, New York, 1968),pp. 103-121. 62. S. Hochstein and R. Shapley, "Quantitative analysis of 86. H. R. Wilson, V. P. Ferrera, and C. Yo, "A psychophysi- retinal ganglion cell classifications,"J. Physiol. (London) cally motivated modelfor two-dimensionalmotion percep- 262, 237-264 (1976). tion," Visual Neurosci.9, 79 (1992). 63. S. Hochsteinand R. Shapley,"Linear and nonlinear spa- 87. G. A. Carpenterand S. Grossberg,"A massively parallel tial subunits in Y cat retina ganglion cells," J. Physiol. architecture for a self-organizingneural pattern recogni- (London)262, 265-284 (1976). tion machine," Comput. Vision Graphics Image Process. 64. N. K. Logothetis,P. H. Schiller, E. R. Charles,and A. C. 37,54-115 (1987). Hurlbert, "Perceptualdeficits and the activity of the color- 88. K Tanaka,Y. Sugita,M. Moriya, and H. A. Saito, "Analy- opponentand broad-bandpathways at isoluminance,"Sci- sis of objectmotion in the ventral part of the medial supe- ence247, 214-217 (1990). rior temporal area of the macaquevisual cortex,"J. Neu- 65. S. Grossberg,"Cortical dynamics of three-dimensional rosci. 69, 128-142 (1993). form, color, and brightness perception. II: Binocular 89. S. Treue and J. H. R. Maunsell, "Attentional modulation theory," Percept.Psychophys. 41, 117-158(1987). of visual motion processing in cortical areas MT and 66. M. Cohenand S. Grossberg,"Neutral dynamicsof speech MST," Nature (London)382,539-541 (1996). and language coding: developmentalprograms, percep- 90. K M. O'Craven,B. R. Rosen,K. K Kwong,A. Treisman, tual grouping, and competition for short term memory," and R. L. Savoy,"Voluntary attention modulatesfMRI ac- Human Neurobiol.5, 1-22 (1986). tivity in human MT-MST," Neuron 18, 591-598 (1997). 67. R. C. Emersonand G. L. Gerstein,"Simple striate neurons 91. E. C. Hildreth, The Measurementof Visual Motion (MIT in !the cat: II. Mechanisms underlying directional Press,Cambridge, Mass., 1984). asYmmetryand directional selectivity," J. Neurophysiol. 92. G. R. Stoner and T. D. Albright, "The influence of 40, 136-155 (1977). foreground/backgroundassignment on transparency and 68. L. Ganz,"Visual cortical mechanismsresponsible for di- motion coherencyin plaid patterns," Invest. Ophthalmol. rection selectivity," Vision Res. 24, 3-11 (1984). Visual Sci. 33, 1050(1992). 69. A. W. Goodwin,G. H. Henry, and P. O. Bishop,"Direction 93. A. Baloch, S. Grossberg,E. Mingolla, and C. A. M. selectivity of simple striate cells: propertiesand mecha- Nogueira,"A neural modelof first-order and second-order nisms,"J. Neurophysiol.38, 1500-1523(1975). motion perception and magnocellular dynamics," Tech. 70. P. Heggelund, "Direction asymmetry by moving stimuli Rep. CAS/CNS-96-030,1996 (BostonU., Boston,Mass.). and static receptivefield plots for simple cells in cat stri- 94. V. P. Ferrera and H. R. Wilson, "Direction specific mask- ate cortex," Vision Res. 24, 13-16 (1984). ing and the analysis of motion in two dimensions,"Vision 71. H. B. Barlow and W. R. Levick, "The mechanismof direc- Res.27, 1783-1796(1987). tionally selectiveunits in rabbit's retina," J. Physiol. (Lon- 95. G. Francisand S. Grossberg,"Cortical dynamicsof bound- don)178, 477-504 (1965). ary segmentationand reset: persistence,afterimages, 72. M. Ariel and N. W. Daw, "Pharmacologicalanalysis of di- and residual traces," Perception26, 543-567 (1996). rectionally sensitive rabbit retinal ganglion cells," J. 96. G. Francis,S. Grossberg,and E. Mingolla, "Cortical dy- Physiol. (London)324, 161-185 (1982). namics of feature binding and reset: control of visual 73. D. H. Hubel and T. N. Wiesel,"Receptive fields of single persistence,"Vision Res.34, 1089-1104(1994). cells in the cat's striate cortex," J. Physiol. (London)148, 97. S. Grossberg,"How is a moving target continuously 574-591 (1959). tracked behind occluding cover?" in High Level Motion 74. D. H. Hubel and T. N. Wiesel,"Receptive fields, binocular Processing,T. Watanabe,ed. (MIT Press, Cambridge, interaction and functional architecture in the cat's visual Mass.,1997). cortex,"J. Physiol. (London)160, 106-154 (1962). 98. C. A. M. Nogueira,E. Mingolla, and S. Grossberg,"Com- 75. S. W. Kuffier, "Dischargepatterns and functional organi- putation of first orderand secondorder motion by a model zation of the mammalian retina," J. Neurophysiol. 16, of magnocellulardynamics," Invest. Ophthalmol. Visual 37-68 (1953). Sci. 34, 1029(1993). 76. P. H. Schiller, "Central connectionson the retinal ON- 99. P. H. Schiller,N. K Logothetis,and E. R. Charles,"Func- and OFF-pathways," Nature (London) 297, 580-583 tions of the colour-opponentand broad-bandchannels of (1982). the visual system,"Science 343, 68-70 (1990). 77. S. Grossberg,"How doesa brain build a cognitive code?" 100. S. Grossbergand E. Mingolla,"Neural dynamicsof motion Psychol.Rev. 87,1-51 (1980). perception: direction fields, apertures, and resonant 78. S. Grossberg,"The quantized geometry of visual space: grouping,"Percept. Psychophys. 53, 243-278 (1993). the coherentcomputation of depth, form, and lightness," 101 A. Gave,S. Grossberg,and E. Mingolla, "Brightness per- Behav. Brain Sci. 6, 625-657 (1983). ception, illusory contours, and corticogeniculate feed- 79. T. D. Albright, R. Desimone,and C. G. Gross,"Columnar back," Visual Neurosci. 12, 1027-1052(1995). 2594 J. Opt. Soc.Am. A/Vol. 14,No. 10/0ctober 1997 Chey et at.

102. G. Govindarajan,S. Grossberg,L. Wyse,and M. Cohen,"A 106. J. Allik, "Competingmotion paths in sequencesof random neural network model of auditory sceneanalysis and dot patterns," Vision Res.32, 157-165 (1992). source segregation," Tech. Rep. CAS/CNS-TR-94-039, 107. M. Shiffrar, x. Li, and J. Lorenceau,"Motion integration 1994)(Boston U., Boston,Mass.). acrossdiffering image features," Vision Res. 35, 2137- 103. S. Grossberg, I. Boardman, and M. Cohen, "Neural 2146 (1995). dynamics of variable-rate speechcategorization," J. Exp. 108. R. J. Douglas,C. Koch, M. Mahowald, K A. C. Martin, Psychol.Hum. Percept.Perf. 23, 481-503 (1997). and H. H. Suarez,"Recurrent excitation in neocorticalcir- 104. R. P. Power and B. Moulden, "Spatial gating effects on cuits," Science269, 981-985 (1995). judged motion of gratings in apertures," Perception21, 109. S. Grossberg,Studies of Mind and Brain (Reidel,Boston, 449-463 (1992). 1982). 105. K Nakayamaand G. H. Silverman,"Temporal and spatial 110. A. L. Hodgkin, The Conductionof the Nervous Impulse characteristicsof the upper displacementlimits for motion (LiverpoolU. Press, Liverpool,UK, 1964). in random dots," Vision Res. 24, 293-299 (1984).