The Barberplaid Illusion

vision Res., Vol. 36, No. 19, pp. 3061–3075,1996 Pergamon Publishedby ElsevierScienceLtd. PII: S0042-6989(96)00064-8 Printedin Great Britain @ 0042-6989/96$15.00+ 000

The Barberplaid Illusion: Plaid Motion is Biased by Elongated Apertures BRENT R.BEUmER,* JEFFREYB.MULLIGAN,*LELANDs.STONE*t Received28August1995;injinalform1 February1996

The perceived direction of motion of plaids windowed by elongated spatial Gaussians is biased toward the window’s long axis. The bias increases as the relative angle between the plaid motion and the long axis of the window increases, peaks at a relative angle of-45 deg, and then decreases. The bias increases as the window is made narrower (at fixed height) and decreases as the component spatial frequency increases (at fixed aperture size). We examine several models of human motion processing (cross-correlation, motion-energy, intersection-of-constraints, and vector-sum), and show that none of these standard models can predict our data. We conclude that spatial integration of motion signals plays a crucial role in plaid motion perception and that current models must be explicitly expanded to include such spatial interactions. Published by Elsevier Science Ltd.

Plaids Motionmodels Direction discrimination Aperture Intersection-of-constraints rule

INTRODUCTION pattern is uniquely determined. Thus, one might expect Human perception of motion depends not only on the that the type of window in which one-dimensional patterns are moved would determinehow this ambiguity physical motion of objects, but also on the conditions is resolved, and thus which of the many possible under which the motion is viewed.A simple yet dramatic directions of motion is actually perceived. On the other example of this is the barberpole illusion, in which the hand, the direction of motion of two-dimensional perceived direction of motion of an obliquely oriented patterns, such as plaids, is unambiguous (Adelson & drifting grating is vertical, when viewed through a Movshon, 1982), because the multiple constraints vertically oriented rectangular aperture; but the same providedby the componentsallow only a single solution motion appears horizontal,when the rectangularaperture [see Fig. l(b)]. The intersection-of-constraints rule is horizontal. The influence of the aperture on the specifieshow this unique resultantpattern velocity could perceived motion of one-dimensionalpatterns has been be computedfrom the componentvelocitiesand orienta- extensively examined (e.g. Wallach, 1935; Mulligan, tions (Fennema & Thompson, 1979; Adelson & Mov- 1991;Power & Mouldon,1992;Kooi, 1993;Mulligan& shon, 1982). Is the human visual system able to extract Beutter, 1994). The question that we examine in this this unique correct velocity independentof the aperture, paper is whether the perceived direction of motion of or does the window shape affect the processing of the two-dimensionalpatterns such as plaids is also affected motion signals? by the type of viewing window.We addressthis question In primates, evidence from anatomy, physiology,and by measuring the perceived direction of moving plaids psychophysicssuggests that motion processing of two- windowed by elongated spatial Gaussians. dimensional velocity appears to occur in a two-stage The direction of motion of a one-dimensionalpattern process. First, directionallyselective mechanismsdetect viewed through a restricted aperture is inherently the motionof localone-dimensionalfeaturesin the image ambiguous, as shown in Fig. l(a). The motion is (Hubel& Wiesel, 1968;Watsonet al., 1980;De Valoiset consistent with any velocity vector that falls on the al., 1982b)and then a second stage integratesthese one- constraint line, because only the velocity component in dimensionalsignals over a larger area of the visual field the direction perpendicular to the orientation of the to extract the two-dimensionalpattern velocity (Adelson & Movshon, 1982; Maunsell & Van Essen, 1983a,b; Albright, 1984; Mikami et al., 1986a,b;Movshonet al., *MS 262-2, Human and Systems Technologies Branch, Flight 1986, 1988; Newsome et al., 1986; Ungerleider & Managementand HumanFactors Division,NASAAmes Research Center, Moffett Field CA 94035-1000,U.S.A. Desimone, 1986; Rodman & Albright, 1989; Welch, ~To whom reprint requests shouldbe addressed. 1989; Stone, 1990). In primate cerebral cortex, the first 3061 3062 B. R. BEUTTERet al.

psychophysicalevidence for noncomponentdriven one- stage velocity estimation [e.g. Derrington & Badcock (1992)]. Additionally, there are clearly other motion processing pathways leading to MT, directly from cortical areas V2 and V3 and from the” Superior Colliculus via the pulvinar, which may also play an important role in velocity estimation (Maunsell & Van Essen, 1983a; Ungerleider et al., 1984; De Yoe & Van Essen, 1985;Ungerleider& Desimone, 1986;Rodmanet al., 1989, 1990). Modelers have also explored the ways primate visual cortex might estimate velocity. These models fall into severalclasses. Bulthoffet al. (1989) have proposedthat velocity is estimated by finding the maximum of an image cross-correlation function. Motion-energy type FIGURE 1. (a) Two one-dimensionalgratingsoriented ~ 45 deg from models (Watson & Ahumada, 1983; van Santen & the vertical are shownin circular apertures.The velocitycomponentin Sperling, 1984, 1985;Adelson & Bergen, 1985;Watson the direction perpendicularto the orientationof the grating is fixedby the constraint line (solid diagonal lines), while the componentin the & Ahumada, 1985; Heeger, 1987) determine perceived parallel directions is ambiguous. Velocities consistent with the velocity using more biologically plausible processes. constraint line are shown by the arrows. The velocity that is usually First, the image is decomposed into its spatio-temporal perceived is shownby the solid filled arrow. (b) When these two one- components (much like what is done in Vi). Then, dimensional gratings are added together, the result is a two- velocityis estimatedby findingthe singlepatternvelocity dimensionalplaid pattern, which has a uniquevelocity. The constraint lines for each grating intersect at a single point, which determinesthe most consistentwith the entire motion-energyspectrum. unique IOC velocity of the plaid (solid filled arrow). The untilled Intersection-of-constraints(IOC) models (Fennema & arrows which represent possible velocities of the one-dimensional Thompson, 1979; Adelson & Movshon, 1982) are also gratings in (a), satisfy the constraint line of only one of the component explicitly two-staged. First, the motions of the compo- gratings,but are inconsistentwith the additionalconstraintimposedby the other grating, and therefore do not correspond to possible plaid nent gratings are estimated separately, and then the IOC velocities. rule is used to compute the pattern velocity that is consistentwith all of the componentconstraints(Fig. 1). Chubb and Sperling (1988, 1989) first proposed the existence of two motion pathways: a Fourier pathway, operating directly on the stimulus, and a non-Fourier pathway that contains a nonlinear preprocessing stage, stage of motion processing occurs in primary visual that performs a rectification or squaring prior to the cortex (VI). Motion-sensitiveneurons in V1 have small motion processing. Wilson et al. (1992) formulated a receptive fields tuned to stimuli of a specific size, motion model incorporatingboth these pathways. Their orientation, and direction of motion (Hubel & Wiesel, vector-sum model measures both Fourier and non- 1968; De Valois et al., 1982b). Each of these cells Fourier motion in separate pathways,and then combines responds maximally to the component of motion in the these signals using a vector-sum rule to compute the direction perpendicular to its preferred orientation and direction of motion of the pattern. shows little or no response to motion parallel to its In this paper, we first describe the effect of aperture preferred orientation. Thus, when an object moves, VI shape on the perceived direction of moving two- neurons respond to the local motion of one-dimensional dimensionalpatterns (plaids). We then use these results features in the image, and therefore cannot individually to determine if any of the above models can predict signal the velocity of the full two-dimensionalpattern. human performance for this type of plaid stimulus. However, the actual two-dimensional pattern velocity Preliminary results have been presented elsewhere can be recovered by combining the ,one-dimensional (Beutter et al., 1994a,b). signalsfrom multipleV1 neurons.Directionallyselective VI neurons project to the middle temporal (MT) area (Maunsell & Van Essen, 1983a;Movshon & Newsome, METHODS 1984), where most neurons appear to respond preferen- tially to motion (Zeki, 1974; Maunsell & Van Essen, Observers 1983b;Albright, 1984;Albrightet al., 1984).Neuronsin Four observers between the ages of 33 and 40yr MT have larger receptivefields(Maunsell& Van Essen, participated in the experiments. One observer, PS,was 1983b) and there is some evidence that MT neurons naive but also strabismic. integrate the one-dimensional edge motion signals to compute the two-dimensionalpattern velocity (Movshon Stimuli and apparatus et al., 1986; Rodman & Albright, 1989; Britten et al., The stimulus, Z(2,t),in these experiments was a 1993). The above notwithstanding, there is also some driftingplaid,P(Z, t), windowedspatiallyby a stationary APERTURESBIAS PERCEIVEDPLAID MOTION 3063 elongated Gaussian, W(7), and temporally by a trapezoidal function,H(t), as described by Eqs (l)-(4). z(i,t)= z~[l + CP(2,t)w(2)H(t)] (1) where P(I, t) = sin [(27r ~, .2 +Jt )1+siw~:”~+fi’)1 (2)

(2.2)2 (e)’ W(2) = exp –y – z~ (3) ( H w ) H(t) = t/Tl, O< t < T, 1 TI < t < T2 (4) 1 – (t– T2)/TI T2 < t < T2+ TI -Th$lplaid, P(X, t), was the sum of two orthogonal V, .f, = O)sine-wave “component” gratings moving with equal speeds.BotJhsine:~ave gratingswere of equal spatial frequencies, (~,1 = ~, I = 0.3, 0.6, or 1.2 c/deg), equal temporal frequencies @t= 4 Hz), and equal peak FIGURE 2. This figure shows a single frame of a two-dimensional plaid windowedby an elongated spatial Gaussian.The windowhas an contrast(c = 0.125).The mean luminance,1., was fixedat aspect ratio of 2.0 and is oriented 40 deg to the right of straight down. 42 cd/m2. Because the plaid was symmetric and its The direction of the plaid motion is straight down as indicated by the component speeds were equal, the plaid direction of filled arrow, while the perceived direction of motion is biased toward motion was always midway between the orientationsof the window orientation, as indicated by the open arrow. the componentgratings.We varied its directionof motion by rotating both componentgratings equally.The spatial window, W(2), was an elongated Gaussian with unequal sum gratingwhose phase is varied by modifyingthe sub- standard deviations, OH(height) and crw (width), in its componentgratings’relativeamplitudesthroughchanges two principal directions, Z and ZL, respectively. In all in the lookup table. The dithering procedure produces experiments,OHwas fixedat 2.5 deg, and the aspectratio, low-contrast spatial artifacts, which are near or below ~H/flw,was varied by setting rsw to be 0.625, 1.25, or threshold (Mulligan & Stone, 1989). The plaid stimulus 1.77 deg. We definedthe absolutewindow orientationto was constructed by creating four sub-component sinu- be the orientationof the unit vector Z.The time course of soidalgratings(a sine and cosine sub-componentpair for the stimuluswas controlledby H(t). It linearlyramped on each plaid component)which were then multipliedby the for 167 msec (7’1),remained constant for 500 msec (T2– elongated spatial Gaussian window function to produce Tl), and then linearly ramped off for 167 msec. An four grating images. Each of these four images was then example of a single of frame of the stimulusis shown in dithered to three levels using a modified error-diffusion Fig. 2. algorithm (Mulligan, 1986). These images were then The stimuli were displayed on a 19” Barco color combined to produce a final image containing 81, (34), monitor (model CDCT 6351B) using an AT Vista video gray levels. For each frame, the appropriatelookup table display system hosted by an IBM 486. The monitor was values of these gray levelswere precomputedand stored. run in the 60-Hz interlaced mode. To minimize interlace Plaid motion was produced by sequentially loading the artifacts, alternate horizontal lines were set equal to one lookup tables on a frame-by-frame basis. For each another by computing a 320 x 243 pixel image and then combination of plaid and window angle, a separate zooming it by a factor of two in both the horizontal and image was created. vertical directions so that it filled the 640x 486 display region. For the spatial and temporal frequencies of our Procedure and data analysis stimuli, no significant aliasing occurred. The display pixel sizes were 0.47 mm horizontally and 0.54 mm We used the method of adjustment to measure the vertically.At the 57-cmviewingdistance,the full display perceived direction of plaid motion. After the stimulus subtended 30 deg x 26 deg. The luminance output of the had been presented, a pointer that subtended 15 deg monitor was calibrated to correct for its gamma appeared in the center of the screen. Two keys allowed nonlinearityusing a lookup table. the observer to rotate the pointer about the center of the The plaid motion was produced by using a dithering display in either a clockwise or a counterclockwise animation method which is described in detail in directiun. The observer was asked to adjust the orienta- Mulligan and Stone (1989). Briefly, to generate a single tion of the pointer so that it was aligned with the drifting sinusoidal grating, it uses two sub-component perceived direction of plaid motion. Although there was gratingsdifferingin spatialphase by 90 deg to produce a no limit to the time allowedto make the setting,observers 3064 B. R. BEUTTERet al.

15 for observerBB, as a function of the window orientation -i To for three separate runs of Experiment 2, with a window aspect ratio of 4.0 and a componentspatial frequency of 0.6 c/deg. The results were symmetric about zero: positive window orientations produced biases approximately equal to, but opposite in sign to, those produced by negativewindow orientationsof the same magnitude. Becauseof thissymmetry,the resultsfor the negativeand positivewindoworientationsfor each run were combined to compute the mean bias for each window orientation and these were then averaged over runs to compute the average bias (filled squares). In subsequentfigures,only -5 the average biases are shown. The error bars are the standard deviations of the mean biases over the + average o runl individualruns. 0 run2 -lo 0 o A run3 RESULTS

IV In Experiment 1, we measured the biases in the -15 A A BB 1 . perceived direction of plaid motion produced by an I I I I I ! -90 -60 -30 0 30 60 90 elongated window, with an aspect ratio of 2, for a plaid Window Orientation (0) composed of 0.6 c/deg sinusoidal gratings. Plots of the biases as a functionof the relativewindoworientationfor four observersare shown in Fig. 4. The resultsfor all the FIGURE3. The biases in perceived direction are plotted as a function of the windoworientationfor observerBB.The stimulushad an aspect observers were similar: they showed biases toward the ratio of 4.0 and a componentspatial frequencyof 0.6 c/deg. The open long axis of the window. For window orientations symbols show the biases (8 settings) found in three separate runs for c-45 deg, the biases increasedas the window orientation each window orientation, while the filled squares show the average increased.For window orientations>-45 deg, the biases biases. The average biases (48 settings) were computedby combining decreased, and even reversed for one observer (PS). the data for positive and negative window orientations for each run, and then averaging over inns. Thus, the magnitudesof the biases for When the plaid movedin the directionof the shortaxisof the positive and negative window orientations are identical. Both are the window (t90 deg), the biases were negligible.The presented for clarity in this and subsequent figures. The error bars peak biases of the four observers ranged from 7.4 to represent the standard deviations over runs. In this and subsequent 11.1 deg, and occurred at a window orientation of figures, the horizontal line represents zero bias. -45 deg. In Experiment2, we investigatedthe effect of varying the window aspect ratio on the perceived direction of generally did so in 1–3 sec. Before data collectionbegan, plaid motion.The biasesproducedby aspectratiosof 1.4, observers were shown sample stimuli and practiced 2.0, and 4.0 for a plaid composedof 0.6 c/deg sinusoidal adjusting the pointer. No feedback was given. gratings are shown in Fig. 5. The results for the two We defined the window aspect ratio to be the ratio of observerstested were similar:the biases increased as the the window’s height to its width, a~low. Window window aspect ratio increased. The pattern of results orientation was defined relative to the plaid as the found for an aspectratio of 2.0 was also found for the 1.4 absolutewindow orientationminusthe absolutedirection and 4.0 aspect ratios: biases increased as the window of motion. Similarly, the bias was defined as the orientationincreased,reached a peak at an angle of about perceived direction of motion relative to the absolute 45 deg, and then decreased.A summaryof theseresultsis shown as the filled circles in Fig. 8(a), in which the plaid direction of motion. In a preliminary experiment, magnitudeof the peak bias is plotted as a functionof the we verified that a circularly symmetric window (aspect aspect ratio. The largest aspect ratio, 4.0, produced the ratio equal to 1.0) produced negligiblebiases. largest biases, while the smallest aspect ratio produced For each run, the spatial frequency of the components the smallest biases. These data show that long narrow and the aspect ratio of the Gaussian window were fixed. windows (high aspect ratios) produce large biases, while Runs consisted of 152 settings: two repetitions of all more circular windows (low aspect ratios) produce combinations of four plaid directions of motion ( t 20 smaller biases. and *4O deg), and of 19 window orientations(O, t 10, In Experiment 3, we examined the effects of varying ~20, ~30, +40, ~50, +60, -L-70,~80, +90 deg). the spatial frequency of the plaid components, while Each observer performed a minimum of three runs in keeping the window constant. Because the window each experiment. Because the results for each absolute parameterswere fixed,varying the spatialfrequencyalso plaid direction were similar, the bias for each window changed the number of visible cycles of the component orientation was computed by first averaging the results gratings. For simplicity, we have chosen to discuss the from the four plaid directions.Figure 3 shows the biases data directlyin termsof the componentspatialfrequency, APERTURESBIAS PERCEIVEDPLAID MOTION 3065

157 15-1 AR 2.0 lo- SF 0.6 C/d 5-

-5- -10-

-15 LS -154 Pv ~ I 1 I I 1 1 1 1 1 1 w -90 -60 -30 0 30 60 90 -90 -60 -30 0 30 60 90 rn .- : ’51 151 lo- lo-

5- 5-

0- 0- -5- -5- -1o- -1o-

-15 Ps -15 BB 1 I 1 I 1 1 I 1 1 1 -90 -60 -30 0 30 60 90 -90 -60 -30 0 30 60 90 Window Orientation (0)

FIGURE4. The average bias in perceiveddirection is plotted as a functionof windoworientationfor a stimuluswith an aspect ratio of 2.0 and componentspatial frequencyof 0.6 c/deg. Results are shownfor four observers,includingnaive observer, PS. The error bars represent the standard deviationover inns.

15- ,,.P...q Aspect Ratio .... i? 4.0 ,.@””” ~..., lo- Q.,,, + 2.0 ..!d” -(3 1.4 5-

..... -10- ‘“’”D.,..,.,a,,’” ~ -15– LS - 1 I I I I I u) m ti 15. ~,,,,...B...... B...... m‘..,, ,, lo- @/ / 5- i

-5-

-lo - ““~....,..=.....+. -15 - 00 I I I I I I -90 -60 -30 60 90 Window O;entati% (0)

FIGURE5. The average bias in perceived direction as a functionof windoworientationis plotted for stimuli with component spatial frequencyfixedat 0.6 c/deg and three aspect ratios: 1.4(0), 2.0 (~), and 4.0 (0). Resultsare shownfor two observers. For clarity, error bars are shown only for the aspect ratio equal to 2.0 data and represent the standard deviationover inns. B. R. BEU’lTERet al.

15- Spatial Frequency ,,,...q ....,13’‘, -m 0.3 Cld ‘$., lo- + 0.6 C/d ,,,,..~’ “!, -0- 1,.2cld ,,,F 5-

....,., -1o- q ~...... ~ ‘,&....” - -15 LS L I I I I I rn at S 15 i ..Q. 10 ,...., ..,.’..’,,,,... i ‘~’”.i,,,,,, 5 ./” -o-cl Q ‘. i T .,’‘ / b. “-””~..., “— A { -. e :. ‘,L,Iir -5 “z;;,‘b‘.‘v--e-- ,.,./ ‘.,,’ ,..., ‘$❑ -10 ‘....., ,,,..’”’ ...... “’~.’” -15 w BB I I I I I I -90 -60 -30 60 90 Window O!ientat%n (0)

FIGURE6. The average biases in perceiveddirectionare plotted as a functionof windoworientationfor stimuliwith an aspect ratio fixed at 2.0 and three componentspatial frequencies: 0.3 (0), 0.6 (e), and 1.2c/deg (0). Results are shown for two observers. For clarity, error bars are shownonly for the 0.6 c/deg data and represent the standard deviationover runs.

rather than in terms of bandwidth or number of visible direction of the long axis of the window. To understand cycles. The biases for plaids composed of sinusoidal how this bias may arise, we examined the predictionsof gratings with spatial frequencies of 0.3, 0.6, and 1.2 several models of human motion processing: a cross- c/deg, and an aspect ratio of 2.0 are shown in Fig. 6. The correlation model, modified IOC models, a motion- results for the two observers were similar: the biases energy model, and Wilson, Ferrera and Ye’s vector-sum decreased as the componentspatial frequency increased. model (1992). The pattern of results found for each of the spatial frequencieswas similar: biases increased as the window Cross-correlationmodels orientation increased, reached a peak at an angle of Cross-correlation models [e.g. Leese et al. (1970); -45 deg, and then decreased.A summaryof these results Bulthoffet al. (1989)] determinethe directionof motion is shown as the filled circles in Fig. 8(b), in which the by computing the translation that produces the maximal magnitudeof the peak bias is plotted as a functionof the overlap between the image at two different times. We component spatial frequency. The lowest spatial fre- calculated the global cross correlationfor plaids moving quency produced the largest biases, while the highest within elongated windows by using the following spatial frequency produced the smallest biases. equation: m cc MODELING CC(A, Ay, t, At) = dx dy ~J (5) Unwindowed moving plaids have an unambiguous Z(x,y, t) .l(x + Ax, y Gy, t ;lt) directionof motionwhich can be foundby using the IOC rule (Fig. 1).Our resultshoweverclearly showthat plaids We determined the predicted direction of motion by in elongatedwindowsare not always seen to move in this computingthe velocity(Ax/At,Ay/At)that maximizedthe direction. Rather, observers consistentlyreport a bias in cross-correlation.* The predicted biases for an aspect the perceived direction of plaid motion toward the ratio of 2.0 and componentspatialfrequencyof 0.6 c/deg APERTURESBIAS PERCEIVEDPLAID MOTION 3067

lo- Predictions Data — CrossCorrelation

? -

-5-

-10-

-90 -60 -30 60 90 Window Or!entatic%(0)

FIGURE7. The predictedbiases in the perceived directionof plaid motionfor a windowwith an aspect ratio equal to 2.0 and plaid componentspatial frequencyof 0.6 c/deg are plottedas a functionof the windoworientationfor the four modelsdescribed in the text (motion-energy,dashedline; cross-correlation,thick line; IOC,dotted line; and vector-summodel,dot-dashedline). The filled circles show the psychophysicaldata averaged over four observers.The error bars represent the standard deviation over observers.

are shown as solid linesin Fig. 7. The mean data from our direction is constrained to lie on a plane in frequency four observersare plotted for comparison(filledcircles). space which is definedby the equation: We also computed the biases predicted by the cross- correlation model for other aspect ratios and component VJ.X + VJY +fi = o (6) spatial frequencies, and found that all of the conditions where VXand VYare the velocitiesin thex andy directions, examined showed similarpatternsof biases as a function and~X,~Y,and~tare the spatial and temporal frequencies of window orientation.These results are summarized in (Watson& Ahumada, 1983).The second stage computes Fig. 8, in which the thick lines show the predicted peak the velocity by finding the plane which optimally biasesplottedboth as a functionof the aspectratio (a) and matches the motion-energy estimates of the first stage. as a function of the component spatial frequency (b). The various models differ in the details of the shapes of Although the cross-correlation model qualitatively pre- the receptivefieldsof the front-endfilters,and in how the dicts the trends in our data, the predicted biases are optimal plane is determined. consistentlytoo small to explain our results. Rather than simulating a specific model, we used the exact motion-energyspectrumas the input to the second Motion-energymodels stage. Our stimulus was windowed both spatially and Many models of human motion processing [e.g. temporally, so that instead of being four points, its Adelson & Bergen (1985); Heeger (1987)] use the Fourier energy spectrum was smeared over four three- stimulusmotion energy to determinevelocity. Generally dimensionalvolumes(four elongatedblobs, two for each these models consist of two stages. In the first stage, component, with coplanar centers). We modeled the arrays of quadrature-pairfiltersare used to extract phase- second stage as estimating the perceived velocity by independentmeasures of the stimulus energy in various finding the velocity plane that best fits the amplitude spatio-temporalfrequency bands. The output of this first spectrum of the stimulus. We began by computing the stage, an estimate of the motion energy, is then analyzed distance, A, of each spectral component @..,&,fi), from by a second stage to determine the direction of motion. the velocity plane as the projection of the component,in The motion energy of an object translating in a fixed the direction perpendicularto the plane.

(LJYJ) “ (kbhl) (7) *The translations, Ax and Ay, producing the maximum cross- A[(vx, vy), (.My,fi)] = correlation are independent of the temporal windowing,because I(vx, Vy,1)[ the temporal window is merely a multiplicative constant which is —.(VX!VYJ) independentof AKand Ay.The maximumhas a small dependence This uses fact that the vector, l(vx,.,,l)l~M a unit vector on both the absolute and relative phases of the two images to be perpendicular to the velocity plane [Eq. (6)]. The cross-correlated, i.e. it is a weak function of both tand At. We deviation of the stimulus for each candidate velocity found that varying t and Arnever changedthe computedbiases by >5~0.For the calculations shown in the figures, we used cosine- was then computed by weighting the square of the phase gratings, and set t to zero and At to be one-tenth of the distance of each spectral component from the velocity temporal period of the gratings. plane by the magnitude of its Fourier amplitude, 3068 B. R. BEUTTERet al.

a) 20- Predictions Data -- MotionEnergy -+ Average 15- — cr~~~ c~rrelati~n : . Ioc - – - VectorSum lo- / /’ - 5- S ...... —.—.—.—.—-——...... %0 —-—-—-—.-. ------. g -s~ 4 21 As~ect Ratio .# b)

\ lo- \

: *, 2 3 45 6789 2 0.1 1 Spatial Frequency (c/d)

FIGURE8. The average psychophysicaldata (o) are comparedto the predictionsof the four models describedin the text. The error bars for the psychophysicaldata represent the standarddeviationover the two observers.(a) The magnitudesof the peak biases are plotted as a functionof the windowaspect ratio while the componentspatial frequencyis held constant at 0.6 c/deg. The predictionsof the vector-sum model are negative because the direction of the predictedbiases was oppositeto that of the other models and the psychophysicaldata. (b) The peak biases for a windowwith an aspect ratio fixed at 2.0 are plotted as a function of the componentspatial frequencyfor the cross-correlationand motion-energymodels.The error bars represent the standard deviationover observers.

W(LM)17 aIIdi@@w overallSPatiO-teIIIPo’alsimulations of other models for comparison. The frequencies to determine the total deviation.* predictedbiases increasedas relativewindoworientation increasedand peaked at -45 deg, and then decreased.We found this same pattern of biases as a functionof window ‘(vxvy)=rmd~~:mdfyrmdfi (8, orientation in our simulationsof other aspect ratios and component spatial frequencies. Figure 8 summarizes w(.fx>~>~)l~A[(vm vy), (tx,f,,fi)]’ these results by showing the predicted peak biases The model output velocity was defined as that (dashed lines) as a function of window aspect ratio (a) correspondingto the minimal deviation.~The predicted and component spatial frequency (b). The predicted biases for an aspect ratio of 2.0 and component spatial biases once again agree qualitativelywith the trends in frequencyof 0.6 cldeg are shownas dashedlines in Fig. 7 our data, and although they are larger than those along with the average data from our four observersand predicted by the cross-correlation model, they are still too small to explain the data. *If each spectral componentis weightedby its power, IAzl,instead of its amplitude, [Al, the results are similar although the predicted biases are somewhat smaller. ModifiedIOC models TTo simplify the calculation, we approximated the trapezoidal We also examined the predictions of models that temporalwindowfunctionby a Gaussianwith a standarddeviation of 0.25 sec. The bias dependsonlyweaklyonthe temporalwindow. explicitly implement the IOC rule (Fennema & Thomp- Varying the standard deviation of the Gaussian from 0.1 to 5 sec son, 1979; Adelson & Movshon, 1982). The IOC rule producedchanges of< 15%. computes an unambiguous velocity of a moving two- APERTURESBIAS PERCEIVEDPLAID MOTION 3069

- ’01 Aspect Rstio .1.4 / )● MJ .- .- -.- 2.0 / – -4.0 / \ m / m 1 —slope = 1 / ~.-.------1

F3~ 90 Window Orientation (0)

FIGURE 9. The biased componentgrating directions and speeds predicted by the cross-correlationcalculation are shownfor three aspect ratios: 1.4 (dotted lines), 2.0 (dot-dashed lines), and 4.0 (dashed lines). (a) The predicted biases in the grating direction of motion are plotted as a function of the window orientation. The thick diagonal line with slope equal to one represents the predictionthat the perceivedmotionis in the directionof the window’sorientation.(b) Thepredictedbiases in the grating speed are plotted as a functionof the windoworientation.The solid line associatedwith the predictionfor each aspect ratio, is the speed definedby the grating’sconstraintline and its directionbias. It is computedas VJCOS(L9B)where OBis the bias in the perceived grating direction and VOis the unbiased grating speed (temporal frequency/spatialfrequency).

dimensional pattern from the ambiguous motion of its If the components are perceived unbiased, then the one-dimensional component gratings (Fig. 1). Models IOC rule prediction is that there will be no bias in the implementingthe IOC rule have two stages: a first stage perceived direction of plaid motion. To obtain a plaid measuresthe grating motion;and a second stage uses the bias, there mustbe a componentbias in at least one of the constraints provided by the gratings to compute the three required measurements. Furthermore, if the per- resultant velocity. ceived directionand speedof each componentgratingare In principle, to determine the constraintfor each com- “co-biased” such that the resultingcomponentvelocities ponent, IOC models require three pieces of information: remain consistentwith the unbiasedconstraintlines, and component orientation, speed, and direction of motion. if the perceivedorientationof the componentsis unbiased Each constraint line’s orientation is parallel to the then the IOC rule predicts no bias. In this case, the component’s orientation and its location is determined computation recovers the original constraint lines and by the component’s speed and direction. To generate a from these computes an unbiased plaid velocity. Other full set of exact IOC predictions from the biases in the types of componentbiases will however produce biases perception of the component gratings, therefore would in plaid direction. require explicit psychophysicalmeasurementsof each of To determinethe potentialeffects of componentbiases these three quantities,for all of the conditionsused in our on perceived plaid motion, it is necessary to know the study. However, even without these data, general biases in component speed, direction, and orientation. conclusionsabout models of this type can still be made Unfortunately only one of these, the direction bias for by examining several possible scenarios. gratingsin elongatedwindows,has been examined,and it 3070 B. R. BEUTTERet al,

60 IOC Predictions — ● 40 I Data 20- ● m. 0 A v -20- -40- Aspect Ratio 4.0 -60 1 1 1 I I (

15, lo- ]1 ~ 5- .; o T T m .5-

-1o- ~1 Aspect Ratio 2.0

15 101 5- 0- -5- ● -10 Aspect Ratio 1.4 i -ls~ 90 Window Orientation (0)

FIGURE10.The biases in plaid directionpredictedby the IOC model,which ignoresthe componentorientationand uses only the biased componentvelocities, are plottedas a functionof windoworientation.Predictionsare shownas the solid lines for the three aspect ratios: 1.4, 2.0, 4.0 (note the larger vertical range in the panel for aspect ratio of 4.0). For comparison,the filled circles show the average psychophysicaldata. The error bars represent the standard deviationover the two observers.

has been measuredonly in a limitednumberof conditions one assumes that the perceived orientations of the (Mulligan, 1991; Mulligan & Beutter, 1994). The gratings are unaffected by the window, the biases in dependence of the grating biases on the window aspect plaid direction predicted from these biased grating ratio is qualitativelysimilar to the predictionsof a cross- velocities are small [Fig. 7 and Fig. 8(a) dotted lines]. correlation model, except that the measured biases Artother possibility is that the motion-processing decrease as spatial frequency increases while those system uses only the component directions and speeds predicted by cross-correlationare independentof spatial as inputs to an IOC computationthat implicitly assumes frequency. Therefore, we have used a cross-correlation that the component orientation is orthogonal to the model to estimate the component biases for our three perceived componentdirection.This is equivalentto the aspect ratios, but have not calculated the predictionsfor orientationbeing misperceived as being in the direction varying the component spatial frequency. perpendicular to the misperceived component direction We first computed the biased directionsand speeds of of motion.The predictedplaid biases for this type of rule gratings moving within elongatedwindows by maximiz- using the biased grating velocities from the cross- ing the stimuluscross-correlation[Eq. (5), with the plaid correlationcomputation(Fig. 9) are shown as solid lines replaced by a grating].The resultsare shown in Fig. 9 for in Fig. 10 for the three aspect ratios along with the several aspect ratios and a grating spatial frequency of averagepsychophysicalresults(filledcircles).While this 0.6 c/deg. Although the model predicts large biases in scenario does predict large biases in plaid direction, the both the component grating direction [Fig. 9(a)] and patternsof biasesdo not even agree qualitativelywith the speed [Fig. 9(b)], these component biases are always data. linked: the biased speed and direction remain approxi- Each of the above IOC predictions depends on the mately consistentwith the originalgratingconstraintline method used to determine the component speed and [Fig. 9(b) solid lines].Therefore, as pointedout above, if directionbiases. However, by focusing on the biases for APERTURESBIAS PERCEIVEDPLAID MOTION 3071 window orientationsof 45 deg, it is possible to examine tions of the IOC rule, and shown that none of them can IOC models independently of any of our previous predict our data: assumptionsabout componentbiases. Because our plaid components were always oriented *45 deg from the 1. If there is no bias in the perceived motion of the plaid direction,when the windoworientationwas 45 deg, componentgrating, there is no bias in the predicted one component was aligned with the major axis of the plaid direction. window,while the other componentwas alignedwith the 2, If the componentorientationsare perceived veridi- minor axis. When the components are aligned with the cally, and the perceived speed and directionof each window, they are perceived to move in the direction component grating are co-biased in manner con- perpendicular to their orientation, such that both their sistentwith its constraintline, there is no bias in the orientations and directions appear unbiased.* Therefore predicted plaid direction. any bias in the perceived directionof plaid motion could 3. If the componentorientationsare perceived veridi- cally, and the perceived speed and directionof each only be caused by misperceptions of the component component grating are those predicted by a cross- speeds. Our psychophysicaldata show that the average correlation model, the biases are smaller than our perceived biases for a window orientationof 45 deg and data. aspect ratios of 4.0, 2.0, and 1.4 are about 14.0, 8.2, and 4. If the perceived componentorientationsare ignored 4.4 deg, respectively. Using the IOC rule for perpendi- (or equivalently presumed orthogonal to the per- cular plaids, the relative speeds of the components ceived direction of motion), and the constraintsare necessary to produce these biases are tan(45-6bi~~),or computed by only using the component speed and 0.60,0.75, and 0.86, respectively.Althoughit is possible direction predicted by a cross-correlation model, that the elongated window may cause small biases in then the biases are qualitatively different than our perceived componentspeed, it is doubtfulthat this effect data. could produce the required 1440% change in relative 5, By examining the 45 deg window orientation case, speed. Because perceived grating speed depends on we showed that our measured biases cannot be contrast (Thompson, 1982; Stone & Thompson, 1992; predicted by any IOC computation using plausible Muller & Greenlee, 1994), it is possible that component biases in the perceived componentmotions. speed biases could result from small inequalities in the effective contrast of the components, that might be Therefore, the IOC model cannot explain our psycho- produced by the elongated window. However, any physical data. contrast effect produced by the window would be too small to be able to produce the large speed differences Vector-summodel required to explain our data: the contrast ratio necessary Wilson et al. (1992) have proposed a vector-sum to produce a -25% change in perceived relative speed is motion processing model (hereafter referred to as the -7 (Stone & Thompson, 1992). Biases in component vector-sum model), which successfully predicts human speed mightalso be causedby differencesin the distances perceptionof movingplaidsunder a variety of conditions traveled (Brown, 1931). But because shorter paths (Ferrera & Wilson, 1990; Wilson et al., 1992; Kim & produce faster perceived speeds, this cannot explain our Wilson, 1993;Wilson & Kim, 1994).Briefly,the vector- results, since it would produce plaid biases in the wrong sum model has a Fourier pathway and a non-Fourier direction (towards the short axis). pathway. The Fourier pathway has an initial filtering In summary, we have examined several implementa- stage whose output is passed through a Reichardt-like motion-energycomputationwhich is followedby a gain- *Resultsfrom Mulligan(1991)and Mulliganand Beutter (1994)show control mechanism. The non-Fourier pathway has an that in the case where the gratingis alignedwith the longaxis of the initial filtering stage whose output is squared, then window, the perceived direction of motion is unbiased. Informal processed by perpendicular filters which are tuned to observationsshow that the perceived direction is also unbiased in lower spatial frequencies, and then is passed through a the case where the grating is alignedwith the shortaxis, andthat, in both of these cases, the orientation appears unbiased. motion-energy mechanism followed by a gain-control ~The spatial receptive fields in the vector sum model are definedas mechanism. The output stage basically computes the average direction of the outputs of these two pathways ~@,Y]=~[exp(~)-Bexp(~)]e xp(~) weighted by the strengthsof their responses. We examined the predictionsof this model by using a The parameter values are a, = 0.098deg, 02= 0.294deg, A = software implementation provided by Wilson. This 1022.7,B =0.333. Instead of specifying LTYdirectly, the angular implementationis designedfor unwindowedplaid stimuli half-bandwidth at half height for the optimal spatial frequency, 1.7c/deg was reported to be 22.5deg. Usingthis,wecalculateda~ of infinitespatial extent.To apply the model to our finite to be 0.405deg. stimuli,we calculatedthe responseof its front-endfilter~ ~Both the filter responses and the resultant predicted directions of to the windowedcomponentgratingsof our stimuliand to motion depend on the position of the mechanisms relative to the identical unwindowed gratings. We computed the center of the stimulus window. Avoiding the problem of responsesby centeringthe receptivefieldon the stimulus determining how these different directions and speeds are integrated into a single percept, we have only calculated the window and then integrating the stimulus weighted by predictionsof a mechanismcentered on the stimulus. the receptive-field filter over all space.$ Each compo- 3072 B. R. BEUTTERet al. nent’s effective contrastwas then calculated as its actual We emphasize that the model’s filter parameters are contrast weighted by the ratio of the windowed-grating not arbitrary:they were empiricallymeasured in a series response to the unwindowed-gratingresponse. We ran of experiments (Wilson & Bergen, 1979; Wilson et al., the simulationsusingtheseeffectivecomponentcontrasts 1983; Phillips & Wilson, 1984; Wilson & Gelb, 1984) as input, in place of the original contrast values.* The and used to model the psychophysicalresults of many predicted biases for the 0.6 c/deg spatial frequency plaid experiments(Ferrera& Wilson, 1990;Wilsonet al., components and an aspect ratio of 2.0 are plotted as a 1992; Kim & Wilson, 1993; Wilson & Kim, 1994). function of the window angle in Fig. 7 as the dot-dashes. Nonethelesswe tunedthe modelparametersin an attempt There are two major differences between the model to improve the match between the predictions and our predictions and the data. The predictions are both too data. Because, as discussedabove, the filtershape causes small and of the wrong sign; instead of being biased the predicted biases to be in the direction opposite to toward the long axis of the window, the predicted biases those of our data, we examinedthe effects of varying the are away from it. A similar pattern of resultsis predicted filter’s height (oY).Experiments using drifting gratings for other aspect ratios. The peak predicted biases are (Anderson& Burr, 1991;Anderson et al., 1991;Watson plotted as a function of aspect ratio in Fig. 8(a). The & Turano, 1995) have shown that the psychophysically negative values indicate that the biases are in the wrong measured receptive fields for moving stimuli have direction. approximately equal height and width. We therefore The vector-sum model predicts biases opposite those reduced the input filter’s height so that it was approxi- of our data because of the shapeof its inputspatialfilters. mately equal to its width, and found that the predicted To understand why this is so, it is again instructive to biases were still in the wrong direction, but they were examine the case in which the window is oriented at smaller. This modification of the filter also had the 45 deg and the componentsmove in the directionsof the undesirable effect of increasing the filter’s angular long and short axes of the window. The Fourier pathway bandwidthfrom 45 to 90 deg.To determineif an extreme responses contain two peaks of unequal magnitude modificationof the filter might improve the predictions, because of the unequal effective component contrasts. we reduced the filter’s height by an additionalfactor of The perhaps counter-intuitiveresult is that the responses 10. This manipulationdid produce biases in the correct to the grating moving in the direction of the short axis of direction,but the biaseswere always c 1 deg, an order of the window are larger than thoseto the gratingmovingin magnitudesmaller than our data. We conclude that even the direction of the long axis of the window. This is with drastic changes in its input filter shape, the vector- caused by the shape of the input spatial filter: its spatial sum model cannot be modifiedto predict our data. extent is larger in the direction perpendicular to the preferred direction of motion (a Gaussian with standard DISCUSSION deviation 0.405 deg) than it is in the parallel direction (a difference of Gaussians whose standard deviations are Our resultsshow that the perceiveddirectionof a plaid 0.098 deg and 0.294 deg). Thus, since the predicted movingwithin an elongatedwindow is biased toward the directionof motion is the weighted averageof the grating long axisof the window.The bias increasesas the relative responses,it is biased in the wrong direction,toward the anglebetween the plaid directionand the long axis of the short axis of the window. The effect of the non-Fourier window increases, peaks at a relative angle of -45 deg, pathway is small: it reduces the amount of the bias and then decreasestoward zero at 90 deg (when the plaid slightly. Because our stimuli are of equal spatial and direction is aligned with the short axis of the window). temporal frequencies, the result of the non-Fourier This pattern of results was observed for all window squaring stage is a grating moving in the direction of aspect ratios and component spatial frequencies tested. plaid motion. (The squaring process produces gratings The magnitudeof the bias increasesas the windowaspect with frequencies that are the sum and difference of the ratio increases and decreases as the plaid spatial component gratings’ spatio-temporal frequencies. The frequency increases (or equivalently as the number of sum moves in the plaid direction and the difference is visible grating cycles increases).Althoughsimilar trends stationary.) This produces non-Fourier pathway re- are present in the predictions of the motion-energy sponses which are centered around the true plaid model, the cross-correlation model, and one of the direction of motion. When included in the averaging modified IOC models, the magnitudes of the biases they merely reduce the magnitude of the biases, but the predicted by each of these models are much too small to biases are still always in the wrong direction. account for our data. The predicted biases of the vector- sum model are in the wrong direction. Thus, none of ‘our component gratings were 0.6 c/deg. Although the optimal these models can predict our psychophysicalresults. frequency of the front end filters is 1.7c/deg, documentation Each of the abovemodelsrespondsto the “first-order” providedby Wilson states that the model operates well for spatial motion of the stimulus. Although our plaids are “first- frequencies between 0.5 and 1.7c/deg. To test whether the order” stimuli and provide strong signals to each of the predictions depended critically on the plaid spatial frequency, we above models, it is possible that higher-order mechan- restimulatedthe modelwith ourstimulusspatiallyresealed (boththe window and the plaid) so that its spatial frequency was 1.7c/deg. isms, such as feature tracking, might also contribute to The biases and trends we found for this 1.7c/deg stimuhrs were the percept [e.g. Lu & Sperling(1995)].Possiblefeatures similar to the results for our actual 0.6 c/deg stimuhrs. of our plaids include the bright or dark blobs and the APERTURESBIAS PERCEIVEDPLAID MOTION 3073 interblob regions. Because the position of these features trials, if observers perceive a partially coherent or even is only slightlyaffectedby the windowshape,any feature possiblyincoherentstimulusand they are forced to make tracking system will likely predict negligible biases. At a directionjudgment, they may respond to the motion of present,no precise feature-trackingmodelexistsfor us to the components.If these responsesare intermingledwith test explicitly, so quantitative estimation of the biases those to trials in which the plaid is perceptually more must await a specific proposal for how feature-tracking coherent, a potential problem arises in interpreting the might be implemented. Nonetheless, it is doubtful that resultant psychometric data. The responses to the any such feature tracking system would generate biases partially coherent trials will produce a bias toward the large enough to explain our data. Furthermore,all of the components, and also cause an increase in response models that we have examined predict biases which are variability which will result in a higher threshold [see too small. Therefore, we can also rule out strategies footnotep. 1061 of Stone et al. (1990)].This worrisome which use combinationsof these processes,because they possibility cannot be distinguished from an actual also would predictbiaseswhich are smallerthan our data. increase in threshold and bias in perceived direction of A number of previousstudieshave shown that varying motion.Thus it is impossibleto rule out this explanation factorssuch as contrast,spatialfrequency,and adaptation of Ferrera and Wilson’s Type II plaid experiments, in state can produce significant biases in the perceived which they find precisely this, both an increase in directionof plaid motion.If the componentgratingshave threshold (-6.5 deg for Type II compared to -1 deg for differentcontrasts,the directionof plaid motion is biased symmetric Type I) and a bias toward the components’ toward the direction of motion of the higher contrast directions (-.7.5 deg for Type 11compared to -O deg for grating (Stone et al., 1990; Kooi et al., 1992). If the symmetricType I). component gratings have different spatial frequencies, To avoid the problem of coherence, we chose to use the direction of plaid motion is biased toward the Type I symmetric, 90 deg, equal spatial and temporal direction of motion of the grating of lower spatial frequency plaids, because they are known to cohere frequency (Smith & Edgar, 1991; Kooi et al., 1992). IAdelson & Movshon (1982); Welch & Bowne (1990); Derrington and Suero (1991) reported that adaptation to Smith & Edgar (1991); Smith (1992); Victor & Conte the direction of one of the components biased the (1992); Kim & Wilson (1993); see however, Farid & perceived direction of plaid motion toward the direction Simoncelli(1994)].Becauseof its simplicity,this type of of the other component.In these cases, the biases cannot plaid provides an extremely direct test of the basic be predicted by cross-correlation models, which are principles of models. The components are identical largely insensitive to these manipulations. However, except for their orientations, and thus issues of the these types of biases are not inconsistent with the interaction of different spatial and temporal frequencies predictions of modified motion-energy models or the are avoided.Additionally,the use of orthogonalcompo- vector-summodel.These data might also be predictedby nents minimizescross-orientationinteractions.When the an IOC model operating on misperceived component window is circularly symmetric,predicting its perceived speeds,if the inputspeedsare modifiedto incorporatethe directionof motionis particularlyeasy, almostany model known effectsof contrast [e.g.Thompson(1982)],spatial (IOC, vector-sum, motion-energy, cross-correlation, frequency [e.g. Smith & Edgar (1990)], and adaptation even average direction) gets it right, but if the window [e.g. Thompson (1981); Muller & Greenlee (1994)]. is elongated, all of the models fail. Our data therefore Thus, these studies do not clearly distinguish between provide a strong challenge to models of human motion many of the leading motion processing models. perception,and suggestthat there may be a basicproblem A different and perhaps more fundamental type of in the way in which all of these models calculate the plaid direction misperception was reported by Ferrera perceived velocity of moving patterns. and Wilson (1990). They examined the perceived How then might the human brain estimate pattern direction of motion of Type II plaids, in which the plaid velocity? The physiology and anatomy suggest a more directionof motionliesoutsidethe componentdirections. elaborate approach in which spatial integration plays a They found directionbiases toward the components.This key role. The stimulusis first processedby neuronswith is not predictedby simplemotion-energyor IOC models, relativelylocal receptivefieldswhich are orientationand but is predicted by the vector-sum model. However, spatial-frequencytuned (De Valois et al., 1982a,b). At interpretingthese results is problematic.The appearance the next stage, motion is analyzed over more extended of these plaids is different than that of symmetrictype I regions (Maunsell & Van Essen, 1983b;Albright, 1984; plaids, and at times Type II plaids appear not to move Mikami et al., 1986b) and across a range of spatial coherently. Because of this, Ferrera & Wilson (1987) frequencies (Movshon et al., 1988) by neurons with originally called Type II plaids, “blobs”, and noted that larger receptive fields. Thus, it appears that motion “(t)he motion of blobs does not always appear to be information is first analyzed locally. These results are absolutely rigid, which might be taken to imply that then combinedacrossspaceand spatialscale to arriveat a coherence is not an all-or-none phenomenon, but that unified global percept. In support of this type of there may be cases of partial coherence” (p. 1788).This processing, Kooi (1993) has shown that small local lack of coherencepresentsa problem for observerswhen changes in a grating barberpole stimuluscan change the asked to make a single direction judgment. On some way in which the aperture problem is resolved. He 3B. R. BEUTTER et 0al.

showed that adding small indentationsin the border of a revealed through distance-driven reversal of apparent motion. rectangularapertureproduceslarge changesin the global Proceedings of the National Academy of Sciences, 86, 2985–2989. percept. To determine if spatial integration of local Derrington, A. M. & Badcock, D. R, (1992). Two-stage analysis of the motion of 2-dimensional patterns, what is the first stage? Vision motion signals might also predict our plaid biases, we Research, 32, 691-98. modified the cross-correlation and IOC models to only Derrington, A. M. & Suero, M. (1991). Motion of complex patterns is look at local stimulus patches. Not unexpectedly, we computed from the perceived motions of their components. Vision foundthat both the predicteddirectionsand speedsvaried Research, 31, 139–149. significantly across space. Generally smaller direction De Valois, R. L., Albrecht, D. G. & Thorell, L. S. (1982a). Spatial frequency selectivity of cells in macaque visual cortex. Vision and speed biases were predicted near the stimuluscenter Research, 22, 545–559. and larger biases near the edges. To produce a unified De Valois, R. L., Yund, E. W. & Hepler, N. (1982b). The orientation percept of a drifting plaid, these local variationsmust be and direction sensitivity of cells in macaque visual cortex. Vision combined. Clearly the resultant pattern direction will Research, 22, 531–544. depend on how this spatial integrationis achieved.None De Yoe, E. A. & Van Essen, D. C. (1985). Segregation of efferent of the models we examined address this issue. As in connections and receptive field properties in visual area V2 of the macaque. Nature, 32, 58-61. previous studies [e.g. Stone et al. (1990); Wilson et al. Farid, H. & Simoncelli, E. P. (1994). The perception of transparency in (1992)], our simulations were either global, a single moving square-wave plaids. Investigative Ophthalmology and measure across the whole visual field, or local, a single Visual Science, 35(Suppl.), 1271. measure of the output of a sensor centered on the Fennema, C. L. & Thompson, W. B. (1979). Velocity determination in stimulus.However,expandingexistingmodelsto include scenes containing several moving objects. Computer Graphics and Image Processing, 9, 301–315. an explicit spatial-integration rule may allow them to Ferrera, V. P. & Wilson, H. R. (1987). Direction specific masking and explain our results. In other words, it is not that we have the analysis of motion in two dimensions. Vision Research, 27, shown that the algorithmstested above do not play a role 1783-1796. in human motion processing but rather that, if they do, Ferrera, V. P. & Wilson, H. R. (1990). Perceived direction of moving they will need to incorporate spatial integration before two-dimensional patterns. Vision Research, 30, 273–287. they can be used to predict human performance. Thus, Heeger, D. J. (1987). Model for the extraction of image flow. Journal of the Optical Society of America A, 4, 1455–1471. our results show that even for a simple plaid stimulus, Hubel, D. H. & Wiesel, T. N. (1968). Receptive fields and functional integration of motion signals across space may play a architecture of monkey striate cortex. Journal of Physiology, 195, critical role in determining the perception of motion. 215-243. Kim, J. & Wilson, H. R. (1993). Dependence of plaid motion coherence on component grating directions. Vision Research, 33, REFERENCES 2479–2489. Adelson, E. H. & Bergen, J. R. (1985). Spatiotempoml energy models Kooi, F. L. (1993). Local direction of edge motion causes and for the perception of motion. Journal of the Optica/ Society of abolishes the barberpole illusion. Vision Research, 33, 1347–1351. America A, 2, 284–299. Kooi, F. L., De Valois, K. K,, Grosof, D. A. & De Valois, R. L. (1992). Adelson, E. H. & Movshon, J. A. (1982). Phenomenal coherence of Properties of the recombination of one-dimensional motion signals moving visual patterns. Nature, 300, 523–525. into a pattern motion signal. Perception and Psychophysics, 52, . Albright, T, D. (1984). Direction and orientation selectivity of neurons 415424. in visual area MT of the macaque. Journal of Neurophysiology, 52, Leese, J. A., Novak, C. S. & Taylor, V. R. (1970). The determination of 11061130. cloud pattern motion from geosynchronous satellite image data. Albright, T. D., Desimone, R. & Gross, C. G. (1984). Columnar Pattern Recognition, 2, 279–292. organization of directionally selective cells in visual area MT of the Lu, Z. L. & Sperling, G. (1995). The functional architecture of human macaque. Journal of Neurophysiology 51, 16–31. visual motion perception. Vision Research, 35, 2697–2722. Anderson, S. J. & Burr, D. R. (1991). Spatial summation properties of Maunsell, J. H. R. & Van Essen, D. C. (1983a). The connections of the directionally selective mechanisms in human vision. Journal of the middle temporal area (MT) and their relationship to cortical Optical Socie~ of Anrerica A, 8, 1330-1339. hierarchy in the macaque monkey. Journal of Neuroscience, 3, Anderson, S. J., Burr, D. R. & Morrone, M. C. (1991). Two- 2563–2586. dimensional spatial and spatial-frequency selectivity of motion- Maunsell, J. H. R. & Van Essen, D. C. (1983b). Functional properties sensitive mechanisms in human vision. Journal of the Optical of neurons in the middle temporal visual of the macaque monkey. 1. Socie& of America A, 8, 1340-1351. Selectivity for stimulus direction, speed, and orientation. Journal of Beutter, B. R., Mulligan, J. B. & Stone, L. S. (1994a). The barberplaid Neurophysiology, 49, 1127–1 147. illusion.Investigative Ophthalmology and Visual Science, Mikami, A., Newsome, W. T. & Wurtz, R. H. (1986a). Motion 35(Suppl.), 2157. selectivity in macaque visual cortex. I. Mechanisms of direction and Beutter, B. R., Mulligan, J. B. & Stone, L. S. (1994b), Direction of speed selectivity in extrastriate area MT. Journal of Neuro- moving plaids is biased by asymmetric viewing windows. Society physiology, 55, 1308-1327. for Neuroscience Abstracts, 20, 772. Mikami, A., Newsome, W. T. & Wurtz, R. H. (1986b). Motion Britten, K. H., Shadlen, M. N., Newsome, W. & Movshon, J. A. selectivity in macaque visual cortex. II. Spatiotemporal range of (1993). Response of neurons in macaque MT to stochastic motion directional interactions in MT and VI. Journal of Neurophysiology, signals. Visual Neuroscience, IO, 1157–1169. 55, 132t+ 1339. Brown, J. F. (1931). The visual perception of velocity. Psychologische Movshon, J. A., Adelson, E. H., Gizzi, M. S. & Newsome, W. T. Forshungen, 14, 199–232. (1986). The analysis of moving visual patterns. Experiment/ Brain Bulthoff, H., Little, J. & Poggio, T. (1989). A parallel algorithm for Research, ll(Suppl.), 117–151. real-time computation of optical flow. Nature, 337, 549–553. Movshon, J. A. & Newsome, W. T. (1984). Functional characteristics Chubb, C. & Sperling, G. (1988). Drift-balanced random stimuli: A of striate cortical neurons projecting to MT in the macaque. Society general basis for studying non-Fourier motion perception. Journal of for Neuroscience Abstracts, IO, 983. the Optical Socie~ of America A, 5, 19862007. Movshon, J. A., Newsome, W. T., Gizzi, M. S. & Levitt, J. B. (1988). Chubb, C. & Sperling, G. (1989). Two-motion perception mechanisms Spatio-temporal tuning and speed sensitivity in macaque visual APERTURESBIAS PERCEIVEDPLAID MOTION 3075

cortical neurons. Investigative Ophthalmologyand VisualScience, to moving stimuli on the perception of subsequentlyseen stimuli. 29(Suppl.),327. VisionResearch, 21, 337–345. Muller,R. & Greenlee, M. W. (1994).Effect of contrastand adaptation Thompson, P. (1982). Perceived rate of movement depends on on the perception of the direction and speed of drifting gratings. contrast. VisionResearch, 22, 377–380. VisionResearch, 34, 2071–2092. Ungerleider, L. G. & Desimone, R. (1986). Cortical connections of Mulligan, J. B. (1986). Minimizing quantization errors in digitally area MT in the macaque.Journal of ComparativeNeurology,248, controlledCRT displays.ColorResearch andApplication,11, S47– 190-222. S51. Ungerleider, L. G., Desimone, R., Galkin, T. W. & Mishkin, M. Mulligan, J. B. (1991). A continuous version of the barber-pole (1984).Subcorticalprojectionsof area MT in the macaque.Journal illusion. Investigative Ophthalmology and Visual Science, of ComparativeNeurology, 223, 368–386. 32(SuppL),829. Victor, J. D. & Conte, M. M. (1992). Coherence and transparencyof Mulligan, J. B. & Beutter, B. R. (1994). Additional twists on a moving plaids composed of Fourier and non-Fourier gratings. continuous barber-pole illusion. Investigative Opthamo[ogy and Perception and Psychophysics,52, 403414. VisualScience, 35(Suppl.), 2157. Wallach, H. (1935). Uber visuell wahrgenommene Bewegungs- Mulligan, J. B. & Stone, L. S. (1989). Halftoning method for the richtung.PsychologischeForschungen,20, 325–380. generation of motion stimuli. Journal of the Optical Sociery of Watson, A. B. & Ahumada, A. J. (1983). A look at motion in the America A, 6, 1217–1227. frequencydomain. In Motion: Perception and representation.New Newsome, W. T., Mikami, A. & Wurtz, R. H. (1986). Motion York: Association for Computing Machinery (also published as selectivity in macaque visual cortex. III. Psychophysics and NASATM-84355). physiology of apparent motion. Journal of Neurophysiology, 55, Watson, A. B. & Ahumada, A. J. (1985). Model of human visual- 1340-1351. motionsensing.Journal of the OpticalSociety ofAmerica A, 2, 322– Phillips, G. C. & Wilson, H. R. (1984). Orientation bandwidths of 341. spatial mechanisms measured by masking.Journal of the Optical Watson,A. B., Thompson,P. G., Murphy,B. J. & Nachmias,J. (1980). Society of America A, 1,226-232. Summation and discrimination of gratings moving in opposite Power, R. P. & Mouldon,B. (1992). Spatial gating effects on judged directions. VisionResearch, 20, 341–347. motion of gratings in apertures. Perception, 21, 449-463. Watson, A. B. & Turano, K. (1995). The optimal motion stimulus. Rodman, H. R. & Albright, T. D. (1989). Single-uqit analysis of VisionResearch, 35, 325–336. pattern-motion selective properties in the middle temporal visual Welch, L. (1989). The perception of moving plaids reveals two area (MT). ExperimentalBrain Research, 75, 53-64. motion-processingstages. Nature, 337, 734736. Rodman,H. R., Gross, C. G. & Albright,T. D. (1989).Afferent basis Welch, L. & Bowne, S. F. (1990). Coherence determines speed of visual responsepropertiesin area MTof the macaque.I. Effectsof discrimination.Perception, 19, 425435. striate cortex removal.Journal of Neuroscience, 9, 2033–2050. Wilson, H. R. & Bergen, J. R. (1979). A four-mechanismmodel for Rodman,H. R., Gross, C. G. & Albright,T. D. (1990).Afferent basis thresholdspatial vision. VisionResearch, 19, 19–32. of visual response properties in area MT of the macaque. IL Effects Wilson, H. R., Ferrera, V. P. & Yo, C. (1992). A psychophysically of superior colliculus removal.Journal of Neuroscience, 10, 1154 motivated model for two-dimensional motion perception. Visual 1164. Neuroscience, 9, 79–97. van Santen,J. P. H. & Sperling,G. (1984).Temporalcovariancemodel Wilson, H. R. & Gelb, D. J. (1984). Modifiedline-element theory for of human motion perception. Journal of the Optical Sociery of spatial-frequencyand width discrimination.Journal of the Optical America A, 1, 451A73. Society of America A, 1, 124-131. van Santen, J. P. H. & Sperling, G. (1985). Elaborated Reichardt Wilson, H. R. & Kim, J. (1994). Perceived motion in the vector sum detectors.Journal of the OpticalSociety of America A, 2,300-320. direction. VisionResearch, 34, 1835-1842. Smith, A. T. (1992). Coherence of plaids comprising componentsof Wilson, H. R., McFarLane, D. K. & Phillips, G. C. (1983). Spatial- disparate spatial frequencies. VisionResearch, 32, 393–397. frequencytuningof orientationselective units estimated by oblique Smith,A. T. & Edgar,G. K. (1990).The influenceof spatial frequency masking. VisionResearch, 23, 873-882. on perceived temporal frequency and perceived speed. Vision Zeki, S. M. (1974). Functional organization of a visual area in the Research, 30, 1467–1474. posteriorbank of the superiortemporalSUICUSof the rhesusmonkey. Smith, A. T. & Edgar, G. K. (1991).Perceived speed and direction of Journal of Physiology (London),236, 546-573. complex gratings and plaids. Journal of the Optical Socie~ of America A, 8, 1161-1171. Stone, L. S. (1990). Precision in the perception of plaid motion. Acknowledgements—Theauthorswouldlike to thankHughWilsonfor Investigative Ophthalmologyand VisualScience, 31(Supp1.),172. kindly providing a MATLABm implementation of the vector-sum Stone, L. S. & Thompson, P. (1992). Human speed perception is motion model (Wilson et al., 1992). We would also like to thank contrast dependent. VisionResearch, 32, 1535–1549. Andrew Watson and Preeti Verghese for their helpful comments on Stone,L. S., Watson,A. B. & Mulligan,J. B. (1990).Effect of contrast earlier drafts and the NASA Ames Vision Group for their general on the perceived direction of a moving plaid. VisionResearch, 30, support.This work was supportedby a NASARTOP 199-16-12-37to 1049–1067. LS, and by a National Research Councilpostdoctoralassociateshipto Thompson,P. (1981).Velocity after-effects: The effects of adaptation BB.