Cognitive Load Effects on Early Visual Perceptual Processing

Attention, Perception, & Psychophysics https://doi.org/10.3758/s13414-017-1464-9

Cognitive load eﬀects on early visual perceptual processing

Ping Liu1 · Jason Forte1 · David Sewell2 · Olivia Carter1

Abstract Contrast-based early visual processing has largely been considered to involve autonomous processes that do not need the support of cognitive resources. However, as spatial attention is known to modulate early visual perceptual processing, we explored whether cognitive load could similarly impact contrast-based perception. We used a dual-task paradigm to assess the impact of a concurrent working memory task on the performance of three different early visual tasks. The results from Experiment 1 suggest that cognitive load can modulate early visual processing. No effects of cognitive load were seen in Experiments 2 or 3. Together, the findings provide evidence that under some circumstances cognitive load effects can penetrate the early stages of visual processing and that higher cognitive function and early perceptual processing may not be as independent as was once thought.

Keywords Cognitive load · Early vision · Dual-task · Spatial attention · Working memory

Introduction low-level visual perceptual mechanisms. The second stream focuses on understanding to what extent cognitive load may At any given moment, our brain is overwhelmed by affect early visual processing when there is no content incoming information from our sensory environment. At overlap between the two (de Fockert et al., 2001;Lavie,2005, the same time, our behavioral goals and the execution of 2010). The current study falls into the second category. actions need to be maintained. The ability of the brain Top-down attention mechanisms supporting perceptual to coordinate concurrent perceptual processing and higher information processing have been the subject of countless cognitive functions is crucial for us to behave in a coherent studies (Carrasco, 2011;Chenetal.,2014; Crist et al., 2001; and efficient manner in daily life. Gilbert & Li, 2013;Lietal.,2004, 2006). Visual attention The influence of cognitive processes on concurrent visual can be selectively directed to different visual properties such perceptual processing has been mainly explored in two as location, color, etc. The majority of such studies have seemingly related but independent literature streams. One looked at how visual spatial attention facilitates the pro- stream focuses on how working memory content can bias cessing of attended information and suppresses unattended concurrent visual processing when there is a content overlap information (Carrasco, 2011; Desimone & Duncan, 1995) between the two (Kosslyn et al., 1999; Scocchia et al., 2013; Whether spatial attention modulates early visual process- Serences et al., 2009). This line of research has provided ing was difficult to prove for more than two decades due to evidence for strong links between cognitive processes and the variety of visual tasks and methodologies employed in spatial attention studies (Carrasco, 2011; Zhaoping, 2014). The flow of visual perceptual processing is believed to Olivia Carter follow an approximately hierarchic feedforward path, i.e., [email protected] from early to high level vision. Each stage is associated Ping Liu [email protected] with its specific category of tasks that have been developed to rigorously assess the relevant level of visual process- 1 Melbourne School of Psychological Sciences, The University ing (Marr, 1982; Zhaoping, 2014). Contrast sensitivity tasks of Melbourne, Parkville, Australia are generally considered to assess early processing stages. 2 School of Psychology, The University of Queensland, Demonstrating spatial attention effects on early visual pro- St Lucia, Australia cessing with only behavioral measures has required rigorous Atten Percept Psychophys control of stimulus configuration and experimental method- Strasburger, 2005; Van der Lubbe & Keuss, 2001). The ology (Dosher & Lu, 2000; Herrmann et al., 2010;Lu& finding and design differences in the research raise the Dosher, 1998; Pestilli et al., 2011). For example, the tar- crucial question as to whether cognitive load can indeed get has to be presented alone free from any distractors and modulate early visual processing. external noise (Dosher & Lu, 2000; Lu & Dosher, 1998; The center-surround antagonistic organization of the Pestilli et al., 2011) and the stimulus size of the target needs receptive field of early visual neurons is thought to be to be carefully controlled in relation to the spatial attention fundamental to optimal contrast-based visual information distribution (Herrmann et al., 2010). processing. The center excitatory drive to the classical The majority of cognitive load studies, however, have receptive field (CRF) establishes a neuron’s basic stimulus not made a clear distinction between the visual tasks selectivity, which can be strongly modulated by the used to assess cognitive load on early versus high level surround inhibition from the extra-classical receptive field visual processing and this has led to some discrepancies (eCRF) in many neurons along the visual pathway (Adelson in the interpretations of results obtained. For example, & Bergen, 1991; Fujita et al., 1992; Hubel & Wiesel, 1962, findings from a class of studies employing flanker tasks 1965). This center-surround interaction has been proposed have been interpreted as suggesting that cognitive load to be one of the most fundamental underlying mechanisms doesn’t modulate early visual processing (de Fockert supporting the efficient encoding of raw visual inputs et al., 2001;Lavie,2005). Flanker tasks represent an (Heeger, 1992; Marr, 1976; Zhaoping, 2014). experimental paradigm known to be more closely associated Neurophysiological findings of top-down modulation with a higher-level visual mechanism, i.e., visual crowding effects on center excitation and surround inhibition suggest (Dayan & Solomon, 2010;Levi,2008;Levietal.,2002; that variations in top-down modulation strength lead to Strasburger, 2005). While these studies are interesting and differential effects on the final output of neural responses informative they cannot be used to rule out an impact of in early visual cortical neurons (Hupe et al., 1998, 2001; cognitive load on early visual processing. Nassi et al., 2013; Sandell & Schiller, 1982;Wangetal., Recently a few studies using early visual tasks have 2010). Specifically, inactivation of feedback to V1 neurons provided some initial indication that such tasks may be has been found to reduce responses in some neurons to sensitive to cognitive load. Cocchi et al. (2011) reported low-contrast stimuli confined to the CRF, suggesting that an unexpected finding that visual spatial working memory cortico-cortical feedback provides a weak, predominantly loads facilitated the performance of a concurrent but excitatory influence on the CRF (Hupe et al., 1998, 2001; independent visual grouping-by-proximity task. Similarly, Sandell & Schiller, 1982;Wangetal.,2010). In contrast, de Fockert and Leiser (2014) showed that high cognitive when assessed using stimuli that engage both the CRF and load enhanced collinear facilitation, which is an established eCRF, eliminating feedback results in strong and consistent early visual perceptual mechanism. The “facilitative” response facilitation, effectively reducing the strength of effects reported in Cocchi et al. (2011) and de Fockert and surround inhibition on center excitation in V1 neurons Leiser (2014) are at odds with the existing research (i.e., (Angelucci et al., 2002; Angelucci & Bullier, 2003; Nassi cognitive load and other dual-task studies) that suggests et al., 2013). Thus, theoretically in the presence of both cognitive load has no impact on concurrent early visual center excitation and surround inhibition, the final outputs processing (Pashler, 1994; de Fockert et al., 2001;Lavie, reflect the balance between these two forces in the absence 2005). However, the grouping-by-proximity task in Cocchi of spatial attention. et al. (2011) and the collinear facilitation task in de Fockert Spatial attention has been argued to shift the balance and Leiser (2014) differ considerably from the flanker between center excitation and surround inhibition, which tasks employed in cognitive load studies. Firstly, both the in turn alters the neural response to visual stimulation. grouping and the collinear facilitation tasks are generally The modulation effects have been characterized by many considered early visual tasks whereas flanker tasks are computational models (Cutzu & Tsotsos, 2003; Pestilli considered a high-level vision task. Secondly, there is et al., 2011; Reynolds & Heeger, 2009). For example, literature suggesting that the grouping and the collinear according to the normalization model of spatial attention facilitation tasks are facilitated by a more distributed visual (Reynolds & Heeger, 2009), the size of the attentional field spatial attention field (Ben-Av et al., 1992; Casco et al., determines how much surround inhibition enters into the 2005; Freeman et al., 2001, 2003; Han et al., 2005a, b; normalization process and, consequently, the final response Ito & Gilbert, 1999; Mack et al., 1992). In contrast, a intensity of a given neuron (Reynolds & Heeger, 2009). focused spatial attention field has been shown to improve The aim of the current study is to explore cognitive load performance on flanker tasks (Chen et al., 2014;Fang& effects on early visual processing. Given center excitation He, 2008; Harrison et al., 2013;Heetal.,1996; Motter, and surround inhibition are the fundamental contrast-based 1993; Petrov & Meleshkevich, 2011; Scolari et al., 2007; early visual processing mechanisms, the current study Atten Percept Psychophys explored cognitive load effects on center excitation and studies (Carrasco et al., 2000; Liu et al., 2009; Skottun et al., surround inhibition separately with established early vision 1987; Smith & Wolfgang, 2004). While it is acknowledged tasks in three experiments. Spatial attention effects were that orientation discrimination is generally regarded as taken into account in the design of the experiments and requiring high-level visual processing (Zhaoping, 2014), the interpretation of results because of its possible modulation processing demand on orientation discrimination in this task effects on the interaction between the two forces. is, however, minimal. A peripheral contrast detection task was employed because it is thought to recruit distributed spatial attention. Cognitive load has been shown to defocus Experiment 1 - Cognitive load eﬀects spatial attention when focused spatial attention is required on center excitation (Caparos & Linnell, 2010; Linnell & Caparos, 2011). By using an early visual task that requires a distributed spatial Converging evidence from psychophysical, neurophysio- attention, the design minimized the chance that cognitive logical, and imaging studies suggests that top-down modu- load effects on center excitation could be confounded by its lation enhances neural response to visual stimulation when effects on altering spatial attention distribution. the dominant driver of the neural response reflects the center Cognitive load was manipulated with a general alphanu- excitation mechanism. Findings from psychophysical stud- meric working memory task similar to that used previously ies of spatial attention suggest that the effects of spatial (de Fockert et al., 2001), in which observers held zero, one attention are equivalent to increasing the contrast of weak or five alphanumeric characters in working memory. If cog- stimuli when the target stimulus is small relative to the spa- nitive load reduces top-down modulation on early visual tial attention field (Herrmann et al., 2010; Ling & Carrasco, processing, our high working memory load condition should 2006; Pestilli et al., 2009). Neurophysiological and imaging result in a relative elevation of contrast detection thresholds. studies of spatial attention have also found spatial attention effects are equivalent to increasing stimulus contrast for Methods small stimuli (Li et al., 2008; Reynolds & Chelazzi, 2004; Reynolds et al., 2000). Together with the neurophysiologi- Participants cal findings that cortico-cortical feedback provides a weak excitatory influence on the CRF for low-contrast stimuli Four graduate students (three females and one male aged 21 (Hupe et al., 1998, 2001; Sandell & Schiller, 1982;Wang to 35 years) from the University of Melbourne participated et al., 2010), these results suggest top-down modulation can in the experiment. Three were experienced psychophysical enhance center excitation. participants and one had no previous experience observing In Experiment 1, cognitive load effects on center psychophysics experiments. All had normal or corrected-to- excitation were assessed. As cognitive load is assumed normal vision. Participants were screened and consented in to tax limited cognitive resources, it was hypothesized accordance with approval from the human research ethics that it may reduce the brain’s ability to provide top-down board of the University of Melbourne. modulation for concurrent early perceptual processing. In other words, the capacity for top-down enhancement of Apparatus center excitation should be diminished causing contrast sensitivity to be lower under high-load conditions. Stimuli were created on a MacPro computer using An orthogonal orientation discrimination task was MATLAB (version 7.8) and the Psychophysics Toolbox 3.0 employed as a proxy for a typical peripheral contrast (Brainard, 1997; Pelli, 1997) and displayed on a gamma- detection task. This task was adapted from previous spatial corrected 17-inch CRT monitor, 1024-by-768-pixel at 85 Hz attention studies and was carefully chosen based on three in a dimly lit room. The background was a uniform gray major considerations. Firstly, the performance on this task with the luminance set to the middle of the monitor’s range, is generally believed to reflect the contrast responses of about 55 cd/m2. The stimuli were viewed binocularly at 80 orientation selective early cortical visual neurons (Skottun cm with participant’s head position stabilized with a chin et al., 1987). Secondly, the orthogonal discrimination rest. and yes-no detection tasks produce equivalent contrast thresholds (Thomas & Gille, 1979). Thirdly, by asking Stimuli participants to judge the orientation contingent dimension of interest (contrast) rather than contrast itself, the task The contrast detection task The target stimuli for the minimizes response bias usually associated with yes-no contrast detection task were Gabor patches (sinusoidal contrast detection tasks (Smith & Wolfgang, 2007). Similar gratings embedded in a Gaussian window) subtending 1◦ of methodology has been adopted in multiple spatial attention visual angle presented at 4◦ eccentricity from the fixation. Atten Percept Psychophys

The Gabor stimuli had a center spatial frequency of 3.6 all remaining locations filled by the tilde. In half of the cycles per degree (cpd). On each trial, a Gabor patch was trials, the probe letter was identical to the one previously presented with equal probability at one of the four corners of presented at the exact location and different in the other an imaginary square, centered on a fixation square (0.2◦ × half. In no-load conditions, a grid of tilde symbols was 0.2◦ of visual angle), which was present at the center of presented to occupy the time. The probe array was presented the screen throughout the perceptual task. Half of the trials in dark green to indicate that this was the probe phase of contained a vertical Gabor and the other a horizontal Gabor. the working memory task. The luminance of the red letters The luminance profile L (x, y) of a static vertical of the memory array and the green letters within the probe Gabor patch as a function of spatial coordinates along the array were matched. horizontal (x) and vertical (y)axeswas (x − x )2 Procedure L (x,y) = L + L · m · exp − 0 0 0 2σ 2 As depicted in Fig. 1, each trial started with a light grey fix- − 2 (y y0) ation square, appearing for 1000 ms, indicating the start of · exp − · cos(2πf (x −x0)+θ) (1) 2σ 2 a new trial. The fixation square was then replaced by the where L0 is the mean luminance of the display, m is the memory grid presented for 1500 ms (individually adjusted amplitude (contrast) of the Gabor function, x0 and y0 are for one of the participants to be 2000 ms). This was then fol- its horizontal and vertical center positions respectively, σ lowed by the presentation of a fixation square for 1200 ms is the standard deviation of the Gaussian envelope, f is before the Gabor patch was presented. The Gabor patch was the frequency of the sinusoid, and θ is the phase of the presented for 50 ms. A square mask consisting of a high ◦ sinusoid with respect to the center of the Gaussian window. contrast checkerboard pattern (subtending 1.1 of visual All Gabors were in cosine phase with θ setat0. angle) was presented for 200 ms at the same location of the To signal the target location and terminate visual target immediately after the offset of Gabor patches. Each perceptual processing, a square mask consisting of a high trial ended with a probe grid that was presented for 800 ms. contrast checkerboard pattern (subtending 1.1◦ of visual The working memory load, location of the Gabor stimulus angle) was presented for 200 ms at the same location of the and contrast level were all randomized within sessions. target immediately after the offset of the Gabor patch. Participants were instructed to fixate on the central The method of constant stimuli was used. All participants square throughout the trial except for reading the memory performed the contrast detection task prior to formal testing grid and the probe grid. With respect to the memory and to establish the contrast levels required to measure the full detection tasks, participants were told to remember and extent of the psychometric function (five or six levels of maintain the memory set online for the full length of contrast linearly spaced on a log scale from chance to each trial and to report the target orientation as accurately asymptote performance level). as possible immediately following the presentation of the Gabor. Their response of orientation (vertical vs. horizontal) The working memory task The working memory set was of the Gabor was indicated by pressing the arrow (left vs. displayed in a 3×3 grid at the center of the monitor right) key on the computer keyboard using a finger (index in font Arial size 18. The grid was made of English vs. middle) of their right hand respectively. Feedback for an consonants randomly selected from the available 20 without incorrect response was given by a high-frequency tone to replacement. The remainder of the grid was filled with encourage stability of decision criteria (Sperling & Dosher, tilde symbols (∼). The entire grid measured approximately 1986). A response window of 1500 ms was provided for 2.5◦ squared, with each letter within the grid subtending the contrast detection task. Participants then responded to approximately 0.6◦. The combination of the letter and tilde the memory task indicating whether the probe letter was the symbols within the grid varied as a function of load (no load, same vs. different to the one in the memory set (in the exact low load, and high load). In the no-load condition, the grid location) by pressing the arrow (left vs. right) key on the consisted purely of tilde symbols. In the low-load condition, computer keyboard using a finger (index vs. middle) of their one letter was presented in the central location of the grid. right hand. A response window of 2000 ms was provided In the high-load condition, five letters were presented with for the working memory task. No feedback was provided one at each corner and one at the center. The memory grid for the working memory task. Responses made outside the was presented in dark red to indicate the encoding phase of response time window for each task were not recorded. the working memory task. Each session of the dual-task paradigm was about 1 h In the low- and high-load conditions, the memory of the long with multiple breaks. The inexperienced participant letter set was later probed by a single letter presented in was trained on the contrast detection task for ten sessions one of the locations previously occupied by a letter, with with a total of 100 trials per contrast level. The three Atten Percept Psychophys

Fig. 1 Schematic representation of a trial sequence in Experiment 1. consisting of either 5, 1, or 0 English constant letters and tilde symbols Participants performed a 2AFC orientation discrimination task as a presented in dark red font. After a response interval of 1500 ms for the proxy for a contrast detection task on a target Gabor patch (horizontal contrast detection task, a single letter was presented in dark green font or vertical) at one of the four corners of an imaginary square centered to probe the working memory. A maximum 2000 ms response window on the fixation. The target was preceded by a working memory array was allowed for the working memory task experienced participants were given one practice session excluded from the final analyses of individual participants each with a total of ten trials per contrast level. All were 3.18, 3.29, 5.87, and 0.54%, respectively. participants performed a total of 20 1-h testing sessions. The trial randomization process ensured that at least 180 The working memory task valid trials were completed per contrast level per working memory load. Trials with missing responses and responses Accuracy and reaction time of the working memory task for with reaction time less than 200 ms for either the visual the four participants are reported in Table 1. The working perceptual task or working memory task (high- and low- memory task performance in the current experiment was load conditions) were excluded. Depending on the number comparable with the 92 and 98% performance and mean of excluded trials, participants typically completed between 180 and 200 trials per condition. See the results section below for specific details regarding the percentage of trials Table 1 Working memory task performance for each participant in excluded for respective participants. Experiment 1 Analysis and results Accuracy (%) Mean reaction time (ms)

Participant Low High Low High All data were analyzed in R (R Development Core Team, 2011). The psychometric function fitting and associated ID 1 92 91 1026 (414) 1082 (419) model comparisons were analyzed using the psychy 0.1-7 ID 2 99 99 1023 (131) 1054 (140) package (Knoblauch & Maloney, 2012). All figures were ID 3 97 94 728 (167) 803 (132) plotted using ggplot2 package (Wickham, 2009). Analyses ID 4 97 93 769 (205) 1180 (318) of performance for both tasks were only made on trials with legitimate responses. The percentages of trials that were Reaction time standard deviations shown in parentheses Atten Percept Psychophys reaction times of 953 and 1394 ms for the low- and high- memory load conditions with the threshold (α) being varied load conditions, respectively, reported in previous cognitive freely but the slope (β) being constrained to be the same. load studies (de Fockert et al., 2001), suggesting that our Finally, in the threshold-slope model, both the threshold (α) cognitive load manipulation was successful. and slope (β) were estimated for each working memory condition. Goodness-of-fit was assessed with deviance scores, The contrast detection task To assess whether cognitive which were calculated as the log-likelihood ratio between load modulates the contrast detection task, a modified nested models (Wichmann & Hill, 2001a, b). The deviance cumulative Gaussian function was fitted to the data from scores of the one-function model and the threshold model each participant, where x is the stimulus contrast, α, β, λ, were compared to assess whether thresholds were different and γ are the fitted model parameters which determine the across working memory load conditions, and the deviance shape of the psychometric function, scores of the threshold model and threshold-slope model were compared to evaluate whether slope differed across F(x,α,β,γ,λ) = γ + (1 − γ − λ)F (x, α, β) (2) working memory load conditions. The results of these fits are summarized in Table 2. and F is the cumulative Gaussian function: Figure 2 shows the psychometric functions for the β x β2 (x − α)2 three working memory load conditions for each of the F(x; α, β) = √ exp − (3) four participants with the fits of the threshold model. As 2π −∞ 2 expected, performance increased as a function of target with α ∈ (−∞, +∞), β ∈ (−∞, +∞). The contrast contrast under all working memory loads. The psychometric threshold (α) and the slope (β) of the psychometric function for the high-load condition shifted to the right functions were left to vary freely and estimated separately compared to the no- and low-load conditions. Although two for the no-, low-, and high-load conditions. The range of participants showed slope (β) changes, the slope effects the asymptote (λ) was constrained to be within 1∼5%, and were not consistent across participants, and potentially additionally, was forced to be equal across all levels of reflected individual differences. working memory load due to the limits of computational Reaction time was evaluated as a secondary measure of capacity of the psychy 0.1-7 package (Knoblauch & the contrast detection task performance. The mean RT for Maloney, 2012). Gamma (γ ) represented the chance per- each participant was fitted with the two parameter Pieron’s´ formance and was set at 0.5 (Wichmann & Hill, 2001a, b). law function (Pieron,´ 1920; Smith et al., 2004): Fits were performed using maximum-likelihood estima- − F(c) = αc β (4) tion. To determine whether there was a change in threshold (α) and a change in slope (β) of the psychometric functions Pieron’s´ law is a power function that describes the decrease under different working memory load, three models were in mean RT with increasing stimulus contrast, c (Smith compared using a nested hypothesis test (Mood et al., 1974). et al., 2004). It describes an empirical rather than a In the one-function model, a single psychometric function theoretical relationship, which is known to characterize the was fit to all the data; the threshold (α) and slope (β)ofthe dependency of RT on stimuli intensity in a variety of tasks psychometric functions for the three working memory loads (Teichner & Krebs, 1972, 1974). As with accuracy data, the were constrained to be the same. In the threshold model, cognitive load effects were quantified by comparing the fits three psychometric functions were fit to the three working of a one-function model in which the scale (α) and exponent

Table 2 GLM model fits and model comparisons for each participant in Experiment 1

Participant One function Threshold Threshold slope Difference model model model

2 2 2 2 2 χ (13)χ(11) χ (9) χo&th p χth&ths p (2) (2)

ID 1 61.43 47.14 32.78 14.29 .001 14.36 .001 ID 2 14.01 9.94 7.77 4.07 .13 2.17 .34 ID 3 41.07 28.59 25.85 12.48 .01 2.74 .25 ID 4 53.09 40.22 17.32 12.88 .001 22.91 .001

2 χ o&th:chi-square improvement from the one-function model to the threshold model; 2 χ th&ths:chi-square improvement from the threshold model to the threshold-slope model Atten Percept Psychophys

ID 1 ID 2 1.00

No Load Low Load High Load .75 .75 1.00 .75 1.00 .50 .50 .50

15100 200 40 155 100 200 400

ID 3 ID 4 1.00 1.00 Proportion Correct .75 .75 .50 .50

1 5 10 20 40 15100 200 400

Contrast (%)

Fig. 2 Psychometric functions for the no- (blue dotted lines and squares), low- (green dashed lines and triangles) and high- (red solid lines and circles) working memory load (proportion correct as a function of stimulus contrast) for individual participants with fits of the threshold model in Experiment 1. The fitted functions are cumulative Gaussian functions. The error bars represent one binomial standard error

(β) were constrained to be the same and a multi-function to be higher in three participants. The same pattern was seen model in which the scale (α) and exponent (β)varied in the fourth participant although the model comparison did with working memory load condition. Goodness-of-fit was not reach significance for this participant. Reaction time assessed with deviance scores, which were calculated as the data suggest that participants were generally slower on the log-likelihood ratio between nested models (Mood et al., contrast detection task under the high working memory 1974). The deviance scores of the single-function and multi- (vs. no- and low-loads), suggesting there was no speed function models were compared to assess whether RT accuracy tradeoff on the contrast detection task. The model changes differed across working memory load conditions. comparisons showed no significant difference between the The model fits are given in Table 3. The mean RT data were better described for all participants by the multi-function model. Plots of mean RT for each participant are shown in Fig. 3. These results show that participants almost always Table 3 Pieron’s´ law model fits for reaction time data for each participant in Experiment 1 responded faster as contrast increased and mean RTs were generally longer under the high-load condition (vs. the no- Participant One-function Multi-function Difference and low-load conditions). model model

Discussion χ 2(13) χ 2(11) χ 2(2) p ID 1 854.42 714.26 139.17 <.001 The aim of Experiment 1 was to evaluate whether cognitive ID 2 71.74 35.05 36.69 <.001 load modulates the strength of the center excitation ID 3 703.55 496 207.54 <.001 mechanism. We assessed the effects of an unrelated but ID 4 1285.99 113.20 1172.78 <.001 concurrent working memory task on the contrast detection thresholds of small Gabors. Under the high working χ 2:chi-square improvement from the one-function model to the memory load, the contrast detection thresholds were found multi-function model Atten Percept Psychophys

ID 1 ID 2 ID 3 ID 4 No load Low load High load

Mean Reaction Time (ms)

350 400 450 500 450 500 550 600 350 375 400 425 400 500 600 20 40 60 80 10 20 30 40 10 20 30 40 10 20 30 40 Contrast (%)

Fig. 3 Mean reaction time as a function of stimulus contrast and working memory condition for individual participants in Experiment 1. Blue dotted lines and squares, green dashed lines and triangles,andred solid lines and circles represent no-, low-, and high-working memory load, respectively. The fitted functions are Pieron’s Law functions. The error bars represent one standard error of the mean low- and no-load conditions in either contrast thresholds or measures demonstrate top-down modulation effects in early reaction time. visual area when stimulus presentation time is less strictly To the best of the authors’ knowledge, this experiment controlled (Buracas & Boynton, 2007; Ito & Gilbert, 1999; represents one of the first demonstrations of cognitive O’connor et al., 2002; Silver et al., 2007;Tootelletal.,1998; load effects on early visual processing with an established Roberts et al., 2007). This issue is discussed further in the early visual perceptual task. We believe that the behavioral general discussion. effects found in Experiment 1 are consistent with a slight reduction in top-down enhancement to center excitation under high cognitive load based on several factors. Firstly, Experiment 2 cognitive load eﬀects the performance on the orientation discrimination task is on surround inhibition generally accepted to be dependent on orientation selective neurons in early visual cortical areas (e.g., V1) (Hubel Under natural viewing conditions, the center excitation and &Wiesel1962, 1968; Skottun et al., 1987). Secondly, surround inhibition mechanisms are believed to function because the single target Gabor was presented against a in a coordinated fashion to best process contrast variations blank background and in the absence of flankers, the current in visual scenes (Bonds, 1989; Mach, 1866;Petrov& experiment maximally reduced the processing demand at McKee, 2006; Tadin et al., 2003). While the visual task later cognitive levels (Dosher & Lu, 2000; Pelli, 1985; in Experiment 1 was designed to optimally measure Pestilli et al., 2011). Thirdly, placing a backward mask at cognitive load effects on center excitation, the aim of only the target location helped minimize spatial uncertainty Experiment 2 was to determine whether cognitive load (Smith, 2000) and associated performance decrements due could also be shown to impact surround inhibition in early to increased decisional noise believed to be related to vision. target selection from multiple spatial channels (Dosher & It is, however, not straight forward to psychophysically Lu, 2000; Pelli, 1985). Any performance difference seen separate out top-down modulation effects on surround inhi- therefore can be more confidently attributed to reductions in bition from its effects on center excitation. Surround inhi- the quality of perceptual representation due to diminished bition by definition is modulatory in nature—behaviorally strength of top-down modulation associated with increased assessing surround inhibition effects usually involves mea- working memory load. While our use of backward masking suring the contrast sensitivity to a central target with versus had some clear advantages, one interesting question that without the presence of a high-contrast surround mask. At arises is whether cognitive load effects on early visual the behavioral level, measuring contrast sensitivity of the processing are only evident when stimulus presentation target recruits spatial attention as participants are usually time is limited. The spatial attention literature suggests explicitly instructed to focus on the central target and ignore that behavioral measures of top-down modulation effects the high contrast mask. This creates a confound as spatial on early visual processing may be dependent on the use attention has been shown to alter the interaction between of backward masking (Cameron et al., 2002; Carrasco center excitation and surround inhibition (Herrmann et al., et al., 2000; Smith, 2000). However, neurophysiological 2010; Reynolds & Heeger, 2009). Atten Percept Psychophys

To measure cognitive load effects on surround inhibition identify evidence consistent with effects of high cognitive strength relatively independent of spatial attention effects, load on surround inhibition. With respect to the motion here we used a motion discrimination task that is thought discrimination task used here, any reduction in surround to represent a perceptual correlate of surround inhibition inhibition should result in better performance on the task (Tadin et al., 2003). One key aspect of this motion task is (i.e., shorter exposure duration thresholds) under high that only one large size stimulus is used as the target for cognitive load (vs. no and low loads). the perceptual task so that there is no distinction between a target and its surrounding. This has the benefit that the task Methods does not require spatial attention to play the typical dual role of focusing on a target while ignoring its surroundings. Participants Tadin et al. (2003) showed that when a high contrast drifting stimulus was presented very briefly, motion direction Four students (one male and three females aged 21 to 35 discrimination deteriorated with increasing stimulus size. years) from the University of Melbourne participated in The results were interpreted as suggesting that the high the experiment. One participant, the first author, PL, had contrast large motion stimulus induces strong surround inhi- previous experience observing the motion task. Two were bition. This, in turn, reduces the motion direction signal experienced psychophysical observers but had no previous rendering the motion direction more difficult to perceive experience observing the motion task. One participant had (Tadin et al., 2003). The counterintuitive psychophysical no prior experience observing psychophysical experiments. observation for the motion task is believed to result from One of the experienced participants also participated in the neuronal surround inhibition in the middle temporal area Experiment 1. All had normal or corrected-to-normal (MT or V5) (Tadin et al., 2011). MT neurons are known to vision. Participants were screened and consented under be highly selective for motion direction, and roughly half approval of the human research ethics board of the of them exhibit inhibitory center-surround interactions at University of Melbourne. high contrasts but show weak or nonexistent surround inhibition at low contrasts (Born, 2000;Born&Bradley,2005; Apparatus Hunter & Born, 2011; Jones et al., 2001; Tsui & Pack, 2011). This surround inhibition is direction-specific and strongest The apparatus was the same as in Experiment 1. for large, slow-drifting stimuli (Pack et al., 2005). It has also been shown that MT neurons with surround inhibi- Stimuli and design tion integrate motion signals relatively quickly compared to MT neurons without surround inhibition (Churan et al., The target stimuli for the motion discrimination task were 2008, 2009). This finding suggests that brief motion stimuli vertical sine gratings of a spatial frequency 1 cpd windowed preferentially probe MT neurons that have strong center- by a stationary two-dimensional Gaussian envelope (Gabor surround configurations (Churan et al., 2008, 2009). patches) subtending 5◦ of visual angle drifting left or right Consistent with neurological findings that top-down at 1◦/s. The contrast of Gabor patches was fixed at 92% and modulation through feedback connections enhances sur- was ramped on and off with a temporal Gaussian envelope, round inhibition, a recent study provided novel evidence that allowing the presentation of brief motion stimulus. Contrast higher cognitive capacity might provide more efficient top- was defined as the peak contrast within the spatial envelope. down modulation, which in turn, might result in stronger Exposure duration of the stimulus was defined as 2 standard surround inhibition. Melnick et al. (2013) found that indi- deviations of the temporal Gaussian envelope. The spatial vidual variability in surround inhibition reflected in the and temporal phase terms were set to zero for simplicity motion task negatively correlated with IQ (r = -0.71), a mea- (same as described in Eq. 1). sure thought to reflect mainly higher cognitive functions. The parameters of the target stimulus were chosen Thus high-IQ individuals exhibited disproportionately large primarily based on the findings from Tadin et al. (2003). impairments in the performance of this motion task when All participants practiced on the motion task prior to the stimulus was large and of high contrast. The finding sug- formal testing to establish the exposure durations required gests that higher cognitive capacity may be associated with to measure the full extent of the psychometric function stronger surround inhibition. (five durations producing performance ranging from chance Taken together, the current literature suggests that strong to asymptote). Three experienced participants were given top-down modulation may increase surround inhibition in two practice sessions of the single motion task each lasting early visual processing. Since the results of Experiment approximately 40 min. The one inexperienced participant 1 were consistent with cognitive load reducing top-down required four practice sessions (approximately 40 min) enhancement of center excitation, Experiment 2 aimed to to achieve stable performance. All participants in this Atten Percept Psychophys

Table 4 Working memory task performance for each participant in Participants were given 1 practice block on the dual-task Experiment 2 paradigm and completed 12 separate 1-h testing sessions. Participant Accuracy (%) Mean reaction time (ms) The method of constant stimuli was used. The working memory loads and exposure durations of the drifting Gabor Low High Low High were randomized within sessions. As in Experiment 1, each participant completed at least 180 valid trials per ID 1 93 92 855 (308) 888 (268) exposure duration per working memory load, resulting in ID 2 99 98 1025 (133) 1051 (114) approximately 180–200 trials per condition. Trial exclusion ID 3 98 95 679 (169) 789 (138) criteria were the same as in Experiment 1. The percentage ID 4 98 96 615 (89) 775 (132) of trials excluded for respective participants are reported in the following results section. Reaction time standard deviations shown in parentheses Analysis and results experiment showed large performance improvement on the motion discrimination task during practice sessions. This The data analyses methods and model fitting procedure performance improvement, termed “perceptual learning” for the motion discrimination task accuracy data were has been found in many psychophysical tasks (Dosher & as described in Experiment 1. The percentages of trials Lu, 1999). During practice, stimulus exposure duration was that were excluded from the final analyses for the four progressively shortened over successive practice blocks as participants were 2.39, 0.2, 0.73, and 0.58%, respectively. the participants became better at the task. As a result, ◦ ◦ the drifting speed was reduced from 2 /s to 1 /s due The working memory task Accuracy and reaction time of to the marked performance improvement during training the working memory task for the four participants are sessions. reportedinTable4. The patterns of results were comparable The working memory stimuli were the same as described with Experiment 1 and previous cognitive load studies (de in Experiment 1. Fockert et al., 2001). Both accuracy and reaction time data showed the general trend that the working memory Procedure task performance was better in the low compared to the high-load condition. The procedure of Experiment 2 was similar to that of Experiment 1 with the motion discrimination task replacing the contrast detection task. For the motion task, participants The motion discrimination task The model fitting proce- were asked to report the motion direction as accurately as dure for the motion discrimination task accuracy data was possible but in a timely manner immediately following the as described in Experiment 1. The results of the model presentation of the drifting Gabor. Participants indicated the fits under the three load conditions are summarized in perceived direction (left vs. right) by pressing the arrow (left Table 5. As expected, performance increased as a function vs. right) key on the computer keyboard using the finger of exposure duration of the drifting Gabor under all three (index vs. middle) of their right hand respectively. Feedback load conditions. However, there were no significant dif- for an incorrect response was given by a high-frequency ferences between psychometric functions of the motion tone. discrimination task under the three load conditions (Fig. 4).

Table 5 GLM model fits and model comparisons for each participant in Experiment 2

Participant One-function Threshold Threshold-slope Difference model model model

2 2 2 2 2 χ (13) χ (11) χ (9) χo&th(2) p χth&ths(2) p

ID 1 4.74 4.61 3.77 .12 .94 .84 .66 ID 2 24.74 21.71 19.95 3.03 .22 1.76 .42 ID 3 14.26 10.02 9.43 4.23 .12 .59 .74 ID 4 40.56 39.49 38.99 1.07 .58 .50 .77

2 2 χ o&th:chi-square improvement from the one-function model to the threshold model; χth&ths:chi-square improvement from the threshold model) to the threshold-slope model Atten Percept Psychophys

Discussion ound inhibition tasks in many aspects. Consequently, the surround inhibition mechanisms measured by such differ- The effects of cognitive load on surround inhibition in a ent tasks may be different. The aim of Experiment 3 was to visual motion task were assessed in Experiment 2. Because assess cognitive load effects on surround inhibition using a the psychometric functions under the three working classical surround inhibition task, adopted from Petrov and memory load conditions did not differ significantly, we McKee (2009). In this task, participants detected a peri- were unable to demonstrate cognitive load effects on the pheral contrast target surrounded by a high contrast mask surround inhibition mechanism in early visual processing. (Fig. 5). The target was a small Gabor presented randomly Taken in isolation, this finding could imply that the in one of two possible peripheral locations at 8◦ eccentricity impact of cognitive load on surround inhibition is either to the fixation. Strong surround inhibition was elicited by absent or not as great as the effects on center excitation. the presentation of a high contrast sine wave mask in half There are, however, potential factors associated with prac- of the trials. tice effects and individual differences that may have con- Given this surround inhibition task naturally involves tributed to the non-significant results found in Experiment 2 focusing spatial attention to the target while ignoring the and are considered in detail in the final discussion. surround mask, spatial attention effects were considered during the experimental design. According to the normaliza- Experiment 3 cognitive load eﬀects tion model of spatial attention (Reynolds & Heeger, 2009), on surround inhibition in a classical the relative size of the stimulus and the spatial attention surround inhibition task field shifts the balance between excitation and inhibition and determines the amount of surround inhibition that is Experiment 2 found no effects of cognitive load on surround included in the normalization process. Previous studies have inhibition using the motion discrimination task from Tadin shown that high cognitive load leads to a more distributed et al. (2003). This motion task differs from classical surr- spatial attention when optimal task performance requires

ID 1 ID 2 1.00

No Load Low Load High Load .75 .75 .50 .50

40 84 128 40 60 80

ID 3 ID 4 1.00 1.00 1.00 Proportion Correct .75 .75 .50 .50

32 60 88 24 56 88 Exposure Duration (ms)

Fig. 4 Psychometric functions for no- (blue dotted lines and squares), low- (green dashed lines and triangles) and high- (red solid lines and circles) working memory (proportion correct as a function of stimulus exposure duration) for individual participants with fits of the threshold model in Experiment 2. Performance increased as a function of exposure duration under all three working memory loads. There is no significant difference between psychometric functions of the motion discrimination task under the three load conditions. The fitted functions are cumulative Gaussian function functions. The error bars represent one binomial standard error Atten Percept Psychophys

Therefore, in the final experiment, high cognitive load was expected to influence the performance on the surround inhibition task in two main possible ways. If high cognitive load diminishes top-down modulation to both center excitation and surround inhibition and if its effects on altering spatial attention distribution are negligible, it should result in improved contrast detection reflecting diminished surround inhibition. In contrast, if high cognitive load only diminishes center excitation but not surround inhibition as suggested by results of Experiment 2 and if cognitive load effects on spatial attention distribution are negligible, contrast detection thresholds for the central target under the surround condition should be elevated. Additionally, if high cognitive load leads to a more distributed spatial attention, this would only further elevate contrast detection thresholds under the surround condition. In all these scenarios, cognitive load was expected to result in elevated contrast detection thresholds under the no-surround condition based on findings from Experiment 1.

Methods

Participants

Three students (two males and one female aged 21 to 30 years) from the University of Melbourne participated in Exper- iment 3. None of these three observers participated in Exper- iments 1 or 2 but two participants had previous experience with psychophysical experiments. All had normal or corrected-to-normal vision. Participhuman research ethics board of the University of Melbourne.

Apparatus Fig. 5 The contrast detection task with and without the high contrast mask in Experiment 3. Note that the thin circles surrounding the two possible target locations are shown at high contrast for illustrative As described for Experiments 1 and 2 with one exception: purposes (actual contrast was 10%) the monitor was fitted with a Bits++ box (Cambridge Research Systems) operating in Mono++ mode to give true focused spatial attention (Caparos & Linnell, 2010; Linnell 14-bit luminance accuracy. & Caparos, 2011). To reduce the confounding effects of spatial attention, we directly combined the working mem- Stimuli and design ory manipulation with the surround inhibition task requiring detection of a peripheral target. This resulted in a multifo- The classical surround inhibition task For the surround inhi- cal spatial attention paradigm, as detection of the peripheral bition task, the target stimulus was a standard horizontally ◦ target required simultaneously monitoring two locations. oriented Gabor patch (diameter 2.6 ) with a center spatial The current evidence relating to spatial attention alloca- frequency of 1.5 cpd. A two-alternative spatial forced- tion suggests that when multiple spatial locations are to choice procedure was employed, in which the Gabor was be simultaneously attended to, attention can concurrently randomly presented in one of two possible peripheral loca- ◦ select multiple locations excluding interposed locations tions, 8 eccentricity left and right to the fixation point. In (Baldauf & Deubel, 2008; Carlson et al., 2007). But spa- half of the trials, the target Gabor was presented with a sur- tial attention in such situation would not be as focused as round mask extending the whole screen. The surround mask in tasks requiring focused spatial attention to a single loca- was a sinusoidal grating of the same orientation and spa- tion. Nevertheless, the possibility that cognitive load also tial frequency as the target. The contrast of the surround ◦ leads to more a distributed spatial attention in tasks requir- mask was fixed at 50%. Faint thin circles of 0.1 wide and ◦ ing multifocal spatial attention cannot be entirely ruled out. 2.8 in diameter were presented surrounding the two possi- Atten Percept Psychophys ble target regions as this has been shown to be particularly Table 6 Working memory task performance for each participant in important for equalizing the spatial uncertainty of targets Experiment 3 presented with and without the surround mask (Petrov & Participant Accuracy (%) Mean reaction time (ms) McKee, 2006). The contrast of the faint circles was 10% and the circles were made of bright and dim dashes of the Low High Low High same spatial frequency of the background sine wave. The faint circles and the fixation square were continuously vis- ID 1 96 95 791 (233) 881 (233) ible throughout the task. In the other half of the trails, the ID 2 99 98 649 (155) 731 (135) Gabor was present without the surround mask. ID 3 98 95 928 (284) 940 (238) The target contrast was determined using a modified weighted up/down adaptive staircase method, where step Reaction time standard deviations shown in parentheses size ratio was 3 to 1 for up and down steps to target 75% correct performance (Kaernbach, 1991). More specifically, 36 contrast levels were chosen linearly spaced in the exclusion criteria were the same as in previous experiments. contrast range between -2 to -0.65 on the log scale, with a The percentages of trials that were excluded from the final step size of ∼0.04 log unit. The initial contrast level in each analyses for the three participants were 1.92, 0.14, and staircase was chosen randomly in the upper middle range 0.66, respectively. Where the results of a staircase did not within the 36 levels (around 15% contrast). On each of the converge, the data from the corresponding session was following trials, the contrast of the stimulus was reduced discarded. Data from 11, 10, and 8 sessions for participant by two step sizes after each correct response and increased one, two and three, respectively, were included in the final by six step sizes after each incorrect response until three analyses. response reversals were reached. After the third response reversal, the contrast of the stimulus was reduced by one The working memory task step size after each correct response and increased by three step sizes after each incorrect response. Accuracy and reaction time of the working memory task for The working memory stimuli were as described in the three participants are reported in Table 6. The patterns Experiments 1 and 2. of results were comparable with Experiment 1 and those reported in cognitive load studies (de Fockert et al., 2001). Procedure The contrast detection task The procedure and testing of the working memory task was the same as in previous experiments. For the surround The staircase results of individual participants were fitted inhibition task, 1400 ms after the presentation of the with a general linear mixed model (GLMM) with a probit memory grid for the working memory task, the Gabor link function (Knoblauch, 2012; Moscatelli et al., 2012). stimulus was presented for 50 ms. Participants pressed the Ordinary GLMs assume that the responses are independent arrow button (left vs. right) to indicate which location (left and conditionally identically distributed. One way to vs. right) contained the target using a finger (index vs. satisfy these assumptions is to provide extensive training middle) of their dominant hand respectively. There was a prior to formal testing in order to stabilize participants’ 1500 ms response window for the surround inhibition task. performance and reduce the variability between sessions. In each testing session, two randomly interleaved When training is intentionally provided at a limited level to staircases (each of 60 trials) were run for each combination minimize practice effects, responses collected from multiple of load (no, low and high load) and surround condition sessions may violate these assumptions. In GLMMs the (no surround and surround), resulting in 12 staircases for overall variability is separated into a fixed and a random each testing session. Participants were given two training component. The fixed component estimates the variance of sessions (four staircases per surround condition per session) the effects of interest (i.e., working memory load), whereas on the surround inhibition task prior to the formal testing. the random component estimates the heterogeneity between Each session of the dual-task paradigm took about 1 h with staircases as an approximation for sessions. In this way, a multiple breaks during which participants remained in the single model was fitted with all data across all staircases dimly lit room. Each participant completed 12 sessions. for each participant, but each staircase was allowed to have a different level of variability. In particular, the following Analyses and results model was adopted: All data were analyzed in R (R Development Core Team, ∗ = + + 2011) with lme4.0 package (Bates et al., 2012). Trial Yij α βxij vij (5) Atten Percept Psychophys

Table 7 GLMM model fits and model comparisons for each participant in Experiment 3

Participant One-function model Surround model Surround-WM model

AIC BIC χ 2 AIC BIC χ 2 AIC BIC χ 2

ID 1 1181 1196 1175 1127 1147 1119 1131 1161 1119 ID 2 931 945 925 811 830 803 812 841 799 ID 3 754 768 748 660 678 652 663 690 651

GLMM model evaluation. Notes: The Akaike Information Criterion (AIC) and the Bayesian Information Criterion (BIC) both favor the surround model (the smaller the value is, the better the model fit is)

∗ where Yij is a latent response variable, assumed to have a under the three working memory load conditions, three ∗ cumulative Gaussian function. The response variable Yij for models were compared using likelihood ratio tests. In the trial j in staircase i is linked to the linear predictors through one-function model, the response was modeled as a sum the probit link function, such that the expected value of of the fixed effect covariate, target contrast, and a random ∗ the latent variable E (Yij ) is the inverse of the cumulative intercept per staircase. In the surround model, a fixed effects Gaussian function ( ) of the response probability: term (surround condition) was added to the one-function − model. In the surround-working-memory model, there were Φ 1 p Y = 1 = α + βx (6) ij ij two fixed effects terms (surround condition and working α and β are the intercept and slope of the fixed effects memory load). The results of the fits are summarized in parameters of the GLMM. The detection threshold can be Table 7. The random and fixed effects fitting parameters calculated as of the model are reported in Tables 8 and 9 respectively. β The surround model appears to be superior to both the one- threshold =− (7) α function model and the surround-working-memory model. Thus, model comparison results suggest adding the fixed The error term υij is the sum of two components μi and εij , such that: effect factor surround explains the variance of the data better than the one-function model with only the contrast covariate υij = μi + εij ∼ 2 whereas no benefits can be obtained by adding the working μi N(0,σμ) 2 memory load factor. The thresholds under no-surround and εij ∼ N(0,σ ) ε surround conditions for all participants calculated according The error-term ij represents the variability within to Eq. 7 are summarized in Fig. 6. staircase and the error-term μi the variability between Data of a single staircase with the fit of the surround- staircases; μi is also known as random effects parameter. In working-memory model for each participant are shown in GLMMs, the errors εij are independent only conditional on Fig. 7. There was significant effect of surround with the the random parameter ui. The estimation of the parameters is based on the maximum likelihood (ML). The guess rate (i.e., asymptotic chance probability) of the probit link Table 9 Fixed effects parameter of the GLMM (the surround model) functionwassetat0.5. for each participant in Experiment 3 To determine whether the contrast detection thresholds Participant Intercept differ under no-surround versus surround conditions and Estimate Std. Error z value p Table 8 Random effects intercept parameter of the GLMM (the ID 1 .04 .06 .72 .47 surround model) for each participant in Experiment 3 ID 2 −.52 .08 −6.78 <.0001 Participant Variance S.D. No. observations No. staircase ID 3 −.59 .09 −6.84 <.0001 Contrast ID 1 <.0001 <.0001 1160 11 ID 1 17.12 1.18 14.82 <.0001 ID 2 <.0001 <.0001 911 10 ID 2 18.82 1.19 14.48 <.0001 ID 3 <.0001 <.0001 703 8 ID 3 33.74 2.04 16.56 <.0001 Surround mask Given the random effects were small, the data could be fit with a model without random effects (i.e., a GLM model). But as suggested ID 1 −.39 .05 −7.50 <.0001 by Knoblauch and Maloney (2012), since the data resulted from an ID 2 −.65 .06 −10.82 <.0001 experiment in which the staircase was be expected to be a random term, ID 3 −.63 .07 −9.64 <.0001 it could be included even though it is small Atten Percept Psychophys

condition) are consistent with the findings in Petrov and McKee (2009), demonstrating that the surround inhibition manipulation in the current experiment was successful. The model comparison indicated there was no significant difference between our two models comparing surround only versus surround with working memory load as an additional parameter. Thus, for the stimulus configurations used in the current experiment, there were no significant effects of cognitive load on surround inhibition in this classical surround inhibition task. The results also suggest that cognitive load did not affect contrast detection thresholds under the no-surround mask condition. One alternative is that the null findings reflect a Fig. 6 Contrast thresholds (%) for the no-surround and surround failure of our stimulus to selectively capture surround conditions and significance levels for each participant in Experiment 3 inhibition effects by inadvertently stimulating both the excitatory center and the inhibitory surround. In the current psychometric function shifting to the right under the experiment, however, we feel confident that this was not surround condition, corresponding to higher contrast the case. In line with the study by Petrov and McKee thresholds under the surround (vs. no-surround) condition. (2009) we selected surround mask parameters believed In contrast the psychometric function did not vary according to cause minimal activation of the excitatory center of to working memory load conditions. target responsive neurons with relatively selective activation of the surround based on existing physiological and Discussion psychophysical data (Angelucci et al., 2002; Angelucci & Bullier, 2003; Angelucci & Bressloff, 2006; Cavanaugh The goal of Experiment 3 was to assess if cognitive et al., 2002; DeAngelis et al., 1994; Sceniak et al., 2001). load modulates surround inhibition in a classical surround The significance of stimulus parameters in interpreting inhibition task in the periphery. The elevated contrast the results of Experiment 3 is considered in the general thresholds under the surround condition (vs. no-surround discussion.

ID1 ID2 ID3

No Surround Surround No Surround Surround No Surround Surround 1.00

0.75 No Load

0.50 No Load Low Load 0.25 High Load

0.00 1.00

0.75 Low Load

0.50

0.25

Proportion Correct 0.00 1.00

0.75 High Load

0.50

0.25

0.00 151015101510151015101510 Contrast (%)

Fig. 7 An example staircase of the GLMM fit for the surround-working-memory model. The psychometric functions indicating the proportion correct for detecting the target as a function of its contrast are displayed for no-surround and surround conditions for no- (blue), low- (green) and high- (red) working memory loads for each participant in Experiment 3.Thedot sizes are proportional to the number of trials at a particular target contrast (the largest size indicates 15 trials). The functions shift to the right under the surround condition (vs. no surround condition) in all participants Atten Percept Psychophys

General discussion decisional noise associated with spatial uncertainty (Smith, 2000). This was important because the peripheral con- The aim of the current study was to explore whether trast detection task involved monitoring multiple locations cognitive load modulates early visual processing. In and was associated with high overall background noise Experiment 1 cognitive load was found to significantly against which the judgment has to be made (Dosher & diminish contrast sensitivity, whereas no effect of cognitive Lu, 2000). Consequently, one might argue that cognitive load was seen in Experiments 2 or 3. Since the three load might have interacted with this high decisional noise established early vision tasks are known to reflect center associated with spatial uncertainty at post-perceptual pro- excitation and surround inhibition effects, it is important to cessing stages in Experiment 1. However, backward masks consider how these results relate to the existing literature have been shown to reduce spatial uncertainty by serving and the potential impact of the stimulus configurations and as peripheral cues that attract exogenous attention to the experiment designs used in the 3 experiments. cued location (Jonides, 1981;Muller¨ & Rabbitt, 1989;Pos- In Experiment 1, the finding of diminished contrast ner, 1980; Smith et al., 2001; Yantis & Jonides, 1990). sensitivity provides arguably the first behavioral evidence of Exogenous attention involves reflexive orienting and cogni- cognitive load effects on early visual processing. Although tive load is known to have no impact on it Jonides (1981). two recent studies claimed to have found cognitive load Consequently, high cognitive load could not have affected effects on early visual processing (Cocchi et al., 2011; the ability of higher cognitive functions to use backward de Fockert & Leiser, 2014), their conclusions may be masks as peripheral cues to reduce spatial uncertainty at premature given their design and technical methodologies. post-perceptual processing stages. Since the mask occurred Cocchietal.(2011) found that concurrent spatial working after target presentations, it could only have reduced spatial memory load improved performance on a grouping by uncertainty at post-perceptual processing stages but should proximity task and concluded that concurrent cognitive not have affected early visual processing of the target prior load facilitated early visual processing. However, these to the mask onsets. Thus, in Experiment 1, the found dimin- results are difficult to interpret because the spatial working ished contrast sensitivity under high cognitive load is best memory task and the grouping task may interact in some explained by a reduced top-down modulation on early visual modality specific ways as both are believed to recruit visual processing. cortex (Harrison and Tong, 2009; Konstantinou et al., 2012; It is crucial to recognize that although backward Kosslyn et al., 1999; Serences et al., 2009). In the case of de masking may be required to control task difficulty in Fockert and Leiser (2014), they showed that high cognitive order to demonstrate cognitive load effects on early load enhanced collinear facilitation - an established early visual processing with only behavioral measures as in visual mechanism. However, it is unclear how the very Experiment 1, this does not suggest cognitive load low contrast stimuli would have been achieved using the only modulates early visual processing when stimulus methodologies described, i.e., 0.3% (low contrast), 0.5% presentation time is limited. In the visual spatial attention (medium contrast), and 0.9% (high contrast) without any literature, it is now accepted that with only behavioral special method to increase luminance resolution such as the measures, the demonstration of spatial attention effects Bits++ box. on early visual processing is dependent on backward The diminished contrast sensitivity observed in Exper- masking in simple contrast detection tasks (Cameron iment 1 is consistent with reduced top-down modulation et al., 2002; Carrasco et al., 2000; Carrasco, 2011; Smith, on early visual processing associated with high cognitive 2000; Smith & Wolfgang, 2004; Smith et al., 2004; load. One important feature of the design of Experiment Smith & Wolfgang, 2007). But with neuroimaging and 1 is the employment of backward masking, which is a neurophysiological methods, spatial attention has been tool regularly used by vision researchers to control the dif- shown to modulate neural activity in visual areas as ficulty of contrast detection tasks by limiting the period early as V1 and the thalamus without backward masking of time observers have visual access to the target (Breit- (Buracas & Boynton, 2007;Chenetal.,2014;Fang meyer, 2007). It served two major functions in Experiment &He,2008; Hupe et al., 1998; Ito & Gilbert, 1999; 1. Firstly, it helped ensure that performance accuracy fell Motter, 1993; O’connor et al., 2002; Pestilli et al., 2011; into a measurable range (Enns & Di lollo, 2000). In our Sestieri et al., 2008; Silver et al., 2007; Somer et al., case, it was instrumental in controlling the task difficulty 1999; Sylvester et al., 2007, 2009; Tootell et al., 1998; such that performance accuracy covered the whole psy- Roberts et al., 2007). Similarly, although backward masking chometric function using luminance resolutions that are is necessary for the demonstration of behavioral effects technically possible with modern computers and monitors. of cognitive load on early visual processing, we believe Secondly, because the backward mask was presented only cognitive load modulates early visual processing regardless at the target location this helped to reduce the impact of the employment of backward masking. Atten Percept Psychophys

We feel that our findings in Experiment 1 are consistent make the motion discrimination task more challenging as with changes in centre excitation associated with a reduction performance improved during the training sessions. Yet, in top-down enhancement because of the consideration the duration thresholds when taken at 82% were still given to ensure the correspondence between our stimuli much shorter (60 ∼ 80 ms) than those reported by Tadin and the underlying CRF that they were designed to et al. (2003). Consistent with a possible role of practice stimulate. The parameters of our stimuli were carefully effects, performance on simple perceptual tasks has been chosen based on existing understanding of neurons in early shown to improve with repeated task exposure, typically visual cortex. Recent neurophysiological and neuroimaging over multiple sessions spreading over several days due to studies have provided converging evidence that the average perceptual learning (Dosher and Lu, 1999;Fahle,2005; size (diameter) of CRFs, measured with small high contrast Gilbert et al., 2001; Goldstone, 1998; Seitz & Watanabe, stimuli, varies from 0.8◦ for low eccentricities to 2.1◦ 2005). Compared to the center excitation mechanism, for large eccentricities (mean=1.0◦) (Angelucci et al., surround inhibition mechanisms seem to be particularly 2002; Angelucci and Bullier, 2003; Angelucci & Bressloff, sensitive to practice effects and it has been reported that 2006; Cavanaugh et al., 2002; DeAngelis et al., 1994; practice effects may even cancel out surround inhibition Sceniak et al., 2001). It is impossible to selectively effects (Dorais & Sagi, 1997). If surround inhibition had activate only the center because the surround is spatially been reduced by repetitive practice on the motion task continuous across the receptive field (Angelucci et al., in Experiment 2, it would be impossible for top-down 2002; Angelucci & Bullier, 2003; Cavanaugh et al., 2002; modulation effects to alter surround inhibition. As a result, it DeAngelis et al., 1994; Sceniak et al., 2001). However, may have rendered cognitive load effects difficult to detect we can be confident that the contribution of the surround behaviorally. is negligible for our small low-contrast stimuli, as the In addition to any impact of practice effects, individual surround is thought to have a much higher threshold than differences may have also contributed to the shorter the center (Angelucci et al., 2002; Angelucci & Bullier, exposure durations seen in Experiment 2 compared to those 2003; Cavanaugh et al., 2002). The target Gabor size reported in Tadin et al. (2003). Our exposure durations were (1◦) in Experiment 1 ensured minimal concurrent surround similar to those reported in another motion perception study inhibition was evoked. Consequently, the reduction in using the same motion task (Lappin et al., 2009)andmay contrast sensitivity observed with increased cognitive load, reflect individual differences in the magnitude of surround provides evidence consistent with a load-induced reduction inhibition that have previously been reported (Cannon & in center excitation. Fullenkamp, 1993;Tadin,2015). If our participants had In Experiment 2, we were unable to demonstrate an relatively limited surround inhibition, top-down modulation impact of cognitive load on surround inhibition in the effects may have been difficult to detect. motion task. However, the potential impact of practice In Experiment 3, although the surround inhibition manip- effects (Dosher & Lu, 1999;Fahle,2005; Gilbert et al., ulation was successful, we were unable to demonstrate an 2001; Goldstone, 1998; Seitz & Watanabe, 2005)and impact of cognitive load on surround inhibition in the classi- individual differences (Cannon & Fullenkamp, 1993;Tadin, cal surround inhibition task in the periphery. The results also 2015) is worth considering in the interpretation of the indicate that cognitive load did not affect contrast detection null results. Firstly, because our participants’ baseline thresholds under the no-surround condition, which suggests performance on the motion discrimination task was not that cognitive load didn’t affect center excitation either. comparable to that reported in Tadin et al. (2003), we Before accepting this interpretation, two alternatives need to believe our results need to be considered with caution and be carefully considered. are likely to have been impacted by the extensive practice One alternative is that the null findings reflect a failure required in Experiment 2. In the initial study by Tadin et al. of our stimulus to selectively capture surround inhibition (2003), a staircase method was used such that the stimulus effects by inadvertently stimulating both the excitatory exposure duration was adjusted based on the participant’s center and the inhibitory surround. If so, it is impossible to performance on previous trials to quickly and successively demonstrate cognitive load effects on surround inhibition. home in on the threshold (40 trials per staircase). In contrast, However, given the carefully chosen stimulus parameters, the method of constant stimuli in Experiment 2 required this interpretation is highly unlikely. Neurophysiological approximately 200 trials per stimulus condition in order findings suggest that the eCRF consists of two fields: a to obtain sufficient number of trials to cover the whole low-contrast summation field immediately outside CRFs psychometric function. In their study, Tadin et al. (2003) and an outer surround field (Angelucci and Bressloff, used a drifting rate of 2◦/s and the duration thresholds 2006; Angelucci & Bullier, 2003; Angelucci et al., 2002; when taken at 82% were in the range of 100∼140 ms. Cavanaugh et al., 2002; DeAngelis et al., 1994; Sceniak In Experiment 2, the drifting rate was reduced to 1◦/s to et al., 2001). The low-contrast summation field, measured Atten Percept Psychophys with low-contrast stimuli, is about twice the diameter of Conclusions the CRF, i.e., 1.6◦ for low eccentricities and 4.2◦ for large eccentricities (mean = 2.0◦) (Angelucci et al., 2002; To summarize, contrast-based early visual processing has Angelucci & Bullier, 2003; Angelucci & Bressloff, 2006; largely been considered to involve autonomous processes Cavanaugh et al., 2002; DeAngelis et al., 1994; Sceniak that do not need the support of cognitive resources. et al., 2001). It can be suppressive at high contrast and However, the reduced contrast sensitivity associated with excitatory at low contrast (Angelucci et al., 2002; Angelucci high cognitive load found in Experiment 1 suggests that & Bullier, 2003; Angelucci & Bressloff, 2006; Cavanaugh cognitive load effects can penetrate the early stages of visual et al., 2002; DeAngelis et al., 1994; Sceniak et al., 2001). perceptual processing. Because the current understanding Outside this low-contrast summation field, there is an outer of top-down modulation effects on the performance of surround field with its diameter varying from 2.5◦ for the surround inhibition tasks adopted in Experiments 2 small eccentricities and to 13◦ for the large eccentricities and 3 is limited, it is premature to consider this to be (mean = 5.1◦). The target Gabor with a size of 2.6◦ at 8◦ definitive evidence of an absence of load effects on surround eccentricity would therefore be expected to cover the CRF inhibition. From the point of view of understanding early and extend into the low-contrast summation field but not visual perceptual mechanisms, it is fully acknowledged that activate surround inhibition in the summation field because null effects of cognitive load in Experiments 2 and 3 suggest the targets were primarily presented around threshold level. that if cognitive load has any impact on surround inhibition, The gap (outer diameter 2.8◦) between the target and such effects are likely to be subtle with minimal impact the surround mask (mainly falling in the outer surround on daily perceptual experience. However, from a systems field) ensured that the surround mask was not presented so neuroscience perspective, these findings give reason to close to the target that the surround mask would stimulate question the previously assumed independence of cognitive the CRF of the neuron responding to the target. The functions and early perceptual processing. Given that real- stimulus configuration also avoided the surround mask world visual processing rarely (if ever) happens in the being coextensive with the target, which may confound absence of concurrent cognitive demands, future research the effects of surround and overlay suppression (Petrov is required to better understanding cognitive load effects & McKee, 2009). Thus, surround inhibition was properly on center excitation and surround inhibition mechanisms manipulated in Experiment 3 and the lack of cognitive load fundamental to early visual processing. effects on surround inhibition is unlikely to have arisen from inappropriate stimulus configurations. Another alternative is that the absence of backward References masking in Experiment 3 may have contributed to the null findings observed. Despite the apparent similarities between Adelson, E. H., & Bergen, J. R. (1991). The plenoptic function and the contrast detection tasks in Experiment 3 (the no sur- the elements of early vision. Computational Models of Visual round condition) and Experiment 1, one of the most cru- Processing, 91(1), 3–20. Angelucci, A., & Bressloff, P. C. (2006). Contribution of feedfor- cial differences is that backward masking was used in ward, lateral and feedback connections to the classical recep- Experiment 1 but not in Experiment 3. As discussed in tive field center and extra-classical receptive field surround Experiment 1, backward masking is probably necessary to of primate V1 neurons, (pp. 93–120). Amsterdam: Elsevier. demonstrate top-down modulation effects on early visual https://doi.org/10.1016/S0079-6123(06)54005-1 Angelucci, A., & Bullier, J. (2003). Reaching beyond the clas- processing with only behavioral measures. To test this sical receptive field of V1 neurons: horizontal or feed- possibility, the seemingly simple solution might be to repeat back axons?. Journal of Physiology, Paris, 97(2-3), 141–154. Experiment 3 with the addition of a pattern backward https://doi.org/10.1016/j.jphysparis.2003.09.001 mask. The issue is, however, more complicated. Adding the Angelucci, A., Levitt, J. B., Walton, E. J. S., Hupe, J.-M., Bullier, J., & LUND, J. S. (2002). Circuits for local and global signal integration backward masking to the no-surround condition is straight- in primary visual cortex. The Journal of Neuroscience : the Official forward. In contrast, the addition of a backward mask to Journal of the Society for Neuroscience, 22(19), 8633–8646. the surround condition could cause the surround mask and Baldauf, D., & Deubel, H. (2008). Properties of atten- the backward mask to be presented in close temporal order. tional selection during the preparation of sequential saccades. Experimental brain research Experimentelle Hirn- This, in turn, could lead to interactions between the two, forschung Expe,rimentation´ cer´ ebrale´ , 184(3), 411–425. which would make the results difficult to interpret (Breit- https://doi.org/10.1007/s00221-007-1114-x meyer, 2007). It is unclear how these potential confounds Bates, D., Maechler, M., & Bolker, B. (2012). lme4: Linear mixed- can be avoided using current behavioral methods. Further effects models using S4 classes. Ben-Av, M. B., Sagi, D., & Braun, J. (1992). Visual-attention and exploration of this issue might therefore require combined perceptual grouping. Perception & Psychophysics, 52(3), 277– imaging or electrophysiological measures. 294. https://doi.org/10.3758/BF03209145 Atten Percept Psychophys

Bonds, A. B. (1989). Role of inhibition in the specification of orien- Churan, J., Richard, A. G., & Pack, C. C. (2009). Interac- tation selectivity of cells in the cat striate cortex. Visual Neuro- tion of spatial and temporal factors in psychophysical esti- science, 2(1), 41–55. https://doi.org/10.1017/S0952523800004314 mates of surround suppression. Journal of Vision, 9(4), 15–15. Born, R. T. (2000). Center-surround interactions in the middle tem- https://doi.org/10.1167/9.4.15 poral visual area of the owl monkey. Journal of Neurophysiology, Cocchi, L., Toepel, U., De Lucia, M., Martuzzi, R., Wood, 84(5), 2658–2669. S. J., Carter, O., & Murray, M. M. (2011). Work- Born, R. T., & Bradley, D. C. (2005). Structure and function of ing memory load improves early stages of independent visual area MT. Annual Review of Neuroscience, 28, 157–189. visual processing. Neuropsychologia, 49(1), 92–102. https://doi.org/10.1146/annurev.neuro.26.041002.131052 https://doi.org/10.1016/j.neuropsychologia.2010.10.021 Brainard, D. H. (1997). The psychophysics toolbox. Spatial Vision, Crist, R. E., Li, W., & Gilbert, C. D. (2001). Learning to 10(4), 433–436. https://doi.org/10.1163/156856897X00357 see: experience and attention in primary visual cortex. Nature Breitmeyer, B. G. (2007). Visual masking: past accomplishments, Neuroscience, 4(5), 519–525. https://doi.org/10.1038/87470 present status, future developments. Advances in Cognitive Cutzu, F., & Tsotsos, J. K. (2003). The selective tuning model Psychology / University of Finance and Management in Warsaw, of attention: psychophysical evidence for a suppressive annulus 3(1-2), 9–20. https://doi.org/10.2478/v10053-008-0010-7 around an attended item. Vision Research, 43(2), 205–219. Buracas, G. T., & Boynton, G. M. (2007). The effect of https://doi.org/10.1016/S0042-6989(02)00491-1 spatial attention on contrast response functions in human Dayan, P., & Solomon, J. A. (2010). Selective Bayes: atten- visual cortex. The Journal of Neuroscience : The Offi- tional load and crowding. Vision Research, 50(22), 2248–2260. cial Journal of the Society for Neuroscience, 27(1), 93–97. https://doi.org/10.1016/j.visres.2010.04.014 https://doi.org/10.1523/JNEUROSCI.3162-06.2007 de Fockert, J. W., Rees, G., Frith, C. D., & Lavie, N. Cameron, E. L., Tai, J. C., & Carrasco, M. (2002). Covert (2001). The role of working memory in visual selective attention affects the psychometric function of con- attention. Science (New York, NY), 291(5509), 1803–1806. trast sensitivity. Vision Research, 42(8), 949–967. https://doi.org/10.1126/science.1056496 https://doi.org/10.1016/0042-6989(93)90034-T de Fockert, J. W., & Leiser, J. (2014). Better target detection Cannon, M. W., & Fullenkamp, S. C. (1993). Spatial interactions in the presence of collinear flankers under high working in apparent contrast: Individual differences in enhancement memory load. Frontiers in Human Neuroscience, 8, 821. and suppression effects. Vision Research, 33(12), 1685–1695. https://doi.org/10.3389/fnhum.2014.00821 https://doi.org/10.1016/0042-6989(93)90034-T DeAngelis, G. C., Freeman, R. D., & Ohzawa, I. (1994). Caparos, S., & Linnell, K. J. (2010). The spatial focus of attention Length and width tuning of neurons in the cat’s primary is controlled at perceptual and cognitive levels. Journal of visual cortex. Journal of Neurophysiology, 71(1), 347–374. Experimental Psychology: Human Perception and Performance, https://doi.org/10.1146/annurev.ne.18.030195.001205 36(5), 1080–1107. https://doi.org/10.1037/a0020367 Desimone, R., & Duncan, J. (1995). Neural mechanisms of selective Carlson, T., Vanrullen, R., & Hogendoorn, H. (2007). Distinguishing visual attention. Annual Review of Neuroscience, 18, 193–222. models of multifocal attention: it’s a matter of time. Journal of https://doi.org/10.1146/annurev.ne.18.030195.001205 Vision, 7(9), 641. https://doi.org/10.1167/7.9.641 Dorais, A., & Sagi, D. (1997). Contrast masking effects Carrasco, M., Penpeci-Talgar, C., & Eckstein, M. (2000). Spatial change with practice. Vision Research, 37(13), 1725–1733. covert attention increases contrast sensitivity across the CSF: https://doi.org/10.1016/S0042-6989(96)00329-X support for signal enhancement. Vision Research, 40(10-12), Dosher, B. A., & Lu, Z.-L. (1999). Mechanisms of perceptual learning. 1203–1215. https://doi.org/10.1016/S0042-6989(00)00024-9 Vision Research, 39(19), 3197–3221. https://doi.org/10.1016/ Carrasco, M. (2011). Visual attention: the past 25 years. Vision S0042-6989(99)00059-0 Research, 51(13), 1484–1525. https://doi.org/10.1016/j.visres.2011. Dosher, B. A., & Lu, Z.-L. (2000). Mechanisms of perceptual attention 04.012 in precuing of location. Vision Research, 40(10-12), 1269–1292. Casco, C., Grieco, A., Campana, G., Corvino, M. P., & https://doi.org/10.1016/S0042-6989(00)00019-5 Caputo, G. (2005). Attention modulates psychophysical Enns, J. T., & Di lollo, V. (2000). What’s new in visual and electrophysiological response to visual texture seg- masking?. Trends in Cognitive Sciences, 4(9), 345–352. mentation in humans. Vision Research, 45(18), 2384–2396. https://doi.org/10.1016/S1364-6613(00)01520-5 https://doi.org/10.1016/j.visres.2005.02.022 Fahle, M. (2005). Perceptual learning: specificity versus gener- Cavanaugh, J. R., Bair, W., & Movshon, J. A. (2002). Nature and alization. Current opinion in neurobiology, 15(2), 154–160. interaction of signals from the receptive field center and surround https://doi.org/10.1016/j.conb.2005.03.010 in macaque v1 neurons. Journal of Neurophysiology, 88(5), 2530– Fang, F., & He, S. (2008). Crowding alters the spatial distribution of 2546. https://doi.org/10.1152/jn.00692.2001 attention modulation in human primary visual cortex. Journal of Chen, J., He, Y., Zhu, Z., Zhou, T., Peng, Y., Zhang, X., & Vision, 6(9), 1–9. https://doi.org/10.1167/8.9.6 Fang, F. (2014). Attention-dependent early cortical suppression Freeman, E., Sagi, D., & Driver, J. (2001). Lateral interactions contributes to crowding. The Journal of neuroscience: The Official between targets and flankers in low-level vision depend on Journal of the Society for Neuroscience, 34(32), 10465–10474. attention to the flankers. Nature Neuroscience, 4(10), 1032–1036. https://doi.org/10.1523/JNEUROSCI.1140-14.2014 https://doi.org/10.1038/nn728 Chen, M., Yan, Y., Gong, X., Gilbert, C. D., Liang, H., & Li, Freeman, E., Driver, J., Sagi, D., & Zhaoping, L. (2003). W. (2014). Incremental integration of global contours through Top-down modulation of lateral interactions in early vision: interplay between visual cortical areas. Neuron, 82(3), 682–694. does attention affect integration of the whole or just per- https://doi.org/10.1016/j.neuron.2014.03.023 ception of the Parts?. Current Biology, 13(11), 985–989. Churan, J., Khawaja, F. A., Tsui, J. M. G., & Pack, C. C. (2008). https://doi.org/10.1016/S0960-9822(03)00333-6 Brief motion stimuli preferentially activate surround-suppressed Fujita, I., Tanaka, K., Ito, M., & Cheng, K. (1992). Columns for visual neurons in macaque visual area MT. Current Biology, 18(22), features of objects in monkey inferotemporal cortex. Nature, R1051–R1052. https://doi.org/10.1016/j.cub.2008.10.003 360(6402), 343–346. https://doi.org/10.1038/360343a0 Atten Percept Psychophys

Gilbert, C. D., & Li, W. (2013). Top-down influences on visual Knoblauch, K., & Maloney, L. T. (2012). Modeling psychophysical processing. Nature Reviews Neuroscience, 14(5), 350–363. data in R. https://doi.org/10.1038/nrn3476 Knoblauch, K. (2012). psyphy: Functions for analyzing psychophysi- Gilbert, C. D., Sigman, M., & Crist, R. E. (2001). The neural cal data in R. 0.1-9. basis of perceptual learning review. Neuron, 31(5), 681–697. Konstantinou, N., Bahrami, B., Rees, G., & Lavie, N. (2012). Visual https://doi.org/10.1016/S0896-6273(01)00424-X short-term memory load reduces retinotopic cortex response to Goldstone, R. L. (1998). Perceptual learning. Annual Review of contrast. Journal of Cognitive Neuroscience, 24(11), 2199–2210. Psychology, 49(1), 585–612. https://doi.org/10.1146/annurev.49. https://doi.org/10.1162/jocn a 00279 1.585. Kosslyn, S. M., Pascual-Leone, A., Felician, O., Camposano, S., Han, S. H., Jiang, Y., Mao, L. H., Humphreys, G. W., & Gu, H. Keenan, J. P., Thompson, W. L., Ganis, G., Sukel, S., & (2005a). Attentional modulation of perceptual grouping in human Alpert, N. M. (1999). The role of area 17 in visual imagery: visual cortex: functional MRI studies. Human Brain Mapping, convergent evidence from PET and rTMS. Science (New York, 25(4), 424–432. https://doi.org/10.1002/hbm.20119 NY), 284(5411), 167–170. https://doi.org/10.1126/science.284. Han, S. H., Jiang, Y., Mao, L. H., Humphreys, G. W., & Qin, J. G. 5411.167 (2005b). Attentional modulation of perceptual grouping in human Lappin, J. S., Tadin, D., Nyquist, J. B., & Corn, A. L. (2009). Spatial visual cortex: ERP studies. Human Brain Mapping, 26(3), 199– and temporal limits of motion perception across variations in 209. https://doi.org/10.1002/hbm.20157 speed, eccentricity, and low vision. Journal of Vision 9.1, 30,1– Harrison, S. A., & Tong, F. (2009). Decoding reveals the contents of 14. https://doi.org/10.1167/9.1.30 visual working memory in early visual areas. Nature, 458(7238), Lavie, N. (2005). Distracted and confused?: selective atten- 632–635. https://doi.org/10.1038/nature07832 tion under load. Trends in Cognitive Sciences, 9(2), 75–82. Harrison, W. J., Mattingley, J. B., & Remington, R. W. (2013). Eye https://doi.org/10.1016/j.tics.2004.12.004 movement targets are released from visual crowding. The Journal Lavie, N. (2010). Attention, distraction, and cognitive control under of Neuroscience : The Official Journal of the Society for Neuro- load. Current Directions in Psychological Science, 19(3), 143– science, 33(7), 2927–2933. https://doi.org/10.1523/JNEUROSCI. 148. https://doi.org/10.1023/A:1023452218225 4172-12.2013 Levi, D. M. (2008). Crowding-an essential bottleneck for object He, S., Cavanagh, P., & Intriligator, J. (1996). Attentional resolution recognition: a mini-review. Vision Research, 48(5), 635–654. and the locus of visual awareness. Nature, 383(6598), 334–337. https://doi.org/10.1016/j.visres.2007.12.009 https://doi.org/10.1038/383334a0 Heeger, D. J. (1992). Normalization of cell responses in cat striate cor- Levi, D. M., Hariharan, S., & Klein, S. A. (2002). Suppressive and tex. Visual Neuroscience, 9(2), 181–197. https://doi.org/10.1017/ facilitatory spatial interactions in peripheral vision: peripheral S0952523800009640. crowding is neither size invariant nor simple contrast masking. Herrmann, K., Montaser-Kouhsari, L., Carrasco, M., & Heeger, D. J. Journal of Vision, 2(2), 167–177. https://doi.org/10.1167/2.2.3 (2010). When size matters: attention affects performance by Li, W., Piech,¨ V., & Gilbert, C. D. (2004). Perceptual learning and top- contrast or response gain. Nature Neuroscience, 13(12), 1554– down influences in primary visual cortex. Nature Neuroscience, 1559. https://doi.org/10.1038/nn.2669 7(6), 651–657. https://doi.org/10.1038/nn1255 Hubel, D. H., & Wiesel, T. N. (1962). Receptive fields, binocular Li, W., Piech,¨ V., & Gilbert, C. D. (2006). Contour saliency in primary interaction and functional architecture in the cat’s visual cortex. visual cortex. Neuron, 50(6), 951–962. https://doi.org/10.1016/j. The Journal of Physiology, 160(1), 106-154.2. neuron.2006.04.035. Hubel, D. H., & Wiesel, T. N. (1968). Receptive fields and functional Li, X., Lu, Z.-L., Tjan, B. S., Dosher, B. A., & Chu, W. architecture of monkey striate cortex. The Journal of Physiology, (2008). Blood oxygenation level-dependent contrast response 195(1), 215–243. https://doi.org/10.1113/jphysiol.1968.sp008455s functions identify mechanisms of covert attention in early Hubel, D. H., & Wiesel, T. N. (1965). Receptive fields and functional visual areas. Proceedings of the National Academy of Sci- architecture in two nonstriate visual areas (18 and 19) of the cat. ences of the United States of America, 105(16), 6202–6207. Journal of Neurophysiology, 28(2), 229–289. https://doi.org/10.1073/pnas.0801390105 Hunter, J. N., & Born, R. T. (2011). Stimulus-dependent modulation of Ling, S., & Carrasco, M. (2006). Sustained and transient suppressive influences in MT. The Journal of Neuroscience : The covert attention enhance the signal via different contrast Official Journal of the Society for Neuroscience, 31(2), 678–686. response functions. Vision Research, 46(8-9), 1210–1220. https://doi.org/10.1523/JNEUROSCI.4560-10.2011 https://doi.org/10.1016/j.visres.2005.05.008 Hupe, J. M., James, A. C., Girard, P., Lomber, S. G., Payne, B. R., & Linnell, K. J., & Caparos, S. (2011). Perceptual and cognitive load Bullier, J. (2001). Feedback connections act on the early part of the interact to control the spatial focus of attention. Journal of responses in monkey visual cortex. Journal of Neurophysiology, Experimental Psychology: Human Perception and Performance, 85(1), 134–145. 37(5), 1643–1648. https://doi.org/10.1037/a0024669 Hupe,J.M.,James,A.C.,Payne,B.R.,Lomber,S.G.,Girard,P., Liu, T., Abrams, J., & Carrasco, M. (2009). Voluntary attention & Bullier, J. (1998). Cortical feedback improves discrimination enhances contrast appearance. Psychological Science, 20(3), 354– between figure and background by v1, v2 and v3 neurons. Nature, 362. https://doi.org/10.1111/j.1467-9280.2009.02300.x 394(6695), 784–787. https://doi.org/10.1038/29537 Lu, Z. L., & Dosher, B. S. (1998). External noise distin- Ito, M., & Gilbert, C. D. (1999). Attention modulates contextual guishes attention mechanisms. Vision Research, 38(9), 1183– influences in the primary visual cortex of alert monkeys. Neuron, 1198. https://doi.org/10.1016/S0042-6989(97)00273-3 22(3), 593–604. https://doi.org/10.1016/S0896-6273(00)80713-8 Mach, E. (1866). Uber¨ die Wirkung der raumlichen¨ Verteilung des Jones, H. E., Grieve, K. L., Wang, W., & Sillito, A. M. (2001). Lichtreizes auf die Netzhaut. Sitzungsberichte der mathematis- Surround suppression in primate v1. Journal of Neurophysiology, chnaturwissenschaftlichen Classe der kaiserlichen Akademie der 86(4), 2011–2028. Wissenschaften, 52, 303–322. Jonides, J. (1981). Voluntary versus automatic control over the mind’s Mack, A., Tang, B., Tuma, R., Kahn, S., & Rock, I. (1992). Perceptual eye’s movement. Attention and Performance IX. organization and attention. Cognitive Psychology, 24(4), 475–501. Kaernbach, C. (1991). Simple adaptive testing with the weighted Marr, D. (1976). Early processing of visual information. Philosophical up-down method. Perception & Psychophysics, 49(3), 227–229. Transactions of the Royal Society B: Biological Sciences, https://doi.org/10.3758/BF03214307 275(942), 483–519. https://doi.org/10.1098/rstb.1976.0090 Atten Percept Psychophys

Marr, D. (1982). Vision: a computational investigation into the human Reynolds, J. H., Pasternak, T., & Desimone, R. (2000). Attention representation and processing of visual information.NewYork: increases sensitivity of v4 neurons. Neuron, 26(3), 703–714. Henry Holt and Co. The MIT Press. https://doi.org/10.1016/S0896-6273(00)81206-4 Melnick, M. D., Harrison, B. R., Park, S., Bennetto, L., & Reynolds, J. H., & Chelazzi, L. (2004). Attentional modulation of Tadin, D. (2013). A strong interactive link between sensory visual processing. Annual review of Neuroscience, 27, 611–647. discriminations and intelligence. Current Biology, 23(11), 1013– https://doi.org/10.1146/annurev.neuro.26.041002.131039 1017. https://doi.org/10.1016/j.cub.2013.04.053 Reynolds, J. H., & Heeger, D. J. (2009). The normalization model Mood, A. M., Graybill, F. A., & Boes, D. C. (1974). Introduction to of attention. Neuron, 61(2), 168–185. https://doi.org/10.1016/j. the theory of statistics. McGraw: Tokyo. neuron.2009.01.002. Moscatelli, A., Mezzetti, M., & Lacquaniti, F. (2012). Model- Roberts, M., Delicato, L. S., Herrero, J., Gieselmann, M. A., & Thiele, ing psychophysical data at the population-level: the general- A. (2007). Attention alters spatial integration in macaque v1 in ized linear mixed model. Journal of Vision, 12(11), 26–26. an eccentricity-dependent manner. Nature Neuroscience, 10(11), https://doi.org/10.1167/12.11.26 1483–1491. https://doi.org/10.1038/nn1967 Motter, B. C. (1993). Focal attention produces spatially selective Sandell, J. H., & Schiller, P. H. (1982). Effect of cooling area processing in visual cortical areas v1, v2, and v4 in the presence of 18 on striate cortex cells in the squirrel monkey. Journal of competing stimuli. Journal of Neurophysiology, 70(3), 909–919. Neurophysiology, 48(1), 38–48. Muller,¨ H. J., & Rabbitt, P. M. (1989). Reflexive and volun- Sceniak, M. P., Hawken, M. J., & Shapley, R. (2001). Visual tary orienting of visual attention: time course of activation spatial characterization of macaque v1 neurons. Journal of and resistance to interruption. Journal of Experimental Psy- Neurophysiology, 85(5), 1873–1887. chology: Human Perception and Performance, 15(2), 315–330. Scocchia, L., Cicchini, G. M., & Triesch, J. (2013). What’s ”up”? https://doi.org/10.1037/0096-1523.15.2.315 Working memory contents can bias orientation processing. Vision Nassi, J. J., Lomber, S. G., & Born, R. T. (2013). Corticocortical Research, 78, 46–55. https://doi.org/10.1016/j.visres.2012.12.003 feedback contributes to surround suppression in v1 of the Scolari, M., Kohnen, A., Barton, B., & Awh, E. (2007). Spatial alert primate. The Journal of Neuroscience : The Official attention, preview, and popout: which factors influence critical Journal of the Society for Neuroscience, 33(19), 8504–8517. spacing in crowded displays?. Journal of Vision, 7(2), 7–7. https://doi.org/10.1523/JNEUROSCI.5124-12.2013 https://doi.org/10.1167/7.2.7 O’connor, D. H., Fukui, M. M., Pinsk, M. A., & Kastner, S. Seitz, A., & Watanabe, T. (2005). A unified model for percep- (2002). Attention modulates responses in the human lateral tual learning. Trends in Cognitive Sciences, 9(7), 329–334. geniculate nucleus. Nature Neuroscience, 5(11), 1203–1209. https://doi.org/10.1016/j.tics.2005.05.010 https://doi.org/10.1038/nn957 Serences, J. T., Ester, E. F., Vogel, E. K., & Awh, E. (2009). Pack, C. C., Hunter, J. N., & Born, R. T. (2005). Contrast Stimulus-specific delay activity in human primary visual cortex. dependence of suppressive influences in cortical area MT of Psychological Science, 20(2), 207–214. https://doi.org/10.1111/ alert macaque. Journal of Neurophysiology, 93(3), 1809–1815. j.1467-9280.2009.02276.x https://doi.org/10.1152/jn.00629.2004 Sestieri, C., Sylvester, C. M., Jack, A. I., d’Avossa, G., Shulman, Pashler, H. (1994). Dual-task interference in simple tasks: data and G. L., & Corbetta, M. (2008). Independence of anticipatory theory. Psychological Bulletin, 116(2), 220–244. https://doi.org/ signals for spatial attention from number of nontarget stimuli in 10.1037/0033-2909.116.2.220 the visual field. Journal of Neurophysiology, 100(2), 829–838. Pelli, D. G. (1985). Uncertainty explains many aspects of visual https://doi.org/10.1152/jn.00030.2008 contrast detection and discrimination. Journal of the Optical Silver, M. A., Ress, D., & Heeger, D. J. (2007). Neural Society of America A: Optics and Image Science, 2(9), 1508–1532. correlates of sustained spatial attention in human early https://doi.org/10.1364/josaa.2.001508 visual cortex. Journal of Neurophysiology, 97(1), 229–237. Pelli, D. G. (1997). The VideoToolbox software for visual psy- https://doi.org/10.1152/jn.00677.2006 chophysics: transforming numbers into movies. Spatial Vision, Skottun, B. C., Bradley, A., Sclar, G., Ohzawa, I., & Freeman, 10(4), 437–442. https://doi.org/10.1163/156856897X00366 R. D. (1987). The effects of contrast on visual orientation and Pestilli, F., Carrasco, M., Heeger, D. J., & Gardner, J. L. (2011). spatial-frequency discrimination—a comparison of single cells Attentional enhancement via selection and pooling of early and behavior. Journal of Neurophysiology, 57(3), 773–786. sensory responses in human visual cortex. Neuron, 72(5), 832– Smith, A. T., Singh, K. D., Williams, A. L., & Greenlee, M. W. (2001). 846. https://doi.org/10.1016/j.neuron.2011.09.025 Estimating receptive field size from fMRI data in human striate Pestilli, F., Ling, S., & Carrasco, M. (2009). A population-coding and extrastriate visual cortex. Cerebral Cortex (New York, NY : model of attention’s influence on contrast response: Estimating 1991), 11.12, 1182–1190. https://doi.org/10.1152/jn.00677.2006 neural effects from psychophysical data. Vision Research, 49(10), Smith, P. L. (2000). Attention and luminance detection: Effects of 1144–1153. https://doi.org/10.1016/j.visres.2008.09.018 cues, masks, and pedestals. Journal of Experimental Psychol- Petrov, Y., & McKee, S. P. (2006). The effect of spatial configuration ogy: Human Perception and Performance, 26(4), 1401–1420. on surround suppression of contrast sensitivity. Journal of Vision, https://doi.org/10.1037/0096-1523.26.4.1401 6(3), 224–238. https://doi.org/10.1167/6.3.4 Smith, P. L., Ratcliff, R., & Wolfgang, B. J. (2004). Attention orienting Petrov, Y., & McKee, S. P. (2009). The time course of contrast masking and the time course of perceptual decisions: response time dis- reveals two distinct mechanisms of human surround suppression. tributions with masked and unmasked displays. Vision Research, Journal of Vision 9.1, 21, 1–11. https://doi.org/10.1167/9.1.21 44(12), 1297–1320. https://doi.org/10.1016/j.visres.2004.01.002 Petrov, Y., & Meleshkevich, O. (2011). Locus of spatial attention Smith, P. L., & Wolfgang, B. J. (2004). The attentional dynam- determines inward-outward anisotropy in crowding. Journal of ics of masked detection. Journal of Experimental Psychol- Vision 11.4. https://doi.org/10.1167/11.4.1 ogy: Human Perception and Performance, 30(1), 119–136. Pieron,´ H. (1920). Nouvelles recherches sur lanalyse du temps de https://doi.org/10.1037/0096-1523.30.1.119 latence sensorialle et sur la loi qui relie ce temps a lintensitede Smith, P. L., & Wolfgang, B. J. (2007). Attentional mechanisms in lexcitation. L’annee´ Psychologique, 22, 58–142. visual signal detection: The effects of simultaneous and delayed Posner, M. I. (1980). Orienting of attention. The Quarterly Journal of noise and pattern masks. Perception & Psychophysics, 69(7), Experimental Psychology, 32(1), 3–25. 1093–1104. https://doi.org/10.3758/BF03193947 Atten Percept Psychophys

Somers, D. C., Dale, A. M., Seiffert, A. E., & Tootell, R. B. H. (1999). Teichner, W. H., & Krebs, M. J. (1974). Laws of visual choice reaction Functional MRI reveals spatially specific attentional modulation time. Psychological Review, 81(1), 75–98. https://doi.org/10. in human primary visual cortex. Proceedings of the National 1037/h0035867 Academy of Sciences of the United States of America, 96(4), 1663– Thomas, J. P., & Gille, J. (1979). Bandwidths of orienta- 1668. https://doi.org/10.1073/pnas.96.4.1663 tion channels in human vision. JOSA, 69(5), 652–660. Sperling, G., & Dosher, B. A. (1986). Strategy and optimization https://doi.org/10.1364/JOSA.69.000652 in human information processing. In Boff, K., Kaufman, L., & Tootell, R. B. H., Hadjikhani, N., Hall, E. K., Marrett, S., Thomas,J.P.(Eds.)Handbook of Perception and Performance, Vanduffel, W., Vaughan, J. T., & Dale, A. M. (1998). The (pp. 1–65). New York: Wiley. retinotopy of visual spatial attention. Neuron, 21(6), 1409–1422. Strasburger, H. (2005). Unfocused spatial attention underlies the https://doi.org/10.1016/S0896-6273(00)80659-5 crowding effect in indirect form vision. Journal of Vision, 5(11), Tsui, J. M. G., & Pack, c. c. (2011). Contrast sensitivity of MT 1024–1037. https://doi.org/10.1167/5.11.8 receptive field centers and surrounds. Journal of Neurophysiology, Sylvester, C. M., Shulman, G. L., Jack, A. I., & Corbetta, M. (2007). 106(4), 1888–1900. https://doi.org/10.1152/jn.00165.2011 Asymmetry of anticipatory activity in visual cortex predicts the Van der Lubbe, R. H. J., & Keuss, P. J. G. (2001). Focused locus of attention and perception. The Journal of Neuroscience: attention reduces the effect of lateral interference in multi-element The Official Journal of the Society for Neuroscience, 27(52), arrays. Psychological Research, 65(2), 107–118. https://doi.org/ 14424–14433. https://doi.org/10.1523/JNEUROSCI.3759-07.2007 10.1007/s0042600000571007/s004260000057 Sylvester, C. M., Shulman, G. L., Jack, A. I., & Corbetta, M. (2009). Anticipatory and stimulus-evoked blood oxygenation Wang, C., Huang, J. Y., Bardy, C., FitzGibbon, T., & Dreher, B. level-dependent modulations related to spatial attention reflect (2010). Influence of ’feedback’ signals on spatial integration in a common additive signal. The Journal of Neuroscience: The receptive fields of cat area 17 neurons. Brain Research, 1328, 34– Official Journal of the Society for Neuroscience, 29(34), 10671– 48. https://doi.org/10.1016/j.brainres.2010.02.069 10682. https://doi.org/10.1523/JNEUROSCI.1141-09.2009 Wichmann, F. A., & Hill, N. J. (2001a). The psycho- Tadin, D. (2015). Suppressive mechanisms in visual motion process- metric function: I. fitting, sampling, and goodness of ing: from perception to intelligence. Vision Research, 115(Pt A), fit. Perception & Psychophysics, 63(8), 1293–1313. 58–70. https://doi.org/10.1016/j.visres.2015.08.005 https://doi.org/10.1016/j.brainres.2010.02.069 Tadin, D., Lappin, J. S., Gilroy, L. A., & Blake, R. (2003). Perceptual Wichmann, F. A., & Hill, N. J. (2001b). The psychomet- consequences of centre-surround antagonism in visual motion ric function: II. Bootstrap-based confidence intervals and processing. Nature, 424(6946), 312–315. https://doi.org/10.1038/ sampling. Perception & Psychophysics, 63(8), 1314–1329. nature01800 https://doi.org/10.3758/BF03194545 Tadin, D., Silvanto, J., Pascual-Leone, A., & Battelli, L. (2011). Wickham, H. (2009). ggplot2: elegant graphics for data analysis. Improved motion perception and impaired spatial suppression Yantis, S., & Jonides, J. (1990). Abrupt visual onsets and selective following disruption of cortical area MT/v5. Journal of Neuro- attention: Voluntary versus automatic allocation. Journal of science, 31(4), 1279–1283. https://doi.org/10.1523/JNEUROSCI. Experimental Psychology: Human Perception and Performance, 4121-10.2011 16(1), 121–134. https://doi.org/10.1037/0096-1523.16.1.121 Teichner, W. H., & Krebs, M. J. (1972). Laws of the simple Zhaoping, L. (2014). Understanding vision: theory, models, and data. visual reaction time. Psychological Review, 79(4), 344–358. Oxford: Oxford University Press. https://doi.org/10.1037/h0032946