Effect of Parietal Lobe Lesions on Saccade Targeting and Spatial Memory in a Naturalistic Visual Search Task Steven S

Neuropsychologia 41 (2003) 1365–1386

Effect of parietal lobe lesions on saccade targeting and spatial memory in a naturalistic visual search task Steven S. Shimozaki a,∗, Mary M. Hayhoe a, Gregory J. Zelinsky b, Amy Weinstein c, William H. Merigan a, Dana H. Ballard a a Center for Visual Science, University of Rochester, Rochester, NY, USA b Department of Psychology, State University of New York, Stony Brook, NY, USA c Department of Neurology, Strong Memorial Hospital, University of Rochester, Rochester, NY, USA Received 8 November 2001; received in revised form 7 January 2003; accepted 7 January 2003

Abstract The eye movements of two patients with parietal lobe lesions and four normal observers were measured while they performed a visual search task with naturalistic objects. Patients were slower to perform the task than the normal observers, and the patients had more fixations per trial, longer latencies for the first saccade during the visual search, and less accurate first and second saccades to the target locations during the visual search. The increases in response times for the patients compared to the normal observers were best predicted by increases in the number of fixations. In order to investigate the effects of spatial memory on search performance, in some trials observers saw a preview of the search display. The patients appeared to have difficulty using previously viewed information, unlike normal observers who benefit from the preview. This suggests a spatial memory deficit. The patients’ deficits are consistent with the hypothesis that the parietal cortex has a role in the selection of targets for saccades, in memory for target location. © 2003 Elsevier Science Ltd. All rights reserved.

Keywords: Eye movements; Visual attention

1. Introduction Neurophysiological studies in primates strongly suggest that another aspect of visuospatial processing involving the Lesions of the parietal cortex have long been known to parietal cortex is saccadic target selection. The parietal cor- cause impairments of visuospatial function. For example, tex projects to areas primarily responsible for eye movement parietal lesions often lead to the syndrome of hemineglect. control, such as the superior colliculus (SC) (Andersen, Occurring predominately with right parietal lesions, hem- Asanuma, Essick, & Siegel, 1990; Fries, 1984; Lynch, ineglect patients tend to ignore the contralesional (left) Graybiel, & Lobeck, 1985) and the frontal eye fields (FEFs) side of the environment. Because this syndrome can be (Andersen et al., 1990; Cavada & Goldman-Rakic, 1989; distinguished from a loss of visual acuity (hemianopia), Schall, Morel, King, & Bullier, 1995). Also, areas in the hemineglect is considered to be a spatial attention deficit posterior parietal cortex, such as 7a and lateral intra parietal (review Rafal, 1994). It has also been shown that parietal le- (LIP), respond both to the anticipation and the execution of sions can cause constructional apraxia (a visuospatial deficit a saccade (e.g. Andersen, Essick, & Siegel, 1987; Colby, in copying and reproducing two- and three-dimensional Duhamel, & Goldberg, 1995; Colby, Duhamel, & shapes) (Benowitz, Moya, & Levine, 1990; Benton, 1967; Goldberg, 1996; Gnadt & Andersen, 1988; Mountcastle, Ruessmann, Sondag, & Beneike, 1988), and deficits in Lynch, Georgopoulos, Sakata, & Acuna, 1975). Andersen short-term spatial memory (De Renzi, Faglioni, & Previdi, (1995) suggests that the LIP activity preceding a saccade en- 1977; De Renzi, Faglioni, & Scotti, 1969; De Renzi & codes the intention to make a saccade, and Duhamel, Colby, Nichelli, 1975). and Goldberg (1992) also found that parietal receptive fields shift in preparation of an intended saccade. Gottlieb, Kusunoki, and Goldberg (1998) found that saccade-related ∗ Corresponding author. Present address: Department of Psychology, University of California, Santa Barbara, CA 93106, USA. activity in LIP depends on behavioral relevance. They sug- Tel.: +1-805-893-3853; fax: +1-805-893-4303. gest that LIP contains a visual ‘salience’ map used in the E-mail address: [email protected] (S.S. Shimozaki). selection of saccadic targets, as proposed by a number of

0028-3932/03/$ – see front matter © 2003 Elsevier Science Ltd. All rights reserved. doi:10.1016/S0028-3932(03)00042-3 1366 S.S. Shimozaki et al. / Neuropsychologia 41 (2003) 1365–1386 computational models of saccades (Findlay & Walker, 1999; and line drawings of objects (Husain et al., 2001). These Rao, Zelinsky, Hayhoe, & Ballard, 2002; Wolfe, 1994). refixations appear to reflect a deficit in the spatial memory Although the attentional deficits caused by parietal le- in the ipsilesional field. Of particular interest is the study sions in humans have been extensively studied, relatively Husain et al. (2001), in which the neglect patients and the less is known about the impact of parietal lesions on eye normal observers explicitly stated whether each object had movements. Several previous studies on the effect of parietal been fixated previously. The hemineglect patients not only lesions on eye movements have focused on hemineglect (e.g. refixated objects more often, they also did not remember Barton, Behrmann, & Black, 1998; Behrmann, Watt, Black, previous fixations. & Barton, 1997; Chedru, Leblanc, & Lhermitte, 1973; Ishiai, This study further explores the effect of posterior pari- Sugishita, Mitani, & Ishizawa, 1992; Karnath, 1994; Walker etal lesions on saccades by examining the saccades of four & Findlay, 1996). Consistent with the attentional effect of normal observers and two patients with parietal lesions dur- hemineglect, these studies find that saccades of hemineglect ing a visual search task. Unlike what has been more com- patients tend to be biased away from the neglected field (both monly studied, the two parietal patients did not have ob- saccade direction and number of fixations). Studies that have servable signs of neglect. In a typical visual search trial, focused more on the eye movements per se, have found that the observer is given a target, views a search display that parietal patients have difficulty in a “double-step” saccade has or does not have the target, and then must indicate the task (Duhamel, Goldberg, Fitzgibbon, Sirigu, & Grafman, presence of the target in the search display. Also, experi- 1992; Heide, Blankenburg, Zimmerman, & Kompf, 1995; ments in visual search often require the observers to hold Heide & Kompf, 1998). In this task, the observer must make their gaze, or have briefly presented stimuli, disallowing the two successive saccades to two targets flashed briefly and use of eye movements (e.g. Palmer, 1994, 1995; Treisman, sequentially in the dark before the first saccade can begin. 1991). If eye movements are allowed, however, observers Thus, to make an accurate second saccade, the observer must tend to make several saccades during a visual search task take into account the spatial displacement of the eye caused (Findlay & Gilchrist, 1998; Zelinsky & Sheinberg, 1997). by the first saccade. Parietal lesions affected this ability, so Zelinsky and Sheinberg (1997) and Scialfa, Thomas, and that inaccurate second saccades were made despite the pres- Joffe (1994), for example, found that the number of saccades ence of accurate first saccades, suggesting an inability to was an excellent predictor of the response times. Another take account of the eye displacement caused by the first sac- study by Zelinsky, Rao, Hayhoe, and Ballard (1997) found cade. Parietal lesions also caused increased saccade latencies that, during visual search, the accuracy of the saccades to during visually guided reflexive saccades (saccades to sud- the target location reliably improved from the first to the last den onset targets in previously unknown locations) (Heide saccade of each trial, showing that saccades reflect the dy- & Kompf, 1998; Pierrot-Deseilligny, Rivuad, Gaymard, & namic evolution of the search process. Thus, under natural Agid, 1991a; Pierrot-Deseilligny, Rivuad, Penet, & viewing conditions, the process of saccadic target selection Rigolet, 1987) and affected saccade latencies and accu- is an integral part of visual search. Given the likely role of racies for memory guided saccades (Pierrot-Deseilligny, the posterior parietal cortex in saccadic target selection, the Rivuad, Gaymard, & Agid, 1991b). Similar effects were question we wish to address is how parietal lesions affect found in normal observers when their posterior parietal the saccadic target selection process during visual search. cortex was temporarily inactivated by transcranial magnetic Another aspect in most visual search studies is that the stimulation (TMS) (Muri, Vermersch, Rivaud, Gaymard, & targets are presented in novel locations on each trial. Thus, Pierrot-Deseilligny, 1996). In addition, several fMRI studies the search must be accomplished solely on the basis of the have found activation in the parietal areas during periods of target’s appearance. In the natural world, however, saccadic visually-guided saccades, compared to periods of fixation targets can be selected either on the basis of the target’s ap- (Corbetta et al., 1998; Darby et al., 1996; Luna et al., 1998; pearance (for example, where is the red sweater?) or on the Petit, Clark, Ingeholm, & Haxby, 1997). basis of previously acquired information about the target’s Some recent studies suggest a specific role of the parietal location (e.g. the keys are on the table). In these cases, search cortex in the use of spatial memory in saccade targeting. In can be based on the memory of the target’s location as well. an fMRI study, Heide et al. (2001), studied a ‘triple-step’ A study by Epelboim et al. (1995), for example, showed that saccade task, which is analogous to the double-step task spatial memory can affect saccadic performance in a search described above, except that observers had three succes- task. They found that the time taken to tap a specified se- sive locations to fixate. They found more activation in the quence of colored lights arrayed on a table rapidly decreased right intraparietal area during the triple-step saccade task, as the task was repeated, suggesting that repeated fixations compared to various types of single-step (visually- and of the locations facilitated the tapping movements. Thus, we memory-guided) saccades that had less demands on spatial were interested to explore saccadic eye movements in both memory. Also, some recent studies of parietal hemineglect kinds of search process, namely, both in appearance- and found a greater number of refixations on the ipsilesional spatial memory-based search. side in a dot-counting task (Zihl & Hebel, 1997), as well A last feature of the experiments was the use of naturalis- as in visual search and cancellation tasks of letters, circles, tic displays. Experiments on visual search typically involve S.S. Shimozaki et al. / Neuropsychologia 41 (2003) 1365–1386 1367 the use of simple stimuli that share common features and are tion within a story, and in the immediate recall of complex easily segmented from the background (e.g. oriented red and nonverbal information (Wechsler Memory Scale—Revised green bars on a white background). In normal scenes, how- (Wechsler, 1987)). Also, RW’s results for the generation of ever, individual objects are visually complex and typically words for specific letters (Controlled Oral Word Associa- have a distinctive set of properties. Additionally, they are tion (Benton & Hamsher, 1989)) and three-dimensional con- usually embedded in a complex background (e.g. a packet of struction (Benton, Hamsher, Varney, & Spreen, 1983) were cereal on a breakfast table). Since confusability with other in the low average range. stimuli, and ease of segregation from the background are Goldmann visual field perimetry tests indicated no signs both likely to affect the search process, we chose search of hemianopia in either patient. Both patients did not show displays that were close to normal scenes. This study there- any clinical signs of hemineglect the day after their brain fore adds to the previous work on eye movements following injury, and later neuropsychological testing also failed to re- parietal lesions by (a) examining saccadic target selection veal evidence of hemineglect. For RW, this included more in the visual search of naturalistic displays, (b) investigating extensive testing using the detection of near-threshold stim- the use of spatial memory in target selection, and (c) using uli in an extinction paradigm. Neither patient showed signs patients with no manifestation of neglect. of an oculomotor deficit based on clinical observation, or had difficulty with the calibration procedure for the eye tracking equipment. The procedure involved fixating a series of 2. Method 25 points for about 1 s each, and was a relatively demand- ing oculomotor task. Both patients had mild peripheral neu- 2.1. Description of observers rological effects from their brain injury. WS experienced numbness and mild pain in his left arm, and RW experi- We identified two patients with lesions localized to a sin- enced numbness and occasional tremors in his right arm and gle hemisphere of the parietal cortex. The first patient (RW) right side of the face. Neuropsychological tests corroborated was a 42-year-old right-handed male with a resection of a these self-reports. Both patients had normal color vision, left posterior lateral parietal tumor (glioma) approximately and normal or corrected-to-normal visual acuity. 4 years prior to his participation in the studies (see Fig. 1, The normal observers were four males with ages ranging drawing of T1-weighted MRI by W. Merigan. A drawing is from 25 to 31 years. All had normal color vision, and normal presented due to the difficulty in observing the lesion in the or corrected-to-normal visual acuity, and no known physical original MRI). The second patient (WS) was a 62-year-old or mental disability. One the authors (GJZ) participated as right-handed male with a right parietal lobe lesion (ante- an observer, and is left-handed. The other three normal ob- rior ventromedial and posterior dorsolateral) caused by an servers were right-handed graduate students initially naive thrombotic stroke approximately 1 year prior to his partici- to the purpose of the experiment. pation in the studies (see Fig. 2, T1-weighted MRI, 3 days after incident). Both patients were tested on a battery of 2.2. Procedure standard neuropsychological tests, including general intelligence, memory, verbal and language skills, motor skills, Both patients and four normal observers participated in the and visual perception. Both patients had high average pre- visual search task while their eye positions were monitored. morbid intelligence, as assessed by the Wechsler Adult In- Observers had to indicate the absence or presence of the telligence Scale—Revised (WAIS-R) (Wechsler, 1981), and target in a search display of naturalistic items. The number as suggested by their personal histories (WS was a retired of items in the search display was one, three or five. schoolteacher, and RW was a computer programmer on In the first condition (No Preview condition, see Fig. 3), disability). Generally the neuropsychological tests for both observers first saw a display of the target for 1 s, followed patients were within or above normal limits, correspond- by a fixation screen for at least 1 s. (This display was not ing with clinical observations that both patients remained removed until the observer’s eye position was within 1◦ of highly functional, showing little overall behavioral effect visual angle of the cross, guaranteeing eye position near the from their brain injury. There were, however, some spe- bottom center at the initiation of search.) Then the search cific focal changes in cognitive functioning for both patients. display was shown until the observer responded, indicating Patient WS scored in the low average range for the 7/24 the presence or absence of the target in the search display. Spatial Recall Test (Rao, Hammeke, McQuillen, Kharti, & In this condition the observer could only use information Lloyd, 1984). Given his above average scores on the other about the target’s appearance to locate the target in the dis- tests, this suggests a possible visuospatial memory deficit. play. In the second condition (Preview condition, see Fig. 4), His scores for Block Design and Digit Span (parts of the two additional displays preceding the sequence in the No WAIS-R) were also in the low average range, suggesting Preview trials, a 2 s preview display, followed by 1 s fixa- possible deficits in spatial construction and short-term mem- tion screen. The preview display contained all the possible ory, respectively. Patient RW exhibited a mild to moderate targets in their correct locations in the search display. In decline in the immediate recall of unrelated verbal informa- other words, if the target appeared in the search display, it 1368 S.S. Shimozaki et al. / Neuropsychologia 41 (2003) 1365–1386

Fig. 1. Drawing of the anatomical MRI (T1-weighted, horizontal) of patient RW describing his glial tumor in the left parietal lobe prior to surgery. The drawings are reversed according to radiological convention, such that the right half of one image represents the left hemisphere, and vice versa. would only appear at the same location as in the preview. pearance and the prior information about target location to For example, in Fig. 4, once the observer learns the target aid search. is the paint scraper, she or he should know to look only in The observer’s right eye position was monitored with an the position indicated by the preview (the far left). Thus, the SRI Generation V dual-Purkinje eye tracker operating at observer could use both information about the target’s ap- 500 Hz with an accuracy of approximately 15 visual angle. S.S. Shimozaki et al. / Neuropsychologia 41 (2003) 1365–1386 1369

Fig. 2. Anatomical MRI (T1-weighted, horizontal) of patient WS 3 days after his thrombotic stroke in the right parietal lobe. The drawings are reversed according to radiological convention, such that the right half of one image represents the left hemisphere, and vice versa.

The observer’s head position was fixed using a dental com- observers. The procedure involved fixating 25 points on the pression (bite) bar, and an eye patch was worn on the left display in random order, and fitting the raw tracker data out- eye. A calibration procedure was performed every 90 trials put to the coordinates of the screen by linear interpolation. for the patients, and every 120 or 180 trials for the normal Offline analyses of the eye movements were performed by 1370 S.S. Shimozaki et al. / Neuropsychologia 41 (2003) 1365–1386

Fig. 3. No Preview trials. The sequence of displays begins at the top left. The search display for a Target Present trial appears at the bottom left, and for a Target Absent trial at the bottom right. Observers indicated their choice for target presence in the search display. Observers ﬁxated the cross in the second dark display before the search display appeared, ensuring the same eye position at the initiation of search.

first converting raw eye position data into eye velocity using were a total of 12 conditions, 2 (Preview/No Preview) × a 17-point velocity filter. A saccade then was identified if the 2 (Target Present/Target Absent) × 3 (number of items, filtered (smoothed) eye velocity exceeded 35◦/s, and if the 1/3/5). Each normal observer participated in 60 trials of resulting analyses met the criterion of a minimum fixation each condition, for 720 total trials. WS participated in duration of 30 ms. 45 trials of each condition, for 540 total trials. RW par- Observers viewed images on an AppleColor 14 in. RGB ticipated in 30 trials of each Preview condition, and 45 color display at 80 cm from the observer, and driven by an trials of each No Preview condition, for 450 total trials. Apple Macintosh IIfx computer. The targets and distractors Trials were blocked by Preview or No Preview trials (90 were digitized naturalistic color objects presented on a back- trials per block for the patients, and 120 or 180 trials ground (12◦ × 16◦) with an appropriate context. There were per block for the normal observers), with target presence three contextual settings; toys in a crib, tools on a work- and number of items randomized within these blocks. bench, and food on a dinner table. The objects for each trial Only the trials with correct responses were analyzed. were chosen randomly from a set of 10 for each context. Analyses of variance were performed at an α level of The object locations were randomized across six locations 0.05 using the statistical package GANOVA (Woodward, in the search display, 7◦ from the initial fixation point at Bonett, & Brecht, 1990). Analyses were performed first the bottom center, and spaced at equal radial angles (22.5◦) with respect to the hemifield (left or right) of the tar- from the fixation point. get. Results for all observers showed no consistent effect Half the search displays contained the target (Target of hemifield, and subsequent results were collapsed over Present), and the other half did not (Target Absent). There hemifield. S.S. Shimozaki et al. / Neuropsychologia 41 (2003) 1365–1386 1371

Fig. 4. Preview trials. The same sequence as Fig. 3, except for the preceding Preview and ﬁxation display. The Preview display contained all the possible targets in their correct locations. For example, in this trial, the target paint scraper can only appear in the far left position. 1372 S.S. Shimozaki et al. / Neuropsychologia 41 (2003) 1365–1386

Table 1 Response times, statistical results Normals RW WS

Patients vs. normals n.a. 1618 (<0.0001) 849.6 (<0.0001) Preview vs. No Preview 47.92 (<0.0001) 89.83 (<0.0001) 16.63 (<0.0001) Preview × patients vs. normals n.a. 127.8 (<0.0001) 41.29 (<0.0001) Linear trend 45.39 (<0.0001) 108.3 (<0.0001) 27.83 (<0.0001) Target Present vs. Target Absent 52.93 (<0.0001) 100.6 (<0.0001) 16.63 (<0.0001)

F-values, with P-values in parentheses. Degrees of freedom = (1, 3652), mean squared error = 1.354 × 106, n.a. = not applicable.

3. Results Statistical results are summarized in Table 1. The response times of both patients were considerably longer than the 3.1. Response times normal observers, about twice as large, with an average increase of 806 ms for RW, and 536 ms for WS. As expected, Fig. 5 presents the response times of the parietal pa- the preview display aided the normal observers in the task tients and the normal observers. RW’s results are in the top by decreasing the response times by 99 ms, on average (this graphs, WS’s results are in the lower graphs, and the nor- rather large effect is concealed somewhat in Fig. 5 by the mal observers’ results are repeated in both patients’ figures. difference between the patients and normal observers). Con- trary to the effect for normal observers, the preview display actually hindered the patients’ performance, by 351 ms for RW, and 136 ms for WS, on an average. This substantially different preview effect for the patients is indicated by a significant Preview × patients versus normals interaction. All observers had an increase of response times with increasing number of items, and had slightly shorter response times with the Target Present than with the Target Absent. These results are consistent with the increased difficulty of the task expected with increasing number of search items and with target absence found by others (e.g. Treisman, 1991; Treisman & Gelade, 1980; Wolfe, 1994). The percentage of incorrect responses (not included in the analyses) reflected the same pattern of results as the response times. The normal observers had fewer errors overall, with errors in 2.4% of all the trials, compared to RW with 6.0% (P<0.005), and WS with 8.5% (P<0.005). The patterns of errors also suggest that the patients had a specific difficulty with the Preview trials. The normal observers had slightly more errors in the Preview trials compared to the No Preview trials, 2.9% and 1.9%, respectively, but this difference was not significant. Both patients, however, had a significantly higher error percentage with the preview than without the preview. For RW, this difference was 12.2% versus 1.8% (P<0.005), and for WS, this difference was 11.5% versus 5.56% (P<0.025).

3.2. Results from eye tracking

Fig. 5. Response times, in ms. The x-axes are the number of items in the The overall percentages of excluded trials due to loss of search display. Results for RW appear in the upper two graphs, and for the eye track signal were 5% for the normal observers, 29% WS in the lower two graphs. The normal observers’ results are repeated for RW, and 2% for WS. The relatively high percentage in the graphs for each patient. The bars indicate the standard errors of the of excluded trials for RW could be due to ptosis (eyelid mean. The Target Present Trials are on the left, and the Target Absent trials are on the right. Circles: Target Present; squares: Target Absent. Open droop) caused by his lesion, or his occasional facial tremors. symbols: No Preview; closed symbols: Preview. Dashed lines: patients; However, RW had no observable ptosis at the time of test- solid lines: normal observers. ing, and incidences of tremor were rare (at which point the S.S. Shimozaki et al. / Neuropsychologia 41 (2003) 1365–1386 1373 experiment was halted until the tremors subsided). It is possible that the different rates of eye track loss reflected a limitation of the capabilities of the eye tracking apparatus, which can vary substantially even for normal observers. The eye tracking results are divided into five parts: the number of fixations per trial in the search display, saccade latencies for the first and second saccades, the contribution of number of saccades versus saccade latencies to the total response time, the distance from the target after the first two saccades in the search display, and the absolute locations of the fixations after the first saccade in the search display.

3.3. Number of ﬁxations

Fig. 6 gives the results for the total number of fixations for each trial, and Table 2 gives the statistical results. Sim- ilar to the response times, both patients had substantially more fixations per trial than the normal observers, with the patients averaging about two more fixations (RW = 2.36, WS = 2.15). The normal observers had a consistent preview effect, with fewer fixations (0.516, on average) with the preview. The patients’ preview effect was weak, and depended upon the target presence, as indicated by a significant Pre- view × target presence interaction. WS had a preview effect

Fig. 7. Latencies, ﬁrst and second saccades in the search display, in ms. Normal observers: upper graph; RW: center graph; WS: lower graph. Dark symbols: latencies for the ﬁrst saccade. Light symbols: latencies for the second saccade. Circles: Target Present; squares: Target Absent. Filled symbols: Preview. Open symbols: No Preview. Dashed lines: patients; solid lines: normal observers. The bars indicate the standard errors of the mean.

only when the target was absent. RW had no preview effect, except for an opposite preview effect in the five-item Target Absent trials (P<0.0001). This corresponds to RW’s response time results, in which he appeared to have particular difficulty in the five-item Preview Target Absent trials.

3.4. Saccade latencies

Given that parietal lesions can cause an increase in the saccade latencies (Pierrot-Deseilligny et al., 1987, 1991a, 1991b), the latencies of the first and second saccades in the search display were analyzed (see Fig. 7 and Tables 3 and 4). There were two main results. First, for the patients, there was a modest increase in first saccade latencies (RW = 73 ms, WS = 35 ms), but the second saccade latencies were very close to the normals. Second, the preview increased the patients’ latencies slightly for both saccades (RW = 44 ms for both, WS = 15 and 18 ms). For normals, on the other hand, there was a small (12 ms) decrease in latency Fig. 6. The mean number of fixations during a trial. See legend for Fig. 5 resulting from the preview for the first saccade, and a small for details. increase (14 ms) for the second saccade. For the first saccade, 1374 S.S. Shimozaki et al. / Neuropsychologia 41 (2003) 1365–1386

Table 2 Number of ﬁxations, statistical results Normals RW WS

Patients vs. normals n.a. 593.4 (<0.0001) 805.6 (<0.0001) Preview vs. No Preview 71.95 (<0.0001) n.s. n.s. Preview × target presence n.s. 10.46 (0.0012) 8.140 (0.0044) Target Present trials only 39.97 (<0.0001) n.s. n.s. Target Absent trials only 32.22 (<0.0001) 6.992 (0.0082) 10.27 (0.0014) Linear trend 66.36 (<0.0001) 109.1 (<0.0001) 124.9 (<0.0001) Target Present vs. Target Absent 10.54 (0.0012) 107.5 (<0.0001) 53.42 (<0.0001) F-values, with P-values in parentheses. Degrees of freedom = (1, 3377), mean squared error = 2.338, n.a. = not applicable, n.s. = not signiﬁcant.

Table 3 First saccade latencies, statistical results Normals RW WS

Patients vs. normals n.a. 623.0 (<0.0001) 273.0 (<0.0001) Preview vs. No Preview 48.31 (<0.0001) 64.38 (<0.0001) 12.45 (0.0004) Preview × patients vs. normals n.a. 95.25 (<0.0001) 35.61 (<0.0001) Linear trend 18.95 (<0.0001) n.s. 33.73 (<0.0001) Number of items × Preview (d.f. = 2, 3375) n.s. 5.780 (0.0031) 20.70 (<0.0001) Preview trials only 11.70 (0.0006) n.s. 62.98 (<0.0001) No Preview trials only 7.434 (0.0064) n.s. n.s. Target Present vs. Target Absent n.s. n.s. n.s. F-values, with P-values in parentheses. Degrees of freedom = (1, 3375), unless indicated, mean squared error = 2085, n.a. = not applicable, n.s. = not signiﬁcant.

Table 4 Second saccade latencies, statistical results Normals RW WS

Patients vs. normals n.a. 19.94 (<0.0001) 113.4 (<0.0001) Target presence × patients vs. normals n.a. 12.35 (0.0004) 13.88 (0.0002) Target Present trials only n.a. 31.61 (<0.0001) 23.94 (<0.0001) Target Absent trials only n.a. n.s. 104.6 (<0.0001) Preview vs. No Preview 21.66 (<0.0001) 27.51 (<0.0001) 8.709 (0.0032) Linear trend n.s. n.s. 13.41 (0.0003) Number of items × Preview (d.f. = 2, 2797) n.s. n.s. 5.203 (0.0056) Preview trials only n.s. n.s. 20.74 (<0.0001) No Preview trials only 5.666 (0.0174) n.s. n.s. Target Present vs. Target Absent 36.06 (<0.0001) n.s. n.s.

F-values, with P-values in parentheses. Degrees of freedom = (1, 2797), unless indicated, mean squared error = 4.486 × 104, n.a. = not applicable, n.s. = not significant. a significant Preview × patient versus normals interaction for fixations, to longer saccade latencies, or both.1 Since the both patients indicated the opposing preview effect. For the increase in saccade latencies was small, it is not likely to be second saccade of the normal observers, the small increase a major contributor to the total reaction time. However, to in latencies probably resulted from the fact that the second assess this more precisely, a multiple linear regression was saccade was more likely to be close to the termination of performed on the response times, with the number of items, the search process in the Preview trials (due to their overall number of fixations, and first and second saccade laten- benefit in seeing the preview), and reflected some influence cies as predictors. Fig. 8 shows the partial correlations for of decision processes. the above oculomotor measures, collapsed across all trials, and Table 5 gives the statistical results. The results for the 3.5. Contribution of number of saccades to response time 1 The saccade durations themselves probably had a negligible effect, The overall longer response times of the patients shown as they typically are brief, and do not vary greatly, except slightly with in Fig. 5 could be related to either the increased number of saccadic distance. S.S. Shimozaki et al. / Neuropsychologia 41 (2003) 1365–1386 1375

possible target location, the patients typically made several fixations while searching for the target. The repetitive scanning and refixations of the objects shown here was almost never seen in normals, and was the most striking indicator of the patients’ impairment in search performance. This is in accordance to other studies showing a higher number of refixations for parietal patients in visual search and cancellation tasks (Husain et al., 2001), and a dot-counting task (Zihl & Hebel, 1997). While it is possible that the patients take longer to make the decision following fixation of the target, or have longer manual response times, it appears to be only a small component of the overall longer reaction times. If we assume a fixed fixation plus saccade duration across all fixations of Fig. 8. Partial correlations from multiple regression analyses. Circles: partial correlations of number of fixations with response time; squares: 215 ms (about the average for the normal observers), the in- partial correlations of first saccade latencies with response time; triangles: crease in number of fixations alone for the patients accounts partial correlations of second saccade latencies with response time. for an increase in response times of 507 ms for RW and 462 ms for WS. This is sufficient to account for all but about 300 ms (RW) or 75 ms (WS) of the total increase in response separate conditions were similar to the collapsed results, and times, and some of this difference can be accounted for by are not shown. For all observers, all the partial correlations increased latency of the first saccade (73 ms for RW, 35 ms were significantly different from zero, indicating that the for WS). oculomotor measures included in the analysis were significantly correlated with response time. Also, for all observers, the partial correlations for number of fixations were signif- 3.6. Distance from the target after the first and second icantly larger than those for either the first or the second saccades saccade latencies. (Significance was tested with a Fisher’s Z transformation and a subsequent pairwise comparison of the To assess more precisely the ability of the observers to use partial correlations.) As the number of items was included the preview in their saccade targeting, we measured the dis- in the analysis, the partial correlations do not include the ef- tance of the eye position from the target location after the first fect of the number of items. The results suggest that the in- and second saccades in degrees of visual angle, summarized creases in response times for the patients compared to in Fig. 10, and Table 6. This indicates how quickly eye posi- the normal observers were mainly related to an increase tion converges on the target. We first analyzed just the Target in the number of fixations. This is consistent with previ- Present trials, in which the target locations were explicitly ous studies of visual search showing response times were defined. The normal observers had a large preview effect, correlated positively to number of fixations, and not to sac- with the preview display reducing the distance from the tar- cade latencies (Scialfa et al., 1994; Zelinsky & Sheinberg, get after both saccades, indicating that they could use the pre- 1997). Fig. 9 gives an illustration of the kind of fixation pat- view information in guiding saccades to the target location. terns shown by the patients, compared to a typical normal The results for the patients clearly indicate difficulty using pattern (in two Preview Target Absent trials with five items). the preview information. Without the preview, the patients’ While the normal observers made saccades directly to the results were generally the same as the normal observers’

Table 5 Partial correlations with response time, statistical results Normals (n = 2110) RW (n = 285) WS (n = 473)

Different from zero correlationa Number of fixations with RT 32.30 (<0.0001) 16.36 (<0.0001) 15.82 (<0.0001) First saccade latency with RT 16.12 (<0.0001) 4.386 (<0.0001) 3.868 (0.0001) Second saccade latency with RT 13.09 (<0.0001) 4.008 (<0.0001) 2.628 (0.0089) Pairwise comparisonsb Number of fixations vs. first saccade latency 10.11 (<0.0001) 7.198 (<0.0001) 7.670 (<0.0001) Number of fixations vs. second saccade latency 12.15 (<0.0001) 7.459 (<0.0001) 8.539 (<0.0001) First saccade latency vs. second saccade latency 2.042 (0.0412) n.s. n.s. n.s. = not significant. a t-Values on each coefficient from multiple regression, with P-values in parentheses. b z-Scores, after Fisher’s Z transformation, with P-values in parentheses. 1376 S.S. Shimozaki et al. / Neuropsychologia 41 (2003) 1365–1386

Fig. 9. Examples of a normal scan path (top) and a scan path from patient RW (bottom) for a Preview Target Absent trial with display size equal to 5. Each circle represents a single ﬁxation, and the radii of the circles indicate the ﬁxation durations.

results, except for WS with one-item displays (P<0.0038). WS’s results were essentially the same with and without the With the preview, however, the patients’ distances from the preview. target were significantly larger than the normal observers’ It is also interesting to note that patient WS was no closer distances (with the exception of RW in the one-item dis- to the target after the second saccade than after the first. After plays). Thus, the patients did not receive the same benefit the second saccade, normal observers and RW moved closer from seeing the preview in their saccade targeting as the nor- to the target as the trial progressed, reflecting the evolution mal observers. RW’s saccades did benefit somewhat from of a correct response during a trial (Zelinsky et al., 1997). the preview, with a consistent reduction of about 1.5◦, while However, WS had errors of 4–5◦ in most conditions even S.S. Shimozaki et al. / Neuropsychologia 41 (2003) 1365–1386 1377

Fig. 10. The distances from the target after the first and second saccade, Target Present trials only. Results for RW appear in the upper two graphs, and for WS in the lower two graphs. The normal observers’ results are repeated in the graphs for each patient. Dark symbols: distances from the target after the first saccade. Light symbols: distances from the target after the second saccade. Filled circles: Preview Target Present. Open circles: No Preview Target Present. Dashed lines: patients; solid lines: normal observers. The bars indicate the standard errors of the mean. after the second saccade, suggesting that he was acutely by Fig. 11 (with statistical results summarized in Table 7), impaired in his ability to use peripheral information to guide the distances from the preview indicate the patients’ diffi- his eyes to the target. culty in using the preview display. Compared to the normal observers, both patients clearly showed larger distances 3.7. Distance from the preview location in the Preview from the preview location. The one exception is RW’s first Target Absent trials saccade in the one-item displays, consistent with his results for the distances from the target. Thus, RW’s tar- While the distances from the target locations cannot be geting deficit was apparent only for the larger three- and measured in the Target Absent trials, there were relevant five-item displays, suggesting that he could use the pre- (previewed) locations during the Target Absent trials with a view display when it was limited to remembering a single preview. If the observers could remember the preview dis- location. play, they knew to look for the target in the search display in For all observers, the distances from the preview loca- the same location as the target in the preview display, even tion after the first and second saccades did not differ, un- though the target did not appear in the search displays on like the distances from the target. One possibility is that the these trials. Thus, in addition to the distances from the target absence led to a different trial termination strategy target, the observers’ ability to use the preview may be as- for both the normal and patient observers. Target absence sessed by analyzing the distance from the preview loca- might have led observers to confirm target absence through- tion in the Target Absent trials with the preview. As shown out the search display, as opposed to ending their search 1378 S.S. Shimozaki et al. / Neuropsychologia 41 (2003) 1365–1386

Table 6 Distance from the target, ﬁrst and second saccades, statistical results Normals RW WS

(a) First saccade Patients vs. normals n.a. 14.838 (0.0001) 83.06 (<0.0001) Preview, Target Present n.a. 6.590 (0.0103) 119.6 (<0.0001) No Preview, Target Present n.a. n.s. n.s. Preview vs. No Preview 342.4 (<0.0001) 14.85 (0.0001) n.s. Linear trend 125.1 (<0.0001) 22.09 (<0.0001) 8.448 (0.0037) Second saccade Patients vs. normals n.a. 16.78 (<0.0001) 360.6 (<0.0001) Preview, Target Present n.a. 4.715 (0.0300) 202.8 (<0.0001) No Preview, Target Present n.a. 16.15 (<0.0001) 158.1 (<0.0001) Preview vs. No Preview 35.89 (<0.0001) 7.698 (0.0056) n.s. Linear trend 54.05 (<0.0001) 14.03 (0.0002) 5.567 (0.0184) (b) First saccade vs. second saccade Overall 394.8 (<0.0001) 26.63 (0.0001) n.s. Preview, Target Present 69.95 (<0.0001) 7.858 (0.0051) n.s. No Preview, Target Present 426.8 (<0.0001) 24.17 (<0.0001) n.s. F-values, with P-values in parentheses. Degrees of freedom = (1, 3062), mean squared error = 4.494, n.a. = not applicable, n.s. = not significant. immediately upon finding the target in the Target Present target positions with increasing number of items. The clus- displays (e.g. Treisman & Gelade, 1980). tering of fixations in the center of the display in the No Pre- view trials appear to be evidence of “center-of-gravity” fix- 3.8. Spatial bias after the first saccade? ations (Findlay, 1982) found also by Zelinsky et al. (1997) in visual search. Zelinsky et al. (1997) suggest that these Several authors have studied the effect of the syndrome of fixations might indicate the initial weighting of the overall hemineglect on eye movements, with the general finding that amount of information in the visual display for saccadic tar- eye movements are biased away from the neglected hemi- get selection during search. field (Barton et al., 1998; Behrmann et al., 1997; Chedru Fig. 13a and b give the scatterplots of the fixation after et al., 1973; Ishiai et al., 1992; Karnath, 1994; Walker & the first saccade in the search display for the patients. Both Findlay, 1996), consistent with the direction of the supposed patients showed some influence of the preview on the scat- attentional deficit. Analyses of the absolute eye position af- terplots. For RW, the fixations moved closer to the target ter the first saccade were performed to assess any spatial positions, reflecting the smaller distances from the target in bias in the patients’ eye movements due to the parietal le- the Preview trials shown in Fig. 10. For WS, the preview sions. Fig. 12 gives the scatterplots of eye position after the changed the pattern of landing points, but did not make them first saccade in the search display for the normal observers, closer to the target. This change of pattern suggests that he with the dashed boxes indicating the possible target loca- was aware of the preview, and that he did not (or could not) tions. The normal observers’ fixations clearly moved closer simply ignore the preview display. to the target locations for the Preview trials, compared to For all the observers, the fixations appear to move to the the No Preview trials, corresponding to the positive preview left with increasing number of items in the search display. effect found for the distances from the target. Also, consis- This is more clearly seen in Fig. 14 and Table 8, which give tent with the linear trends found for the distances from the the deviations from the vertical midline for the first fixation target, the fixations tended to move further away from the after the first saccade in the search display. In degrees of visual angle, positive values indicate deviations to the right, and negative values indicate deviations to the left. As sug- Table 7 gested by the scatterplots, the deviations tended to move to Distance from the preview location, first and second saccades, statistical the left with increasing number of items, with significant lin- results ear trends found for all observers. One possible explanation Normals RW WS is an increased reliance on a left-to-right serial scanning, or Patients vs. n.a. 50.83 (<0.0001) 146.2 (<0.0001) ‘reading’, strategy, as the visual search task became more normals difficult (Zelinsky, 1996). For RW in the No Preview tri- < < Linear trend 80.15 ( 0.0001) 34.76 ( 0.0001) 4.260 (0.0392) als, his leftward bias was larger than the normal observers’ F-values, with P-values in parentheses. Degrees of freedom = (1, 1425), biases (due to the difference in the three- and five-item mean squared error = 6.316, n.a. = not applicable. S.S. Shimozaki et al. / Neuropsychologia 41 (2003) 1365–1386 1379

Fig. 11. The distances from the preview location target after the first and second saccade, Preview Target Absent trials only. Results for RW appear in the upper two graphs, and for WS in the lower two graphs. The normal observers’ results are repeated in the graphs for each patient. Dark symbols: distances from the preview location after the first saccade. Light symbols: distances from the preview location after the second saccade. Dashed lines: patients; solid lines: normal observers. The bars indicate the standard errors of the mean. displays, P<0.0001). This result is consistent with pat- ticeable neglect of the right visual field. These results, and tern of hemineglect caused by his left lesion. However, this the relatively uncommon occurrence of hemineglect for left bias was not found in the one-item trials, and extensive neu- parietal lesions, suggest that the increased bias for RW was ropsychological testing on RW also did not reveal any no- not due to hemineglect. In the No Preview Target Present

Table 8 Deviations of the ﬁrst saccade from the vertical meridian, statistical results Normals RW WS

Patients vs. normals n.a. 5.193 (0.0227) n.s. Preview × patients vs. normals n.a. 5.093 (0.0241) n.s. Preview trials only n.a. n.s. n.s. No Preview trials only n.a. 13.82 (0.0002) n.s. Preview vs. No Preview 13.68 (0.0002) 12.65 (0.0004) n.s. Preview × target presence n.s. n.s. n.s. Target Present trials only 4.024 (0.0449) 13.32 (0.0003) n.s. Target Absent trials only 10.13 (0.0003) n.s. n.s. Linear trend 35.83 (<0.0001) 13.17 (0.0003) 21.50 (<0.0001) Target Present vs. Target Absent n.s. n.s. n.s. F-values, with P-values in parentheses. Degrees of freedom = (1, 3380), mean squared error = 9.797, n.a. = not applicable, n.s. = not signiﬁcant. 1380 S.S. Shimozaki et al. / Neuropsychologia 41 (2003) 1365–1386

Fig. 12. Scatter plots for the end point of the first saccade in the search display, normal observers. The Preview conditions appear on the left, and the No Preview conditions appear on the right. Each graph presents results by number of items in the search display. The dashed boxes indicate the possible locations of the targets. trials with and three and five items, WS’s deviations were effects were specific to damage to the parietal cortex,2 the also marginally more to the left than the normal observers results suggest that the parietal cortex has a role in these two (P<0.04), opposite to the pattern predicted by hemine- processes. Compared to the normal observers over all con- glect from his right parietal lesion. For the normal observers, ditions, the patients showed an increase in response times, and for RW in the Target Present trials, the No Preview number of fixations, and first saccade latencies, and the pa- trials were significantly more to the left than the Preview tients were farther from the target than normals after the trials. second saccade. These results suggest that the patients appeared to have a general appearance-based difficulty with the visual search task. In addition, the patients had a specific 4. General discussion difficulty with using the preview information, suggesting a memory-based deficit in their visual search. For response A visual search task in a realistic setting may be character- times, number of fixations, and first saccade latencies, the ized as having two components, one founded on the current normal observers received a strong benefit from the preview, visual stimulus, or an appearance-based component, and an- whereas the patients generally showed either a disadvantage other founded on previously viewed spatial information, or or no advantage. Also, both the distances from the target in a spatial memory-based component. We studied the effect the Target Present trials, and the distances from the preview of parietal lesions on both of these processes, as evidenced location in the Target Absent trials with a preview, indicated by the effect on their saccadic targeting. Of particular interest is the spatial memory-based process, which has been 2 In fact, it seems likely that other areas such as the FEF (Andersen studied less extensively in previous studies of visual search. et al., 1990; Cavada & Goldman-Rakic, 1989; Schall et al., 1995), superior colliculus (Andersen et al., 1990; Fries, 1984; Lynch et al., 1985), and The two parietal patients clearly demonstrated deficits in dorsolateral prefrontal cortex (Funahashi, Bruce, & Goldman-Rakic, 1989; both the appearance- and memory-based aspects of visual Jonides et al., 1993; McCarthy et al., 1994; Moscovitch, Kapur, Kohler, search. While it cannot be assumed from this study that the & Houle, 1995) are involved. S.S. Shimozaki et al. / Neuropsychologia 41 (2003) 1365–1386 1381

Fig. 13. Scatter plots for the end point of the ﬁrst saccade in the search display, patients. Results for RW appear in (a), and for WS in (b). See legend for Fig. 11 for details. 1382 S.S. Shimozaki et al. / Neuropsychologia 41 (2003) 1365–1386

Fig. 14. Horizontal deviations of the first saccade from the vertical meridian, in degrees of visual angle. Positive: rightward deviations; negative: leftward deviations. See legend for Fig. 5 for other details. that the patients did not receive the preview benefit in their logical studies of eye movements (Gnadt & Andersen, 1988). saccade targeting that the normal observers did. Typically in this task the observer fixates a central location Unlike the other variables, the patients generally had sec- and is shown briefly a peripheral target location. After a de- ond saccade latencies that were only slightly larger or less lay period, the observer makes a saccade to the target loca- than the normal observers’. Thus, the patients’ increase in tion, forcing the observer to retain that location throughout first saccade latencies did not extend to the second saccade, the delay period. As in delayed saccade task, the observers suggesting that the patients’ difficulty with search did not had to maintain a target location in memory for a brief pe- arise from a deficit in initiating saccades, consistent with riod (1 s) before the search display appeared. The behavioral the clinical observations. Also, most of the increase in re- task differed, with the observer responding to target pres- sponse times can be ascribed to the increase in the number ence, instead of making a saccade, and the observers had to of fixations. This suggests a difficulty with the target selec- remember up to five spatial locations. Also, most delayed tion process (or, in other words, the mechanism that uses the saccade tasks are performed without any direct visual guid- peripheral visual information to guide the eye movements), ance, while in this task, at least in the Target Present trials, rather than, again, the mechanics of the eye movement itself. the target was present in the periphery. The oculomotor be- This is consistent with physiological and clinical studies that havior of the normal observers, nonetheless, mirrored that of indicate the parietal cortex’s role in the target selection of the monkeys in the delayed saccade task, making saccades saccades discussed earlier. directly to the targets after viewing the preview. The visual search task with the preview bears some simi- With one-item displays, RW’s targeting performance as larity to the delayed saccade task used commonly in physio- measured by the distance from the target, was close to the S.S. Shimozaki et al. / Neuropsychologia 41 (2003) 1365–1386 1383 normal observers (Figs. 10 and 11). Thus, it appears that his of 24 objects. After the 10 s inspection, the observers were use of spatial memory was taxed with the larger displays, asked to recall as many locations or objects as possible. As and not with the one-item displays. Also, the preview did as- expected, WS could not recall as many locations as the nor- sist his saccade targeting in the three- and five-item displays mal observers (3.6 versus 5.4), but could recall just as many to some extent (see Fig. 10), though not at the level of the objects (5.7 versus 5.9). normal observers. This suggests that RW maintained some ability to use the preview information for targeting saccades, 4.1. Effect of spatial preview as opposed to WS, who showed little effect on saccade targeting with the preview. Possibly these differences in deficit As mentioned earlier, other studies have shown difficulty are due to hemispheric differences, as the right parietal with short-term spatial memory (De Renzi & Nichelli, 1975; lobe is assumed to be more specialized for spatial informa- De Renzi et al., 1969, 1977) and memory-guided saccades tion (Battersby, Bender, Pollack, & Kahn, 1956; Bisiach, after parietal lesions (Husain et al., 2001; Pierrot-Deseilligny Cornacchia, Sterzi, & Vallar, 1984; Vallar, Rusconi, et al., 1991b; Zihl & Hebel, 1997), and memory-guided sac- Geminiani, Berti, & Cappa, 1991). Other factors might cades after temporary parietal inactivation by TMS (Muri include age, and the sizes and etiologies of the lesions. et al., 1996). Also, fMRI studies of spatial memory tasks For WS, age might have lead to some general memory (Courtney, Ungerleider, Keil, & Haxby, 1996; McCarthy deficits, or a general loss in oculomotor ability, which also et al., 1994; Mountcastle et al., 1975) and memory-guided affected his performance in the visual search task. Typically, saccades (Heide et al., 2001) show activation of the pari- reported increases in response times for aged observers in etal cortex. Finally, several physiological studies also sug- visual search (first noted by Rabbitt (1965)) vary with diffi- gest that neurons in LIP hold the memory of a target lo- culty of the search, from about 100 to 1000 ms (e.g. Madden, cation for a saccade (Andersen et al., 1987; Colby et al., Pierce, & Allen, 1996; Plude & Doussard-Roosevelt, 1989; 1995, 1996; Gnadt & Andersen, 1988; Goldberg, Colby, & Plude & Hoyer, 1989; Scialfa, Esau, & Joffe, 1998; Scialfa Duhamel, 1990). Thus, it seems likely that the parietal cor- & Joffe, 1997), well within the range found for WS. Also, tex had a role in the visual search task with the preview for a study by Scialfa et al. (1994), found that aging increases the normal observers. Also, the two patients with parietal le- in response time correlated well with the number of fixa- sions had difficulty with the preview information, in which tions. For general oculomotor ability, consistent aging effects they were disadvantaged by looking at the preview display, have been found for peak saccadic velocity (Abel, Troost, consistent with the assumed function of the parietal lobe. & Dell Osso, 1983; Spooner, Sakala, & Baloh, 1980), pre- Given that the preview information is not necessary for the saccadic spike potential (Doig & Boylan, 1989), and sac- behavioral task of target presence, however, and only adds cade latencies, typically from 30 to 85 ms (Abel et al., 1983; information, the patients’ impairment of performance by the Carter, Obler, Woodward, & Albert, 1983; Scialfa, Hamaluk, preview is somewhat surprising, particularly in the 1-item Skaloud, & Pratt, 1999; Spooner et al., 1980). Also, the re- displays in which there were no competing items for target- sults of one study suggest that older observers are slightly ing saccades. What is the basis for the impaired performance more prone to ‘averaging’ (center-of-gravity) saccadic tar- with the spatial preview? geting errors when conflicting stimuli are presented with a As mentioned earlier, a common concept in the modeling saccade target (Scialfa et al., 1999). Some aspects of WS’s of saccades is a spatial salience map encoding the informa- results suggest a clear spatial memory deficit, however, be- tion guiding saccades (Findlay & Walker, 1999; Rao et al., yond a generalized aging effect. First, a general aging deficit 2002; Wolfe, 1994), with the salience of a particular loca- does not seem to wholly explain the reversed effect of the tion related to its visual characteristics (appearance), previ- preview on WS’s results (increase of response time with pre- ous information or exposure (memory), and/or behavioral view), or the lack of difference between WS and the normal relevance (reward or punishment). In particular, Gottlieb observers in the second saccade latencies. Also, WS’s neu- et al. (1998) suggest that the parietal cortex might hold a ropsychological results suggest that he retained an above av- general spatial salience map, through its localized activity, erage level of both memory and cognitive function, both for which could guide saccades. A possible means to describe his age and compared to a normal adult population, while the patients’ deficit is through a change in a salience map in also suggesting a specific spatial memory deficit. Finally, a parietal cortex. Assuming the existence of a salience map, previous study we performed on WS showed clear spatial a plausible decision rule would be to choose the location memory (and not object memory) deficits (Shimozaki et al., with the maximum activity (and hence, salience) as the lo- 1997). In this study, we compared WS to normal observers in cation for the next saccade. For the normal observers, the the 7/24 Spatial Recall Test (Rao et al., 1984), and an equiv- preview display might be seen as adding activity (salience) alent object memory test with a base set of 24 line drawings only to the target location, while for the patients, the pre- of objects chosen from a larger standard set (Snodgrass & view information, being poor in quality, adds activity across Vanderwart, 1980). In both tests, observers were given 10 s the salience map (‘noise’), and not specifically to the tar- to inspect a display, either 7 checkers placed randomly in get location. Thus, the preview information aids the normal a grid with 24 locations, or 7 objects chosen from the set observers in making the peak activity more likely to be the 1384 S.S. Shimozaki et al. / Neuropsychologia 41 (2003) 1365–1386 target location, and thus easier to find, and hurts the pa- indicate that patients with hemineglect may also exhibit ocu- tients in making the peak activity less likely to be the tar- lomotor behaviors suggesting deficits in spatial memory. get location, and thus harder to find. This concept of the ‘signal-to-noise’ ratio guiding visual behavior has been em- ployed in different contexts in visual perception (Green & Acknowledgements Swets, 1974), and has been used to model visual search and saliency (Eckstein, 1998; Eckstein, Thomas, Palmer, & This study was supported by grants from the National Shimozaki, 2000; Palmer, 1994, 1995; Palmer, Verghese, & Institutes of Health (EY-05729 and EY-06742). The au- Pavel, 2000; Shaw, 1984). One aspect of the preview infor- thors wish to thank Deborah Bancroft, Russell Loeber, and mation is that it appears to be obligatory, and cannot be ig- Christopher Chizk for their assistance, Darrell Anderson, nored by the patients, even if it degrades their performance. Ben Singer and Yasser Mufti for programming assistance, If, as mentioned earlier, salience is the combination of infor- and David Bensinger, Keith Karn, and Rajesh Rao for par- mation from vision, memory, and behavioral relevance, per- ticipating as observers. haps combining these sources of information into a single salience map, by default, prevents the exclusion of a single source of information. References It is possible, in fact, that such salience noise would con- tinue to affect performance throughout the trial. Another as- Abel, L. A., Troost, B. T., & Dell Osso, L. F. (1983). The effects of age on pect of the patients’ behavior was the increased number of normal saccadic characteristics and their variability. Vision Research, fixations, and, in particular, the curious apparent pattern of 23, 33–37. fixations on previously viewed locations (see Fig. 9). While Andersen, R. A. (1995). Encoding of intention and spatial location in the posterior parietal cortex. Cerebral Cortex, 5, 457–469. the refixations might suggest difficulties with identifying the Andersen, R. A., Asanuma, C., Essick, G., & Siegel, R. M. (1990). target objects, the neuropsychological testing of the two pa- Corticocortical connections of anatomically and physiologically defined tients did not indicate a specific deficit in object identifi- subdivisions within the inferior parietal lobule. Journal of Comparative cation (visual agnosia). Also, as mentioned earlier, another Neurology, 296, 65–113. study based on the 7/24 Spatial Recall Test (Rao et al., Andersen, R. A., Essick, G. K., & Siegel, R. M. (1987). Neurons of area 7 activated by both visual stimuli and oculomotor behavior. Experimental 1984) found evidence for a specific spatial (and not object) Brain Research, 67, 316–322. memory deficit for WS (Shimozaki et al., 1997). One possi- Barton, J. J. S., Behrmann, M., & Black, S. E. (1998). Ocular search ble explanation is that degraded information in the salience during line bisection: The effects of hemi-neglect and hemianopia. map lead to spurious peaks, which in turn generate saccades Brain, 121, 1117–1131. to these locations during search. Thus, the spurious peaks Battersby, W. S., Bender, M. B., Pollack, M., & Kahn, R. L. (1956). Unilateral “spatial agnosia” (“inattention”) in patients with cerebral might continually drag the eye away to a new target before lesions. Brain, 79, 68–93. the decision process (with respect to target presence) is com- Behrmann, M., Watt, S., Black, S. E., & Barton, J. J. S. (1997). Impaired plete, increasing the number of fixations, including those to visual search in patients with unilateral neglect: An oculographic previously fixated locations, needed to perform the task. analysis. Neuropsychologia, 35, 1445–1558. Benowitz, L. I., Moya, K. L., & Levine, D. N. (1990). Impaired verbal 4.2. Relationship to hemineglect reasoning and constructional apraxia in subjects with right hemisphere damage. Neuropsychologia, 28, 231–244. Benton, A. L. (1967). Constructional apraxia and the minor hemisphere. While the patients’ lesions were in opposite hemispheres, Confinia Neurologica, 29, 1–16. in general, the patterns of results for the two patients in the Benton, A. L., & Hamsher, K. deS. (1989). Multilingual aphasia visual search task were similar. Also, analyses including the examination. Iowa City, IA: AJA Associates. hemifield of the target showed no consistent effect of hemi- Benton, A. L., Hamsher, K. deS., Varney, N. R., & Spreen, O. (1983). Contributions to neuropsychological assessment. New York: Oxford field for either patient. An asymmetry in the spatial process- University Press. ing of the two parietal lobes, with the right parietal lobe Bisiach, E., Cornacchia, L., Sterzi, R., & Vallar, G. (1984). Disorders of having a larger role than the left, has been assumed based on perceived auditory lateralization after lesions of the right hemisphere. the neurological finding that hemineglect is more common Brain, 107, 37–52. with right parietal lobe lesions (e.g. Battersby et al., 1956; Carter, J. E., Obler, L., Woodward, S., & Albert, M. L. (1983). The effect Bisiach et al., 1984; Vallar et al., 1991). As neither patient of increasing age on the latency of saccadic eye movements. Journal of Gerontology, 38, 318–320. showed any signs of hemineglect, this asymmetry may not Cavada, C., & Goldman-Rakic, P. S. (1989). Posterior parietal cortex in be relevant to these patients, and it is possible that this study rhesus monkey. II. Evidence for segregated corticocortical networks addresses a different function of the parietal lobe than that linking sensory and limbic areas with the frontal lobe. Journal of revealed by hemineglect. Also, differences in the sizes and Comparative Neurology, 287, 422–445. locations of the lesions may account for the apparent lack of Chedru, F., Leblanc, M., & Lhermitte, F. (1973). Visual searching in normal and brain-damaged subjects (contribution to the study of asymmetry. While the two current patients appear not to fit unilateral inattention). Cortex, 9, 94–111. into the syndrome of neglect, other hemineglect studies pre- Colby, C. L., Duhamel, J. R., & Goldberg, M. E. (1995). Oculocentric viously mentioned (Husain et al., 2001; Zihl & Hebel, 1997) spatial representation in parietal cortex. Cerebral Cortex, 5, 470–481. S.S. Shimozaki et al. / Neuropsychologia 41 (2003) 1365–1386 1385

Colby, C. L., Duhamel, J. R., & Goldberg, M. E. (1996). Visual, Heide, W., Binkofski, F., Seitz, R. J., Posse, S., Nitshchke, M. F., Freund, presaccadic, and cognitive activation of single neurons in monkey H. -J., & Kompf, D. (2001). Activation of frontoparietal cortices during lateral intraparietal area. Journal of Neurophysiology, 76, 2841–2852. memorized triple-step sequences of saccadic eye movements: An fMRI Corbetta, M., Akbudak, E., Conturo, T. E., Snyder, A. Z., Ollinger, J. study. European Journal of Neuroscience, 13, 1177–1189. M., Drury, H. A., Linenweber, M. R., Peterson, S. E., Raichle, M. E., Heide, W., Blankenburg, M., Zimmerman, E., & Kompf, D. (1995). Van Essen, D. C., & Shulman, G. L. (1998). A common network of Cortical control of double-step saccades: Implications for spatial functional areas for attention and eye movements. Neuron, 21, 761– orientation. Annals of Neurology, 38, 739–748. 773. Heide, W., & Kompf, D. (1998). Combined deficits of saccades and Courtney, S. M., Ungerleider, L. G., Keil, K., & Haxby, J. V. (1996). visuo-spatial orientation after cortical lesions. Experimental Brain Object and spatial visual working memory activate separate neural Research, 123, 164–171. systems in human cortex. Cerebral Cortex, 6, 39–49. Husain, M., Mannan, S., Hodgon, T., Wojciulik, E., Driver, J., & Darby, D. G., Nobre, A. C., Thangaraj, V., Edelman, R., Mesulam, M. Kennard, C. (2001). Impaired spatial working memory across saccades -M., & Warach, S. (1996). Cortical activation in the human brain contributes to abnormal search in parietal neglect. Brain, 5, 941–952. during lateral saccades using EPISTAR functional magnetic resonance Ishiai, S., Sugishita, M., Mitani, K., & Ishizawa, M. (1992). Leftward imaging. NeuroImage, 3, 53–62. search in left unilateral spatial neglect. Journal of Neurology, De Renzi, E., Faglioni, P., & Previdi, P. (1977). Spatial memory and Neurosurgery, and Psychiatry, 55, 40–44. hemispheric locus of lesion. Cortex, 13, 424–433. Jonides, J., Smith, E. E., Koeppe, R. A., Awh, E., Minoshima, S., & De Renzi, E., Faglioni, P., & Scotti, G. (1969). Impairment of memory Mintun, M. A. (1993). Spatial working memory in humans as revealed for position following brain damage. Cortex, 5, 274–284. by PET. Nature, 363(17), 623–625. De Renzi, E., & Nichelli, P. (1975). Verbal and non-verbal short-term Karnath, H. -O. (1994). Spatial limitation of eye movements during ocular memory impairment following hemispheric damage. Cortex, 11, 341– exploration of simple line drawings in neglect syndrome. Cortex, 30, 354. 319–330. Doig, H. R., & Boylan, C. (1989). Changes in presaccadic spike potential Luna, B., Thulborn, K. R., Strojwas, M. H., McCurtain, B. J., Berman, with age. Electroencephalography and Clinical Neurophysiology, 73, R. A., Genovese, C. R., & Sweeney, J. A. (1998). Dorsal cortical 549–551. regions subserving visually guided saccades in humans: An fMRI Duhamel, J. -R., Colby, C. L., & Goldberg, M. E. (1992). The updating study. Cerebral Cortex, 8, 40–47. of the representation of visual space in parietal cortex by intended eye Lynch, J. C., Graybiel, A. M., & Lobeck, L. B. (1985). The differential movements. Science, 255(3), 90–92. projection of two cytoarchitectonic subregions of the inferior parietal Duhamel, J. -R., Goldberg, M. E., Fitzgibbon, E. J., Sirigu, A., & Grafman, lobule of macaque upon the deep layers of the superior colliculus. J. (1992). Saccadic dysmetria in a patient with a right frontoparietal Journal of Comparative Neurology, 235, 241–254. lesion: The importance of corollary discharge for accurate spatial Madden, D. J., Pierce, T. W., & Allen, P. A. (1996). Adult age differences behaviour. Brain, 115, 1387–1402. in the use of distractor homogeneity during visual search. Psychology Eckstein, M. P. (1998). The lower efficiency for conjunctions is due to and Aging, 11, 454–474. noise and not serial attentional processing. Psychological Science, 9, McCarthy, G., Blamire, A. M., Puce, A., Nobre, A. C., Bloch, G., Hyder, 111–118. F., Goldman-Rakic, P., & Shulman, R. G. (1994). Functional magnetic Eckstein, M. P., Thomas, J. P., Palmer, J., & Shimozaki, S. S. (2000). resonance imaging of human prefrontal cortex: Activation during a A signal detection model predicts effects of set size on visual search spatial working memory task. Proceedings of the National Academy accuracy for feature, conjunction, triple conjunction and disjunction of Sciences of the United States of America, 91, 8690–8694. displays. Perception & Psychophysics, 62, 425–451. Moscovitch, M., Kapur, S., Kohler, S., & Houle, S. (1995). Distinct Epelboim, J., Steinman, R. M., Kowler, E., Edwards, M., Pizlo, Z., neural correlates of visual long-term memory for spatial location and Erkelens, C. J., & Collewijn, H. (1995). The function of visual search object identity: A positron emission tomography study in humans. and memory in sequential looking tasks. Vision Research, 35, 3401– Proceedings of the National Academy of Sciences of the United States 3422. Findlay, J. M. (1982). Global visual processing for saccadic eye of America, 92, 3721–3725. movements. Vision Research, 22, 1003–1045. Mountcastle, V. B., Lynch, J. C., Georgopoulos, A., Sakata, H., & Findlay, J. M., & Gilchrist, I. D. (1998). Eye guidance and visual search. Acuna, C. (1975). Posterior parietal association cortex of the monkey: In G. Underwood (Ed.), Eye guidance in reading and scene perception Command functions for operations within extrapersonal space. Journal (pp. 295–312). Amsterdam, The Netherlands: Elsevier. of Neurophysiology, 38, 871–905. Findlay, J. M., & Walker, R. (1999). A model of saccade generation Muri, R. M., Vermersch, A. -I., Rivaud, S., Gaymard, B., & based on parallel processing and competitive inhibition. Brain and Pierrot-Deseilligny, C. (1996). Effects of single-pulse transcranial Behavioral Sciences, 22, 661–721. magnetic stimulation over the prefrontal and posterior parietal cortices Fries, W. (1984). Cortical projections to the superior colliculus in the during memory-guided saccades. Journal of Neurophysiology, 76, macaque monkey: A retrograde study using horseradish peroxidase. 2102–2106. Journal of Comparative Neurology, 230, 55–76. Palmer, J. (1994). Set-size effects in visual search: The effect of attention Funahashi, S., Bruce, C. J., & Goldman-Rakic, P. S. (1989). Mnemonic is independent of the stimulus for simple tasks. Vision Research, 34, coding of visual space in the primate dorsolateral prefrontal cortex. 1703–1721. Journal of Neurophysiology, 61, 814–833. Palmer, J. (1995). Attention in visual search: Distinguishing four causes Gnadt, J. W., & Andersen, R. A. (1988). Memory related motor planning of set-size effects. Current Directions in Psychological Science, 4, activity in posterior parietal cortex of macaque. Experimental Brain 118–123. Research, 70, 216–220. Palmer, J., Verghese, P., & Pavel, M. (2000). The psychophysics of visual Goldberg, M. E., Colby, C. L., & Duhamel, J. -R. (1990). The search. Vision Research, 40, 1227–1268. representation of visuomotor space in the parietal lobe of the monkey. Petit, L., Clark, V. P., Ingeholm, J., & Haxby, J. V. (1997). Dissociation Cold Spring Harbor Symposia on Quantitative Biology, 55, 729–739. of saccade-related and pursuit-related activation in human frontal eye Gottlieb, J. P., Kusunoki, M., & Goldberg, M. E. (1998). The fields as revealed by fMRI. Journal of Neurophysiology, 77, 3386– representation of visual salience in monkey parietal cortex. Nature, 3390. 391(29), 481–484. Pierrot-Deseilligny, C., Rivuad, S., Gaymard, B., & Agid, Y. (1991a). Green, D. M., & Swets, J. A. (1974). Signal detection theory and Cortical control of reflexive visually-guided saccades. Brain, 114, psychophysics. Los Altos, CA: Peninsula Publishing. 1473–1485. 1386 S.S. Shimozaki et al. / Neuropsychologia 41 (2003) 1365–1386

Pierrot-Deseilligny, C., Rivuad, S., Gaymard, B., & Agid, Y. (1991b). Shimozaki, S., Weinstein, A., Hayhoe, M., Merigan, W., Zelinsky, G., & Cortical control of memory-guided saccades in man. Experimental Ballard, D. (1997). Eye tracking of a right parietal patient and normal Brain Research, 83, 607–617. subjects during the 7/24 Spatial Recall Test and object memory tests. Pierrot-Deseilligny, C., Rivuad, S., Penet, C., & Rigolet, M. -H. (1987). Brain and Cognition, 35, 388–392. Latencies of visually guided saccades in unilateral hemispheric cerebral Snodgrass, J. G., & Vanderwart, M. (1980). A standardized set of 260 lesions. Annals of Neurology, 21, 138–148. pictures: Norms for name agreement, image agreement, familiarity, Plude, D. J., & Doussard-Roosevelt, J. A. (1989). Aging, selective and visual complexity. Journal of Experimental Psychology: Human attention, and feature integration. Psychology and Aging, 4, 98–105. Learning and Memory, 6, 174–215. Plude, D. J., & Hoyer, W. J. (1989). Age and the selectivity of visual Spooner, J. W., Sakala, S. M., & Baloh, R. W. (1980). Effect of aging information processing. Journal of Psychology and Aging, 1, 4–10. on eye tracking. Archives of Neurology, 37, 575–576. Rabbitt, P. (1965). An age-decrement in the ability to ignore irrelevant Treisman, A. M. (1991). Search, similarity and integration of features information. Journal of Gerontology, 20, 233–237. between and within dimensions. Journal of Experimental Psychology: Rafal, R. D. (1994). Neglect. Current Opinion in Neurobiology, 4, 231– Human Perception and Performance, 17, 652–675. 236. Treisman, A. M., & Gelade, G. (1980). A feature integration theory of Rao, S. M., Hammeke, T. A., McQuillen, M. P., Kharti, B. O., & Lloyd, D. attention. Cognitive Psychology, 12, 97–136. (1984). Memory disturbance in chronic progressive multiple sclerosis. Vallar, G., Rusconi, M. L., Geminiani, G., Berti, A., & Cappa, S. F. (1991). Archives of Neurology, 41, 625–631. Visual and nonvisual neglect after unilateral brain lesions: Modulation Rao, R. P., Zelinsky, G. J., Hayhoe, M. M., & Ballard, D. H. (2002). Eye by visual input. International Journal of Neuroscience, 61, 229– movements in iconic visual search. Vision Research, 42, 1447–1463. 239. Ruessmann, K., Sondag, H. -D., & Beneike, U. (1988). On the Walker, R., & Findlay, J. M. (1996). Saccadic eye movement programming cerebral localization of constructional apraxia. International Journal in unilateral neglect. Neuropsychologia, 34, 493–508. of Neuroscience, 42, 59–62. Wechsler, D. (1981). Wechsler adult intelligence scale—revised manual. Schall, J. D., Morel, A., King, D. J., & Bullier, J. (1995). Topography New York: The Psychological Corporation. of visual cortex connections with frontal eye ﬁeld in macaque: Wechsler, D. (1987). Wechsler memory scale—revised manual. San Convergence and segregation of processing streams. Journal of Antonio, TX: The Psychological Corporation. Neuroscience, 15, 4464–4487. Wolfe, J. M. (1994). Guided search 2.0: A revised model of visual search. Scialfa, C. T., Esau, S. P., & Joffe, K. M. (1998). Age, target–distractor si- Psychonomic Bulletin and Review, 1, 202–238. milarity, and visual search. Experimental Aging Research, 24, 337–358. Woodward, J. A., Bonett, D. G., & Brecht, M. L. (1990). Introduction Scialfa, C. T., Hamaluk, E., Skaloud, P., & Pratt, J. (1999). Age differences to linear models and experimental design. New York: Harcourt Brace in saccadic averaging. Psychology and Aging, 14, 695–699. Jovanovich. Scialfa, C. T., & Joffe, K. M. (1997). Age differences in feature and Zelinsky, G. J. (1996). Using eye saccades to assess the selectivity of conjunction search: Implications for theories of visual search and gene- search movements. Vision Research, 36, 2177–2187. ralized slowing. Aging, Neuropsychology, and Cognition, 4, 227–246. Zelinsky, G. J., Rao, R. P. N., Hayhoe, M. M., & Ballard, D. H. (1997). Scialfa, C. T., Thomas, D. M., & Joffe, K. M. (1994). Age differences in Eye movements reveal the spatiotemporal dynamics of visual search. the useful ﬁeld of view: An eye movement analysis. Optometry and Psychological Science, 8, 448–453. Vision Science, 71, 736–742. Zelinsky, G. J., & Sheinberg, D. L. (1997). Eye movements during Shaw, M. L. (1984). Division of attention among spatial locations: A parallel–serial visual search. Journal of Experimental Psychology: fundamental difference between detection of letters and detection of Human Perception and Performance, 23, 244–262. luminance increments. In H. Bouma, & D. G. Bouwhuis (Eds.), Zihl, H., & Hebel, N. (1997). Patterns of oculomotor scanning in Attention and performance (Vol. X, pp. 109–121). Hillsdale, NJ: patients with unilateral posterior parietal or frontal lobe damage. Erlbaum. Neuropsychologia, 35, 893–906.