Matching Two Imagined Clocks: the Luigi Trojano, Dario Grossi1, David E.J. Linden2,3,Elia Formisano4,5, Hans Hacker5, Friedhelm E. Zanella5, Rainer Functional Anatomy of Spatial Analysis in Goebel2 and Francesco Di Salle5,6 the Absence of Visual Stimulation Salvatore Maugeri Foundation, IRCCS, Center of Telese, Loc. S. Stefano in Lanterria, I-82037 Telese Terme (BN), Departments of 1Neurological Sciences, 4Electronic Engineering and 6Radiological Sciences, Federico II University, Nuovo Policlinico, Via S. Pansini 5, I-80131 Naples, Italy, 2Max-Planck-Institut für Hirnforschung, Deutschordenstrasse 46, D-60528 Frankfurt am Main and Departments of 3Neurology and 5Neuroradiology, Klinikum der Johann Wolfgang Goethe-Universität, Schleusenweg 2–16, D-60528 Frankfurt am Main, Germany

Do spatial operations on mental images and those on visually requiring only a verbal–semantic judgement, which did not rely presented material share the same neural substrate? We used the on spatial or imagery processes. high spatial resolution of functional magnetic resonance imaging to In the second experiment, we compared the activity during determine whether areas in the parietal lobe that have been the mental clock test (imagery) with that evoked by the same implicated in the spatial transformation of visual percepts are also operation performed on visually presented material (perception) activated during the generation and spatial analysis of imagined and by a different non-spatial control task (syllable counting), objects. Using a behaviourally controlled mental imagery paradigm, the attentional load of which was comparable to that of the which did not involve any visual stimulation, we found robust imagery task. activation in posterior parietal cortex in both hemispheres. We could From the lesion-based evidence on the cortical substrate thus identify the subset of spatial analysis-related activity that is of spatial mental imagery we expected to see an increase of involved in spatial operations on mental images in the absence of neuronal activity, as reflected by the blood oxygen level-depend- external visual input. This result clarifies the nature of top-down ent (BOLD) signal, in the parietal lobes bilaterally during the processes in the dorsal stream of the human cerebral cortex and mental clock test. In addition, our analysis included the entire provides evidence for a specific convergence of the pathways of imagery and visual perception within the parietal lobes. cortex in order to assess the coactivation of other brain areas.

Introduction Materials and Methods Focal lesions of cerebral cortex may lead to selective defects of visual mental imagery (Farah, 1984; Trojano and Grossi, 1994). Subjects This suggests that the ability to revisualize objects or events in We recruited seven right-handed post-graduate students (four male, three the mind relies on the integrity of specific neural structures. female; mean age 27 years; range 23–32), who gave their informed One major issue in the search for the neural basis of mental consent to participate in the study. None of the subjects was taking any medication or was affected by neurologic or psychiatric conditions. All of imagery is whether perception and imagery share a common them were unaware of the purposes and predictions of the experiment at neural substrate. On the basis of studies suggesting that there are the time of testing. All seven subjects participated in experiment 1; four two visual systems, one for identifying objects and one for of them also participated in experiment 2. locating them (Ungerleider and Mishkin, 1982), it has been proposed that visual object and spatial imagery are functionally Magnetic Resonance (MR) Hardware and Sequences independent processes, which must rely on different underly- The MR scanner used for imaging was a 1.5 T whole-body supercon- ing neural systems (Levine et al., 1985). Patients with selective ducting system (MAGNETOM Vision, Siemens Medical Systems, Erlangen, deficits in the performance of visual or spatial imagery tasks have Germany) equipped with a standard head coil, an active shielded gradient been reported in the literature (Farah et al., 1988; Luzzatti et al., coil (25 mT/m) and Echo Planar (EPI) sequences for ultra-fast MR imaging. 1998). Several brain mapping studies (Mellet et al., 1998) tried to For functional imaging, we used a BOLD sensitive single shot EPI identify the neural basis of spatial mental imagery in the absence sequence [echo time (TE) = 66 ms; flip angle (FA) = 90°; matrix size = of visual stimulation, but none of them verified actual execution 128 × 128, voxel dimensions = 1.4 × 1.4×4mm]withaninterscan temporal spacing of 5 s. Each functional time series consisted of 5 of the imagery task on-line. ‘resting’ volumes (25 s, discarded from further analysis) followed either In the present study we used functional magnetic resonance by 100 (experiment 1) or 72 (experiment 2) acquisitions during which imaging (fMRI) to explore the neural correlates of a behaviour- the imagery condition was periodically alternated with a control condi- ally controlled spatial imagery task. The experimental task tion every ten (experiment 1) or eight scans (experiment 2). was derived from the mental clock test (Paivio,1978; Grossi et After the functional acquisitions, a high-resolution three-dimensional al., 1989, 1993), a paradigm particularly suitable for the fMRI data set covering the whole brain (referred to as 3-D anatomical image) investigation of mental imagery because it involves a behavioural was collected for each subject with a 3-D magnetization-prepared rapid control that can be performed on-line during scanning. Subjects acquisition gradient echo (MP-RAGE) sequence (TR = 9.7 ms, TE = 4 ms, FA = 12°, matrix = 256 × 256, thickness = 1 mm, number of partitions = are asked to imagine pairs of times that are presented acoustic- 170–180, Voxel dimensions = 1mm × 1mm × 1mm). For the four subjects ally and to judge at which of the two times the clock hands form participating in experiment 2, we furthermore performed a 3-D T1 the greater angle. weighted fast low-angle shot measurement at the same resolution for The spatial imagery task was used in two experiments. In subsequent surface reconstruction and surface-based statistical analysis the first experiment it was alternated with a control condition (see Data Analysis).

© Oxford University Press 2000 Cerebral Cortex May 2000;10:473–481; 1047–3211/00/$4.00 Experiment 1 tion) and on parameters of the T1-weighted 3-D MP-RAGE measurement (number of sagittal partitions, shift mean, off-centre read, off-centre Experimental Paradigm phase, resolution) with respect to the initial overview measurement Subjects were asked to imagine two analogue clock faces based on the (scout). times that were presented verbally by the examiner [e.g. 9.30 and 10.00; For each subject the structural 3-D data sets were transformed into inter-stimulus interval (ISI) = 1 s] and to judge at which of the two times Talairach space. The Talairach transformation was performed in two the clock hands form the greater angle (imagery). We selected 50 pairs of steps. The first step consisted in rotating the 3-D data set of each subject times, involving only half-hours (i.e. 7.30) or hours (i.e. 9.00); in half of to be aligned with the stereotaxic axes. For this step the location of the the conditions, the correct answers corresponded to numerically greater anterior commissure (AC), the posterior commissure (PC) and two times (i.e. 3.00 versus 1.00), and in the other half to numerically smaller rotation parameters for midsagittal alignment had to be specified times (i.e. 8.30 versus 11.00) in order to avoid subjects considering only manually in the 3-D MP-R AGE data set. In the second step the extreme the numerically greater pairs. The clock faces were balanced for the side points of the cerebrum were specified. These points together with the AC on which the hands had to be imagined and presented in pseudo-random and PC coordinates were then used to scale the 3-D data sets into order. Subjects had to push a button with their right index finger if the the dimensions of the standard brain of the Talairach and Tournaux atlas hands of the first clock formed the greater angle, or their left index finger (Talairach and Tournaux, 1988) using a piecewise affine and continuous for the second. Subjects’ responses were registered by an optic fibre transformation for each of the 12 defined subvolumes. answer box and analysed for accuracy. The individual Talairach 3-D maps were averaged across subjects and As a control task, we asked subjects to judge which of two times was superimposed on a normalized anatomical 3-D data set. Prior to averag- numerically greater. Materials, presentation and response modality were ing, the functional 3-D maps were smoothed with a Gaussian kernel of the same as in the imagery task, but the control condition required 5 mm FWHM. Activated brain areas were selected in the average map by only a verbal–semantic judgement, which did not rely on the activation of imposing lower overall correlation values (r > 0.25) to take into account imagery processes. the interindividual spatial variability of activated clusters. The two tasks were alternated in blocks of ten trials (total number of The high-resolution T1-weighted 3-D recording of one subject was trials = 100); every trial block was preceded by the appropriate instruc- used for a surface reconstruction of both hemispheres. The white/grey tion. Trials were presented every 5 s (1 trial/volume). matter border was segmented using a region-growing algorithm. The In a pre-experimental session all subjects completed an angle com- discrimination between white and grey matter was improved by several parison task on visually presented pairs of analogue clocks and several manual interactions (e.g. labelling subcortical structures as ‘white practice trials of both the imagery and the control task, to familiarize matter’). The white/grey matter border was finally tesselated in a single them with materials, tasks and response modality. step using two triangles for each side of a voxel located at the margin of Following the common procedure for imagery paradigms, subjects white matter. The tesselation of a single hemisphere typically consists were asked to keep their eyes closed during the scanning session. To of roughly 240 000 triangles. The reconstructed surface is subjected exclude the possible confounding effect of eye movements, subjects were to iterative corrective smoothing (100–200 iterations). An interactive also instructed to keep their eye position steady. Control scans with open morphing algorithm (Goebel et al., 1998a) was used to let the surface eyes and infrared eye-tracking were performed on two subjects (see grow smoothly into the grey matter. Through visual inspection, this Results). process was halted when the surface reached the middle of the grey matter (approximately layer 4 of the cortex). The resulting surface was used as the reference mesh for the visualization of functional data. The Data Analysis iterative morphing algorithm was further used to inflate each hem- Data analysis, registration and visualization were performed with the isphere. An inflated hemisphere possesses a link to the folded reference fMRI software package BrainVoyager 2.0 (Goebel et al., 1998a,b). mesh so that functional data may be shown at the correct position of Prior to statistical analysis, the time series of functional images were the inflated representation. This link was also used to keep geometric aligned for each slice in order to minimize the signal changes related to distortions during inflation to a minimum with a morphing force that small motions of the subject during the acquisition. The realigned time keeps the area of each triangle of the inflated hemisphere as close series were at first spatially filtered by convolving each EPI – image with as possible to the value of the folded reference mesh. This display of a bidimensional Gaussian smoothing kernel with full width at half max- functional maps on an inflated hemisphere allows the topographic imum (FWHM) = 2 pixels, and then temporally filtered by convolving representation of the 3-D pattern of cortical activation without loss of the each pixel’s time course with a smoothing Gaussian kernel with FWHM = lobular structure of the telencephalon (Goebel et al., 1998a; Linden et al., 2 samples. Furthermore, the linear drifts of the signal with respect to time 1999). were removed from each pixel’s time course. After these pre-processing steps, the cerebral regions responding to Experiment 2 the stimulus were identified by means of a cross-correlation analysis (Bandettini et al., 1993). The reference vector used in our analysis was a ‘box-car’ ideal vector which assumed a value of 0 during the control Experimental Paradigm period and 1 during the imagery period. In order to take into account the In this experiment, we compared the activity during the imagery variations in the haemodynamic delay of the activation signal throughout condition with that evoked by the same cognitive operation on visually the cortex, the cross-correlation coefficient was evaluated for each pixel presented material (perception) and by a different non-spatial control task using the vector obtained by shifting the original reference vector by one (syllable counting). sample. The maximum of the two cross-correlation coefficients was then The imagery condition was the same as in experiment 1, and considered in the activation map. Activated brain areas were selected in employed 48 pairs of times different from those of the previous experi- the cross-correlation maps by imposing a conservative intensity threshold ment but counterbalanced in the same way. for the cross-correlation coefficients (r > 0.5, corresponding to P <10–7; In the perception condition we used 48 pairs of analogic clock faces; uncorrected). the clock faces of each pair were generated on a computer screen and Two-dimensional statistical maps were converted into polychromatic projected one at a time (ISI = 1 s) in the central visual field. The subjects images and incorporated into the 3-D MP-R AGE data sets through had to decide at which of the two times the clock hands formed the interpolation to the same resolution (voxel size: 1.0 × 1.0 × 1.0 mm3). greater angle. This made it possible to superimpose 3-D statistical maps onto the During the syllable counting condition, we asked subjects to count 3-D anatomical data sets. Since the 2-D functional and 3-D structural the syllables of each of 48 auditorily presented pairs of times and to report measurements were performed within the same recording session, whether the total syllable number was odd or even. co-registration of the respective data sets could be computed directly During the imagery and perception condition, materials and response based on the Siemens slice position parameters of the T2*-weighted modality were the same. In the latter case, however, the pairs of times measurement (number of slices, slice thickness, distance factor, Tra-Cor were presented visually to the subject while in the imagery condition angle, FOV, shift mean, off-centre read, off-centre phase, in plane resolu- they had to be visualized mentally.

474 The Functinal Anatomy of Spatial Imagery • Trojano et al. In the syllable counting condition, material, presentation and nonparametrically adjusting for multiple comparisons while preserving response modality were the same as in the imagery task, but here a from an excessive loss of statistical power of the conventional Bonferroni verbal–phonological judgement was required that was matched for approach (Goebel and Singer, 1999). response time with the imagery conditions. Statistical results were then visualized through projecting 3-D Three sessions of 72 measurements were conducted for each subject. statistical maps on folded and inflated surface reconstructions of the Within each session, the three conditions were alternated in blocks of cortical sheet. For significantly activated voxels, the relative contribution eight trials (1 trial/measurement, 8 measurements/block) and separated RC between two selected sets of conditions in explaining the variance of by blocks of four ‘resting’ measurements. The sequence of stimulation a voxel time course were computed as RC =(bs1 – bs2)/(bs1 + bs2) where bsi conditions consisted of: imagery, perception, syllable counting.This is the sum of the estimates of the standardized regression coefficients sequence was repeated twice per session. In all the conditions, the of all conditions included in set si. The RC index was visualized with a inter-trial interval was 5 s (1 trial/scan), and every trial block was red–green pseudo-colour scale. An RC value of 1 (red) indicates that a preceded by the appropriate instruction. voxel time course is solely explained with predictor set s1 whereas an RC In this experiment subjects had to press a button with their right value of –1 (green) indicates that a voxel time course is explained solely index or middle finger; responses were registered by an optic fibre with predictor set s2. An RC value of 0 indicates that a voxel time course answer box and analysed for accuracy and response times. Mean correct is explained with an equal contribution of both predictor sets. In the reaction times for each subject were used for the speed of performance contrast maps, only |RC| values >0.7 were visualized. analysis. Because of the low number of subjects, statistical analysis was performed by nonparametric methods (Mann–Whitney test for two- group comparisons). Subjects were asked to keep their eyes open during Results the scanning session and foveate a fixation cross in order to avoid eye movements. Experiment 1

Data Analysis Behavioural Results As this experiment comprised three conditions, the statistical analysis All subjects performed the imagery task without difficulty, and was based on the application of the multiple regression analysis to time none of them made more than three errors (mean correct series of task-related functional activation (Friston et al., 1995). Further- responses: 47.6 ± 1.3). The semantic task proved to be slightly more, a method based on the reconstruction of a cortical sheet was used easier than the imagery task (mean correct responses: 48.9 ± for adjusting the significance values for multiple comparisons (Goebel 1.1). The difference in accuracy between the two tasks was not and Singer, 1999). These extended analytical tools were implemented in BrainVoyager 3.0 (Goebel et al., 1998a,b; Dierks et al., 1999). significant (t = 2). Talairach transformation (see experiment 1) was performed for the complete set of functional data of each subject, yielding a 4-D data FMRI Results representation (volume time course: 3 × space, 1 × time). Prior to The contrast between imagery and control conditions yielded statistical analysis, the time series of functional images were aligned in an increase of the BOLD signal in several parietal and frontal order to minimize the effects of head movements. The central volume of regions. the time series was used as a reference volume to which all other volumes The individual correlation maps (r > 0.5) showed a bilateral were registered, using a 3-D motion correction that estimates the three activation in the posterior parietal cortex (PPC) [primarily translation and three rotation parameters of rigid body transformation. Spatial and temporal filtering of the image time series was the same as in Brodmann area (BA) 7], in the fronto-basal region centred on the experiment 1. ascending ramus of the lateral sulcus and on parts of the inferior For each subject, the high-resolution T1-weighted 3-D recording was frontal gyrus (BA 45), and in the anterior insula in all subjects. used for the surface reconstruction of the grey/white matter boundary of The middle frontal gyrus including the dorsolateral prefrontal the brain. Since the surface reconstruction and the 4-D functional data set cortex (DLPFC, BAs 9, 46) was bilaterally activated in five of were co-registered, it was possible to tag those voxels of the functional seven subjects, the superior frontal gyrus including the supple- data which were within 0 and 2 mm with respect to the reconstructed mentary motor area in four volunteers, and the frontal eye fields cortical sheet and to limit the statistical analysis to these voxels. The (FEF) in 3 subjects. The inferior occipito-temporal region (BA general linear model (GLM) of the experiment was then computed from 37) was also activated (4/7 left, 3/7 right). the 12 (four subjects, three sessions per subject) tagged and z-normalized volume time courses. The signal values during the imagery, perception The analysis of the averaged correlation map (n =7,r > 0.25) and syllable counting conditions were considered the effects of interest. confirmed the main results of the individual analysis. The largest The corresponding predictors, obtained by convolution of an ideal clusters of activation were located bilaterally in the PPC, mainly box-car response (assuming the value 1 for the time points of task along the intraparietal sulcus IPS (Fig. 1). Smaller foci of activity presentation and the value 0 for the remaining time points) with a linear were also visible in the inferior frontal and perisylvian region of model of the haemodynamic response (Boynton et al., 1996), were used both sides (Table 1). These areas of group activation corres- to build the design matrix of the experiment. The global level of the signal ponded to those that were found consistently in the individual time courses in each session was considered to be a confounding effect. correlation maps: each centre of mass of a cluster in the average To analyse the effects of conditions compared with baseline and contrasts between conditions, 3-D individual and group statistical maps were map lay within two standard deviations of the distribution of the generated by associating each tagged voxel with the F value corres- centres of mass of the respective clusters in the individual maps ponding to the specified set of predictors and calculated on the basis of (Table 1). The DLPFC, FEF and occipito-temporal cortex, which the least mean squares solution of the GLM. Effects were only accepted as were activated only in some volunteers, did not appear in the significant when, considering an F distribution with n1 and n2 degrees of averaged map at the selected rigorous threshold. freedom (n1 = number of orthogonal predictors and n2 = number of time ′ –3 samples – n1 –1), the associated P value yielded P = P/N 10 , where N Eye Movements represented the number of independent statistical tests. In single-subject The experiment was repeated with two subjects, who were analysis, N was chosen as the total number of voxels indexed from the requested to keep their eyes open while their eye movements surface mesh (28 118, 27 267, 28 528 and 26 771 respectively for the four subjects) and the uncorrected P threshold was adequately adjusted in were monitored by the fMRI-compatible Ober2 (Permobil Medi- order to keep the corrected P ′ threshold at 10–3. In multi-subject analysis, tech, Timra, Sweden) infrared eye tracking system (Aisenberg, N was chosen as the number of voxels resulting from the logic OR of the 1996), sampling the horizontal and vertical positions of both individual surface meshes. This procedure provided an effective means of eyes at 120 Hz. This system has already been used for fixation

Cerebral Cortex May 2000, V 10 N 5 475 Figure 1. Experiment 1. 3-D correlation map of BOLD activation during imagery versus control averaged over seven subjects and superimposed on (A) a folded (axial cut at z = 44) and (B) an inflated surface-rendered individual normalized 3-D anatomy. The white lines indicate the descending portions of the intraparietal sulci. control in a recent study of attentional effects on fMRI activity in the imagery (2517 ± 860 ms; 45.5 ± 6.4 correct responses) or human MT/MST (O’Craven et al., 1997). We could thus confirm the perception conditions (1433 ± 618 ms; 47.1 ± 0.8 correct that subjects maintained fixation during the entire experiment. responses). However, only the comparison between the per- The magnetic artefacts produced by the eye monitoring ception and syllable counting conditions yielded significant device affected the quality of the EPI signal, but only in areas differences both in accuracy (Mann–Whitney test: z = 2.39; P = anterior to the clusters of activation described above. The BOLD 0.02) and in reaction time (Mann–Whitney test: z = 2.30; P = activation pattern during eye monitoring did not differ sig- 0.02), whereas the behavioural differences between the syllable nificantly from that obtained without the eye tracking system. counting and imagery conditions were not significant.

Experiment 2 FMRI Results Behavioural Results All three tasks, compared with baseline, were accompanied by The syllable counting task proved to be the most difficult an increased activation of left sensorimotor cortex (contralateral condition, with higher reaction times (2940 ± 1039 ms) and to the hand of the button press response). The two tasks that error rates (39.5 ± 4.9 correct responses/48 trials) than either involved auditory stimulation (imagery, syllable counting) were

476 The Functinal Anatomy of Spatial Imagery • Trojano et al. Table 1 Experiment 1: locations, extension and significance of activated brain areas during the mental clock task

Anatomical area Brodmann area Tailarach coordinates Vox. r

XYZ

Left posterior parietal cortex 7 –24 (–20 ± 4.3) –70 (–74 ± 3.3) 44 (42 ± 4.2) 243 0.73 –22 (–23 ± 2.0) –72 (–76 ± 4.7) 31 (33 ± 2.5) 53 0.63 Right posterior parietal cortex 7 22 (21 ± 2.5) –76 (–75 ± 2.7) 43 (42 ± 0.8) 261 0.72 24 (25 ± 1.8) –80 (–78 ± 2.7) 31 (33 ± 1.7) 52 0.63 Left inferior parietal lobule 39–40 –31 (–33 ± 3.1) –48 (–46 ± 4.4) 38 (34 ± 2.7) 77 0.64 Left perisylvian cortex 45/insula –31 (–33 ± 5.5) 23 (22 ± 3.0) 12 (8 ± 4.0) 39 0.58 Right perisylvian cortex 45/insula 44 (43 ± 4.5) 16 (17 ± 2.5) 4 (5 ± 1.7) 96 0.57 The position of each area is given as the Tailarach coordinates of the centre of mass of the supra-threshold (r > 0.25) clusters in the 3-D-average map. The mean values (± SD of seven subjects) of the Tailarach coordinates of the centre of mass of each cluster as selected in the individual correlation maps (r > 0.5; P < 10–7; uncorrected) are reported in brackets. Vox = number of voxels (1.4 × 1.4 × 4 mm) of each area; r = cross-correlation coefficient of the cluster time-courses averaged across subjects.

Table 2 Experiment 2: locations of brain areas activated during the imagery, perception and syllable counting conditions

Anatomical area Brodmann area Imagery versus baseline Perception versus baseline Syllable counting versus baseline

XYZXYZXYZ

Left posterior parietal cortex 7 –17 –70 46 –26 –65 46 – – – Right posterior parietal cortex 7 24 –69 46 27 –65 55 – – – Left inferior parietal lobule 39/40 –30 –50 38 –26 –50 41 –46 –48 44 Right inferior parietal lobule 39/40 – – – 32 –45 39 44 –45 42 Left perisylvian cortex 45/ins –31 16 15 – – – –35 16 3 Right perisylvian cortex 45/ins 38 9 3 – – – 46 9 5 Left superior frontal gyrus 6/8 – – – –6 4 46 –5 –16 61 Right superior frontal gyrus 6/8 4 28 39 4 –3 46 4 –5 54 Left precentral gyrus 4 –33 –26 55 –31 –26 54 –41 –31 55 Left gyrus of Heschl 41/42 –57 –24 9 – – – –51 –20 9 Right gyrus of Heschl 41/42 63 –16 6 – – – 54 –17 –3 Right prefrontal cortex 46 38 36 22 – – – 45 25 35 Left occipitotemporal cortex 37/19 – – – –41 –67 –5 – – – Right occipitotemporal cortex 37/19 – – – 38 –60 –13 – – – Left middle occipital gyrus 19 – – – –35 –79 3 – – – Right middle occipital gyrus 19 – – – 32 –80 2 – – – The position of each area is given as the Talairach coordinates of the centre of mass of the supra-threshold clusters (P ′ < 10–3, corrected) of the group analysis.

accompanied by activation of auditory cortex in the temporal syllable counting task, activity increased bilaterally in the lobes bilaterally (Table 2). inferior parietal lobules and right prefrontal cortex (Fig. 3a). The comparison imagery versus baseline yielded a prominent In the contrast between the imagery and perception con- increase in activation of the PPC in all four subjects. Increased ditions no activation was evident in the PPC. In the imagery activity was also present in the left inferior parietal lobule, in the condition increased activity was only present in right prefrontal right pre-frontal cortex, and in the inferior frontal and cortex and in mesial frontal areas bilaterally. In the perception perisylvian region, bilaterally. Moreover, a cluster of activation condition, occipital and inferior temporal areas were activated was seen in the right superior mesial frontal region (Table 2). bilaterally (Fig. 3b, Table 3). The comparison between the perception and baseline conditions showed an activation of the posterior and inferior Discussion parietal cortex very similar to that observed during the imagery The mental clock test requires the generation of multipart mental condition. Increased activity was also found in the basal and images (Trojano and Grossi, 1994). These mental images served lateral surfaces of the occipital lobe. More consistently observed as the substrate of a spatial comparison task, which all of our in the left hemisphere, this activity extended anteriorly in the subjects performed at a high level of accuracy. basal surface of the temporal lobe. An activation was also seen in The most striking results of our two experiments demonstrate the superior mesial frontal region bilaterally (Table 2). that cortical activation (as measured by an increase of the fMRI In the syllable versus baseline contrast increased activation BOLD signal) during the mental clock test was most prominent was recognizable bilaterally in the inferior parietal lobule, in the in the posterior parietal lobes of both hemispheres. inferior frontal and perisylvian region, in the superior mesial The absence of imagery-related activation in early visual areas frontal region and in the right prefrontal cortex (Table 2). The in both experiments provides new data bearing on the debate contrast between the imagery and syllable counting conditions regarding the cortical representations of mental imagery, yielded, in all four subjects, a bilateral activation of the PPC, especially because the present study employed an on-line mainly along the IPS (Table 3), closely corresponding to the behavioural control to ensure the actual generation of mental largest cluster of experiment 1 (Figs 1, 2 and 3a). During the images. Our results therefore confirm the recent findings that

Cerebral Cortex May 2000, V 10 N 5 477 Figure 2. Experiment 2. The contrast maps for imagery versus syllable counting (P ′ <10–3, corrected) superimposed on folded surface reconstructions showing the activation during the imagery condition of the posterior parietal cortex in all four subjects (axial cuts at z1 = 51, z2 = 48, z3 = 48, z4 = 44). Left hemispheres are shown on the right side.

certain imagery conditions, particularly those which rely on equal strength during the imagery and syllable counting abstract patterns and schematic figures, produce increases in conditions were observed in superior mesial frontal cortex. activity in primary visual areas to only a small extent (Goebel et Activation of this area, unlike that of the PPC, was thus related to al., 1998a) or not at all (Mellet et al., 1996). the attentional demand of the task and not to its specific nature. The activation of the prefrontal cortex, which was observed Likewise, the activation of perisylvian areas in experiment 1 in a number of our subjects in the first experiment, has been should be considered not to be specific for mental imagery associated with the attentional demand involved in different because a similar activation pattern was found for syllable kinds of mental imagery (Mellet et al., 1996; Goebel et al., counting versus imagery in experiment 2. The activation of the 1998a) and with tasks of working memory (Ungerleider et al., inferior parietal lobule during the syllable counting task can be 1998). Accordingly, increased prefrontal activity was evident in explained by the requirement of phonetic segmentation and the second experiment only when a task was compared with the short-term memory (Wildgruber et al., 1999). baseline or when a more difficult task was compared with an Increased activity in the posterior parietal lobes, particularly easier one. In the second experiment, activations of roughly along the IPS, was observed in experiment 1 and was par-

478 The Functinal Anatomy of Spatial Imagery • Trojano et al. Figure 3. Experiment 2. Multi-subject contrast maps (P ′ <10–3, corrected) superimposed on inflated representations of the cortical sheet of an individual normalized 3-D anatomy. (A) Imagery (green) versus syllable counting (red). (B) Imagery (green) versus perception (red). The white lines indicate the descending portions of the intraparietal sulci. ticularly evident when we compared the imagery condition Petrides, 1997), the right PPC was seen to participate in the pro- of experiment 2 (execution of the mental clock test) with the cessing of mirror images of letters or digits. Activation of the baseline. This activation maintained the contrast with a non- superior parietal lobule and of the IPS in both hemispheres has spatial control task that was performed on the same auditorily been described in a recent fMRI study of the analysis of spatially presented material and matched for difficulty (imagery versus transformed words and phrases, which also distinguished this syllable counting, Fig. 3a) and was consistently observed spatial transformation-related activation from general attentional between individual subjects (Fig. 2). This differential activation effects (Goebel et al., 1998b). The construction of 3-D mental of the posterior parietal lobes by the spatial task confirms the images from auditory instructions, as studied by PET (Mellet et specific role of the parietal cortex in spatial processing, rather al., 1996), involved a distributed system of frontal, occipital and than an unspecific activation related to attentional demand. parietal areas. The parietal activation, however, was most prom- Activation of posterior parietal areas has been observed in inent in the right precuneus and supramarginal gyrus and did not conjunction with the spatial transformation of visually presented involve the IPS region. This absence of IPS activation might stimuli (Cohen et al., 1996; Alivisatos and Petrides, 1997) and reflect the specific nature of the task of Mellet et al. (Mellet et the orientation discrimination of tactile stimuli (Sathian et al., al., 1996), which did not require the spatial comparison be- 1997). In a recent PET study of mental rotation (Alivisatos and tween objects and was not performed through mental rotation.

Cerebral Cortex May 2000, V 10 N 5 479 Table 3 Experiment 2: locations, extension and significance of contrasts between the imagery and control conditions

Anatomical area BA Imagery (I) vs syllables (S) Imagery (I) versus perception (P)

Task XYZVoxel R Task xyzVoxel R

Left posterior parietal cortex 7 I –19 –67 48 1010 0.76 – (–16.3 ± 3.9) (–69.8 ± 5.6) (50.5 ± 3.7) Right posterior parietal cortex 7 I 17 –64 43 726 0.65 – (15.8 ± 4.6) (–68.5 ± 4.0) (45.3 ± 2.2) Left perisylvian cortex 45/ins S –49 15 0 507 0.64 – (–46.5 ± 5.2) (11.3 ± 3.0) (–0.5 ± 3.3) Right perisylvian cortex 45/ins S 44 13 1 984 0.52 – (42.5 ± 3.4) (10.5 ± 3.1) (1.5 ± 0.6) Left inferior parietal lobule 39/40 S –45 –46 44 612 0.70 – (–43.5 ± 3.7) (–45.5 ± 4.2) (43.0 ± 2.0) Right inferior parietal lobule 39/40 S 52 –45 43 532 0.72 – (49.5 ± 4.4) (–45.0 ± 3.6) (44.8 ± 1.7) Left superior frontal gyrus 6/8 – I –3 28 37 171 0.54 (–2.8 ± 1.7) (31.3 ± 4.1) (34.8 ± 3.8) Right superior frontal gyrus 6/8 – I 4 28 40 625 0.50 (5.3 ± 2.3) (23.7 ± 6.7) (38.7 ± 5.1) Right prefrontal cortex 9 S 43 21 39 662 I 53 13 26 435 0.64 (43.3 ± 2.9) (13.5 ± 4.4) (35.5 ± 4.7) (50.8 ± 4.8) (13.8 ± 1.5) (26.5 ± 6.6) Left occipitotemporal cortex 37/19 – P –45 –57 –9 556 0.68 (–44.3 ± 2.2) (–58.8 ± 4.3) (–7.3 ± 2.6) Right occipitotemporal cortex 37/19 – P 44 –65 –12 1181 0.70 (41.8 ± 4.5) (–61.5 ± 4.4) (–10.5 ± 1.7) Left middle occipital gyrus 19 – P –35 –79 3 515 0.56 (–37.5 ± 4.8) (–77.5 ± 5.8) (0.3 ± 2.5) Right middle occipital gyrus 19 – P 32 –80 2 401 0.54 (34 ± 3.4) (–81.5 ± 3.7) (3.3 ± 2.2) The position of each area is given as the Tailarach coordinates of the centre of mass of the supra-threshold (P ′ <10–3, corrected) clusters in the multi-subject maps. The mean values (± SD of four subjects) of the Tailarach coordinates of the centre of mass of each cluster (P ′ <10–3, corrected) as selected by the single-subject analysis are reported in brackets. Vox = number of voxels (1× 1 × 1 mm) of each area; R = multiple regression coefficient of the cluster time-courses averaged across subjects.

two ways. First, the superior IPS might be instrumental in the Table 4 computation of spatial transformations, regardless of whether Meta-analysis of activated brain areas in the posterior parietal cortex during spatial tasks the material is present in the visual field or merely as a mental Authors Brodman Tailarach coordinates image. Alternatively, any spatial transformation task, whether area it involves visually perceived or imagined material, or indeed XYZ tactile stimulation (Sathian et al., 1997), might require the Goebel et al. (1998b) (mirror reading) 7 L –23 –68 47 implicit generation of mental visual representations (Kosslyn and 7 R –25 –71 25 29 –68 28 Sussmann, 1995). Alivisatos and Petrides (1997) (mental rotation) 7 R 12 –69 51 In the second experiment the perception condition (spatial Alivisatos and Petrides (1997) (mirror image) 7 R 31 –62 45 matching of visually presented clocks), compared with the Sathian et al. (1997) (orientation discrimination) 7/19 L –20 –71 32 fixation baseline, yielded only weak activation of primary visual cortex (probably because of the low contrast of the visual stimulus, and the TR of 5 s, which was relatively long compared with the short presentation of the stimulus), but very prominent The majority of earlier studies of mental rotation and related bilateral activation of the inferior temporal and the inferior and tasks suffered from the possible confound of saccade-related lateral occipital lobes. These areas, which include the fusiform activation in the parietal lobe (Milner and Goodale, 1995). The gyri, have been shown to be involved in the cortical processing spatial transformation-related activation of the IPS region that of visual objects (Ungerleider and Mishkin, 1982; Malach et al., was observed by Goebel et al. (Goebel et al., 1998b) could be 1995). In the posterior parietal lobes, a strong activation of the separated from the activity related to overt (saccades) and covert IPS was observed during this task. The direct contrast between attention shift by additional control tasks and correlation analysis the perception and imagery conditions (i.e. the clock test per- of the BOLD signal time course. In the present study, we ex- formed on visually presented and mentally imagined material, cluded the contribution of saccadic activity to the task-related respectively) led to the disappearance of this activity. The com- BOLD signal changes by controlling eye movements during the parison of the present results with the data on transformations scanning session. of visually presented material from our (Goebel et al., 1998a) and Goebel et al. (Goebel et al., 1998b) identified a ‘spatial trans- other groups (Cohen et al., 1996; Alivisatos and Petrides, 1997) formation area’ in the parietal lobe. Their spatial transformation suggested a close spatial overlap between the two sets task, like those of Alivisatos and Petrides (Alivisatos and Petrides, of paradigms (Table 4).The new evidence from visuospatial 1997) and Cohen et al. (Cohen et al., 1996), was performed on perceptual and mental imagery tasks performed by the same visually presented material. The present study shows that the subjects in the same experimental session confirmed that IPS area in the superior IPS is active when the spatial task is activation is common to both conditions. Our results reveal a performed on mental images, in the absence of any visual stimu- striking similarity between brain activation during spatial com- lation. This similarity of activation patterns can be explained in parisons of visually perceived and imagined material.

480 The Functinal Anatomy of Spatial Imagery • Trojano et al. Conclusion Goebel R, Linden DEJ, Lanfermann H, Zanella FE, Singer W (1998b) In the present study we used a behaviourally controlled Functional imaging of mirror and inverse reading reveals separate coactivated networks for oculomotion and spatial transformations. paradigm to verify the neural basis of visuospatial imagery in Neuroreport 9:713–719. the absence of visual stimulation and eye movements. Our high- Goebel R, Singer W (1999) Cortical surface-based statistical analysis of resolution fMRI findings demonstrate that the PPC is strongly functional magnetic resonance imaging data. Neuroimage 9:S64. involved in the processing of spatially coded material in the Grossi D, Modafferi A, Pelosi L, Trojano L (1989) On the different roles of imagery domain. Whilst our study did not address the separation the two cerebral hemispheres in mental imagery: the ‘O’Clock test’ in of the single subcomponents of spatial mental imagery (Trojano two clinical cases. Brain Cogn 10:18–27. and Grossi, 1994), the comparison of our results with those Grossi D, Angelini R, Pecchinenda A, Pizzamiglio L (1993) Left imaginal neglect in heminattention: experimental study with the o‘clock test. of recent studies of spatial transformations of visually presented Behav Neurol 6:155–158. material indicates that the analysis of visual space in perception Kosslyn SM, Sussmann AL (1995) Roles of imagery in perception: or, and imagery has a common neural basis in the parietal lobes. It there is no such thing as immaculate perception. In: The cognitive is suggested that the neural networks involved in the processes neurosciences (Gazzaniga M, ed.), pp. 1035–1042. Cambridge, MA: of spatial transformation might be shared by several cognitive MIT Press. functions, including visuospatial imagery. Levine DN, Warach J, Farah M (1985) Two visual systems in mental imagery: dissociation of ‘what’ and ‘where’ in imagery disorders due to bilateral posterior cerebral lesions. Neurology 35:1010–1018. Notes Linden DEJ, Prvulovic D, Formisano E, Völlinger M, Zanella FE, Goebel R, The authors are grateful to Jim Waltz and Lars Muckli for comments Dierks T (1999) The functional neuroanatomy of target detection: on the manuscript. This paper is dedicated to Prof. Giuseppe Caruso, an fMRI study of visual and auditory oddball tasks. Cereb Cortex 9: unforgettable teacher. 515–523. Address correspondence to Dr Luigi Trojano, Salvatore Maugeri Luzzatti C, Vecchi T, Agazzi D, Cesa-Bianchi M, Vergani C (1998) A Foundation, IRCCS, Center of Telese, Loc. S. Stefano in Lanterria, I–82037 neurological dissociation between preserved visual and impaired Telese Terme (BN), Italy. Email: [email protected]. spatial processing in mental imagery. Cortex 34:461–469. Malach R, Reppas JB, Benson RR, Kwong KK, Jiang H, Kennedy WA, Ledden PJ, Brady TJ, Rosen BR, Tootell RB (1995) Object-related References activity revealed by functional magnetic resonance imaging in human Aisenberg S (1996) Measurement and analysis of binocular rapid eye occipital cortex. Proc Natl Acad Sci USA 92:8135–8139. movement, including saccades, saccadic prediction, fixations, nystag- Mellet E, Tzourio N, Crivello F, Joliot M, Denis M, Mazoyer B (1996) mus, and smooth pursuit. Invest Ophthalmol Vis Sci Suppl 37:S274. Functional anatomy of spatial mental imagery generated from verbal Alivisatos B, Petrides M (1997) Functional activation of the human brain instructions. J Neurosci 16:6504–6512. during mental rotation. Neuropsychologia 35:111–118. Mellet E, Petit L, Mazoyer B, Denis M, Tzourio N (1998) Reopening the Bandettini PA, Jesmanowicz A, Wong EC, Hyde JS (1993) Processing mental imagery debate: lessons from functional anatomy. Neuroimage strategies for time-course data sets in functional MRI of the human 8:129–139. brain. Magn Reson Med 30:161–173. Milner AD, Goodale MA (1995) The visual brain in action. Oxford: Boynton GM, Engel SA, Glover GH, Heeger DJ (1996) Linear systems Oxford University Press. analysis of functional magnetic resonance imaging in human V1. J O’Craven KM, Rosen BR, Kwong KK, Treisman A, Savoy RL (1997) Neurosci 16:4207–4241. Voluntary attention modulates fMRI activity in human MT-MST. Cohen MS, Kosslyn SM, Breiter HC, DiGirolamo GJ, Thompson WL, Neuron 18:591–598. Anderson AK, Brookheimer SY, Rosen BR, Belliveau JW (1996) Paivio A (1978) Comparisons of mental clocks. J Exp Psychol (Hum Changes in cortical activity during mental rotation. A mapping study Percept) 4:61–71. using functional MRI. Brain 119:89–100. Sathian K, Zangaladze A, Hoffmann JM, Grafton ST (1997) Feeling with Dierks T, Linden DE, Jandl M, Formisano E, Goebel R, Lanfermann H, the mind’s eye. Neuroreport 8:3877–3881. Singer W (1999) Activation of Heschl’s gyrus during auditory hal- lucinations. Neuron 22:615–621. Talairach J, Tournoux P (1988) Co-planar stereotaxic atlas of the human Farah MJ (1984) The neurological basis of mental imagery: a compo- brain: 3-dimensional proportional system: an approach to cerebral nential analysis. Cognition 18:245–72. imaging. Stuttgart: Thieme. Farah MJ, Hammond KM, Levine DN, Calvanio R (1988) Visual and spatial Trojano L, Grossi D (1994) A critical review of mental imagery defects. imagery: dissociable systems of representation. Cogn Psychol 20: Brain Cogn 24:213–243. 439–462. Ungerleider LG, Mishkin M (1982) Two cortical visual systems. In: Friston KJ, Holmes AP, Worsley KJ, Poline JB, Frith CD, Frackowiak RSJ Analysis of visual behavior (Ingle DJ, Goodale MA, Mansfield RJW, (1995) Statistical parametric maps in functional imaging: a general eds), pp. 549–586. Cambridge, MA: MIT Press. linear approach. Hum Brain Map 2:189–210. Ungerleider LG, Courtney SM, Haxby JV (1998) A neural system for Goebel R, Khorram-Sefat D, Muckli L, Hacker H, Singer W (1998a) human visual working memory. Proc Natl Acad Sci USA 95:883–890. The constructive nature of vision: direct evidence from functional Wildgruber D, Kischka U, Ackermann H, Klose U, Grodd W (1999) magnetic resonance imaging studies of apparent motion and motion Dynamic pattern of brain activation during sequencing of word imagery. Eur J Neurosci 10:1563–1573. strings evaluated by fMRI. Cogn Brain Res 7:285–94.

Cerebral Cortex May 2000, V 10 N 5 481