Effects of Strategies on Mental Rotation and Hemispheric Lateralization: Neuropsychological Evidence

Barbara Tomasino and Raffaella I. Rumiati Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/16/5/878/1758193/089892904970753.pdf by guest on 18 May 2021

Abstract & We can predict how an object would look if we were to view that what matters is the type of strategy adopted in MR. see it from different viewpoints by imagining its rotation. This Thus, independently of the type of stimulus, patients with left essential human ability, called mental rotation (MR), guides hemisphere lesions showed a selective deficit in MR as a individuals’ actions by constantly updating their environmental consequence of their manual activity, whereas patients with consequences. It is, however, still under debate whether the right hemisphere lesions were found impaired in MR by means way in which our brain accomplishes this operation is deter- of a visual strategy. We conclude that MR is achieved by mined by the type of stimulus or rather by a mental strategy. recruiting different strategies, implicitly triggered or prompted Here we present neuropsychological evidence sustaining the at will, each sustained by a unilateral brain network. &

INTRODUCTION selectively impaired when deciding whether a hand is Typically, mental rotation (MR) is studied by asking left or right despite being still able to mentally rotate subjects to decide whether two images—of which one Shepard and Metzler’s stimuli (Rumiati et al., 2001). On is either a mirror or an identical rotated image of the the other hand, a patient (JB) with a bilateral infero- other—are the same or different (Corballis, 1997; She- temporal lesion was reported as having a deficit in per- pard & Metzler, 1971; Shepard & Cooper, 1982). The forming MR of Shepard and Metzler’s stimuli (Sirigu analysis of the response times (RTs) shows that subjects & Duhamel, 2001). However, JB’s ability to mentally create an internal image of an object and that they rotate rotate motor images of body parts was not investigated. it until it is congruent with the target object. A linear Nor was this ability assessed in other posterior left increase in RTs with increasing angular disparity is usu- (Morton & Morris, 1995; Metha & Newcombe, 1991; ally found for rotated stimuli such as two-dimensional Kosslyn, Holtzman, Farah, & Gazzaniga, 1985) or right alphanumeric characters (Corballis & Sergent, 1989) and (Bricolo, Shallice, Priftis, & Meneghello, 2000; Ditunno three-dimensional Shepard and Metzler’s stimuli (Cohen & Mann, 1990; Farah & Hammond, 1988; Ratcliff, 1979) et al., 1996; Shepard & Metzler, 1971; Shepard & Coo- brain-damaged patients who have also been described per, 1982). MR operations can be distinguished accord- as having a deficit of MR operations. In a recent study ing to the type of stimulus involved (Tomasino, Toraldo, (Tomasino et al., 2003), a direct comparison between & Rumiati, 2003; Rumiati, Tomasino, Vorano, Umilta`,& right hemisphere (RH) and left hemisphere (LH) pa- DeLuca, 2001; Kosslyn, DiGirolamo, Thompson, & Al- tients’ performance on MR of both types of stimuli pert, 1998), the reference frame (Zacks, Rypma, Gabrieli, showed that lesions in the LH impaired MR of hands, Tversky, & Glover, 1999; Zacks, Mires, Tversky, & Ha- while lesions in the RH affected MR of external objects zeltine, 2000; Zacks, Ollinger, Sheridan, & Tversky, 2002; (e.g., a puppet and flag shapes). Wraga, Creem, & Proffitt, 1999), or the strategies used Psychophysical studies indicate that different mecha- (Kosslyn, Thompson, Wraga, & Alpert, 2001). nisms may be selected in MR according to the frame As far as the stimulus type is concerned, different of reference used for the rotation (Zacks et al., 2000). operations may be recruited in MR depending on Thus, MR can be accomplished taking as a reference whether the stimulus type is a body part or a two- frame the object itself (i.e., allocentric view) or the or three-dimensional object. Neuropsychological studies viewer’s (i.e., egocentric view). It has been shown that indicate that these two types of transformations can be the allocentric and egocentric transformations lead to selectively affected. For instance, a patient with left different chronometric patterns. A strong correlation hemisphere (LH) brain damage was described as being between the angle of rotation and RTs has been consist- ently obtained in subjects who mentally rotated the stimuli using the allocentric view (Zacks et al., 1999, Scuola Internazionale Superiore di Studi Avanzati (SISSA) 2000, 2002; Wraga et al., 1999; Corballis, 1997; Shepard

D 2004 Massachusetts Institute of Technology Journal of Cognitive Neuroscience 16:5, pp. 878–888 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/089892904970753 by guest on 27 September 2021 & Metzler, 1971; Shepard & Cooper 1982). When trans- The present study was designed first to provide formations are based on the viewer’s perspective, the neuropsychological evidence that object- and viewer- RTs reflect biomechanical constraints that mirror those based transformations are dissociable as an effect of of the real movements (Zacks et al., 2000; Parsons, brain damage. We assumed that any lesion of the brain 1987a, 1987b, 1994). network (i.e., parietal or premotor cortex) that is known Functional imaging studies have investigated how MR to underlie MR abilities will selectively impair object- or can be modulated as an effect of the type of stimuli, the egocentric-based transformation as a consequence of type of strategies, and reference frames. In a PET study damage to the RH or to the LH, respectively. We (Kosslyn et al., 1998) with healthy participants, it was also aimed to reconcile the neuropsychological findings,

shown that MR of hands enhanced activation in the left calling for a hemispheric specialization in stimulus- de- Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/16/5/878/1758193/089892904970753.pdf by guest on 18 May 2021 parietal lobe, left motor and premotor areas, whereas MR pendent MR (Tomasino et al., 2003), with the recently of Shepard and Metzler’s stimuli induced a bilateral proposed ‘‘strategy account’’ (Kosslyn et al., 2001). activation of the parietal lobe and BA 19 only. Based on We addressed these issues by asking patients with these activations, Kosslyn et al. (1998) argued that MR of unilateral brain lesions and healthy control subjects the hands but not of Shepard and Metzler’s stimuli were instructed to adopt a motor (egocentric transfor- recruits motor processes. Interestingly, the same activa- mation) and, in a different block, a visual strategy (allo- tion patterns were obtained in another PET study (Koss- centric transformation) when performing MR of hand lyn et al., 2001) in which subjects performed MR of shapes (Experiment 1) or Shepard and Metzler’s stimuli Shepard and Metzler’s stimuli using an internal or exter- (Experiment 2). nal strategy. Thus, they imagined rotating a stimulus as Independently of the stimulus to be rotated, patients a consequence of their own hand action (i.e., internal with LH lesions are expected to make considerably strategy) or as if they were observing the stimulus more errors than RH patients in applying the motor rotating in the visual space (i.e., external strategy). The strategy, as it is well known that the LH sustains tasks left primary motor cortex—the region that in Kosslyn’s that require actual and simulated hand movements PET study (Kosslyn et al., 1998) was activated in associ- (Tomasino et al., 2003; Rumiati et al., 2001; Kosslyn ation with MR of hands only—was here enhanced when et al., 1998, 2001; Ganis, Keenan, Kosslyn, & Pascual subjects mentally rotated Shepard and Metzler’s stimuli, Leone, 2000; Parsons & Fox, 1998; Sirigu et al., 1996; simulating a sort of manual rotation. The effect of the Decety et al., 1994). In sharp contrast, given that the RH reference frame on MR was investigated in a series of is held to be involved in spatial operations (Kosslyn studies (Zacks et al., 1999, 2002). First, it was shown that et al., 2001; Bricolo et al., 2000; Harris et al., 2000; when subjects decided which arm of a puppet was out- Davidoff & Warrington, 1999; Zacks et al., 1999; Corbal- stretched, the activation within the PTO junction was lis, 1997; Ditunno & Mann, 1990; Farah & Hammond, lateralized to the LH when the stimuli were upright 1988; Ratcliff, 1979), we predict that RH patients should (egocentric perspective MR), whereas it was stronger be less accurate than the LH patients in performing the in the RH for upside-down pictures (object-based MR) MR using the visual strategy. To date, we refer to LH/ (Zacks et al., 1999). These results were partially repli- RH as patients with lesions in any region of the brain cated in a successive fMRI study (Zacks et al., 2002), network (i.e., parieto-premotor) sustaining MR in the where subjects were presented with disoriented pairs of LH and RH, respectively. human shapes depicted with an extended arm. The activation was lateralized to the RH posterior cortex when subjects performed same–different judgments RESULTS (object-based MR) compared with when they made left–right judgments on which arm was extended (viewer- Independent of the type of stimulus (i.e., hand shapes or based MR). However, the opposite comparison (i.e., Shepard and Metzler’s stimuli), a double dissociation was viewer-based transformation vs. object-based transfor- found between deficits of motor and visual strategies in mation) did not show any lateralized activity. performing MR. Thus, RH patients (RHD1 and RHD2) Zacks et al. (1999, 2002) proposed a model that pre- were more accurate than LH patients (LHD1 and LHD2) dicts the presence of a double dissociation between when they imagined physically turning the stimulus. regions specifically associated with MR obtained by On the contrary, LH patients (LHD1 and LHD2) were adapting the egocentric reference frame of the viewer better than RH patients (RHD1 and RHD2) when they to that of the object, and those associated with MR imagined the stimulus as rotating in the visual space. accomplished by aligning the object’s reference frame to the frame of the viewer. So far, however, only a single Experiment 1 dissociation, namely, the enhancement of the RH pos- terior regions when object-based MR was compared Accuracy Data with viewer-based MR, has been documented (Zacks For MR of hand shapes, LHD2 was more accurate than et al., 2002). RHD1 and RHD2 when using the visual strategy (Wilcox-

Tomasino and Rumiati 879 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/089892904970753 by guest on 27 September 2021 on, both p < .05), whereas RHD1 and RHD2 were more respectively), while it fell within the normal range when accurate than LH2 in using the motor strategy (Wilcoxon, they solved the task by means of the motor strategy both p < .0001) (see Figure 1a). LHD1 was significantly ( p > .05). more accurate than RHD1 (Wilcoxon, p < .001) when using the visual strategy, but his performance was not RT Data significantly different than that of RHD2 ( p >.05, ns). In turn, RHD1 (Wilcoxon, p < .005) and RHD2 (Wilcoxon, One way to ascertain that subjects apply the motor p <. 01) were more accurate than LHD2 in using the strategy is looking whether RTs reflect arm–hand bio- motor strategy (see Table 1). logical constraints (Parsons, 1987a, 1987b, 1994). In-

Compared with control subjects, LHD1’s (z = À8.22, deed, subjects were faster with stimuli oriented toward Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/16/5/878/1758193/089892904970753.pdf by guest on 18 May 2021 p < .0001) and LHD2’s (z = À8.19, p < .0001) ability to the body midsagittal plane (i.e., medial) than when apply the motor strategy was pathological, whereas they they were oriented away from the body midsagittal were as accurate as controls in using the visual strategy plane (i.e., lateral). This pattern is clearly indicated by ( p > .05 for both). Conversely, RHD1’s and RHD2’s the presence of a significant main effect of the lateral/ performance on the visual strategy was pathological medial gradient when subjects applied the motor (z = À 10.7, p < .0001 and z = À 1.5, p < .05, strategy [F(1,43) = 4.15, p < .05 for LHD2; F(1,66) =

Figure 1. Experiment 1: MR of hands. (A) Percent of correct responses adopting either the motor or the visual strategy. (B) The lateral (L)–medial (M) gradient. Mean RTs (msec) of correct responses as a function of the lateral and medial stimulus orientation in performing the MR of hands using the motor (B) and the visual (C) strategy. The effect of L–M gradient and different hand positions on RTs in the motor (D) and the visual (E) tasks: mean RTs just for LHD2 and controls as an example. However, all patients showed the same pattern of results.

880 Journal of Cognitive Neuroscience Volume 16, Number 5 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/089892904970753 by guest on 27 September 2021 Table 1. LHD1, LHD2, RHD1, and LHD2’s Performance in for control subjects. When they applied the motor Experiment 1 (MR of Hand Shapes) and Experiment 2 (MR strategy, a significant main effect of L–M orientation of Shepard and Metzler’s Stimuli) in Using the Motor and [F(1,13) = 83.82, p < .001], view [F(2,26) = 27.86, p < Visual Strategy .001] as well as the interactions View  L–M orienta- Strategy Comparison Significance tion and View  Angle were found [F(2,26) = 11.41, p < .001 and F(1,13) = 63.38, p < .001, respectively]. Experiment 1 However, such effects disappeared when controls per- Motor RHD1 vs. LHD2 p < .0001 RHD1 > LHD2 formed MR using the visual strategy: L–M orientation RHD2 vs. LHD2 p < .0001 RHD2 > LHD2 [F(1,13) = 0.66], view [F(2,26) = 0.19], and the inter- actions View  L–M orientation and View  Angle Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/16/5/878/1758193/089892904970753.pdf by guest on 18 May 2021 RHD1 vs. LHD1 p < .005 RHD1 > LHD1 [F(2,26) = 2.04 and F(1,13) = 0.29, respectively: all RHD2 vs. LHD1 p < .01 RHD2 > LHD1 p > .05, ns) (see Figure 1D and E). In addition, we argue that all subjects who re- LHD1 vs. LHD2 p < .05 LHD1 = LHD2 sponded according to whether the target stimulus RHD1 vs. RHD2 p < .05 RHD1 = RHD2 appeared on the right or left side of the screen Visual LHD2 vs. RHD1 p < .05 LHD2 > RHD1 (instead of the side of the line drawing of the hand) when performing the visual strategy failed to mentally LHD2 vs. RHD2 p < .05 LHD2 > RHD2 reorient the stimuli. In fact, subjects could respond LHD1 vs. RHD1 p < .001 LHD1 > RHD1 that the thumb was on the left or right of the screen or LHD1 vs. RHD2 p < .05 LHD1 > RHD2 on the left or right of the line drawing of the hand. If they responded according to the side of the screen, it LHD1 vs. LHD2 p < .05 LHD1 = LHD2 meant that they did not rotate the stimulus. Errors of RHD1 vs. RHD2 p < .05 RHD1 = RHD2 this sort can occur only at 1508 with both L- and M- orientations. In fact, in this condition, the responses given according to whether the thumb is on the left or Experiment 2 right side of the screen and responses based on the Motor RHD1 vs. LHD1 p < .001 RHD1 > LHD1 required spatial judgment are incompatible. The effect RHD2 vs. LHD1 p < .001 RHD2 > LHD1 of angle was not significant neither for controls’ accu- racy [F(1,13) = 0.06, p > .05, ns] nor for patients’ RHD1 vs. LHD2 p < .05 RHD1 > LHD2 accuracy (Wald(1) = .0330 for RHD1; Wald(1) = .0122 RHD2 vs. LHD2 p < .05 RHD2 > LHD2 for LHD1, Wald(1) = .0303 for RHD2 and Wald(1) = LHD1 vs. LHD2 p < .05 LHD1 = LHD2 .0097 for LHD2). RHD1 vs. RHD2 p < .05 RHD1 = RHD2

Visual LHD1 vs. RHD1 p < .005 LHD1 > RHD1 Experiment 2 LHD1 vs. RHD2 p < .05 LHD1 > RHD2 Accuracy Data LHD2 vs. RHD1 p < .05 LHD2 > RHD1 In MR of Shepard and Metzler’s stimuli, LHD1 per- LHD2 vs. RHD2 p < .01 LHD2 > RHD2 formed the visual operation better than RHD1 and RHD2 (Wilcoxon, p < .005, and p < .05, respectively), LHD1 vs. LHD2 p < .05 LHD1 = LHD2 who in turn were more accurate than LHD1 in mentally RHD1 vs. RHD2 p < .05 RHD1 = RHD2 rotating the stimuli by means of the motor strategy (Wilcoxon, both p < .001, see Figure 2A). LHD2 per- Levels of significance are from the Wilcoxon Test. formed the visual task better than RHD1 (Wilcoxon, p < .05) and RHD2 (Wilcoxon, p < .01), whereas RHD1 and RHD2 were more accurate than LHD2 with the 6.09, p < .05 for LHD1; F(1,69) = 9.44, p < .005 for motor strategy but the difference did not reach signifi- RHD1; F(1,38) = 6.09, p < .05 for RHD2] but not the cance (all p > .05, see Table 1). visual strategy (all p > .05, ns) (see Figure 1B and C). LHD1’s (z = À11.24, p < .0001) and LHD2’s (z = In addition, when patients performed the motor stra- À3.06, p < .005) ability to imagine turning the Shepard tegy, we found a significant main effect of view ( p < .05 and Metzler’s stimuli as if they turn it with their own for LHD1, LHD2, and RHD2; p < .01 for RHD1) and a hand was pathological when compared with controls, significant interaction View  L–M orientation  Hand- but in the normal range when they adopted the visual edness ( p < .005 for LHD2; p < .05 for LHD1, RHD1 strategy ( p > .05). Unlike for the visual strategy (RHD1: and RHD2). By contrast, such effects were not significant z = À11.3, p < .0001; RHD2: z = À6.6, p < .0001), when she performed the visual task (all p > .05, ns) (see RHD1 and RHD2’s ability to mentally rotate stimuli using Figure 1D and E). The same pattern of results was found the motor operation was normal ( p > .05).

Tomasino and Rumiati 881 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/089892904970753 by guest on 27 September 2021 RT Data Keller, Eddy, & Thulborn, 1999; Iwaki, Ueno, Imada, & Onoike, 1999; Kosslyn et al., 1998; Thagaris et al., 1998; The RTs analysis showed that the lateral–medial gradi- Alvisatos & Petrides, 1997; Richter et al., 1997; Cohen ent was significant only when patients solved the MR et al., 1996), in the premotor region (Kosslyn et al., 1998; using the motor strategy [F(1,44) = 6.51, p < .05 for Parsons & Fox, 1998; Richter et al., 1997, 2000, 2002; LHD1; F(1,34) = 6.54, p < .05 for LHD2; F(1,37) = Cohen et al., 1996), and in the primary motor cortex for 4.98, p < .05 for RHD1; and F(1,58) = 4.55, p < .05 for body parts (Ganis et al., 2000; Kosslyn et al., 1998). RHD2] but not the visual strategy (all p > .05, ns, see In this study, patients were selected for having a Figure 2B and C). lesion affecting any of the areas found activated in the

above studies. Thus, independent of the stimulus to be Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/16/5/878/1758193/089892904970753.pdf by guest on 18 May 2021 rotated, RHD patients were found to be selectively DISCUSSION impaired in performing operations based on the refer- Based on neuropsychological reports, it would seem ence frame of the object (‘‘visual strategy’’), but still able that MR is stimulus-specific with the RH specialized for to perform viewer-based MR (‘‘motor strategy’’). By external objects and the LH for body parts (Tomasino contrast, LHD patients were found to be selectively et al., 2003; Rumiati et al., 2001). However, in recent impaired in MR based on the viewer’s reference frame, neuroimaging studies, it was argued that MR is not leaving object-based MR intact. stimulus-bound but is rather influenced by the strategy We suggest that the reference frame and the strategy adopted for the transformation (Kosslyn et al., 2001) accounts are different ways of describing the same and/or the frame of reference in which MR takes place mental operation. On the one hand, MR as a conse- (Zacks et al., 1999, 2000, 2002). In the present study, quence of subjects’ own action can be interpreted as a we provided evidence in favor of a view that combines viewer-based operation. This interpretation is supported these proposed accounts. We described a double disso- here also by the effect of biological constraints on sub- ciation between deficits in object- or viewer-based MR as jects’ RTs as the motor transformation is best modeled a consequence of a lesion affecting any one area of in a somatic or biomechanical space (involving somato- a brain network sustaining MR. Functional imaging topic representations) and not in the visual space. Our studies on MR abilities have reported robust activations results replicate those of Parsons and Fox (1998), who in the visual areas (Jordan, Heinze, Lutz, Kanowski, & tested the effect of biological constraints on MR of hand Jancke, 2001; Kosslyn et al., 1998; Richter, Ugurbil, shapes only. However, in the present study, we demon- Georgopoulos, & Kim, 1997; Cohen et al., 1996), in strated that, if explicitly required, MR of external objects the parietal cortex (Zacks et al., 1999, 2002; Jordan too can be performed in a somatic or biomechanical et al., 2001; Podzebenko, Egan, & Watson, 2001; Vinger- space. On the other hand, the absence of an effect of hoets et al., 2001; Harris et al., 2000; Carpenter, Just, biological constraints on MR of the stimulus in the visual

Figure 2. Experiment 2: MR of 3-D cubes. (A) Percent of correct responses adopting either the motor or the visual strategy. (B) The L–M gradient. Mean RTs (msec) of correct responses as a function of the lateral and medial stimulus orientation in performing the MR of hands using the motor (B) and the visual (C) strategy.

882 Journal of Cognitive Neuroscience Volume 16, Number 5 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/089892904970753 by guest on 27 September 2021 space indicates that this is an operation that takes the ing a different brain network either within the LH or the object as a reference frame. RH. In addition, the fact that in performing the same task As far as the idea that MR is stimulus-bound, we argue some subjects may have implicitly adopted the motor that the nature of the stimulus can have a role in strategy, but others the visual strategy, can explain why triggering the motor or the visual strategy. Are these in some studies a bilateral activation within the MR net- results in conflict with previous neuropsychological re- work is found ( Jordan et al., 2001; Carpenter et al., 1999; ports calling for a hemispheric specialization in MR Thagaris et al., 1998; Richter et al., 1997; Cohen et al., according to the type of stimulus (Tomasino et al., 1996) However, in order to better characterize the brain 2003)? In that particular study, LHD patients were im- network in the RH and the LH, more observations need

paired in MR of hands but not of external objects, to be collected. Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/16/5/878/1758193/089892904970753.pdf by guest on 18 May 2021 whereas RHD patients showed the opposite pattern. That different brain networks can support a cognitive In the present study, where patients were explicitly en- operation according to its purpose has been argued for couraged to apply either the motor or the visual strategy, instance by Milner and Goodale (1995), who showed LHD patients failed to rotate both types of stimuli when that visual information can be used either ventrally for the operation was solved by means of a motor strategy, identifying objects (the ‘‘what’’ stream) or dorsally for whereas they succeeded when the alternative visual guiding action (the ‘‘how’’ stream). Goodale, Milner, strategy was used. Kosslyn et al. (1998) suggested that, Jacobson, and Carey (1991) and Milner et al., 1991 in the absence of clear instructions, subjects may spon- extensively studied DF, a patient with visual form agno- taneously adopt one or the other strategy to perform MR. sia following a bilateral occipital lesion, and found that However, if this were true, then patients, for instance she was severely impaired at perceptually judging the with an impairment in rotating hands using an implicit orientation of a line and at showing with her fingers the motor strategy, could complete MR by applying the dimensions of objects visually presented. Conversely, DF spared visual strategy. But this is not what LHD patients was able to orient her hand in a posting task as well as did to overcome the deficit of MR of hands in Tomasino to execute normal reaching–grasping movements. The et al.’s (2003) study. The two rotation methods might be opposite pattern was observed in patient RV, with a implicitly selected according to whether the mental bilateral occipital lesion, who failed to grasp objects operation intrinsically requires imagining limb move- whose visual shape he was almost perfectly able to ments (somatomotor operation) or the motion of visual identify (Goodale et al., 1994). Note that apparently objects (visuospatial operation). the lesion affects the occipital lobes in both patients, The behavioral double dissociation between motor although it is smaller in RV than in DF. However, for and visual strategies in MR reflects complementary patient RV a disconnection between the occipital and specializations of the opposite hemispheres. Object- the parietal cortex is held to be the cause of impaired based spatial transformations appear to be particularly reaching and grasping movements. In the same vein, dependent on the RH, whereas LH regions seem to be Weiss, Marshall, Zilles, and Fink (2003) showed that dedicated to the transformation and alignment of one’s differential neural mechanisms were enhanced when egocentric reference frame to that of the stimulus (Zacks subjects solved the line bisection task either manually et al., 1999, 2002). The double dissociation, logically (action) or as perceptual judgments (perception). In used in neuropsychology to argue for the modularity of particular, in the latter condition, an activation lateral- two processes or systems underlying two tasks (Shallice, ized to the RH (inferior parietal cortex, anterior cingu- 1988), in this context, allows us to argue that there are late, dorsolateral prefrontal cortex) and bilaterally to the two qualitatively distinct ways of imagining a stimulus extrastriate and superior temporal cortex was found. By rotating, and that these two operations are performed contrast, manual bisection task enhanced activation in by unilateral brain networks. Therefore, the apparent the extrastriate, superior parietal, and premotor cortex inconsistency between brain imaging results with re- bilaterally. In addition, Ruby and Decety (2000) showed spect to whether MR is lateralized to the RH (Zacks that when subjects imagined themselves manipulating et al., 1999, 2002; Podzebenko et al., 2001; Harris et al., an object, only regions in the LH were enhanced (i.e., 2000), or to the LH (Vingerhoets et al., 2001; Alvisatos & inferior parietal lobe, precentral gyrus, superior frontal Petrides, 1997) is explained by the argument that sub- gyrus occipito-temporal junction), whereas when they jects carry out MR using either the motor or the visual imagined the experimenter doing the same, areas in the strategy each involving a different brain network either RH were enhanced (inferior parietal, precuneus, poste- within the LH or the RH. Therefore, the apparent rior cingulated, and frontopolar cortex). Finally, Vogeley inconsistency between brain imaging results with re- and Fink (2003) reviewed several imaging studies show- spect to whether MR is lateralized to the RH (Zacks ing how spatial cognitive tasks activated left parieto- et al., 1999, 2002; Podzebenko et al., 2001; Harris et al., temporal and frontal regions when solved from a first 2000), or to the LH (Vingerhoets et al., 2001; Alvisatos & person perspective (1PP) and right parietal and frontal Petrides, 1997) can be due to subjects carrying out MR regions from a third person perspective (3PP). These using either the motor or the visual strategy each involv- results are likely to reflect the LH dominance for action

Tomasino and Rumiati 883 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/089892904970753 by guest on 27 September 2021 and goal-directed motor behavior (and apraxia) and the 1966). However, RHD’s ability in mentally rotating ex- RH dominance for space. We add that this pattern of ternal objects was pathological (Flags task, 17/24) (Thur- hemispheric specialization is comparable with the one stone & Jeffrey, 1983). we found in the MR domain. RHD2 is a 51-year-old, right-handed woman with 13 years of education, admitted to the neurosurgery METHODS unit with a meningioma affecting the right premotor area (for more neuroradiological details, see Figure 3d The Cases and 4d). RHD2 did not show aphasia or short-term In our study, patients were selected for having a lesion memory problems (WAIS Memory Scale, verbal span,

within the brain network supporting MR operations, 6, Wechsler, 1981). RHD2’s ability to mentally rotate Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/16/5/878/1758193/089892904970753.pdf by guest on 18 May 2021 thus the inclusion criteria were the presence of a lesion external objects was impaired (17/24, Thurstone & affecting any area of that circuit, that is either the Jeffrey, 1983). None of the patients had right–left disori- posterior parietal cortex or the posterior frontal premo- entation as assessed in the baseline of the Little Man tor area. All four patients met these criteria and there Test (Ratcliff, 1979) or visuospatial agnosia based on were also some overlapping lesions across patients. Visual Object and Space Perception [VOSP] (Warrington LHD1 is a right-handed Italian man born in 1951 with & James, 1991). All patients were tested 6 months after 17 years of education, who in 1998 was admitted to the the illness onset. neurological ward with a right hemiparesis and aphasia. The patients’ performance on the MR experiment was CT scans showed that the lesion involved left Brod- compared with the performance of 14 control subjects of mann’s area (BA) 45 subcortically and BAs 41, 42, 43, comparable age (mean = 48.41 ± 8.8 years) and educa- and 44 superficially. Part of the left superior (BA 7) and tion (mean = 14.35 ± 2.3 years). Informed consent was inferior parietal lobe (BA 40) was also involved, sparing obtained from all patients and subjects. BA 5 and BA 39 (for more neuroradiological details, see Figure 3A and 4A). LHD1 had a normal intelligence Procedure (WAIS, 93) (Wechsler, 1984), but had aphasia of Broca’s During the experimental session, patients sat at a table in type and apraxia (imitation of actions, 45/72) (De Renzi, front of a computer screen and maintained their hands Motti, & Nichelli, 1980); pantomime of the use of still on their knees. Subjects performed two experiments objects following visual (17/28), tactile (19/28), and in which they were required to perform MR of hands verbal (23/28) presentation (Zanini, Bearzotti, Vorano, (Experiment 1) and of Shepard and Metzler’s stimuli De Luca, & Rumiati, 1999). He performed pathologically (Experiment 2) in which they were instructed to use a on an MR of hand shapes task (15/21) (Luria, 1966), but motor and a visual strategy (see Figure 5A and B). normally on an MR of external objects such as flag Accuracy and RTs were collected. The study was ap- shapes (50/60) (Thurstone & Jeffrey, 1983). proved by the SISSA Ethical Committee. LHD2, a right-handed woman, born in 1920, with 8 years of schooling, was admitted to the hospital with Experiment 1: MR of Hands a left ischemic cortico-subcortical lesion of the temporo- parietal regions. The lesion involved BAs 40, 39, and 22 Twenty-four line drawings of open hand shapes were (for more neuroradiological details see Figure 3b and presented one at time on a computer screen 20 cm away 4b). Normally intelligent (WAIS, 94) (Wechsler, 1984), from the subjects. Hands varied for handedness (50% LHD2 had Wernicke’s aphasia as well as apraxia (imita- were left, and 50% right), view (palm, back and wrist), tion of actions, 42/72) (De Renzi et al., 1980); pantomime and orientation (hands were rotated by steps of 608 in of objects use visual (17/28), tactile (19/28), and verbal the picture plane). In the ‘‘motor task,’’ subjects were (23/28) presented (Zanini et al., 1999). LHD2 failed in a asked to decide whether the stimulus depicted a right test in which she had to perform MR operations (Flags or a left hand (see Figure 5A, upper left panel). To solve task, 29/40) (Thurstone & Jeffrey, 1983) and in deciding this task, it is known that subjects mentally rotate the the handedness of rotated hand postures (Luria’s task, motor image of their own hand (i.e., egocentric perspec- 11/21, Luria, 1966). tive) and match it with the target stimulus (Parsons, RHD1 is a right-handed man born in 1930 with 5 years 1987a, 1994), while the frame of reference of the stimulus of education who was admitted to the hospital with an hand was irrelevant for the task. In the ‘‘visual task,’’ ischemic lesion in the territory of the right middle patients were asked to imagine the stimulus rotating cerebral artery. The cortico-subcortical lesion involved until it reached the upright position and then decide the right parietal lobe and the basal ganglia (for more whether the thumb was on the left or on the right side of neuroradiological details, see Figure 3c and 4c). RHD1 the line drawing of the hand, while ignoring their own showed no sign of aphasia, visual agnosia or neglect hands (see Figure 5B, upper right panel). In solving and had no memory deficits (WAIS Memory Scale, digit the visual task, an MR operation is also required but it span, 6, Wechsler, 1981). RHD1 succeeded in deciding implies a mental transformation based on the object’s whether a rotated hand was a left or a right one (Luria, reference frame (i.e., allocentric prospective) in order to

884 Journal of Cognitive Neuroscience Volume 16, Number 5 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/089892904970753 by guest on 27 September 2021 Figure 3. Patients’ CT or fMRI scans. (A) LHD1: Hemorrhagic lesion involves the superior parietal cortex and the posterior frontal area (the primary motor area was involved but not the somatosensory area), part of the temporal lobe (Sylvian region), and the basal ganglia all in the LH. (B) LHD2: Ischemic cortico- subcortical lesion in the left inferior temporo-parietal Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/16/5/878/1758193/089892904970753.pdf by guest on 18 May 2021 junction. The lesion involves also the white matter in the frontal region. (C) RHD1: She had a meningioma affecting the right premotor area. (D) RHD2: Hemorrhagic cortico- subcortical lesion in the RH parietal lobe and basal ganglia.

reorient the picture and then make the spatial decision, that subjects did not mentally reorient the stimuli. In while the reference frame based on their own hands was particular, if subjects respond in this way, all the errors irrelevant for the task. Responses given according to occur for stimuli at 1508 both L- and M-orientations. In whether the thumb appeared on the left or the right of fact, in this condition, the response given according to the computer screen instead of taking the picture as the whether the thumb is on the left or right side of the reference for the spatial judgment were wrong, showing screen and responses based on the required spatial

Tomasino and Rumiati 885 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/089892904970753 by guest on 27 September 2021 Figure 4. Templates of lesion reconstruction in the patients. For mapping the templates of lesion reconstruction, we followed the method suggested by Luzzatti, Scotti, and Gattoni (1979). Octagon texture ( ) represents subcortical and stone-wall texture ( ) cortical lesions. (A) LHD1; (B) LHD2; (C) RHD1; (D) RHD2. Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/16/5/878/1758193/089892904970753.pdf by guest on 18 May 2021

judgment are incompatible (i.e., subjects respond right Experiment 2: MR of Shepard and Metzler’s Stimuli if, for instance, the thumb is on the right side of the screen, but the correct response is left, because it is on Twenty-four pairs of pictures of three-dimensional the left side of the line drawing of the hand shape). Thus, branching forms were presented one at a time on a in order to disambiguate this type of error, we examined computer screen. For each pair, stimuli were placed one the effect of DR on accuracy data. (oriented upright) above the other (the rotated one). In

Figure 5. Examples of the stimuli used in Experiments 1 (i.e., hands) and 2 (Shepard and Metzler’s cubes) are shown. For each experiment, the left- side panel shows examples of stimuli (A and C) administered in the motor task, whereas the right-side panel shows examples of stimuli (B and D) administered in the visual task.

886 Journal of Cognitive Neuroscience Volume 16, Number 5 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/089892904970753 by guest on 27 September 2021 the ‘‘motor task,’’ subjects were required to decide Carpenter, P. A., Just, M. A., Keller, T. A., Eddy, W., & Thulborn, whether a counterclockwise (CTW) or a clockwise K. (1999). Graded functional activation in the visuospatial system with the amount of task demand. Journal of (CW) rotation would be the shortest path from that Cognitive Neuroscience, 11, 9–24. orientation to upright (see Figure 5C, lower left panel). Cohen, M. S., Kosslyn, S. M., Breiter, H. C., DiGirolamo, G. J., They were asked to imagine grasping the Shepard and Thompson, W. L., Anderson, A. K., Bookheimer, S. Y., Rosen, Metzler stimulus with their left hand and rotating it in B. R., & Belliveau, J. W. (1996). Changes in cortical activity order to align this stimulus in the orientation shown by during mental rotation: A mapping study using functional MRI. Brain, 119, 89–100. the target cube. In the ‘‘visual task,’’ the same pairs of Corballis, M. C. (1997). Mental rotation and the right stimuli (n = 24) were presented but now the rotated hemisphere. Brain and Language, 57, 100–121. cube (the one below) contained a red square on its left Corballis, M. C., & Sergent, J. (1989). Hemispheric Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/16/5/878/1758193/089892904970753.pdf by guest on 18 May 2021 or on its right face (see Figure 5D, lower right panel). specialization for mental rotation. Cortex, 25, 15–25. Subjects were required to decide on which face (the left Davidoff, J., & Warrington, E. (1999). In G. W. Humphreys (Ed.), Case studies in neuropsychology of vision or the right one) of the cube the red square was (pp. 59–79). UK: Psychology Press. depicted. Responses given according to whether the Decety, J., Perani, D., Jeannerod, M., Bettinardi, V., Tadary, B., red square appeared in the left or right space of the Woods, R., Mazziotta, J. C., & Fazio, F. (1994). Mapping computer screen and not on the L–R face of the Shepard motor representations with positron emission tomography. and Metzler stimulus were considered failures and Nature, 371, 45–52. De Renzi, E., Motti, F., & Nichelli, P. (1980). Imitating gestures: showed that subjects did not perform an MR. The order A quantitative approach to ideomotor apraxia. Archives of of the task (motor and visual) and the type of stimulus Neurology, 37, 6–10. (hands and Shepard and Metzler stimuli) was counter- Ditunno, P. L., & Mann, V. A. (1990). Right hemisphere balanced. The same experiment (MR of hands, MR of 3D specialization for mental rotation in normals and brain cubes) was performed three times in the same session damaged subjects. Cortex, 26, 177–188. Farah, M. J., & Hammond, K. M. (1988). Mental rotation and for a total of 240 trials. LH and RH patients and control orientation-invariant object recognition: Dissociable subjects responded by pressing two different buttons processes. Cognition, 29, 29–46. with the index and the middle fingers of the left and right Ganis, G., Keenan, J. P., Kosslyn, S. M., & Pascual Leone, A. hand, respectively. (2000). Transcranial magnetic stimulation of primary motor How do we know that patients and control subjects are cortex affects mental rotation. Cerebral Cortex, 10, 175–180. rotating the stimulus according to one or the other Goodale, M. A., Meenan, J. P., Bulthoff, H. H., Nicolle, D. A., strategy? If subjects are adopting the motor strategy, Murphy, K. J., & Racicot, C. I. (1994). Separate neural RTs are predicted to be faster for medial than for pathways for the visual analysis of object shape in perception lateral orientations, that is, orientations toward and away and prehension. Current Biology, 4, 604–610. from the body’s midsagittal plane, respectively. This Goodale, M. A., Milner, A. D., Jakobson, L. S., & Carey, D. P. (1991). A neurological dissociation between perceiving effect is due to the physical arm–hand constraints on objects and grasping them. Nature, 349, 154–156. MR (Parsons, 1987a, 1987b, 1994). When subjects imag- Harris, I. M., Egan, G. F., Sonkkila, C., Tochon-Danguy, H. J., ine the stimulus rotating in the visual space (i.e., visual Paxinos, G., & Watson, J. D. G. (2000). Selective right parietal strategy), such an effect should disappear. These RT lobe activation during mental rotation. Brain, 123, 65–73. patterns would be consistent with the view that the Iwaki, S., Ueno, S., Imada, T., & Tonoike, M. (1999). Dynamic cortical activation in mental image processing revealed by motor strategy and the visual strategy are based on biomagnetic measurement. NeuroReport, 10, 1793–1797. egocentric and allocentric transformations, respectively. Jordan, K., Heinze, H. J., Lutz, K., Kanowski, M., & Jancke, L. (2001). Cortical activations during the mental rotation of different visual objects. Neuroimage, 13, 143–152. Acknowledgments Kosslyn, S. M., DiGirolamo, G. J., Thompson, W. L., & Alpert, N. M. (1998). Mental rotation of objects versus hands: Neural We thank Ms. Lorenza Vorano for helping to collect some of mechanisms revealed by positron emission tomography. the data, Prof. Miran Skrap and Dr. D’Agostino for neuro- Psychophysiology, 35, 151–161. radiological information, and the members of the Cognitive Kosslyn, S. M., Holtzman, J. D., Farah, M. J., & Gazzaniga, M. S. Neuroscience Sector for helpful comments. (1985). A computational analysis of mental image Reprint requests should be sent to Raffaella Rumiati, Scuola generation: Evidence from functional dissociations in Internazionale Superiore di Studi Avanzati (SISSA), Via Beirut split-brain patients. Journal of Experimental Psychology: 2–4, 34014 Trieste, Italy, or via e-mail: [email protected]. General, 114, 311–341. Kosslyn, S. M., Thompson, W. L. Wraga, M., & Alpert, N. M. (2001). Imagining rotation by endogenous versus exogenous REFERENCES forces: Distinct neural mechanisms. NeuroReport, 12, 2519–2525. Alvisatos, B., & Petrides, M. (1997). Functional activation of the Luria, A. R. (1966). Higher cortical functions in man. London: human brain during mental rotation. Neuropsychologia, 35, Tavistock. 111–118. Luzzatti, C., Scotti, G., & Gattoni, A. (1979). Further Bricolo, E., Shallice, T., Priftis, K., & Meneghello, F. (2000). suggestions for cerebral CT-localization. Cortex, 15, 483–490. Selective space transformation deficit in a patient with Metha, Z., & Newcombe, F. (1991). A role for the left spatial agnosia. Neurocase, 6, 307–319. hemisphere in spatial processing. Cortex, 27, 153–167.

Tomasino and Rumiati 887 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/089892904970753 by guest on 27 September 2021 Milner, A. D., & Goodale, M. A. (1995). The visual brain in & Agid, Y. (1996). The mental representation of hand action. Oxford: Oxford University Press. movements after parietal cortex damage. Science, 273, Milner, A. D., Perret, D. I., Johnston, R. S., Benson, P. J., 1564–1568. Jordan, T. R., Heeley, D. W., Bettucci, D., Mortara, F., Thagaris, G. A., Kim, S. G., Strupp, J. P., Andersen, P., Ugurbil, Mutani, R., Terrazzi, E., & Davidson, D. L. W. (1991). K., & Georgopoulos, A. P. (1998). Mental rotation by Perception and action in visual form agnosia. Brain, 114, functional magnetic resonance imaging at high field 405–428. (4 tesla): Performance and cortical activation. Journal of Morton, N., & Morris, R. G. (1995). Image transformation Cognitive Neuroscience, 9, 419–432. dissociated from visuospatial working memory. Cognitive Thurstone, L. L., & Jeffrey, T. E. (1983). Flags: A test of Neuropsychology, 12, 767–791. spatial thinking. In J. Eliot & I. Macfarlane Smith (Eds.), Parsons, L. M. (1987). Imagined transformation of one’s body. An international directory of spatial tests (pp. 197–198). Journal of Experimental Psychology: General, 116, 172–191. Atlantic Highlands, NJ: Humanities Press. Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/16/5/878/1758193/089892904970753.pdf by guest on 18 May 2021 Parsons, L. M., & Fox, P. T. (1998). The neural basis of implicit Tomasino, B., Toraldo, A., & Rumiati R., I. (2003). Dissociation movements used in recognizing hand shape. Cognitive between the mental rotation of visual images and motor Neuropsychology, 15, 583–615. images in unilateral brain-damaged patients. Brain & Parsons, M. L. (1987). Imagined spatial transformations of Cognition, 51, 368–371. one’s hands and feet. Cognitive Psychology, 19, 178–241. Vingerhoets, G., Santens, P., Van Laere, K., Lahorte, P., Parsons, M. L. (1994). Temporal and kinematic proprieties of Dierckx, R. A., & De Reuck, J. (2001). Regional brain activity motor behavior reflected in mentally simulated action. during different paradigms of mental rotation in healthy Journal of Experimental Psychology: Human Perception volunteers: A position emission tomography study. and Performance, 20, 709–730. Neuroimage, 13, 381–391. Podzebenko, K., Egan, G. F., & Watson, J. D. G. (2001). Vogeley, K., & Fink, G. R. (2003). Neural correlates of the first- Widespread dorsal stream activation during a parametric person-perspective. Trends in Cognitive Sciences, 7, 38–42. mental rotation task, revealed with functional magnetic Warrington, E. K., & James, M. (1991). VOSP The Visual Object resonance imaging. Neuroimage, 0, 1–12. and Space Perception Battery. Bury St. Edmunds: Thames Ratcliff, G. (1979). Spatial thought, mental rotation and the Valley Test Company. right cerebral hemisphere. Neuropsychologia, 17, Wechsler, D. (1981). Wechsler Memory Scale.OS 49–54. Organizzazioni Speciali Firenze. Richter, W., Ugurbil, K., Georgopoulos, A., & Kim, S. G. Wechsler, D. (1984). Wechsler Adults Intelligence Scale. OS (1997). Time-resolved fMRI of mental rotation. NeuroReport, Organizzazioni Speciali Firenze. 8, 3697–3702. Weiss, P. H., Marshall, J. C., Zilles, K., & Fink, G. R. (2003). Are Ruby, P., & Decety, J. (2000). Effect of subjective perspective action and perception in near and far space additive or taking during simulation of action: a PET investigation of interactive factors? Neuroimage, 18, 837–846. agency. Nature Neuroscience, 4, 546–550. Wraga, M., Creem, S. H., & Proffitt, D. R. (1999). The influence Rumiati, R. I., Tomasino, B., Vorano, L., Umilta`, C., & of spatial reference frames on imagined object and viewer De Luca, G. (2001). Selective deficit of imagining finger rotations. Acta Psychologica, 102, 247–264. configurations. Cortex, 37, 730–733. Zacks, J., Mires, J., Tversky, B., & Hazeltine, E. (2000). Mental Shallice, T. (1988). From neuropsychology to mental spatial transformations of objects and perspective. Spatial structure. Cambridge: Cambridge University Press. Cognition and Computation, 2, 315–332. Shepard, R. N., & Cooper, L. A. (1982). Mental images and Zacks, J. M., Ollinger, J. M., Sheridan M. A., & Tversky, B. their transformations. Cambridge: MIT Press. (2002). A parametric study of mental spatial transformations Shepard, R. N., & Metzler, J. (1971). Mental rotation of of bodies. Neuroimage, 6, 857–872. three-dimensional objects. Science, 171, 701–703. Zacks, J., Rypma, B., Gabrieli, J. D. E., Tversky, B., & Glover, Sirigu, A., & Duhamel, J. R. (2001). Motor and visual imagery G. H. (1999). Imagined transformations of bodies: An fMRI as two complementary but neurally dissociable mental investigation. Neuropsychologia, 37, 1029–1040. processes. Journal of Cognitive Neuroscience, 13, Zanini, S., Bearzotti, F., Vorano, L., De Luca, G., & Rumiati, R. I. 910–919. (1999). The assessment of utilization apraxia. Europa Sirigu, A., Duhamel, J. R., Cohen, L., Pillon, B., Dubois, B., Medicophysica, 35, 119–124.

888 Journal of Cognitive Neuroscience Volume 16, Number 5 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/089892904970753 by guest on 27 September 2021