From: AAAI Technical Report FS-96-02. Compilation copyright © 1996, AAAI (www.aaai.org). All rights reserved. Mental Rotation - A Case of Embodied Action Andreas Wohlschläger Max-Planck-Institute for Psychological Research Unit Cognition and Action Leopoldstraße 24 D-80802 München Germany [email protected] Abstract movement makes it convenient to call it a mental rotation. A new view of mental rotation in humans is presented. These results demonstrate how analog operations can be Rather than being a perceptual phenomenon, mental rota- performed by our brains and perhaps mental object rota- tion of objects is supposed to be an imagined action in the tion is performed by similar changes in the activity pat- sense that its only difference to real action is the absence terns of cell assemblies. Notably, these results were ob- of motor output. A series of experiments is reported which tained by measuring cell activities in the motor cortex. shows that the difference in speed between mental and Cognitive psychologists, so far, mainly have tried to manual rotation are negligible and that performing rota- tional hand movements interferes with mental rotation and explain the analog nature of mental rotation by looking vice versa. It also could be shown that the preparation of for the relation between mental rotation and the percep- rotational hand movements is already sufficient to influ- tion of rotary motion. Shepard & Judd (1976) addressed ence mental rotation. The general role of motor processing this question by investigating apparent rotational motion in dynamic visual imagery is discussed, considering the and its relation to mental rotation. However, mental rota- underlying neurophysiology. tion speed was about 50 to 60°/s, whereas the slope of the optimum apparent motion function calculates to 1000°/s. Mental Rotation, Perception and Action Furthermore, Friedman & Harding (1990) showed that mental rotation speed depends on the axis of rotation, Mental rotation occurs when a decision has to be made whereas the apparent motion illusion does not. They con- whether two objects differing in orientation are identical cluded that mental rotation and apparent motion don't or mirror-images of each other. The angular disparity have much in common. In addition mental rotation is between stimuli is varied systematically and response strategic (Just & Carpenter 1985) compared to the largely times (RT) and errors are measured. The typical, most automatic processing of apparent motion perception. intriguing result found in many mental rotation studies is Hence, the consultation of the perception of real motion the almost perfect linear increase of RT with angular as the main explanation for the mental rotation phenome- stimulus disparity. Together with introspective reports non is somewhat unsatisfactory. Mental rotation is obvi- from participants and experimenters, these core findings ously a much higher-level process than the perception of led to the term "mental rotation", since it resembles the real or apparent motion. time course of a physical rotation with constant angular Nevertheless, there have been further, more direct at- velocity. tempts to prove the participation of rotary movement This analog nature of the mental rotation process, most perception in mental rotation. Corballis & McLaren ( impressively demonstrated by Cooper (1976) piqued psy- 1982) could show, that inducing a rotary aftereffect by chologists' interest. The resemblance of mental rotation to means of a rotating textured disk influenced RT of the external physical rotation, however, demands for a mental mental rotation of subsequently presented alphanumeric process mimicking external physical rotation, and it is characters. Compared to the standard experiment, RT still not clear how such an analog process is implemented were increased when the aftereffect was in the opposite in our brains. Recent electrophysiological studies could direction (discordant condition) of the presumed mental measure continuous changes in the activity pattern of cell rotation. However, Jolicoeur & Cavanagh (1992) could assemblies in monkeys performing a visuomotor mental exclude the participation of low-level motion analysis rotation task (Georgopoulos et al. 1989). They found that centers in the mental rotation process. Although finding the neuronal population vector - calculated from the cell slight differences in overall RT level when presenting assemblies' activity pattern - continuously changed its alphanumeric stimuli in different surface media, there was direction prior to the onset of a movement pointing 90° to no effect of surface medium on mental rotation rate. A the left of a target light. The fact, that the neuronal popu- pronounced effect was only produced when rotating the lation vector rotated prior and not in parallel to the characters about a small angle by means of apparent mo tion. Concordant rotation of presented characters acceler- individuals the opportunity to turn the stimuli on the dis- ated RT of mental rotation, whereas rotations opposite play by means of a knob fixed to an axis. (discordant) to the presumed mental rotation direction led Subjects' mean correct RT as well as the direction and to delayed responses, compared to a neutral condition. amount of manual rotation were measured and analyzed. Obviously, figure motion interacts to some degree with Similar to standard mental rotation experiments, in both, mental rotation. Summarizing their results, Jolicoeur & the mental and the manual rotation condition, RT in- Cavanagh (1992) concluded that mental rotation occurs at creased linearly with the amount of angular disparity a relatively high and perhaps abstract level of processing. between stimuli. Speed of manual as well as speed of This agrees with a principal difference between motion mental rotation depended on the axis of rotation. Manual perception and mental rotation. Whereas motion percep- and mental rotation functions matched each other with the tion is a rather automatic process, mental rotation is stra- exception of the oblique B-axis, running upward and away tegic and shares some characteristics with voluntary ac- from the subject. Because the B-axis differed from the tions. Mental rotation can be started and stopped volun- Cartesian axes in two other aspects, its results will be tarily (Cooper 1976) and even its speed is at free choice reported and discussed later on. When comparing both (Cooper & Shepard 1973). The processes engaged in rotation conditions, speeds of manual and mental rotation motion perception might well participate in imagining were not different for any of the three Cartesian axes. The rotating objects, but there should be a higher process same rank order of speeds among these axes was found as steering these dynamic imaginations, because they are in Parsons's (1987) experiment, and it was identical for quasi completely under voluntary control. Considering its the manual and mental rotation condition.. Thus, it can be similarity to voluntary actions, it might be possible that concluded that, at least for Cartesian axes, mental and motor or premotor processes are involved in mental rota- manual rotation are similar processes in a phenomenal tion, a conclusion which Kosslyn (1994) meanwhile sense. draws complementary to his earlier work. More precisely, What about the oblique B-axis? It differed in two as- this means that the processes engaged in rotary object pects from the three Cartesian axes. First, manual rotation manipulation might contribute to mental rotation. In this speed for the oblique B-axis was about twice as fast as view, rotating something mentally is rather an imagined mental rotation speed. Second, the threshold function for action than the perception-like imagination of an object in the shortest direction of rotation was substantially differ- rotation. Let us assume that rotating an object mentally is ent from those of the other axes. For trials with 120° something very similar to rotating it physically (e.g., with through 240° stimulus orientation students rotated in one's hand) in the sense that the same processes initiate either direction with equal frequency. Obviously, indi- and rule the rotation, whether it turns up mentally or it is viduals in the B-axis condition often could not figure out actually performed. This "common-processing" assump- the shortest direction. This is probably due to a general tion implies essentially two things. First, mental rotation inability to conceive the position of the axis and the angle should be commensurate with rotary object manipulation: of shortest path rotation between two orientations of an factors that affect mental rotation should have the same object, when there is no coincidence of the object's princi- effect on actual rotation. Second, both processes should pal axes, the rotation axis, and the viewer axis (Parsons, be functionally connected, i.e., depend on each other. 1995). The results of Experiment 1 show that in mental This could be investigated by having both tasks executed rotation shortest path rotation is not used in the case of the simultaneously, i.e., performing mental rotation while oblique B-axes. Otherwise the participants in the manual making rotational hand movements. rotation condition should have been able to figure out the shortest direction of rotation more accurately. Regarding the correspondence of manual and mental Experiment 1 rotation observed with the Cartesian axes, there is good Experiment 1 was designed to