theoretical perspective From Imaginal to Digital: Mental Imagery and the Computer Image Space Nick Lambert a b s t r a c t The author suggests that the intangible characteristics of computer graphics bear some resemblance to the brain’s ability to construct mental images, as outlined by veteran s a historian of computer art, I am fascinated Kosslyn and his team at Harvard researcher Stephen M. Kosslyn. A An analogy might also be drawn by the nature of digital images and the way in which people have been looking at the visual with the “Imaginal World” of the apprehend them. Images that are rendered on a screen ap- psychology and neurophysiology Sufis, as described by Henri parently exist in a plane separate to the rest of our world, of mental imagery for several de- Corbin, which exists in a space analogous to the pictorial space developed by artists but with cades. Using PET and fMRI scan- of its own. As computer graph- an animate and interactive aspect wholly its own. Their basis is ners, Kosslyn and his collaborators ics have emerged as an artistic not visual but mathematical and process-driven, although the have discovered that particular medium, one may consider how this internalized ability influences computer has evolved into a device for generating imagery. As brain areas related to color and the artist’s response to the com- we become ever more adept at creating and navigating a range shape perception activate during puter, especially as new display of virtual worlds (whether the mundane desktop or the exotic the imaginary phase. Nor is this technologies emerge. unreality of Second Life), the computer facilitates the exten- solely for imagery; imagined audi- sion of our visual imagination in the apparently unbounded tory experiences seem to activate virtual space created by mathematics. areas associated with hearing, and motor activities (e.g. imagined walking) bring responses from the corresponding areas for movement. Says Kosslyn: Mental IMagery With this in mind, it is worth considering the inherent hu- Many neuroimaging studies of “mental rotation” have been re- ported, all of which have shown that multiple brain areas are man faculty for generating wholly imaginary images, or “men- activated during mental rotation. For example, Richter et al. mea- tal imagery.” Mental imagery is succinctly defined by one of sured brain activation with fMRI while subjects mentally rotated its major exponents and researchers, Stephen M. Kosslyn of the three-dimensional multi-armed angular stimuli invented by Harvard’s Department of Psychology, quoted here by Samuel Shepard and Metzler [4]. Moulton: If this is indeed the case, then mental images are a crucial Mental imagery occurs “when a representation of the type cre- component of our cognition. They enable us to conceive of ated during the initial phases of perception is present but the objects and situations in our minds and to maneuver them. stimulus is not actually being perceived; such representations Similarly, computer graphics has given us a platform for creat- preserve the perceptible properties of the stimulus and ulti- ing controllable animate forms in a non-physical space that we mately give rise to the subjective experience of perception” [1]. have developed over the past 50 years. The basic contention is that when an imaginary scene is After numerous tests involving people imagining visual ob- called forth, for instance if one is asked to think of the shape jects, and comparing the brain activity and descriptions of this of an Alsatian dog’s ears (a typical Kosslyn example), then process to actual vision, Kosslyn has proposed the following: one visualizes the scene in the mind’s eye. The question is Because imagery relies on perceptual representations, it makes whether the brain actually forms an image or simply processes explicit and accessible the same types of information that are non-visual data. The veracity of mental imagery has been hotly registered by the senses during perception (including proprio- debated throughout the 20th century; Michael Tye’s seminal ception and kinaesthetic information). Mental imagery may book The Imagery Debate [2] outlines the basic argument be- best be understood in the context of mental simulation, specifi- cally as a kind of mental emulation [5]. tween imagery’s supporters and detractors. Zenon Pylshyn is one of the most prominent opponents of the idea that the Kosslyn goes on to use a computational analogy here: When brain forms actual images: his position is that “[internal rep- we see an object in our mind’s eye we can simulate how it resentations] are essentially conceptual and propositional, rather might appear under different conditions, translate it into vari- than sensory or pictorial, in nature” [3]. ous contexts or investigate it more closely. In a sense, we are calling an immaterial object to mind in order to study it. Obvi- ously there are elements of memory and recall in this process, and since Classical times this ability to compose active mental Nick Lambert (lecturer), Department of History of Art and Screen Media, Birkbeck, images has been deployed as a memory tool. The historian University of London, 43 Gordon Square, London WC1H 0PD, U.K. E-mail: <n.lambert @bbk.ac.uk>. of visual culture Frances Yates, in her magnum opus The Art See <www.mitpressjournals.org/toc/leon/44/5> for supplemental files associated with of Memory, famously described how this and the technique of this issue. creating “loci,” or imaginative sites, to serve as the locations for ©2011 ISAST LEONARDO, Vol. 44, No. 5, pp. 439–443, 2011 439 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/LEON_a_00245 by guest on 29 September 2021 specific stored memories developed into BrIdgIng the aBstract and means of realizing the images. The Byz- an important, though obscure, area of the MaterIal antine distinction between noetos and Western thought during the Renaissance aisthetos seemingly remains current. The There is a degree of resemblance be- [6]. Kosslyn, too, refers to it obliquely: question is to what degree the computer tween the imaginative simulation of the enables an interchange between both ar- mundus imaginalis and the digital simu- Memorial processes must retrieve and eas via mathematical processes rendered lation of a three-dimensional reality. later encode episodic information. in visual form. Much evidence supports this interplay I believe that one can extend the idea between memory and imagery. One uses of the “imaginal world” to the wholly imagery to simulate what one would per- ceive if one were in a specific situation; externalized digital worlds that subsist the coMputer as a this is as true of imagery used to retrieve inside a space of their own, one that is VIsual MedIuM memories as it is of imagery used to pre- seen through the computer screen. My Although computer images can be de- dict the future. As such, imagery and contention is that computer users appre- scribed as grids of pixels, they really simulation are joined at the hip, and hend the intangible visual space of the should be studied together [7]. consist of instructions: In other words, screen with similar processes to those their constituents are not visual (or physi- used for internalized mental imagery. cal). The artist can entirely redefine the This was underscored for me when I read the IMagInal realM structure of the program that gives rise to the following in Foley and Van Dam’s The word “imaginal” may be unfamiliar these images and sits outside this tightly primer on computer graphics: but it is appropriate in this context. The defined reality formed of instructions. French philosopher and researcher of [Computer graphics involves] the picto- In the earliest days of computer image- Sufism Henri Corbin identified a facet rial synthesis of real or imaginary objects making, there was a conscious decision of Sufi practice that involved the con- from their computer-based models . we to adopt some of the conventions of per- can make pictures not only of concrete templation of the alam al-mithal, a world “real-world” objects but also of abstract, spectival geometry to make computer created by the visual imagination that was synthetic objects [and] of data that have graphics represent 3D space properly not “imaginary” in a pejorative sense but no inherent geometry [10]. on a flat screen. While Ivan Sutherland’s rather had an independent existence be- Sketchpad of 1961 had introduced interac- tween the physical and spiritual planes. This informed my concept that com- tive vector graphics, it was still 2D only. Also called the “Eighth Climate” or Na- puter art differs from all traditional visual Timothy Johnson, working on MIT’s kojd-Abad, the “land of No-where,” this art in two important respects. Firstly, the Lincoln TX-2 computer, which was one zone was described as a realm with iden- artist manipulates information directly, of the few powerful enough to support tifiable cities and geography. These were without the limitations that are intrin- graphics applications, created Sketchpad imbued with symbolic meaning, where sic to a physical medium; and secondly III as a 3D version of the original. Around specific architectural features evoked the computer can respond to the artist 1963, Lawrence G. Roberts (more famous certain doctrinal and philosophical during production and to the viewer af- as a pioneer of the ARPANET, precursor points. For this reason, Corbin invented ter production, or even act as a creative to the Internet) wrote his Ph.D. thesis the term mundus imaginalis to describe agent in its own right if programmed to on 3D models created by processing 3D this place: do so. This is possible because the digi- information contained in scanned pho- tal image is intangible, like the mental tographs.