COGNITIVE SCIENCE 13, 51-78 (1989) ReinterpretingVisual Patterns in Mental Imagery RONALDA.FINKE Texas A&M University STEVENPINKER Massachusetts Institute of Technology MARTHAJ.FARAH Carnegie-Mellon University In a recent paper, Chambers and Reisberg (1985) showed that people cannot reverse classical ambiguous figures in imagery (such OS the Necker cube, duck/ rabbit, or Schroeder staircase). In three experiments, we refute one kind of ex- planation for this difficulty: that visual images da not contain information about the geometry of a shape necessary for reinterpreting it or that people connot apply shape classification procedures to the information in imagery. We show, that given suitable conditions, people con assign novel interpretations to ambigu- ous images which have been constructed out of parts or mentally transformed. For example, when asked to imagine the letter “D” on its side, affixed to the top of the letter “J”, subjects spontaneously report “seeing” an umbrella. We also show that these reinterpretations are not the result of guessing strategies, and that they speak directly to the issue of whether or not mental images of ambigu- ous figures can be recanstrued. Finally, we show that arguments from the phiios- aphy literature on the relation between images and descriptions are not relevant to the issue of whether imoges can be reinterpreted, and we suggest possible ex- planations for why classical ambiguous figures do not spontaneously reverse in imagery. At least since Berkeley’s time, the question of whether mental images can be ambiguous has held a central place in the debate over the nature of imagery. It is easy to see why the two issues are so closely related. The process of per- ception begins with the geometry of the retinal images, and ends with a description of objects in the world. The controversy over imagery has largely concerned whether images are like early perceptual representations contain- ing information about the geometric properties of visual inputs, or like later This research was supported by NIMH Grant lROlMH3980901 to Ronald A. Finke, by NSF Grant 85-18774 to Steven Pinker, and by ONR Grant NOOO14-86-K-0094 and NIH Pro- gram Project Grant NS-06209-21, and NIH Grant R23-NS-23458-01 to Martha J. Farah. We thank Ned Block, James Greeno, Stephen Kosslyn, Howard Kurtzman, Steven Palmer, ROSS Thompson, and Barbara Tversky for helpful comments and suggestions. Correspondence and requests for reprints should be sent to Steven Pinker, Department of Brain and Cognitive Sciences, MIT, Cambridge MA 02139. 51 52 FINKE, PINKER, AND FARAH cognitiverepresentations containing information about the conceptualcate- goriesof interpretedobjects (Kosslyn & Pomerantz,1977; Pylyshyn, 1973). If memory imagespreserve some of the geometricinformation in perceptual representations,it should be possiblefor the imager to recognizethe pres- enceof an object categoryin an imagethat wasnot originally assignedwhen the object wasfirst seen.In the most dramaticcase, an imagershould be able to observean ambiguousfigure, suchas a Neckercube or a duck/rabbit, see it as one object (e.g., a duck), form a visual imageof it when it is no longer present,and then be able to seeit as the other object (e.g., a rabbit). On the other hand, if memory imagesare records of the conceptualcategory or in- terpretationassigned to the stimulus when it was viewed, and information about its geometricproperties is lost or not readily accessibleto interpreta- tive processes,then a reassignmentof the categoryof an object should be impossible;the imager should be stuck with whateverinterpretation he or sheassigned to the stimulus when it was visible. Severalexperimental investigations have cast doubt on people’sability to recategorizeimages of ambiguousfigures. An experimentreported by Reed (1974)explored whether subjects could detect“hidden” figuresin imagesof patternsthat were composedof combinationsof geometricforms. For ex- ample, oneof the patternswas formed by superimposingtwo equilateraltri- angles,one pointing up, and the other pointing down, positionedsuch that the vertexof one was centeredon the baseof the other. After a brief reten- tion interval, the subjectswere shown a secondpattern, and their task was to say whether or not that pattern was a part of the first pattern. Reed found that subjectscould easilydetect only thoseparts that would enterinto a structural descriptionof the pattern, such as one of the equilateraltri- angles,but not a part that cuts acrossthe elementsof such a description, suchas a parallelogram.Reed and Johnson(1975) later found that the parts not fitting into the original composition of a complex pattern could be detectedmuch more easilywhen subjectscould inspectthe original patterns at the time of testing, than when they had to rely on a memory image. Be- causesubjects in theseexperiments could rarely detectthe hidden parts in their images,these results suggestedthat images,unlike visually perceived forms, cannot be reinterpretedor reorganized.Rather, what is detectedin an imagemay dependentirely on how the imaginedpattern was initially con- ceived(see also the relevant work of Hinton, 1979,and Stevens8r Coupe, 1978). Thesefindings conflict with the observationsof other imagery theorists who haveclaimed that the ability to “see” new patternsin an image is one of the prime functions of imagery, for example,in scientific and artistic creativity (Shepard,1978). More importantly, thereare demonstrations that peoplecan detect new patternsin transformed images.Pinker and Finke (1980)reported a seriesof experimentsin which subjectswere able to “see” shapesthat emergedin the projection of a three-dimensionalconfiguration REINTERPRETING IMAGES 53 of objects after it was mentally rotated. Shepardand Feng (reportedin Shepard& Cooper, 1982)demonstrated that subjectscould quickly name the letter resulting from a transformation (rotation, reflection, or some combination of the two) of a starting letter. For example,when given the transformation “rotate 90 degrees”and the starting letter “N,” subjects could reconstruethe resultingimage as a “Z.” In an experimentsimilar to thoseof Reed, Slee(1980, Experiment 3) found that subjectswere able to judge, with successrates greaterthan chance,whether various geometric forms were presentas embeddedfigures in patternsthey had imagined. In anotherexperiment, Slee demonstrated that subjectscould constructa men- tal image from separatelyviewed piecesand then detect emergentforms resulting from a reorganizationof the imagined piecesaccording to the Gestaltlaws of proximity and common fate. Hollins (1985)had a group of subjectsimagine a grid and mentally fill in certain squaresspecified in terms of their Cartesiancoordinates. On different trials, the experimenterdictated patternsof filled-in squaresresembling a dog, a pitcher, a wall plug, a car, and a telephone.Subjects were able to saywhat the resultingimage depicted on about half of the trials. Relatedindirect evidencecomes from experimentson visual synthesisof parts. Palmer (1977)had subjectsmentally synthesizepatterns by mentally superimposingtwo visually presentedparts consistingof connectedline seg- ments.They then had to match the synthesizedpattern against visual probes. The task was easiestwhen the subpatternscorresponded to perceptually “good” geometricfigures such as triangles and boxes,as opposedto open or disconnectedcollections of line segments.However, subjectsreported that evenwhen the original subpatternswere not “good,” they “looked” for emergent“good” figuresin the synthesizedwhole, with greateror lesser successon different trials. Apparently, at least some subjectswere quite successfulwith this strategy:Their matching times were uniformly fast for shapessynthesized out of good, moderatelygood, and bad parts.Thompson and Klatzky (1978)obtained this effect more uniformly by having subjects mentallysuperimpose sets of visually presentedangles and linesthat together definedunified geometricshapes such as a parallelogram.They found that subjectsreally did treat the result as an emergentsingle form: When match- ing thesepatterns against probe stimuli, they wereno slowerwhen they had synthesizedthe pattern by superimposingtwo or threeparts than whenthey had actually seenthe pattern in its entirety. However, a paper has appearedrecently whose authors try to make a strongcase that the reconstrualof mental imagesis impossible.Chambers and Reisberg(1985) conducted a set of four experimentsaimed at assessing whetherpeople can reinterpretan ambiguousfigure stored in a mentalimage. In their experiments,subjects inspectedambiguous forms, such as the “duck/rabbit” figure commonly used to demonstratemultistability in visual perception(e.g., Attneave, 1971),and werethen instructedto form 54 FINKE, PINKER, AND FARAH mental images of the forms and to try to see the reversals in their images. Al- though the subjects were previously trained in detecting such reversals using other types of reversible figures, they never once reported the correct rever- sal in their imagery. This negative finding persisted even when the subjects were screened for high imagery vividness. In addition, the subjects were able to reverse the figures when they later drew the figures from memory and inspected their drawings. Chambers and Reisberg concluded that men- tal images are therefore not subject to reconstrual, in contrast to visually perceived forms, because images do not contain uninterpreted information; the implication is that images are nothing but