The Interpretation of 10 Visual Illusions by Donald D. Hoffman December 1983 The visual system apparently organizes ambiguous retinal images according to rules of inference that exploit certain regularities in the external world ision is a process of inference. however, the organization changes: now One reason the visual system orga- What you see when you look each colored contour, instead of lying in nizes and interprets retinal images is V around depends not only on what a trough between two rings, appears to simply that many possible configura- is there to be seen but also on how your trace the crest of a ring. (Try it.) Evi- tions in the real world are consistent visual system organizes and interprets dently the visual system does more than with any given retinal image. In other the images that fall on your retinas. An passively transmit signals to the brain. It words, retinal images need organization intriguing demonstration of this aspect . actively takes part in organizing and in- and interpretation because they are fun- of perception is presented by the appar- terpreting them. damentally ambiguous. Their ambigui- ent surface that is formed by rotating This finding raises three questions. ty is due in part to the fact that the world a cosine wave around a vertical axis First, why does the visual system need is three-dimensional and each retina viewed obliquely [see illustration on this to organize and interpret the images is essentially two-dimensional. To de- page]. When you first look at the figure, formed on the retinas? Second, how scribe the world in its full three-dimen- it appears to be organized into a set of does it remain true to the real world in sional glory necessarily involves some raised concentric rings, with the bound- the process? Third, what rules of infer- rather sophisticated inferences by the aries between the rings delineated ap- ence does it follow? The answers to visual system, inferences that for the proximately by the colored circular con- these questions call for a closer exami- most part proceed without any con- tours. If you turn the page upside down, nation of such figures. scious awareness. For example, the co- sine surface at the right, like your retinal image of it, is two-dimensional. Yet it appears, quite compellingly, as three- dimensional. The appearance of depth is entirely inferred, or, to put it anoth- er way, hallucinated. This conclusion should be cause for some concern. If, as I suggest, such hallucinations are not an exception but the rule, and if they are in fact a necessary concomitant of visual perception, how can one justify one's faith in perception? How is it still possi- ble that in general seeing is believing? hat is needed for an understanding W of vision, therefore, is an explana- tion of why such visual inferences usu- ally bear a nonarbitrary relation to the real world. A promising line of investi- gation begins with the observation that the visible world, far from being com- pletely chaotic, obeys certain laws and exhibits numerous regularities. If the visual system is adapted to exploit these laws and regularities in its organization and interpretation of retinal images, and if it is constrained somehow to prefer the interpretation that is most credible, given both the image and a knowledge of these laws and regularities, then it might be possible to understand how it is that one's visual hallucinations bear AMBIGUOUS SURFACE is made by rotating a cosine wave about a vertical axis. The surface a nonarbitrary and even useful relation initially appears to be organized into raised concentric rings, with the colored circular contours to the external world. lying in the troughs between the rings. When the page is turned upside down, however, the A particularly clear example of this organization appears to change: each colored contour is now seen to trace the crest of a ring. approach is the research into visual mo- tion perception done by Shimon Ullman tem exploits the laws of projection, and nar Johansson; SCIENTIFIC AMERICAN, of the Massachusetts Institute of Tech- if it exploits the fact that the world con- June, 1975]. Johansson put small light nology. Ullman has explored the re- tains rigid objects, then in principle a bulbs on the major joints of a person markable ability of the human visual uniq ue and correct interpretation can and took motion pictures as the person system to perceive the correct three-di- be obtained. In particular he showed moved about in a dark room. A single mensional structure and motion of an that three views of four noncoplanar frame of such a film looks like a random object solely from its moving two-di- light bulbs are enough to solve the prob- collection of white dots on a black back- mensional projection, an ability Hans lem. The key point is that an inference ground. When the film is set in motion, Wallach and Donald N. O'Connell of rule, based on a law (the law of projec- however, one immediately sees the cor- Swarthmore College call the kinetic- tion) and a regularity (namely the fact rect three-dimensional structure of the depth effect. For instance, if a trans- that the world includes rigid objects), dots and recognizes that there is an in- parent beach ball with tiny light bulbs enables the visual system to make a cor- visible person walking about [see lower mounted randomly on its surface is set rect interpretation. illustration below]. spinning in a dark room, one immedi- At this stage, however, a puzzle arises. When my colleague Bruce E. Flinch- ately perceives the correct spherical lay- The same mathematical precision that baugh, who is now at Bell Laboratories, out of the lights [see upper iliustration shows the rigidity regularity is sufficient and I considered this problem. what below]. When the spinning stops, so in principle to interpret the rotating puzzled us was that it is possible to see does the perception of the spherical ar- beach ball also shows the rigidity regu- the correct three-dimensional structure ray. How does one see the correct three- larity by itself is insufficient to interpret even though, according to Ullman's re- dimensional structure when infinitely a similar display. This display was first sults, one lacks the appropriate informa- many three-dimensional structures are devised by Gunnar Johansson of the tion to do so. To infer a correct three-di- consistent with the moving two-dimen- University of Uppsala as an example mensional structure on the basis of the sional retinal projection? Ullman showed of what he calls biological motion [see rigidity regularity it is necessary to have mathematically that if the visual sys- "Visual Motion Perception," by Gun- three snapshots of at least four nonco- ROT ATlNG SPHERE is seen when the three dot patterns represent- be adopting the most rigid three-dimensional interpretation for the ed here are shown in rapid succession. The visual system seems to moving dots that is consistent with the two-dimensional projections. WALKING PERSON is seen when these dot patterns are shown in based on an experiment conducted by Gunnar Johansson of the U ni- rapid succession. In this case the visual system seems to adopt the versity of Uppsala in which small light bulbs were put on a person's most rigid and planar three-dimensional interpretation that is con- major joints (shoulder, elbow, wrist, hip, knee and ankle) and a mo- sistent with the two-dimensional motions of the dots. The display is tion picture was made as the person moved about in a dark room. planar points in a rigid configuration. In biological motion displays, on the other hand, at best only pairs of points are rigidly connected, such as the ankle and the knee or the knee and the hip. Rigid quadruplets of points just do not exist. he rigidity regularity, then, is insuf- T ficient by itself, leading us to ask: What further regularity might the visual system be exploiting? After several false starts it occurred to us there is an ana- tomical regularity that might do the trick. Each weight-bearing limb of most animals is constrained, because of the TRANSVERSALITY, a kind of regularity commonly observed in the external world, under- construction of its joints, to swing in a lies a unified account of several visual illusions. According to the rule of transversality (as de- single plane in a normal gait. We call fined by Whitman A. Richards and the author), when any two surfaces penetrate each other at this the planarity regularity. random, they always meet at a concave discontinuity, indicated here by the colored contour. In fact, the planarity regularity is suf- ficient to correctly interpret biological- motion displays of gait. The correct three-dimensional structure can be in- ferred either from three snapshots of two points swinging rigidly in a plane or from two snapshots of three points (such as an ankle, a knee and a hip) forming rigid pairs and swinging in one plane. These results comport nicely with Jo- hansson's observation that only two or three frames of his films need be seen before subjects correctly perceive the biological motion. In addition it turns out not only that all three-dimensional motions governed by the planarity regu- larity can be given a correct interpreta- tion but also that whenever an interpre- tation is found for image motion based on the planarity regularity or the rigidity PARTITIONING RULE based on the transversality regularity is demonstrated with the aid regularity the interpretation is correct. of these two figures that reverse when they are looked at steadily.