Algorithms for Single-Image Random-Dot Stereograms

Displaying 3D Images: Algorithms for Single-Image Random-Dot Stereograms

Harold W. Thimbleby, University of Stirling Stuart Inglis and Ian H. Witten, University of Waikato

e perceive depth in various ways: perspective, shadows, color and in- tensity effects, hazing, and changes in size. Artists know these effects well W and can manipulate them in pictures to give an impression of depth - indeed, pictures of solid objects drawn without perspective look odd. In three dimensions, additional methods are available for perceiving depth. Because our eyes are separated, each has a slightly different view of a distant object. and the angular difference varies with distance. When both eyes look at an object, they turn toward it (convergence) and the retinas receive slightly different images (stereo disparity). Also, the eyes focus differently for objects at different distances. The fusing of the two eyes’ images to produce the effect of depth is called stereopsis. Stereopsis requires the cooperation of both eyes to construct a single mental picture. This article discusses how to use stereopsis to re-create the appearance of depth from a purely flat picture. To understand how this is done, we need only consider the geometry of stereo vision, not its physiological or psychological basis. Stereopsis occurs naturally when viewing solid objects with both eyes. It is possible with two-dimensional pictures only when each eye receives a separate image, A new, simple, and corresponding to the view it would have of the actual 3D object depicted. There are symmetric algorithm various ways to produce stereo images; special equipment is normally required. either for making the image or for viewing it. In a stereoscope,‘.?an optical instrument can be implemented similar to binoculars, each eye views a different picture and thereby receives the spe- cific image that would have arisen naturally. An early suggestion for a color stereo that results nn higher computer display involved a rotating filter wheel held in front of the eyes.3 levels of detail in solid In contrast. this article describes a method for viewing on paper or on an ordinary computer screen without special equipment, although it is limited to the display of 3D objects than previously monochromatic objects. (The image can be colored, say, for artistic reasons, but the possible with method we describe does not allow colors to be allocated in a way that corresponds to an arbitrary coloring of the solid object depicted.) The image can easily be con- autostereograms. structed by computer from any 3D scene or solid object description.

Authorized licensed use limited to: SIMON FRASER UNIVERSITY. Downloaded on January 26, 2009 at 14:22 from IEEE Xplore. Restrictions apply. Solid Background and overview

Julesz and Miller4 were the first to show clearly that a sense of depth could arise purely from stereopsis, without re- lying on other cues such as perspective or contours. They used random patterns of dots that were meaningless when viewed by a single eye but nevertheless created an impression of depth when Figure 1. A solid viewed in a stereoscope. object can be Normal It might seem that stereopsis necessar- simulated on a / :: - - -convergence on ily requires two separate pictures, or at 2D image plane to happlane least some method of splitting a single for stereoscopic Lines of -convergence on picture to give each eye a separate view viewing. to SQlUplane (using, for instance, red/green filters, po- larized light, or interference, as in holo- grams). Recently, however, Tyler and Clarke’ realized that a pair of random- of the visual processes involved in inter- three black images on the plane, two of dot stereograms can be combined, re- preting random-dot stereograms. Gulick the images coinciding.Notice that the dis- sulting in a “single-image random-dot and Lawsony offer an excellent general tance between each pair of dots is differ- stereogram” or, more generally, an au- survey of the psychologicalprocesses and ent: The farther the corresponding point tostereogram. Essentially, this is an over- history of stereopsis. Such background is on the object is behind the image plane, lay of the two separate random-dot pat- useful, though it does not bear directly the farther apart are its two image points. terns, with the dots carefully placed so on the technique described in this article. The central dot shown on the image that each serves simultaneously for two plane in the figure represents two sepa- parts of the image. All that is necessary is rate locations on the object. Therefore, to suitably constrain the dot pattern. Or- these two locations must have the same dinary stereograms, such as photographs Stereo vision and color. In turn, then, the other two dots or wireframe models, cannot be com- autostereograms shown in the image plane must be the bined into a single image, since the result same color. Overall, of course, some dots is merely a double picture. (An effect Figure 1 shows an image plane placed must be of different colors, or else the im- called the “wallpaper illusion” occurs between the eyes and a solid object. Imag- age plane would appear uniform and not when lines of horizontally repeating pat- ine that it is a sheet of glass. (In fact, it present any useful information about the terns are perceived to lie at different could be a computer screen or a piece of object lying behind it. Such considera- depths; however, since the patterns re- paper.) Light rays coming from the ob- tions constrain the surface coloring of the peat monotonously in wallpaper, they ject pass through the image plane and en- object. It is sufficient to use only two col- convey no useful information.) ter each eye. As far as the eyes are con- ors (for example, black and white), but It turns out that very convincing images cerned, two rays pass through each point gray levels and a full spread of colors can with a vivid effect of depth can be con- in the image plane, one for each eye. If be used too. There is considerable flexi- structed in this way, and the advantage of both rays are the same color and inten- bility in choosing the palette, and colors this ingenious approach is that no special sity, they can be conveniently reproduced may be chosen for artistic impact. viewing equipment is required. It takes a by a single light source in the image plane. Figure 1also illustrates the task of view- little practice to see depth in the pictures, Hence, although the object can be seen ing autostereograms. of seeing depth in but once experienced it is very satisfying. stereoscopically, there need be only one the initially meaningless arrangement of Tyler and Clarke5 described a simple image in the image plane, not two, and it random dots. Suppose the image plane is but asymmetric algorithm, which meant can be shared by both eyes. This solves transferred to a sheet of opaque paper. If that some people could see the intended the problem of seeing a stereoscopic pic- the eyes converge ioview the paper in the effect only when the picture was held up- ture without any special equipment. normal way, they dre not converged in a side down. This article presents a new, The problem of generating the au- way that allows reconstruction of the solid simple, and symmetric algorithm for gen- tostereogram amounts to illuminating the image. The same effect occurs when you erating single-image stereograms from screen in such a way that it simulates a look at a mark on a window: Objects be- any solid model. It also avoids several mi- pattern of light that could have come hind the window appear double. In the nor problems with the original algorithm. from a solid object lying behind the case of random-dot stereograms, seeing Further background on stereograms and screen. In general, each point of the ob- the solid image “double” is tantamount to the development of our work is available ject will map into two points on the image not seeing it at all. To view it correctly, in Thimbleby and Neeshan6 plane. Now, if two locations on the solid one must deliberately deconverge the The literature on the psychology of object are chosen carefully, as shown in eyes, as explained in the sidebar “View- stereopsis is vast. For example, Marr and Figure 1, and both are black dots, then it ing a single-image random-dot stereo- Poggi~~.~discuss computational models can be arranged that they generate just gram.”

October 1994 39

Authorized licensed use limited to: SIMON FRASER UNIVERSITY. Downloaded on January 26, 2009 at 14:22 from IEEE Xplore. Restrictions apply. An algorithm for Viewing a single-image random-dot stereogram generating single- Figure A1 is a random-dot stereogram of a large annulus, exactly the same size image random-dot and texture as that of Figure A2, floating above the middle of the picture. The stereograms challenge of successful viewing is that the viewer must decouple the eyes’ convergence from their focusing. Since this is never done in normal circumstances (except perhaps when tired), it may be difficult to achieve a successful 3D effect In Figure 1 we can see that constraints at first. The trick is to converge the eyes at a distance well beyond the paper on affect only points along a line that lies in which the dots lie, yet focus them on the dots themselves. Working in bright light the same plane as the two eyes. This pro- can help, as contraction of the iris causes the eyes’ depth of field to increase, and vides a clue to making the algorithm effi- as a result they rely less on focusing. cient: It can construct the image line by Altering the eyes’ convergence is not easy, but the two circular marks near the line. The inevitable disadvantage is that bottom of the picture can help. By blinking (or covering) the eyes alternately, you there are no constrainis at any other an- can make the marks produce four images, two with the right eye and two with the gle, and therefore the stereo effect is left. Now deconverge your vision so that all four images can be seen with both eyes achievable only when the picture is up- open. The aim is to move the two pairs of images closer together until the inner two right (or upside down). Tyler and Clarke’ coincide and only three distinct marks are seen. The center mark is binocular, while briefly discuss the possibility of having the outer two are seen monocularly (with only one eye each). Fixate on the center orthogonal lines of constraints, but we mark to stabilize the image, then carefully allow your gaze to wander around the will not pursue it here. whole picture. Placing the convergence marks in the center of the picture instead of Our algorithm is based on the geome- near the bottom can make for easier viewing but usually spoils the picture. try shown in Figure 2. The object to be When viewing the autostereogram, you should strive to converge your eyes on portrayed lies between two planes called the far plane. In Figure A1 this is the same distance beyond the paper as the the near and far planes. The latter is cho- paper is from the eyes. Copying the stereogram onto a transparency greatly aids sen to be the same distance D behidd the viewing. Mark an X on a plain sheet of paper and hold the transparency about image plane as the eyes are in front of it. halfway between your eyes and the paper. View the paper through the trans- This is a convenient value because: when parency and converge your eyes on the X. Carefully allow your gaze to wander viewing the autostereogram the eyes across the whole picture. Once you have some experience, you can dispense should converge on the far plane, and you with the transparency and work directly from the printed image. may be able to catch your reflection in Do not be content with an inferior 3D effect. You may become dimly aware of the screen and use this, to assist the con- some circularity in the dot pattern, along with some tenuous 3D effects, but this is vergence process. Sincle at first you don‘t not the full illusion. Viewed properly, the annulus should leap out of the plane at know what you are looking for, it helps if you, and the effect should be stable and persist through sustained and detailed the eyes can initially converge on a large examination of each part of the image, both foreground and background. Stereop- equidistant target. The near plane is a dis- sis has hysteresis: Once you have begun to fuse an image, it becomes clearer as tance pD in front of tbe far plane, and in you look. The farther your eyes are from the paper, the greater the depth effect. If the program p is chosen to be 11.3. The you see two interlocking rings instead of a single ring, your eyes have converged separation between the near and far incorrectly, at twice the distance apart that they should be. (This and other arti- planes determines the (depthof field (not facts are described in the article.) Reexamine the convergence dots and strive to the depth of focus, for all dots actually lie make the four images into three. If you see the annulus behind the plane rather on the image plane and both eyes should than in front of it, you are viewing the stereogram with your eyes crossed, that is, focus there). Increasing the depth of field converging on a point in frontof the paper. When properly diverged, the eyes by increasing p bring,s the near plane converge on a point behindthe paper. (The latter is sometimes referred to as closer to the screen and causes greater “wall-eyed” or “boss-eyed.”) Some people can cross their eyes or diverge them at difficulty in attaining proper convergence will; others can only diverge them. of the eyes. Even if you can capture the 3D picture without using the convergence marks, it It is convenient to define the “image is worth learning to use them properly. If you wear spectacles to correct for short- stereo separation” of a point on the sur- sightedness, it is advantageous to remove them: Your natural focus will be closer face of the solid object viewed in 31) to be than the patterns appear to be when the eyes are correctly converged. If your the distance between its image points ly- spectacles also correct for other sight defects, such as astigmatism, removing ing in the image plane..This quantity re- them may not help. lates directly to the conventional mea- Allow plenty of time: It is often necessary to work for some minutes to achieve sure of stereo disparity, which is the the full 3D effect for the first time. Viewing becomes much easier with practice. difference in angle subtended at the eyes. If viewing difficulty persists, photocopy two identical transparencies from Figure Since the background is taken to be the A1 , line them up carefully, and slide one horizontally over the other until the annu- same distance behind the screen as the lus appears. Although the picture is not in stereo, this will at least serve to con- screen is from the eyes, the separation for vince your eyes that there is an annulus hidden in the dots. Two identical trans- the far plane is half the distance between parencies are recommended, rather than one and the original, since the the eyes. photocopier will tend to stretch the transparencies and the annulus may only be Figure 2 shows the stereo separations visible if both images are distorted similarly. for a point with a specified z-coordinate. The z-values range from 0 (which corre-

40 COMPUTER

Authorized licensed use limited to: SIMON FRASER UNIVERSITY. Downloaded on January 26, 2009 at 14:22 from IEEE Xplore. Restrictions apply. 1

Figure! A. (1) Stereo- gram showing an annulus tloatlng above a surface; (2) the annulus by itself.

October 1994 41

Authorized licensed use limited to: SIMON FRASER UNIVERSITY. Downloaded on January 26, 2009 at 14:22 from IEEE Xplore. Restrictions apply. D D

IParai’-9 error

Far plane

Figure 2. The geometry forming the basis for the stereogram-generationalgorithm.

sponds to the far plane) to 1 (which cor- same[]array. In general, each pixel may right, should we set same[left]= right, responds to the near plane). Thus, the be constrained to be the same color same[right]= left. or both? Actually, it is point is a distance pzD in front of the far (black or white) as several others. How- unnecessary to set both. When the time plane. or (1 -pz)D from the image plane. ever, it can be arranged that each element comes to draw the pixels, the line will be By similar triangles, of the array need only specify a single scanned either left to right or right to left, constraint on each pixel. The same[]array and in the latter case it is only necessary is initialized by settingsame[x] =x for ev- to ensure that same[left]= right. It doesn’t ery pixel. representing the fact that in the matter which of the two directions is absence of any depth information. each used. We choose right to left (line 53), set pixel is necessarily constrained to be the same[left] = right, and require, as an in- which gives the stereo separation 7 as a same as itself (line 24). variant, that same[x]< y whenever x < y. function of 2. This is the fundamental re- Then the picture is scanned, giving, at Now.sarne[left] may have already been lationship on which the algorithm is built. point x, a separation s between a pair of set in the course of processing a previous equal pixels that corresponds to the im- constraint. Recall that .same[x] records a Implementation of the algorithm. The age stereo separation at that point. Call- pixel to the right of x that is constrained to algorithm. programmed in C. is shown in ing these two pixels left and right. then be the same color as x. If same[lefr]is al- Figure 3. Lines 1-1 1 set the scene. The left is at .r - .si2 and right is at x + sI2; but ready constrained, that constraint is fol- screen is rrzaxX by maxY pixels. and the just in cast‘s is odd, we can more accu- lowed rightward (lines 42-48. using vari- object’s ,--value is Z[x11.].The depth of rately position right at lefi + s (lines 28- able k to follow the same links) to find a field ,u is chosen to be 13.Neither the eye 29). To avoid a systematic bias in round- pixel that is not otherwise constrained. In separation nor the screen resolution is crit- ing off when s is odd, line 28 adds I to .s on following the constraints, the variable k ical: approximate values will do in both odd-numbered lines (we might instead may “jump” over righr. If this happens cases. The image stereo separation corre- have made the offset random rather than (line 43). lines 46-47 preserve the assump- sponding to the far plane is called,fcrr. in alternate directions on alternate lines. tion that lefi < right by swapping them. The program processes one scan line as suggested here). The pair of pixels. left Once the constraints have been set for at a time (using the large loop in lines 16- and righr. must be constrained to have all pixels in a scan line, the line is res- 58 of Figure 3). The key to solving the the same color. This is accomplished by canned in tiecreasingx order. that is, from constraint equations is to record what the recording the fact in the same[] array. right to left as mentioned above, allocat- constraints are in an initial pass (lincs 26- (However. there may be geometric rea- ing pixel values (lines 53-57). When a 52). A second pass alllocates a random sons why the corresponding point on the pixel is unconstrained (same[x] = x). a pixel value (black or white) whenever solid objecl is not visible along both lines value is chosen for it randomly and stored there is ;I free choice (line 51): when there of sight: this is checked in lines 35-39. and in pix[x].Otherwise, its value must be is not. the pixel is set to the value that we discuss it in detail below.) constrained to be the same as some pixel obeys the relevant constraint (line 55). To ensure that the pixel at left is farther to the right, and sopixjs]is set to Pixel constraints are specified by the recorded as being the same as the pixel at pix[samr[.r]].The routine Set_Pixel()

42 COMPUTER

Authorized licensed use limited to: SIMON FRASER UNIVERSITY. Downloaded on January 26, 2009 at 14:22 from IEEE Xplore. Restrictions apply. 1. /* Algorithm for drawing an autostereogram */ 2. 3. #define round(X) (int)((X)+0.5)/* Often need to round rather than truncate */ 4. #define DPI 72 /* Output device has 72 pixels per inch */ 5. #define E round(2.5*DPI) /* Eye separation is assumed to be 2.5 inches */ 6. #define mu (1/3.0) /* Depth of field (fraction of viewing distance) */ 7. #define separation(2) round((l-rrm*Z)*E/(2-m~*2)) 8. corresponding to position Z */ 9. #define far separation(0) ing to far plane, Z=O */ 10. #define maxX 256 /* Image and QbjQct are both maxX by maxY gixels */ 11. #define maxY 256 12. 13. void DrawAutoStereogram(f1oat Z[l[l) 14. { /* Object’s depth is Z[xl [yl (between 0 and 1) */ 15. int x, y; /* Coordinates of the current point */ 16. for( y = 0; y c maxY; y++) /* Convert each scan line independently */ 17. ( int pix[maxX]; /* Color of this pixel */ 18. int same[-] ; /* Points to a pixel to the right . . . */ 19. /* ... that is constrained to be this color */ 20. int s; /* Stereo separation at this (x,yl pint */ 21. int left, right; /* X-values corresponding to left and right eyes */ 22. 23. for( x = 0; x < maxX; x++ ) 24. same[x] = x; /* Each pixel is initially linked with itself */ 25. 26. for( x = 0; x c maxX; x++ 1 27. i s = separation(2 [XI [yl);

28. left = x - (s+(s&y&1))/2; /* Pixels at left and right I */ 29. right = left + s; /* ... must be the same ... */ 30. if( 0 <= left && right < maxX ) /* ... or must they? */ 31. { int visible;/* First, perform hidden-surface removal */ 32. int t = 1; /* We will check the points (x-t,y)and (x+t,y) */ 33. float zt; /* Z-coord of ray at these two points */ 34. 35. do 36. { zt = Z[x] [y] + 2*(2 - mu*Z[xl [yl)*t/(mu*E); 37. visible = Z[x-t] [ylizt && Z[x+t] [ylczt; /* False if obscured */ 38. t++; 39. 1 while( visible && zt < 1 ); /* Done hidden-surface removal .,. */ 40. if( visible ) /* ... so record the fact that pixels at */ 41. { int k; */ 42. 43. if( k < right )/* But first, juggle the pointers ... */ 44. left = k; /* ... until either same[leftl=left */ 45. else /* . . . or same[rightl=left */ 46. { left = right; 47. right = k; 48. ) 49. same[left] = right;/* This is where we actually record it */ 50. 1 51. 1 52. 1 /* 53 (I for( x = maxx-1; x >= 0; x- ) Now set the pixels on this scan line */ 54. { if( same[x] == x ) pix[x] = random()&l; /* Free choice; do it rancomly */ 55. else pix[x] = pix[same[x]J; /* Constrained choice; obey constrain: */ 56. Set-Pixel (x, y, pix [XI ) i 57. 1 58. 1 59. DrawCircle(maxX/2-far/2,maxY*19/20) ; /* Draw convergence dots at far plane, */ 60. DrawCircle(maxX/2+far/2,maxY*19/20); /* near the bottom of the screen */ 61. 1

Figure 3. C-language algorithm for drawing an autostereogram.

Authorized licensed use limited to: SIMON FRASER UNIVERSITY. Downloaded on January 26, 2009 at 14:22 from IEEE Xplore. Restrictions apply. then sets the pixel to black or white on the near to the far plane, say). a surface in to the much larger one with the same the screen as soon as its color is known. the foreground obscures one eye's view apex but is based on half of the line join- (If desired. the number of calls to of a more distant point. Hidden-surface ing the eyes, and so Set-Pixe/() can be halved by first drawing removal is a technical detail, and very few a white line across the scan line and then objects make it visibly worthwhile. Yet t-El2 setting only the black pixels.) the advantage is that for any part of the (z, -dPD (2-PZoP The code is followed by drawing two object that is strictly hidden, there is one circular marks with their centers near the less constraint to process. In turn, this (the relevant distances in the denomina- bottom of the screen. separated by a dis- gives the algorithm greater flexibility in tors are marked in Figure 4). from which tance proportional to far (lines 59-60). allocating pixel colors, which reduces the it follows that This separation places them on the back- problem of artifactual "echoes," as dis- ground of the 3D picture. Some viewers cussed in the next section. The hidden- find it easier to see the stereograms by surface-removal code is in lines 35-39 of converging their eyes on the near plane, Figure 3. and this is facilitated by separating the Figure 4 shows a point on the object This is calculated in line 36 of the algo- marks by separarion( 1 ) instead of far = that would cause two pixels of the image rithm in Figure 3. The crucial inequality .sr,parurion(O).Alternatively. if the marks to be linked were it not for an obscuring zl 2 z,appears in line 37, and at the same obscure part of the object, it may be bet- object that interrupts one eye's view. The time a test IS made to determine whether ter to determine the distance from the crucial inequality for hidden-surface re- the other eye's view is obscured. The depth of the object at that point. to en- moval is that zI t z,, where z1is the z-co- whole operation is repeated for differ- sure that the eyes' separation is correct ordinate of a point on the obscuring ob- ent values oft, from I up to the maxi- for that part of the picture. Initial viewing ject and z. is the z-coordinate of a point mum possible value, hhich occurs when may be slightly easier if the marks are on the ray from the eye to the original z,= 1. placed at the center of i.he screen. though object. The x-coordinate is governed by This algorithm checks for obscuring for aesthetic reasons vue prefer to place the distance f, and if such an interruption surfaces at each point of the image plane, them near the bottom. occurs for any value oft greater than 0 but if a geometric object were being ren- and up to the point where z, = 1. then the dered rather than the Z[x][y]array, a Hidden-surface removal. Sometimes, original point is no longer visible to this more efficient implementation could be because of a transition in the object (from eye. The small shaded triangle is similar devised using surface intersections.

D *

Point on object ...... $- -,J -,J

obscures one - eye's view .. .. +w; (2,- z,)yl: I .. .. Image .. plane .. z, z, I1 1 -Z 0

Figure 4. The geometry for hidden-surface removal, which is performed by the algorithm.

44 COMPUTER

Authorized licensed use limited to: SIMON FRASER UNIVERSITY. Downloaded on January 26, 2009 at 14:22 from IEEE Xplore. Restrictions apply. S

on whithe figure is drawn. Consider the two rays f” a point on the far plane to the eyes, shown in Figure 2. These intersect the screen at iwo pixels; the dktance between these pixels is dledthe stereo separation corresponding to the far plane. If the background is the same distance behind the screen as the screen is from the eyes, then this stereo separation is half the distance between the eyes, say 1.25 inches (half the distance typically separating the eyes). The width of the initial strip should be the stereo separation currespondlng to the far plane. Suppose this is 90 pixels, corresponding to approximately half the distance betwdn the eyes, measured in pixels on a typical display screen with 72 pixels/lnch. To give a depth impression gf the far plane, the first dot would be repeated at position 91,

Geometrical to the plane containing the eyes. Hidden- separation is not a linear function of z. surface decisions are made by projecting Previous schemes for calculating single- limitations backward from each scan line in a direction image random-dot stereograms (see the orthogonal to the viewing plane, and this is sidebar above) interpolate the z-values Although there are some geometrical accurate only when the object subtends a linearly into the space between the near limitations to the approach, in practice small vertical angle at the eyes. Neither the and far planes, creating a small depth dis- none actually detract from the depth horizontal nor the vertical distortion af- tortion. The algorithm in Figure 3 cor- effect. fects the depth geometry, and neither is rects this problem by using the true non- noticeable at normal viewing distances. linear separation function (line 7). Geometrical distortion. As Figure 2 il- For distant viewing, both errors disappear lustrates, the object being viewed will have (the lateral one disappears because points False fusion: echoes. Suppose there are a slight lateral distortion in the x direction. A and A’of Figure 2 coincide). constraints on pixels such that a = b and Suppose a value of x has been reached The amount of available depth resolu- b = c. It follows that a = c. This last con- (Figure 3, line 26) that corresponds with tion is the same as the distance far - near straint, interpreted geometrically, corre- point a on the image plane. To determine + 1, in pixels, where far and near are the sponds to a third point in 3D space, and the stereo separation, the program will stereo separations corresponding to the if pixel b lies between pixels a and c, the look along the line aA to find the appro- far and near planes, that is, separation(0) new point lies farther from the eyes than priate depth, which in this case locates a and separation(1) in the program. This is the points constraining a = b or b = c. point (marked A) on the far plane (line because points on the far plane are sepa- There may be no such point on the ob- 27). However, the stereo separation should rated by far pixels on the screen, points on ject! This phenomenon, which follows in- really be governed by point A’ instead. the near plane are separated by near pix- exorably from the transitivity of equal- There might be something intervening be- els, and intermediate values are interpo- ity, can produce visible artifacts that we tween the eyes and A’, but not between lated. For the parameters in the program, call echoes. the eyes and A. While it would be possible far = 90 and near = 72, giving a total of 19 Most transitive pixel constraints do not to mod@ the program to correct the prob- separate depth planes. The number of cause a visible echo. Iltwo pixels are con- lem, this could only be done for a fixed depth planes increases in proportion with strained to be equal but are separated by viewing position. the output device resolution. more thanfar, the echo is conceptually be- There is an analogous distortion in the The depth planes are not exactly the hind the object -indeed, if the pixels are vertical (y-axis) direction, perpendicular same distance apart because the stereo separated by more than E, the echo is im-

October 1994 45

Authorized licensed use limited to: SIMON FRASER UNIVERSITY. Downloaded on January 26, 2009 at 14:22 from IEEE Xplore. Restrictions apply. Figure 5. Two more views of Figure A, with echoes appearing in the same plane as the annulus: (a) for the algorithm of Figure 3 without hidden-surface removal, (b) for the previous algorithm, described in the sidebar on page 45.

False fusion: artifacts. Even though there may be no geometrical constraint that a = h, it can happen that a and h are given the same color purely by chance. This will result in a visual artifact that may Figure 6. Incor- be distracting to the eye. rect convergence With a computer screen display, artifacts of the eyes can can be effectively eliminated by displaying lead to false a different versions of the picture in rapid fusion effect. succession. This can be done efficiently by storing all the constraints in an array and then repeating lines 57-61. but with differ- possible to see anyway. since it corre- happens by chance, so the resulting image ent random numbers. This means that ar- sponds to a virtual image behind the is mottled, but the visible echoes appear as tifacts appear only intermittently and are viewer’s eyes. However. Cor regular ob- the large connected black regions visible therefore hardly noticed. If the display can jects such as plane figures, echoes may in Figure Sa. If the procedure is repeated be recomputed fast enough, or if several combine in complex but systematic ways with the hidden-surface code in place: the versions are generated in advance and cy- right across the image. IPeriodicities in con- corresponding picture shows just the sin- cled through quickly, a striking “shimmer- straints may result in echoes that appear in gle large annulus without any echoes. Vis- ing” effect is obtained. ‘This makes it much the same depth plane where a legitimate ible echoes are also produced by Tyler and easier to see the depth illusion because the part of the object appears. Such echoes Clarke’s scheme,’ described in the side- eyes cannot latch on to chance patterns or can be very distracting, not to mention bar “Previous method for generating focus on individual dots and the only con- quite misleading when they occur in un- single-image random-dot stereograms”: stant constraints are those that represent known or unfamiliar objects. these echoes appear in Figure Sb. The the 3D image. Fortunately. hidden-surface removal re- asymmetry of their scheme is clearly evi- It is also possible to ameliorate artifacts duces constraints for plane figures to the dent in this figure. by using gray levels. Rather than color the extent that echoes in the same plane as the A visible echo often creates such an im- pixels black and white (half the time any figure cease to be a noticeable problem. plausible distortion that it remains unno- two particular pixels will be the same), we For example. Figure 5a shows the echoes ticed - cerlainly when the object is famil- can give them gray levels (or colors) from that arise from the test pattern of Figure A iar, such as ;I sphere c)r text written in relief. a larger palette, say eight gray levels. (in the sidebar) if the hidden-surface code Nevertheless. once an echo has been no- Then chance artifacts become less likely. is suppressed. Figure Sa was generated as ticed. it may become hard to ignore. Echoes follows. First. a stereogram of Figure A2 often persist. although for some people False fusion: incorrect convergence. was created without hidden-surface re- they come and go of their own accord (like False fusion can also occur simply because moval. Then it was scanned for pairs of depth reversal in the Necker cube illusion). the eyes arc incorrectly converged. In our pixels with the same value (black or white) These effecls probably occur because both experience. false fusion is most commonly that have a separation corresponding to the background plane and the echoes in the caused by having the eyes converged at the near plane. that is, srpauulion( I). foreground are simultaneously present in far too grt:at a distance. thercby seeing Whenever this occurre’rl,the midpoint was the autostereogram. and the brain must the virtual solid object that is imaged by marked black. Of course, it frequently choose which to see. alternate image pixels in the image plane.

46 COMPUTER

Authorized licensed use limited to: SIMON FRASER UNIVERSITY. Downloaded on January 26, 2009 at 14:22 from IEEE Xplore. Restrictions apply. gorithm and produced satisfactory but not ideal results (see sidebar on the next page). Our symmetric algorithm is efficient and in practice as fast as the earlier algorithm. Correct implementation of geometrical constraints, together with the hidden- surface feature, enables higher levels of detail in the solid object to be faithfully recorded in the stereogram. The “shimmering” effect that may occur on a dynamic display renders stereograms much easier to view. These features Figure 7. Use of the correct viewing technique will reveal a hemisphere bulging out of a plane this in give artists greater stereogram. freedom and scope for creative use of this new Figure 6 shows the effect: Instead of see- mately pixelwide ambiguity.) medium and provide the basis for many ing the annulus in Figure A, one sees two Care was taken to avoid the systematic interesting experiments with optical illu- interlocking rings. In fact, if the eyes really bias created by always rounding in the sions. are separated by 2.5 inches, as assumed same direction when the stereo separa- in the program, this effect will be achieved tion is halved (line 28 of Figure 3). How- precisely when the gaze is directed at in- ever, in low-resolution autostereograms, Acknowledgments finity. The risk of perceiving 3D objects it is better to round consistently (either with the eyes set at an incorrect conver- up or down) to preserve straight vertical We would like to acknowledge the contri- gence makes guide dots essential, espe- lines, even though they may be systemat- bution of Kerry Guise, who assisted us with ically misaligned. some of our experiments and wrote the X cially for more complex objects. Windows demonstration program. Low-resolution autostereograms. It is References possible to draw low-resolution au- utostereograms are very satis- tostereograms using letters (say, 0.5 inch fying to view and are becoming 1. C. Wheatstone, “Contributions to the Phys- in height), for example, instead of pixels. A increasingly popular, particu- iology of Vision, Part I: On Some Remark- Some of the fine points of drawing au- larly as recreational posters and in scien- able, and Hitherto LJnobserved, Phenom- tific illustrations. A public-domain X ena of Binocular Vision,” Royal Soc. of tostereograms, discussed above, were London Philosophical Trans., Vol. 128, based on an assumption of high resolu- Windows demonstration program for dis- 1838, pp. 371-394. tion. With a low-resolution picture, the playing and printing autostereograms is pixels themselves become a salient fea- availablefrom ftp.cs.waikato.ac.nz. It in- 2. C. Wheatstone, “Contributions to the Phys- ture, and the geometrical correctness of corporates the basic algorithm of Figure iology of Vision., Part 11: On Some Re- markable, and Hitherto Unobserved, Phe- the picture may have to be compromised 3, along with an interface that provides nomena of Binocular Vision (continued),” to reduce distracting effects. control over all the features discussed in The London, Edinburgh, and Dublin Philo- Hidden-surface removal can cause am- this article. As a final example, and to il- sophical Magazine and J. of Science, Series biguous edges around sudden changes in lustrate an object that requires continu- 4. Vol. 3.1852, pp, 504-523. depth. It is generally best, then, not to use ous change in depth, Figure 7 shows a 3. R.I. Land and I.E. Sutherland, “Real-Time, hidden-surface removal with low-resolu- hemisphere bulging out of a plane. Color, Stereo Computer Displays,” Applied tion autostereograms. (With high resolu- Previous attempts at drawing auto- Optics, Vol. 8, No. 3, Mar. 1969, pp. 721- tion. the eye doesn’t notice the approxi- stereograms relied on an asymmetric al- 723.

October 1994 47

Authorized licensed use limited to: SIMON FRASER UNIVERSITY. Downloaded on January 26, 2009 at 14:22 from IEEE Xplore. Restrictions apply. The advantage of symmetric eye convergence

For a viewer to perceive stereoscopic depth across a range (0S z 5 1) have a disparity of less than 0.5 degree. of depths without seeing double, the angular separation of The previous algorithm (described in the sidebar on page corresponding parts of the retinas' images must not be too 45) assumes that one eye looks straight at the image (e= 0). great. Specifically,the angle subtended on a retina - the For the full range of zand the same viewing conditions, this stereo disparity - must be less than about 0.5 degrees. results in a huge disparity of 5.6 degrees. Put another way, Under certain circumstances, larger or smaller disparities can for a disparity of at most 0.5 degree, the effective range of z be maintained. For example, a rod held on the nose and is halved. Although this assumption makes constructing the pointing away from the face subtends too high a disparity to image easy, it distorts the image asymmetrically and reduces be seen as a single rod near the face, but you know it's a the complexity of the images that some people can see single rod, and this impression suffices to sense, if not clearly. Note that a slight rotation of the head has no serious patently see, depth uniformly along its length. impact on these arguments, since the eyes continue to con- The range of angles subtended on a retina depends on verge at the same point (though the effective narrowing of which direction the eye itself is pointing. Suppose one eye is the head reduces all angles). converged inward at a fixed angle eto a perpendicular to the In contrast, the advantages of the symmetric method we image plane. Setting e higher for one eye reduces it for the describe are twofold: The range of stereo disparity required other, given that both eyes converge on the same point in the for viewing is reduced, and parallax distortion is both reduced image plane. Hence, equal, or symmetric, eye convergence and symmetric. Further advantages are hidden-surface permits the largest range of depth change within any given removal and correct depth interpolation, though these refine- limitation of disparity. Take the eye separation to be 2.5 ments could also be applied to the previous algorithm. Al- inches, p = 1/3,the viewing distance to be 15 inches from the though the symmetric convergence approach places multiple eye to the image plane, and consider a symmetric conver- constraints for both eyes on each image pixel, these con- gence angle @=arctan(H4D). Then all depth changes straints are readily solved.

4. B. Iulesz and J.E. Miller, "Automatic 6. H.W. Thimbleby and C. Neesham, "How 8. D. Marr andT. Poggio, "A Computational Stereoscopic Presentation of Functions of to Play Tricks with Dots," ,Ntw .Scientist. Theory of Human Stereo Vision." Proc. Twci Variables," Bell Sy.skm Techniccrl I.. Vol. 140. Oct. 9. Issue No. 1.894. 1993. pp. Royd Toc. of London, Vol. 8203, Royal Vol. 41. Mar. 1Y62. pp. 663-676. 26-20. Society. London. 1970. pp. 304-328.

r and M.EI. C'larke, "'l'lic Aut+ 7. D. Mxi andT', Poggio,"('ooperativc Con- 9. W.L. iulick and K.B. Lawson. HIINIO~~ stereogram." SI'IE Srrwo.vc,opicDisp/cy.s iriitl putatioii 01'Stcrco Disparity." .Scic,izc~r,.Vol. Srercv)pvis: A l'.s~'cli~)l~)~i~~(ilAtiol! vis, Ox- Applicmrior~s.Vol 1,258. 19'90, pp. I IC - 1%. 1W. ()cl. IS, 1976. pp. 283-287. lord University Press, Oxford. UK, 1976.

Harold W. Thimbleby i:s professor ofcomput- Stuart lnglis is a doctoral student at the Uni- Ian H. Witten is professor of computer science ing research at Middlesex University. Eng- versity of Waikato. New Zealand, w8here he at the LJniversity of Waikato, New Zealand. land. Formerly a lecturer iit Queen Mary graduated with a degree in computer science. having been previously affiliated with the Uni- College (now Queen Mary and Westt'icld Col- He was rcccntly a research assistant at the Uni- versity of ('algary. Canada. and Essex Univer- lege) of London Univi2rsity and York Uni- versity ol Calgary, (-'anada. He is interested in sity. England. His current interests include pre- versity. England, he joined Stirling University. all aspects olcomputer graphics and is pursu- diction and modeling, machine learning, and Scotland, in 1988 as profesor of information ing research on compression-based methods programming by cxample. Hc has published technology and moved to Middlesex in 1994. for templatc matching and the compression of widely on machine learning, speech synthesis He has written more than 2OOpublications. in- textual iniaqcs. and signal processing. text compression, hy- cluding three books. and sits on several rdito- pertext. and computer typography. He has au- rial boards. thored or coauthored five books. Thimbleby received B I'hD from London Wittcn received dcgrces in mathematics. University in 1981. He is a chartered engineer computer science. and electrical engineering and a lellow of the Royal Society of Art. In from Cambridge LJniveraity. the University of 1981 he was awarded 1.he British Computer Calgary. and Essex University. respectively. Societ! 's Wilkes Medal. Hc is a member of the IEEE Computer Society.

Thimblcbly can be contacted at the Department of Psychology, University of Stirling, Stirling, Scotland, F K9 JLA, e-mail [email protected] ling.ac.uk. Inglis and Pv'itten are at the Department of ('ompuler Science, University of Waikato. Hamilton. New Zealand. e-mail (singlis. ihw)@'aaikato.ac.nz.

48 COMPUTER

Authorized licensed use limited to: SIMON FRASER UNIVERSITY. Downloaded on January 26, 2009 at 14:22 from IEEE Xplore. Restrictions apply.