Overview Vision & Visuals ‡ Human Visual Systems ‡ Visual Perceptions ‡ 3D Depth Cues ‡ 3D Stereographics Terminology ‡ Stereo Approximation ‡ 3D Displays & Auto-Stereoscopic Displays 029515 ‡ Desiggpyn Issues for VR Visual Displays 2010년 봄학기 3/29/2010 박경신

Human Visual System Human Eyes

‡ Light is focused by the cornea and the lens onto the retina ‡ Light passing through the center of the cornea and thlhe lens hi hihfts the fovea (or Macu l)la) ‡ Iris permits the eye to adapt to varying light levels, contro lling th e amoun t o f lig ht en ter ing the eye. ‡ Retina is optically receptive layer like a film in a camera ‡ Retina translate light into nerve signals. ‡ Retina has photoreceptors (rods & cones) and inter- neurons. Photoreceptors (Rods & Cones) Rods & Cones Distribution

‡ Rods ‡ Rods & Cones Distribution „ Operate at lower illumination levels ‡ Fovea „ Luminance-only „ only cone receptors with „ The most sensitive to light very hihigh h dens itity „ At night where the cones cannot detect the light, the rods provide us with a black and white view of the world „ no rods „ The rods are also more sensitive to the blue end of the „ no S-cones (blue cones) spectrum „ responsible for high visual ‡ Cones acuity „ operate at higher illumination levels ‡ Blind Spot „ provide better spatial resolution and contrast sensitivity „ no receptors „ provide color vision (currently believed there are 3 types of „ Axons o f a ll gangli lion ce lls cones in human eye, one attuned to red, one to green and pass through the blind spot one to blue) [Young-Helmholtz Theory] on the way to the brain „ provide visual acuity

Visual Spectrum Color Perception

‡ We perceive electromagnetic ‡ There are three types of energy having wavelengths cones, referred to as S, M, in the range 400 nm ~ 700 and L. They are roughly nm as visible light. equivalent to blue, green, ‡ ~450nm : violet and red sensors. Their peak 458nm ~ 480nm : blue sensitivities are located at 420 nm, 534 nm, 564 nm. 510nm ~ 550nm : green ‡ Color ppperception results 570nm ~ 590nm : ye llow Spectral Sensitivity of Cones from the simultaneous 590nm ~ 630nm : orange stimulation of the 3 cone 630nm ~ : re d types. ‡ Colorblindness results from a deficiency of one cone type. Chromatic Color Visual Perception

‡ Hue ‡ Visual Acuity „ distinguishes between colors „ Visual acuity is measured as the angle subtended by the human eye. ‡ Saturation „ Snellen fraction 20/X where the viewer sees at 20 feet detail „ how far is the color from a that the average person can see at X feet gray of equal intensity „ The standard definition of normal visual acuity (20/20 vision) „ vivid colors (red, blue) are is the ability to resolve a spatial pattern separated by an angle highly saturated, further from of 1 arc min gray „ A resolution of about 1 arc min is necessary to get to the „ pastltel co lors (ik(pink, s kky bl)blue) region of peak sensitivity are lightly saturated, closer to gray ‡ Brightness CIE (Commission Internationale „ perceived the intensity of a lum inous o bjec t d’Eclairage) Chromaticity Diagram 20/20 CRT 20/40 HMD 20/425 BOOM 20/85 CAVE 20/110

Visual Perception Visual Perception

‡ Temporal Resolution ‡ Color „ The real world doesn’t flicker. CRTs do flicker because the „ Most visual perceptual processes are driven by intensity not image is constantly being refreshed. color. „ We perceive flickering if the image on CRT isn’ t refreshed fast „ Motion system is color blind, depth perception is color blind, enough. object recognition is color blind „ Most people stop perceiving the flicker between 15 Hz (for „ But uniquely colored objects are easy to find dark image) and 50 Hz (for bright images). ‡ Field of view (FOV) „ Some people can perceive flicking even at 60 Hz for a bright „ Each eye has a horizontal fov of 150° (60° towards the nose display with a large field of view. and90d 90° to the s ide ) an d a ver tica l fov o f 120° (50° up and70d 70° „ Very large display may require up to 85 Hz. down). ‡ Luminance „ Most HMDs have a FOV of about 40 to 60. „ The eye has a dynamic range of 7 orders of magnitude. ‡ Motion „ The eye is sensitive to ratios of intensities rather than to „ Motion of the visual field causes a sense of motion even abltdiffbsolute differences. BihtBrightness = Lum inance ^033^0.33. without physical motion. Studies have shown that a low „ To make something appear n times brighter the luminance resolution peripheral image contributes strongly to the effect must be increased by n^3. Visual Depth Cues Visual 3D Depth Cues - Pii“dh”ihPerceiving “depth” with one eye c ldlosed ‡ Linear perspective „ Objec ts ge t sma ller the fur ther away they are an d para lle l line converge in distance. ‡ Size of known objects „ We expect certain object to be smaller than others. ‡ Detail (texture gradient) „ Close objects appear in more detail, distant objects less. ‡ Occlusion (hidden surfaces) „ An object that blocks another is assumed to be in the foreground. ‡ Lighting and Shadows „ Closer objects are brighter, distant ones dimmer. Shadow is a ffliform of occlusion. ‡ Relative motion (motion parallax due to head motion) „ Objjyects further away seem to move more slowl yjy than objects in the foreground. 13

Perspective Size

‡ The observance that parallel lines converge at a single ‡ Compare the size of objects with our memory of similar vanihiishing po int. objec ts to approx ima te how far away the o bjec t is from ‡ Relies on the assumption that the object being viewed is us. construc te d o f para lle l lines, suc h as mos t bu ildings. ‡ Comparing the size of objects with respect to other objects of the same type to determine the relative distance between objects. Detail Occlusion

‡ Our eyes cannot discern as much detail of a texture at a ‡ An object occludes our view of another distance as compare d w it h up c lose. ‡ The strongest depth cue ‡ Atmospheric effects, such as haze and fog, cause more dis tan t o bjec ts to be v isua lly less dis tinc t.

Liggghting and Shadow Motion Parallax

‡ Brighter objects are perceived as being closer. ‡ As an observer moves, nearby objects appear to move rapidly while far objects appear to move slowly. ‡ With one source of light, all shadows lie in same ‡ Come from the parallax created by the changing relative direction. ppjgosition between the head and the object being ‡ The object covered by the shadow is perceived to be observed. further away than the object in the light. ‡ Generally more important than stereoscopy for VR. ‡ A form o f s ha ding tha t in dica te the pos itiona l relationship between two objects. Visual Depth Cues Stereoscopy - UiUsing bo th eyes ‡ Binocular vision occurs when ‡ Binocular cues: binocular disparity (stereopsis) two eyyges look at the same thing „ This is the difference in the images projected onto the back of at a slightly different angle, the eye (and then onto the visual cortex) because the eyes are resulting in two slightly different seppyyarated horizontally by the interocular distance. images. ‡ Oculomotor cues: accommodation & convergence ‡ The brain must match points „ Based on information from muscles in the eye between the two separate „ Accommodation (focus) images seen by the two eyes. left right ‡ This is the muscle tension needed to change the focal length of the ‡ The slight difference between eye lens in order to focus at a particular depth. the v iewpo in ts o f your two eyes „ Convergence is called binocular disparity ‡ This is the muscle tension required to rotate each eye so that it is fifacing t hfhe foca l po int. ‡ Stereopsis is depth perception „ Accommodation and Convergence work together (when eyes due to binocular disparity converggye to a certain distance, automatically accommodates and ‡ Possiblyypp 12% of people have no vice versa) stereo vision or some problem with stereo vision.

Accommodation (()focus) Convergence Some Terminology Homologous Points

‡ Horizontal Parallax (Binocular Disparity) „ When the re tina l images o f an o bjec t fa ll on dispara te po in ts on the two retinas, these points differ only in their horizontal position. „ Thlhe value gi ibven by R – L ‡ Stereo Window (Stereo Plane) „ The point at which there is no difference in parallax between the two eye views „ Usually at the same depth as the monitor screen or the projection surface. Scan line ‡ Homologous Points „ Points which correspppyond to each other in the separate eye views. ‡ Vertical Displacement „ Vertical parallax between homologous points relative to the line Stereo Plane that the two eyes form.

Some Terminology Some Terminology

‡ Interocular Distance ‡ Positive Parallax „ The distance between the left and right eyes, usually about „ The point lies behind the stereo window (On the opposite side 2.5 inches (i.e., 6.5 cm) from the observer) ‡ Hypostereo/Giantism ‡ Negative Parallax „ Decreasing the distance between the left and right eyes to „ The point lies in front of the stereo window (On the same side as show stereoscopic detail on small items the observer) ‡ Hyperstereo/Lilliputism ‡ Zero Parallax „ Increasing the distance between the left and right eyes to „ The point is at the same depth as the stereo window (Both eyes shidililhow stereoscopic detail in large scenes see thi)he same image) ‡ Interocular Crosstalk (Ghosting) „ Each eye should only see it’ s view but sometimes it can see part of the other eye view as well. This is distracting and causes eye fatigue. Positive Parallax Negative Parallax AdtiAccommodation-convergence mithismatch AdtiAccommodation-convergence mithismatch

R L

R L

‡ Iffff Objects are too close in front of the projection plane, negative parallax will increase. The left and riggyht eye ima gpjges projected on the screen ‡ If negative parallax is wider than eye separation, then result is pain.

Zero Parallax When the o bjec t is ac tua lly on the screen Stereo Approximation

Left Eye Viewing a point in a scene ftfrom two difference camera positions produces differing view planes Right Eye

32 Stereo Approximation Correct Stereo Comppputer Graphics

Left Using parallel Eye views (i. e., The correct symmetric view approach using frustums) produces parallel views a silingle v iew p lane, and asymmetric but images must view frustrum be trimmed to area produces a of overlap - silingle v iewp lane Right Projection Planes and overlapped Eye are not the same image ‡ Some software (e.g. Blitz3D) does not have an easy way (yet) to create asymmetric view frustums. ‡ There is potential for eye discomfort for objects that are too close because an object may appear to be cut off at the e dges for one o f the eyes. Asymmetric View Frustum ‡ Enlarging eye separation makes the problem worse.

Stereo Images on the GeoWall Off-axis Persppjective Projection in a CAVE

‡ Creates 1 wide window 2048x768 and creates 2 viewports In CAVEs, the view frustum will often fall off center. (1 for le ft eye image, 1 for r ig ht eye image ) front wall view ‡ Each viewport goes to 1 of the graphics card’s outputs to a projector .

left wall view right wall view eye Making the virtual world look true to How to Generate Stereo Images size Left Present a distinct image to each eye: Eye ‡ Free-viewing ‡ Optics (lenses)

f d ‡ Chromadepth h ‡ Pulfrich Effect Right ‡ Anaglyph (color) Eye ‡ Polarization ‡ Set camera properties to be the same as real world ‡ Active Shuttering properties: ‡ Autostereo „ Set user ’s distance to screen (i.e. foca l lengt h d) „ Measure the screen’s height (h) „ Compute the field of view (f = 2*atan(h/2d)) „ Use real world eye separation distance (2.5 inches)

Free-viewing Optics

‡ Two slightly different images are displayed next to each ‡ Use lenses and physical separation, present a separate other. image to each eye. ‡ The viewer must focus his or her eyes properly to fuse the titwo images: e ithlllither parallel or cross-eyed.

Stereoscope, invented by Charles Wheatstone in early 1800s Optics Optics

2 computer monitors: one for each eye

Viewmaster Slidemaster

HMD BOOM

Chromadepth Pulfrich Effect

‡ Special filters that cause different colors to appear at ‡ Physiological effect discovered by astronomer, Carl Pulfrich. different dept hs. ‡ Pulfrich effect glass have one dark lens and one clear lens. ‡ Red objects appear close; blue objects appear distant. ‡ Images viewed through a darkened lens reach the brain slower than those viewed through a clear lens. ‡ When something moves across the visual field, the brain fuses titwo images from s lihtldiffttilightly different times. MtiMotion is thus converted into stereo parallax. Anaggyplyph

‡ Colored filtered are used – one eye sees just red elhlements, other eye sees bl()blue (or green or cyan) elements. ‡ The co lore d lenses ma ke one image more v is ible to one of your eyes and less visible to your other eye.

Passive Polarization Passive Polarization

ImmeraDesk4@EVL – Circular Polarization

iDesk GeoWall – Linear Polarization @EVL Linear Polarization Circular Polarization

‡ A single “ray” of light has a particular ‡ Combining a linear polarizer and a polar iza tion direc tion, perpen dicu lar quarter-wave retarder produces to direction of propagation. circular polarization. ‡ Circular polarization can be ‡ Ordinary light usually is polarized clockwise or counter-clockwise. equally in all directions. ‡ Circular polarization is immune to ‡ A polarizing filter allows only light the “hea d-tilt pro blem. ” polarized a certain direction to pass ‡ Works better because light is through circularly polarized. ‡ Less expensive. ‡ Problem: Many project screen ‡ Problem: tilting the viewer’s head materials de-ppgolarize light; mirrors affects filtering and hurts stereo can also de-polarize light, at larger effect. angles of reflection; LCD projects polarize light internally (green one way, red & blue another way)

Active Shutter Glasses Autostereo

‡ Glasses have liquid crystal lenses which ‡ Glass-free stereo can bdkbe darkene d&ld & cleare d rap idly - at „ Image broken into sets of vertical strips. Each set is a different any time one lens is clear and one is eye-view dark. ‡ Autostereoscopic ‡ Glasses are synchronized with video „ Parallax barrier: Barrier strip (PHSCologram, Synthagram, etc) display - one eye sees odd frames, seppyarate layer with stri ps that block all but one ima ge from other eye sees even frames. any viewpoint „ Lenticular: lens like stripes ‡ Reqq(uires fast video refresh rate (> „ LltItlhtLenslet: Integral photograp hith or integram 90Hz) to prevent flicker. ‡ 3D displays ‡ LCD projects are not capable of high „ Hologram frequencies. Affordable DLP projectors „ Volumetric not programmed to support high „ Stereoscopp,,pic: Active stereo, Passive stereo, Autostereoscopic frequencies. Hologram Volumetric Display

‡ MARK-II @ MIT Media Lab ‡ Spinning Screen Display ‡ Slice-Stacking Display

Actuality Systems LightSpace Tech

Autostereo 3D Display Autostereo 3D Display

‡ Integral-Imaging System

Pickup Display

Philips, 42” WOWvx 42-3D6C01 -Lenticular, support for multiusers Pick up device Object Display panel Integrated image Lens array Lens array

Pavonine, 17’’ /19’’ Dimen -Backlight switchable barrier

Sharp, Actius RD3D Notebook -Backlight switchable barrier Parallax Barrier Lenticular

‡ A vertical slit plat placed in front of a specially ‡ Need no special viewing equipment prepared image made of strips of alternating left and ‡ Made from strips of cylindrical lenses right eye views

Lenslet How to Generate GrappQyhics Quickly

‡ Uses spherical lenses instead of cylindrical ones to ‡ Naive Approach present horizontally and vertically varying directional „ poll head & hand sensor information, thus producing a full parallax image „ update virtual world „ dldflf&ihdraw world for left & right eye „ display images ‡ In VR sys tems like the CAVE this is more comp lica te d because the scene must be drawn for the left and riggyht eye on multi ppjple projection places. ‡ The rate that images are drawn will always be based on the update rate. We tend to run at either 96hz or 120hz (48hz or 60hz per eye ) w hic h g ives flic ker free images based on most people's temporal resolution. Updating Visuals based on Head How to Generate GrappQyhics Quickly Tracking

‡ Often when moving through a virtual world, the world ‡ Update the graphics every frame in order to use the will seem to speed up or slow down based on the mostthditit recent head positions. complexity of the scene. ‡ Since there will be jitter in the tracker values and latencies to deal with in magnetic tracking, this may ‡ Models can be replaced by models with less detail. result in the image jittering. ‡ 3D models of far away objects can be replaced by ‡ One wayyypg to avoid this is to only update the image tex ture mappe d billboar ds. when the head has moved (or rotated) a certain ‡ The horizon can be moved in – moving in Z-far and amount perhaps covering this with fog. ‡ Another option is to interpolate between the previous ‡ A less complex lighting model can be used. and current position and rotation in order to smooth out this motion. This results in smoother transitions but will also increase the lag slightly ‡ Another option is to extrapolate into the future

Simulator Sickness Simulator Sickness

‡ A small percentage of people gets sick from immersive ‡ The organ of the inner ear VR. This percentage will increase as latency increases. contiitaining three ‡ Symptoms semicircular ducts at right „ Nausea, blurred vision, difficultyyg concentrating, headache, angles to one another. drowsiness, discomfort, dizziness, fatigue ‡ Responsible for maintaining ‡ Causes the body’s orientation in „ Still unknown but one common hypothesis is mismatch between space, blbalance, an d pos ture; visual motion (what your eyes tell you) and the vestibular system (what your ears tell you) regulates locomotion and oth er m ov em en ts an d k eeps „ HMDs are more lik e lly to cause it than pro jec tion VR sys tems objects in visual focus as „ low resolution, low frame rate, high latency are also like causes the body moves. „ Another hypothesis deals with the lack of a rest frame. When a user views images on a screen with an obvious border that border locates the user in the real world. Without that border the user loses his/her link to the real world and the affects of motion in thlldhe virtual worlds are more pronounce d. Remedies for Simulator Sickness Reference

‡ Close your eyes. ‡ http://www.evl.uic.edu/aej/528/lecture04.html & lecture05.html ‡ Or take a walk around the block if you can do it ‡ Denn is Pro ffitts SIGGRAPH’94 Deve lop ing Advance d VR Applications course notes without falling over. ‡ Lou Harrison SIGGRAPH’97 Stereo Computer Graphics for Virtual ‡ Break the illusion of motion by NOT covering your RliReality notes entire field of view with the screen. (i.e. sit back so ‡ http://www.siggraph.org/education/materials/HyperVis/virtual.env/ that yygyou can see the edges of your monitor) percept.iss/percept.htm ‡ Do not play in a dark room- for the same reason as ‡ http://www.wmin.ac.uk/ITRG/IS/DPI/HIW/Human%20Visual%20Syst above. em.pdf ‡ http://www.3dglassesonline.com/ ‡ Avo id scenes w here you are ro lling a bout t he Z ax is. ‡ http://web.cs.wpi.edu/~matt/courses/cs563/talks/stereohtml/stere ‡ Higher frame rates can actually INCREASE the sense of o.html motion. ‡ http://web.media.mit.edu/~ halazar/autostereo/autostereo.html ‡ http://www.mlab.uiah.fi/nmc/stereo/masters/eng/vocabulary.html ‡ But the jarring effect from lower frame rates can cause ‡ http://www.3dnshop.com/dic/list.php eye strain. ‡ http://local. wasp. uwa. edu. au/~pbourke/projection/caev/