COMS W4172 Perception, Displays, and Devices 3 Steven Feiner Department of Computer Science Columbia University New York, NY 10027 www.cs.columbia.edu/graphics/courses/csw4172 February 16, 2021 1 Stereoscopic Viewing Wheatstone, 1838 Passive . Spatial multiplexing . “Free viewing” http://www.luminous-lint.com/app/image/06751152706639519425853376/ . Stereoscope . Presents each eye with its own view Brewster, 1839 Holmes, 1861 http://www.gilai.com/product_763/Holms-Wood-Stereoscope-with-Green-Velvet-Lining-and-Stereocard. 2 Feiner, COMS W4172, Spring 2021 Stereoscopic Viewing Passive . Spatial multiplexing . “Free viewing” . Stereoscope presents each eye with its own view 3 What’s wrong with these pictures? 4 Feiner, COMS W4172, Spring 2021 Commercial versions (w/o camera support): • www.amazon.com/Hasbro-Viewer-touch- Stereoscopic Viewing iPhone-Black/dp/B004T7VI2Y • www.samsung.com/global/galaxy/gear-vr For commodity • arvr.google.com/cardboard • arvr.google.com/daydream/smartphonevr Passive mobile devices • holokit.io/ (optical see-through) . Spatial multiplexing . VR: Stereoscopic viewer for smartphone/tablet . AR: 1. Cheat: Separate left/right stereoscopic views of virtual objects combined with L–R shifted copies of a monoscopic camera view 2. Full video see-through stereo using additional camera lens (e.g., www.kula3d.com/kula-bebe) Stereo viewer used with Columbia’s Goblin XNA and Nokia Lumia 800 3. Optical see-through stereo using 2012 additional optics (e.g., holokit.io) Google Cardboard, 2014 https://arvr.google.com/cardboard/ Stereo viewer for Nokia Lumia 800 courtesy of USC ICT MxR Lab, 2012 https://web.archive.org/web/20190314093037/http://projects.ict.usc.edu/mxr/diy/fov2go-viewer/ 5 Autostereoscopic Viewing Passive . Spatial multiplexing . Parallax barrier approaches . Parallax stereogram (F. Ives, 1901) . Viewer’s eyes need to be positioned properly for each to see only its parts of the pair http://paulbourke.net/stereographics/lenticular/ 6 Feiner, COMS W4172, Spring 2021 Autostereoscopic Viewing Passive RRRLL L . Spatial multiplexing . Parallax barrier approaches . Parallax panoramagram L R Y B (C. Kanolt, 1915) …Y Presents different stereo G… pairs at different locations . Can use physically moving mechanical or virtually moving liquid crystal barrier to create active display for tracked user . mrl.nyu.edu/~perlin/demos/autostereo.html http://paulbourke.net/stereographics/lenticular/ 7 Autostereoscopic Viewing Passive . Spatial multiplexing . Lenticular approach Note: Refraction (H. Ives, 1930) not modeled properly in figure! . Array of cylindrical lenses . Brighter than barrier Every stereo postcard you’ve seen https://earthx.org/product/turtle-postcard/ http://paulbourke.net/stereographics/lenticular/ 8 Feiner, COMS W4172, Spring 2021 Autostereoscopic Viewing Reconfigurable from mono to stereo . Goal: Display that can switch HTC Evo 3D Nintendo 3DS from Was used in SHARP 3D desktop displays and laptops autostereoscopic 3D to 2D . Solution: Parallax barrier made from switching LC panel that can be turned on or off http://sharp-world.com/products/lcd_monitor/sharp_lcd/wide_variety/index.html 9 Stereoscopic Viewing Passive . Spectral multiplexing . “Red–cyan” (earlier “red-blue”) anaglyph stereo . Display overlapped left/right images in red/cyan . Can create by Displaying only red channel of left image and green/blue channels of right image Converting left (right) image to greyscale and displaying as red (cyan) http://www.stereoscopy.com/3d-images/photography.html http://mars.jpl.nasa.gov/MPF/mpf/anaglyph-arc.html 10 Feiner, COMS W4172, Spring 2021 Stereoscopic Viewing Passive . Spectral multiplexing . ColorCode 3-D . Display overlapped left/right images in amber/blue Left eye provides color (mostly) Right eye provides second eye’s geometry (mostly) . Better color rendition than red/cyan anaglyph . Imagery looks better without eyewear . www.colorcode3d.com . Other color pairs used by other vendors 11 Stereoscopic Viewing Passive . Spectral multiplexing . Infitec . Display overlapped left/right images using different RGB primaries for each eye Special narrow bandpass filters used on projector and in eyewear . Passive eyewear (expensive) . Metallized screen not needed . Very low crosstalk . Slight L–R color shift . www.infitec.net http://www.infitec.net http://infitec.net/wp-content/uploads/2015/04/60.5L_Stereoscopic-laser-display-system-using-six-primary-colors-.pdf 12 Feiner, COMS W4172, Spring 2021 Stereoscopic Viewing Passive . Spectral multiplexing . Infitec Excellence [improved version] . Display overlapped left/right images using different RGB primaries for each eye Special narrow bandpass filters used on projector and in eyewear R, G, B: one band each for Right; two bands each for Left. Long B/short G and long G/short . Passive eyewear R are combined, totalling four bands for Left. (expensive) Each single-band primary is a metamer of its . Metallized screen two-band primary! not needed . Very low crosstalk . Much lower L–R color shift . www.infitec.net 12 3 4 http://www.infitec.net https://infitec.net/wp-content/uploads/2015/04/SIDDIG_2011.pdf 13 Stereoscopic Viewing Passive (single image) . Spectral multiplexing . Chromostereoscopy . Different colors focus at slightly different points on retina (red closer to temple than blue for most people) . Caused by chromatic aberration Perceived fused depth (blue) Left eye seen from above Perceived fused depth (red) Actual position of object (black) and optically shifted positions due to red and blue focusing at different offsets on retina. Optical axis Fovea is off center relative to eye’s optical axis. Index of refraction varies with wavelength (red < blue) Exaggerated! http://users.ph.tum.de/cucke/ftp/lectures/netduis.pdf 14 Feiner, COMS W4172, Spring 2021 Stereoscopic Viewing Passive (single image) . Spectral multiplexing . ChromaDepthTM stereo . Inspired by chromostereoscopy . Refractive/diffractive optics over right eye shift colors differentially to the left (red > blue) Actual position (black) and optically shifted positions Perceived fused depth (blue) Perceived fused depth (red) Optics Effect is more extreme if optics are used for both eyes Exaggerated! www.chromatek.com 15 Stereoscopic Viewing Passive (single moving image) . Intensity multiplexing . Pulfrich effect https://the3dmarket.com/products/pulfrich-3d-glasses . Dark filter over one eye causes perceptual delay . Note: Works with moving images only (e.g., http://michaelbach.de/ot/lum-Pulfrich/index.html http://www.youtube.com/watch?v=1mnWI_u_zBg) . Dark filter on eye at leading edge of horizontal motion brings it forward Perceived depth Left–right motion Right–left motion Actual position (black) and Actual position (black) and perceived position (grey) perceived position (grey) Perceived depth Optics Optics https://youtu.be/1mnWI_u_zBg Use filter over left eye! 16 Feiner, COMS W4172, Spring 2021 Stereoscopic Viewing Passive . Polarization state multiplexing . Linear (horizontal/vertical) http://reald.com . Circular (left- handed/right-handed) http://www.barco.com/VirtualReality/en/stereoscopic/passive.asp Polarized light has the tip of the electric field vector following a particular path 17 Stereoscopic Viewing Passive . Linearly polarized Polarized light has the tip of the electric field vector following a particular path https://en.wikipedia.org/wiki/Polarizer 18 Feiner, COMS W4172, Spring 2021 See the 19th C. machine Wheatstone designed to demonstrate the propagation of polarized light: https://www.youtube.com/watch?v=bw4R5qXaIww Stereoscopic Viewing Passive . Circularly polarized Polarized light has the tip of the electric field vector following a particular path https://en.wikipedia.org/wiki/Polarizer 19 Stereoscopic Viewing Passive . Polarization state multiplexing . Linear (rotated by 90°) . Circular (left- handed/right-handed) Linearly . Robust to head roll polarized . Screen must preserve polarization (metallized) Circularly polarized https://en.wikipedia.org/wiki/Polarizer 21 Feiner, COMS W4172, Spring 2021 Stereo Content Caveats . Geometry . Avoid window violations: Objects at screen edges that extend in front of the screen and are cut off by the edge . Avoid extreme parallax: Difficult to verge and make L–R cross-talk obvious . Avoid rapid/discontinuous changes in depth over time . Color coding for anaglyph . Avoiding objects whose light is sent to only a single eye after filtration (e.g., red or blue objects in simplistic coding approaches) . Some anaglyph approaches (https://www.flickr.com/groups/anachrome3d/) allow slight leakage of red into the cyan filter to support redness cues in both eyes Image with front of landscape protruding from screen, violating window edges (NASA Mars Pathfinder) Image after shifting L and R views horizontally (and cropping out edge portions with only one view) to adjust front of landscape to be behind screen http://en.wikipedia.org/wiki/Anaglyph_3D 22 Stereoscopic Viewing Active . Temporal multiplexing . Liquid crystal shutter eyewear . Right/left lenses alternatively opacify in synchrony with display of CrystalEyes LC shutter eyewear left/right views L R L R … . Synchronized to wireless (e.g., infrared, Bluetooth) emitter https://lennylipton.wordpress.com/2009/02/24/thank-you-puff-the-magic-dragon/ 23 Feiner, COMS W4172, Spring 2021 Stereoscopic Viewing Active . Temporal multiplexing . Liquid crystal shutter eyewear . Can support multiple users at decreased fps L1 L2 R1 R2 … CrystalEyes LC shutter eyewear Two-user stereo workbench Left