Human Visual System & 3D Visualization Systems
Felix G. Hamza-Lup, Ph.D Associate Professor, Director NEWS Lab Computer Science and Information Technology Armstrong Atlantic State University Savannah, Georgia, USA
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 1 Outline
• Human Visual System • Components • Limitations/Deficiencies • Optical Illusions
• Light and Colors • Light Properties • Artificial Light Sources • Color Models
• 3D Visualization Systems
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 2 The Human
• Information i/o … • visual, auditory, haptic, movement
• Information stored in memory • sensory, short-term, long-term
• Information processed and applied • reasoning, problem solving, skill, error
• Emotion influences human capabilities
• Each person is different
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 3 Human Visual System (HVS) (1)
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 4 HVS (2)
• Rods and Cones excited by electromagnetic energy in the range 350-780 nm
• Sizes of rods and cones determines the resolution of HVS – our visual acuity
• The sensors in the human eye do not react uniformly to the light energy at different wavelengths
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 5 HVS (3) - Human Fovea
125 million Rods/eye Only 5-7 million Cones/eye Rods are 1000 times more sensitive to light Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 6 HVS (4)
• Three different cones in HVS: • Blue, green & yellow – often reported as red for compatibility with camera & film
• Transducin is a protein that resides in the retina and is able to effectively convert light energy into an electrical signal.
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 7 HVS (5) - Rod Cells
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 8 Vision (1)
• Two stages in vision • physical reception of stimulus • processing and interpretation of stimulus
• Resolution: our ability to discern separate pixels on a display depends: • display resolution • display size a. b. • distance from it.
Average is ~ 1' (minute of arc, 1/60 of degree) • Example: • 1' at reading distance 25 cm = 350ppi (pixels per inch). • iPhone 4 "Retina Display” • 3.5" display with 640×960 resolution = 326ppi
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 9 Vision (2) • Size and depth • visual angle indicates how much of view object occupies (relates to size and distance from eye) • visual acuity is the ability to perceive detail (limited) • familiar objects perceived as constant size (in spite of changes in visual angle when far away) • cues like overlapping help perception of size and depth
• Brightness • subjective reaction to levels of light • affected by luminance of object • measured by just noticeable difference • visual acuity increases with luminance as does flicker • Colour • made up of hue, intensity, saturation • cones sensitive to colour wavelengths • blue acuity is lowest • 8% males and 1% females colour blind
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 10 Reading
• Several stages: "Aoccdrnig to a rscheearch at Cmabrigde Uinervtisy, • visual pattern perceived it deosn't mttaer in waht oredr the ltteers in a wrod • decoded using internal representation of are, the olny iprmoatnt tihng is taht the frist and lsat language ltteers be at the rghit pclae. The rset can be a toatl • interpreted using knowledge of syntax, mses and you can sitll raed it wouthit porbelm. Tihs is semantics, pragmatics bcuseae the huamn mnid deos not raed ervey lteter by istlef, but the wrod as a wlohe." • Reading involves saccades and fixations • Perception occurs during fixations • Word shape is important to recognition • Negative contrast improves reading from computer screen
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 11 HVS Deficiencies & Limitations
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 12 Color – Trichromacy, Dichromacy
• Normal cones and pigment sensitivity enable an individual to distinguish all the different colors as well as subtle mixtures of hues - normal color vision is known as trichromacy.
• Dichromacy, a form of color blindness, or color deficiency, occurs when one of the pigments is seriously deviant in its absorption characteristics. • Protanopia – missing L • Deuteranopia – missing M (red-green) • Tritanopia – missing S
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 13 Color Deficiencies
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 14 HVS Limitations • In human vision, a significant degree of image processing takes place in the brain
• 1 million myelinated optical nerve fibers carry the information from 125 million rods and 5-7 million cones to the brain
• The retina is also involved in a wide range of processing tasks
• Multiple processes in the brain in parallel • edge sharpening • contrast enhancement • spatial summation • noise averaging • other forms of signal processing that have not yet been discovered.
• Optical limitations: • Illusions due to factors like retina memory • may conceal 2D/3D visualization system technical limitations Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 15 Color Perception – Limitation From 100.000 to 10 million (subjective)
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 16 HVS Optical Illusions
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 17 Optical Illusions - Interpreting the Signal
• The visual system compensates for: • movement • changes in luminance.
• Context is used to resolve ambiguity
• Optical illusions sometimes occur due to over compensation
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 18 Perception – Optical Illusion
Retina adaptation to neighboring color
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 19 Perception – Optical Illusion (1) How many colors ?
Only 3 pink, white, green When the green and pink colors are placed side by side, they enhance each other’s darker tones, making them look like completely new colors. Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 20 Perception – Optical Illusion (2)
Retina memory
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 21 Perception – Optical Illusion (3)
Are the horizontal lines parallel or do they slope?
Because we are trying to rationalize either the black vertical lines or white vertical lines, we cannot make sense of the untidiness of the columns and, to our eyes, the horizontal lines are sloping downwards
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 22 Perception – Optical Illusion (4)
Concentric Circles
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 23 Perception – Optical Illusion (5) How many legs ?
Flat 2D picture but the brain tries to make it 3D Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 24 Perception – Optical Illusion (6) Static/Dynamic ?
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 25 Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 26 Outline
• Human Visual System • Components • Limitations/Deficiencies • Optical Illusions
• Light and Colors • Light Properties • Artificial Light Sources • Color Models
• 3D Visualization Systems
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 27 Light & Color
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 28 Physics of Light
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 29 Light + Eye = Colors
• Light is a form of electromagnetic radiation • Visible spectrum 350 – 780 nm • No light => no color (proves the importance of the light rendering system in computer graphics)
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 30 Light & Color
• Color is the perceptual characteristic of light described by a color name. • Specifically, color is light, and light is composed of many colors—those we see are the colors of the visual spectrum: red, orange, yellow, green, blue, and violet.
• Objects absorb certain wavelengths and reflect others back to the viewer. We perceive these wavelengths as color.
• A color is described in three ways: • by its name (hue), • how pure or de-saturated it is • its value or lightness.
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 31 Light can Do …
• Reflection
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 32 Light Source Reflection (GBR)
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 33 Light can Do
• Refraction
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 34 Light can Do …
• Diffraction
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 35 Shadows
• Can improve the realism
• Illustrates spatial relationships among objects
• Soft Shadows • Made by area light • Umbra – totally blocked from the light source • Penumbra – partially blocked from the light source
• Can be modelled by a collection of point light sources
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 36 Measuring Light
• Light is measured with two main alternative sets of units:
• radiometry consists of measurements of light power at all wavelengths
• photometry measures light with wavelength weighted with respect to a standardized model of human brightness perception (to quantify illumination intended for human use).
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 37 Measuring Light - Radiometry
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 38 Measuring Light - Photometry
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 39 Relative Luminance
Relative luminance follows the photometric definition of luminance, but with the values normalized to 1 or 100 for a reference white
Relative luminance in colorimetric spaces
Y = 0.2126 R + 0.7152 G + 0.0722 B
The formula reflects the luminosity function: green light contributes the most to the intensity perceived by humans, and blue light the least.
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 40 Artificial Light Sources
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 41 Basic Artificial Light Sources
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 42 Rendering … effects (1)
• Texture-mapping — a method of applying detail to surfaces
• Bump-mapping — a method of simulating small-scale bumpiness on surfaces
• Fogging/participating medium — how light dims when passing through non-clear atmosphere or air
• Shadows — the effect of obstructing light • Hard shadows • Soft shadows — varying darkness caused by partially obscured light sources
• Transparency (optics), transparency (graphic) or opacity — sharp transmission of light through solid objects
• Translucency — highly scattered transmission of light through solid objects
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 43 Rendering ….effects (2)
• Reflection, Refraction, Diffraction
• Indirect illumination — surfaces illuminated by light reflected off other surfaces, rather than directly from a light source (also known as global illumination)
• Caustics (a form of indirect illumination) — reflection of light off a shiny object, or focusing of light through a transparent object, to produce bright highlights on another object
• Depth of field — objects appear blurry or out of focus when too far in front of or behind the object in focus
• Motion blur — objects appear blurry due to high-speed motion, or the motion of the camera
• Non-photorealistic rendering — rendering of scenes in an artistic style, intended to look like a painting or drawing
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 44 CoLoR Models
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 45 4 color models
1. RGB (CMY): The red, green, blue and cyan, magenta, yellow models are closely related, the primary colors of each form the secondary colors of the other.
2. HSB/HLS: Hue, saturation, and brightness and hue, lightness, and saturation are two variations of a similar model that is a standard for computer graphics and that closely models the qualities most apparent to human perception of color.
3. Munsell color system: is one of the most influential systems developed for ordering colors that can be used for production. While its practical application is mostly outside of print production, it still forms the basis for most other work on color modeling.
4. CIE: color models are highly influential systems for measuring color and distinguishing between colors. We will examine three CIE models: CIEXYZ, CIELUV, and CIELAB.
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 46 Definitions - Hue
• Hue: defines the color itself, for example, red in distinction to blue or yellow.
• The values for the hue axis run from 0–360° beginning and ending with red and running through green, blue and all intermediary colors like greenish-blue, orange, purple, etc.
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 47 Definitions - Saturation • Saturation: indicates the degree to which the hue differs from a neutral gray. Alternate terms: Fullness, Richness, Depth, Boldness.
• The values run from 0%, which is no color saturation, to 100%, which is the fullest saturation of a given hue at a given percentage of illumination.
Chroma (or Collorfulness) are closely related. Often confused with saturation, chroma has more to do with the sharp brightness of a color rather than the general fullness of it. When a color is said to be "chromatic", it contains a hue that is spectrally pure. Chroma is defined as "colorfulness" of an object relative to the brightness of a white object similarly illuminated, which allows for the fact that a surface of a given chroma displays increasing "colorfulness" as the level of illumination increases
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 48 Definitions - Lightness
• Lightness (or value or tone): indicates the level of illumination. Refers to the perceived reflectance of a surface, or to relative luminous reflectance of a colored surface patch
• The values run as percentages; • 0% appears black (no light) while • 100% is full illumination, which washes out the color (it appears white)
Lightness is the perceptual correlate of reflectance – perceived reflectance Brightness is the perceptual correlate of luminance – perceived luminance Value is defined as the relative lightness or darkness of a color (!) Value is used with a different meaning in the HSV model.
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 49 Definitions - Relative Luminance
Relative luminance (sometimes brightness) follows the photometric definition of luminance, but with the values normalized to 1 or 100 for a reference white
Relative luminance in colorimetric spaces Y = 0.2126 R + 0.7152 G + 0.0722 B
The formula reflects the luminosity function: green light contributes the most to the intensity perceived by humans, and blue light the least.
Luminance is the amount of visible light that comes to the eye from a surface do not confuse with Illuminance is the amount of light incident on a surface L=I*R do not confuse with Reflectance is the proportion of light reflected from a surface
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 50 Definitions - Luminosity
• Luminosity typically (and incorrectly) refers to relative luminance. This usage was popularized by Adobe Photoshop
• In the documentation of the CS3 version, the luminosity blending mode is still present, but is described in terms of luminance: “Luminosity: Creates a result color with the hue and saturation of the base color and the luminance of the blend color”.
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 51 Definitions – Shade & Tint
• Shade and tint are terms that refer to a variation of a hue.
• Shade: A hue produced by the addition of black.
• Tint: A hue produced by the addition of white.
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 52 1. RGB (CMY) – Color Models
• Red, green, and blue are the primary stimuli for human color perception and are the primary additive colors
• Cyan, magenta, and yellow correspond roughly to the primary colors in art production: red, blue, and yellow.
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 53 2. HSV & HSL – Color Models
• Hue Saturation Value (HSV) & Hue Saturation Lightness (HSL) • two related representations of points in an RGB color model that attempt to describe perceptual color relationships more accurately than RGB, while remaining computationally simple.
• (HCL) Hue Chroma Luminosity
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 54 HSV representations
HSV – as a cylinder HSV – as a cone
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 55 HSL vs. HSV
(!) Lightness: Various color models have an explicit term for this property. The Munsell color model uses the term value, while the HSL color model and Lab color space use the term lightness. The HSV model uses the term value a little differently: a color with a low value is nearly black, but one with a high value is the pure, fully-saturated color.
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 56 3. The Munsell Color System
• Munsell modeled his system as an orb around whose equator runs a band of colors.
• The axis of the orb is a scale of neutral gray values with white as the north pole and black as the south pole.
• Extending horizontally from the axis at each gray value is a gradation of color progressing from neutral gray to full saturation.
• With these three defining aspects, any of thousands of colors could be fully described. Munsell named these aspects, or qualities, Hue, Value, and Chroma.
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 57 4. CIE - Model
• Different HVS response for single frequency light – red/green/blue
• Relative brightness response at different frequencies
• Commision Internationale de L’Eclairage (CIE) standard observer curve
• Most sensitive to GREEN wave.
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 58 CIE & Colorimetry • Colorimetry is the science of measuring colors. • CIE has defined • a standard observer • a set of standard conditions for performing color measuring experiments • after many experiments using light sources of the wavelengths red=700nm, green=546.1nm and blue 435.8nm [WySt82] color matching curves were proposed by CIE.
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 59 CIE xy chromaticity diagram & color space
• We 3 types of color sensors that respond to different ranges of wavelengths, a full plot of all visible colors is a 3D figure.
• However, the concept of color can be divided into two parts: brightness and chromaticity. • E.g. the color white is a bright color, while the color grey is considered to be a less bright version of that same white. In other words, the chromaticity of white and grey are the same while their brightness differs.
• The CIE XYZ color space was deliberately designed so that • the Y parameter was a measure of the brightness or luminance of a color. • chromaticity of a color was then specified by the two derived parameters x and y, two of the three normalized values which are functions of all three tri-stimulus values X, Y, and Z: • The derived color space specified by x, y, and Y is known as the CIE xyY color space and is widely used to specify colors in practice.
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 60 Colors - Spectra
The Visible spectrum consists of billions of colors, a monitor can display millions, a high quality printer is only capable of producing thousands, and older computer systems may be limited to 216 cross-platform colors.
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 61 Outline
• Human Visual System • Components • Limitations/Deficiencies • Optical Illusions
• Light • Light and Colors • Reflection/Intensity • Artificial Light Sources
• 3D Visualization Systems - Hardware
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 62 Visualization Systems – a review
• Displays • Resolution Evolution • Non-immersive vs. Immersive • Mobile vs. Fixed • 2D vs 3D • Other Factors
• GPU Evolution
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 63 Resolution Evolution (1)
• A bit of History (1980-2000)
• I grew up with a VGA (Video Graphics Array) and it was fantastic
• 8 bit vs 24 bit color system
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 64 Resolution Evolution (2)
• Evolution in computer screen resolution 2000 - now
• Changes in screen sizes happen slowly
• Big screens are now the most common class
• In usability studies, the cardinal rule is always to test on the equipment used by the most customers
• Avoiding "frozen" designs and instead ensuring that your design can scale across a range of screen resolutions
• Since 2010, the core design target has been the big screen
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 65 Non-Immersive Displays (1)
Multiple Flatscreens 1990 Display Technology Flatscreen VR
- CRT screens ERA - Mainly 2D with some depth cues
- Multiple screens Gaming Applications
Panoram Tech 3-Screen Desktop Veridian System
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 66 Non-Immersive Displays (2)
18.1” LCD display, Resolution: 12801024
2000 Display Technology
- LCD screens ERA 17” LCD display, Resolution: SXGA (12801024) - Still 2D with some 3D capability Auto-stereoscopic 3-D Display(courtesy of Dimension - Significantly increased resolution Technologies Co.) Virtual Window 3-D Display (courtesy of Virtual Research Co.) and framerate due to GPU
FloStation
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 67 Non-Immersive Displays (3)
2000 Display Technology
• Specialized with 3D capability • Crystal Eyes 3D – Active Shutter System
Mini-Workbench The ImmersaDesk
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 68 Non-Immersive Displays (4) • 2010 – Now, Display Technology • 3D TV cornerstone (plasma, LCD or LED) • Active vs. Passive 3D • Active 3D widely introduced in 2010 • Glasses use liquid crystal shutters • Don't show jagged-edge artifacts and line structure • better when seen from extreme angles • Passive 3D introduced in 2011 • Glasses use simple polarizing lenses • Lighter and more comfortable • Cheaper • Don't introduce flicker • Not available on plasma screens • Touch screens
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 69 Immersive Displays (1)
• Cave Automatic Virtual Environment (CAVE) [Cruz-Neira,92] • Many variations over the past 2 decades • Projection based system • walls of a CAVE are typically made up of rear-projection screens • Stereoscopic LCD shutter glasses
• Head tracking (acoustic + inertial) CAVE 3-D large volume display (Fakespace Co.)
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 70 Immersive Displays (2)
Head Mounted Displays (1990-2010) Head Mounted - Fully immersive Displays (HMD)... - Partially immersive (AR type): - Video see-through - Optical see-through - Glasses type – conceptual level
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 71 Mobile vs Fixed Displays
• Smart Phones …..
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 72 Flexible Displays
• OLED
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 73 Touch-sensitive displays
• Detect the presence of finger or stylus on the screen. • works by interrupting matrix of light beams, capacitance changes or ultrasonic reflections • direct pointing device
• Advantages: • fast, and requires no specialised pointer • good for menu selection • suitable for use in hostile environment: clean and safe from damage.
• Disadvantages: • finger can mark screen • imprecise (finger is a fairly blunt instrument!) • lifting arm can be tiring Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 74 Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 75 Graphics Processing - GPU Evolution -
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 76 GPU Evolution
• GPU/CPU evolution last decade
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 77 Thank you FelixLup.net
Felix G. Hamza-Lup, Ph.D - Fulbright Specialist 2013 (Thailand) 78