Human visual perception - topics n Visual acuity n Weber-Fechner Law n Lateral inhibition and excitation n Transfer functions of the color channels n Spatial and temporal masking n Eye movements Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 1 Anatomy of the human eye iris cornea pupil fovea lens visual axis optic axis optic disk optic nerve retina sclera object retinal image Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 2 1 Optical properties of the human eye n Deviations from ideal perspective projection due to l Aperture of the eye l Focus errors (spherical aberration) l Chromatic aberration l Dispersion n Effects can be summarized by a 2D convolution with the optical point-spread function (PSF). n Instead of a PSF, an optical line-spread function (LSF) is often given, which can be measured more easily. Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 3 Optical LSF of the human eye LSF measured for different pupil diameters (Campbell+Gubisch) LSF calculated from eye aperture (due to diffraction) Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 4 2 Optical modulation transfer function (MTF) of the human eye n MTF is measured directly with sinewave gratings. n The optical modulation transfer function (MTF) can be interpreted as contrast ratio image:object Fourier transform of the optical LSF. spatial frequency [cpd] Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 5 Sine wave grating L - L contrast ratio = 2 1 L2 + L1 intensity position Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 6 3 Human retina I light Signal propagation in the retina: optic nerve ganglion cells (=optic nerve) fibers ganglion cells inner synaptic bipolar cells layer amacrine cells bipolar cells horizontal cells outer synaptic layer receptors receptor nuclei receptors pigmented layer (epithelium cells) Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 7 Human retina II receptors in 1000/mm2 Rods Cones high sensitivity low sensitivity low light vision day light vision > 1 cd/m2 rods monochrome color cones "scotopic vision" "photopic vision" blind spot fovea nose angle [degree] Video displays Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 8 4 Visual acuity n Spatial resolution in lines/arcmin: cones rods blind spot fovea nose angle [degree] n Minimum distance of adjacent cones in the central fovea limits spatial resolution. (2 - 2.3 mm 25 . 29 sec of arc) Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 9 Weber-Fechner law, I n Experiment: n Result: 2 surround /m luminance LS background cd stimulus luminance LB area L in D CONST threshold 2 LB, background luminance (cd/m ) Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 10 5 Weber-Fechner law, II n "Weber-Fechner Law” DL = cLB c = 0.01 ... 0.02 n Implies logarithmic relationship between physical luminance and subjectively perceived brightness. n Other proposed nonlinearities: square-root, cube-root, polynomials n g-characteristic of CRT displays is approximate inverse of nonlinearity of human brightness perception. Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 11 Inhibition and excitation in the retina n Receptive field of a ganglion cell (=fiber of the optic nerve) shows „center-surround response“ with both l Lateral inhibition l Lateral excitation receptive field center surround activity average of neuron activity Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 12 6 Spatial contrast sensitivity n Lateral inhibition and excitation together lead to a bandpass characteristic of the contrast sensitivity function of the human visual system ) 2 /m cd contrast sensitivity (@ 500 spatial frequency (cpd) Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 13 Color vision: opponent color theory n Retina carries out “matrix operation“ to represent colors in the opponent color system (Y, Y-B, R-G) Opponent color model: Opponent color space: B G R white green Y Luminance signal YB blue yellow Chromatic RG channels red black B G R Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 14 7 Color vision: contrast sensitivity in opponent color space n Spatial frequency response of Y-B and R-G channel (Girod, 1988): spatial frequency (cpd) n Bandwidth Y:RG:YB approximately 8:5:3. n Some researchers have observed bandpass characteristic also for chromaticity channels. Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 15 Spatial masking, I dark light Experiment: distance D dynamic noise bar change in video signal amplitude [0 . 255] 80 ® 230 visibility threshold 80 ® 180 (w-Modell, Girod, 1987) 6 dB 80 ® 130 visibility threshold [log units] -20 -10 0 10 20 distance from edge [arcmin] Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 16 8 Spatial masking, II Visibility threshold for the g-predistorted video signal (w-Modell, Girod, 1987): "g-shift" 6 dB change of video amplitude [0 . 255] 80 ® 230 80 ® 180 80 ® 130 visibility threshold [log units] -20 -10 0 10 20 distance from edge [arcmin] Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 17 Spatiotemporal contrast sensitivity of luminance perception n Spatiotemporal contrast sensitivity of the luminance channel has bandpass characteristic. n Contrast sensitivity function separable for high spatial and temporal frequencies only. n Plot of contrast sensitivity function (from Kelly): just modulation perceptable spatial frequency temporal frequency Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 18 9 Temporal masking n Visibility thresholds for g-predistorted video signal after luminance discontinuity (w-model, Girod, 1987): visibility threshold [arbitrary log units] time after discontinuity [ms] Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 19 Eye movements slow corrective saccade saccade; v up to 800 °/s drift saccade SPEM ; v < 30 °/s SPEM: smooth pursuit eye movement Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 20 10 Temporal masking and SPEMs t=0 t=t0 t0 < t < t1 t1 < t Eye fixates screen Eye tracks moving edge x x dark dark t 0 t 0 bright bright t 1 t trajectory of eye 1 trajectory of eye movement movement t t Temporal masking Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 21 Eye movements and spatiotemporal frequency response of the human visual system, I n Assume SPEM of constant velocity: x' = x - v x t y' = y - v y t t' = t retina coordinate system display coordinate system n Coordinate transformation in spatiotemporal frequency space (“Doppler effect“) w w x ' = x wy ' = wy w w w w t ' = t + x vx + y vy Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 22 11 Eye movements and spatiotemporal frequency response of the human visual system, II relative velocity between eye and coordinate system Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 23 Eye movements and spatiotemporal frequency response of the human visual system, III relative velocity between eye and coordinate system Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 24 12 Perception of a temporally sampled image signal, without movement spatial frequency temporal frequency "window of perception" Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 25 Perception of a temporally sampled image signal, translatory movement spatial frequency 3 pel/frame temporal frequency n Phosphors with fast decay reproduce more than 20 temporal baseband replicas "window of perception", n No spectral overlap -> perfect with SPEM reconstruction of original signal possible Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 26 13 Anatomy of the human visual system (enlarged) retina visual cortex overlapping visual fields Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 27 Human visual perception - summary n Spatial frequency components visible up to 60 cpd n Logarithmic relationship between luminance and subjective brightness perception n Lateral inhibition -> spatial bandpass characteristic n Chromaticity channels have lower bandwidth n Visibility threshold often increased in the vicinity of edges, but sometimes decreased (“masking”). n SPEMs shear window of perception in spatiotemporal frequency space. Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 28 14.