Human Visual Perception - Topics

n Deviations from ideal perspective projection due to l Aperture of the eye l Focus errors (spherical aberration) l Chromatic aberration l Dispersion n Effects can be summarized by a 2D convolution with the optical point-spread function (PSF). n Instead of a PSF, an optical line-spread function (LSF) is often given, which can be measured more easily.

Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 3

Optical LSF of the human eye

LSF measured for different pupil diameters (Campbell+Gubisch)

LSF calculated from eye aperture (due to diffraction)

Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 4

2 Optical modulation transfer function (MTF) of the human eye

n MTF is measured directly with sinewave gratings. n The optical modulation transfer function (MTF) can

be interpreted as contrast ratio image:object Fourier transform of the optical LSF.

spatial frequency [cpd]

Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 5

Sine wave grating

L - L contrast ratio = 2 1 L2 + L1 intensity

position

Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 6

3 Human retina I

light Signal propagation in the retina:

optic nerve

ganglion cells (=optic nerve)

fibers

ganglion cells

inner synaptic bipolar cells layer

amacrine cells bipolar cells horizontal cells

outer synaptic layer

receptors receptor nuclei

receptors

pigmented layer (epithelium cells)

Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 7

Human retina II

receptors in 1000/mm2 Rods Cones high sensitivity low sensitivity low light vision day light vision > 1 cd/m2 rods monochrome color cones "scotopic vision" "photopic vision"

blind spot fovea nose angle [degree]

Video displays

Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 8

4 Visual acuity

n Spatial resolution in lines/arcmin:

cones

rods

blind spot fovea nose angle [degree] n Minimum distance of adjacent cones in the central fovea limits spatial resolution. (2 - 2.3 mm 25 . . . 29 sec of arc)

Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 9

Weber-Fechner law, I

n Experiment: n Result: 2 surround /m luminance LS background cd

stimulus luminance LB area L in D

CONST threshold

2 LB, background luminance (cd/m )

Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 10

5 Weber-Fechner law, II

n "Weber-Fechner Law”

DL = cLB c = 0.01 ... 0.02 n Implies logarithmic relationship between physical luminance and subjectively perceived brightness. n Other proposed nonlinearities: square-root, cube-root, polynomials n g-characteristic of CRT displays is approximate inverse of nonlinearity of human brightness perception.

Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 11

Inhibition and excitation in the retina

n Receptive field of a ganglion cell (=fiber of the optic nerve) shows „center-surround response“ with both l Lateral inhibition l Lateral excitation

receptive field

center

surround

activity average of neuron activity

Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 12

6 Spatial contrast sensitivity

n Lateral inhibition and excitation together lead to a bandpass characteristic of the contrast sensitivity function of the human visual system ) 2 /m cd contrast sensitivity (@ 500

spatial frequency (cpd)

Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 13

Color vision: opponent color theory

n Retina carries out “matrix operation“ to represent colors in the opponent color system (Y, Y-B, R-G)

Opponent color model: Opponent color space:

B G R

white green Y Luminance signal YB blue yellow Chromatic RG channels

red black

B G R

Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 14

7 Color vision: contrast sensitivity in opponent color space n Spatial frequency response of Y-B and R-G channel (Girod, 1988):

spatial frequency (cpd) n Bandwidth Y:RG:YB approximately 8:5:3. n Some researchers have observed bandpass characteristic also for chromaticity channels.

Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 15

Spatial masking, I

dark light Experiment: distance D

dynamic noise bar

change in video signal amplitude [0 . . . 255] 80 ® 230 visibility threshold 80 ® 180 (w-Modell, Girod, 1987) 6 dB 80 ® 130 visibility threshold [log units] -20 -10 0 10 20 distance from edge [arcmin]

Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 16

8 Spatial masking, II

Visibility threshold for the g-predistorted video signal (w-Modell, Girod, 1987):

"g-shift"

6 dB change of video amplitude [0 . . . 255]

80 ® 230 80 ® 180 80 ® 130

visibility threshold [log units] -20 -10 0 10 20 distance from edge [arcmin]

Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 17

Spatiotemporal contrast sensitivity of luminance perception n Spatiotemporal contrast sensitivity of the luminance channel has bandpass characteristic. n Contrast sensitivity function separable for high spatial and temporal frequencies only. n Plot of contrast sensitivity function (from Kelly):

just modulation perceptable

spatial frequency temporal frequency

Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 18

9 Temporal masking n Visibility thresholds for g-predistorted video signal after luminance discontinuity (w-model, Girod, 1987): visibility threshold [arbitrary log units]

time after discontinuity [ms]

Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 19

Eye movements

slow corrective saccade saccade; v up to 800 °/s drift

saccade SPEM ; v < 30 °/s

SPEM: smooth pursuit eye movement

Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 20

10 Temporal masking and SPEMs

t=0 t=t0 t0 < t < t1 t1 < t

Eye fixates screen Eye tracks moving edge x x

dark dark

t 0 t 0

bright bright

t 1 t trajectory of eye 1 trajectory of eye movement movement t t Temporal masking

Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 21

Eye movements and spatiotemporal frequency response of the human visual system, I n Assume SPEM of constant velocity:

x' = x - v x t y' = y - v y t t' = t retina coordinate system display coordinate system n Coordinate transformation in spatiotemporal frequency space (“Doppler effect“)

w w x ' = x

wy ' = wy w w w w t ' = t + x vx + y vy

Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 22

11 Eye movements and spatiotemporal frequency response of the human visual system, II

relative velocity between eye and coordinate system

Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 23

Eye movements and spatiotemporal frequency response of the human visual system, III

relative velocity between eye and coordinate system

Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 24

12 Perception of a temporally sampled image signal, without movement

spatial frequency

temporal frequency

"window of perception"

Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 25

Perception of a temporally sampled image signal, translatory movement

spatial frequency 3 pel/frame

temporal frequency

n Phosphors with fast decay reproduce more than 20 temporal baseband replicas

"window of perception", n No spectral overlap -> perfect with SPEM reconstruction of original signal possible

Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 26

13 Anatomy of the human visual system (enlarged)

retina

visual cortex overlapping visual fields

Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 27

Human visual perception - summary

n Spatial frequency components visible up to 60 cpd n Logarithmic relationship between luminance and subjective brightness perception n Lateral inhibition -> spatial bandpass characteristic n Chromaticity channels have lower bandwidth n Visibility threshold often increased in the vicinity of edges, but sometimes decreased (“masking”). n SPEMs shear window of perception in spatiotemporal frequency space.

Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 28