RPI LRC Capturing the Lighting Edge New Color Metrics Mark Fairchild

Color Appearance of Displays, etc.

RGC scheduled September 28, 2012 from 6:45 AM to 9:45 AM RPI LRC Capturing the Lighting Edge New Color Metrics

Oct. 3, 2012

Mark Fairchild Rochester Institute of Technology, College of Science

1 Some Adaptation Demos

2 Color

3 4 5 6 Blur (Sharpness)

7 8 9 10 Noise

11 12 13 14 Chromatic, Blur, and Noise Adaptation

15 Outline

•Color Appearance Phenomena •Chromatic Adaptation •Metamerism •Color Appearance Models •HDR

16 Color Appearance Phenomena

17 Color Appearance Phenomena

If two stimuli do not match in color appearance when (XYZ)1 = (XYZ)2, then some aspect of the viewing conditions differs.

Various color-appearance phenomena describe relationships between changes in viewing conditions and changes in appearance.

Bezold-Brücke Hue Shift Abney Effect Helmholtz-Kohlrausch Effect Hunt Effect Simultaneous Contrast Crispening Helson-Judd Effect Stevens Effect Bartleson-Breneman Equations Chromatic Adaptation Color Constancy Memory Color Object Recognition

18 Simultaneous Contrast

The background in which a stimulus is presented inﬂuences the apparent color of the stimulus. Stimulus Indicates lateral interactions and adaptation.

Stimulus Color- Background Background Change Appearance Change

Darker Lighter

Lighter Darker

Red Green

Green Red

Yellow Blue

Blue Yellow

19 Simultaneous Contrast Example

(a)

(b)

20 Josef Albers

21 Complex Spatial Interactions

22 Hunt Effect

Corresponding chromaticities across indicated relative changes in luminance (Hypothetical Data) For a constant chromaticity, perceived 0.6 colorfulness increases with luminance.

0.5 As luminance increases, stimuli of lower colorimetric purity are required to match 1 10 a given reference stimulus. y 0.4 100 1000 10000

10000 1000 100 10 1 0.3 Indicates nonlinearities in visual processing.

0.2 0.2 0.3 0.4 0.5 0.6 x

23 Stevens Effect

1 Perceived lightness contrast increases with increasing adapting luminance.

As adapting luminance increases: dark colors look darker and light colors look lighter. L (Dark) L (63 dB) Indicates luminance-dependent nonlinearities. L (79 dB)

L (97 dB) 0.1 0.1 1 Y/Yn

24 Stevens & Hunt Effects

0.1 cd/m2 1.0 cd/m2 10 cd/m2

100 cd/m2 1000 cd/m2 10,000 cd/m2

25 Bartleson-Breneman

Perceived lightness as a function of relative luminance for various surround relative luminances 100 90 Apparent contrast in complex stimuli (i.e. images) 80 increases with increasing surround luminance. 70 60 Decreased surround luminance increases the 50 brightness of all image colors, but the effect is 40 greater for dark colors. 30 Lightness (Average) 20 Lightness (Dim) Indicates a differential contrast effect (white-point 10 Lightness (Dark) resetting). 0 0 10 20 30 40 50 60 70 80 90 100 Relative Luminance, Y

26 Surround Effect Demo

27 28 Adaptation

Light Adaptation: Decrease in visual sensitivity with increases in luminance. (Automatic Exposure Control)

Dark Adaptation: Increase in visual sensitivity with decreases in luminance. (Automatic Exposure Control)

Chromatic Adaptation: Independent sensitivity regulation of the mechanisms of color vision. (Automatic Color Balance)

29 Local Adaptation

Linear Mapping Perceptual Mapping

30 Chromatic Adaptation

1.25 The three cone types, LMS, are capable of 1 independent sensitivity regulation. (Adaptation occurs in higher-level mechanisms as well.) 0.75 Magnitudes of chromatic responses are dependent 0.5 on the state of adaptation (local, spatial, temporal). Afterimages provide evidence. 0.25

0 400 450 500 550 600 650 700 Wavelength (nm)

31 32 A Cyan Filter

33 34 35 Color Constancy (Discounting)

0.54 Adapting Chromaticities Achromatic Chromaticities Incandescent We perceive the colors of objects to remain 0.52 unchanged across large changes in illumination <-- Hands <-- No Hands color. •Not True v' 0.50 •Chromatic Adaptation •Poor Color Memory •Cognitive Discounting-the-Illuminant 0.48 Daylight

<-- Hands <-- No Hands 0.46 0.18 0.20 0.22 0.24 0.26

36 Diamonds

37 With Tips

38 On Backgrounds

39 Both Tips & Backgrounds!

40 Real-World Discounting

41 42 Chromatic Adaptation Modeling

Chromatic Adaptation: Largely independent sensitivity regulation of the (three) mechanisms of color vision.

1.25

0.75

0.5

0.25

0 400 450 500 550 600 650 700 Wavelength (nm) 43 Chromatic Adaptation Models Model: X1Y1Z1 La = f(L, Lwhite ,...) 3x3 Ma = f(M, Mwhite,...) L1M1S1 S a = f(S, Swhite,...) VC1

LaMaSa

Transform (CAT): VC2 L M S XYZ 2 = f(XYZ1, XYZwhite ,...) 2 2 2 3x3

X2Y2Z2

44 XYZ-to-LMS

2 z 1

S M L 1.5

y 0.75 s t i e v i u l t i a s L 0.400 0.708 −0.081 X v

n x

e y s s u

0.5 1 l e u v i m t i t a l M = −0.226 1.165 0.046 Y s i e r R 0.25 T S 0.000 0.000 0.918 Z 0.5 0 380 480 580 680 780 Wavelength, nm 0 380 480 580 680 780 Wavelength, nm

45 Johannes von Kries

Johannes von Kries "Father of Chromatic-Adaptation Models"

46 von Kries Hypothesis

"This can be conceived in the sense that the individual components present in the organ of vision are completely independent of one another and each is fatigued or adapted exclusively according to its own function." -von Kries, 1902

La = kLL

Ma = k MM

S a = kSS

Von Kries thought of this “proportionality law” as an extension of Grassmann’s Laws to span two viewing conditions.

47 Some Evolution of CATs •Nayatani et al. Nonlinear Model •Fairchild (1991) & (1994) Models •Bradford Model •CIELAB & CIELUV •CAT02

48 Metamerism: Illuminant, Observer, & Metrics

49 Illuminant Metamerism

• Same Observer, Change Illuminantion • In theory, must have tristimulus equality for a reference illuminant. • Calculate Color Difference for Test Illuminant: CIE Special Index of Metamerism • Calculate Spectral Difference: General Index of Metamerism

50 In Practice, Paramers, not Metamers

Corrected Standard Batch batch

X 19.93 21.96 19.93

Y 14.29 14.87 14.29

Z 7.23 7.91 7.23

0.8 HJ HJ HJ J 0.6 H r o t Standard

c B a f

e J J Batch c

n 0.4 H a t B H Corrected batch c

e B l f B e B

R B J B 0.2 H B HJ HJ H H J B B BJ H J J BHJ H H BHJ BHJ BJ B B 0 400 450 500 550 600 650 700 Wavelength (nm) 51 CIE Special Index of Metamerism, Change in Illuminant

• Must have an exact tristimulus match for a single viewing and observer condition. • Calculate Color Difference for Test Illuminant • Workﬂow: • Measure Parameric Pair • Correct Batch as Needed for Reference Illuminant • Calculate CIE DE2000 for Test Illuminant

52 General Index of Metamerism is Spectral Based

• Weighted RMS Error (Using Diagonal of Matrix R)

n ⎛ ⎞2 ⎜ w ΔR ⎟ ∑⎝ λ λ ⎠ WRMS = λ=1 n w = diag R λ ( )

53 Observer Metamerism 1931 2° vs. 1964 10° standard observers

1.5 s e u l a v

u 1 l u m i t s i r T 0.5

0 380 480 580 680 780 Wavelength, nm

Dashed lines, 1964 observer

54 Variability in Color Matching Functions

Wyszecki and Stiles, “Color Science” 55 CIE Standard Deviate Observer

2.2 • Based on Stiles and Burch 1.8 data s e

u 1.4 l a v

s Normalization of primaries

u 1 l • u and V-lambda minimized m i t s

i 0.6 r true variance T 0.2 Not recommended -0.2 • 380 480 580 680 780 Wavelength, nm • CIE Pub. 80 20-year old (solid lines) 60-year old (dashed lines)

56 CIE TC 1-36 (2006)

⎣⎡− Dτ ,max,macula ⋅Dmacula,relative (λ)− Dτ ,ocul (λ)⎦⎤ l (λ) = αi,l (λ)⋅10

⎣⎡− Dτ ,max,macula ⋅Dmacula,relative (λ)− Dτ ,ocul (λ)⎦⎤ m(λ) = αi,m (λ)⋅10

⎣⎡− Dτ ,max,macula ⋅Dmacula,relative (λ)− Dτ ,ocul (λ)⎦⎤ s (λ) = αi,s (λ)⋅10

Color-matching Cone spectral Macula spectral Lens and ocular functions absorptances density media spectral as a function of density as a ﬁeld size function of age (i=internal)

57 Examples of Cone Fundamentals

L-, M-, & S-Cone Fundamentals (2- & 10-Deg. @ Age 32) L-, M-, & S-Cone Fundamentals (10-Deg. @ Ages 20 & 80)

1.00 1.00 l-bar (2) l-bar (20) 0.90 m-bar (2) 0.90 m-bar (20) s-bar (2) s-bar (20) 0.80 l-bar (10) 0.80 l-bar (80) m-bar (10) m-bar (80) 0.70 0.70 s-bar (10) s-bar (80)

0.60 0.60

0.50 0.50

0.40 0.40

Relative Sensitivity 0.30 Relative Sensitivity 0.30

0.20 0.20

0.10 0.10

0.00 0.00 390 440 490 540 590 640 690 740 390 440 490 540 590 640 690 740 Wavelength (nm) Wavelength (nm)

Normalized to Peak Height

58 Effect of Age vs. Field Size

2 2 xbar 1931 xbar 1931 ybar 1931 ybar 1931 zbar 1931 zbar 1931 x25yrs,2degree x25,10degree 1.5 1.5 y25yrs,2degree y25,10degree z25yrs,2degree z25,10degree x65yrs,2degree x65,10degree y65yrs,2degree y65,10degree 1 1 z65yrs,2degree z65,10degree ristimulus values ristimulus values

T 0.5 T 0.5

0 0

-0.5 -0.5 380 430 480 530 580 630 680 730 780 380 430 480 530 580 630 680 730 780 Wavelength (nm) Wavelength (nm)

59 Some Metamers

0.7 0.8 Vinyl-1 O Vinyl-1 G 0.7 0.6 Vinyl 2-A Vinyl 2-B 0.6 Fabric-A 0.5 Fabric-B

0.5 0.4

0.4

0.3 Reﬂectance factor Reﬂectance factor 0.3

0.2 0.2

0.1 0.1

0 0 380 430 480 530 580 630 680 730 780 380 430 480 530 580 630 680 730 780 Wavelength (nm) Wavelength (nm)

Grays Materials

60 Observer Metamerism

2,25 2,65

1931

10,25 61 10,65 Illuminant Metamerism

1931, D65 1931, A

1931, HP Na 1964, HP Na

62 Color “Blindness”

63 Cone Spectral Sensitivities

S M L

y 0.75 t i v i t i s n e s

0.5 e v i t a l e

R 0.25

0 380 480 580 680 780 Wavelength, nm

64 Color Deﬁciencies

Dichromats

Protanopia: Lacking “L-cone response” Deuteranopia: Lacking “M-cone response” Tritanopia: Lacking “S-cone response” Also “Anomalous Trichromats” ~8% of Males have Color Vision Deﬁciencies

Various tests available at: , . 65 Cone Mosaic

“Normal” Deuteranope

66 Color Vision Deﬁciencies

Original

Protanope (Missing L Cones)

Pseudo-isochromatic plates Dueteranope (Missing M Cones)

Tritanope (Missing S Farnsworth- Cones) Munsell 100-Hue vischeck.com Test 67 Chromatic Adaptation Reminder

68 Deﬁnitions • Chromatic adaptation: changes in the visual system that approximately compensate for changes in the spectral quality of illumination. • Light adaptation: changes in the visual system that approximately compensate for changes in the level of illumination.

Luminance

0.0001 .001 .1 to 1 5 20 50 to 150 150 to 600 3000 6000 +10000 Luminance (cd/m2)

69 Deﬁnitions

• Corresponding color stimuli: Pairs of color stimuli that look alike when one is seen in one set of adaptation conditions and the other is seen in a different set. • Chromatic adaptation transform (CAT) – computational approach to calculate corresponding colors,

70 Corresponding Color Experiment

A color chip is selected under daylight that matches the perception of the sample viewed under tungsten

71 Color Inconstancy Index

72 Three-Color Gray is Color Inconstant

0.5 0.5

0.4 0.4

0.3 0.3 Factor Factor

0.2 0.2 Reflectance 0.1 Reflectance 0.1

0.0 0.0 380 440 500 560 620 680 380 440 500 560 620 680 Wavelength (nm) Wavelength (nm)

Daylight 73 Incandescent Color Inconstancy (Left) vs. Metamerism (Right)

One Object Two Objects

74 Corresponding Color Calculation

⎛ X ⎞ ⎛ X⎞ ⎜ c⎟ ⎜ ⎟ −1 ⎜ Yc ⎟ = MCAT02MVKMCAT02⎜ Y⎟ ⎜ Z ⎟ ⎜ Z⎟ ⎝ c ⎠ ⎝ ⎠

Daylight: XcYcZc Incandescent: XYZ

75 Color Inconstancy Index (CII) Calculation

1. Measure spectral reﬂectance factor of object 2. Calculate tristimulus values for a standard observer and all illuminants of interest (e.g., A, F2, F11, F8, D50) 3. Calculate corresponding colors from each illuminant to D65. (D65 is selected because CIELAB has its best visual uniformity for this illuminant.)

76 Color Inconstancy Index (CII) Calculation

4. Calculate color inconstancy index (CII):

1/2 ⎡ 2 2 2⎤ ⎛ * ⎞ ⎢ ΔL ⎛ ΔC* ⎞ ⎛ ΔH* ⎞ ⎥ CII = ⎜ ⎟ + ⎜ ab ⎟ + ⎜ ab ⎟ ⎢⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎥ ⎢⎝ 2SL ⎠ ⎝ 2S C ⎠ ⎝ SH ⎠ ⎥ ⎣ ⎦

77 Color Appearance Models

78 What About Appearance?

Chromatic-adaptation models provide nominal scales for color appearance.

Two stimuli in their relative viewing conditions match each other.

BUT what color are they??

We need color-appearance models to get interval and ratio scales of: Lightness, Brightness, Hue, Chroma, and Colorfulness.

79 Flow Chart

Measure Physical Stimuli Φ(λ)

Stimulus, Background, Surround, etc.

Calculate Tristimulus Values XYZ (LMS)

Stimulus, Background, Surround, etc.

Calculate Correlates of Perceptual Attributes

Lightness, Brightness, Chroma, Colorfulness, Hue, etc.

80 Structure of CAMs

1.25

Chromatic Adaptation Transform 0.75 (to Implicit or Explicit Reference Conditions) Corresponding Colors 0.5

0.25

0 400 450 500 550 600 650 700 Wavelength (nm)

Color Space Construction Cone Responses Opponent Responses Appearance Correlates

81 CIELAB as an Example

CIELAB Does: •Model Chromatic Adaptation •Model Response Compression •Include Correlates for Lightness, Chroma, Hue •Include Useful Color Difference Measure

CIELAB Doesn't: •Predict Luminance Dependent Effects •Predict Background or Surround Effects •Have an Accurate Adaptation Transform

82 CIELAB as a CAM

Chromatic Adaptation Lightness L X/Xn, Y/Yn, Z/Zn

Yellowness +b Opponent Processes X-Y Greenness –a Redness +a Y- Z Uniform Spacing Blueness –b Constants 116, 500, 200 Cube Root

83 CIELAB Equations

L* = 116f(Y/Yn ) − 16

a* = 500[f(X/Xn ) − f(Y/Yn )]

b* = 200[f(Y/Yn )− f(Z/Zn )]

f(ω) = (ω)1/3 ω > 0.008856 f (ω) = 7.787(ω) + 16/116 ω ≤ 0.008856

84 CIELAB Lightness

L* = 116f(Y/Yn ) − 16

f(ω) = (ω)1/3 ω > 0.008856 f (ω) = 7.787(ω) + 16/116 ω ≤ 0.008856

10 9 8 7 6 5 4 Munsell Value 3 2 1 0 0 1 2 3 4 5 6 7 8 9 10 L*/10

85 CIELAB Chroma

1/ 2 C* = a * 2 +b *2 ( )

Increasing Chroma, C*

Neutrals Have Zero Chroma, C* = 0.0

86 Saturation in CIELAB

Due to the lack of a related chromaticity diagram, saturation is not ofﬁcially deﬁned in CIELAB.

However recalling the deﬁnitions of chroma (colorfulness/ brightness of white), lightness (brightness/brightness of white), and saturation (colorfulness/brightness).

L* Constant Constant Chroma Saturation

Constant Lightness Saturation = C*/L*

C* 87 CIELAB Hue

90° +b*

h° ab +a* −1⎛ b *⎞ 0° h ab = tan 180° ⎝ a *⎠ -a*

-b* Relative Hue Scale — Where's Red? 270°

Approximate Hue Angles of NCS Unique Hues R — 24° Y — 90° G — 162° B — 246°

Note: The number of discriminable hue steps is not equal between each of the unique hues.

88 Why Not Just CIELAB? Positive Aspects: •Accounts for Chromatic Adaptation •Works Well for Near-Daylight Illuminants (also Medium Gray Background & Surround and Moderate Luminance Levels) • Negative Aspects: •Does Not Account for Changes in: Background Surround Luminance Cognition •Cannot Predict Brightness & Colorfulness •"Wrong" von Kries Transform Works Poorly for Large Changes From Daylight •Constant-Hue Predictions could be Improved (especially Blue) CIELAB Makes a Good, Simple Baseline for Comparison

89 Extending CIELAB

90 Beyond CIELAB

• More Accurate Adaptation Transform

• Luminance Dependencies

• Surround Dependencies

• Brightness & Colorfulness

• Hue Linearity

(Hunt, Nayatani, RLAB, LLAB, CIECAM97s, CIECAM02)

91 Extending CIELAB

•Main Limitation is “Wrong” von Kries •Can be Replaced with More Accurate CAT •CIELAB Under Daylight a Very Good Color Space

92 CIELAB plus CAT Concept

Stimulus & Viewing Conditions Colorimetry

Accurate CAT (e.g., CAT02)

Corresponding Colors (XYZ) and

Reference Illuminant (XnYnZn)

CIELAB Equations

CIELAB Lightness, Chroma, Hue (L*C*h) to Describe Appearance

93 CIELAB plus CAT Example

Step 1: Obtain colorimetric data for stimulus and viewing conditions.

Step 2: Use CAT02 to compute corresponding colors for CIE Illuminant D65 (and 1000 lux).

Step 3: Compute CIELAB coordinates using corresponding colors from step 2 and D65 white.

Step 4: Use CIELAB L*C*h as appearance correlates.

94 CIECAM02

95 Inputs

2 LA: Adapting Field Luminance in cd/m (often 20% of the luminance of white)

XYZ: Relative Tristimulus Values of the Sample

XwYwZw: Relative Tristimulus Values of the White

Yb: Relative Luminance of the Background

D: Speciﬁes the Degree of Adaptation: D = 1.0, (Complete Adaptation or Discounting) D = 0.0, (No Adaptation) D in Between, (Various Degrees of Incomplete Adaptation)

96 Outline of Model Structure

• Chromatic Adaptation Transform (to Implicit Ill. E Reference Conditions) Corresponding Colors

• Color Space Construction Cone Responses Opponent Responses Appearance Correlates

97 Chromatic Adaptation Transform: CAT02

• von Kries Normalization • Linear (normal von Kries) • “Sharpened” “Cone” Responses (Optimized) • Generally Very Good Performance

98 Transform to RGB Responses

⎡ R⎤ ⎡ X⎤ ⎢ ⎥ ⎢ ⎥ G M Y ⎢ ⎥ = CAT 02⎢ ⎥ ⎣⎢ B⎦⎥ ⎣⎢ Z⎦⎥ ⎡ 0.7328 0.4296 −0.1624⎤ ⎡ 1.0961 −0.2789 0.1827⎤ ⎢ ⎥ −1 ⎢ ⎥ M 0.7036 1.6975 0.0061 M 0.4544 0.4735 0.0721 CAT 02 = ⎢− ⎥ B = ⎢ ⎥ ⎣⎢ 0.0030 0.0136 0.9834 ⎦⎥ ⎣⎢− 0.0096 −0.0057 1.0153⎦⎥

99 Adaptation Transform

Rc = [YW (D/Rw ) + (1− D)]R

Gc = [YW (D/Gw ) + (1− D)]G

B Y D/B (1 D) B c = [ W ( w ) + − ]

100 Adapted Cone Responses

⎡ R'⎤ ⎡ Rc ⎤ ⎢ ⎥ 1 ⎢ ⎥ G' M M− G ⎢ ⎥ = H CAT 02⎢ c⎥ ⎣⎢ B'⎦⎥ ⎣⎢ Bc ⎦⎥ ⎡ 0.38971 0.68898 −0.07868⎤ ⎡1. 9102 −1.1121 0.2019⎤ M ⎢ 0.22981 1.18340 0.04641 ⎥ M −1 ⎢0.3710 0.6291 0.00 ⎥ H = ⎢ − ⎥ H = ⎢ ⎥ ⎢ 0.00 0.00 1.00 ⎥ ⎢ 0.00 0.00 1.00 ⎥ ⎣ ⎦ ⎣ ⎦

0.42 400(FL R'/100) R'a = 0.42 + 0.1 F R'/100 + 27.13 [( L ) ] 0.42 400(FLG'/100) G'a = 0.42 + 0.1 F G'/100 + 27.13 [( L ) ] 0.42 400(FL B'/100) B'a = 0.42 + 0.1 F B'/100 + 27.13 [( L ) ]

101 Opponent Responses

A = [2R'a +G'a +(1/20)B'a −0.305]Nbb

a = R' a −12G' a /11 + B' a /11

b = (1/9)(R ' a +G' a −2B'a )

102 Appearance Correlates

• Brightness, Lightness • Colorfulness, Chroma, Saturation • Hue

• Built Up to Fit Experimental Data • Need 5 of 6 to Fully Describe Appearance

103 Hue

The degree to which a stimulus can be described as similar to or different from stimuli that are described as red, green, blue, and yellow.

−1 1 ⎡ ⎛ π ⎞ ⎤ h = tan b /a e = ⎢cos ⎜ h + 2⎟ + 3.8⎥ ( ) 4 ⎣ ⎝ 180 ⎠ ⎦

Red: h = 20.14, e = 0.8, H = 0 or 400, Yellow: h = 90.00, e = 0.7, H = 100, Green: h = 164.25, e = 1.0, H = 200, Blue: h = 237.53, e = 1.2. H = 300

104 Lightness

The brightness of a stimulus relative to the brightness of a stimulus that appears white under similar viewing situations.

cz J 100 A/A = ( w)

105 Brightness

The perceived quantity of light emanating from a stimulus.

0.5 0.25 Q = (4 /c)(J /100) (Aw + 4)FL

106 Chroma

The colorfulness of a stimulus relative to the brightness of a stimulus that appears white under similar viewing conditions.

2 2 1/ 2 (50000/13)NcNcbe a + b t = ( ) R'a +G'a +(21/20)B'a

0.5 0.73 C = t 0.9(J /100) (1.64 − 0.29n )

107 Colorfulness

The perceived quantity of hue content (difference from gray) in a stimulus.

Colorfulness increases with luminance.

0.25 M = CFL

108 Saturation

The colorfulness of a stimulus relative to its own brightness.

s =100 M Q

109 Chroma/Saturation s s s e s n t e h n t g i h L g i L

Saturation Chroma

110 High Dynamic Range Perception

111 The HDR Photographic Survey • Online High-Dynamic Range Image Database • Calibrated Images • Colorimetric Scene Measurements • In Situ Color Appearance Scaling • Pleasant Scenes (Psychophysics) • Technical Challenges (Rendering)

• 40 Scenes ... 112 Examples

113 9-Stop Mosaic

114 Linear OpenEXR

115 Locally Rendered

116 9-Stop Mosaic

117 Linear OpenEXR

118 Locally Rendered

119 Comparison

Linear

CS2 Local

120iCAM06 Comparison 2

Linear

CS2 Local

121iCAM06 Read More ... Questions??

• Thank you ...

123