Andrew Stockman

INTRODUCTION

Advanced Colour

NEUR 3001/G001/M001 Advanced Visual Andrew Neuroscience Stockman

Light 400 - 700 nm is important for vision

How dependent are we on colour?

Advanced Colour 1 Andrew Stockman

No colour… Colour…

ACHROMATIC COMPONENTS

Split the image into...

But just how important is colour? CHROMATIC COMPONENTS

Advanced Colour 2 Andrew Stockman

CHROMATIC COMPONENTS ACHROMATIC COMPONENTS

Chromatic information by itself provides relatively Achromatic information is important for fine detail … limited information…

How do we see colours? Human photoreceptors Rods Cones An image of the world is . Achromatic . Daytime, achromatic projected by the cornea and night vision and chromatic vision lens onto the rear surface of . 1 type . 3 types the eye: the . Rod Long‐wavelength‐ sensitive (L) or “red” cone

Middle‐wavelength‐ sensitive (M) or “” cone The back of the retina is carpeted by a layer of light‐ Short‐wavelength‐ sensitive photoreceptors sensitive (S) or “blue” (rods and cones). cone

Advanced Colour 3 Andrew Stockman

Rod and cone distribution

Central fovea is rod-free, and the very central is rod- and S-cone free

0.3 mm of eccentricity is about 1 deg of visual angle

Chromophore (chromo- color, + -phore, producer) Light-catching portion of any molecule

11-cis retinal. The molecule is twisted at the 11th carbon.

Photopigment molecule (cone) From Sharpe, Stockman, Jägle & Nathans, 1999

Advanced Colour 4 Andrew Stockman

Chromophore Chromophore

A photon is absorbed A photon is absorbed

the energy of which initiates a conformational change to…

Chromophore Chromophore

A photon is absorbed

This process is binary: all (1) or nothing (0). the energy of which initiates a conformational change to…

What does that tell us about how wavelength all-trans retinal be encoded? (side chains omitted for simplicity).

Advanced Colour 5 Andrew Stockman

Colour isn’t just about physics. For example:

Is colour, as we perceive it, mainly a These two property of physics or biology? metamers of yellow

+

though physically very different, can appear identical.

There are many other such metamers or matches…

Advanced Colour 6 Andrew Stockman

COLOUR MIXING

Advanced Colour 7 Andrew Stockman

What was colour mixing able to tell us about colour vision (before we Human vision is knew about the trichromatic. underlying biology)?

Colour TV

Trichromacy is exploited in colour reproduction, since the myriad of colours perceived can be produced by mixing together small dots of three colours.

If you look closely at a colour television (or this projector output)…

Trichromacy means that 3-coloured dots 3-coloured bars colour vision is relatively simple.

It is a 3 variable system…

Advanced Colour 8 Andrew Stockman

The dots produced by a TV or projector are so small that they are mixed together by the eye and thus appear as uniform patches of colour

The laws of colour mixing that apply to projected lights or combinations of dots are called “additive”.

The laws of colour mixing that apply to pigments or paints But what about mixing paints? are different because they depend on what is absorbed or “subtracted” from the reflected light by the pigment.

WHITE PIGMENT Photo: Jozsef Szasz-Fabian

Advanced Colour 9 Andrew Stockman

WHITE PIGMENT RED PIGMENT

A white pigment reflects red, green and blue lights. A red pigment subtracts green and blue and reflects red.

GREEN PIGMENT BLUE PIGMENT

A green pigment subtracts red and blue and reflects green. A blue pigment subtracts red and green and reflects blue.

Advanced Colour 10 Andrew Stockman

Subtractive colour mixing (of paints or pigments)

Laws of subtractive colour mixing YELLOW PIGMENT (of paints or pigments)

MAGENTA PIGMENT CYAN PIGMENT

Subtractive colour mixing Subtractive colour mixing

YELLOW PIGMENT YELLOW PIGMENT += + = GREEN PIGMENT RED PIGMENT

CYAN PIGMENT PIGMENT

Advanced Colour 11 Andrew Stockman

Subtractive colour mixing SUBTRACTIVE COLOUR MIXING

CYAN PIGMENT + = BLUE PIGMENT

MAGENTA PIGMENT

SUBTRACTIVE ADDITIVE SUBTRACTIVE ADDITIVE COLOUR MIXING COLOUR MIXING COLOUR MIXING COLOUR MIXING

NOTE THAT THESE MIXING “LAWS” ARE BOTH But, why is normal human vision a CONSISTENT WITH HUMAN COLOUR VISION trichromatic, three variable system? BEING A TRICHROMAT, THREE VARIABLE SYSTEM.

Advanced Colour 12 Andrew Stockman

One reason, of course, is because just three cone But trichromacy also depends on the fact photo‐receptors underlie daytime colour vision. that the output of each photoreceptor is:

“UNIVARIANT”

What does univariant mean? Short-wavelength- Middle-wavelength- Long-wavelength- sensitive or “blue” sensitive or “green” sensitive or “red”

Use Middle‐wavelength‐sensitive (M) cones as an example…

UNIVARIANCE UNIVARIANCE Crucially, the effect of any absorbed photon is independent of its The effect of any absorbed photon is independent of its wavelength. wavelength.

M-cone M-cone

Once absorbed a photon produces the same change in So, if you monitor the cone output, you can’t tell which photoreceptor output whatever its wavelength. “colour” of photon has been absorbed.

Advanced Colour 13 Andrew Stockman

UNIVARIANCE UNIVARIANCE Crucially, the effect of any absorbed photon is independent of its wavelength. What does vary with wavelength is the probability that a photon will be absorbed.

M-cone

This is reflected in what is called a “spectral sensitivity function”. All the photoreceptor effectively does is to count photons.

Imagine the sensitivity to these photons…

0

-1 M-cone

-2 In order of M‐cone sensitivity: quantal sensitivity quantal 10 >>>> >> Log Changes in light intensity are confounded -3 with changes in colour (wavelength)

400 450 500 550 600 650 700 Wavelength (nm)

Advanced Colour 14 Andrew Stockman

UNIVARIANCE A change in photoreceptor output can be caused by a change in intensity or by a change in colour. There is no way of telling which. Univariance

If a cone is n times less sensitive to light A than to light B, then if A is set to be n times brighter than B, the two lights will appear identical whatever their wavelengths.

Each photoreceptor is therefore ‘colour blind’, and is unable to distinguish between changes in colour and changes in intensity.

If we had only one photoreceptor, we would be colour-blind…

People with normal colour vision have three univariant cones with different spectral sensitivities…

Examples: night vision, blue cone monochromats

Advanced Colour 15 Andrew Stockman

0

L -1 M Their colour vision is therefore three dimensional or: S

-2 TRICHROMATIC quantal sensitivity 10 Log -3

400 450 500 550 600 650 700 Wavelength (nm)

Trichromacy means our colour vision is actually limited So, if each photoreceptor is colour- blind, how do we see colour? We confuse many pairs of colours that are spectrally very different. Such pairs are known as metameric pairs.

Or to put it another way: Many of these confusions would be obvious to a being with 4 cone photoreceptors—just as the confusions of How is colour encoded at the colour deficient people are obvious to us. input to the ?

Advanced Colour 16 Andrew Stockman

Colour is encoded by the relative cone outputs

Blue light Red light Blue light Red light

If we want to be able to predict the relative responses of the three Green light Purple light Purple light Green light cones signals to lights of any spectral composition, we need to know the three cone spectral sensitivities… Yellow light White light Yellow light White light

In other words:

DETERMINING CONE SPECTRAL SENSITIVITIES We want to measure how the sensitivity of each cone type varies with wavelength.

How might we do that?

Advanced Colour 17 Andrew Stockman

Spectral sensitivity measurements Spectral sensitivity measurements

Flashing or flickering light Flashing or flickering light

Observer sets the threshold for Observer sets the threshold for detecting the flash or flicker as a detecting the flash or flicker as a Intensity Intensity function of the wavelength of the light. function of the wavelength of the light. Space (x) Space (x)

Spectral sensitivity measurements Spectral sensitivity measurements

Flashing or flickering light Flashing or flickering light

Observer sets the threshold for Observer sets the threshold for detecting the flash or flicker as a detecting the flash or flicker as a Intensity Intensity function of the wavelength of the light. function of the wavelength of the light. Space (x) Space (x)

Advanced Colour 18 Andrew Stockman

But the cone spectral sensitivities overlap throughout the spectrum.

S M L 0 M- and L-cone measurements

-1

-2 Use two special types of subjects: -3 Log sensitivity Deuteranopes -4 400 500 600 700 Protanopes Wavelength (nm)

Consequently, to measure them separately we have to use special subjects or special conditions.

Normal Protanope Normal Deuteranope

L M S L M S L M S L M S Protanopia Deuteranopia

Advanced Colour 19 Andrew Stockman

-8 Mean L‐cone spectral sensitivity function L-cone 0

L -9 L -1

Adjusted L- -10 quantal sensitivity cone data 10 Deuteranopes Log Macular and lens adjusted -2 -11 quantal sensitivity 10 400 450 500 550 600 650 700 Log 0.5 -3

0.0 difference 10 -0.5 400 450 500 550 600 650 700 Log 400 450 500 550 600 650 700 Wavelength (nm) Wavelength (nm)

Mean L‐ and M‐ cone spectral sensitivity functions 0 S-cone measurements L -1 M Two types of subjects: -2 quantal sensitivity

10 S‐cone (or blue cone) monochromats

Log Colour normals -3

400 450 500 550 600 650 700 Wavelength (nm)

Advanced Colour 20 Andrew Stockman

-6 Normal S‐cone monochromat

-8 S-cone data S -10

-12

-14

-16 quantal spectral sensitivity 10 -18 Normals

Log AS CF BCMs The Normal data were HJ FB -20 LS KS obtained on an intense orange TA PS adapting background that adapted (suppressed) the L- L M S L M S -22 400 450 500 550 600 and M-cones.

Wavelength (nm)

Mean spectral sensitivity functions 0 Why study spectral sensitivities?

L -1 M A knowledge of the spectral sensitivities of the cones is S important because it allows us to accurately and simply specify colours and to predict colour matches—for both -2 colour normal and colour deficient people (and to quantal sensitivity understand the variability between individuals). 10 These mean functions have enabled us to derive Log “standard” cone spectral Practical implications for colour printing, colour -3 sensitivity functions. reproduction and colour technology.

400 450 500 550 600 650 700 Wavelength (nm)

Advanced Colour 21 Andrew Stockman

Normal Tritanope Deuteranope

Credit: Euro Puppy Blog

Dogs are dichromats with only two cones peaking at 429 and 555 nm L M S L M S L M S Tritanopia

Normal Deuteranope

COLOUR VISION AND

MOLECULAR GENETICS Protanope

How do red-green colour vision deficiencies arise?

From Sharpe, Stockman, Jägle & Nathans, 1999

Advanced Colour 22 Andrew Stockman

Amino acid differences between photopigment opsins

Photopigment molecule From Sharpe, Stockman, Jägle & Nathans, 1999 From Sharpe, Stockman, Jägle & Nathans, 1999

Amino acid differences LWS all with OH between photopigment group POLAR opsins

MWS

NON-POLAR Why are the M‐ and L‐ cone opsins so similar? 180 277 285

Spectral shifts ~ 5 nm ~ 25 nm From Sharpe, Stockman, Jägle & Nathans, 1999

Advanced Colour 23 Andrew Stockman

Phylogenetic tree of visual pigments Mammals

About 460 – 520 nm Rh1 490 - 500 nm

About 460 – 520 nm Rh2 lost

About 420 – 480 nm SWS2 lost

About 355 – 450 nm SWS1 365 - 450 nm Basis for dichromatic colour vision

About 490 – 570 nm LWS 490 - 565 nm

Credit: Bowmaker

Humans

490 - 500 nm Gene duplication on the X-chromosome

lost Mammal lost

365 - 450 nm Human/ Old world Basis for primate trichromatic L-cone M-cone colour vision photopigment photopigment opsin gene opsin gene

Gene duplication Credit: Bowmaker

Advanced Colour 24 Andrew Stockman

Crossovers during meiosis are common:

Because these two genes are in a tandem array, and are very similar… Intergenic crossover

L-cone M-cone photopigment photopigment opsin gene opsin gene Intergenic crossovers produce more or less L and M- cone genes on each X chromosome

From Sharpe, Stockman, Jägle & Nathans, 1999

Intragenic crossovers produce hybrid or mixed L and M-cone genes 0 M Intergenic crossover -1

More M‐cone like -2 quantal sensitivity quantal 10

Intragenic crossover Log Hybrid (mixed) -3 L/M genes More L‐cone 400 450 500 550 600 650 700 like Wavelength (nm) Intragenic crossover The spectral sensitivities of the hybrid photo‐pigments vary between those of L the M‐ and L‐cones depending on where the crossover occurs. From Sharpe, Stockman, Jägle & Nathans, 1999

Advanced Colour 25 Andrew Stockman

Single-gene dichromats Multiple-gene dichromats

Protanope

With a single gene Male observers with two similar male observers must genes may also be effectively be dichromats dichromats if the two genes produce similar photopigments.

Deuteranope

Anomalous trichromats Male observers with two different genes are 1.0 1.0 “anomalous” trichromats Severe Normal Severe 0.5 Deutan 0.5 Protanomalous 0.0 0.0 350 450 550 650 750 350450550650750

Mild

1.0 1.0

Severe Mild Deuteranomalous 0.5 Protan 0.5 Protan Severe 0.0 0.0 350 450 550 650 750 350 450 550 650 750

Advanced Colour 26 Andrew Stockman

Main types of colour vision defects with approximate proportions of occurrence in the population. XY inheritance

percent in UK

Condition Male Female

Protanopia no L cone 1.0 0.02 Protanomaly milder form 1.0 0.03 Deuteranopia no M cone 1.5 0.01 Deuteranomaly milder form 5.0 0.4 Tritanopia no SWS cone 0.008 0.008

No red-green discrimination

The emergence of two longer wavelength (M‐ and L‐cones) is thought to have occurred relatively recently in primate evolution.

Why is it important?

Source: Hans Irtel

Advanced Colour 27 Andrew Stockman

Red-green discrimination

DIAGNOSING COLOUR VISION DEFICIENCIES

Source: Hans Irtel

Ishihara plates

Advanced Colour 28 Andrew Stockman

Tests measuring colour discrimination

Farnsworth-Munsell D-15 test

Farnsworth-Munsell D-15 Farnsworth-Munsell D-15

From: Ted Sharpe From: Ted Sharpe

Advanced Colour 29 Andrew Stockman

Mild Severe

Protan

D15 results

Deutan POSTRECEPTORAL COLOUR VISION Tritan

Rod monochromat

Credit: Jenny Birch

How is colour processed after the cones? The colour opponent theory of Hering

Advanced Colour 30 Andrew Stockman

Some ganglion cells are The colour opponent theory of Hering colour opponent

is opposed to R-G

is opposed to Y-B

RED On-centre How might this be related to visual GREEN Off-surround processing after the cones?

Red-green colour opponency Some ganglion cells are colour opponent

Four variants

GREEN On-centre RED Off-surround

Advanced Colour 31 Andrew Stockman

Summary Blue/yellow pathway

Trichromatic stage

Colour opponent stage

Source: David Heeger

So far, we’ve mainly been talking about the colours of isolated patches of light. But the colour of a patch depends also upon:

(i) What precedes it (in time) COLOUR AFTER-EFFECTS COLOUR AFTER-EFFECTS

(ii) What surrounds it (in space) (what precedes the patch) COLOUR CONTRAST COLOUR ASSIMILATION

Advanced Colour 32 Andrew Stockman

Colour after-effects

You don’t have to see things for them to produce an after-effect...

Beer & MacLeod

Advanced Colour 33 Andrew Stockman

Beer & MacLeod Mediafire

Lilac chaser or Pac-Man illusion

Jeremy Hinton

Advanced Colour 34 Andrew Stockman

Lilac chaser or Pac-Man illusion

COLOUR CONTRAST

(what surrounds the patch)

http://michaelbach.de/ot/index.html Michael Bach and Jeremy Hinton

Color contrast Color contrast

Advanced Colour 35 Andrew Stockman

Color contrast Color contrast

Color contrast

MacLeod

Advanced Colour 36 Andrew Stockman

COLOUR ASSIMILATION

MacLeod

Colour assimilation Colour assimilation

Advanced Colour 37 Andrew Stockman

Colour assimilation

Colour assimilation

Munker illusion

COLOUR CONSTANCY

Advanced Colour 38 Andrew Stockman

Credit: Gegenfurtner Colour constancy Colour constancy red green

g

blue yellow

Chromatic adaptation Colour and the illuminant and colour constancy

The change in colour appearance following adaptation is due to chromatic adaptation. Chromatic adaptation is adaptation to the colour is of the ambient illumination.

Amazing Art, Viperlib

Advanced Colour 39 Andrew Stockman

Colour and brightness

Monet, Claude The Regatta at Argenteuil c. 1872 19 x 29 1/2 in. (48 x 75 cm) Musee d'Orsay, Paris

S-CONE MEDIATED VISION IS UNUSUAL

Advanced Colour 40 Andrew Stockman

In other retinal regions, the S‐cone mosaic remains sparse. S-cones are unusual

S-cones form between 5 and 10% of the cone population.

Central area of the fovea lacks S- cones and is therefore tritanopic.

Curcio et al.

S‐cones make little or no contribution to luminance Primary S‐cone pathway is an ON pathway

S-cone ON bipolar L+M OFF bipolar

small blue-yellow bistratified ganglion cell From Rodieck (1998)

Advanced Colour 41 Andrew Stockman

LGN pathways S-cone vision is slow

Ganglion cell projections

Chromatic aberration

Why is S-cone vision sparse?

Base picture: Digital camera world

Advanced Colour 42 Andrew Stockman

Effect of chromatic blur on eye chart

COLOUR AND COGNITION

Jim Schwiegerling, U. Arizona

TEST 1 Stroop effect RED GREEN BLUE YELLOW Say to yourself the colours of the ink in which the following words are written -- as fast as you can. ORANGE BLUE GREEN BROWN WHITE

GREEN YELLOW PINK RED ORANGE So, for RED, say “red”.

But for RED, say “green” BROWN RED WHITE BLUE YELLOW

Ready, steady… WHITE ORANGE GREEN BROWN RED

How long?

Advanced Colour 43 Andrew Stockman

TEST 2

BLUE PINK WHITE RED BROWN

BROWN RED BLUE GREEN ORANGE

YELLOW BLUE RED ORANGE WHITE

BROWN RED GREEN WHITE RED

RED PINK BLUE GREEN WHITE

How long?

Advanced Colour 44