Andrew Stockman

McCollough effect adapting pattern

Colour Vision

MSc Neuroscience course

Andrew Stockman LONG-TERM “CONTINGENT” ADAPTATION Akiyoshi Kitaoka

Light 400 - 700 nm is important for vision

INTRODUCTION

Lecture notes at http://www.cvrl.org

Click on “MSc Neuroscience” in left menu.

Colour Vision 1 Andrew Stockman

No colour…

How dependent are we on colour?

Colour…

But just how important is colour?

Colour Vision 2 Andrew Stockman

ACHROMATIC COMPONENTS CHROMATIC COMPONENTS

Split the image into...

CHROMATIC COMPONENTS

By itself chromatic information provides relatively limited information…

ACHROMATIC COMPONENTS How do we see colours?

An image of the world is projected by the cornea and lens onto the rear surface of the eye: the .

The back of the retina is carpeted by a layer of light-sensitive photoreceptors (rods and cones). Achromatic information important for fine detail …

Colour Vision 3 Andrew Stockman

Rods and Human photoreceptors cones Rods Cones . Achromatic . Daytime, achromatic night vision and chromatic vision . 1 type . 3 types

Rod Long‐wavelength‐ sensitive (L) or “red” cone

Middle‐wavelength‐ sensitive (M) or “” cone

Short‐wavelength‐ sensitive (S) or “blue” cone Webvision

Rod and cone systems are optimized for different light levels

Moonlight Sunlight Typical ambient light levels Starlight Indoor lighting

Photopic retinal illuminance -4.3 -2.4 -0.5 1.1 2.7 4.5 6.5 8.5 (log phot td)

Why do we have rods and cones? Scotopic retinal illuminance -3.9 -2.0 -0.1 1.5 3.1 4.9 6.9 8.9 (log scot td)

SCOTOPIC MESOPIC PHOTOPIC Visual function

Absolute rod Cone Rod saturation Damage threshold threshold begins possible

Rod system Cone system

A range of c. 106 A range of > 108

Colour Vision 4 Andrew Stockman

Rod and cone distribution Moonlight Sunlight Typical ambient light levels Starlight Indoor lighting

Photopic retinal illuminance -4.3 -2.4 -0.5 1.1 2.7 4.5 6.5 8.5 (log phot td)

Scotopic retinal illuminance -3.9 -2.0 -0.1 1.5 3.1 4.9 6.9 8.9 (log scot td)

SCOTOPIC MESOPIC PHOTOPIC Visual function

Absolute rod Cone Rod saturation Damage threshold threshold begins possible

Scotopic levels Mesopic levels Photopic levels (below cone threshold) where rod and cone (above rod saturation) where rod vision vision function where cone vision functions alone. together. functions alone. 0.3 mm of eccentricity is A range of c. 103 A range of c. 103 A range of > 106 about 1 deg of visual angle

Rod density peaks at about 20 deg eccentricity During the day, you have to look at Cones peak at the centre things directly to see them in detail of vision at 0 deg

At night, you have to look away from things to see them in more detail

Colour Vision 5 Andrew Stockman

Original photograph The human is a foveating system

Simulation of what we see when we fixate with cone vision.

Central fovea is rod-free, and the very central is rod- and S-cone free

Credit: Stuart Anstis, UCSD

Chromophore (chromo- color, + -phore, producer) Light-catching portion of any molecule

11-cis retinal. The molecule is twisted at the 11th carbon.

Photopigment molecule (cone) From Sharpe, Stockman, Jägle & Nathans, 1999

Colour Vision 6 Andrew Stockman

Chromophore Chromophore

A photon is absorbed A photon is absorbed

the energy of which initiates a conformational change to…

Chromophore Chromophore How can this process A photon is absorbed encode wavelength?

the energy of which initiates a conformational change to… 11-cis retinal.

all-trans retinal all-trans retinal. (side chains omitted for simplicity). Can it?

Colour Vision 7 Andrew Stockman

Four human photoreceptors with different spectral sensitivities 441 500 541 566 max (nm, corneal, quantal)

0 Vision at the photoreceptor stage is Note logarithmic scale relatively simple because the output -1

of each photoreceptor is: -2

UNIVARIANT -3 Rods quantal sensitivity quantal L 10 -4

Log S -5 M

400 450 500 550 600 650 700 We’ll come back to this… Wavelength (nm)

Colour… Colour isn’t just about physics. For example:

Two metamers of yellow

+

Is it mainly a property of physics or biology? though physically very different, can appear identical.

Colour Vision 8 Andrew Stockman

There are many other such metamers or matches…

Colour Vision 9 Andrew Stockman

Colour mixing

What can colour mixing tell us about colour vision?

Trichromacy means that colour vision is relatively simple.

Human vision is It is a 3 variable system… trichromatic

Colour Vision 10 Andrew Stockman

Colour TV The dots produced by a TV or projector are so small that they are mixed together by the eye and Trichromacy is exploited in colour reproduction, since thus appear as uniform patches of colour the myriad of colours perceived can be produced by mixing together small dots of three colours.

If you look closely at a colour television (or this projector output)…

3‐coloured dots 3‐coloured bars

Why is human vision trichromatic? The main reason is because just three cone photoreceptors are responsible for daytime colour vision.

Short‐wavelength‐ Middle‐wavelength‐ Long‐wavelength‐ sensitive or “blue” sensitive or “green” sensitive or “red”

But it is also depends upon the fact that...

Colour Vision 11 Andrew Stockman

UNIVARIANCE Crucially, the effect of any absorbed photon is independent The output of each photoreceptor is: of its wavelength.

“UNIVARIANT”

M-cone What does univariant mean?

Once absorbed a photon produces the same change Use Middle‐wavelength‐sensitive (M) cones as an example… in photoreceptor output whatever its wavelength.

UNIVARIANCE UNIVARIANCE Crucially, the effect of any absorbed photon is independent Crucially, the effect of any absorbed photon is independent of its wavelength. of its wavelength.

M-cone M-cone

So, if you monitor the cone output, you can’t tell All the photoreceptor effectively does is to count which “colour” of photon has been absorbed. photons.

Colour Vision 12 Andrew Stockman

M-cone spectral sensitivity function UNIVARIANCE High sensitivity (less photons needed to have an effect) 0 What does vary with wavelength is the probability that a photon will be absorbed. -1

-2 quantal sensitivity quantal

10 Low sensitivity (more photons needed Log to have an effect) -3 This is reflected in what is called a “spectral sensitivity function”. 400 450 500 550 600 650 700 Wavelength (nm)

Imagine the sensitivity to these photons…

0 0

-1 -1 If you have four lights of the same intensity (indicated here by their heights) -2 In order of M‐cone sensitivity: -2 quantal sensitivity quantal quantal sensitivity quantal

10 The green will look brightest, then 10 >>>> >> yellow, then blue and lastly the Log Log red will be the dimmest -3 -3

400 450 500 550 600 650 700 400 450 500 550 600 650 700 Wavelength (nm) Wavelength (nm)

Colour Vision 13 Andrew Stockman

0

-1 M-cone We can adjust the intensities to compensate for the sensitivity differences. -2 quantal sensitivity quantal

10 When this has been done, the four lights will look completely Log identical. Changes in light intensity are confounded with -3 changes in colour (wavelength)

400 450 500 550 600 650 700 Wavelength (nm)

UNIVARIANCE A change in photoreceptor output can be caused by a change in intensity or by a change in colour. There is no way of telling which.

Colour or intensity Univariance in suction electrode recordings change??

Each photoreceptor is therefore ‘colour blind’, and is unable to distinguish between changes in colour and changes in intensity.

Colour Vision 14 Andrew Stockman

If we had only one photoreceptor, we would be colour‐blind…

Univariance

If a cone is n times less sensitive to light A than to light B, then if A is set to be n times brighter than B, the two lights will appear identical whatever their wavelengths.

Examples: night vision, blue cone monochromats

With three cone photoreceptors, our colour vision is trichromatic…

So, if each photoreceptor is colour-blind, how do we see colour?

Or to put it another way: How is colour encoded?

Colour Vision 15 Andrew Stockman

Colour is encoded by the relative cone outputs Trichromacy actually means our colour vision is limited Blue light

0 We confuse many pairs of colours that are spectrally L very different. Such pairs are known as metameric -1 M pairs. S uantal sensitivity

q -2

Many of these confusions would be obvious to a 10

being with 4 cone photoreceptors—just as the Log

confusions of colour deficient people are obvious to -3 us. 400 450 500 550 600 650 700 Wavelength (nm)

Colour is encoded by the relative cone outputs Colour is encoded by the relative cone outputs

Blue light Blue light

0 0 L L Green light -1 M -1 M S S uantal sensitivity uantal sensitivity Red light q q -2 -2 10 10 Log Log

-3 -3 Red light

400 450 500 550 600 650 700 400 450 500 550 600 650 700 Wavelength (nm) Wavelength (nm)

Colour Vision 16 Andrew Stockman

Colour is encoded by the relative cone outputs

Blue light Red light

DETERMINING CONE SPECTRAL SENSITIVITIES Green light Purple light

Yellow light White light

In other words…

The cone spectral sensitivities overlap S M L extensively throughout 0 the spectrum. -1

-2 Consequently, we have How can we measure how the sensitivity of each cone -3 to use special subjects Log sensitivity type varies with wavelength (or spectral colour)? or special conditions to -4 be able isolate the the 400 500 600 700 response of a single cone type. Wavelength (nm)

Colour Vision 17 Andrew Stockman

Normal Protanope

M- and L-cone measurements

Use two special types of subjects:

Deuteranopes Protanopes

L M S L M S Protanopia

Normal Deuteranope -8 L-cone

L -9

(Adjusted) L‐cone -10 quantal sensitivity

data 10 Log Macular and lens adjusted -11

400 450 500 550 600 650 700

0.5

0.0 difference 10 L M S L M S -0.5 Log Deuteranopia 400 450 500 550 600 650 700 Wavelength (nm)

Colour Vision 18 Andrew Stockman

Mean L‐ and M‐cone spectral sensitivity functions 0 S-cone measurements L -1 M

Two types of subjects: -2 quantal sensitivity

10 S‐cone (or blue cone) monochromats Colour normals Log -3

400 450 500 550 600 650 700 Wavelength (nm)

-6 Normal S‐cone monochromat

-8 S-cone data S -10

-12

-14

-16 The Normal data were quantal spectral sensitivity

10 obtained on an intense orange -18 Normals adapting background that was Log AS there to suppress the L- and CF BCMs HJ FB M-cone sensitivities. -20 LS KS TA PS

L M S L M S -22 400 450 500 550 600

Wavelength (nm)

Colour Vision 19 Andrew Stockman

Mean spectral sensitivity functions 0 Why study spectral sensitivities?

L -1 M A knowledge of the spectral sensitivities of the cones is S important because it allows us to accurately and simply specify colours and to predict colour matches—for both -2 colour normal and colour deficient people (and to quantal sensitivity

10 These mean functions understand the variability between individuals). have enabled us to derive Log “standard” cone spectral Practical implications for colour printing, colour -3 sensitivity functions. reproduction and colour technology.

400 450 500 550 600 650 700 Wavelength (nm)

Normal Tritanope Deuteranope

Credit: Euro Puppy Blog

Dogs are dichromats with only two cones peaking at L M S L M S 429 and 555 nm L M S Tritanopia

Colour Vision 20 Andrew Stockman

But what happens next (i.e., how is colour encoded after the photoreceptors)?

POSTRECEPTORAL COLOUR VISION

Colour phenomenology The colour opponent theory of Hering Can provide clues about how colours are processed after the photoreceptors…

Which pairs of colours coexist in a single, uniform patch of colour? Which pairs never coexist?

Reddish‐? Reddish‐yellows? Bluish‐reds? Bluish‐yellow? ?? Reds can get bluer or yellower but not greener

WHY?

Colour Vision 21 Andrew Stockman

The colour opponent theory of Hering The colour opponent theory of Hering

Yellows can get greener or redder but not bluer Greens can get bluer or yellower but not redder

The colour opponent theory of Hering

The colour opponent theory of Hering

Blues can get greener or redder but not yellower

Colour Vision 22 Andrew Stockman

Some ganglion cells are The colour opponent theory of Hering colour opponent

is opposed to R-G

is opposed to Y-B

Imagine that this is the region of space that the cell “sees” in How might this be related to visual the external world processing after the cones?

Some ganglion cells are Some ganglion cells are colour opponent colour opponent

A red light falling on the central area A green light falling on the surround area excites the cell (makes it fire faster) inhibits the cell (makes it fire slower)

Colour Vision 23 Andrew Stockman

Red-green colour opponency Blue/yellow pathway

Four variants

Source: David Heeger

LGN cell responses

LESS COMMON

Colour Vision 24 Andrew Stockman

Summary So that’s colour (chromatic) ACHROMATIC COMPONENTS encoding, but what about “luminance” (achromatic) encoding? Trichromatic stage

CHROMATIC COMPONENTS Colour opponent stage

In addition to neural pathways that signal colour there are also pathways that signal intensity or luminance:

Chromatic and achromatic pathways

Colour Vision 25 Andrew Stockman

Luminance is encoded by summing the L‐ and M‐cone signals:

Blue light Red light

L+M + + Colour is in many ways secondary Green light Purple light to luminance L+M + +

Yellow light White light

L+M + +

+ +

Colour Vision 26 Andrew Stockman

+ +

+ +

Colour Vision 27 Andrew Stockman

Boynton illusion

Rob van Lier, Mark Vergeer & Stuart Anstis

Boynton Boynton illusion illusion

Colour Vision 28 Andrew Stockman

Boynton Fading dot illusion

Watercolour effect

What are the postreceptoral neural substrates of the chromatic and luminance pathways?

Colour Vision 29 Andrew Stockman

Magnocellular

Luminance pathways, which produce Diffuse bipolar diffuse cells sum across achromatic (colourless) percepts, have bipolar M‐ and L‐cones been linked to the magnocellular stream.

parasol ganglion cells From Rodieck (1998)

Koniocellular

Chromatic pathways, which produce diffuse bipolar diffuse chromatic percepts, have been linked to the S-cone bipolar koniocellular stream for S‐(L+M)… bipolar

blue-yellow bistratified ganglion cell From Rodieck (1998)

Colour Vision 30 Andrew Stockman

Parvocellular

Chromatic pathways, which produce chromatic percepts, have been linked to the koniocellular stream for S‐(L+M)... In the fovea, midget bipolar cells have single M‐ and L‐cone midget bipolars inputs.

And also to the parvocellular retinal stream for L‐M.

midget ganglion cells From Rodieck (1998)

[Based on Boynton (1979)]

Although luminance pathways have been linked to the magnocellular stream, they must also depend on the parvocellular stream.

Parvocellular Koniocellular Magnocellular pathway pathway pathway

Colour Vision 31 Andrew Stockman

Austin Roorda, 2004 The parvocellular pathway must be double‐duty supporting finely detailed luminance vision as well as much more coarse colour vision.

To be able to resolve this E at the resolution limit, there must be enough samples.

The parvocellular pathway, with its one‐to‐one cone to bipolar connections, provides enough samples.

The magnocellular pathway, with diffuse bipolar cells, does

not. Austin Roorda, 2004

Magnocellular pathway: Parvocellular pathway: High temporal frequencies (motion) High spatial frequencies (detail) Low spatial frequencies Low temporal frequencies Achromatic Chromatic Multiplexing colour and Higher contrast sensitivity Lower contrast sensitivity luminance information

From Rodieck (1998)

Colour Vision 32 Andrew Stockman

So far, we’ve mainly been talking about the colours of isolated patches of light. But the colour of a patch depends also upon:

(i) What precedes it (in time) COLOUR AFTER-EFFECTS COLOUR AFTER-EFFECTS

(ii) What surrounds it (in space) (what precedes the patch) COLOUR CONTRAST COLOUR ASSIMILATION

Colour after-effects

Colour Vision 33 Andrew Stockman

You don’t have to see things for them to produce an after‐effect...

Beer & MacLeod

Beer & MacLeod Mediafire

Colour Vision 34 Andrew Stockman

Lilac chaser or Pac-Man illusion

Jeremy Hinton

Color contrast

COLOUR CONTRAST

(what surrounds the patch)

Colour Vision 35 Andrew Stockman

COLOUR ASSIMILATION

Colour assimilation

Colour Vision 36 Andrew Stockman

Munker illusion

COLOUR CONSTANCY

Credit: Gegenfurtner Colour constancy Colour constancy red green

g

blue yellow

Colour Vision 37 Andrew Stockman

Chromatic adaptation Colour and the illuminant and colour constancy

The change in colour appearance following adaptation is due to chromatic adaptation. Chromatic adaptation is adaptation to the colour is of the ambient illumination.

Amazing Art, Viperlib

Colour and brightness

Monet, Claude COLOUR AND The Regatta at Argenteuil c. 1872 19 x 29 1/2 in. (48 x 75 cm) COGNITION Musee d'Orsay, Paris

Colour Vision 38 Andrew Stockman

TEST 1 Stroop effect RED GREEN BLUE YELLOW Say to yourself the colours of the ink in which the following words are written -- as fast as you can. ORANGE BLUE GREEN BROWN WHITE

GREEN YELLOW PINK RED ORANGE So, for RED, say “red”.

But for RED, say “green” BROWN RED WHITE BLUE YELLOW

Ready, steady… WHITE ORANGE GREEN BROWN RED

How long?

TEST 2 McCollough effect test pattern

BLUE PINK WHITE RED BROWN

BROWN RED BLUE GREEN ORANGE

YELLOW BLUE RED ORANGE WHITE

BROWN RED GREEN WHITE RED

RED PINK BLUE GREEN WHITE

How long? Version by Akiyoshi Kitaoka

Colour Vision 39 Andrew Stockman

McCollough effect adapting pattern McCollough effect test pattern

LONG-TERM “CONTINGENT” ADAPTATION Version by Akiyoshi Kitaoka Akiyoshi Kitaoka

Tritanope

COLOUR VISION, COLOUR DEFICIENCIES AND MOLECULAR GENETICS

Colour Vision 40 Andrew Stockman

Appearance of visible spectrum

Colour deficiency simulations From Sharpe, Stockman, Jägle & Nathans, 1999

Normal Deuteranope

COLOUR VISION AND MOLECULAR GENETICS Tritanope Protanope

From Sharpe, Stockman, Jägle & Nathans, 1999

Colour Vision 41 Andrew Stockman

The spectral sensitivity of the photopigment depends on the energy required to flip the chromophore from it 11‐ cis form to its all‐trans form.

But the same chromophore is used in all four human photo‐ pigments.

The energy is modified by changes in the amino acids in the surrounding opsin molecule.

Photopigment molecule Photopigment molecule (cone) From Sharpe, Stockman, Jägle & Nathans, 1999 From Sharpe, Stockman, Jägle & Nathans, 1999

Amino acid differences between photopigment opsins LWS all with OH group POLAR

MWS NON-POLAR

180 277 285

From Sharpe, Stockman, Jägle & Nathans, 1999

Colour Vision 42 Andrew Stockman

MWS LWS Amino acid Tuning Site differences between 180 alanine serine OH- ~ 5 nm photopigment opsins

277 phenylalanine tyrosine OH-

~ 25 nm Why are the M‐ and L‐ cone opsins so similar? 285 alanine threonine OH-

From Sharpe, Stockman, Jägle & Nathans, 1999

Phylogenetic tree of visual pigments Mammals

About 460 – 520 nm Rh1 490 - 500 nm

About 460 – 520 nm Rh2 lost

About 420 – 480 nm SWS2 lost

About 355 – 450 nm SWS1 365 - 450 nm Basis for dichromatic colour vision

About 490 – 570 nm LWS 490 - 565 nm

mya (million years ago) mya (million years ago) Credit: Bowmaker

Colour Vision 43 Andrew Stockman

Humans

490 - 500 nm Gene duplication on the X-chromosome

lost Mammal lost

365 - 450 nm Human/ Old world Basis for primate trichromatic L-cone M-cone colour vision photopigment photopigment opsin gene opsin gene

Duplication mya (million years ago) Credit: Bowmaker

Crossovers during meiosis are common: Because these two genes are in a tandem array, and are very similar… Intergenic crossover

L-cone M-cone photopigment photopigment opsin gene opsin gene Intergenic crossovers produce more or less numbers of L and M-cone genes on each X chromosome

From Sharpe, Stockman, Jägle & Nathans, 1999

Colour Vision 44 Andrew Stockman

Intragenic crossovers Intragenic crossovers produce hybrid or mixed L and M-cone genes 0 M Intergenic crossover -1

-2 quantal sensitivity quantal 10

Intragenic crossover Log Hybrid (mixed) -3 L/M genes 400 450 500 550 600 650 700 Wavelength (nm) Intragenic crossover The spectral sensitivities of those hybrid L photopigments vary between those of the M- and L-cones depending on where the

From Sharpe, Stockman, Jägle & Nathans, 1999 crossover occurs.

M Single-gene dichromats

Protanope More M-cone like With just one gene a male (with only one X- chromosome) must be a dichromat More L-cone like Deuteranope L

Colour Vision 45 Andrew Stockman

Deuteranope Protanope

Males with two different Multiple-gene dichromats Anomalous trichromats genes are anomalous trichromats

Severe Protanomalous Males with two very similar genes may also be effectively (or behaviourally) dichromats, simply Mild because the spectral sensitivities of the two cone types are so similar. Deuteranomalous

Severe

Colour Vision 46 Andrew Stockman

Main types of colour vision defects with approximate 1.0 1.0 proportions of appearance in the population. Severe Normal 0.5 Deutan 0.5 percent in UK

0.0 0.0 350 450 550 650 750 350450550650750 Condition Male Female

Protanopia no L cone 1.0 0.02 Protanomaly milder form 1.0 0.03 1.0 1.0 Deuteranopia no M cone 1.5 0.01 Deuteranomaly milder form 5.0 0.4 Severe Mild 0.5 Protan 0.5 Protan Tritanopia no SWS cone 0.008 0.008

0.0 0.0 350 450 550 650 750 350450550650750

XY inheritance The emergence of two longer wave‐ length (M‐ and L‐cones) is thought to have occurred relatively recently in primate evolution.

Why is it important?

Colour Vision 47 Andrew Stockman

No red-green discrimination Red-green discrimination

Source: Hans Irtel Source: Hans Irtel

Ishihara plates

DIAGNOSING COLOUR VISION DEFICIENCIES

Colour Vision 48 Andrew Stockman

S-cones are unusual

S-CONE MEDIATED VISION IS UNUSUAL

Central area of the fovea lacks S‐ cones and is therefore tritanopic.

Colour Vision 49 Andrew Stockman

In other retinal regions, the S-cone mosaic remains sparse. S‐cone pathway ON pathway

S-cones form between 5 and 10% of the cone population.

Curcio et al.

Chromatic aberration

Why is S-cone vision sparse?

Base picture: Digital camera world

Colour Vision 50 Andrew Stockman

Effect of chromatic blur on eye chart McCollough effect test pattern

Version by Akiyoshi Kitaoka Jim Schwiegerling, U. Arizona

Colour Vision 51