Saliency-Based Color Accessibility Opening the Umwelt

Saliency-based Color Accessibility Opening the Umwelt

Reference: Tajima, S. & Komine, K., IEEE Trans. Imag. Proc., 24(3):1115–1126, (2015) Background

Satohiro Tajima (“Sato”)

2007-09 U Tokyo (Masato Okada, Ikuya Murakami, …) Bayes models of vision 2009-13 NHK (Japan Broadcasting Corporation) Image processing, natural stats etc. 2013 PhD (Engineering), U Tokyo 2013-14 RIKEN Dynamical systems 2014- U Geneva (Alex Pouget) + Decision-making, consciousness

tajima - consciousness / embedding Background

Satohiro Tajima (“Sato”)

2007-09 U Tokyo (Masato Okada, Ikuya Murakami, …) 2009-13 NHK (Japan Broadcasting Corporation) 11.3.2011: Earthquake 9.0M 2013 PhD (Engineering), U Tokyo Tsunami Fukushima power plant accident 2013-14 RIKEN 2014- U Geneva (Alex Pouget)

tajima - consciousness / embedding What you see ≠ What others see.

Original image An actual evacuation map for the nuclear plant accident L M S Fukushima nuclear plant

Deuteranopia (*simulation)

L S Fukushima nuclear plant

Saliency-based color accessibility Color vison polymorphism

Color deficiency ~280 million (8% of male) Anomalous Dichromacy trichromacy L M S Common ● ● ● Male Female Male Female

1-1.3% 0.02% 1.3% 0.02% Protanope ∙ ● ●

1-1.2% 0.01% 5.0% 0.35% Deuteranope ● ∙ ●

0.001% 0.03% 0.01% 0.01% Tritanope ● ● ∙ Conventional simulation method Common Protanope

Tritanope Deuteranope

(Viénot et al., Nature, 1995) Saliency-based color accessibility Conventional simulation method Problems Common • No objective criteria for image quality • Simulated color distance  Perceptual difference?

Tritanope

(Viénot et al., Nature, 1995) (Brettel et al., JOSA A, 1997) Saliency-based color accessibility More fundamental problem Problems • No objective criteria for image quality • Simulated color distance  Perceptual difference? • Can we compare subjective perceptions between individuals having different sensory properties?

L M S L S

Saliency-based color accessibility More fundamental problem Problems • No objective criteria for image quality • Simulated color distance  Perceptual difference? • Can we compare subjective perceptions between individuals having different sensory properties?

L M S

Umwelt (self-centered world) J. J. B. von Uexküll

L M S L S

Umwelt (self-centered world)

Saliency-based color accessibility A (partial but practical) solution…

• Objective criteria We propose: • Distance in a perceptual space • Attentional cue as a common subspace of perception

“Saliency-based accessibility”

Saliency-based color accessibility Concept Salience: cue for bottom-up visual attention

(Itti & Koch, Nat. Rev. Neurosci., 2001) Saliency-based color accessibility Method Common vision

Uncommon vision (e.g., deuteranopia) Method Control Deuteranope

Difference: Lost information Model outputs

Saliency-based color accessibility Cf. Dichromatic vision can have advantages in the wild.

Animal Behaviour, 73:205-214 (2007)

Animal Behaviour 83:479-486 (2012) Experiment

• Participants: 5 color-deficient, 18 control observers • Task: 2AFC judgement of subjective salience “how easily the visual information is extracted from images”

W/ and w/o spatial context:

1) Original vs. Dichromat simulation 2) Original vs. Saliency-based recoloring

Time

Saliency-based color accessibility Concept of experiment Saliency-based recoloring (SBR)

Original image

Dichromat simulation (DS)

Saliency-based color accessibility Results

W/ context W/O context

Saliency-based color accessibility Results

W/ context W/O context

Saliency-based color accessibility Applications to TV program production

Original

Improved

Saliency-based color accessibility Summary New method to analyze proposed, color accessibility was experimentally evaluated (roughly), and applied to TV program productions.

What you see ≠ What others see But you can make: What you attend = What others attend

“Saliency-based accessibility”

L M S L S

Saliency-based color accessibility