Saliency-based Color Accessibility Opening the Umwelt Satohiro Tajima [email protected] Reference: Tajima, S. & Komine, K., IEEE Trans. Imag. Proc., 24(3):1115–1126, (2015) Background Satohiro Tajima (“Sato”) 2007-09 U Tokyo (Masato Okada, Ikuya Murakami, …) Bayes models of vision 2009-13 NHK (Japan Broadcasting Corporation) Image processing, natural stats etc. 2013 PhD (Engineering), U Tokyo 2013-14 RIKEN Dynamical systems 2014- U Geneva (Alex Pouget) + Decision-making, consciousness tajima - consciousness / embedding Background Satohiro Tajima (“Sato”) 2007-09 U Tokyo (Masato Okada, Ikuya Murakami, …) 2009-13 NHK (Japan Broadcasting Corporation) 11.3.2011: Earthquake 9.0M 2013 PhD (Engineering), U Tokyo Tsunami Fukushima power plant accident 2013-14 RIKEN 2014- U Geneva (Alex Pouget) tajima - consciousness / embedding What you see ≠ What others see. Original image An actual evacuation map for the nuclear plant accident L M S Fukushima nuclear plant Deuteranopia (*simulation) L S Fukushima nuclear plant Saliency-based color accessibility Color vison polymorphism Color deficiency ~280 million (8% of male) Anomalous Dichromacy trichromacy L M S Common ● ● ● Male Female Male Female 1-1.3% 0.02% 1.3% 0.02% Protanope ∙ ● ● 1-1.2% 0.01% 5.0% 0.35% Deuteranope ● ∙ ● 0.001% 0.03% 0.01% 0.01% Tritanope ● ● ∙ Conventional simulation method Common Protanope Tritanope Deuteranope (Viénot et al., Nature, 1995) Saliency-based color accessibility Conventional simulation method Problems Common • No objective criteria for image quality • Simulated color distance Perceptual difference? Tritanope (Viénot et al., Nature, 1995) (Brettel et al., JOSA A, 1997) Saliency-based color accessibility More fundamental problem Problems • No objective criteria for image quality • Simulated color distance Perceptual difference? • Can we compare subjective perceptions between individuals having different sensory properties? L M S L S Saliency-based color accessibility More fundamental problem Problems • No objective criteria for image quality • Simulated color distance Perceptual difference? • Can we compare subjective perceptions between individuals having different sensory properties? L M S Umwelt (self-centered world) J. J. B. von Uexküll Saliency-based color accessibility More fundamental problem Problems • No objective criteria for image quality • Simulated color distance Perceptual difference? • Can we compare subjective perceptions between individuals having different sensory properties? L M S L S Umwelt (self-centered world) Saliency-based color accessibility A (partial but practical) solution… • Objective criteria We propose: • Distance in a perceptual space • Attentional cue as a common subspace of perception “Saliency-based accessibility” Saliency-based color accessibility Concept Salience: cue for bottom-up visual attention (Itti & Koch, Nat. Rev. Neurosci., 2001) Saliency-based color accessibility Method Common vision Uncommon vision (e.g., deuteranopia) Method Control Deuteranope Difference: Lost information Model outputs Saliency-based color accessibility Cf. Dichromatic vision can have advantages in the wild. Animal Behaviour, 73:205-214 (2007) Animal Behaviour 83:479-486 (2012) Experiment • Participants: 5 color-deficient, 18 control observers • Task: 2AFC judgement of subjective salience “how easily the visual information is extracted from images” W/ and w/o spatial context: 1) Original vs. Dichromat simulation 2) Original vs. Saliency-based recoloring Time Saliency-based color accessibility Concept of experiment Saliency-based recoloring (SBR) Original image Dichromat simulation (DS) Saliency-based color accessibility Results Saliency-based color accessibility Results Saliency-based color accessibility Results W/ context W/O context Saliency-based color accessibility Results W/ context W/O context Saliency-based color accessibility Applications to TV program production Original Improved Saliency-based color accessibility Summary New method to analyze proposed, color accessibility was experimentally evaluated (roughly), and applied to TV program productions. What you see ≠ What others see But you can make: What you attend = What others attend “Saliency-based accessibility” L M S L S Saliency-based color accessibility.