Chapter 5 - Encodings & Design Principles

Raghu Machiraju

Slides: Courtesy Profs. Pﬁster, Moeller, und Weiskopf Outline

- Role of human perception - relative vs. absolute judgements. - Theory of marks and visual channels, - Principles of expressiveness and effectiveness - accuracy - discriminability - separability - propensity for visual popout

- Rankings of channels according to data type Judging ? Read M. Livingstone, “Vision and Art” Human Visual System

• Massive parallel processing • Left / Right • Shape / Color / Motion • Luminance: Depth and motion • Color: Form and function Weber’s Law

• Just Noticeable Difference (JND) I S = k I • We can detect 0.5% change in brightness independent of the overall illumination • Only applies to small patches Weber’s Law

Most continuous variations are perceived as discrete steps

Based on slide from Agrawala Based on slide from Mazur Based on slide from Mazur

Luminance Constancy

• Light levels can vary by six orders of magnitude • Our visual system deals with that by responding to relative differences • Can cause errors in simple graphical environments • In the real world we make use of additional information Simultaneous Contrast

Gray patch on dark looks brighter than gray patch on light background Contrast Crispening

Differences are perceived large when samples are similar to the background color

Color Processing

Read Color Contrast

Colors are also perceived relative to their overall context Color Size Illusion

Cleveland & McGill, “A Color-Caused Optical Illusion on a Statistical Graph”, 1983 Which Area Bigger? Cleveland & McGill, “A Color-Caused Optical Illusion on a Statistical Graph”, 1983 Marks Channels Marks and Channels Channel Types

Provide information on • What something is information • circle, square ? • categorical attributes • How much information • quantify/measure: size, etc. ? • ordered attributes - ordinal and quantitative Mark Types

• Mark is an item • Connection marks • Containment marks (areas) Expressiveness - visual encoding should express all of, and only, the information in the dataset attributes. - E.g. - Our perceptual system intrinsically senses as ordered if data is indeed ordered - Conversely too Effectiveness

- Importance of the attribute should match salience of channel

- Most important attributes encoded with the most effective channels

- Effectiveness has meaning in context of visual encoding Channel Rankings

Univariate Data Representations

http://www.smartmoney.com/marketmap/

Based on slide from Stasko Mapping to Positional Quantities • Mapping to positional quantities: 1.000 Position • 0.775 • Size 0.550 0.325 Orientation • 0.100 0.05 0.288 0.525 0.763 1 • Geometric mapping • Typically, very effective visual parameters

32 Bivariate Date Representations

Based on slide from Stasko Mapping to Positional Quantities • Point diagrams • E.g. scatter plots: visual recognition of correlations

34 Mapping to Positional Quantities • Line and curve diagrams: • Effective perception of differences in • Position and • Length 10000

8750

7500

6250

5000 35 0 7.5 15 22.5 30 Mapping to Positional Quantities • Bar graph: • Discrete independent variable (domain): nominal/ ordinal/quantitative Quantitative dependent variable (data) • 10000

8750

7500

6250

5000 36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Trivariate Data Representations

Based on slide from Stasko Alternatives

Based on slide from Stasko Channel Effectiveness

• Accuracy • Discriminability • Separability • Popout Effectiveness - Accuracy Read Effectiveness of Mappings

[Kandel et al.: Principles of neural science] 42

Steven’s Power Law

S = kIp

From Wilkinson 99, based on Stevens 61 Steven’s Power Law Sensation Exponent Brightness 0.33 - 0.5 Smell 0.55 (Coffee) - 0.6 (Heptane) Loudness 0.6

Underestimating Vibration 0.6 (250 Hz) - 0.95 (60 Hz) Taste 0.6 (Saccharine) - 1.3 (Salt) Temperature 1.0 (Cold) - 1.6 (Warm) Duration 1.1 Pressure 1.1 Heaviness 1.45

Overestimating Electric Shock 3.5

Stevens 61, “Psychophysics of Sensory Functions” Area Judgements

• Follow Steven’s power law S = kAp • Stevens: p = 0.7 • Flannery: p = 0.87, k = 0.98 • Cleveland: p = 1.0

Cleveland et. al, “Judgments of Circle Sizes on Statistical Maps”, 1982 Accuracy of Channels • Directly map human response to visually encoded information • Explicit rankings of perceptual accuracy for each channel type • Position along common scale is most accurately perceived • Position along nonaligned scales, • Length and direction and angle judgements without scale • Area judgements are notably less accurate • Volume and curvature judgements worse yet • Greyscale shading and color saturation were the least accurate

Discriminability

Channel limited by number of bins for use Bins in a Channel

Large line width makes appear polygon Best when # values to encode is small Separability

+ Separable channel pair best for two different data attributes + Position and color hue is a good choice + For single data attribute with three categories - + Integral channel pair of horizontal and vertical size is a reasonable choice Popout Popout Popout

• All support popout individually • A few channel pairs support popout • Popout not possible with >3 channels • Popout depends on channel and how different target is from surrounding Channel Characteristics

• Planar Position • Color • Size • Tilt/Angle • Shape Planar Position Planar Position

• Attributes encoded with position will dominate users mental model

• Vertical and horizontal position are combined into planar • Differences between up-down and side-to-side axes are relatively subtle

• Perceive height differences along up-down axis more than horizontal differences • Why ? Physical effects of gravity

•Information density considerations sometimes override this concern •Why ? Aspect ratio gives horizontal salience Visual Metaphors

Semiology of Graphics [J. Bertin, 83] Color/Luminance Processing Human Visual System

• Massive parallel processing • Left / Right • Shape / Color / Motion • Luminance: Depth and motion • Color: Form and function Newton Cartoon Spectra

Red Cyan

400 500 600 700 nm 400 500 600 700 nm

Green Magenta

400 500 600 700 nm 400 500 600 700 nm Blue Yellow

400 500 600 700 nm 400 500 600 700 nm Additive Color Mixing

• Colors combine by adding the spectra • Examples: Monitors, projectors, lights, etc.

Red

+ 400 500 600 700 nm

Green = Yellow

400 500 600 700 nm 400 500 600 700 nm Subtractive Color Mixing

• Colors combine by multiplying the spectra • Examples: Crayons, paint, most ﬁlms, ﬁlters

Cyan (“blue” in crayons)

* 400 500 600 700 nm

Yellow = Green

400 500 600 700 nm 400 500 600 700 nm 1 RefractiveP ower = Focal distance F ocalDistance(m) Wandell, Foundations of Vision Bear, Connors, Paradiso, “Neuroscience” Bear, Connors, Paradiso, “Neuroscience” M. Livingstone, “Vision and Art” Center-Surround Organization Color-Opponent Cells

Bear, Connors, Paradiso, “Neuroscience” Color Opponency

Ware, “Information Visualization” Color Blindness

M. Livingstone, “Vision and Art” Recap

• Massive parallel processing • Left / Right - depth • Center / surround - light / dark • P / M - detail / low-resolution vision • P / NonM-NonP - red-green / blue-yellow • Luminance: Depth and motion • Color: Form and function Further Reading Luminance conveys high-resolution edge information, while the red-green and blue-yellow channels are lower resolution. Color Systems • Our perception registers: – Hue – Saturation – Lightness or brightness • Artists often specify colours in terms of – Tint – Shade – Tone

• Given spectral power distribution S(λ)

• Given S1(λ) , S2(λ), if the X, Y, and Z responses are same then they are metamers wrt to the sensor • Used to show that three sensor types are same

Basis for industrial color standards and “pointwise” color models

• Three pure light sources: R = 700 nm, G = 546 nm, B = 436 nm.

• r, g, b can be negative

• 3 hypothetical light sources which yield positive matching curves

• Use linear combinations of real lights –R, G-2R,B+R • One of the lights is grey and has no hue • Two of the lights have zero luminance and provide hue

• Gives X, Y, Z color values • Y corresponds to achromatic (no color) channel • Chromaticity values: • x=X/(X+Y+Z); y=Y/(X+Y+Z) • Typically use x,y,Y

X x = X + Y + Z Y y = X + Y + Z

€

• When 2 colors are added together, the new color lies along the straight line between the original colors – E.g. A is mixture of B (spectrally pure) and C (white light) – B - dominant wavelength – AC/BC (as a percentage) is excitation purity of A – The closer A is to C, the whiter and less pure it is.

• D and E are complementary colors • can be mixed to produce white light • color F is a mix of G and C • F is non-spectral its dominant wavelength is the complement of B

• area of colors that a physical device can represent • hence - some colors can't be represented on an RGB screen

• Just a change of basis • Need detailed monitor information to do this right – Used in high quality settings (movie industry, lighting design, publishing) • Normalized (lazy) way: – (1,1,1) in RGB <-> (1,1,1) in XYZ – matrices exist ⎡ X⎤ ⎡0 .5149 .3244 .1607⎤⎡ R⎤ ⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢Y ⎥ = ⎢0 .2654 .6704 .0642⎥⎢ G ⎥ ⎣⎢ Z⎦⎥ ⎣⎢0 .0248 .1248 .8504⎦⎥⎣⎢ B⎦⎥ © Machiraju/Möller

€ The RGB Cube

• RGB color space is perceptually non-linear • Dealing with > 1.0 and < 0 ! • RGB space is a subset of the colors human can perceive • Con: what is ‘bloody red’ in RGB?

• CMY(K) – used in printing • LMS – sensor response • HSV – popular for artists • Lab, UVW, YUV, YCrCb, Luv, • Opponent color space – relates to brain input: – R+G+B(achromatic); R+G-B(yellow-blue); R-G(red-green) • All can be converted to/from each other – There are whole reference books on the subject

• Cyan, Magenta, Yellow (Black) – CMY(K) • A subtractive color model dye color absorbs reﬂects Cyan red blue and green Magenta green blue and red yellow blue red and green Black all none

Converting between RGB and CMY

⎡ C ⎤ ⎡1 ⎤ ⎡ R⎤ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ M⎥ = ⎢1 ⎥ −⎢G ⎥ ⎣⎢ Y ⎦⎥ ⎣⎢1 ⎦⎥ ⎣⎢ B⎦⎥

⎡ C ⎤ ⎡max( R,G,B)⎤ ⎡ R ⎤ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ M max(R,G,B) G ⎢ ⎥ =€⎢ ⎥ −⎢ ⎥ ⎢ Y ⎥ ⎢max( R,G,B)⎥ ⎢ B ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ K ⎦ ⎣ 1 ⎦ ⎣max( R,G,B)⎦ © Machiraju/Möller

€ RGB and CMY

• Based on polar coordinates, not Cartesian coordinates. • HSV is a non-linearly transformed (skewed) version of RGB cube – Hue: quantity that distinguishes colour family, say red from yellow, green from blue – Saturation (Chroma): colour intensity (strong to weak). Intensity of distinctive hue, or degree of colour sensation from that of white or grey – Value (luminance): light colour or dark colour

• S = sensation / brightness • I = intensity Problem - HSV

• Not perceptual • L* a* b*

• A color model for which, a unit change in luminance and chrominance are uniformly perceptible • U = 13 W* (u - uo ); V = 13 W* (v - vo); W = 25 ( 100 Y ) 1/3 - 17 • where Y , u and v can be calculated from : • X = O.607 Rn + 0.174 Gn + 0.200Bn • Y = 0.299 Rn + 0.587 Gn + 0.114Bn • Z = 0.066 Gn + 1.116 Bn • x = X / ( X + Y + Z ) • y = Y / ( X + Y + Z ) • z = Z / ( X + Y + Z ) • u = 4x / ( -2x + 12y + 3 ) • v = 6y / ( -2x + 12y + 3 )

• Lab is most linear along all three axis

• Luv is derived from UVW and Lab, with all components guaranteed to be positive

• Initially, for PAL analog video, it is now also used in CCIR 601 standard for digital video • Y (luminance) is the CIE Y primary. Y = 0.299R + 0.587G + 0.114B • It can be represented by U and V -- the color differences. U = B – Y; V = R - Y • YCrCb is a scaled and shifted version of YUV and used in JPEG and MPEG (all components are positive) Cb = (B - Y) / 1.772 + 0.5; Cr = (R - Y) / 1.402 + 0.5

• Issues: • What kind of data can be color-coded? • What kind of information can be efﬁciently visualized? • How do you reason about channels for color ? Mapping to Color • Areas of application • Provide information coding • Designate or emphasize a speciﬁc target in a crowded display • Provide a sense of realism or virtual realism • Provide warning signals or signify low probability events • Group, categorize, and chunk information • Convey emotional content • Provide an aesthetically pleasing display Luminance Summary

• How much channel ! • Contrast is the only to encode ﬁne detail • Hue, saturation do not • Using grey-scale encoding is a waste of perceptual resources • Contrast effects reduce accuracy - six discernible steps • Maximize contrast with background if outlines of shapes are important • Use medium-grey background if subtle graduation in grey scales are important . Get it right in black and white. Equi-luminous Colors

M. Livingstone, “Vision and Art”

Saturation Channel

• How much channel ! • Low accuracy • Interferes with size channel • For large regions, use low-saturation in pastel range - turquoise, lavender, primrose yellow, baby pink, baby blue, jade green, peach, apricot, salmon pink are all pastels. • For small regions use bright, highly saturated colors • Saturation and Hue not separable Tufte, VDQI (Vol. 1), p. 77 Tufte, VDQI (Vol. 1), p. 76 Hue Channel

• What channel ! • High ranked after spatial channel • Works well for contiguous regions • Not for separate regions • Categorical Color - 6-12 bins Comparative Genomics: Synteny between whole genomes. Figure shows the synteny blocks for human and mouse genomes. The genome shown in the right panel (mouse in this case) is the source genome and the genome on the left (human) is the target genome. All chromosomes of the source genome are shown in unique colors. Each chromosome of the target genome is shown as composed of segments of some chromosome of the source genome, as indicated by the corresponding color. For example, the majority of human chromosome 1 is composed of (i.e. is syntenic to) mouse chromosomes 4, 3 and 1. Information in color and value InformationValueInformation is Informationperceived in as color ordered andinand Color valuevalue ∴ Encode ordinal variables (O) ValueValue isis perceivedperceived as ordered ∴ Encode ordinal variables (O) ∴ •Encode(Grey)Value ordinal is variables perceived (O) as ordered (O) ∴ Encode continuous variables (Q) [not as well]

∴∴•EncodeEncodeCan encode continuouscontinuous quantitative variablesvariables values (Q)(Q) (Q) [not[not [not asas well] well]as well] Hue is normally perceived as unordered ∴•EncodeHue isnominal normally variables perceived (N) as using unordered color (N) HueHue isis normallynormally perceived asas unorderedunordered Encode nominal variables (N) using color ∴∴ Encode nominal variables (N) using color

Based on slide from Agrawala

Visual variables Visual variables Visual! Position variables ! Size ! Value ! TexturePosition !! PositionColorSize !! SizeOrientationValue !! ValueShapeTexture !! TextureColor !! ColorOrientation !! OrientationShape ! Shape

Note: Bertin does not consider 3D or time Note: Card and Mackinlay extend the number of vars. Note: Bertin does not consider 3D or time Note:Note: BertinCard and does Mackinlay not consider extend 3D the or numbertime of vars. Note: Card and Mackinlay extend the number of vars.

15 15 Colors for Sequences

• Order these colors (low to high)

Based on slide from Stasko Which One ? Which One ? • Hue does not have an implicit perceptual ordering, • Reliably order luminance, always placing grey in between black and white.

• With saturation place the less saturated pink between fully saturated • red and zero-saturation white. • Does not work for red, blue, green, and yellow Color for Sequences

Based on slide from Stasko Colors for Categories

• Ware suggests: • red, green, yellow, blue, black white • pink, cyan, grey, orange, brown, purple

Ware, “Information Visualization” Color Summary

• Do not use color for quantitative data • Use color for nominals and ordinals (w/ care) • Do not use highly saturated colors (pastels are better) • Beware of equiluminous colors next to each other Color Schemes for Maps

http://www.cdc.gov/ﬂu/weekly/usmap.htm

Possible Problems

• Possible problems: • Distract the user when inadequately used • Dependent on viewing and stimulus conditions • Ineffective for color deﬁcient individuals (use redundancy) • Results in information overload • Unintentionally conﬂict with cultural conventions • Cause unintended visual effects and discomfort

127 Other Channels Size • How much channel • Best for ordered data • Size interferes with other color hue and saturation • Judgement of length is good; area is worse and volume is worse Size Issues Shape

• What Channel with point marks • Not line Marks • For large point marks discriminable bins is many • Shape and size have strong relationship • Shape interferes with other channels Tilt/Angle Stipple/Texture

• Separable from Color • Interacts with luminance • Number of discriminant bins depends on region size Mapping to Texture • Goal: • Avoid visual "crosstalk“ • “Orthogonal” perceptual channels • Restricts range of parameters • E.g. approximately 30 degrees difference in orientation needed to distinguish textures • Main application for textures: nominal data • Some applications for direct visualization of orientations Mapping to Texture

• Main parameters for texture • Orientation • Size • Contrast [C. Ware, Visualization] Information

135 Mapping to Texture • Generate texture • Gabor func. as primitives • Parameters: • Orientation • Size • Contrast

• Randomly splatter down [C. Ware, Visualization] Information Gabor functions

• Blending yields continuous Visualization of a magnetic field coverage Stochastic texture model • 136 Mapping to Texture

• Other stochastic texture models: • LIC (Line integral convolution) for vector ﬁeld visualization • Structural models • Procedural description of texture generation • E.g. Lindenmayer systems (L-systems)

137 Effectiveness of Mappings • Effectiveness of visual variables • According to Mackinlay [J. Mackinlay: Automating the design of graphical presentations of relational information. ACM Transactions on Graphics, 5(2), 110-141, 1986]

138