Fundamentals (I)

Color Image Processing

„ The visible spectrum is continuous Image Processing with Biomedical „ Six Broad regions: Applications ‰ Violet, , , , , and ELEG-475/675 „ Object color depends on what wavelengths it reflects „ Achromatic light is void of color (flat spectrum) Prof. Barner ‰ Characterization: intensity (gray level)

Image Processing Prof. Barner, ECE Department, Color Image Processing University of Delaware 3

Color Image Processing Color Fundamentals (II)

„ Full-color and pseudo-color processing „ „ representations „ Color processing „ Chromatic light spectrum: 400-700 nm ‰ Correction „ Descriptive quantities: ‰ Enhancement ‰ Radiance – total energy that flows from a light source (Watts) ‰ Luminance – amount of energy and observer perceives from a ‰ Smoothing/sharpening light source (lumens) ‰ Segmentation ‰ Brightness – subjected descriptor of intensity

Image Processing Prof. Barner, ECE Department, Image Processing Prof. Barner, ECE Department, Color Image Processing University of Delaware 2 Color Image Processing University of Delaware 4

1 Vision Brightness and Response „ Cone response: „ Brightness – notion of intensity ‰ 6-7 million receptors „ – an attribute associated with the dominant ‰ Red sensitive: 65% wavelength (color) ‰ ‰ Green sensitive: 33% The color of an object determines its hue ‰ Blue sensitive: 2% „ Saturation – relative purity, or the amount of „ Most sensitive receptors light mixed with a hue ‰ Pure spectrum are fully saturated, e.g., red „ Primary colors: red (R), green (G), blue (B) ‰ Saturation is inversely proportional to the amount of white „ International Commission on Illumination (CIE) light in a color standard definitions: „ Chromaticity is hue and saturation together ‰ Blue (435.8 nm), Green (546.1 nm), Red (700 nm) ‰ A color may be characterized by its brightness and „ Defined in 1931 – doesn’t exactly match human perception chromaticity

Image Processing Prof. Barner, ECE Department, Image Processing Prof. Barner, ECE Department, Color Image Processing University of Delaware 5 Color Image Processing University of Delaware 7

Primary and Secondary Colors Tristimulus Representation

„ Add primary colors to obtain „ Tristimulus values: X – red; Y – green; Z –blue secondary colors of light: „ Trichromatic coefficients: ‰ Magenta, cyan, and yellow X x = XYZ+ + „ Primarily colors of: Y y = ‰ Light – sources X +YZ+ Z „ Red, green, blue z = X + YZ+ ‰ Pigments – absorbs (subtracts) a x + yz+=1 of light and reflects „ (transmits) the other two alternate approach: chromaticity diagram „ Magenta (absorbs green), cyan ‰ Gives color composition as a function of red (x) and green (absorbs red), and yellow (absorbs (y) blue) „ Solve for blue (z) according to the above „ Secondary pigments: ‰ Projects 3-D color space on to two dimensions ‰ Red, green, and blue

Image Processing Prof. Barner, ECE Department, Image Processing Prof. Barner, ECE Department, Color Image Processing University of Delaware 6 Color Image Processing University of Delaware 8

2 Chromaticity The RGB (Space) Diagram

„ Pure colors are on the „ RGB is the most widely boundary used hardware-oriented ‰ Fully saturated „ Interior points are color space mixtures ‰ Graphics boards, monitors, ‰ A line between two cameras, etc. testing colors indicates all possible mixtures of the ‰ Normalized RGB values two colors ‰ is a diagonal line „ Color gamut – triangle defined by three colors through the cube ‰ Three color mixtures are ‰ Quantization determines restricted to the gamut ‰ No three-color gamut completely encloses the „ Full-color: 24-bit chromaticity diagram representations (16,777,216 colors)

Image Processing Prof. Barner, ECE Department, Image Processing Prof. Barner, ECE Department, Color Image Processing University of Delaware 9 Color Image Processing University of Delaware 11

Color Gamut Color Image Generation Examples

„ RGB monitor color gamut „ Monochrome images ‰ Regular (triangular) shaped represent each color ‰ Based on three highly component controllable light ‰ Hyperplane examples: primaries „ Fix one dimension „ Printing device color gamut „ Example shows three hidden sides of the color ‰ Combination of additive cube and subtracted „ Acquisition process – ‰ Difficult control process reverse operations „ Neither gamut includes all colors ‰ Filter light to obtain RGB components ‰ Monitor is better

Image Processing Prof. Barner, ECE Department, Image Processing Prof. Barner, ECE Department, Color Image Processing University of Delaware 10 Color Image Processing University of Delaware 12

3 Safe RGB Colors (I) The CMY and CMYK Color Spaces

„ Consistent color reproduction is problematic „ CMY – cyan, magenta, and yellow ‰ Plethora of hardware from different manufacturers ‰ CMYK – adds „ Define a subset of colors to be faithfully reproduced on all hardware „ Black is difficult (and costly) to produce with CMY ‰ 256 colors „ Four- „ Sufficient number to produce good images ‰ Subtracted primaries – widely used in printing „ Small enough set to be accurately reproduced „ 40 of these yield hardware specific results ‰ De facto safe RGB/Web/browser colors: 216 colors ⎡ CR⎤⎡⎤⎡⎤1 „ Formed as RGB triplets of values below ⎢ ⎥⎢⎥⎢⎥ ⎢M ⎥⎢⎥⎢⎥=−1 G ⎣⎢ YB⎦⎣⎦⎣⎦⎥⎢⎥⎢⎥1

Image Processing Prof. Barner, ECE Department, Image Processing Prof. Barner, ECE Department, Color Image Processing University of Delaware 13 Color Image Processing University of Delaware 15

Safe RGB Colors (II) The HSI Color Space (I)

„ 216 safe RGB colors „ Hue, saturation, intensity ‰ human perceptual descriptions „ 256 color RGB system of color includes 16 gray levels ‰ Decouples intensity (gray level) ‰ Six are in the 216 safe from hue and saturation colors (underlined) ‰ Rotate RGB cube so intensity is the vertical axis „ RGB said-color cube „ The intensity component of any color is its vertical component „ Saturation – distance from vertical axis ‰ Zero saturation: colors (gray values) on the vertical axis ‰ Fully saturated: pure colors on the cube boundaries „ Hue – primary color indicated as an angle of rotation

Image Processing Prof. Barner, ECE Department, Image Processing Prof. Barner, ECE Department, Color Image Processing University of Delaware 14 Color Image Processing University of Delaware 16

4 The HSI Color Space (II) HSI to RGB Conversion – Three Cases

„ View the HSI space „ Case 1: RG sector (0°≤H≤120°) from top down B = IS(1− ) ‰ Slicing plane ⎡ SHcos ⎤ perpendicular to RI=+⎢1 ⎥ intensity ⎣ cos(60°−H ) ⎦ „ Intensity – height of GRB=1(−+ ) slicing plane „ Case 2: GB sector (120°≤H240°) „ Saturation – distance from HH= −°120 center (intensity R = IS(1− ) axis) ⎡ SHcos ⎤ GI=+⎢1 ⎥ „ Hue – rotation ⎣ cos(60°−H )⎦ angle from Red B = 1(−+RG ) „ Natural shape: „ Case 3: BR sector (240°≤H≤360°) hexagon HH= −°240 ‰ Normalized to circle or triangle GI= (1− S ) ⎡ SHcos ⎤ BI=+⎢1 ⎥ ⎣ cos(60°−H )⎦ R = 1(−+GB )

Image Processing Prof. Barner, ECE Department, Image Processing Prof. Barner, ECE Department, Color Image Processing University of Delaware 17 Color Image Processing University of Delaware 19

RGB to HSI HSI Component Example (I) Conversion

„ Common HSI representations „ RGB to HSI conversion θ if BG≤ H = { 360−>θ if BG ⎧⎫1 []()()RG−+− RB −1 ⎪⎪2 θ = cos ⎨⎬1 ⎪⎪⎡⎤()()()RG−+−2 RBGB − 2 ⎩⎭⎣⎦ 3 SRGB=−1min(,,)[] „ HSI representations of the color cube ()RGB++ 1 ‰ Normalized values represented as gray values I =++()RGB 3 ‰ Only values on surface of cube shown ‰ Result for normalized (circular) HSI representation „ Explain:

‰ Take care to note which HSI ‰ Sharp transition in hue representation is being used! ‰ Dark and light corners in saturation

‰ Uniform intensity

Image Processing Prof. Barner, ECE Department, Image Processing Prof. Barner, ECE Department, Color Image Processing University of Delaware 18 Color Image Processing University of Delaware 20

5 HSI Component Example (II) Density Slicing Example (I)

„ Primary and secondary colors „ Eight color density slicing of thyroid Phantom ‰ HSI ‰ Density slicing enables visualization of variations representation and details

Image Processing Prof. Barner, ECE Department, Image Processing Prof. Barner, ECE Department, Color Image Processing University of Delaware 21 Color Image Processing University of Delaware 23

Pseudocolor Image Processing Density Slicing Example (II)

„ Assigning colors to gray „ X-ray image of a values yields Pseudocolor weld () images ‰ Density slicing to help ‰ assignment criteria is application-specific visualize cracks „ Intensity (density) slicing

‰ Assign colors based on gray value relation to slicing plane

f (,xy )=∈ ckk if f(, xy ) V

„ Special case: Thresholding

Image Processing Prof. Barner, ECE Department, Image Processing Prof. Barner, ECE Department, Color Image Processing University of Delaware 22 Color Image Processing University of Delaware 24

6 Density Slicing Example (III) Example: Airport x-ray Scanning System

„ Sinusoidal color mappings ‰ Phase changes between components yield different results ‰ Greatest color changes at sinusoidal troughs „ Largest derivative „ First mapping: ‰ Highlights explosives „ Second mapping: ‰ Explosives and bag have similar mappings „ Explosive is “transparent”

Image Processing Prof. Barner, ECE Department, Image Processing Prof. Barner, ECE Department, Color Image Processing University of Delaware 25 Color Image Processing University of Delaware 27

Gray Level to Color Transformations Multispectral Extensions

„ Each color can be a dependent/independent „ Pseudo coloring is often used in the visualization of function of gray level multispectral images ‰ Example: RGB processing ‰ Examples: Satellite and astronomy images ‰ Goal: highlight (color) „ , infrared, radio waves, etc. objects or features of ‰ Transformations are applications and spectral band interest dependent

Image Processing Prof. Barner, ECE Department, Image Processing Prof. Barner, ECE Department, Color Image Processing University of Delaware 26 Color Image Processing University of Delaware 28

7 Wash. DC LANDSAT Example (I) Galileo Spacecraft Example

„ Multispectral image of

‰ Jupiter’s moon: Ito

‰ Multispectral bands are chemical composition sensitive „ Pseudocolor image

‰ Highlights volcanic activity „ New deposits: red „ Old deposits: yellow

Image Processing Prof. Barner, ECE Department, Image Processing Prof. Barner, ECE Department, Color Image Processing University of Delaware 29 Color Image Processing University of Delaware 31

Wash. DC LANDSAT Full-Color Image Processing Example (II)

„ Images in bands 1-4 „ Samples in „ Color composite image using observation window ‰ Band 1 (visible blue) as blue ‰ Vectors ‰ Band 2 (visible green) as green ⎡cxyR (, )⎤⎡ Rxy (, ) ⎤ ‰ Band 3 (visible red) as red cxy(, )==⎢ c (, xy )⎥⎢ Gxy (, ) ⎥ ⎢ G ⎥⎢ ⎥ ‰ Result is difficult to analyze ⎣⎢cxyB (, )⎦⎣⎥⎢ Bxy (, ) ⎦⎥ „ Color composite image using „ General transformation: ‰ Bands 1 and 2 as above g(,xy )= T[ f (, xy )] ‰ Band 4 (near infrared) as red ‰ Better distinguishes between „ Restrict transformation to be a set {T1,T2, …,Tn} of biomass (red dominated) and transformations or color mappings man-made structures sii= Trr(12 , ,..., r n ) ‰ RGB: n=3; HSI: n=3; CMYK: n=4

Image Processing Prof. Barner, ECE Department, Image Processing Prof. Barner, ECE Department, Color Image Processing University of Delaware 30 Color Image Processing University of Delaware 32

8 Image & Color Complements Components

„ Image and CMYK, RGB, and HSI components

„ Simple application:

‰ Intensity scaling

‰ HSI space: „ Color circle ‰ Circular connection of s3=kr3 visible spectrum ‰ RGB space: „ Color complementation ‰ Color negatives si=kri i=1,2,3 ‰ Shown transformations ‰ CMY space: „ RGB: exact „ HSI: approximation si=kri +(1-k) i=1,2,3 ‰ S component not independent of H&I

Image Processing Prof. Barner, ECE Department, Image Processing Prof. Barner, ECE Department, Color Image Processing University of Delaware 33 Color Image Processing University of Delaware 35

Scaling Result Systems (CMS)

„ All devices have their own profile „ Goal: device independent color model ‰ Must be able to represent the entire color gamut „ Scaling result for k=0.7 ‰ Shown: ‰ Shown: RGB, CMY, and HSI transformations „ RGB monitor gamut „ (HS and I transformations swapped) „ Full gamut

Image Processing Prof. Barner, ECE Department, Image Processing Prof. Barner, ECE Department, Color Image Processing University of Delaware 34 Color Image Processing University of Delaware 36

9 CIE L*a*b* Color Space (I) Tone Corrections

„ Desired color space attributes „ Change intensity, not ‰ Color metric – colors perceived as matching are identically coded color ‰ Perceptually uniform – color differences among various are perceived uniformly ‰ RGB and CMYK „ Distance in color space: uniformly scale space matches components perceived difference in colors ‰ HSI space: scale ‰ Device independent – intensity (luminance) independent of specific device display characteristics „ Gamut encompasses entire visible spectrum

Image Processing Prof. Barner, ECE Department, Image Processing Prof. Barner, ECE Department, Color Image Processing University of Delaware 37 Color Image Processing University of Delaware 39

CIE L*a*b* Color Space (II) Color Imbalances

„ Tristimulus to L*a*b* „ Components: „ Color in balances are conversion: ‰ Intensity (): L* * ⎛⎞Y Lh=⋅116⎜⎟ − 16 normally addressed in ⎝⎠YW ‰ Color: „ the RGB or CMYK * ⎡⎤⎛⎞⎛⎞XY Red minus green: a* ah=−500 ⎢⎥⎜⎟⎜⎟ h „ ⎣⎦⎝⎠⎝⎠XYWW Green minus blue: b* spaces ⎡⎤ * ⎛⎞⎛YZ ⎞ „ Appropriate for applications ‰ Corrective mappings bhh=−200⎢⎥⎜⎟⎜ ⎟ ⎝⎠⎝YZWW ⎠ that require: where ⎣⎦ shown ‰ Full color space hq= 3 qq > 0.008856 representation () {7.787qq+≤ 16/116 0.008856 ‰ Color space distance and „ Reference white tristimulus perceptual difference values: matching ‰ ‰ X =0.3127, Y =0.3290, and Drawbacks: computational W W cost ZW=1-XW-YW

Image Processing Prof. Barner, ECE Department, Image Processing Prof. Barner, ECE Department, Color Image Processing University of Delaware 38 Color Image Processing University of Delaware 40

10 Histogram HSI Processing Processing

„ Perform histogram equalization on Intensity ‰ Avoids generation of new colors „ Independent component processing is undesirable ‰ Improves statistics of intensity ‰ Does impact „ Alternative approach: process Intensity vibrancy of colors „ Solution: ‰ Useful for extending grayscale procedures to color increase saturation

Image Processing Prof. Barner, ECE Department, Image Processing Prof. Barner, ECE Department, Color Image Processing University of Delaware 41 Color Image Processing University of Delaware 43

Separable RGB HSI Smoothing Comparison Functions

„ Simple separable linear functions can be applied to components independently ‰ Example: spatial averaging 1 cc(,xy )= ∑ (, xy ) K (,)xy∈ Sxy ⎡⎤1 ⎢⎥R(,xy ) K ∑ ⎢⎥(,)xy∈ Sxy ⎢⎥1 c(,x yGxy )= ⎢⎥∑ (, ) ⎢⎥K (,)xy∈ Sxy ⎢⎥ 1 „ Similar, but not identical results ⎢⎥B(,xy ) ⎢⎥K ∑ ⎣⎦(,)xy∈ Sxy ‰ RGB processing introduces new colors ‰ Apply on RGB components

Image Processing Prof. Barner, ECE Department, Image Processing Prof. Barner, ECE Department, Color Image Processing University of Delaware 42 Color Image Processing University of Delaware 44

11 RGB HSI Sharpening Comparison Vector Gradient

„ RGB unit vectors: r, g, b „ Directional derivatives: ∂R ∂∂GB ur=+g +b ∂x ∂∂xx ∂R ∂∂GB vrgb=+ + ∂yyy∂∂

„ Dot products: ∂R 222∂∂GB g =⋅=uu uuT = + + xx ∂x ∂∂xx 222 ∂R ∂∂GB g =⋅=vv vvT = + + yy ∂y ∂∂yy ∂R ∂∂∂∂∂RGGBB g =⋅=uv uvT = + + xy ∂x ∂∂∂∂∂yxyxy ⎡⎤ 1 2gxy „ Direction of maximum change: θ =⎢tan−1 ⎥ 2 ⎢⎥gg− „ Laplacian reduces to component-wise application ⎣⎦()xx yy ⎡⎤∇2 R(,xy ) 22⎢⎥ ∇=∇[]c(,x yGxy )⎢⎥ (, ) „ ⎢⎥2 Magnitude of maximum change: ⎣⎦∇ B(,xy ) 1 ⎧1 ⎫2 „ Application on Intensity yields similar results Fggggg()θθθ=++−+⎨ [(xx yy )() xx yy cos2 2 xy sin2]⎬ ⎩⎭2

Image Processing Prof. Barner, ECE Department, Image Processing Prof. Barner, ECE Department, Color Image Processing University of Delaware 45 Color Image Processing University of Delaware 47

Edge Detection in Color Images Gradient Example

„ Shown ‰ Input image ‰ RGB space vector gradient ‰ RGB space independent component „ Gradient operators applied independently to color components yields poor gradient results „ Results „ RGB example: step edges in individual color planes summed ‰ Case 1: aligned edges ‰ Case 2: two aligned edges, one orthogonal edge ‰ Difference ‰ Both cases yield identical gradients at image center image „ Color change more significant in Case 1

Image Processing Prof. Barner, ECE Department, Image Processing Prof. Barner, ECE Department, Color Image Processing University of Delaware 46 Color Image Processing University of Delaware 48

12 Component Gradients Noise and Color Space Conversion

„ Independent Gaussian noise in the RGB channels

‰ Resulting color image

‰ Note introduced colors „ RGB component gradients

‰ Note broken edges in individual components

Image Processing Prof. Barner, ECE Department, Image Processing Prof. Barner, ECE Department, Color Image Processing University of Delaware 49 Color Image Processing University of Delaware 51

Noise in Color Images HSI Representation of Noisy Image

„ The general degradation model holds in the color case „ Noise affecting individual color planes usually has the same characteristics ‰ Usually modeled as independent „ Hue and saturation are severely degraded „ Possible differences: ‰ Nonlinear transformations from the RGB space ‰ Differences in illumination levels „ Involves cosine and minimum operators „ Red (filtered) channel in a CCD camera tends to have lower illumination (higher noise) „ Intensity component is smoothed ‰ Bad sensors in an individual channel ‰ Average of RGB components

Image Processing Prof. Barner, ECE Department, Image Processing Prof. Barner, ECE Department, Color Image Processing University of Delaware 50 Color Image Processing University of Delaware 52

13 Single Channel Corruption

„ Single channel corruption ‰ Salt and pepper noise in the green channel ‰ p=0.05 „ Color space conversion ‰ Spreads noise ‰ Changes statistics ‰ Shown: HSI components

Image Processing Prof. Barner, ECE Department, Color Image Processing University of Delaware 53

14