Lecture 8. Color Image Processing

EL512 Image Processing

Note: Part of the materials in the slides are from Gonzalez’s Digital Image Processing and Onur’s lecture slides Lecture Outline

• Color perception and representation – Human perception of color – Trichromatic color mixing theory – Different color representations • Color image display – True color image – Indexed color images • Pseudo color images • Color image enhancement

Fall 2003 EL512 Image Processing Lecture 8, Page 2 Light is part of the EM wave

Fall 2003 EL512 Image Processing Lecture 8, Page 3 Human Perception of Color

• Retina contains receptors – Cones • Day vision, can perceive color tone • Red, green, and blue cones – Rods • night vision, perceive brightness only • Color sensation – Luminance (brightness) – Chrominance From http://www.macula.org/anatomy • Hue (color tone) /retinaframe.html • Saturation (color purity)

Fall 2003 EL512 Image Processing Lecture 8, Page 4 Frequency Responses of Cones

Ci = ∫C(λ)ai (λ)dλ, i = r, g,b, y Fall 2003 EL512 Image Processing Lecture 8, Page 5 Illuminating and Reflecting Light • Illuminating sources: – emit light (e.g. the sun, light bulb, TV monitors) – perceived color depends on the emitted frequency • Reflecting sources: – reflect an incoming light (e.g. the color dye, matte surface, cloth) – perceived color depends on reflected frequency (=emitted frequency - absorbed frequency

Fall 2003 EL512 Image Processing Lecture 8, Page 6 Tri-chromatic Color Mixing

• Tri-chromatic color mixing theory – Any color can be obtained by mixing three primary colors with a right proportion • Primary colors for illuminating sources: – Red, Green, Blue (RGB) – Color monitor works by exciting red, green, blue phosphors using separate electronic guns – follows additive rule: R+G+B=White • Primary colors for reflecting sources (also known as secondary colors): – Cyan, Magenta, Yellow (CMY) – Color printer works by using cyan, magenta, yellow and black (CMYK) dyes – follows subtractive rule: R+G+B=Black

Fall 2003 EL512 Image Processing Lecture 8, Page 7 RGB vs CMY

Magenta = Red + Blue Magenta = White - Green Cyan = Blue + Green Cyan = White - Red Yellow = Green + Red Yellow = White - Blue Fall 2003 EL512 Image Processing Lecture 8, Page 8 A Color Image

Red

Green Blue

Fall 2003 EL512 Image Processing Lecture 8, Page 9 Tristimuls Values • Tristimulus value – The amounts of red, green, and blue needed to form any particular color are called the tristimulus values, denoted by X, Y, and Z. – Trichromatic coefficients X Y Z x = , y = , z = . X +Y + Z X +Y + Z X +Y + Z – Only two chromaticity coefficients are necessary to specify the chrominance of a light. x + y + z =1

Fall 2003 EL512 Image Processing Lecture 8, Page 10 CIE Chromaticity Diagram

CIE (Commission Internationale de L’Eclairage, International Commission on Illumination ) system of color specification

x axis: red y axis: green

The point marked with GREEN x: 25%, y: 62%, z: 13%.

Fall 2003 EL512 Image Processing Lecture 8, Page 11 Color Models

• Specify three primary or secondary colors – Red, Green, Blue. – Cyan, Magenta, Yellow. • Specify the luminance and chrominance – HSB or HSI (Hue, saturation, and brightness or intensity) – YIQ (used in NTSC color TV) – YCbCr (used in digital color TV) • Amplitude specification: – 8 bits per color component, or 24 bits per pixel – Total of 16 million colors – A 1kx1k true RGB color requires 3 MB memory

Fall 2003 EL512 Image Processing Lecture 8, Page 12 RGB Color Model

RGB 24-bit color cube

Fall 2003 EL512 Image Processing Lecture 8, Page 13 CMY and CMYK Color Models

• Conversion between RGB and CMY ⎡ C ⎤ ⎡1⎤ ⎡R⎤ ⎡R⎤ ⎡1⎤ ⎡ C ⎤ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢M ⎥ = ⎢1⎥ − ⎢G⎥, ⎢G⎥ = ⎢1⎥ − ⎢M ⎥. ⎣⎢ Y ⎦⎥ ⎣⎢1⎦⎥ ⎣⎢B⎦⎥ ⎣⎢B⎦⎥ ⎣⎢1⎦⎥ ⎣⎢ Y ⎦⎥

• Equal amounts of Cyan, Magenta, and Yellow produce black. In practice, this produce muddy-looking black. To produce true black, a fourth color, black is added, which is CMYK color model.

Fall 2003 EL512 Image Processing Lecture 8, Page 14 HSI Color Model

• Hue represents dominant color as perceived by an observer. It is an attribute associated with the dominant wavelength. • Saturation refers to the relative purity or the amount of white light mixed with a hue. The pure spectrum colors are fully saturated. Pink and lavender are less saturated. • Intensity reflects the brightness.

Fall 2003 EL512 Image Processing Lecture 8, Page 15 The HSI Color Model

Fall 2003 EL512 Image Processing Lecture 8, Page 16 Conversion Between RGB and HSI

• Converting color from RGB to HSI ⎧ 1 ⎫ [](R − G) + (R − B) ⎧ θ if B ≤ G −1 ⎪ 2 ⎪ H = ⎨ , with θ = cos ⎨ ⎬ 360 −θ if B > G 1 ⎩ ⎪[](R − G)2 + (R − B)(G − B) 2 ⎪ ⎩ ⎭ 3 S =1− [min(R,G, B)] (R + G + B) 1 I = [R + G + B] 3 • Converting color from HSI to RGB RG sector (0≤H<120) GB sector (120≤H<240) BR sector (240≤H<360) B = I(1− S) R = I(1− S) G = I(1− S) ⎡ S cos(H −120) ⎤ ⎡ ⎤ ⎡ S cos H ⎤ G = I 1+ S cos(H − 240) R = I ⎢1+ ⎥ ⎢ ⎥ B = I ⎢1+ ⎥ ⎣ cos(60 − H )⎦ ⎣ cos(60 − (H −120))⎦ ⎣ cos(60 − (H − 240))⎦ G =1− (R + B) B =1− (R + G) R =1− (G + B) Fall 2003 EL512 Image Processing Lecture 8, Page 17 YIQ Color Coordinate System

• YIQ is defined by the National Television System Committee (NTSC) – Y describes the luminance, I and Q describes the chrominance. – A more compact representation of the color. – YUV plays similar role in PAL and SECAM. • Conversion between RGB and YIQ ⎡Y ⎤ ⎡0.299 0.587 0.114 ⎤⎡R⎤ ⎡R⎤ ⎡1.0 0.956 0.621 ⎤⎡Y ⎤ ⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ I ⎥ = ⎢0.596 − 0.274 − 0.322⎥⎢G⎥, ⎢G⎥ = ⎢1.0 − 0.272 − 0.649⎥⎢ I ⎥ ⎣⎢Q⎦⎥ ⎣⎢0.211 − 0.523 0.311 ⎦⎥⎣⎢B⎦⎥ ⎣⎢B⎦⎥ ⎣⎢1.0 −1.106 1.703 ⎦⎥⎣⎢Q⎦⎥

Fall 2003 EL512 Image Processing Lecture 8, Page 18 Criteria for Choosing the Color Coordinates • The type of representation depends on the applications at hand. – For display or printing, choose primary colors so that more colors can be produced. E.g. RGB for displaying and CMY for printing. – For analytical analysis of color differences, the difference in the trisitumulus values are linearly related to the chrominance differences. HSI is more suitable. – For transmission or storage, choose a less redundant representation, eg. YIQ or YUV

Fall 2003 EL512 Image Processing Lecture 8, Page 19 Comparison of Different Color Spaces

Fall 2003 EL512 Image Processing Lecture 8, Page 20 Demo Using Photoshop

• Show the RGB, CMY, HSI models – Using the “window->info” tool and the “window>show color” tool (in “show color”, click on right arrow button to choose different color sliders) – Sample image: RGBadd, CMYsub

Fall 2003 EL512 Image Processing Lecture 8, Page 21 Color Image Display and Printing

• Display: – Need three light sources projecting red, green, blue components respectively at every pixel – Analog display: raster scan – Digital display: directly projecting at all pixel locations • Printing: – Need three (or more) color dyes (Cyan, Magenta, Yellow, and Black) – Analog printing – Digital printing – Out of gamut color

Fall 2003 EL512 Image Processing Lecture 8, Page 22 Color Image Display

Input Output 0 1 … 255

Red signal Red Red LUT Gun

Green signal Green Red LUT Gun

Blue signal Blue Red LUT Gun

Fall 2003 EL512 Image Processing Lecture 8, Page 23 Color Gamut

Each color model has different color range (or gamut). RGB model has a larger gamut than CMY. Therefore, some color that appears on a screen may not be printable and is replaced by the closest color in the CMY gamut.

Fall 2003 EL512 Image Processing Lecture 8, Page 24 Gamma Correction

• The intensity to voltage response curve of the computer monitor is not linear.

Sample Input to Monitor Graph of Input

Graph of Output L=V2.5 Output from Monitor • Gamma correction Sample Input to Monitor Graph of Input

1/2.5 Gamma corrected Input Graph of Correction L’=L

Monitor Output Graph of Output

Fall 2003 EL512 Image Processing Lecture 8, Page 25 Demo with Photoshop

• Using photoshop to show how to replace a out of gamut color by its closest in-gamut color. • Choose “window->show swatch”, choose “blue”

Fall 2003 EL512 Image Processing Lecture 8, Page 26 Color Quantization

• Select a set of colors that are most frequently used in an image, save them in a look-up table (also known as color map or color palette) • Any color is quantized to one of the indexed colors • Only needs to save the index as the image pixel value and in the display buffer • Typically: k=8, m=8 (selecting 256 out of 16 million)

Input index (k bits) Red color (m bits) Green color (m bits) Blue color (m bits)

Index 1 … … …

……. … … …

Index 2^k … … …

Fall 2003 EL512 Image Processing Lecture 8, Page 27 Uniform vs. Adaptive Quantization • Uniform (scalar quantization) – Quantize each color component uniformly • E.g. 24 bit-> 8 bit can be realized by using 3 bits (8 levels) for red, 3 bits (8 levels) for green, 2 bits (4 levels) for blue • Do not produce good result • Adaptive (vector quantization) – Treating each color (a 3-D vector) as one entity. Finds the N colors (vectors) that appear most often in a given image, save them in the color palette (codebook). Replace the color at each pixel by the closest color in the codebook – The codebook (I.e. color palette) varies from image to image -> adaptive Fall 2003 EL512 Image Processing Lecture 8, Page 28 Illustration of the Vector Quantization

y Codebook size: 25 y

x x

Uniform Quantization Vector Quantization Fall 2003 EL512 Image Processing Lecture 8, Page 29 Example of Color Image Quantization

24 bits -> 8 bits

Uniform quantization Adaptive (non-uniform) quantization (3 bits for R,G, 2 bits for B) (vector quantization)

Fall 2003 EL512 Image Processing Lecture 8, Page 30 Web Colors: 216 Safe RGB Colors

These colors are those that can be rendered consistently by different computer systems. They are obtained by quantizing the R,G,B component independently using uniform quanitization. Each component is quantized to 6 possible values: 0(0x00), 51(0x33), 102(0x66), 153(0x99), 204(0xCC), 255(0xFF).

Fall 2003 EL512 Image Processing Lecture 8, Page 31 Color Dithering

• Color quantization may cause contour effect when the number of colors is not sufficient • Dithering: randomly perturb the color values slightly to break up the contour effect – fixed pattern dithering – diffusion dithering (the perturbed value of the next pixel depends on the previous one) – Developed originally for rendering gray scale image using black and white ink only Original Dithered value value (R,G, or B)

Dithering value

Fall 2003 EL512 Image Processing Lecture 8, Page 32 Example of Color Dithering

8 bit uniform 8 bit uniform without dithering with diffusion dithering

Fall 2003 EL512 Image Processing Lecture 8, Page 33 Demo Using Photoshop

• Show quantization results with different methods using “image->mode->index color”

Fall 2003 EL512 Image Processing Lecture 8, Page 34 Pseudo Color Image

• Why? – Human eye is more sensitive to changes in the color hue than in brightness. • How? – Use different colors (different in hue) to represent different image features in a monochrome image.

Fall 2003 EL512 Image Processing Lecture 8, Page 35 Pseudo Color Display

• Intensity slicing: Display different gray levels as different colors – Can be useful to visualize medical / scientific / vegetation imagery • E.g. if one is interested in features with a certain intensity range or several intensity ranges • Frequency slicing: Decomposing an image into different frequency components and represent them using different colors.

Fall 2003 EL512 Image Processing Lecture 8, Page 36 Intensity Slicing

Color

f0 =0 f1 f2 f3 f4 Gray level

Pixels with gray-scale (intensity) value in the range of (f i-1 , fi) are rendered with color Ci

Fall 2003 EL512 Image Processing Lecture 8, Page 37 Example

Fall 2003 EL512 Image Processing Lecture 8, Page 38 Another Example

Fall 2003 EL512 Image Processing Lecture 8, Page 39 Pseudo Color Display of Multiple Images

• Display multi-sensor images as a single color image – Multi-sensor images: e.g. multi-spectral images by satellite

Fall 2003 EL512 Image Processing Lecture 8, Page 40 An Example

Fall 2003 EL512 Image Processing Lecture 8, Page 41 Example

Fall 2003 EL512 Image Processing Lecture 8, Page 42 Color Image Enhancement • Enhance each primary color component independently using the techniques for monochrome images – Will change the color hue of the original image • Convert the tri-stimulus representation into a luminance / chrominance representation, and enhance the contrast of the luminance component only. – Use HSI representation, where I truly reflects the luminance information.

Fall 2003 EL512 Image Processing Lecture 8, Page 43 Example of Color Image Enhancement

Fall 2003 EL512 Image Processing Lecture 8, Page 44 Example of Color Image Enhancement

Fall 2003 EL512 Image Processing Lecture 8, Page 45 Example of Color Image Enhancement

Fall 2003 EL512 Image Processing Lecture 8, Page 46 Homework

1. (Computer Assignment) Write a program which first performs high–pass filtering (you can use matlab func ’conv2’ for this part) of an input gray scale image using the following filter: ⎡ 0 −1 0 ⎤ 1 ⎢−1 4 −1⎥ 4 ⎢ ⎥ ⎣⎢ 0 −1 0 ⎦⎥ Scale the filtered image to range between 0 and 255. Then displays the filtered image using 3 pseudo colors, using the following color transformation: Color Red for values 0-80, Color Green for values 81-160, color blue for values 161-255. In matlab, you can use the function colormap to change the colormap and use imshow to display an image using a specified colormap. Comment on the visual effect, e.g. each color represents what attributes of the image? 2. (Computer Assignment) Choose a 24 bit RGB color image, perform the following operations: 1) convert it to YIQ format, save the resulting images in three separate files (for Y, I and Q components respectively), each with 8 bits/pixel; 2) perform histogram equalization to the Y image; 3) convert the enhanced Y image and the original I and Q image back to the RGB image. View the original and enhanced color RGB images and comment on your observations. 3. (Computer Assignment) Choose a 24 bit color RGB image, quantize the R, G, and B components to 3, 3, and 2 bits, respectively, using a uniform quantizer in the range 0-256. Display the original and quantized color image using the original colormap associated with the image. Comment on the difference in color accuracy. Make sure you use a computer that has a 24 bit color display, and the test image has good color contrast.

Fall 2003 EL512 Image Processing Lecture 8, Page 47 Reading

• Prof. Yao Wang’s Lecture Notes, Chapter 6. • R. Gonzalez, “Digital Image Processing,” Chapter 6. • A. K. Jain, “Fundamentals of Digital Image Processing,” Section 3.7 ~ 3.11, 7.7 ~ 7.8.

Fall 2003 EL512 Image Processing Lecture 8, Page 48