Color Image Perception, Representation and Contrast Enhancement
Total Page:16
File Type:pdf, Size:1020Kb
Color Image Perception, Representation and Contrast Enhancement Yao Wang Tandon School of Engineering, New York University Why Study Image and Video Processing? • Purpose of image and video processing – Improvement of pictorial information for human interpretation – Compression of image and video data for storage and transmission – Preprocessing to enable object detection, classification, and tracking • Typical application areas – Television Signal Processing – Satellite Image Processing – Medical Image Processing – Robotics – Visual Communications – Law Enforcement – Etc. © Yao Wang, 2016 EL-GY 6123 2 Breakdown of the Course • Image processing basics • Video processing basics • Image and video compression • Stereo and multiview image/video basics © Yao Wang, 2016 EL-GY 6123 3 Today’s Lecture • Color perception • Color production using primary colors • Color specification • Color image representation • Image capture and display • Image contrast measurement using histograms • Image contrast enhancement © Yao Wang, 2016 EL-GY 6123 4 Color Perception and Specification • Light -> color perception • Human perception of color • Type of light sources • Trichromatic color mixing theory • Specification of color – Tristimulus representation – Luminance/Chrominance representation • Color coordinate conversion © Yao Wang, 2016 EL-GY 6123 5 Light is part of the EM wave from [Gonzalez02] © Yao Wang, 2016 EL-GY 6123 6 Illuminating and Reflecting Light • Illuminating sources: – emit light (e.g. the sun, light bulb, TV monitors) – perceived color depends on the emitted freq. – follows additive rule • R+G+B=White • Reflecting sources: – reflect an incoming light (e.g. the color dye, matte surface, cloth) – perceived color depends on reflected freq (=emitted freq- absorbed freq.) – follows subtractive rule • R+G+B=Black © Yao Wang, 2016 EL-GY 6123 7 Eye Anatomy From http://www.stlukeseye.com/Anatomy.asp © Yao Wang, 2016 EL-GY 6123 8 Eye vs. Camera Camera components Eye components Lens Lens, cornea Shutter Iris, pupil Film Retina Cable to transfer images Optic nerve send the info to the brain © Yao Wang, 2016 EL-GY 6123 9 Human Perception of Color • Retina contains photo receptors – Cones: day vision, can perceive color tone • Red, green, and blue cones • Different cones have different frequency responses • Tri-receptor theory of color vision [Young1802] – Rods: night vision, perceive brightness From http://www.macula.org/ only anatomy/retinaframe.html • Color sensation is characterized by – Luminance (brightness) – Chrominance • Hue (color tone) • Saturation (color purity) © Yao Wang, 2016 EL-GY 6123 10 Frequency Responses of Cones and the Luminous Efficiency Function Ci = ∫C(λ)ai (λ)dλ, i = r, g,b, y © Yao Wang, 2016 EL-GY 6123 11 Three Attributes of Color • Luminance (brightness) • Chrominance – Hue (color tone) and Saturation (color purity) • Represented by a “color cone” θ © Yao Wang, 2016 EL-GY 6123 12 Trichromatic Color Mixing • Trichromatic color mixing theory – Any (actually most) color can be obtained by mixing three primary colors with a right proportion C = ∑TkCk , Tk : Tristimulus values k=1,2,3 • Primary colors for illuminating sources: – Red, Green, Blue (RGB) – Color monitor works by exciting red, green, blue phosphors using separate electronic guns • Primary colors for reflecting sources (also known as secondary colors): – Cyan, Magenta, Yellow (CMY) – Color printer works by using cyan, magenta, yellow and black (CMYK) dyes © Yao Wang, 2016 EL-GY 6123 13 RGB vs CMY Magenta = Red + Blue Magenta = White - Green Cyan = Blue + Green Cyan = White - Red Yellow = Green + Red Yellow = White - Blue © Yao Wang, 2016 EL-GY 6123 14 red Green Blue © Yao Wang, 2016 EL-GY 6123 15 Tristimulus Values and Chromaticity • Tristimulus values – The amounts of three primary colors needed to form any particular color are called the tristimulus values, denoted by X, Y, and Z (or any other three symbols). • Trichromatic (or chromaticity) coefficients X Y Z x = , y = , z = . X +Y + Z X +Y + Z X +Y + Z – Only two chromaticity coefficients are necessary to specify the chrominance of a light. x + y + z =1 © Yao Wang, 2016 EL-GY 6123 16 Problems with Using Primary Colors for Color Representation • Tristimulus values depend on the primary colors used by the camera/display • Tristimulus values may be negative for some visible colors • How to compare colors obtained using different devices • Need for standardization • Standard using physical primary colors (CIE RGB) • Standard using imaginary primary colors (CIE XYZ): device invariant • Colors captured by cameras using different physical RGB primaries can be converted to the standard XYZ representation • CIE: International commission on illumination © Yao Wang, 2016 EL-GY 6123 17 CIE XYZ Color Space • Use imaginary color primaries X, Y, Z so that the tristimulus values are all non-negative • Encompasses all visible colors by an average person • Device invariant color representation • Color primaries used by each device can be specified using XYZ values • Colors captured by cameras using different physical RGB primaries can be converted to the standard XYZ representation © Yao Wang, 2016 EL-GY 6123 18 CIE XYZ Color Matching Functions https://en.wikipedia.org/wiki/ CIE_1931_color_space • Y approximates the brightness ~= green cone response of an average person (standard observer) • Z approximates the blue cone response of an average person • X is NOT red, rather derived so that XY,Z values for all visible colors are all >=0. © Yao Wang, 2016 EL-GY 6123 19 CIE Chromaticity Diagram • Colors on the boundary: spectrum colors, highest saturation • Shows all visible colors by humans • Mixing any three colors in the visible range can generate colors in the triangle formed by these three points. © Yao Wang, 2016 EL-GY 6123 from [Gonzalez02] 20 Color Gamut of Monitors and Printers • Not all visible colors can be reproduced by RGB primaries used for display, or CMY primaries used for printing. • The triangle: gamut of color monitors using the Red, Green, and Blue colors at the corner as primary colors • The irregular region inside: gamut of most color printers • A printer cannot reproduce all colors that you see on the screen! © Yao Wang, 2016 EL-GY 6123 21 CIE RGB Color Space • Using single spedtral color for primary colors: – 700 nm (R), 546.1 nm (G), 435.8nm (B) • Color matching functions are the amounts of primaries needed to match each single https://en.wikipedia.org/wiki/CIE_1931_color_space spectral color © Yao Wang, 2016 EL-GY 6123 22 Color Primary Defined by ITU Rec. 709 for HDTV ! R $ ! 3.240479 −1.537150 −0.498535 $! X $ # & # &# & # G & = # −0.969256 1.875992 0.041556 &# Y & # & # &# Z & " B % " 0.055648 −0.204043 1.057311 %" % ! X $ ! 0.412453 0.357580 0.180423 $! R $ # & # &# & # Y & = # 0.212671 0.715160 0.072169 &# G & # Z & # &# & " % " 0.019334 0.119193 0.950277 %" B % The three columns in the RGB-> XYZ conversion are the tristimulus values of the R,G,B primaries defined in terms of CIE XYZ primaries. From these, you could derive the chromaticity coefficients of RGB primaries and correspondingly locate them in the CIE diagram (HW!) © Yao Wang, 2016 EL-GY 6123 23 Comparison of Colors Defined Using Different Primiaries • To compare colors defined using different primaries, we should convert them all to CIE XYZ primaries and compare their XYZ values © Yao Wang, 2016 EL-GY 6123 24 Color Representation Models • Specify the tristimulus values associated with the three primary colors – RGB, CMY, XYZ • Specify the luminance and chrominance – HSI (Hue, saturation, intensity) – YIQ (used in NTSC color TV) – YCbCr (used in digital color TV) • Amplitude specification: – 8 bits for each color component, or 24 bits total for each pixel – Total of 16 million colors – A true RGB color display of size 1Kx1K requires a display buffer memory size of 3 MB • High dynamic range (HDR) Image: up to 16 bits/component © Yao Wang, 2016 EL-GY 6123 25 HSI Color Model • Hue represents dominant color as perceived by an observer. It is an attribute associated with the dominant wavelength. • Saturation refers to the relative purity or the amount of white light mixed with a hue. The pure spectrum colors are fully saturated. Pink and lavender are less saturated. • Intensity reflects the brightness. © Yao Wang, 2016 EL-GY 6123 26 Conversion Between RGB and HSI • Converting from RGB to HSI ⎧ 1 ⎫ [(R − G) + (R − B)] ⎧ θ if B ≤ G 1⎪ ⎪ H with cos− 2 = ⎨ θ = ⎨ 1 ⎬ ⎩360 −θ if B > G 2 ⎪ (R G) (R B)(G B) 2 ⎪ ⎩⎪[ − + − − ] ⎭⎪ 3 S = 1− [min(R,G, B)] (R + G + B) 1 I = [R + G + B] 3 • Converting from HSI to RGB RG sector (0≤H<120) GB sector (120≤H<240) BR sector (240≤H<360) B = I(1− S) R = I(1− S) G = I(1− S) ⎡ S cos H ⎤ ⎡ S cos(H −120) ⎤ ⎡ S cos(H − 240) ⎤ R I 1 G = I 1+ B = I 1+ = ⎢ + ⎥ ⎢ cos(60 (H 120))⎥ ⎢ ⎥ ⎣ cos(60 − H )⎦ ⎣ − − ⎦ ⎣ cos(60 − (H − 240))⎦ G =1− (R + B) B =1− (R + G) R =1− (G + B) © Yao Wang, 2016 EL-GY 6123 from [Gonzalez02] 27 YIQ Color Coordinate System • YIQ is defined by the National Television System Committee (NTSC) for US analog color TV system – Y describes the luminance, I and Q describes the chrominance. – A more compact representation of the color. – YUV plays similar role in PAL and SECAM. • Conversion between RGB and YIQ ⎡Y ⎤ ⎡0.299 0.587 0.114 ⎤⎡R⎤ ⎡R⎤ ⎡1.0 0.956 0.621 ⎤⎡Y ⎤ ⎢ I ⎥ ⎢0.596 0.274 0.322⎥⎢G⎥, ⎢G⎥ ⎢1.0 0.272 0.649⎥⎢ I ⎥ ⎢ ⎥ = ⎢ − − ⎥⎢ ⎥ ⎢ ⎥ = ⎢ − − ⎥⎢ ⎥ ⎣⎢Q⎦⎥ ⎣⎢0.211 − 0.523 0.311 ⎦⎥⎣⎢B⎦⎥ ⎣⎢B⎦⎥ ⎣⎢1.0 −1.106 1.703 ⎦⎥⎣⎢Q⎦⎥ © Yao Wang, 2016 EL-GY 6123 28 YUV/YCbCr Coordinate • YUV is the color coordinate