Color Image Perception, Representation and Contrast Enhancement

Yao Wang Tandon School of Engineering, New York University Why Study Image and Video Processing?

• Purpose of image and video processing – Improvement of pictorial information for human interpretation – Compression of image and video data for storage and transmission – Preprocessing to enable object detection, classification, and tracking • Typical application areas – Television Signal Processing – Satellite Image Processing – Medical Image Processing – Robotics – Visual Communications – Law Enforcement – Etc.

© Yao Wang, 2016 EL-GY 6123 2 Breakdown of the Course

• Image processing basics • Video processing basics • Image and video compression • Stereo and multiview image/video basics

© Yao Wang, 2016 EL-GY 6123 3 Today’s Lecture

perception • Color production using primary • Color specification • Color image representation • Image capture and display • Image contrast measurement using histograms • Image contrast enhancement

© Yao Wang, 2016 EL-GY 6123 4 Color Perception and Specification

-> color perception • Human perception of color • Type of light sources • Trichromatic theory • Specification of color – Tristimulus representation – Luminance/ representation • Color coordinate conversion

© Yao Wang, 2016 EL-GY 6123 5 Light is part of the EM wave

from [Gonzalez02]

© Yao Wang, 2016 EL-GY 6123 6 Illuminating and Reflecting Light

• Illuminating sources: – emit light (e.g. the sun, light bulb, TV monitors) – perceived color depends on the emitted freq. – follows additive rule • R+G+B= • Reflecting sources: – reflect an incoming light (e.g. the color dye, matte surface, cloth) – perceived color depends on reflected freq (=emitted freq- absorbed freq.) – follows subtractive rule • R+G+B=

© Yao Wang, 2016 EL-GY 6123 7 Eye Anatomy

From http://www.stlukeseye.com/Anatomy.asp

© Yao Wang, 2016 EL-GY 6123 8 Eye vs.

Camera components Eye components Lens Lens, cornea Shutter Iris, pupil Film Retina Cable to transfer images Optic nerve send the info to the brain

© Yao Wang, 2016 EL-GY 6123 9 Human Perception of Color

• Retina contains photo receptors – Cones: day vision, can perceive color tone • , , and cones • Different cones have different frequency responses • Tri-receptor theory of [Young1802] – Rods: night vision, perceive brightness From http://www.macula.org/ only anatomy/retinaframe.html • Color sensation is characterized by – Luminance (brightness) – Chrominance • (color tone) • Saturation (color purity)

© Yao Wang, 2016 EL-GY 6123 10 Frequency Responses of Cones and the Luminous Efficiency Function

Ci = ∫C(λ)ai (λ)dλ, i = r, g,b, y

© Yao Wang, 2016 EL-GY 6123 11 Three Attributes of Color

• Luminance (brightness) • Chrominance – Hue (color tone) and Saturation (color purity) • Represented by a “color cone”

θ

© Yao Wang, 2016 EL-GY 6123 12 Trichromatic Color Mixing

• Trichromatic color mixing theory – Any (actually most) color can be obtained by mixing three primary colors with a right proportion

C = ∑TkCk , Tk : Tristimulus values k=1,2,3 • Primary colors for illuminating sources: – Red, Green, Blue (RGB) – Color monitor works by exciting red, green, blue phosphors using separate electronic guns • Primary colors for reflecting sources (also known as secondary colors): – Cyan, Magenta, (CMY) – Color printer works by using cyan, magenta, yellow and black (CMYK) dyes

© Yao Wang, 2016 EL-GY 6123 13 RGB vs CMY

Magenta = Red + Blue Magenta = White - Green Cyan = Blue + Green Cyan = White - Red Yellow = Green + Red Yellow = White - Blue

© Yao Wang, 2016 EL-GY 6123 14 red

Green Blue

© Yao Wang, 2016 EL-GY 6123 15 Tristimulus Values and

• Tristimulus values – The amounts of three primary colors needed to form any particular color are called the tristimulus values, denoted by X, Y, and Z (or any other three symbols).

• Trichromatic (or chromaticity) coefficients

X Y Z x = , y = , z = . X +Y + Z X +Y + Z X +Y + Z – Only two chromaticity coefficients are necessary to specify the chrominance of a light.

x + y + z =1

© Yao Wang, 2016 EL-GY 6123 16 Problems with Using Primary Colors for Color Representation

• Tristimulus values depend on the primary colors used by the camera/display • Tristimulus values may be negative for some visible colors • How to compare colors obtained using different devices • Need for standardization • Standard using physical primary colors (CIE RGB) • Standard using imaginary primary colors (CIE XYZ): device invariant • Colors captured by using different physical RGB primaries can be converted to the standard XYZ representation • CIE: International commission on illumination

© Yao Wang, 2016 EL-GY 6123 17 CIE XYZ

• Use imaginary color primaries X, Y, Z so that the tristimulus values are all non-negative • Encompasses all visible colors by an average person • Device invariant color representation • Color primaries used by each device can be specified using XYZ values • Colors captured by cameras using different physical RGB primaries can be converted to the standard XYZ representation

© Yao Wang, 2016 EL-GY 6123 18 CIE XYZ Color Matching Functions

https://en.wikipedia.org/wiki/ CIE_1931_color_space

• Y approximates the brightness ~= green cone response of an average person (standard observer) • Z approximates the blue cone response of an average person • X is NOT red, rather derived so that XY,Z values for all visible colors are all >=0.

© Yao Wang, 2016 EL-GY 6123 19 CIE Chromaticity Diagram

• Colors on the boundary: spectrum colors, highest saturation

• Shows all visible colors by humans

• Mixing any three colors in the visible range can generate colors in the triangle formed by these three points.

© Yao Wang, 2016 EL-GY 6123 from [Gonzalez02] 20 Color Gamut of Monitors and Printers

• Not all visible colors can be reproduced by RGB primaries used for display, or CMY primaries used for printing. • The triangle: gamut of color monitors using the Red, Green, and Blue colors at the corner as primary colors • The irregular region inside: gamut of most color printers • A printer cannot reproduce all colors that you see on the screen!

© Yao Wang, 2016 EL-GY 6123 21 CIE RGB Color Space

• Using single spedtral color for primary colors: – 700 nm (R), 546.1 nm (G), 435.8nm (B) • Color matching functions are the amounts of primaries needed to match each single https://en.wikipedia.org/wiki/CIE_1931_color_space

spectral color

© Yao Wang, 2016 EL-GY 6123 22 Color Primary Defined by ITU Rec. 709 for HDTV

! R $ ! 3.240479 −1.537150 −0.498535 $! X $ # & # &# & # G & = # −0.969256 1.875992 0.041556 &# Y & # & # &# Z & " B % " 0.055648 −0.204043 1.057311 %" %

! X $ ! 0.412453 0.357580 0.180423 $! R $ # & # &# & # Y & = # 0.212671 0.715160 0.072169 &# G & # Z & # &# & " % " 0.019334 0.119193 0.950277 %" B %

The three columns in the RGB-> XYZ conversion are the tristimulus values of the R,G,B primaries defined in terms of CIE XYZ primaries. From these, you could derive the chromaticity coefficients of RGB primaries and correspondingly locate them in the CIE diagram (HW!)

© Yao Wang, 2016 EL-GY 6123 23 Comparison of Colors Defined Using Different Primiaries

• To compare colors defined using different primaries, we should convert them all to CIE XYZ primaries and compare their XYZ values

© Yao Wang, 2016 EL-GY 6123 24 Color Representation Models

• Specify the tristimulus values associated with the three primary colors – RGB, CMY, XYZ • Specify the luminance and chrominance – HSI (Hue, saturation, intensity) – YIQ (used in NTSC color TV) – YCbCr (used in digital color TV) • Amplitude specification: – 8 bits for each color component, or 24 bits total for each – Total of 16 million colors – A true RGB color display of size 1Kx1K requires a display buffer memory size of 3 MB • High dynamic range (HDR) Image: up to 16 bits/component

© Yao Wang, 2016 EL-GY 6123 25 HSI

• Hue represents dominant color as perceived by an observer. It is an attribute associated with the dominant wavelength. • Saturation refers to the relative purity or the amount of white light mixed with a hue. The pure spectrum colors are fully saturated. and lavender are less saturated. • Intensity reflects the brightness.

© Yao Wang, 2016 EL-GY 6123 26 Conversion Between RGB and HSI

• Converting from RGB to HSI ⎧ 1 ⎫ [(R − G) + (R − B)] ⎧ θ if B ≤ G 1⎪ ⎪ H with cos− 2 = ⎨ θ = ⎨ 1 ⎬ ⎩360 −θ if B > G 2 ⎪ (R G) (R B)(G B) 2 ⎪ ⎩⎪[ − + − − ] ⎭⎪ 3 S = 1− [min(R,G, B)] (R + G + B) 1 I = [R + G + B] 3 • Converting from HSI to RGB RG sector (0≤H<120) GB sector (120≤H<240) BR sector (240≤H<360) B = I(1− S) R = I(1− S) G = I(1− S) ⎡ S cos H ⎤ ⎡ S cos(H −120) ⎤ ⎡ S cos(H − 240) ⎤ R I 1 G = I 1+ B = I 1+ = ⎢ + ⎥ ⎢ cos(60 (H 120))⎥ ⎢ ⎥ ⎣ cos(60 − H )⎦ ⎣ − − ⎦ ⎣ cos(60 − (H − 240))⎦ G =1− (R + B) B =1− (R + G) R =1− (G + B)

© Yao Wang, 2016 EL-GY 6123 from [Gonzalez02] 27 YIQ Color Coordinate System

• YIQ is defined by the National Television System Committee (NTSC) for US analog color TV system – Y describes the luminance, I and Q describes the chrominance. – A more compact representation of the color. – YUV plays similar role in PAL and SECAM. • Conversion between RGB and YIQ

⎡Y ⎤ ⎡0.299 0.587 0.114 ⎤⎡R⎤ ⎡R⎤ ⎡1.0 0.956 0.621 ⎤⎡Y ⎤ ⎢ I ⎥ ⎢0.596 0.274 0.322⎥⎢G⎥, ⎢G⎥ ⎢1.0 0.272 0.649⎥⎢ I ⎥ ⎢ ⎥ = ⎢ − − ⎥⎢ ⎥ ⎢ ⎥ = ⎢ − − ⎥⎢ ⎥ ⎣⎢Q⎦⎥ ⎣⎢0.211 − 0.523 0.311 ⎦⎥⎣⎢B⎦⎥ ⎣⎢B⎦⎥ ⎣⎢1.0 −1.106 1.703 ⎦⎥⎣⎢Q⎦⎥

© Yao Wang, 2016 EL-GY 6123 28 YUV/YCbCr Coordinate

• YUV is the color coordinate used in color TV in PAL system, somewhat different from YIQ. • YCbCr is the digital equivalent of YUV, used for digital TV, with 8 bit for each component, in range of 0-255

© Yao Wang, 2016 EL-GY 6123 29 Comparison of Different Color Spaces

from [Gonzalez02]

© Yao Wang, 2016 EL-GY 6123 30 Color Coordinate Conversion

• Conversion between different primary sets are linear (3x3 matrix) • Conversion between primary and XYZ/YIQ/YUV are also linear • Conversion to LSI/Lab are nonlinear – LSI and Lab coordinates • coordinate Euclidean distance proportional to actual color difference • Conversion formulae between many color coordinates can be found in [Gonzalez92] • Color space of HDTV by Recommendation 709 can be found in [Woods2012]

© Yao Wang, 2016 EL-GY 6123 31 Image Specification

• Each pixel value represents the brightness of the pixel. With 8-bit image, the pixel value of each pixel is 0 ~ 255 • Matrix representation: An image of MxN is represented by an MxN array, each element being an unsigned integer of 8 bits

⎡160 162  166 154⎤ ⎢162 158 122 69 ⎥ ⎢  ⎥ M = ⎢      ⎥ ⎢ ⎥ ⎢ 60 55  79 94 ⎥ ⎣⎢ 58 55  99 109⎦⎥

© Yao Wang, 2016 EL-GY 6123 32 Color Image Specification

• Three components – M = {R, G, B}

⎡73  87⎤ ⎡66  98⎤ ⎡31  61⎤ R ⎢ ⎥, G ⎢ ⎥, B ⎢ ⎥ = ⎢    ⎥ = ⎢    ⎥ = ⎢    ⎥ ⎣⎢27  17⎦⎥ ⎣⎢36  13⎦⎥ ⎣⎢36  14⎦⎥

G B R

Red nose is brightest! Blue Cheek is brightest © Yao Wang, 2016 EL-GY 6123 33 Chrominance Subsampling Formats

4:4:4 4:2:2 4:1:1 4:2:0 For every 2x2 Y Pixels For every 2x2 Y Pixels For every 4x1 Y Pixels For every 2x2 Y Pixels 4 Cb & 4 Cr Pixel 2 Cb & 2 Cr Pixel 1 Cb & 1 Cr Pixel 1 Cb & 1 Cr Pixel (No subsampling) (Subsampling by 2:1 (Subsampling by 4:1 (Subsampling by 2:1 both horizontally only) horizontally only) horizontally and vertically)

Y Pixel Cb and Cr Pixel

© Yao Wang, 2016 EL-GY 6123 34 Image Capture and Display

• Light reflection physics • Imaging operator • Color capture • Color display

© Yao Wang, 2016 EL-GY 6123 35 Image Capture

• Light source (λ: wavelength of the source) – E(x, y, z, λ): incident light on a point (x, y, z world coordinates of the point) • Each point of the scene has a reflectivity function. – r(x, y, z, λ): reflectivity function • Light reflects from a point and the reflected light is captured by an imaging device. – c(x, y, z, λ) = E(x, y, z, λ) * r(x, y, z, λ): reflected light.

E(x, y, z, λ) c(x, y, z, λ) = E(x, y, z, λ)*r(x, y, z, λ)

Camera( c(x, y, z, λ) )

Courtesy of Onur Guleryuz © Yao Wang, 2016 EL-GY 6123 36 More on Image/Video Capture

• Reflected light to camera – Camera absorption function (3 such functions to capture color images)

ψ (X,t) = ∫C(X,t,λ)ac(λ)dλ

– Projection from 3-D to 2-D X→x P ψ (P(X),t) =ψ (X,t) or ψ (x,t) =ψ (P−1(x),t)

• The projection operator is non-linear – Perspective projection – Othorgraphic projection

© Yao Wang, 2016 EL-GY 6123 37 Gray and Color Image Capture

• Gray images are captured by a single sensor, sensitive to the entire , similar to the rods • Color images are captured by having three sensors, each sensitive to a different , similar to the cones – Alternatively using a single sensor proceeded by different color filters • Each sensor or filter has its own frequency response, which may differ from the cones responses in the human retina.

© Yao Wang, 2016 EL-GY 6123 38 Color Imaging Using Color Filters

• Single sensor array, with different color filters to separate the RGB primary components • Sensors: CCD, CMOS

Bayer

From http://en.wikipedia.org/wiki/Bayer_filter

© Yao Wang, 2016 EL-GY 6123 39 1: Original scene; 2: Output of a 120-pixel × 80-pixel sensor with a ; 3: Output color-coded; 4: Reconstructed image after interpolating missing color information From: https://en.wikipedia.org/wiki/Bayer_filter © Yao Wang, 2016 EL-GY 6123 40 Gray and Color Image Display/Printing

• Gray images are displayed by a single light sensitive diode, with intensity proportional to gray level. • LCD monitors display color images by having three phosphors at slightly shifted positions near each pixel, each generating a different primary color (red, green, blue) • Color images are printed by having three color inks (cyan, magenta, yellow). High end printers use more inks to produce a larger color gamut and more vivid colors.

© Yao Wang, 2016 EL-GY 6123 41 Gamma Correction

• Displayed light intensity may be nonlinearly related to the actual intensity following the Gamma rule

g = af r • Gamma correction precompensate this nonlinearity

h = f 1/r

g = ahr = af

• Gamma value depends on the display device

© Yao Wang, 2016 EL-GY 6123 42 Captured and Displayed Color

• Original lighted scene has a spectrum distribution (in terms of wavelength) at a particular space and time (x,y,z,t). Call it c(l) • If the human observes it directly, the cones’ responses (each with

frequency responses ai(l)) will be Ai = ∫ C(λ)ai (λ)dλ, i = r, g, b, y • If the scene is captured by a camera, with three sensors, each with

frequency response bi(l), the captured RGB values will be

Ci = ∫ C(λ)ci (λ)dλ, i = r, g, b • The display will use three phosphors, each with different spectrum

di(l) and the displayed spectrum is the weighted sum of the three

spectrums ! ! = !!!! ! ! ! • The displayed spectrum again is viewed by the human, generating

three other responses Bi = ∫ D(λ)ai (λ)dλ, i = r, g, b, y

• Bi should be close to Ai for the perceived scene to be similar to the original scene

© Yao Wang, 2016 EL-GY 6123 43 Summary (Color Image Perception, Production, Representation, Capture and Display)

• What is light? Difference between illuminating and reflecting light. Which light attribute determines the color? • How human perceive color? Functions of cones and rods in the retina. • Three attributes of color: luminance, hue, saturation • How to produce different colors ? • How to represent a color? Different color models. • Color image representation using a matrix • Color image capture using color filters • Color image display

© Yao Wang, 2016 EL-GY 6123 44 MATLAB For Image Processing

• Toolbox: images • Useful functions: • Imread( ), Imwrite( ), Imshow( ), Imresize( ) • Plot, subplot • Color conversion: rgb2ycbcr( ), rgb2hsv( ), etc. • Use “help images” to see all functions • Use “help imshow” to see all options of imshow

© Yao Wang, 2016 EL-GY 6123 45 Some Caveats to Remember!

• Images are generally stored in array of unit8 (8 bit unsigned integer) • Convert to integer or double before any processing! – Img=imread(‘lena.jpg’); – Img=int8(img) or img=double(img) • Array index starts at 1, not 0! Top-left pixel is img(1,1); • Convert back to uint8 when saving an image – First scale the processed image so that min=0, max=255, change the type to uint8 – Then use imwrite( ) • Image display: Imshow(), imagesc() – Be very careful when using imshow( ) on images of single/double. Default range may not show all possible values – Imshow(img, [low, high]): low: black, high: white – Imshow(img,[ ]): low=min, high=max

© Yao Wang, 2016 EL-GY 6123 46 Image Contrast Enhancement

• Introduction • Linear stretching • Nonlinear stretching • Histogram equalization • Histogram specification • Adaptive histogram modification

© Yao Wang, 2016 EL-GY 6123 47 What is Contrast Enhancement

Original image with low contrast Enhanced image

© Yao Wang, 2016 EL-GY 6123 48 How to enhance the contrast?

• Low contrast à image values concentrated near a narrow range (mostly dark, or mostly bright, or mostly medium values) • Contrast enhancement à change the image value distribution to cover a wide range • Contrast of an image can be revealed by its histogram

© Yao Wang, 2016 EL-GY 6123 49 Histogram

• Histogram of a monochrome image with L possible gray levels, l = 0, 1, …, L-1.

– P(l) = nl / n,

• nl is the number of pixels with gray level l. • n is the total number of pixels in the image.

© Yao Wang, 2016 EL-GY 6123 50 Histogram vs. Image Contrast

Images with figure captions in this and other slides are from [Gonzalez08]

© Yao Wang, 2016 EL-GY 6123 51 Histogram Calculation

function h = histogram(imgname) img = imread(imgname); % method 2 figure; imshow(img); img = int8(img);

h = zeros(256,1); % method 1 for i=1:M, h = zeros(256,1); for j=1:N, for l = 0:255 f = img(i,j); for i = 1:N, h(f+1) = h(f+1) + 1; for j = 1:M, end if img(i, j) == l, end h(l + 1) = h(l + 1) + 1;

end % method 3 end h = zeros(256,1); end for l = 0 : 255, end h(l + 1)=sum(sum(img == l)); figure; bar(h); end

Photoshop has extensive histogram display tools Matlab: imhist( ): can compute and display histograms

© Yao Wang, 2016 EL-GY 6123 52 Very Different Images May Have Same Histogram!

Histogram reflects the pixel intensity distribution, not the spatial distribution!

© Yao Wang, 2016 EL-GY 6123 53 Previous Example

Original image with low contrast Enhanced image

© Yao Wang, 2016 EL-GY 6123 54 Histograms of Example Images

1600 1600 1400 1400 1200 1200 1000 1000

800 800

600 600

400 400

200 200

0 0

0 50 100 150 200 250 0 50 100 150 200 250

Original girl image with low Enhancement image with contrast histogram equalization

© Yao Wang, 2016 EL-GY 6123 55 How to change the histogram? Using Point-Wise Transformation

• Use a “function” g(f) to generate a new image B from a given image A via:

B(i, j) = g(A(i, j)), i = 0,..., N −1, j = 0,...,M −1

• The function g(f) operates on each image pixel independently. All pixels with original gray level f are changed to have gray level g(f) • Properties that g(f) should satisfy – Monotonically non-decreasing, so that relative brightness of pixels do not change. – G(f) in the same range as original f, i.e. with same min (e.g. 0) and max values (e.g. 255), and be integers for digital images. • Rounding/truncation may be needed

© Yao Wang, 2016 EL-GY 6123 56 How to Determine the Transformation Function?

• How to design the transformation function g(f)?

– depends on the histogram of the original image hA(f) and the desired histogram of the transformed image hB(f).

– To enhance contrast, we like hB(f) to be as flat as possible. • Different approaches – Using fixed functional forms: linear, non-linear

– Using adaptive transform, that is determined from hA(f) and hB(f):

• Histogram equalitzation (hB(f) is uniform): Fully automatic! • Histogram specification or matching

© Yao Wang, 2016 EL-GY 6123 57 Illustration of Linear Stretching

p(f) g(f) p(g) 255

f f g 100 150 100 150 255

Linear stretching

© Yao Wang, 2016 EL-GY 6123 58 Piece-Wise Linear for Middle Range Image

© Yao Wang, 2016 EL-GY 6123 59 Nonlinear Stretching

• Nonlinear functions with a fixed form • Fewer parameters to adjust

• Satisfying 0 = fmin ≤ g ≤ fmax = L −1 • Examples – Logarithmic transformation g = blog(af +1) • Stretch dark region, suppress bright region – Exponential transformation g = b(eaf −1) • Expand bright region k – Power Law g = af • K = 2: square law, similar to exponential • K = 1/3: cubic root, similar to logarithmic

© Yao Wang, 2016 EL-GY 6123 60 Example of Log Transformation

Eq. (3.2-2) “log” function: g=c log (1+f)

© Yao Wang, 2016 EL-GY 6123 61 Power Law (Gamma) Transformations

© Yao Wang, 2016 EL-GY 6123 62 Histogram Equalization

• Transforms an image with an arbitrary histogram to one with a flat histogram

– Suppose f has PDF pF(f), 0 ≤ f ≤ 1 – Transform function (continuous version) f g( f ) = pF (t)dt – g is uniformly distributed in (0, 1) ∫0

Histogram Equalization

© Yao Wang, 2016 EL-GY 6123 63 Proof

f g( f ) = pF (t)dt, ∫fmin

pF ( f ) pG (g) = , g ∈(0,1) dg df dg = p ( f ) df F

pG (g) = 1, g ∈(0,1)

© Yao Wang, 2016 EL-GY 6123 64 Example

⎧ 4 f f ∈(0,1/ 2) pF ( f ) = ⎨ ⎩4(1− f ) f ∈(1/ 2,1) f ⎧ 4 fdf = 2 f 2 f ∈(0,1/ 2) ⎪ ∫0 g( f ) = 1/ 2 f ⎨ 2 ⎪ 4 fdf + 4(1− f )df =1− 2( f −1) f ∈(1/ 2,1) ⎩∫0 ∫1/ 2

pG (g) =1, g ∈(0,1)

p (f) g(f) p (g) F 1 G

2 1/2 1

f f 0 1/2 1 0 1/2 1 0 1 g

© Yao Wang, 2016 EL-GY 6123 65 Discrete Implementation

• For a discrete image f which takes values k=0,…,K-1, use

l ~ g(l) = ∑ pF (k),l = 0,1,...,K −1. k=0 • To convert the transformed values to the range of (0, L-1):

l ⎪⎧⎛ ⎞ ⎪⎫ g(l) = round ⎜ p (k)⎟*(L −1) ⎨⎜∑ F ⎟ ⎬ ⎩⎪⎝ k=0 ⎠ ⎭⎪ – Note: {x} is the rounding of x

© Yao Wang, 2016 EL-GY 6123 66 Example

l g~ = p (k) g [g~ *7] fk pF(l) l ∑k=0 F l = l pG(l) gk 0 0.19 0.19 [1.33]=1 0 0 1 0.25 0.44 [3.08]=3 0.19 1 2 0.21 0.65 [4.55]=5 0 2 3 0.16 0.81 [5.67]=6 0.25 3 4 0.08 0.89 [6.03]=6 0 4 5 0.06 0.95 [6.65]=7 0.21 5 6 0.03 0.98 [6.86]=7 0.16+0.08=0.24 6 7 0.02 1.00 [7]=7 0.06+0.03+0.02=0.11 7

pF(l) pG(l)

l l 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 We don’t get perfectly flat histogram © Yao Wang, 2016 EL-GY 6123 with discrete implementation! 67 Sample Matlab Code

function histogram_eq(inimgname)

img=imread(imgname); %perform mapping figure; imshow(img); for (i=1:M) [M,N]=size(img); for (j=1:N) f=int8(img(i,j))+1; histeqimg(i,j)=C(f); H=imhist(img); end; H=H/(M*N); end; figure; bar(H);

%note the above loop can be replaced by: %Computing the mapping function %histeqimg=C(int8(img)+1); for (k=1:256) %this will be much faster! C(k)=uint8(sum(H(1:k))*255); end; figure; % C = uint8(cumsum(H)*255); imshow(histeqimg);

figure;plot(C);

© Yao Wang, 2016 EL-GY 6123 68 Adaptive Histogram Modification

• Local histogram equalization

© Yao Wang, 2016 EL-GY 6123 69 Contrast Enhancement for Color Images

• How should we apply the previous techniques to color images – To all three color components (RGB or CMY) separately – To the Intensity component only while keeping the Hue and Saturation in the HSI coordinate – Can also enhance saturation for more vivid colors – Can change individual color components to add certain tone to an image

© Yao Wang, 2016 EL-GY 6123 70 Examples for Color Image (1)

From [Gaonzalez08] © Yao Wang, 2016 EL-GY 6123 71 Examples for Color Image (2)

From [Gaonzalez08] © Yao Wang, 2016 EL-GY 6123 72 MATLAB functions

• Imhist( ) • Histeq() • Adapthisteq( ) • Imadjust( ) • Imcontrast() • Brighten() • stretchlim()

• Use help to learn what does each do, and play around with these functions to see the effect

© Yao Wang, 2016 EL-GY 6123 73 Summary (Contrast Enhancement)

• What is image histogram? • How to tell whether an an image have a good contrast from its histogram? • Given the histogram of an image, can you sketch a transformation that will likely improve the image contrast. • The principle of histogram equalization • The principle of histogram specification • Color image enhancement

© Yao Wang, 2016 EL-GY 6123 74 Reading Assignments

• Image formation, perception and representation: – Wang et al, Digital video processing and communications. Chap 1 (Sec. 1.3-1.4 optional) – Or – Woods, Multidimensional signal, image, and video processing and coding, Chap 6. (Sec. 6.2-6.3 optional) • Contrast enhancement – Gonzalez & Woods, “ Processing”, Prentice Hall, 2008, Chapter 3 (Section 3.1 – 3.3)

© Yao Wang, 2016 EL-GY 6123 75 Written Homework

1. Consider the Rec. 709 color primaries. Determine the chromaticity coefficients (x,y) associated with these primary colors and roughly indicate their location in the CIE chromaticity diagram. 2. Following figure shows the histogram of an image and two transformation functions. Sketch the histograms of the images obtained by applying the two functions to the original image.

g2(f) 255 h(f) g1(f) a 128 128

0 128 255 f 0 128 255 f 0 128 255 f

3. Following figure (next slide) shows the histograms of three different images, and three possible point functions. For each original image, which point operation can best equalize its histogram? Briefly explain your reasoning.

© Yao Wang, 2016 EL-GY 6123 76

Written Homework (cnt’d)

3. For the histogram h3(f) given in the following figure, determine analytically the histogram equalizing function (assuming the original image value can vary continuously in the range of 0-255). 4. An 8-level images have histograms h=[.1, .2, .3, 0, .1, 0, 0, .3]. Find a point function g(f) that will equalize this histogram. Show the resulting mapping function and the histogram. Is the resulting histogram flat? If not, what possibe mechanism you can use to make it better? h (f) h (f) 1 2 h3(f) a a a

0 128 255 f 0 64 192 f 0 128 255 f g (f) g (f) g (f) 1 2 3 255 255 255

128 128 128

0 64 192 255 f 0 128 255 f 0 128 255 f

© Yao Wang, 2016 EL-GY 6123 77 MATLAB Assignment

1. If you do not have a copy of MATLAB, get one installed by IS helpdesk or use the PC Lab at Poly. If you have not used Matlab before, go through Matlab getting started guide at http://www.mathworks.com/help/pdf_doc/matlab/getstart.pdf. Go through all exercises step by step. Make sure that you know basic operations and how to write m-files (scripts or functions). 2. If you have not used MATLAB for image processing, go through the supplementary document on course website (MATLAB for image manipulation) 3. Take a picture of yourself or anything interesting and upload to your computer. 4. Write a matlab program that can i) read your color image into a matrix (you can use imread() function in MATLAB); ii) covert it to a 8-bit grayscale image using the RGB to Y conversion formula in the RGB to YCbCr formula provided (You should not use the MATLAB built-in function rgb2ycbcr), iii) generate a digital negative version of your grayscale image, and iv) display both the color image, the original gray scale and the negative image. REMEMBER THE CAVEATS! 5. Write a Matlab function that can compute the histogram of a grayscale image (assuming 256 levels of gray). Try the three possible ways described in slide 7, and see which one is faster. Finalize your program with the fastest method. In a separate main program, apply the program to a test image, and display the histogram as a stem plot besides the image (using “subplot” function). You are not allowed to simply use the “imhist” or “hist” function in Matlab, although you are encouraged to compare your results with those obtained using these functions.

© Yao Wang, 2016 EL-GY 6123 78 MATLAB Assignment (cnt’d)

5. Write a Matlab program that performs histogram equalization on a grayscale image. Your program should: i) compute the histogram of the input image by calling your own histogram function from previous problem; ii) compute the histogram equalizing transformation function; iii) apply the function to the input image; iv) compute the histogram of the equalized image; v) display (and print) the original and equalized images, as well as their corresponding histograms, all in one figure. You are not allowed to simply use the “histeq” function in Matlab, although you are encouraged to compare your results with those obtained using these functions.

For Prob. 4 and 5: try to capture an image with low contrast so that you could see the effect of contrast enhancement. Alternatively you could use the test images provided in the class homepage.

Note: For this (and future) computer assignment, you should submit a lab report that includes a copy of your matlab programs, the pictures you created, as well as a description of your findings if applicable. All printed figures should have a caption that explains what they are. You should submit a printed copy of your report together with your other written homework.

© Yao Wang, 2016 EL-GY 6123 79