Image Enhancement and Restoration Algorithms

CCHCS M.Sc- Computer Vision

Image enhancement and filtering

Introduction

Images may suffer from the following degradations:

· Poor contrast due to poor illumination or finite sensitivity of the imaging device.

· Electronic sensor noise or atmospheric disturbances leading to broad band noise.

· Aliasing effects due to inadequate sampling.

· Finite aperture effects or motion leading to spatial blurring.

We will consider simple algorithms for image enhancement based on lookup tables and image restoration based on filtering which can improve the visual quality of images with these degradations.

Image enhancement using histogram equalisation

In an image of low contrast, the image has grey levels concentrated in a narrow band.

Consider the grey level histogram h(i ) of such an image where :

h(i ) = number of pixels with grey level i

We want to 'stretch' the histogram to the full (0-255) dynamic range. The simplest technique is to use a sigmoid lookup table to map input to output grey levels :

h' (i )

h' (i )  h( g 1(i ))

i 1 g(i )  1 exp  i sigmoid function :

 controls the position of maximum slope

 controls the slope

Problem - we need to determine the optimum sigmoid parameters  and  for each image.

A better method would be to determine the best mapping function g(i ) from the image data. A general histogram stretching algorithm

We require a transformation g(i ) such that from any histogram h(i ) :

h' (i )   h( j )  constant j:i g( j )

Constraints :

255 1.  h' (i )  N 2 (for an N x N , 8 bit image) i0

2. i1  i2  g(i1 )  g(i2 )

g(i) is an ordered transformation - the grey levels in the output histogram should not 'cross over'. The required transformation is based on the cumulative histogram H(i ) . H(i ) = number of pixels with grey levels  i i   h( j ) j0

If the required histogram is flat, the required cumulative histogram will be a ramp :

Let the actual histogram and cumulative histogram be h(i ) and H(i ).

Let the desired histogram and desired cumulative histogram be h' (i ) and H' (i ).

Let the transformation be g(i ).

For an N x N, 8-bit image :

N 2 g(i ) H' ( g(i ))  255 ( H' (255)  N 2 ,H' (0)  0)

Key point :

g(i ) is an ordered transformation.

i1  i2  g(i1 )  g(i2 )

Therefore :

N 2 g(i ) H' ( g(i ))  H(i )  255 Thus : 255H(i ) g(i )  N 2 h(i ) H(i )

i i h' ( g(i )) H' ( g(i )) i  g(i )

g(i ) g(i ) Worked example

32 x 32 bit image with grey levels quantised to 3 bits.

7H(i ) g(i )  1024

h' (i )   h( j ) j:i g( j )

i h(i ) H(i ) g(i ) h' (i ) 0 197 197 1.351 - 1 256 453 3.103 197 2 212 665 4.555 - 3 164 829 5.676 256 4 82 911 6.236 - 5 62 993 6.657 212 6 31 1004 6.867 246 7 20 1024 7.07 113

Original histogram 150 Stretched histogram

0 0 1 2 3 4 5 6 7 Examples on real images

Histogram before stretching : h(i) 2000.00

1500.00

1000.00

500.00

i 0.00 0.00 50.00 100.00 150.00 200.00 250.00 Histogram after stretching : h(i) 2000.00

1500.00

1000.00

500.00

i 0.00 0.00 50.00 100.00 150.00 200.00 250.00 Histogram before stretching h(i) 3000.00

2500.00

2000.00

1500.00

1000.00

500.00

i 0.00 0.00 50.00 100.00 150.00 200.00 250.00 Histogram after stretching h(i) 3000.00

2500.00

2000.00

1500.00

1000.00

500.00

i 0.00 0.00 50.00 100.00 150.00 200.00 250.00 Image filtering

Introduction

Simple image operators can be classified as 'pointwise' or 'neighbourhood' (filtering) operators.

I n p u t i m a g e O u t p u t i m a g e

( x , y ) ( x , y )

p o i n t w i s e t r a n s f o r m a t i o n

I n p u t i m a g e O u t p u t i m a g e

( x , y ) ( x , y )

n e i g h b o u r h o o d t r a n s f o r m a t i o n

Co-ordinates ( x,y ) refer to the discrete image grid. Thus x  0,1,2..... N  1, y  0,1,2..... N  1. Types of filtering operations

The output g(x,y) can be a linear or non-linear function of the set of input pixel grey levels  f ( x  M , y  M ).. f ( x  M , y  M ) .

I n p u t i m a g e f ( x , y ) O u t p u t i m a g e g ( x , y )

( x - 1 , y - 1 )

( x , y ) ( x , y )

( x + 1 , y + 1 )

Examples

g( x,y )  h1 f ( x  1,y  1) h2 f ( x,y  1)

.....h9 f ( x  1,y  1)

This defines a linear filter with the h1 ..h9 being the filter coefficients.  f ( x  1, y  1), f ( x, y  1) g( x, y )  median   ..... f ( x  1, y  1) 

This defines a median filter which is useful for removing some types of noise from images. Linear filtering and convolution

Suppose we wanted to create an output image consisting of the local 3x3 arithmetic means of an input image (ignoring floating point ® byte rounding).

( x - 1 , y - 1 )

( x , y ) ( x , y )

( x + 1 , y + 1 )

1 g( x,y )  ( f ( x  1,y  1)  f ( x,y  1)  9 ..... f ( x  1,y  1)) We can define a 3x3 'convolution mask' H :

 1 1 1   9 9 9  1 1 1 H     9 9 9  1 1 1    9 9 9

Place the central point of the mask at pixel (x,y).

Multiply and add the overlapping mask and image points to produce the output image value at pixel (x,y).

( x , y ) ( x , y ) 

I m a g e p o i n t F i l t e r m a s k p o i n t Mathematically, we can write this operation as follows :

1 1 g( x,y )    h( x' ,y' ) f ( x  x' ,y  y' ) x'  1 y'  1 1 1 1    f ( x  x' ,y  y' ) 9 x'  1 y'  1

The first equation generally describes convolution of an input image f ( x,y ) with a filter h( x, y ). Example

An image corrupted with additive broad-band noise can be smoothed with a Gaussian low-pass filter.

( x 2  y 2 ) g( x,y )  exp(  ) 22  g( x )g( y ) x 2 g( x )  exp(  ) 2 2 where  determines the width of the filter and hence the amount of smoothing.

g(x) 1.00

0.60 

0.00 -6 -4 -2 0 2 4 x Original image. Noise corrupted image

Gaussian filtered   1.5 Gaussian filtered   3.0 Convolution in the frequency domain

An N  N image can be represented in the spatial domain as f ( x,y ) or in the frequency domain by F(u,v )which is the discrete Fourier transform (DFT) of f ( x,y ) .

N 1 N 1 2j F(u,v )    f ( x,y )exp(  (ux  vy )) x0 y0 N

u,v  0..N  1

The inverse DFT is given by :

1 N 1 N 1 2j f ( x,y )  2   F(u,v )exp( (ux  vy )) N x0 y0 N x,y  0..N  1

f ( x,y ) is the grey level of the image at position ( x, y ). F(u,v ) is the frequency content of the image at spatial frequency position (u,v ).

Smooth regions of the image contribute low frequency components to F(u,v ). Abrupt transitions in grey level (lines and edges) contribute high frequency components to F(u,v ). ( 0 . 0 ) x

f ( x , y )

( N - 1 , N - 1 )

y D F T I D F T ( 0 , 0 ) u

F ( u , v )

( N - 1 , N - 1 )

v Example

Input image f ( x,y).

Log magnitude of the DFT log(1 F(u,v ) ).

(The origin of the (u,v ) axis is the centre of the image.) Computation of the DFT

An N  N point DFT would require N 2 floating point multiplications per output point. Since there are N 2 output points , the computational complexity of the DFT is N 4 .

N 4  4  109 for N  256

- many hours on a workstation!

In the 1960's the FFT algorithm was developed. An N  N 2 point DFT only requires 2N log2 N multiplications using the FFT algorithm.

2 6 2N log2 N  10 for N  256

- only a few minutes on a workstation. Convolution theorem

Suppose we have an N  N image f ( x,y) to be convolved with an M  M convolution mask h( x, y ) to produce an output g( x, y ).

M 1 M 1 g( x,y )    h( x' ,y' ) f ( x  x' ,y  y' ) x'  0 y' 0  f ( x,y )* h( x,y )

This is a slightly different definition of the convolution. The mask h( x, y ) is shifted and inverted before overlapping and summing.

Let G( u,v ), F(u,v ), H( u,v ) be DFT’s of g( x, y ), f ( x, y ),h( x, y ). G(u,v )  H(u,v )F(u,v )

Convolution theorem.

This provides the 'filtering' interpretation of convolution. We can demonstrate this filtering interpretation by performing a simple operation on the DFT of f ( x,y). Suppose we set all values in F( u,v ) to zero for spatial frequencies u,v such that u2  v 2  R where R defines the bandwidth of the filter.

This is equivalent to multiplying F( u,v ) by H(u,v ) where :

H(u,v )  0 for u2  v 2  R  1 otherwise G(u,v )  H(u,v )F(u,v ) g( x,y )  h( x,y )* f ( x,y ) DFT

Only high frequency regions, corresponding to edge regions in the original image, remain. We could get the same result by convolving the image with h( x,y ) which, in this case, would be an edge detector (see later.)

( x , y ) ( x , y ) 

F i l t e r m a s k h ( x , y ) Implementation

G( u,v ), F(u,v ), H( u,v ) are N  N DFT’s of g( x, y ), f ( x, y ),h( x, y ).

Usually M  N . In this case the M  M filter mask is 'zero- padded' out to N  N .

The output image g( x,y ) is of size N  M  1 N  M  1. The filter mask ‘wraps around’ truncating g( x,y ) to an N  N image? F i l t e r m a s k h ( x , y )

z e r o p a d d i n g I n p u t i m a g e f ( x , y )

x x x x x

D F T D F T H ( u , v ) F ( u , v )

H ( u , v ) F ( u , v ) f ( x , y ) * h ( x , y )

I D F T I n p u t i m a g e f ( x , y ) O u t p u t i m a g e g ( x , y )

( x , y ) ( x ' , y ' )

x ' = x m o d u l o N F i l t e r m a s k h ( x , y ) y ' = y m o d u l o N Computational complexity of convolution (Spatial domain v frequency domain)

N  N image is to be convolved with an M  M filter. Spatial domain convolution requires M 2 floating point multiplications per output point or N 2 M 2 in total.

Frequency domain implementation requires 2 2 3 (2N log2 N )  N floating point multiplications ( 2 DFTs + 1 IDFT + N 2 multiplications of the DFTs).

Example

N  512, M  7

Spatial domain implementation requires 1.3  107 floating point multiplications.

Frequency domain implementation requires 1.4  107 floating point multiplications.

N  512, M  32

Spatial domain implementation requires 2.7  108 floating point multiplications.

Frequency domain implementation requires 1.4  107 floating point multiplications.

For smaller mask sizes, it would appear that spatial or frequency domain implementations have about the same computational complexity. However, we can speed up the frequency domain implementation significantly by splitting the image into blocks and filtering the blocks separately in the frequency domain. Examples of simple filters

We can consider some simple filters in the spatial and frequency domains.

1. Smoothing (low-pass) filter

1 1 1 1 1   1 1 1 1 1 1 1 1     1 1   H3  1 1 1 H5  1 1 1 1 1 etc 9   25   1 1 1 1 1 1 1 1   1 1 1 1 1

These masks are useful for smoothing images corrupted by additive broad band noise. Spatial domain

h( x )

x Frequency domain

This has the frequency characteristic of a low-pass (smoothing filter) but notice the ‘ringing’ at high frequencies. This explains why more rounded filters (such as the Gaussian) are preferred. 2. Simple edge detector

1 0  1   H  1 0  1   1 0  1

H responds to horizontal edges

- positive response Þ left to right edge

- negative response Þright to left edge

f ( x )

x f ( x )*( 1 0 1)

x We can evaluate the DFT of the 1-D edge detector h( x )  (1 0 -1) very easily.

N 1 2jux H(u )   h( x )exp(  ) x0 N 4ju  1 exp(  ) N 2ju  2ju 2ju   exp(  ) exp( ) exp(  ) N  N N  2ju 2u  2 j exp(  )sin( ) N N The magnitude of the response is thus :

2u H(u )  sin( ) N H(u )

The frequency response has a band-pass characteristic. Thus the filter responds to areas in the image containing medium frequencies such as edges and texture and has zero response to areas containing a slowly varying greylevel.