Image Enhancement
Introduction to Signal and Image Processing
Prof. Dr. Philippe Cattin
MIAC, University of Basel
April 5th/12th, 2016
Introduction to Signal and Image Processing April 5th/12th, 2016
1 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Contents
Contents
Abstract 2 1 Geometrical Image Corrections 1.1 Geometrical Image Transformations Introduction 5 Geometrical Image Transformations 6 Basic Steps of a Geometric Transform 7 The Hierarchy of Geometric Transformations 8 Class I: Isometries 9 Class I: Isometries (2) 10 Class I: Isometries Example 11 Class I: Practical Issues 12 Class II: Similarity Transformations 13 Class II: Similarity Transformations (2) 14 Class II: Similarity Transformations Example 15 Class II: Practical Issues 16 Class III: Affine Transformations 17 Class III: Affine Transformations (2) 18 Class III: Affine Transformation Example 19 Class III: Practical Issues 20 Class IV: Projective Transformations 21 Class IV: Projective Transformations (2) 22 Class IV: Projective Transformation Example 23 Practical Issues and Common Pitfalls 24 Determination of the Matrix Coefficients 25 Determination of the Matrix Coefficients (2) 26 1.2 Pixel Interpolation Methods Applying Spatial Transformations to Images 28 Introduction to Signal and Image Processing April 5th/12th, 2016
2 of 188 22.02.2016 09:18 Forward Mapping 29 Forward Mapping Example 30 Backward Mapping 31 Interpolation Methods 32 Nearest Neighbour Interpolation 33 Nearest Neighbour Interpolation Example 34 Bilinear Interpolation 35 Bilinear Interpolation Example 36 Cubic Convolution Interpolation 37 Cubic Convolution Interpolation (2) 38 Cubic Convolution Interpolation Principle 39 What Interpolation Kernels are There? 40 Conditions for Cubic Convolution 41 Interpolation Cubic Convolution Kernel with Order L=3 42 Cubic Convolution Kernel with Order L=4 43 Bi-Cubic Convolution Interpolation 44 Bi-Cubic Convolution Interpolation Example 45 Summary Cubic Convolution Interpolation 46 Comparison of NN, Bilinear, and Bi-Cubic 47 Interpol Comparison Bilinear vs Bi-Cubic Convolution48 Comparison NN, BL, CC & Sinc 49 Interpolation References 50 1.3 Camera Calibration Camera Calibration 52 The Camera Model 53 The Camera Model (2) 54 The Camera Model (3) 55 The Camera Model (4) 56 The Camera Model (5) 57 Introduction to SignalExtrinsic and Image Camera Processing Parameters April 5th/12th, 201658
3 of 188 22.02.2016 09:18 Intrinsic Camera Parameters 59 How-To Determine the Camera Parameters 60 Camera Calibration Example 61 References 62 2 Filtering in the Spatial Domain 2.1 Fundamentals of Spatial Filtering Background 65 Spatial Domain Methods 66 Spatial Domain Methods (2) 67 Spatial Filtering 68 Linear Spatial Filter 69 Non-Linear Spatial Filter 70 Solutions to the Border Problem 71 2.2 Basic Grey-Level Transformations Intensity Transformation Functions 73 Image Negatives 74 Gamma Correction 75 Gamma Correction of Displays 76 Gamma Correction Example 77 Gamma Correction Example (2) 78 Contrast Stretching Problem 79 Contrast Stretching 80 Contrast Stretching Example 81 Thresholding 82 General Thresholding 83 General Thresholding Example 84 Dynamic Range Compression 85 2.3 Histogram Processing Histogram Equalisation 87 Histogram Equalisation (2) 88 89 Introduction to SignalHistogram and Image Equalisation Processing (4) April 5th/12th, 201690
4 of 188 22.02.2016 09:18 Histogram Equalisation (5) 91 Histogram Equalisation Example 92 Histogram Equalisation Example (2) 93 Histogram Equalisation vs. Homomorphic 94 Filtering Histogram Specification 95 Local Histogram Enhancement 96 Local Histogram Enhancement (2) 97 But Beware... 98 2.4 Image Averaging Image Averaging 100 Image Averaging Example 101 2.5 Smoothing Spatial Filters Smoothing Spatial Filters 103 2.5.1 General Properties General Properties of Smoothing 105 Filters Zero Shift 106 Preservation of the Mean 107 Isotropy 108 Monotonically Decreasing Transfer 109 Function 2.5.2 Box Filter Box Filter (1) 111 Box Filter Example 112 What is the Problem with the Box 113 Filter? Where do these Phase Reversals 114 come from?? 2.5.3 Lowpass Gauss Filter Lowpass Gauss Filter 116 Lowpass Gauss Filter Example 117
Introduction to Signal andGauss Image FilterProcessing on the Conic AprilCircle 5th/12th, 2016118
5 of 188 22.02.2016 09:18 Example Problem with Linear Spatial Filtering 119 2.6 Sharpening Spatial Filters Foundation for Sharpening Spatial Filters 121 Foundation for Sharpening Spatial Filters 122 (2) Using the 2nd Derivative (Laplacian) for 123 Sharpening Other Reasoning for Sharpening Spatial 124 Filters Basic Highpass Spatial Filtering 125 Unsharp Masking and High-boost Filtering 126 High-boost Filtering Example 127 2.7 Mean Filters Mean Filters 129 Arithmetic Mean Filter 130 Geometric Mean Filter 131 Arithmetic vs. Geometric Mean Filter 132 Arithmetic vs. Geometric Mean Filter (2) 133 Harmonic Mean Filter 134 Contraharmonic Mean Filter 135 Contraharmonic Mean Filter Example (1) 136 Contraharmonic Mean Filter Example (2) 137 Contraharmonic Mean Filter Example (3) 138 Mean Filter Discussion 139 2.8 Order-Statistics Filters Order-Statistics Filter 141 Median Filter 142 Median Filter (2) 143 Median Filter (3) 144 Median vs Arithmetic Mean vs Geometric 145 Mean Filter Introduction to SignalMedian and Image Filter Processing Example April 5th/12th, 2016146
6 of 188 22.02.2016 09:18 Min and Max Filters 147 Min and Max Filter Example 148 Midpoint Filter 149 Midpoint Filter Example 150 Alpha-Trimmed Mean Filter 151 Alpha-Trimmed Mean vs Arith. Mean vs. 152 Median Filter 2.9 Adaptive Filters Adaptive Filters 154 Adaptive, Local Noise Reduction Filters 155 Adaptive, Local Noise Reduction Filters (2) 156 Remarks 157 Adaptive, Local Noise Reduction Filter 158 Example Adaptive Median Filter 159 Adaptive Median Filter Algorithm 160 Adaptive Median Filter Example 161 3 Filtering in the Frequency Domain Background 163 3.1 Introduction Frequency Domain Methods Principle of Frequency Domain Methods 165 3.2 Smoothing Frequency Domain Filters Smoothing Frequency Domain Filters 167 (Lowpass) Ideal Lowpass Filter 168 Ideal Lowpass Filter Example 169 Ideal Lowpass Filter Discussion 170 ILPF Transfer Function in the Spatial 171 Domain A Simple ILPF Example 172 Butterworth Lowpass Filter 173 Butterworth Lowpass Filter Example 174 Introduction to SignalGaussian and Image Lowpass Processing Filter April 5th/12th, 2016175
7 of 188 22.02.2016 09:18 Gaussian Lowpass Filter Example 176 3.3 Sharpening Frequency Domain Filters Sharpening Frequency Domain Filters 178 Ideal Highpass Filter 179 Ideal Highpass Filter Example 180 Butterworth Highpass Filter 181 Butterworth Highpass Filter Example 182 Gaussian Highpass Filter 183 Gaussian Highpass Filter Example 184 The Laplacian in the Frequency Domain 185 Laplacian Example 186 Laplacian Highboost Filter Example 187 Unsharp Masking and High-boost Filtering 188 Unsharp Masking and Highboost Filtering 189 Example 3.4 Homomorphic Filtering Homomorphic Filtering 191 Homomorphic Filtering (2) 192 Homomorphic Filtering Example 193 Homomorphic Filtering Example (2) 194
Introduction to Signal and Image Processing April 5th/12th, 2016
8 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement
Abstract (2)
The principal objective of the enhancements techniques is to process the images so that the results are more suitable than the original images for a specific application. The approaches discussed in this chapter fall mainly the categoriy of geometrical image corrections.
Introduction to Signal and Image Processing April 5th/12th, 2016
9 of 188 22.02.2016 09:18 Geometrical Image Corrections
Geometrical Image Transformations
Introduction (5)
Almost all capturing processes involve unwanted geometric transforms. They are caused by
Perspective distortions Optical distortions due to lens errors or aberration Capturing process inherent limitations and so forth
Geometric transforms permit to eliminate, to a large extent, these distortions. Only after correcting these errors it would be possible to
Derive accurate metric measurements from the images Compare the same or similar objects in different images
Introduction to Signal and Image Processing April 5th/12th, 2016
10 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Geometrical Image Transformations
Geometrical Image (6) Transformations
Definition:
A geometric transform is a vector function that maps all the pixels in the source image to a new Fig 5.1: Geometrical position in the transform rectified coordinate system with .
The transformation is either known in advance or can be determined from several known pixel correspondences in an original and transformed image pair.
Depending on the geometrical distortion one has to select the most appropriate geometrical transformation from a class of transformations.
Introduction to Signal and Image Processing April 5th/12th, 2016
11 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Geometrical Image Transformations
Basic Steps of a Geometric (7) Transform
Such geometric transforms consist of two basic steps
1. Determining the pixel coordinates in the transformed image Mapping of the coordinates in the input image to Fig 5.2: Geometrical the point in the output transform image The output coordinates generally don't fall onto exact pixel coordinates 2. Determining the point in the digital raster which matches the transformed point and determining its brightness/colour Brightness/colour is usually computed as an interpolation of several points in the neighbourhood
Introduction to Signal and Image Processing April 5th/12th, 2016
12 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Geometrical Image Transformations
The Hierarchy of Geometric (8) Transformations
The projective transformations form in a mathematical sense a group known as the projective linear group. This group can be split into various sub-groups with special properties and increasing complexity (degrees-of-freedom DOF).
Class I: Isometries (translation, rotation, rigid) Class II: Similarity transformations Class III: Affine transformations Class IV: Projective transformations
The most general case of geometric transformation is the
Free-form or curved transformation
It, however, doesn't fit in the nice mathematical framework of the projective linear group.
Introduction to Signal and Image Processing April 5th/12th, 2016
13 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Geometrical Image Transformations
Class I: Isometries (9)
Isometries are transformations of the plane that preserve Euclidean distance thus the term isometry (from iso = same, metric = measure).
Fig 5.3: Isometry = Translation + Rotation (5.1)
where . If then the isometry is orientation- preserving and is composed of a translation and rotation. If then the isometry reverses orientation.
The above equation can be written more concisely in block form as
(5.2)
where is a (orthogonal) rotation matrix, a translation vector, and a null vector.
Introduction to Signal and Image Processing April 5th/12th, 2016
14 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Geometrical Image Transformations
Class I: Isometries (2) (10)
Degrees of Freedom
Isometries have 3 degrees-of-freedom (DOF): one for rotation ( ) and two for translation The Isometry can be computed from two point correspondences
Invariants Lengths (distance between two points) Angles (angle between two lines) Area
Introduction to Signal and Image Processing April 5th/12th, 2016
15 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Geometrical Image Transformations
Class I: Isometries (11) Example
Isometric transformation with
and
yielding a transformation Fig 5.4: Original matrix image Fig 5.5: Image after isometric transformation
Beware, that the definition of the affine transformation matrix in MATLAB is different!
Beware, that the origin (zero point) in MATLAB is the top-left corner of images!
Introduction to Signal and Image Processing April 5th/12th, 2016
16 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Geometrical Image Transformations
Class I: Practical Issues (12)
Never try to estimate the matrix elements of directly
(5.3)
Estimate the Euler angle and translation parameters instead.
Introduction to Signal and Image Processing April 5th/12th, 2016
17 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Geometrical Image Transformations
Class II: Similarity (13) Transformations
A similarity transformation is an isometry composed with an isotropic scaling, i.e.
(5.4) Fig 5.6: Similarity = Translation + Rotation + Isotropic Scaling
The above equation can be written more concisely in block form as
(5.5)
where is a (orthogonal) rotation matrix, a translation vector, the scalar the isotropic scaling, and a null vector.
Introduction to Signal and Image Processing April 5th/12th, 2016
18 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Geometrical Image Transformations
Class II: Similarity (14) Transformations (2)
Degrees of Freedom
Similarity Transformations have 4 DOF: one for rotation , two for translation , and one for scaling A similarity can be computed from two point correspondences
Invariants
Angles (angle between two lines) Parallel lines are mapped to parallel lines The length between to points is not invariant, but the ration of two lengths is Similarly the ratio of areas is invariant (because the squared scaling cancels out)
Introduction to Signal and Image Processing April 5th/12th, 2016
19 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Geometrical Image Transformations
Class II: Similarity (15) Transformations Example
Similarity transformation with
and
yielding a transformation Fig 5.7: Original matrix image Fig 5.8: Image after similarity transformation
Introduction to Signal and Image Processing April 5th/12th, 2016
20 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Geometrical Image Transformations
Class II: Practical Issues (16)
Never try to estimate the matrix elements of directly
(5.6)
Estimate the Euler angle , scaling , and translation parameters instead.
Introduction to Signal and Image Processing April 5th/12th, 2016
21 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Geometrical Image Transformations
Class III: Affine (17) Transformations
An affine transformation is a non-singular linear transformation (rotation, scaling and skewing) followed by a translation, i.e.
(5.7) fFig 5.9: Affine = Non-singular linear transformation + Translation
The above equation can be written more concisely in block form as
(5.8)
where is a non-singular matrix, a translation vector, and a null vector.
An affinity is orientation-preserving or -reversing if is positive or negative respectively.
Introduction to Signal and Image Processing April 5th/12th, 2016
22 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Geometrical Image Transformations
Class III: Affine (18) Transformations (2)
Degrees of Freedom
Affine Transformations have 6 DOF: four for the non-singular matrix , and two for translation An affinity can be computed from three point correspondences
Invariants
Parallel lines are mapped to parallel lines Ratio of lengths of parallel line segments Ratio of areas
Introduction to Signal and Image Processing April 5th/12th, 2016
23 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Geometrical Image Transformations
Class III: Affine (19) Transformation Example
Affine transformation with
yielding a transformation Fig 5.10: Original matrix image Fig 5.11: Image after affine transformation
Introduction to Signal and Image Processing April 5th/12th, 2016
24 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Geometrical Image Transformations
Class III: Practical Issues (20)
The use of the affine transform rather than the similarity transform (rigid) does not greatly increase its applicability in medicine, as there are no organs that only stretch or shear. Tissues normally deform in more complex non-rigid ways.
There are, however, scanner introduced errors than can result in skewing terms
Tilted gantry in CT acquisitions Tilted plane in optical image acquisitions such as microscope or endoscope
and affine transforms are used to overcome these problems.
Nevertheless, the affine transformation is often used for registration, as no care has to be taken that form a proper rotation matrix.
Introduction to Signal and Image Processing April 5th/12th, 2016
25 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Geometrical Image Transformations
Class IV: Projective (21) Transformations
A projective transformation is a general non-singular linear transformation of homogeneous coordinates, i.e.
Fig 5.12: Affine = General (5.9) non-singular linear transformation
The above equation is often written more concisely in block form as
(5.10)
where is a vector , a non-singular matrix, a translation vector, and the scaling parameter.
Introduction to Signal and Image Processing April 5th/12th, 2016
26 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Geometrical Image Transformations
Class IV: Projective (22) Transformations (2)
Degrees of Freedom
Projective Transformations have 8 DOF A projective transformation can be computed from four point correspondences (with no three collinear on either plane)
Invariants
The most fundamental projective invariant is the cross ratio (ratio of ratio) of four collinear points
Introduction to Signal and Image Processing April 5th/12th, 2016
27 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Geometrical Image Transformations
Class IV: Projective (23) Transformation Example
Projective transformation with a transformation matrix
Fig 5.13: Original image Fig 5.14: Image after projective transformation
Introduction to Signal and Image Processing April 5th/12th, 2016
28 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Geometrical Image Transformations
Practical Issues and (24) Common Pitfalls
Beware of the rotation centre! Are transformations commutative thus is ?
Introduction to Signal and Image Processing April 5th/12th, 2016
29 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Geometrical Image Transformations
Determination of the (25) Matrix Coefficients
The coefficients of the transformation matrices are generally not known. They can, however, be infered from point correspondences.
For an affine transformation three non-collinear points are required. With these three points Eq 5.7 becomes
(5.11)
or short
(5.12)
from which we can compute the transformation with
(5.13)
The inverse of exist only if the three points are linerarily independent → geometrical interpretation: they do not lie on one line
Introduction to Signal and Image Processing April 5th/12th, 2016
30 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Geometrical Image Transformations
Determination of the (26) Matrix Coefficients (2)
Beware, that this approach only works for the affine transform of Eq 5.7 and the projective transform shown in Eq 5.9.
The individual matrix parameters in the transformation matrix of the isometries and similarity transforms are not independent!
Introduction to Signal and Image Processing April 5th/12th, 2016
31 of 188 22.02.2016 09:18 Pixel Interpolation Methods
Applying Spatial (28) Transformations to Images
Once the spatial transformation is known, the input image has to be mapped into the output image. Two approaches are common: Fig 5.15: Spatial mapping Forward mapping: Backward mapping:
During forward mapping each pixel from the input image is transformed and copied to the output image. Problems with this approach:
Pixel coordinates generally do not fall onto exact pixel locations Two or more input pixels could be mapped to the very same output pixel Some output pixels might not get a value assigned at all (gaps)
During backward mapping for each output pixel the coordinates in the input image are calculated and copied over. Problem with this approach:
The pixel coordinate generally does not fall onto an exact pixel location
Introduction to Signal and Image Processing April 5th/12th, 2016
32 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Pixel Interpolation Methods
Forward Mapping (29)
To avoid
the problem of several input pixels assigned to the same output pixels and the gaps in the output image
a more sophisticated approach is required.
One such possibility is mapping all four corners of each input pixel into a quadrilateral mesh in the output image.
Output pixels are calculated by how much area each input pixel occupies.
→ Although the approach produces good results, it is complex and computationally expensive to implement
Introduction to Signal and Image Processing April 5th/12th, 2016
33 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Pixel Interpolation Methods
Forward Mapping Example (30)
Fig 5.16: Forward mapping without Fig 5.17: Forward mapping with interpolation interpolation
Introduction to Signal and Image Processing April 5th/12th, 2016
34 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Pixel Interpolation Methods
Backward Mapping (31)
In backward mapping for each output pixel the corresponding location in the input image is calculated using .
As the pixel coordinates in the input image generally do not fall onto exact pixel locations, the pixel intensity/colour is interpolated among the nearest input image pixels.
Backward mapping is easier to implement and computationally faster than forward mapping Fig 5.18 Backward mapping → Problem with this approach is, that the inverse of can not always be found
The problem, however, how to interpolate (resample) the image pixels is still unanswered
Introduction to Signal and Image Processing April 5th/12th, 2016
35 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Pixel Interpolation Methods
Interpolation Methods (32)
It is known from Signalprocessing Theory that the sinc-function
(5.14)
would be the theoretically ideal interpolation kernel for band limited signals. It is, however,
computationally very unattractive as it has infinite support.
Nearest-Neighbour Interpolation Bilinear Interpolation
very attractivefar from idealbetter trade-off
Cubic Convolution Methods
Introduction to Signal and Image Processing April 5th/12th, 2016
36 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Pixel Interpolation Methods
Nearest Neighbour (33) Interpolation
Set the intensity equal to the intensity of the closest pixel
Nearest Neighbour (NN) interpolation is very fast NN interpolation does not introduce
new grey values! Fig 5.19: As the NN interpolation shows heavy distance between artefacts (see example) and is smallest, is set to .
Introduction to Signal and Image Processing April 5th/12th, 2016
37 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Pixel Interpolation Methods
Nearest Neighbour (34) Interpolation Example
Fig 5.20: Original image Fig 5.21: Rotation with Nearest-Neighbour Interpolation
Introduction to Signal and Image Processing April 5th/12th, 2016
38 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Pixel Interpolation Methods
Bilinear Interpolation (35)
Bilinear interpolation determines the grey level from the weighted average of the four closest pixels and can be defined as (5.17) where
(5.18)
Fig 5.23: Schematic of bilinear interpolation
Bilinear Interpolation has the following properties:
Better than nearest-neighbour interpolation Computationally efficient to implement (about five times slower then NN) But it smooths and blurs the image and thus reduces spatial resolution
Remark: 1D Linear, 2D Bilinear, and in 3D Trilinear Interpolation
Introduction to Signal and Image Processing April 5th/12th, 2016
39 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Pixel Interpolation Methods
Bilinear Interpolation (36) Example
Fig 5.24: Original image Fig 5.25: Rotation with bilinear interpolation
Introduction to Signal and Image Processing April 5th/12th, 2016
40 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Pixel Interpolation Methods
Cubic Convolution (37) Interpolation
An interpolation function is a special type of approximating function as they must exactly coincide with the sampled data at the interpolation nodes (sample points). In other words, if is a sampled function, and if is the corresponding interpolated function, then Fig 5.26: Cubic convolution whenever is an interpolation interpolation node.
Introduction to Signal and Image Processing April 5th/12th, 2016
41 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Pixel Interpolation Methods
Cubic Convolution (38) Interpolation (2)
For equally spaced sample points, many interpolation functions (linear, cubic spline, and cubic convolution) can be written in the form
(5.19)
Fig 5.27: Cubic convolution interpolation where are parameters which depend upon the sampled data, with an inter-sample-distance of , the interpolation kernel, and the interpolant at any arbitrary
As Cubic convolution is separable it can be computed row-by-row and then column-by-column. The one-dimensional Cubic convolution can thus be easily extended to a two-dimensional CC.
Introduction to Signal and Image Processing April 5th/12th, 2016
42 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Pixel Interpolation Methods
Cubic Convolution (39) Interpolation Principle
Cubic convolution interpolation principle using the Catmull-Rom Interpolation kernel with support .
Step 0 Step 1 Step 2
Step 3 Step 4 Step 5
Introduction to Signal and Image Processing April 5th/12th, 2016
43 of 188 22.02.2016 09:18 Step 6 Step 7 Step 8
Introduction to Signal and Image Processing April 5th/12th, 2016
44 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Pixel Interpolation Methods
What Interpolation (40) Kernels are There?
Theory tells us that the sinc is the perfect interpolation kernel for band limited signals → it, however, has infinite support → computational very complex. Furthermore, it turns out, that bilinear Fig 5.28: Optimal sinc interpolation is a special case of cubic kernel convolution interpolation see Fig. 5.29.
Are there any better interpolation kernels?
Fig 5.29: Linear kernel
Introduction to Signal and Image Processing April 5th/12th, 2016
45 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Pixel Interpolation Methods
Conditions for Cubic (41) Convolution Interpolation
The cubic convolution algorithm is derived from a set of conditions imposed on the interpolation kernel and the parameters to maximise accuracy for a given level of computational effort.
(5.20) Fig 5.30: Cubic convolution interpolation
The parameters and the kernel are selected so that the interpolation condition, for each , is satisfied. If the interpolation kernel assumes and → , thus the are the sampled data The cubic convolution interpolation kernel is symmetric and composed of piecewise cubic polynomials defined in the interval The interpolation kernel must be continuous and have a continuous first derivative
Introduction to Signal and Image Processing April 5th/12th, 2016
46 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Pixel Interpolation Methods
Cubic Convolution Kernel (42) with Order L=3
Using the aforementioned conditions we can derive several different variants of Cubic convolution interpolation kernels.
For this kernel the additional requirement that the Taylor series expansion for agrees for as many terms as possible with Fig 5.31: Catmull-Rom the cubic convolution interpolation kernel function. Furthermore, the support is limited to i.e. a grid of . This interpolation kernel is called Catmull-Rom, and defined as
(5.21)
It provides superior results than other kernels in the CC family.
Introduction to Signal and Image Processing April 5th/12th, 2016
47 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Pixel Interpolation Methods
Cubic Convolution Kernel (43) with Order L=4
As with the previous kernel of order this kernel consists of piecewise third- degree polynomials, however, the support is enhanced to a grid of pixels to match an additional term of the Taylor series expansion. Fig 5.32: CC kernel A further extension of the support would with order not allow for a CC kernel with order (instead an increase of order of the polynoms would be required)
(5.22)
Introduction to Signal and Image Processing April 5th/12th, 2016
48 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Pixel Interpolation Methods
Bi-Cubic Convolution (44) Interpolation
For the case of two dimension, a sequential process is used.
First, the points ( ) along each row are calculated Second, convolution is applied along the dashed line through the previously Fig 5.33: Bi-Cubic convolution interpolation determined → yielding the point at position
Introduction to Signal and Image Processing April 5th/12th, 2016
49 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Pixel Interpolation Methods
Bi-Cubic Convolution (45) Interpolation Example
Fig 5.34: Original image Fig 5.35: Rotation with Bi-cubic interpolation
Introduction to Signal and Image Processing April 5th/12th, 2016
50 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Pixel Interpolation Methods
Summary Cubic (46) Convolution Interpolation
Cubic convolution is better at retaining the original grey values than NN and bilinear interpolation Computational requirement is up to 10 times higher than NN interpolation Computational requirement is up to 2 times higher than bi-linear interpolation
Introduction to Signal and Image Processing April 5th/12th, 2016
51 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Pixel Interpolation Methods
Comparison of NN, (47) Bilinear, and Bi-Cubic Interpol
(a) (b) (c) Fig 5.36: (a) NN Interpolation, (b) Bilinear Interpolation, and (c) Bi-Cubic Interpolation
Introduction to Signal and Image Processing April 5th/12th, 2016
52 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Pixel Interpolation Methods
Comparison Bilinear vs (48) Bi-Cubic Convolution
Fig 5.37 Bilinear interpolation Fig 5.38: Bi-Cubic interpolation
The visual difference between bilinear and bi-cubic interpolation is minor (mouse over).
Introduction to Signal and Image Processing April 5th/12th, 2016
53 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Pixel Interpolation Methods
Comparison NN, BL, CC & (49) Sinc Interpolation
Fig 5.39: Comparison of Nearest Neighbour (top/right) vs Bi-Linear (top/right) vs Cubic- Convolution (bottom/left) and Sinc interpolation (bottom/right). Interpolation every for . Introduction to Signal and Image Processing April 5th/12th, 2016
54 of 188 22.02.2016 09:18 Introduction to Signal and Image Processing April 5th/12th, 2016
55 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Pixel Interpolation Methods
References (50)
Robert G. Keys, → Cubic Convolution Interpolation for Digital Image Processing [http://people.ee.ethz.ch/~pcattin /SIP/bibliography/KEYS1981.pdf], IEEE Trans. Acoustics, Speech, and Signal Processing, 29(6):1153-60, 1981.
Introduction to Signal and Image Processing April 5th/12th, 2016
56 of 188 22.02.2016 09:18 Camera Calibration
Camera Calibration (52)
Camera calibration is a special geometrical image correction procedure that tries to undo the imperfections of the image acquisition process, in particular it can correct for
non-square pixels, shifts in the optical axes, and Fig 5.40: radial lens distortions. Endoscope image of a chessboard pattern All these factors need to be corrected prior to further analysis.
Introduction to Signal and Image Processing April 5th/12th, 2016
57 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Camera Calibration
The Camera Model (53)
The transformation from 3D World Coordinate to the 2D Camera Coordinate system is a four step process:
1. Rigid body transformation from world coordinates to camera coordinates →
2. Perspective projection with pin hole geometry → 3. Radial lens distortion →
4. Transformation into the pixel coordinate system →
Introduction to Signal and Image Processing April 5th/12th, 2016
58 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Camera Calibration
The Camera Model (2) (54)
Step 1: Rigid body transformation from the object world coordinate system to the 3D camera coordinate system :
(5.23)
where is a rotation matrix (3 DOF, the Euler angles yaw, pitch and roll), and is the translation vector.
Parameters to calibrate: Rotation matrix , translation
Introduction to Signal and Image Processing April 5th/12th, 2016
59 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Camera Calibration
The Camera Model (3) (55)
Step 2: Transformation from the 3D camera coordinate system to the ideal 2D image coordinate system using perspective projection with pinhole camera geometry. Using the similarity of triangles we get
(5.24)
where is the effective focal length.
[5-Enhancement- figs/camera_model_step2.png]
Parameters to calibrate: The focal length (1 DOF)
Introduction to Signal and Image Processing April 5th/12th, 2016
60 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Camera Calibration
The Camera Model (4) (56)
Step 3: Radial lens distortion is defined as
(5.25)
where is the distorted (or true) image coordinate on the image plane and
(5.26)
Parameters to calibrate: Radial distortion coefficients (in general one term is enough → 1 DOF)
Introduction to Signal and Image Processing April 5th/12th, 2016
61 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Camera Calibration
The Camera Model (5) (57)
Step 4: Real image coordinate is projected to computer image coordinate using an affine transformation
(5.27)
where are the row/column of the pixel and of the principle point in the computer image coordinate system. are the number of pixels per unit length in the horizontal and vertical direction respectively.
indicates how strong the shape of the pixels deviates from being rectangular ( corresponds to rectangular pixels).
Parameters to calibrate: , and . The parameters are provided by the manufacturer.
Introduction to Signal and Image Processing April 5th/12th, 2016
62 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Camera Calibration
Extrinsic Camera (58) Parameters
The parameters in Step 1 (transformation from the 3D world coordinates to 3D camera coordinate system centred at the optical centre) are called extrinsic parameters. There are six extrinsic parameters:
The Euler angles yaw , pitch , roll defining the rotation matrix , and
(5.28)
The three components for the translation
(5.29)
.
Introduction to Signal and Image Processing April 5th/12th, 2016
63 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Camera Calibration
Intrinsic Camera (59) Parameters
The parameters determined during the Steps 2-4 are called intrinsic camera parameters. They are used for projecting the points in the 3D camera coordinate system into the 2D computer image coordinate system. There are six intrinsic parameters:
focal length (distance of the image plane from the projective centre) radial lens distortion coefficient skewness of the pixels pixel coordinates of the principal point
Note: For many applications only the intrinsic camera parameters are required.
Introduction to Signal and Image Processing April 5th/12th, 2016
64 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Camera Calibration
How-To Determine the (60) Camera Parameters
Equation summary Step 1 Step 2 Step 3 Step 4
Method 1. Take several images of a calibration object such as the chessboard or cube with pattern 2. Measure for multiple points 3. Determine their corresponding coordinates in the camera coord. system 4. Optimise the above equations to find the calibration parameters →
Introduction to Signal and Image Processing April 5th/12th, 2016
65 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Camera Calibration
Camera Calibration (61) Example
Undistortion example of an endoscope image using camera calibration. Observe the nearly straight lines in the undistorted image compared to the original image.
Fig 5.41: Original image of a Fig 5.42: Using the camera chessboard taken with an endoscope calibration parameters the image can be rectified/undistorted
Introduction to Signal and Image Processing April 5th/12th, 2016
66 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Camera Calibration
References (62)
Publications
Robert Y. Tsai, A Versatile Camera Calibration Technique for High-Accuracy 3D Machine Vision Metrology using Off-the-Shelf TV Cameras and Lenses, JRA, 3(4):323-44, 1987. Janne Heikkilä, Geometric Camera Calibration Using Circular Control Points, TPAMI, 22(10):1066-77, 2000.
Other Resources
→ A nice camera calibration toolbox for Matlab [http://www.ee.oulu.fi/~jth/calibr/]
Introduction to Signal and Image Processing April 5th/12th, 2016
67 of 188 22.02.2016 09:18 Filtering in the Spatial Domain
Fundamentals of Spatial Filtering
Background (65)
One possible way of classifying image enhancement methods is by their working domain, in particular we distinguish between the
Spatial domain image enhancement methods, and the Frequency domain image enhancement methods
The following slides discuss the fundamental ideas underlying the spatial domain enhancement methods
Introduction to Signal and Image Processing April 5th/12th, 2016
68 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Fundamentals of Spatial Filtering
Spatial Domain Methods (66)
Spatial domain methods are procedures that operate directly on the pixels composing an image. These spatial domain functions may be expressed mathematically as
(5.30)
where is the input image, is the processed output image, and is an operator on , defined over some neighbourhood of .
In addition can also operate on a set of input images
(5.31)
such as pixel-by-pixel averaging of images for noise reduction.
Introduction to Signal and Image Processing April 5th/12th, 2016
69 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Fundamentals of Spatial Filtering
Spatial Domain Methods (67) (2)
The principal approach to defining a neighbourhood about is to use a square or rectangular subimage (often called kernel) centred at .
The centre of the subimage is moved from pixel to pixel and applying the Fig 5.43: A operator at each location to neighbourhood about a point yield .
For subimage sizes , the pixels at the edge of the image are not well defined.
To avoid ambiguities odd kernel sizes are taken generally.
Introduction to Signal and Image Processing April 5th/12th, 2016
70 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Fundamentals of Spatial Filtering
Spatial Filtering (68)
The use of spatial masks for image processing usually is called spatial filtering and the masks themselves are called spatial filter. In this part we describe linear and non-linear spatial filters for image enhancement.
The neighbourhood processing of spatial filtering consists of:
1. Defining a centre point 2. Performing an operation that only involves the pixels in a predefined neighbourhood about the centre point 3. The result of this operation is the value of the process at point Fig 5.44: Principle of spatial filtering with a -mask
4. Repeat this process for every pixel in the image
If the computations performed on the pixels of the neighbourhoods are linear, the operation is called linear spatial filtering or convolution, otherwise it is called non-linear spatial filtering.
Introduction to Signal and Image Processing April 5th/12th, 2016
71 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Fundamentals of Spatial Filtering
Linear Spatial Filter (69)
Linear spatial filtering is, in contrast to frequency domain filtering, performed in the pixel or spatial domain.
The linear operations of interest in this chapter consist of multiplying each pixel in the neighbourhood by a corresponding coefficient and summing the results to obtain the response at each point , thus .
The coefficients are arranged as an matrix, often called a filter, mask, kernel, template, or window. Fig 5.45: Linear spatial filter principle with a mask Remark: are typically odd as a filter mask with a unique centre point is more intuitive.
Introduction to Signal and Image Processing April 5th/12th, 2016
72 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Fundamentals of Spatial Filtering
Non-Linear Spatial Filter (70)
Non-linear spatial filters also operate on neighbourhoods.
In general, however, their operation is directly based on the values of the pixels in the neighbourhood under consideration, and they do not explicitly use coefficients as described before, thus
(5.32)
where is a non-linear function. Fig 5.46: Non-linear spatial filter principle with a neighbourhood
Introduction to Signal and Image Processing April 5th/12th, 2016
73 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Fundamentals of Spatial Filtering
Solutions to the Border (71) Problem
Several solutions have been proposed to circumvent/minimise the border problem when filtering an image, namely padding with a specific value, replicating the border pixels, symmetric mirroring of the image, circular extension of the image.
Replication of Mirror- Zero (black) Original image the border reflecting at Circular padding values borders
Introduction to Signal and Image Processing April 5th/12th, 2016
74 of 188 22.02.2016 09:18 Basic Grey-Level Transformations
Intensity Transformation (73) Functions
The simplest form of the Transformation is when the neighbourhood in Fig. 5.43 is of size . The value of at location thus only depends on the intensity of at that point, and reduces to an intensity or grey-level transformation function.
As only depends on the pixel intensity, and not explicitly on , intensity transformation functions can be written in simplified form as
(5.33)
where denotes the intensity of and the intensity of . Fig 5.47: Look-up Table In other words reduces to a simple Look-up Table (LUT).
Introduction to Signal and Image Processing April 5th/12th, 2016
75 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Basic Grey-Level Transformations
Image Negatives (74)
Negatives of digital images are useful in numerous applications, such as displaying medical X-Ray images.
The negative of a digital image is obtained using the transformation function shown in Fig. 5.48, where is the number of grey levels.
Fig 5.48: LUT to obtain the negative image
Fig 5.49: Fig 5.50: Inverse Mammogram Mammogram
Introduction to Signal and Image Processing April 5th/12th, 2016
76 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Basic Grey-Level Transformations
Gamma Correction (75)
The Power-Law or Gamma Transformations have the basic form:
(5.34)
where and are positive constants. Fig 5.51: Plots of Eq 5.34 for various values of
Power-law curves with:
: map a narrow band of dark values into a wider range of output intensities : map a narrow band of bright values into a wider range of output intensities.
Introduction to Signal and Image Processing April 5th/12th, 2016
77 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Basic Grey-Level Transformations
Gamma Correction of (76) Displays
Medical viewing devices have to be corrected (Gamma correction) to compensate for any power-law transformations in the capturing and displaying chain.
The problem of missing dynamic range compared to Fig 5.52: Gamma correction in medical displaying X-Ray film devices still exists.
Introduction to Signal and Image Processing April 5th/12th, 2016
78 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Basic Grey-Level Transformations
Gamma Correction (77) Example
The power-law or gamma correction can also be used to manipulate image contrast as shown in this example.
Original image Fig 5.53: Sample image with various gamma correction values
Introduction to Signal and Image Processing April 5th/12th, 2016
79 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Basic Grey-Level Transformations
Gamma Correction (78) Example (2)
There exist, however, better methods for contrast enhancement.
Original image Homomorphic filtering Fig 5.54: Comparison gamma correction and homomorphic filtering for contrast enhancement
Introduction to Signal and Image Processing April 5th/12th, 2016
80 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Basic Grey-Level Transformations
Contrast Stretching (79) Problem
Although we injected contrast agent, the thorax CT image has poor contrast.
The problem is the poor dynamic range of the display.
Gamma correction does not help here either, see the MeVisLab example [MeVisLab/gamma_vs_windowing.mlab] Fig 5.55: Abdominal CT with contrast agent
Introduction to Signal and Image Processing April 5th/12th, 2016
81 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Basic Grey-Level Transformations
Contrast Stretching (80)
A low-contrast image can result from poor illumination, lack of dynamic range in the imaging sensor, or even wrong setting of a lens aperture during image acquisition. The aim of contrast stretching is to increase the dynamic range of the grey levels in the image Fig 5.56: Typical contrast being processed, see Fig. 5.56 stretching transformation function In medical imaging the special case shown in Fig. 5.57 called windowing is often used to visualise details otherwise hidden by locally stretching the contrast.
Fig 5.57: Windowing LUT often used in medical radiology
Introduction to Signal and Image Processing April 5th/12th, 2016
82 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Basic Grey-Level Transformations
Contrast Stretching (81) Example
Properly set windows help to visualise the data.
Fig 5.58: Abdominal CT with contrast Fig 5.59: Same slice with windowing agent for improved contrast
Introduction to Signal and Image Processing April 5th/12th, 2016
83 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Basic Grey-Level Transformations
Thresholding (82)
If , , and the transformation becomes a thresholding function that creates a binary image and is thus a simple segmentation technique.
is the thresholding level, where , and the output either Fig 5.60: Thresholding is another special case of a or . transformation function In some cases an interval with two thresholds is defined.
Introduction to Signal and Image Processing April 5th/12th, 2016
84 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Basic Grey-Level Transformations
General Thresholding (83)
Thresholding must not necessarily lead to binary images. In general thresholding only parts of the grey-value range are mapped to a constant.
The general threshold filter of itk (→ Insight Tool Kit [http://www.itk.org]) for example allows three different modes:
1. All grey-values below a certain Fig 5.61: Original MR head scan threshold are mapped to the constant
2. All grey-values above a certain threshold are mapped to the constant
3. All grey-values below or above , thus outside, are mapped to the constant
Fig 5.62: Below mode ( )
Introduction to Signal and Image Processing April 5th/12th, 2016
85 of 188 22.02.2016 09:18 Fig 5.63: Above mode ( )
Fig 5.64: Outside mode ( )
Introduction to Signal and Image Processing April 5th/12th, 2016
86 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Basic Grey-Level Transformations
General Thresholding (84) Example
Fig 5.65: Red colour Fig 5.66: All white pixels Fig 5.67: All black pixels channel of a Jet Aircraft exceed the threshold are in the interval
Fig 5.68: Grey scale Fig 5.69: Grey scale Fig 5.70: Grey scale histogram histogram histogram
Introduction to Signal and Image Processing April 5th/12th, 2016
87 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Basic Grey-Level Transformations
Dynamic Range (85) Compression
Sometimes the dynamic range of a processed image exceeds the capabilities of the display and only the brightest parts are visible. An effective way to compress the dynamic range is the following intensity transformation:
(5.35)
where is a scaling constant, and the logarithm performs the desired compression.
Fig 5.71: Test pattern Fig 5.72: Fourier Fig 5.73: Fourier spectrum of the pattern spectrum with a compressed dynamic range
Introduction to Signal and Image Processing April 5th/12th, 2016
88 of 188 22.02.2016 09:18 Histogram Processing
Histogram Equalisation (87)
It often happens that the dynamic range of the camera, i.e. the range of possible intensities, is not effectively used. Possible causes can be under- or overexposed images.
Fig 5.74: Aerial image of the Nasca Fig 5.75: Corresponding intensity lines in Peru histogram
The image above has poor contrast as can also be seen in the intensity histogram. Prior to further processing, such as line detection, image contrast should be optimised.
Introduction to Signal and Image Processing April 5th/12th, 2016
89 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Histogram Processing
Histogram Equalisation (88) (2)
Suppose that the image intensities fall in the interval . Further suppose that the original histogram, i.e. the grey level probability density function, is a continuous function .
The underlying idea of Histogram equalisation (aka Histogram flattening) is to produce a new histogram, with a more flat outlook instead. Evenly distributing the histogram in fact maximises the entropy of the image. Maximising the entropy of a distribution maximises its information content and thus histogram equalisation maximises the information content of the output image.
The individual pixels after transformation retain their value order but the values are shifted, so that as far as possible, an equal number of pixels have each possible value.
Introduction to Signal and Image Processing April 5th/12th, 2016
90 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Histogram Processing
Histogram Processing (89)
Fig 5.76: Principle of histogram equalisation
Introduction to Signal and Image Processing April 5th/12th, 2016
91 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Histogram Processing
Histogram Equalisation (90) (4)
From the previous slide we know that our transformation must fulfil
(5.36)
rewriting this equation we thus Fig 5.77: Cumulative probability get for transformation distribution of the original tire image (5.37)
where is the cumulative probability distribution of the original image. This grey-level transform forces the resulting histogram to be flat.
Introduction to Signal and Image Processing April 5th/12th, 2016
92 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Histogram Processing
Histogram Equalisation (91) (5)
The effects of histogram equalisation are
1. grey level intervals with many pixels packed together are expanded 2. intervals with few pixels are compressed.
Fig 5.78: Original grey Fig 5.79: Histogram Fig 5.80: Resulting scale image equalised image equalised histogram
Due to discretisation effects the resulting histogram will almost never be entirely flat.
Introduction to Signal and Image Processing April 5th/12th, 2016
93 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Histogram Processing
Histogram Equalisation (92) Example
(a) Original image (b) Histogram (c) Image part (d) Improved equalised image used to determine histogram the cumulative equalised image probability distribution
Fig 5.81: Histogram equalisation example of the Scott Monument in Edinburgh, Scotland
Introduction to Signal and Image Processing April 5th/12th, 2016
94 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Histogram Processing
Histogram Equalisation (93) Example (2)
(a) Original chest X-ray (b) After histogram equalisation
Introduction to Signal and Image Processing April 5th/12th, 2016
95 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Histogram Processing
Histogram Equalisation vs.(94) Homomorphic Filtering
(a) Histogram equalisation (b) Homomorphic filtering
Introduction to Signal and Image Processing April 5th/12th, 2016
96 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Histogram Processing
Histogram Specification (95)
Histogram specification or Histogram matching is a common variant of Histogram equalisation. Histogram equalisation is limited in that it is capable of producing only a uniform intensity distribution. Sometime it is, however, desirable to Fig 5.82: Histogram be able to control the shape of the specification output histogram in order to highlight certain intensity levels in an image.
The method is simple in principle. Let's consider continuous levels that are normalised in the interval , and let and denote the input and output intensity levels. With the given input probability density function, and the desired output PDF.
We know from previous discussion that
(5.38)
results in intensity levels that have uniform PDF . Suppose now that we define a variable with the property
(5.39)
Since we are after an image with intensity levels , which have the specified PDF we can reformulate the two preceding equations
(5.40)
Introduction can be to easily Signal andfound Image from Processing the input image April(histogram 5th/12th, 2016
97 of 188 22.02.2016 09:18 equalisation). As long as we can find the above equation can be used to find the transformed levels whose PDF was specified by .
When working with discrete variables, the inverse of exists, if has unit area and none of its entries is zero. As with histogram equalisation, the discrete implementation only yields an approximation to the specified histogram.
Introduction to Signal and Image Processing April 5th/12th, 2016
98 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Histogram Processing
Local Histogram (96) Enhancement
Histogram equalisation and Histogram specification are global methods in the sense that pixels are modified by a transformation function based on the grey-level distribution of the entire image. It is often necessary to enhance details over small areas as is illustrated in Figure 5.83
(a) Original (b) Result of global (c) Local histogram image histogram equalisation equalisation with neighbourhood
Fig 5.83: Example showing the usefulness of local histogram equalisation
Introduction to Signal and Image Processing April 5th/12th, 2016
99 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Histogram Processing
Local Histogram (97) Enhancement (2)
The histogram equalisation and specification methods can be easily adapted for local enhancement with either one of the following approaches:
1. At each location the histogram of Fig 5.84: Method 1 the neighbourhood is improved and the centred pixel in the neighbourhood taken. The neighbourhood is then moved to the next adjacent pixel. 2. The second computationally more efficient approach uses non-overlapping regions → the Fig 5.85: Method 2 approach usually produces a checkerboard effect
Introduction to Signal and Image Processing April 5th/12th, 2016
100 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Histogram Processing
But Beware... (98)
Applying the described histogram based methods to the RGB channels of a colour image separately can produce significant colour shift in the transformed image → change colour space
(a) Original (b) Histogram (c) Histogram (d) Gamma 0.4 petroglyph image equalisation equalisation of the improved original that lacks contrast enhanced value component image and brightness petroglyph image in HSV space Fig 5.86: Sample with a histogram equalised colour image.
Introduction to Signal and Image Processing April 5th/12th, 2016
101 of 188 22.02.2016 09:18 Image Averaging
Image Averaging (100)
Consider a noisy image formed by the addition noise to the original image
(5.41)
where the assumption is that at every pixel position the noise is uncorrelated and has zero mean. If the noise satisfies these constraints and an image is formed by averaging different noisy images,
(5.42)
then it follows that the expected value of is
(5.43)
that the variance of every pixel is reduced by
(5.44)
where and are the variances of and
This means that approaches as the number of noisy images used in the averaging process increases. To avoid blurring the images must be properly registered.
Introduction to Signal and Image Processing April 5th/12th, 2016
102 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Image Averaging
Image Averaging (101) Example
(a) Single slice (b) Average
Fig 5.87: Averaging example (a) single sagittal slice of a liver (b) average over 5 sagittal slices
Introduction to Signal and Image Processing April 5th/12th, 2016
103 of 188 22.02.2016 09:18 Smoothing Spatial Filters
Smoothing Spatial Filters (103)
Smoothing filters are used for blurring or noise reduction.
Blurring is frequently used in preprocessing to (1) remove small details from an image prior to large object extraction (2) and to bridge small gaps in lines or curves. Noise reduction can either be accomplished with a linear filter and also by non-linear filters
Introduction to Signal and Image Processing April 5th/12th, 2016
104 of 188 22.02.2016 09:18 General Properties
General Properties of (105) Smoothing Filters
Convolution forms the mathematical framework for these elementary filters. These filters have a number of desired properties in common that are discussed in the following:
Zero Shift Preservation of the Mean Isotropy Monotonically Decreasing Transfer Function
Introduction to Signal and Image Processing April 5th/12th, 2016
105 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement General Properties
Zero Shift (106)
Property:
For object detection the most important feature of a convolution filter is that it does not shift the objects position.
Implication:
To cause no shift, the transfer function of a filter must be real thus the imaginary part zero → no phase.
Comment:
A real transfer function originates from a symmetric filter mask.
Introduction to Signal and Image Processing April 5th/12th, 2016
106 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement General Properties
Preservation of the Mean (107)
Property:
A smoothing filter should preserve the mean value → the brightness of the image does not change.
Implication:
This condition implies the sum of all filter coefficients of the mask is 1, thus
Comment:
An area with a constant value should not change its value. This is only the case if all the weights add up to 1.
Introduction to Signal and Image Processing April 5th/12th, 2016
107 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement General Properties
Isotropy (108)
Property:
In most applications, the smoothing should be the same in all directions thus isotropic.
Implication:
Both the filter mask and the transfer function should be isotropic.
Comment 1:
The filter mask only depends on the distance from the center pixel and the transfer function on the magnitude of the frequency.
Comment 2:
In the discrete space, of course, this condition can only be met approximately.
Introduction to Signal and Image Processing April 5th/12th, 2016
108 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement General Properties
Monotonically (109) Decreasing Transfer Function
Property:
A smoothing operator should attenuate smaller structures more than coarser structures.
Implication:
This implies that the transfer function decreases monotonically with increasing frequency
Comment:
Together with the preservation of the mean value, this means that the transfer function decreases monotonically from one to zero for each smoothing filter.
Introduction to Signal and Image Processing April 5th/12th, 2016
109 of 188 22.02.2016 09:18 Box Filter
Box Filter (1) (111)
Figure 5.88 indicates that to implement a lowpass (smoothing) spatial filter all coefficients have to be positive.
Although the shape of the spatial filter could be modeled with a Gaussian, the key requirement is that all coefficients be positive The simplest arrangement for a spatial filter would be a mask with all coefficients → box filter As the result would be the Fig 5.88: (Top) cross sum of grey levels of nine pixels (and section of circularly possibly exceed the grey-level range) symmetric frequency has to be scaled by thus domain filter, (bottom) cross section of corresponding spatial With all the response would filter simply be the average of the pixels in the mask (thus called neighbourhood averaging)
Introduction to Signal and Image Processing April 5th/12th, 2016
110 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Box Filter
Box Filter Example (112)
Successive blurring of an MR image with a box filter ( ) of mask size 3 to 11. Note in particular the loss of sharpness in areas with fine detail.
Cropped original Kernel Kernel
MR liver image
Kernel Kernel Kernel
Fig 5.89: Linear lowpass spatial filter example with a MR liver image
Introduction to Signal and Image Processing April 5th/12th, 2016
111 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Box Filter
What is the Problem with (113) the Box Filter?
Fig 5.90: Original image with conic Fig 5.91: Conic circles filtered with a circles box filter
Introduction to Signal and Image Processing April 5th/12th, 2016
112 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Box Filter
Where do these Phase (114) Reversals come from??
Let's have a look at the Fourier transform of the box filter :
Fig 5.92: , the Fourier Transform of the box filter (notice the negative values)
Since becomes negative for some some frequencies undergo a complete phase reversal (shift over ).
Introduction to Signal and Image Processing April 5th/12th, 2016
113 of 188 22.02.2016 09:18 Lowpass Gauss Filter
Lowpass Gauss Filter (116)
The lowpass Gaussian Filter has nicer properties, as the Fourier Transform of a Gaussian is also a Gaussian curve → thus no phase reversals due to negative amplitudes. Fig 5.93: Gauss Kernel with For discrete calculations the ideal filter size depends on the filters sigma is normally chosen as
(5.45)
Namely, the largest odd integer greater than thus more than of the volume under the Gaussian curve is captured.
Introduction to Signal and Image Processing April 5th/12th, 2016
114 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Lowpass Gauss Filter
Lowpass Gauss Filter (117) Example
Successive blurring of an MR image with a Gauss filter and .
Cropped original Gauss kernel with Gauss kernel with
MR liver image
Gauss kernel with Gauss kernel with Gauss kernel with
Fig 5.94: Linear lowpass spatial filter example with a MR liver image
Introduction to Signal and Image Processing April 5th/12th, 2016
115 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Lowpass Gauss Filter
Gauss Filter on the Conic (118) Circle Example
Fig 5.95: Original image with conic Fig 5.96: Conic circles filtered with a circles of increasing frequency Gauss filter with
The Gauss Filter behaves much nicer than the box filter.
Introduction to Signal and Image Processing April 5th/12th, 2016
116 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Smoothing Spatial Filters
Problem with Linear (119) Spatial Filtering
The noise is nicely reduced, but the edges are blurred too.
Introduction to Signal and Image Processing April 5th/12th, 2016
117 of 188 22.02.2016 09:18 Sharpening Spatial Filters
Foundation for (121) Sharpening Spatial Filters
Because averaging (smoothing) is analogous to integration, it is logical to conclude, that sharpening can be accomplished by spatial differentiation.
The principle of sharpening is thus to highlight these differences i.e. transitions in intensity. This can be accomplished either by the
1st derivative
or
2nd derivative.
Introduction to Signal and Image Processing April 5th/12th, 2016
118 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Sharpening Spatial Filters
Foundation for (122) Sharpening Spatial Filters (2)
The definition of the derivatives by finite differences is:
First Derivative Fig. 5.97: 1st vs. 2nd-derivative
(5.46)
Second Derivative
(5.47)
As edges in natural images are often ramps, the first derivative would result in thick edges, whereas the second derivative produces a double edge of one pixel thicknes, separated by a zero-crossing.
Introduction to Signal and Image Processing April 5th/12th, 2016
119 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Sharpening Spatial Filters
Using the 2nd Derivative (123) (Laplacian) for Sharpening
The Laplacian is defined as
(5.48)
from the two Eqs 5.47 from the previous slide, it follows
(5.49)
this Equation can be implemented using a linear filter mask, see Fig 5.98.
Fig. 5.98: Fig. 5.99: Fig. 5.100: Neg. Fig. 5.101: Neg. Laplacian filter Laplacian filter Laplacian filter Laplacian filter mask isotropic in mask isotropic in mask isotropic in mask isotropic in increments increments increments increments
Introduction to Signal and Image Processing April 5th/12th, 2016
120 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Sharpening Spatial Filters
Other Reasoning for (124) Sharpening Spatial Filters
The principle objective of sharpening is to highlight fine detail. Figure 5.102 indicates that highpass spatial filters should have positive coefficients near its centre, and negative coefficients in the outer periphery.
Fig 5.102: (Top) cross section of circularly symmetric highpass frequency domain filter, (bottom) cross section of corresponding spatial filter
Introduction to Signal and Image Processing April 5th/12th, 2016
121 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Sharpening Spatial Filters
Basic Highpass Spatial (125) Filtering
The Equation below (Laplacian) shows the classic implementation of a sharpening filter mask:
(5.50)
Fig 5.103: Nasca Note that lines in Peru
the sum of the coefficients is , so that the output of the mask is zero or small over areas of constant or slowly varying grey level → suppression of the mean this filter eliminates the zero-frequency term → significantly reducing the overall Fig 5.104: Highpass contrast in the image filtered image
The latter property is not always desired.
Introduction to Signal and Image Processing April 5th/12th, 2016
122 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Sharpening Spatial Filters
Unsharp Masking and (126) High-boost Filtering
The process of Unsharp Masking and High-boost Filtering in the spatial domain consists of the following steps:
1. Blur the original image 2. Subtract the blurred image from the original
3. Add a fraction of the mask to the original
Unsharp masking is thus defined in the spatial domain as
(5.51)
where is a smoothed image, then
(5.52)
defines unsharp masking for and highboost filtering when .
Introduction to Signal and Image Processing April 5th/12th, 2016
123 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Sharpening Spatial Filters
High-boost Filtering (127) Example
(a) Original unsharp (b) High-boost with (c) High-boost with image Fig 5.105: High-boost filtering example of a girl
High-boost filters as any other highpass filter enhance image noise!
Introduction to Signal and Image Processing April 5th/12th, 2016
124 of 188 22.02.2016 09:18 Mean Filters
Mean Filters (129)
Image Enhancement and Image Restoration become virtually indistinguishable disciplines when filtering additive noise.
The mean filters is a class of spatial filters whose performance is in many Fig 5.106 Principle of mean cases superior to the filters filters discussed in the Image Enhancement section.
Common to all mean filters is that they operate on a rectangular subimage of size centred at point whose set of coordinates is denoted by .
Introduction to Signal and Image Processing April 5th/12th, 2016
125 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Mean Filters
Arithmetic Mean Filter (130)
The simplest mean filter is the arithmetic mean filter aka box filter.
Let represent the set of coordinates in a rectangular subimage window of size , centred at point . Fig 5.107 Arithmetic mean filter (5.53)
The arithmetic mean filter restores the image by replacing the point at each by the arithmetic mean in the defined subregion .
The arithmetic mean filter is separable and can be efficiently implemented using a convolution mask in which all coefficients have value .
Introduction to Signal and Image Processing April 5th/12th, 2016
126 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Mean Filters
Geometric Mean Filter (131)
Let represent the set of coordinates in a rectangular subimage window of size , centred at point .
(5.54) Fig 5.108 Geometric mean filter The geometric mean filter restores image by replacing each grey value at position by the -root of the product of all values in the defined subregion .
The geometric mean filter achieves a smoothing performance similar to the arithmetic mean filter, but tends to lose less image detail in the process.
Introduction to Signal and Image Processing April 5th/12th, 2016
127 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Mean Filters
Arithmetic vs. Geometric (132) Mean Filter
Fig 5.109: Original image Fig 5.110: Added Gaussian noise with
Fig 5.111: Arithmetic mean filter of Fig 5.112: Geometric mean filter of size size
Introduction to Signal and Image Processing April 5th/12th, 2016
128 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Mean Filters
Arithmetic vs. Geometric (133) Mean Filter (2)
Fig 5.114 PCB X-ray with gauss noise Fig 5.115: PCB X-ray with gauss noise filtered with an arithmetic mean filter filtered with a geometric mean filter
Both show good performance in attenuating the Gaussian noise. The geometric mean filter does, however, not blur the image as much as the arithmetic filter (b/w transitions are sharper).
Introduction to Signal and Image Processing April 5th/12th, 2016
129 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Mean Filters
Harmonic Mean Filter (134)
Let represent the set of coordinates in a rectangular subimage window of size , centred at point . The harmonic mean filter is then defined as
(5.55) Fig 5.116 Harmonic mean filter The harmonic mean filter works well for salt noise, but fails for pepper noise. It does also well with other noise types like Gaussian noise.
Introduction to Signal and Image Processing April 5th/12th, 2016
130 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Mean Filters
Contraharmonic Mean (135) Filter
Let represent the set of coordinates in a rectangular subimage window of size , centred at point . The contraharmonic mean filter is then defined as
Fig 5.117 Contraharmonic mean filter (5.56)
where is the order of the filter. If the contraharmonic filter reduces to the arithmetic filter, and if to the harmonic filter. The contraharmonic mean filter is well suited to eliminate salt-and-pepper noise. It can not do both simultaneously. If it eliminates pepper noise, and if it removes salt noise.
Introduction to Signal and Image Processing April 5th/12th, 2016
131 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Mean Filters
Contraharmonic Mean (136) Filter Example (1)
Fig 5.118: Original image Fig 5.119: pepper noise added
Fig 5.120: Contraharmonic mean Fig 5.121: Contraharmonic mean filter with filter with The contraharmonic mean filter is good at removing salt-and- pepper noise, but has to be chosen correctly.
Introduction to Signal and Image Processing April 5th/12th, 2016
132 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Mean Filters
Contraharmonic Mean (137) Filter Example (2)
Fig 5.122: Original image Fig 5.123: salt noise added
Fig 5.124: Contraharmonic mean Fig 5.125: Contraharmonic mean filter with filter with
Introduction to Signal and Image Processing April 5th/12th, 2016
133 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Mean Filters
Contraharmonic Mean (138) Filter Example (3)
Fig 5.127 Contraharmonic filtered ( Fig 5.128: Contraharmonic filtered ( ) image with pepper noise ) image with salt noise The positive-order filter ( ) does a better job at removing noise from the background, at the expense of blurring the dark areas. The opposite is true for the negative-order filter.
Introduction to Signal and Image Processing April 5th/12th, 2016
134 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Mean Filters
Mean Filter Discussion (139)
In general, the arithmetic and geometric mean filter are
well suited for random noise like Gaussian or uniform noise.
The contraharmonic mean filter in contrast is well suited for
impulse noise like salt-and-pepper noise.
It must, however, be known in advance if salt or pepper noise is present in order to choose the correct sign for . Failing to select the correct sign is disastrous as was shown in Figures 5.121 & 5.124.
Introduction to Signal and Image Processing April 5th/12th, 2016
135 of 188 22.02.2016 09:18 Order-Statistics Filters
Order-Statistics Filter (141)
Order-statistics filters are based on the ordering of the pixels contained in the area encompassed by the filter.
As the order-statistics filters are non-linear they have unique properties not seen in linear filters Fig 5.129: Order-statistics presented so far. Filter
Introduction to Signal and Image Processing April 5th/12th, 2016
136 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Order-Statistics Filters
Median Filter (142)
The most widely spread order- statistics filter in Computer Vision is the Median Filter. As its name already implies, the median filter replaces the centre pixel at position with the median of the region , thus Fig 5.130: Median filter (5.57) Median Filters are particularly effective in the presence of both bipolar and unipolar impulse noise (salt-and-pepper noise).
Introduction to Signal and Image Processing April 5th/12th, 2016
137 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Order-Statistics Filters
Median Filter (2) (143)
Median filtering is a non-linear noise reduction method that does not blur edges and other sharp details.
Median filters replace each pixel with the median of the neighbourhood
It is in particular useful when the noise pattern consists of strong, spikelike components.
(a) Original noisy signal (b) Box filter (c) Median filtered signal
Fig 5.131: Comparison of a spatial filter with the median filter of a salt-pepper noisy image
Introduction to Signal and Image Processing April 5th/12th, 2016
138 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Order-Statistics Filters
Median Filter (3) (144)
(a) Original satellite image (b) Original with 5% unipolar salt noise
(c) Average smoothing filter size 5 (d) Median filtered image
Fig 5.132: Comparison of a smoothing filter vs. a median filter for impulse noise images
Introduction to Signal and Image Processing April 5th/12th, 2016
139 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Order-Statistics Filters
Median vs Arithmetic (145) Mean vs Geometric Mean Filter
Fig 5.133: Original image Fig 5.134: salt and pepper noise added
Fig 5.135: Filtered with a Fig 5.136: Filtered with a arithmetic mean filter geometric mean filter
Introduction to Signal and Image Processing April 5th/12th, 2016
140 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Order-Statistics Filters
Median Filter Example (146)
Fig 5.137: salt and pepper Fig 5.138: After 1 iteration with a noise added median filter
Fig 5.139: After 2 iterations with a Fig 5.140: After 3 iterations with a median filter median filter (mouseover) original
Introduction to Signal and Image Processing April 5th/12th, 2016
141 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Order-Statistics Filters
Min and Max Filters (147)
As the median represents the percentile of the sorted grey values, the Min filter represents the percentile
(5.58)
Fig 5.141: Min filter and the Max filter the percentile of the ordered grey values.
(5.59)
The min filter is suitable to find the darkest point in an image, and it Fig 5.142: Max filter also reduces salt noise.
The max filter is suitable to find the brightest point in an image, and it also reduces pepper noise.
Introduction to Signal and Image Processing April 5th/12th, 2016
142 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Order-Statistics Filters
Min and Max Filter (148) Example
Fig 5.143: Original plus salt noise Fig 5.144: Original plus pepper added noise added
Fig 5.146: Image 5.144 filtered with Max filter of size (mouseover) median filtered version (1 iteration) Fig 5.145: Image 5.143 filtered with a Min filter of size (mouseover) median filtered version (1 iteration)
Introduction to Signal and Image Processing April 5th/12th, 2016
143 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Order-Statistics Filters
Midpoint Filter (149)
The Midpoint filter calculates the midpoint between the maximum and minimum grey values in the area encompassed by the filter, thus
(5.60) Fig 5.147: Midpoint filter The Midpoint filter is a combination of an order-statistics filter and an averaging filter. It works best with randomly distributed noise, such as Gaussian or uniform noise.
Introduction to Signal and Image Processing April 5th/12th, 2016
144 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Order-Statistics Filters
Midpoint Filter Example (150)
Fig 5.148: Image with uniform noise Fig 5.149: Filtered with a midpoint filter
Fig 5.150: Filtered with a Fig 5.151: Filtered with a arithmetic filter geometric filter
Introduction to Signal and Image Processing April 5th/12th, 2016
145 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Order-Statistics Filters
Alpha-Trimmed Mean (151) Filter
Suppose the lowest and highest grey level pixels in the neighbourhood are ignored. Let represent the remaining pixels. The Alpha-trimmed mean filter averages these remaining pixels, thus Fig 5.152: Alpha-trimmed mean filter (5.61)
where . When the alpha-trimmed filter reduces to the arithmetic mean filter. If is selected, the filter becomes the median filter.
The alpha-trimmed filter is suitable for noise combinations, such as salt-and-pepper with Gaussian noise.
Introduction to Signal and Image Processing April 5th/12th, 2016
146 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Order-Statistics Filters
Alpha-Trimmed Mean vs (152) Arith. Mean vs. Median Filter
Fig 5.153: Image with Gaussian noise Fig 5.154: Alpha-trimmed mean filter , and salt, and with pepper noise
Fig 5.155: Arithmetic mean of size Fig 5.156: Median filtered image filtered image (size )
Introduction to Signal and Image Processing April 5th/12th, 2016
147 of 188 22.02.2016 09:18 Adaptive Filters
Adaptive Filters (154)
The filter discussed thus far disregard image content and how image characteristics might vary from one point to the other.
Two adaptive filters whose behaviour changes based on the image characteristics in the neighbourhood defined by the rectangular window .
Adaptive filters have superior performance compared to the filters discussed so far. The improved filtering power comes at the cost of algorithm complexity.
Introduction to Signal and Image Processing April 5th/12th, 2016
148 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Adaptive Filters
Adaptive, Local Noise (155) Reduction Filters
An adaptive noise filter requires statistical measures to adjust its behaviour. The simplest measures of a random variable are its (1) mean and (2) variance. The mean is a measure for the average grey value, and the variance a measure for the local contrast in that region. Fig 5.157: Adaptive, local noise reduction filter The adaptive filter response at any point is thus be based on the four quantities:
1. The grey value of the noisy image at position 2. The estimated variance of the additive noise term
3. The local mean in the neighbourhood 4. The local variance in the neighbourhood
Introduction to Signal and Image Processing April 5th/12th, 2016
149 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Adaptive Filters
Adaptive, Local Noise (156) Reduction Filters (2)
Using these parameters, the behaviour of the filter should be as follows
1. If the estimated noise variance is zero → return as 2. If the local variance is high relative the noise variance → return a value close to 3. If the variances are equal → return the arithmetic mean in the neighbourhood of the pixel
Based on these assumptions the adaptive, local noise reduction filter could be formulated as
Introduction to Signal and Image Processing April 5th/12th, 2016
150 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Adaptive Filters
Remarks (157)
Some important assumptions regarding the method have been made
1. The noise variance must be estimated in advance 2. All other parameters are calculated for each pixel position from pixels in the local neighbourhood 3. An assumption of the method is, that in
Equation
Although it is a reasonable assumption, as the noise in our model is additive and position independent, this condition must be enforced by setting the ratio to 1 if . 4. This enforcement makes the filter non-linear but prevents nonsensical results such as negative pixel values.
Introduction to Signal and Image Processing April 5th/12th, 2016
151 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Adaptive Filters
Adaptive, Local Noise (158) Reduction Filter Example
Fig 5.158: Image with Gaussian noise Fig 5.159: Arithmetic mean filter size
Fig 5.160: Geometric mean filter size Fig 5.161: Adaptive, local noise reduction filter size
Introduction to Signal and Image Processing April 5th/12th, 2016
152 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Adaptive Filters
Adaptive Median Filter (159)
The median filter performs well as long as the spatial density of the impulse noise is not large (rule of thumb ). The Adaptive median filter can handle impulse noise with even larger probability.
In addition the adaptive median Fig 5.162: Adaptive median filter tries to smooth nonimpulse filter noise whilst preserving image detail.
The adaptive median filter also works on a local neighbourhood but in contrast to all previous filter the neighbourhood can increase depending on certain conditions. The following terminology is required to understand the algorithm
minimum/maximum grey level value in median of grey levels in grey level at position maximum allowed size of the neighbourhood
Introduction to Signal and Image Processing April 5th/12th, 2016
153 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Adaptive Filters
Adaptive Median Filter (160) Algorithm
The adaptive median filter has three main objectives (a) removing impulse noise, (b) reduce distortion such as thinning, and (c) provide smoothing of other nonimpulse noise.
Level A: win_size = 3 While(win_size < ) If Goto Level B Else increase win_size End End Output
Level B: If Output Else Output End
Introduction to Signal and Image Processing April 5th/12th, 2016
154 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Adaptive Filters
Adaptive Median Filter (161) Example
Fig 5.163: Image with bipolar Fig 5.164: Arithmetic mean filter impulse noise (size )
Fig 5.165: 2 Iterations median filter Fig 5.166: Adaptive median filter (size ) with max window size 9
Introduction to Signal and Image Processing April 5th/12th, 2016
155 of 188 22.02.2016 09:18 Filtering in the Frequency Domain
Background (163)
One possible way of classifying image enhancement methods is by their working domain, in particular we distinguish between the
Spatial domain image enhancement methods, and the Frequency domain image enhancement methods
The following slides discuss the fundamental ideas underlying the frequency domain enhancement methods
Introduction to Signal and Image Processing April 5th/12th, 2016
156 of 188 22.02.2016 09:18 Introduction Frequency Domain Methods
Principle of Frequency (165) Domain Methods
1. Given a noisy signal
2. Transform the original signal into the frequency domain
3. Multiply the frequency domain signal with the filter transfer function , thus
4. Backtransform with the inverse Fourier transform into the spatial domain →
Introduction to Signal and Image Processing April 5th/12th, 2016
157 of 188 22.02.2016 09:18 Smoothing Frequency Domain Filters
Smoothing Frequency (167) Domain Filters (Lowpass)
Edges and other sharp transitions (such as noise) in an image contribute significantly to the high-frequency content. Hence blurring (smoothing) is achieved in the frequency domain by attenuating a specific range of high-frequency components, thus called Lowpass. Mathematically this can be expressed as
(5.62)
where is the Fourier transform of the image to be smoothed, is the transfer function of the filter, and is the Fourier transform of the smoothed signal. The inverse Fourier transform then yields .
We only consider filter transfer functions that affect the real and imaginary parts of in exactly the same manner and therefore don't alter the phase of the signal → zero-phase-shift filters.
Introduction to Signal and Image Processing April 5th/12th, 2016
158 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Smoothing Frequency Domain Filters
Ideal Lowpass Filter (168)
An ideal lowpass filter (ILPF) is one whose transfer function satisfies the relation, Fig. 5.167
(5.63) Fig 5.167: ILPF transfer function
where is the specified non-negative cutoff frequency, and is the distance from point to the origin of the frequency plane; that is Fig 5.168: Filter cross section (5.64)
The name ideal filter indicates that all frequencies inside the circle of radius are passed with no attenuation, whereas all frequencies outside this circle are completely attenuated.
An ideal lowpass filter cannot be realised with electronic components
Introduction to Signal and Image Processing April 5th/12th, 2016
159 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Smoothing Frequency Domain Filters
Ideal Lowpass Filter (169) Example
(a) Original image (c) Cutoff frequency (e) Cutoff frequency at radius 100 radius 50
(b) Log2 power (d) Cutoff frequency (f) Cutoff frequency spectrum radius 75 radius 25 Fig. 5.169: With cutoff radius smaller than 75 heavy ringing artefacts are visible
Introduction to Signal and Image Processing April 5th/12th, 2016
160 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Smoothing Frequency Domain Filters
Ideal Lowpass Filter (170) Discussion
As the cutoff radius decreases more and more high-frequency energy is removed, resulting in more image blurring.
However, starting with cutoff radius of 75 the images are characterised with heavy ringing artefacts.
The blurring and ringing properties of the ILPF can be easily explained using the convolution theorem. Since the images are related in the Fourier domain by equation
(5.65)
it can be expressed as a convolution in the spatial domain
(5.66)
where is the inverse Fourier transform of the filter transfer function .
Introduction to Signal and Image Processing April 5th/12th, 2016
161 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Smoothing Frequency Domain Filters
ILPF Transfer Function in(171) the Spatial Domain
The key to understand the blurring and ringing process in ILPF lies in the nature of , see Fig. 5.170.
The radii of the concentric rings in are inversely proportional to the cutoff frequency , thus
severe filtering in the frequency Fig 5.170: Blurring domain (small ) produces a function for an ideal lowpass filter relatively small number of broad rings (contrast improved for that cause heavy blurring, see Fig better visualisation) 5.169(f) as the cutoff frequency increases more finely spaced rings with less blurring are produced, see Fig 5.169(d)
Introduction to Signal and Image Processing April 5th/12th, 2016
162 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Smoothing Frequency Domain Filters
A Simple ILPF Example (172)
Suppose that is a simple image composed of two bright pixels on a black background, Fig. 5.171. As the two bright points are approximations of two impulses, the convolution of and is simply a process of copying to each location of each impulse.
Fig 5.171: Simple image with two Fig 5.172: ILPF filter result (contrast white dots enhanced)
Remark: The two copies of in Fig. 5.172 are slightly distorted due to the discrete nature of the calculation process.
Introduction to Signal and Image Processing April 5th/12th, 2016
163 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Smoothing Frequency Domain Filters
Butterworth Lowpass (173) Filter
To circumvent the ringing problems seen with ILPF the Butterworth Lowpass Filter (BLPF) was introduced. The transfer function of the BLPF of order and cutoff frequency is defined as Fig. 5.173: Butterworth lowpass filter (5.67)
A BLPF of order 1 shows no ringing and is normally imperceptible for filters with an order of 2 but become significant in filters of higher order. Fig. 5.174: Radial cross An order of 2 is normally a good section for compromise between effective lowpass filtering and ringing.
Unlike the ILPF, the BLPF transfer function does not have a sharp discontinuity that establishes a clear cutoff between passed and filtered frequencies. For smooth transfer functions the locus of the cutoff frequency is normally defined at the point where is down to a certain fraction of its maximum value.
In case of Eq. 5.67 is where . Another value commonly used is leading to a slightly modified equation for the BLPF.
Introduction to Signal and Image Processing April 5th/12th, 2016
164 of 188 22.02.2016 09:18 (5.68)
The butterworth filter family is suitable for analog hardware implementations.
Introduction to Signal and Image Processing April 5th/12th, 2016
165 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Smoothing Frequency Domain Filters
Butterworth Lowpass (174) Filter Example
(a) Original image (c) Cutoff frequency (e) Cutoff frequency at radius 100 radius 50
(b) Log2 power (d) Cutoff frequency (f) Cutoff frequency spectrum radius 75 radius 25
Fig. 5.175: Compared to the ILPF the BLPF shows no ringing artefacts
Introduction to Signal and Image Processing April 5th/12th, 2016
166 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Smoothing Frequency Domain Filters
Gaussian Lowpass Filter (175)
The Gaussian Lowpass Filters are more common in Computer Vision than the Butterworth filters presented in the previous section. The transfer function of the Gaussian lowpass filter (GLPF) is given by Fig. 5.176: Transfer function of a Gaussian lowpass filter (5.69)
where is the standard deviation. By letting , we obtain the following expression in terms of the cutoff parameter Fig. 5.177: Radial cross section (5.70)
when the filter is down to of its maximum value of 1.
Introduction to Signal and Image Processing April 5th/12th, 2016
167 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Smoothing Frequency Domain Filters
Gaussian Lowpass Filter (176) Example
(a) Original image (c) Cutoff frequency (e) Cutoff frequency at radius 100 radius 50
(b) Log2 power (d) Cutoff frequency (f) Cutoff frequency spectrum radius 75 radius 25
Fig. 5.178: Compared to the ILPF the BLPF shows no ringing artefacts
Introduction to Signal and Image Processing April 5th/12th, 2016
168 of 188 22.02.2016 09:18 Sharpening Frequency Domain Filters
Sharpening Frequency (178) Domain Filters
Just as lowpass filtering blurs the image, the opposite process, highpass filtering, sharpens the image by attenuating the low frequencies and leaving the high frequencies of the Fourier transform relatively unchanged.
Introduction to Signal and Image Processing April 5th/12th, 2016
169 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Sharpening Frequency Domain Filters
Ideal Highpass Filter (179)
Similar to the ILPF, the ideal highpass filter (IHPF) is one whose transfer function satisfies the relation, Fig. 5.179)
Fig 5.179: IHPF (5.71) transfer function
where is the specified non-negative cutoff frequency. Fig 5.180: Filter cross The name ideal filter indicates that all section frequencies outside the circle of radius are passed with no attenuation, whereas all frequencies inside this circle are completely attenuated.
An ideal highpass filter cannot be realised with electronic components
Introduction to Signal and Image Processing April 5th/12th, 2016
170 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Sharpening Frequency Domain Filters
Ideal Highpass Filter (180) Example
(a) Original image (b) Cutoff frequency radius 15
(c) Cutoff frequency radius 30 (d) Cutoff frequency radius 80
Fig. 5.181: Ideal highpass filter example
Introduction to Signal and Image Processing April 5th/12th, 2016
171 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Sharpening Frequency Domain Filters
Butterworth Highpass (181) Filter
The transfer function of the Butterworth highpass filter (BHPF) of order and with cutoff frequency is defined by the relation
Fig. 5.182: Butterworth (5.72) highpass filter
where is given by Eq. 5.64. Similar to the BLPF the BHPF the cutoff frequency is normally defined at the point where is down to a certain fraction of its maximum Fig. 5.183: Radial cross value. In case of Eq. 5.72 is section for where . Another value commonly used is leading to a slightly modified equation for the BHPF.
(5.73)
The butterworth filter family is suitable for analog hardware implementations.
Introduction to Signal and Image Processing April 5th/12th, 2016
172 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Sharpening Frequency Domain Filters
Butterworth Highpass (182) Filter Example
(a) Original image (b) Cutoff frequency radius 15
(c) Cutoff frequency radius 30 (d) Cutoff frequency radius 80
Fig. 5.184: Butterworth highpass filter example
Introduction to Signal and Image Processing April 5th/12th, 2016
173 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Sharpening Frequency Domain Filters
Gaussian Highpass Filter (183)
The Gaussian Highpass Filters are more common in Computer Vision than the Butterworth filters. The transfer function of the Gaussian highpass filter (GHPF) is given by Fig. 5.185: Transfer function of a Gaussian highpass filter (5.74)
where is the standard deviation. By letting , we obtain the following expression in terms of the Fig. 5.186: Radial cross cutoff parameter section
(5.75) when the filter is down to of its maximum value of 1.
Introduction to Signal and Image Processing April 5th/12th, 2016
174 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Sharpening Frequency Domain Filters
Gaussian Highpass Filter (184) Example
(a) Original image (b) Cutoff frequency radius 15
(c) Cutoff frequency radius 30 (d) Cutoff frequency radius 80
Fig. 5.187: Gaussian highpass filter example
Introduction to Signal and Image Processing April 5th/12th, 2016
175 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Sharpening Frequency Domain Filters
The Laplacian in the (185) Frequency Domain
As we have seen previously, the Laplacian of a 2-dimensional function is defined as
(5.76) Fig 5.188: Transfer With the definition of the Fourier Transform function of the Laplacian (top of we get dome is zero)
(5.77)
Yielding for the Laplacian a frequency domain representation of Fig 5.189: Transfer function in the (5.78) spatial domain (black , and white ) The Laplacian operator is useful for outlining edges and improve sharpness in an image.
Introduction to Signal and Image Processing April 5th/12th, 2016
176 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Sharpening Frequency Domain Filters
Laplacian Example (186)
Fig 5.191: Laplacian of the sample Fig 5.190: Sample pattern pattern
Introduction to Signal and Image Processing April 5th/12th, 2016
177 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Sharpening Frequency Domain Filters
Laplacian Highboost (187) Filter Example
Fig 5.193: Moon after Laplacian Fig 5.192 Original blurry moon image highboosting
Introduction to Signal and Image Processing April 5th/12th, 2016
178 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Sharpening Frequency Domain Filters
Unsharp Masking and (188) High-boost Filtering
On this slide Unsharp Masking and High-boost Filtering in the frequency domain are discussed. As already seen unsharp masking is defined in the spatial domain as
(5.79)
where is a smoothed image, then
(5.80)
defines unsharp masking for and highboost filtering when .
This equation can be expressed entirely in the frequency domain
(5.81)
or using a highpass filter
(5.82)
Highpass filters set the dc term (zero frequency) to null. The above filter does not have this problem because of the 1 that is added. The constant controls the proportion of high frequencies added. A slightly more general formulation called High-frequency emphasis filter is
(5.83)
where controls the offset of the original image and controls the contribution of high frequencies.
Introduction to Signal and Image Processing April 5th/12th, 2016
179 of 188 22.02.2016 09:18 Introduction to Signal and Image Processing April 5th/12th, 2016
180 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Sharpening Frequency Domain Filters
Unsharp Masking and (189) Highboost Filtering Example
Fig 5.195: Chest after highboost Fig 5.194 Original chest X-ray filtering
Introduction to Signal and Image Processing April 5th/12th, 2016
181 of 188 22.02.2016 09:18 Homomorphic Filtering
Introduction to Signal and Image Processing April 5th/12th, 2016
182 of 188 22.02.2016 09:18 Homomorphic Filtering (191)
Homomorphic filtering is a generalised technique that simultaneously normalises the brightness across an image and increases its contrast. The illumination-reflectance model introduced in the Section "A Simple Image Model" is used as the basis for a frequency domain procedure. An image can be expressed as the product of its illumination and reflectance components by
(5.84)
where is the illumination, and the reflectance component of the image. The above equation can not be directly used to operate separately on the frequency components of illumination and reflectance because the Fourier transform of the product of two functions is not separable;
(5.85)
With the logarithm we can, however, separate the multiplication into a sum;
(5.86)
with Fourier transform we thus get
(5.87)
or
(5.88)
where and are the Fourier transforms of and , respectively.
But what do we gain with this reformulation? Introduction to Signal and Image Processing April 5th/12th, 2016
183 of 188 22.02.2016 09:18 As the illumination component of an image is generally characterised by slow spatial variations (low frequencies), whilst the reflectance component tends to vary abruptly (high frequencies) we gained rough control over the illumination and reflectance components of an image if we select the filter function appropriately.
(5.89)
where is the Fourier transform of the result and in the spatial domain,
(5.90)
By letting and Eq. 5.90 can be expressed as
(5.91)
Finally, the logarithm has to be reversed yielding the desired enhanced image
(5.92)
where is the enhance output image.
Introduction to Signal and Image Processing April 5th/12th, 2016
184 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Homomorphic Filtering
Homomorphic Filtering (192) (2)
The described homomorphic filtering approach is summarised in Fig. 5.197.
Homomorphic filtering allows good control over the illumination and Fig 5.196:Cross section of reflectance components of an image. circular symmetric This, however, requires a filter homomorphic filter that affects the low- and high-frequency components in different ways.
If the parameters are chosen so that and , the filter function shown in Fig. 5.196 tends to decrease the low frequencies and amplify the high frequencies. The result is simultaneous dynamic range compression and contrast enhancement.
Introduction to Signal and Image Processing April 5th/12th, 2016
185 of 188 22.02.2016 09:18 Fig 5.197: Homomorphic filter approach for image enhancement
Introduction to Signal and Image Processing April 5th/12th, 2016
186 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Homomorphic Filtering
Homomorphic Filtering (193) Example
Fig 5.198: Original image with Fig 5.199: Resulting image corrected uneven lightning with a homomorphic filter
Introduction to Signal and Image Processing April 5th/12th, 2016
187 of 188 22.02.2016 09:18 Ph. Cattin: Image Enhancement Homomorphic Filtering
Homomorphic Filtering (194) Example (2)
Fig 5.200: Homomorphic filter example of a fetoscope image
Introduction to Signal and Image Processing April 5th/12th, 2016
188 of 188 22.02.2016 09:18