Image Filtering: Noise Removal, Sharpening, Deblurring Yao Wang Polytechnic University, Brooklyn, NY11201 http://eeweb.poly.edu/~yao Outline • Noise removal by averaging filter • Noise removal by median filter • Sharpening (Edge enhancement) • Deblurring ©Yao Wang, 2006 EE3414: Image Filtering 2 Noise Removal (Image Smoothing) • An image may be “dirty” (with dots, speckles,stains) • Noise removal: – To remove speckles/dots on an image – Dots can be modeled as impulses (salt-and-pepper or speckle) or continuously varying (Gaussian noise) – Can be removed by taking mean or median values of neighboring pixels (e.g. 3x3 window) – Equivalent to low-pass filtering • Problem with low-pass filtering – May blur edges – More advanced techniques: adaptive, edge preserving ©Yao Wang, 2006 EE3414: Image Filtering 3 Example ©Yao Wang, 2006 EE3414: Image Filtering 4 Averaging Filter • Replace each pixel by the average of pixels in a square window surrounding this pixel • Trade-off between noise removal and detail preserving: – Larger window -> can remove noise more effectively, but also blur the details/edges ©Yao Wang, 2006 EE3414: Image Filtering 5 Example: 3x3 average 100 100 100 100 100 100 200 205 203 100 100 195 200 200 100 100 200 205 195 100 100 100 100 100 100 100 100 100 100 100 100 144 167 145 100 100 167 200 168 100 100 144 166 144 100 100 100 100 100 100 ©Yao Wang, 2006 EE3414: Image Filtering 6 Example ©Yao Wang, 2006 EE3414: Image Filtering 7 Weighted Averaging Filter • Instead of averaging all the pixel values in the window, give the closer-by pixels higher weighting, and far-away pixels lower weighting. L L g(m,n) = ∑ ∑h(k,l)s(m − k,n − l) l=−L k=−L • This type of operation for arbitrary weighting matrices is generally called “2-D convolution or filtering”. When all the weights are positive, it corresponds to weighted average. • Weighted average filter retains low frequency and suppresses high frequency = low-pass filter ©Yao Wang, 2006 EE3414: Image Filtering 8 Graphical Illustration ©Yao Wang, 2006 EE3414: Image Filtering 9 Example Weighting Mask 1 1 1 1 2 1 1 × 1 1 1 1 × 2 4 2 9 16 1 1 1 1 2 1 All weights must sum to one ©Yao Wang, 2006 EE3414: Image Filtering 10 Example: Weighted Average 100 100 100 100 100 100 200 205 203 100 100 195 200 200 100 1 1 1 1 2 1 1 1 × 1 1 1 100 200 205 195 100 × 2 4 2 9 16 1 1 1 100 100 100 100 100 1 2 1 100 100 100 100 100 100 100 100 100 100 100 144 167 145 100 100 156 176 158 100 100 167 200 168 100 100 174 201 175 100 100 144 166 144 100 100 156 175 156 100 100 100 100 100 100 100 100 100 100 100 ©Yao Wang, 2006 EE3414: Image Filtering 11 Filtering in 1-D: a Review • Continuous-Time Signal ∞ Time Domain : g(t) = s(t)∗h(t) = h(τ )s(t −τ )dτ ∫−∞ Frequency Domain : G( f ) = S( f )H ( f ) ∞ Filter frequency response : H ( f ) = h(t)e− j2πft dt ∫−∞ • Extension to discrete time 1D signals Time Domain (linear convolution) : s(n) = s(n)∗h(n) = ∑ h(m)s(n − m) m Frequency Domain : G( f ) = S( f )H ( f ) Filter frequency response (DTFT) : H ( f ) = ∑ h(n)e− j 2πfn n H ( f ) is periodic, only needs to look at the range f ∈(-1/ 2,1/ 2 ) 1/2 corresponds to fs / 2 ©Yao Wang, 2006 EE3414: Image Filtering 12 Filtering in 2D Weighted averaging = 2D Linear Convolution l1 k1 g(m,n) = ∑ ∑ h(k,l)s(m − k,n − l) = = l l0 k k0 = In 2D frequency domain G( f1, f2 ) S( f1, f2 )H ( f1, f2 ) Frequency response of the 2D Filter l1 k1 − π + = j2 ( f1m f2n) H ( f1, f2 ) ∑ ∑ h(m,n)e = = m l0 n k0 H ( f1, f2 ) is periodic, only needs to look at the square region ∈ − ∈ − f1 ( 1/ 2,1/ 2), f2 ( 1/ 2,1/ 2). ©Yao Wang, 2006 EE3414: Image Filtering 13 Frequency Response of Averaging Filters 0 0 0 0 0 • Averaging over a 3x3 0 1 1 1 0 1 window h(m,n) = 0 1 1 1 0 9 0 1 1 1 0 0 0 0 0 0 1 − π ⋅ − + ⋅ − − π ⋅ + ⋅ − − π ⋅ + ⋅ − H ( f , f ) = ()e j2 ( f1 ( 1) f2 ( 1)) + e j2 ( f1 (0) f2 ( 1)) + e j2 ( f1 (1) f2 ( 1)) 1 2 9 1 − π ⋅ − + ⋅ − π ⋅ + ⋅ − π ⋅ + ⋅ + ()e j2 ( f1 ( 1) f2 (0)) + e j2 ( f1 (0) f2 (0)) + e j2 ( f1 (1) f2 (0)) 9 1 − π ⋅ − + ⋅ − π ⋅ + ⋅ − π ⋅ + ⋅ + ()e j2 ( f1 ( 1) f2 (1)) + e j2 ( f1 (0) f2 (1)) + e j2 ( f1 (1) f2 (1)) 9 1 π − π π 1 π − π 1 π − π − π = ()e j2 f1 +1+ e j2 f1 e j2 f2 + ()e j2 f1 +1+ e j2 f1 ⋅1+ ()e j2 f1 +1+ e j2 f1 e j2 f2 9 9 9 1 π − π π − π 1 = ()()e j2 f1 +1+ e j2 f1 e j2 f2 +1+ e j2 f2 = ()()1+ 2cos 2πf 1+ 2cos 2πf 9 9 1 2 ©Yao Wang, 2006 EE3414: Image Filtering 14 = H ( f1, f2 ) H ( f1)H ( f2 ) 1 1 H ( f ) = ()1+ 2cos 2πf , H ( f ) = ()1+ 2cos 2πf 1 3 1 2 3 2 Sketch H(f1) 1 0.5 0 -0.5 30 25 20 20 15 10 10 5 0 0 ©Yao Wang, 2006 EE3414: Image Filtering 15 Frequency Response of Weighted Averaging Filters 1 b 1 1 = 1 2 = 1 [] H 2 b b b 2 b 1 b 1 ; (1+ b) (1+ b) 1 b 1 1 H (u,v) = ()()b + 2cos(2πu) b + 2cos(2πv) /(1+ b)2 1 0.8 0.6 0.4 0.2 0 30 25 20 20 15 10 10 5 0 0 ©Yao Wang, 2006 EE3414: Image Filtering 16 Averaging vs. Weighted Averaging 1 1 1 1 1 b 1 1 = 1 = [] = 1 2 = 1 [] H 1 1 1 1 1 1 1; H 2 b b b 2 b 1 b 1 ; 9 (1+ b) (1+ b) 1 1 1 1 1 b 1 1 H (u,v) = ()()1+ 2cos(2πu) 1+ 2cos(2πv) / 9 H (u,v) = ()()b + 2cos(2πu) b + 2cos(2πv) /(1+ b)2 1 1 0.8 0.5 0.6 0 0.4 0.2 -0.5 30 0 30 25 20 20 25 20 15 20 10 10 15 5 10 10 0 0 5 0 0 b=2 ©Yao Wang, 2006 EE3414: Image Filtering 17 Interpretation in Freq Domain Filter response Low-passed image spectrum Original image spectrum Noise spectrum 0 f Noise typically spans entire frequency range, where as natural images have predominantly lower frequency components ©Yao Wang, 2006 EE3414: Image Filtering 18 Median Filter • Problem with Averaging Filter – Blur edges and details in an image – Not effective for impulse noise (Salt-and-pepper) • Median filter: – Taking the median value instead of the average or weighted average of pixels in the window • Median: sort all the pixels in an increasing order, take the middle one – The window shape does not need to be a square – Special shapes can preserve line structures • Order-statistics filter – Instead of taking the mean, rank all pixel values in the window, take the n-th order value. – E.g. max or min ©Yao Wang, 2006 EE3414: Image Filtering 19 Example: 3x3 Median 100 100 100 100 100 100 200 205 203 100 100 195 200 200 100 100 200 205 195 100 100 100 100 100 100 100 100 100 100 100 100 100 200 100 100 100 200 200 200 100 100 100 195 100 100 100 100 100 100 100 Matlab command: medfilt2(A,[3 3]) ©Yao Wang, 2006 EE3414: Image Filtering 20 Example ©Yao Wang, 2006 EE3414: Image Filtering 21 Matlab Demo: nrfiltdemo Original Image Corrupted Image Filtered Image Can choose between mean, median and adaptive (Wiener) filter with different window size ©Yao Wang, 2006 EE3414: Image Filtering 22 Noise Removal by Averaging Multiple Images ©Yao Wang, 2006 EE3414: Image Filtering 23 Image Sharpening • Sharpening : to enhance line structures or other details in an image • Enhanced image = original image + scaled version of the line structures and edges in the image • Line structures and edges can be obtained by applying a difference operator (=high pass filter) on the image • Combined operation is still a weighted averaging operation, but some weights can be negative, and the sum=1. • In frequency domain, the filter has the “high- emphasis” character ©Yao Wang, 2006 EE3414: Image Filtering 24 Frequency Domain Interpretation Filter response (high emphasis) sharpened image spectrum Original image spectrum 0 f ©Yao Wang, 2006 EE3414: Image Filtering 25 Highpass Filters • Spatial operation: taking difference between current and averaging (weighted averaging) of nearby pixels – Can be interpreted as weighted averaging = linear convolution – Can be used for edge detection • Example filters 0 1 0 0 −1 0 1 1 1 −1 −1 −1 1 − 4 1; −1 4 −1; 1 −8 1; −1 8 −1; 0 1 0 0 −1 0 1 1 1 −1 −1 −1 – All coefficients sum to 0! ©Yao Wang, 2006 EE3414: Image Filtering 26 Example Highpass Filters 0 −1 0 − − − 1 1 1 −1 4 −1 −1 8 −1 − 0 1 0 −1 −1 −1 8 6 12 10 4 8 2 6 4 0 30 2 0 20 30 25 20 10 25 20 20 15 15 10 10 10 0 5 0 5 0 0 ©Yao Wang, 2006 EE3414: Image Filtering 27 Example of Highpass Filtering Original image Isotropic edge detection Binary image ©Yao Wang, 2006 EE3414: Image Filtering 28 Designing Sharpening Filter Using High Pass Filters • The desired image is the original plus an appropriately scaled high-passed image • Sharpening filter = + λ fs f fh = δ + λ hs (m,n) (m,n) hh (m,n) f(x) g(x)=f(x)*hh(x) fs(x)=f(x)+ag(x) x x x ©Yao Wang, 2006 EE3414: Image Filtering 29 Example Sharpening Filters f(x) g(x)=f(x)*hh(x) fs(x)=f(x)+ag(x) x x x 0 −1 0 0 −1 0 = 1 − − ⇒ = 1 − − λ = H h 1 4 1 H s 1 8 1 with 1.