Digital Image Processing Chapter 5: Image Restoration Concept of Image Restoration
Image restoration is to restore a degraded image back to the original image while image enhancement is to manipulate the image so that it is suitable for a specific application.
Degradation model:
g(x, y) = f (x, y) ∗h(x, y) +η(x, y)
where h(x,y) is a system that causes image distortion and
η(x,y) is noise. (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition. Concept of Image Restoration g(x, y) = f (x, y) ∗h(x, y) +η(x, y) We can perform the same operations in the frequency domain, where convolution is replaced by multiplication, and addition remains as addition, because of the linearity of the Fourier transform. G(u,v) = F(u,v)H(u,v) + N(u,v) If we knew the values of H and N we could recover F by writing the above equation as N(u,v) F(u,v) = G(u,v) − H(u,v) However, as we shall see, this approach may not be practical. Even though we may have some statistical information about the noise, we will not know the value of n(x,y) or N(u,v) for all, or even any, values. A s well , di vidi ng b y H(i,j) will cause diff ulti es if there are values which are close to, or equal to, zero. Noise
WdfiibddiihiilWe may define noise to be any degradation in the image signal, caused by external disturbance. If an image is being sent electronically from one place to another , via satellite or wireless transmission, or through networked cable, we may expect errors to occur in the image sig nal. These errors will app ear on the imag e output in different ways depending on the type of disturbance in the signal.
Usually we know what type of errors to expect, and hence the type of noise on the image; hence we can choose the most appropriate method for reducing the effects. Cleaning an image corrupted by noise is thus an important area of image restoration.
We will investigate some of the standard noise forms, and the differen t meth od s of eli mi nati ng or red uci ng th ei r eff ect s on th e image. Salt and pepper noise
Also call ed i mpul se noi se, sh ot noi se, or bi nary noi se. Thi s d egrad ati on can b e caused by sharp, sudden disturbances in the image signal; its appearance is randomly scattered white or black (or both) pixels over the image. To add noise, we use the Matlab function imnoise, which takes a number of different parameters. To add salt and pepper noise: >> cameraman=imread('cameraman.tif'); >> figure;imshow(cameraman); >> cameraman_sp=imnoise(cameraman,'salt & pepper'); >> figure;imshow(cameraman_sp); to add more or less noise we include an optional parameter, being a value between 0 and 1 indicating the fraction of pixels to be corrupted. Thus,for example >> cameraman_sp20=imnoise(cameraman,'salt & pepper',0.2); >> figure;i msh ow( cameraman_sp20) ; Gaussian noise
Gauss ian noi se i s an id eali zed f orm of whi te noi se, whi ch i s caused by random fluctuations in the signal. We can observe white noise by watching a television which is slightly mistuned to a particular channel. Gaussian noise is white noise which is normallyygp distributed. If the image is represented as I, and the Gaussian noise by N, then we can model a noisy image by simply adding the two: I + N >> cameraman_gn=i mnoi se( cameraman,‘ gaussi an') ; >> figure;imshow(cameraman_gn); The gaussian parameter also can take optional values, giving the mean and variance of the noise. The default values are 0 and 0.1 Speckle noise
Whereas G aussi an noi se can b e mod dlldbelled by rand om val ues add ddded to an image; speckle noise (or more simply just speckle) can be modelled by random values multiplied by pixel values , hence it is also called multiplicative noise. Speckle noise is a major problem in some radar applications. As above, imnoise can do sp eckle: >> cameraman_spk=imnoise(cameraman,'speckle'); >> figure;imshow(cameraman_spk); In Matlab, speckle noise is implemented as I(1 + N) Although Gaussian noise and speckle noise appear superficially similar, they are formed by two totally different methods, and, as we shall see, require different approaches for their removal. Periodic noise
If th e i mage si gnal i s subj ect to a peri odi c, rath er th an a rand om disturbance, we might obtain an image corrupted by periodic noise. The effect is of bars over the image. The function imnoise does not have a periodic option, but it is quite easy to create, by addinggp a periodic matrix ( using a tri gonometric function) , to an image: >> s=size(cameraman); >> [x , y]=meshgrid(1:s(1), 1:s(2)); >> p=sin(x/3+y/5)+1; >> cameraman_pn=(im2double(cameraman)+p/2)/2; >> fi gure;i msh ow( cameraman_pn) Salt and pepper noise, Gaussian noise and speckle noise can all be cleaned by using spatial filtering techniques . Periodic noise, however, requires the use of frequency domain filtering. This is because whereas the other forms of noise can b e mod dlldelled as l ocal ld degrad dtiations, peri idiodic noi se i s a global effect. Noise Models
NiNoise cannot tb be predi ditdcted btbut can b e approxi mat tlely d escrib ibdied in statistical way using the probability density function (PDF) Gaussian noise: μ 1 2 2 p(z) = e−( z− ) / 2σ 2πσ Rayleigh noise ⎧2 2 ⎪ (z − a)e−( z−a) / b for z ≥ a p(z) = ⎨b ⎩⎪ 0 for z < a Erlang (Gamma) noise ⎧ ab zb−1 ⎪ (z − a)e−az for z ≥ 0 p(z) = ⎨(b −1)! ⎩⎪ 0 for z < 0 Noise Models (cont.)
Exponential noise
p(z) = ae−az Uniform noise ⎧ 1 ⎪ for a ≤ z ≤ b p(z) = ⎨b - a ⎩⎪ 0 othiherwise
Il(lt&)iImpulse (salt & pepper) noise
⎧Pa for z = a ⎪ p(z) = ⎨Pb for z = b ⎪ ⎩0 otherwise PDF: Statistical Way to Describe Noise
PDF tells how much each z value occurs.
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition. Image Degradation with Additive Noise
g(x, y) = f (x, y) +η(x, y)
Degraded images
Original image
Histogram
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition. Image Degradation with Additive Noise (cont.)
g(x, y) = f (x, y) +η(x, y)
Degraded images
Original image
Histogram
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition. Estimation of Noise
We cannot use the image histogram to estimate noise PDF.
It i s b ett er t o use th e histogram of one area of an image that has constant intensity to estimate noise PDF.
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition. Periodic Noise
Periodic noise looks like dots In the frequency domain
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition. Periodic Noise Periodic Noise Periodic Noise Reduction by Freq. Domain Filtering
Degraded image DFT Periodic noise can be red uced b y setting frequency components corresponding to noise to zero.
Band reject filter Restored image (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition. Band Reject Filters
UtliitfUse to eliminate frequency component tis in some b bdands
Periodic noise from the previous slide that is Filtered out.
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition. Notch Reject Filters
A no tc h re ject filt er i s used t o eli mi nat e some f requency component s.
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition. Notch Reject Filter: Notch filter Degraded image DFT (freq. Domain)
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition. Noise Restored image Example: Image Degraded by Periodic Noise Degraded image
DFT (no shift)
DFT of noise Noise Restored image (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition. Mean Filters Degradation model:
g(x, y) = f (x, y) ∗h(x, y) +η(x, y)
To remove this part
Arithmetic mean filter or moving average filter (from Chapter 3) 1 fˆ(x, y) = ∑ g(s,t) mn (s,t)∈Sxyy Geometric mean filter mn = size of moving window
1 ⎛ ⎞ mn fˆ(x, y) = ⎜ g(s,t)⎟ ⎜ ∏ ⎟ ⎝ (s,t)∈Sxy ⎠ Geometric Mean Filter: Example
Image OOgriginal corrupte image d by AWGN
Image Image obtained obtained using a 3x3 using a 3x3 arithmetic geometitric mean filter mean filter
(Images from Rafael C. Gonzalez and Richard E. AWGN: Additive White Gaussian Noise Wood, Digital Image Processing, 2nd Edition. Harmonic and Contraharmonic Filters Harmonic mean filter mn fˆ(x, y) = 1 WkWorks well llf for salt lti noise but fails for pepper noise ∑ g(s,t) (s,t)∈Sxy
Contraharmonic mean filter mn = size of moving window
Q+1 ∑ g(s,t) Positive Q is suitable for fˆ(x, y) = (s,t)∈Sxy elimi nati ng pepper noi se. g(s,t)Q Negative Q is suitable for ∑ eliminating salt noise. (s,t)∈Sxy Q = the filter order For Q = 0, the filter reduces to an arithmetic mean filter. For Q = -1, the filter reduces to a harmonic mean filter. Contraharmonic Filters: Example
Image Image corrupted corrupted by pepper by salt noise with noise with prob. = 010.1 prob01b. = 0.1
Image Image obtained obtained using a 3x3 using a 3x3 contra- contra- harmonic harmonic mean filter mean filter With Q = 1.5 With Q=-1.5
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition. Contraharmonic Filters: Incorrect Use Example
Image Image corrupted corrupted by pepper by salt noise with noise with prob. = 010.1 prob01b. = 0.1
Image Image obtained obtained using a 3x3 using a 3x3 contra- contra- harmonic harmonic mean filter mean filter With Q=-1.5 With Q=1.5
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition. OrderOrder--StatisticStatistic Filters: Revisit Original image subimage
Statistic parameters MMdiMdMean, Median, Mode, Min, Max, Etc.
Moving window
Output image OrderOrder--StatisticsStatistics Filters
Me dian filter fˆ(x, y) = media n{g(s,t)} (s,t)∈Sxy Max filter
fˆ(x, y) = max {g(s,t)} Reduce “dark” noise (s,t)∈Sxy (pepper noise) Min filter fˆ(x, y) = min {g(s,t)} RdReduce “bih”“bright” noi se (s,t)∈Sxy (salt noise) Midpoint filter 1 ⎛ ⎞ fˆ(x, y) = ⎜ max {g(s,t)}+ min {g(s,t)}⎟ 2 ⎝ (s,t)∈Sxy (s,t)∈Sxy ⎠ Median Filter : How it works A median filter is good for removing impulse , isolated noise
Salt noise Pepper noise Median
Sorted Degraded image Moving array window Salt noise Filter output Pepper noise Normally, impulse noise has high magnitude and is isolated. When we sort pixels in the moving window, noise pixels are usually at the ends of the array. Therefore, it’s rare that the noise pixel will be a median value. Median Filter : Example 1 2 Image corrupted blby salt- and-pepper noise with
pa=pb= 0.1
3 4
Images obtained using a 3x3 median (Imagesfilter from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition. Max and Min Filters: Example
Image Image corrupted corrupted by pepper by salt noise with noise with prob. = 0.1 prob01b. = 0.1
Image Image obtained obtained using a 3x3 using a 3x3 max filter min filter
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition. AlphaAlpha--trimmedtrimmed Mean Filter
Formula:
ˆ 1 f (x, y) = ∑ gr (s,t) mn − d (s,t)∈Sxyy
where gr(s,t) represent the remaining mn-d pixels after removing the d/2 highest and d/2 lowest values of g(s,t).
Total mn pixels Lowest Highest Gray Level d/2 d/2 Gray Level
This filter is useful in situations involving multiple types of noise such as a combination of salt -and-pepper and Gaussian noise. AlphaAlpha--trimmedtrimmed Mean Filter: Example 1 2 Image additionally Image corrupted corrupted by additive by additive salt-and- uniform pepper noise noise
Image 2 Image 2 obtained obtained using a 5x5 using a 5x5 arithme tic geometric mean filter mean filter
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition. AlphaAlpha--trimmedtrimmed Mean Filter: Example (cont.) 1 2 Image additionally Image corrupted corrupted by additive by additive salt-and- uniform pepper noise noise
Image 2 Image 2 obtained obtained using a 5x5 using a 5x5 alpha- median filt er trimmed mean filter with d = 5 AlphaAlpha--trimmedtrimmed Mean Filter: Example (cont.)
Image Image obtained obtained using a 555x5 using a 555x5 arithmetic geometric mean filter mean filter
Image Image obtained obtained using a 5x5 using a 5x5 alpha- median filt er trimmed mean filter with d = 5 Adaptive Filter
GlGeneral concept : -Filter behavior depends on statistical characteristics of local areas iidinside mxn moviiding window - More complex but superior performance compared with “fixed” filters Statistical characteristics: Local mean: Noise variance: 1 2 σ mL = ∑ g(s,t) η mn (s,t)∈Sxy Local variance:
2 1 2 σ L = ∑(g(s,t) − mL ) mn (s,t)∈Sxy Adaptive, Local Noise Reduction Filter Purpose: want to preserve edges
Concept:
2 1. If ση is zero, Æ No noise the filter should return g(x,y) because g(x,y) = f(x,y)
2 2 2. If σL is high relative to ση , Æ Edges (should be preserved), the filter should return the value close to g(x,y)
2 2 3. If σL = ση , Æ Areas inside objects the filter should return the arithmetic mean value mL Formula: σ 2 ˆ η f (x, y) = g(x, y) − 2 (g(x, y) − mL ) σ L Adaptive Noise Reduction Filter: Example
Image Image corrupted obtained by additive using a 7x7 Gaussian arithmetic noise with mean filter zero mean and σ2=1000 Image Image obtained obtained using a7a 7x7 using a 7x7 adaptive geometric noise mean filter reduction filter
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition. Adaptive Median Filter Purpose: want to remove impulse noise while preserving edges
Algorithm:Level A: A1= zmedian –zmin A2= zmedian – zmax If A1 > 0 and A2 < 0, goto level B Else increase window size
If window size <= Smax repeat level A Else return zxy Level B: B1= zxy – zmin B2= zxy –zmax If B1 > 0 and B 2 < 0, return zxy Else return zmedian where zmin = minimum gray level value in Sxy zmax = maximum gray level value in Sxy zmedian = median of ggyray levels in Sxy zxy = gray level value at pixel (x,y) Smax = maximum allowed size of Sxy Adaptive Median Filter: How it works
LlLevel A: A1= zmedian – zmin DiDetermine A2= zmedian –zmax whether zmedian If A1 > 0 an d A2 < 0, go to level B is an impulse or not Else Æ Window is not big enough increase window size
If window size <= Smax repeat level A Else return zxy
Level B: Æ zmedian is not an impulse Determine
B1= zxy –zmin whether zxy B2= zxy –zmax is an impulse or not If B1 > 0 and B2 < 0, Æ zxy is not an impulse return zxy Æ to preserve original details Else return zmedian Æ to remove impulse Adaptive Median Filter: Example
Image corrupted Image obtained Image obtained by salt-and-pepper using a 7x7 using an adaptive noise with median filter median filter with
pa=pb= 0.25 Smax = 7
More small details are preserved
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition. Estimation of Degradation Model Degradation model:
g(x, y) = f (x, y) ∗h(x, y) +η(x, y) or G(u,v) = F(u,v)H(u,v) + N(u,v)
Purpose: to estimate h(x,y) or H(u,v) Why? If we know exact ly h(x,y), regardless of noi se, we can d o deconvolution to get f(x,y) back from g(x,y).
Methods: 1. Estimation by Image Observation
2. Estimation by Experiment
3. Estimation by Modeling (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition. Estimation by Image Observation Original image (unknown) Degraded image
f(x,y) f(x,y)*h(x,y) g(x,y)
Observation Subimage Estimated Transfer DFT Gs (u,v) g (x, y) function s G (u,v) Restoration H (u,v) ≈ H (u,v) = s ppocessbyrocess by s ˆ Fs (u,v) estimation DFT Reconstructed This case is used when we Fˆ (u,v) know only g(x,y) and cannot s Subimage ˆ repeat the experiment! fs (x, y) Estimation by Experiment Used when we have the same equipment set up and can repeat the experiment. Response image from Input impulse image the system
System H()( )
Aδ (x, y) g(x, y) DFT DFT DFT{Aδ (x, y)}= A G(u,v) G(u,v) H(u,v) = A (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition. Estimation by Modeling Used when we know physical mechanism underlying the image formation process that can be expressed mathematically. OiOrig inal li image StblSevere turbulence Example: Atmospheric TblTurbulence mod dlel
2 2 5/ 6 H (u,v) = e−k (u +v ) k = 0.0025 Mild turbulence Low turbulence
k = 0.001 k = 0.00025 (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition. Estimation by Modeling: Motion Blurring
Assume that camera velocity is (x0 (t), y0 (t)) The blurred image is obtained by
T g(x, y) = f (x + x (t), y + y (t))dt ∫ 0 0 0 where T = exposure time.
∞∞ G(u,v) = ∫ ∫ g(x, y)e− j2π (ux+vy)dddxdy −∞∞− ∞ ∞ ⎡T ⎤ = f (x + x (t), y + y (t))dt e− j2π (ux+vy)dxdy ∫ ∫ ⎢∫ 0 0 ⎥ −∞∞− ⎣ 0 ⎦ T ⎡ ∞ ∞ ⎤ = f (x + x (t), y + y (t))e− j2π (ux+vy)dxdy dt ∫∫∫⎢ 0 0 ⎥ 0 ⎣−∞−∞ ⎦ Estimation by Modeling: Motion Blurring (cont.) T ⎡ ∞ ∞ ⎤ G(u,v) = f (x + x (t), y + y (t))e− j2π (ux+vy)dxdy dt ∫ ⎢ ∫ ∫ 0 0 ⎥ 0 ⎣−∞∞− ⎦ T = ∫[F(u,v)e− j2π (ux0 (t)+vy0 (t)) ]dt 0 T = F(u,v)∫ e− j2π (ux0 (t)+vy0 (t))dt 0 Then we get, the motion blurring transfer function: T H (u,v) = ∫ e− j2π (ux0 (t)+vy0 (t))dt 0
For constant motion (x0 (t), y0 (t)) = (at,bt) T T H (u,v) = ∫ e− j2π (ua+vb)ddt = si(in(π (ua + vb))e− jπ (ua+vb) 0 π (ua + vb) Motion Blurring Example For constant motion T H (u,v) = sin(π (ua + vb))e− jπ (ua+vb) π (ua + vb)
Original image Motion blurred image a = b =01= 0.1, T =1= 1
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition. Motion Blur in Matlab >> f=checkerboard(8); >> PSF=fspecial('motion',7,45); >> PSF
PSF =
0 0 0 0 0 0.0145 0 000000 0 0 0 0.0376 0 .1283 0 .0145 0 0 0 0.0376 0.1283 0.0376 0 0 0 0.0376 0.1283 0.0376 0 0 0 0.0376 0.1283 0.0376 0 0 0 0.0145 0.1283 0.0376 0 0 0 0 000 0.0145000000145 0 0 0 0 0
>> gb= imfilter(f, PSF,'circular' ); >> imshow(f);figure;imshow(gb); (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition. Inverse Filter From degradation model: G(u,v) = F(u,v)H(u,v) + N(u,v)
after we obtain H(u,v), we can estimate F(u,v) by the inverse filter: G(u,v) N(u,v) Fˆ (u,v) = = F(u,v) + H(u,v) H(u,v)
Noise is enhanced when H(u,v) is small. To avoid the side effect of enhancing noise, we can apply this formulation to freq. component (u,v) with in a
radius D0 from the center of H(u,v). In practical, the inverse filter is not Popularly used. (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition. Inverse Filter
>> w=id('imread('cameraman.t if');; >> wf=fftshift(fft2(w)); >> b=lbutter(256,256 ,15 ,2); >> wb=wf.*b;
% image degraded >> wba=abs(ifft2(fftshift(wb))); >> wba=ui nt8(255* mat2 gray( wb a)) ; >> figure;imshow(wba);
% recover >> w1=fftshift(fft2(wba))./b; >> w1a=abs(ifft2(fftshift(w1))); >> figure; ims how (ma t2gray (w 1a ));
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition. Inverse Filter: Example
Result of applying Result of applying Original image the full filter the filter with D0=40
Blurred image Result of applying Result of applying Due to Turbulence the filter with D0=70 the filter with D0=85 2 2 5/ 6 H (u,v) = e−0.0025(u +v ) Wiener Filter: Minimum Mean Square Error Filter Objective: optimize mean square error: e2 = E{( f − fˆ)2 } Wiener Filter Formula: ⎡ H *(u,v)S (u,v) ⎤ ˆ f F(u,v) = ⎢ 2 ⎥G(u,v) ⎢⎣S f (u,v) H(u,v) + Sη (u,v)⎦⎥ ⎡ H *(u,v) ⎤ = ⎢ 2 ⎥G(u,v) ⎣⎢ H (u,v) + Sη (u,v) / S f (u,v)⎦⎥ 2 ⎡ 1 H (u,v) ⎤ = ⎢ ⎥G(u,v) H (u,v) 2 ⎢⎣ H(u,v) + Sη (u,v) / S f (u,v)⎦⎥ where H(u,v) = Degradation function
Sη(u,v) = Power spectrum of noise Sf(u,v) = Power spectrum of the undegraded image (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition. Approximation of Wiener Filter Wiener Filter Formula:
2 ⎡ 1 H (u,v) ⎤ Fˆ (u,v) = ⎢ ⎥G(u,v) H (u,v) 2 ⎣⎢ H (u,v) + Sη (u,v) / S f (u,v)⎦⎥
Difficult to estimate Approximated Formula:
⎡ 2 ⎤ ˆ 1 H (u,v) F(u,v) = ⎢ 2 ⎥G(u,v) ⎣⎢ H (u,v) H (u,v) + K ⎦⎥ Practically, K is chosen manually to obtained the best visual result! Wiener Filter: Example
Result of the Result of the inverse Original image full inverse filter filter with D0=70
Blurred image Result of the Due to Turbulence full Wiener filter Wiener Filter: Example (cont.)
Result of the inverse
Origgginal image filter with D0=70
Blurred image Result of the DTblDue to Turbulence Wiener fil ter Example: Wiener Filter and Motion Blurring Image Result of the Result of the degraded inverse filter Wiener filter by motion blur + AWGN 2 ση =650
2 ση =325 Note: K is chosen manually
2 ση =130 (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition. Constrained Least Squares Filter
Degradation model: Writt en i n a mat ri x f orm g(x, y) = f (x, y) ∗h(x, y) +η(x, y) g = Hf + η Objective: to find the minimum of a criterion function MN−−11 2 C = ∑∑[∇2 f (x, y)] xy==00 Subject to the constraint 2 2 g − Hfˆ = η 2 where w = wT w We get a constrained least square filter ⎡ * ⎤ ˆ H (u,v) F(u,v) = ⎢ 2 2 ⎥G(u,v) ⎣⎢ H (u,v) + γ P(u,v) ⎦⎥ where ⎡ 0 −1 0 ⎤ ⎢−1 4 −1⎥ P(u,v) = Fourier transform of p(x,y) = ⎢ ⎥ ⎣⎢ 0 −1 0 ⎦⎥ Constrained Least Squares Filter: Example Constrained least square filter
⎡ * ⎤ ˆ H (u,v) F(u,v) = ⎢ 2 2 ⎥G(u,v) ⎣⎢ H (u,v) + γ P(u,v) ⎦⎥ γ is adaptively adjusted to achieve the best result.
Results from the previous slide obtained from the constrained least square filter Constrained Least Squares Filter: Example (cont.) Image Result of the Result of the degraded Constrained Wiener filter by motion Least square blur + filter AWGN 2 ση =650
2 ση =325
2 ση =130 Constrained Least Squares Filter:Adjusting γ
T 2 Define r = g − Hfˆ It can be shown that φ(γ ) = r r = r
2 2 We want to adjust gamma so that r = η ± a 1
1. Specify an initial value of γ where a = accuracy factor
2 2. Compute r
3. Stop if 1 is satisfied Otherwise return step 2 after increasing γ if r 2 < η 2 − a
or decreasing γ if r 2 > η 2 + a Use the new value of γ to recompute ⎡ * ⎤ ˆ H (u,v) F(u,v) = ⎢ 2 2 ⎥G(u,v) ⎢⎣ H(u,v) + γ P(u,v) ⎦⎥ Constrained Least Squares Filter:Adjusting γ (cont.)
⎡ * ⎤ ˆ H (u,v) F(u,v) = ⎢ 2 2 ⎥G(u,v) ⎣⎢ H(u,v) + γ P(u,v) ⎦⎥ 2 R(u,v) = G(u,v) − H(u,v)Fˆ (u,v) For computing r
M −1N −1 2 1 r = ∑∑r2 (x, y) MN x=0 y=0
1 MN−−11 mηη = ∑∑η(x, y) MN xy==00
MN−−11 2 2 1 2 For computing η σ = ∑∑[η(x, y) − mη ] MN xy==00
2 η2 η = MN[σ − mη ] Constrained Least Squares Filter: Example
OiOrigi nal li image Use correct noise parameters Correct parameters: Initial γ = 10-5 Correction factor = 10-6 a = 0.25 2 -5 ση = 10
Blurred image Use wrong noise Wrong noise parameter Due to Turbulence parameters 2 -2 ση = 10
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition. Results obtained from constrained least square filters Geometric Mean filter This filter represents a family of filters combined into a single expression
1−α ⎡ ⎤ α ⎢ ⎥ ⎡ H *(u,v) ⎤ ⎢ H *(u,v) ⎥ Fˆ (u,v) = ⎢ ⎥ G(u,v) 2 ⎢ ⎡ ⎤⎥ ⎢⎣ H (u,v) ⎦⎥ 2 Sη (u,v) ⎢ H(u,v) + β ⎢ ⎥⎥ S (u,v) ⎣⎢ ⎣ f ⎦⎦⎥
α = 1 Æ the inverse filter α = 0 Æ the Parametric Wiener filter α = 0, β = 1 Æ the standard Wiener filter β = 1, α < 0.5 Æ More like the inverse filter β = 1, α > 0.5 Æ More like the Wiener filter
Another name: the spectrum equalization filter Geometric Transformation
These t ransf ormati ons are oft en call ed rubber-shttheet transf ormati ons: Printing an image on a rubber sheet and then stretch this sheet according to some predefine set of rules .
A geometric transformation consists of 2 basic operations: 1. A spatial transformation : Define how pixels are to be rearranged in the spatially transformed image. 2. Gray level interpolation : Assign gray level values to pixels in the spatially transformed image. Spatial Transformation Spatial Transformation
One of the most commonly used forms of spatial transformations is affine transform , which can be written.
This transfomation can scale, rotate, translate, or shear a set ofif points, d ependi ng on th e val ues ch osen f fhlor the elements of T, which can be create in Matlab : Spatial Transformation Spatial Transformation Image Registration
IiihdkliifhImage registration method seek to align two images of the same scene . For example, it may be of interest to align two ore more images taken at roughly the same time, but using different instruments, such an MRI scan and a PET scan. Image Registration Geometric Transformation : Algorithm
(x, y) (x′, y′)
Image f to be 1 3 Distorted image g restdtored
1. Select coordinate ((y)x,y) in f to be restored 2. Compute x′ = r(x, y) 4. get pixel value at g(x′, y′) By gray level interpolation y′ = s(x, y) 5. store that value in pixel f(x,y) 3. Go to pixel ( x ′, y ′ ) in a distorted image g 5 Spatial Transformation To map between pixel coordinate (x,y) of f and pixel coordinate (x’,y’) of g x′ = r(x, y) y′ = s(x, y)
For a bilinear transformation mapping between a pair of Quadrilateral regions
x′ = r(x, y) = c1x + c2 y + c3xy + c4
y′ s(x, y) c x c y c xy c = = 5 + 6 + 7 + 8 (x′, y′) (x, y)
To obtain r(x,y) and s(x,y), we need tkto know 4 paifirs of coordi ditnates (x, y) and its corresponding (x′, y′) which are called tiepoints.
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition. Gray Level Interpolation: Nearest Neighbor
Sibididince ( x ′ , y ′ ) may not be at an integer coordinate, we need to Interpolate the value of g(x′, y′)
Example interpolation methods that can be used: 1. Nearest neighbor selection 2. Bilinear interpolation 3. Bicubic interpolation
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition. Gray Level Interpolation: Bilinear interpolation Geometric Distortion and Restoration Example Original image and Tiepoints of distorted tiepoints image
Distorted image Restored image
Use nearest neighbor intepolation
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition. Geometric Distortion and Restoration Example (cont.) Original image and Tiepoints of distorted tiepoints image
Distorted image Restored image
Use bilinear itintepol ltiation
(Images from Rafael C. Gonzalez and Richard E . Wood, Digital Image Processing, 2nd Edition. Example: Geometric Restoration Original image Geometrically distorted Image Use the same Spatial Trans (()Bilinear). as in the previous example Restored image
Difference between (Images from Rafael C. Gonzalez and Richard E. 2 above images Wood, Digital Image Processing, 2nd Edition.