Chapter 5 Image Restoration
國立雲林科技大學 資訊工程研究所 張傳育(Chuan-Yu Chang ) 博士 Office: EB 212 TEL: 05-5342601 ext. 4516 E-mail: [email protected] Web: http://MIPL.yuntech.edu.tw Chapter 5 Image Restoration
Image Degradation/Restoration Process The objective of restoration is to obtain an estimate fˆ ( x , y ) of the original image. fˆ(x, y) will be close to f(x,y).
Restoration的目的在於獲得原始影像的 估計影像，此估計影像應儘可能的接近 原始影像。 2 Image Degradation/Restoration Process
The degraded image is given in the spatial domain by Degradation function g(x, y) = h(x, y)* f (x, y) +η(x, y) (5.1-1) noise The degraded image is given in the frequency domain by
G(u,v) = H (u,v)F(u,v) + N(u,v) (5.1-2)
3 Noise Models: Some Important Probability Density Functions
The principal sources of noise arise during Image acquisition and transmission Gaussian noise (normal noise): The PDF of a Gaussian random variable, z, is given by
1 2 2 p(z) = e−(z−µ ) / 2σ 2πσ Rayleigh noise: The PDF of Rayleigh is given by
2 2 (z − a)e−(z−a) /b for z ≥ a p(z) = b 0 for z < a
2 b(4 −π ) µ = a + πb / 4,σ = − ab zb 1 4 e−az for z ≥ 0 p(z) = (b −1)! Erlang (Gamma) noise 0 for z < 0 b b µ = ,σ 2 = a a2 4 Noise Models ae−az for z ≥ 0 p(z) = < Exponential noise 0 for z 0 1 1 µ = ,σ 2 = a a2 Uniform noise 1 if a ≤ z ≤ b p(z) = b − a 0 otherwise a + b (b − a)2 µ = ,σ 2 = 2 12 Impulse (salt and pepper) noise
Pa for z = a p(z) = Pb for z = b 0 otherwise 5 Some important probability density function 偏離原點的位移， 向右傾斜
6 Example 5.1 Sample noisy images and their histograms
下圖為原始影像，將在此圖中加入不同的雜訊。
7 Example 5.1 (cont.) Sample noisy images and their histograms
8 Example 5.1 (cont.) Sample noisy images and their histograms
9 Periodic Noise
Periodic Noise Arises typically from electrical or electromechanical interference during image acquisition. It can be reduced via frequency domain filtering. Estimation of Noise Parameters Estimated by inspection of the Fourier spectrum of the image. Periodic noise tends to produce frequency spikes that often can be detected by visual analysis. From small patches of reasonably constant gray level. The heights of histogram are different but the shapes are similar. 10 Example
影像遭受 sinusoidal noise的破壞
有規則性的亮點
影像的 spectrum
11 Fig 5.4(a-c)中的某小塊影像的histogram
Histogram的形狀幾乎和Fig.4 (d,e,k)的形狀一樣但高度不同。 12 Periodic Noise
The simplest way to use the data from the image strips is for calculating the mean and variance of the gray levels.
µ = ∑ zi p(zi ) (5.2-15) zi∈S 2 2 (5.2-16) σ = ∑(zi − µ) p(zi ) zi∈S
The shape of the histogram identifies the closest PDF match.
13 Restoration in the presence of noise only-spatial filtering When the only degradation present in an image is noise, Eq. (5.1-1) and (5.1-2) become
g(x, y) = f (x, y) +η(x, y) (5.3-1) G(u,v) = F(u,v) + N(u,v) (5.3-2)
The noise terms (η(x,y), N(u,v)) are unknown, so subtracting them from g(x,y)or G(u,v) is not a realistic option. In periodic noise, it is possible to estimate N(u,v) from the spectrum of G(u,v).
14 Restoration in the presence of noise only-spatial filtering
Mean Filter Arithmetic mean filter
Let Sxy represent the set of coordinates in a rectangular subimage windows of size mxn, centered at point (x,y). The arithmetic mean filtering process computes the average value of the corrupted image g(x,y) in the area defined by Sxy. 1 fˆ(x, y) = ∑ g(s,t) mn (s,t)∈Sxy
This operation can be implemented using a convolution mask in which all coefficients have value 1/mn. Noise is reduced as a result of blurring
15 Mean Filter (cont.)
Geometric mean filter Each restored pixel is given by the product of the pixels in the subimage window, raised to the power 1/mn.
1 mn fˆ(x, y) = ∏ g(s,t) ∈ (s,t) Sxy
A geometric mean filter achieves smoothing comparable to the arithmetic mean filter, but it tends to lose less image detail in the process.
16 Example 5.2 被平均值0，變異數 400的加成性高斯雜訊 Illustration of mean filters 破壞的結果
Result of arithmetic mean filter Result of geometric mean filter
17 Restoration in the presence of noise only-spatial filtering
Harmonic mean filter ˆ mn 可濾除salt noise, f (x, y) = 但對pepper noise則失敗 ∑ 1 (s,t)∈Sxy g(s,t)
Contra-harmonic mean filter
Q+1 若 Q>0 可濾除pepper noise, ∑ g(s,t) 若 Q<0 可濾除salt noise, ∈ 若 為算數平均 ˆ (s,t) Sxy Q=0 f (x, y) = 若 為 ∑ g(s,t)Q Q=-1 Harmonic mean (s,t)∈Sxy
18 被機率0.1的salt雜訊 被機率0.1的pepper雜 Chapter 5 破壞的結果 訊破壞的結果 Image Restoration
Contraharmonic mean filter, Q=1.5 濾波的結果
The positive-order filter did a better job of cleaning the background. In general, the arithmetic and geometric mean filters are well suited for random noise. The contra-harmonic filter is well 19 suited for impulse noise Contraharmonic mean filter, Q= -1.5 濾波的結果 Results of selecting the wrong sign in contra- harmonic filtering The disadvantage of contraharmonic filter is that it must be known whether the noise is dark or light in order to select the proper sign for Q. The result of choosing the wrong sign for Q can be disastrous. 在contra-harmonic filter中選取錯誤正負號所導致的結果
20 Order-Statistics Filters The response of the order-statistics filters is based on ordering (ranking) the pixels contained in the image area encompassed by the filter. Median filter Replaces the value of a pixel by the median of the gray levels in the neighborhood of that pixel. (5.3-7) fˆ(x, y) = median{g(s,t)} (s,t)∈Sxy
Median filter provide excellent noise-reduction capabilities, with considerably less blurring than linear smoothing filters of similar size. Median filters are particularly effective in the presence of both bipolar and unipolar impulse noise. 21 Order-Statistics Filters (cont.)
Max filter This filter is useful for finding the brightest points in an image. It reduces pepper noise fˆ(x, y) = max {g(s,t)} (5.3-8) (s,t)∈Sxy
Min filter This filter is useful for finding the darkest points in an image It reduces salt noise. (5.3-9) fˆ(x, y) = min {g(s,t)} (s,t)∈Sxy
22 Order-Statistics Filters (cont.)
Midpoint filter This filter works best for randomly distributed noise, such as Gaussian or uniform noise. 1 fˆ(x, y) = max {g(s,t)}+ min {g(s,t)} (5.3-10) ∈ ∈ 2 (s,t) Sxy (s,t) Sxy
Alpha-trimmed mean filter We delete the d/2 lowest and the d/2 higest gray-level values of g(s,t) in the neighborhood Sxy. 1 fˆ(x, y) = g (s,t) − ∑ r (5.3-11) mn d (s,t)∈Sxy
先刪除0.5d最大與最小的灰階值，再求剩下的灰階平均值 當d=0，則為mean filter 當d=(mn-1)/2時，為median filter 23 Result of one pass with a median filter of size Example 5.3 3x3, several noise points are still visible. Illustration of order-statistics filters
Image corrupted by salt and pepper noise with probabilities Pa=Pb=0.1
Result of processing (c) with median filter again
Result of processing (b) with median filter again
24 Example 5.3 Illustration of order-statistics filters
Result of filtering with Result of filtering with a max filtering a min filtering
25 Example 5.3 Illustration of order-statistics filters
Image corrupted by Image corrupted by additive uniform additive salt-and- noise (variance 800 pepper noise and zero mean) (Pa=Pb=0.1)
Result of filtering (b) Result of filtering (b) with a 5x5 arithmetic with a geometric mean filter mean filter
Result of filtering (b) Result of filtering (b) with a median filter with a alpha-trimmed mean filter with d=5
26 Adaptive Filter Adaptive Filter The behavior changes based on statistical characteristics of the image inside the filter region defined by the m x n rectangular windows Sxy. The price paid for improved filtering power is an increase in filter complexity. Adaptive, local noise reduction filter The mean gives a measure of average gray level in the region. The variance gives a measure of average contrast in that region. The response of the filter at any point (x,y) on which the region is centered is to be based on four quantities: g(x,y): the value of the noisy image. 2 ση , the variance of the noise corrupting f(x,y) to form g(x,y) mL, the local mean of the pixels in Sxy. 2 σ L , the local variance of the pixels in Sxy. 27 Adaptive local noise reduction filter
The behavior of the adaptive filter to be as follows: If the variance of noise is zero, the filter should return simply the value of g(x,y). If the local variance is high relative to the variance of noise, the filter should return a value close to g(x,y). If the two variances are equal, return the arithmetic mean value of the pixels in Sxy. An adaptive expression for obtaining estimated fˆ(x, y) based on these assumptions may be written as The only quantity that σ 2 ˆ = − η ( )− Needs to be known f (x, y) g(x, y) 2 [g x, y mL ] (5.3-12) σ L
28 Example 5.4 Illustration of adaptive, local Arithmetic mean noise-reduction filtering 7*7
Gaussian noise
geometric mean Adaptive filter 7*7
29 Adaptive median filter
Adaptive median filtering can handle impulse noise, it seeks to preserve detail while smoothing nonimpulse noise. The adaptive median filter changes the size of Sxy during filter operation, depending on certain conditions. Consider the following notation: zmin: minimum gray level value in Sxy. zmax: maximum gray level value in Sxy. zmed: median of gray levels in Sxy. zxy: gray level at coordinates (x,y). Smax: maximum allowed size of Sxy.
30 Adaptive median filter (cont.) The adaptive median filtering algorithm
Level A: A1=zmed-zmin A2=zmed-zmax if A1>0 and A2<0, goto level B 判斷zmed是否為 impulse noise else increase the window size if window size <=Smax, repeat level A else output zxy Level B: B1=zxy-zmin B2=zxy-zmax if B1>0 and B2 <0, output zxy else output zmed
31 Adaptive median filter (cont.)
The objectives of the adaptive median filter Remove the slat-and-pepper noise Preserve detail while smoothing nonimpulse noise Reduce distortion The purpose of level A is to determine in the median filter output, zmed is an impulse or not.
32 Example 5.5 Illustration of adaptive median filtering
Corrupted by salt- Result of filtering with a Result of adaptive median and pepper noise 7x7 median filter filtering with Smax=7 with probabilities Pa=Pb=0.25
Preserved sharpness The noise was effectively removed, the filter and detail caused significant loss of detail in the image 33 Periodic Noise Reduction by Frequency Domain Filtering
Bandreject Filter Remove a band of frequencies about the origin of the
Fourier transform. W band的寬度 1 if D(u,v) < D − 0 2 W W = − ≤ ≤ + (5.4-1) Ideal Bandreject filter H (u,v) 0 if D0 D(u,v) D0 2 2 W 1 f D(u,v) > D + 0 2 n order Butterworth filter 1 H (u,v) = 2n D(u,v)W (5.4-2) 1+ 2 2 D (u,v) − D0
2 1 D2 (u,v)−D2 Gaussian Bandreject filter − 0 (5.4-3) H (u,v) =1− e 2 D(u,v)W 34 Periodic Noise Reduction by Frequency Domain Filtering (cont.)
35 Periodic Noise Reduction by Frequency Domain Filtering (cont.)
Image corrupted by sinusoidal noise
Butterworth bandreject filter of order 4
36 Periodic Noise Reduction by Frequency Domain Filtering (cont.)
Bandpass Filters A bandpass filter performs the opposite operation of a bandreject filter.
The transfer function Hbp(u,v) of a bandpass filter is obtained from a corresponding bandreject filter with
transfer function Hbr(u,v) by
(5.4-4) Hbp (u,v) =1− Hbr (u,v)
37 Periodic Noise Reduction by Frequency Domain Filtering (cont.)
Bandpass filtering is quit useful in isolating the effect on an image of selected frequency bands. 以帶通濾波器所獲得 的圖5.16(a)影像的雜 訊圖樣
This image was generated by (1) Using Eq(5.4-4) to obtain the bandpass filter. (2) Taking the inverse transform of the bandpass- filtered transform
38 Periodic Noise Reduction by Frequency Domain Filtering (cont.) Notch Filters Rejects frequencies in predefined neighborhoods about a center frequency. 因為FT對稱性的關係 0 if D (u,v) ≤ D or D (u,v) ≤ D (5.4-5) H (u,v) = 1 0 2 0 1 otherwise 1 2 2 2 D1(u,v) = [(u − M / 2 − u0 ) + (v − N / 2 − v0 ) ] (5.4-6) 1 2 2 2 D2 (u,v) = [(u − M / 2 + u0 ) + (v − N / 2 + v0 ) ] (5.4-7)
Due to the symmetry of the Fourier transform, notch filters must appear in symmetric pairs about the origin
39 Periodic Noise Reduction by Frequency Domain Filtering (cont.)
order n Butterworth notch filter
1 H (u,v) = n D2 1+ 0 D (u,v)D (u,v) 1 2 (5.4-8)
Gaussian notch reject filter
1 D (u,v)D (u,v) − 1 2 2 2 H (u,v) =1− e D0 (5.4-9)
These three filters become highpass filters if u0=v0=0.
40 Periodic Noise Reduction by Frequency Domain Filtering (cont.)
Ideal notch
order 2 Butterworth notch filter
Gaussian notch filter
若u0=v0=0，上述三種濾波器，則退化成高通濾波器 41 Periodic Noise Reduction by Frequency Domain Filtering (cont.)
Notch pass filters We can obtain notch pass filters that pass the frequencies contained in the notch areas. Exactly the opposite function as the notch reject filters.
H np (u,v) = 1− H nr (u,v) (5.4-10)
Notch pass filters become lowpass filters when u0=v0=0.
42 Periodic Noise Reduction by Frequency Domain Filtering (cont.) 佛羅里達州和墨西哥灣的衛 Spectrum 星影像 image (存在水平掃描線)
Notch filter
空間域的雜 訊影像
43 Periodic Noise Reduction by Frequency Domain Filtering (cont.)
Optimal Notch filtering Clearly defined interference patterns are not common. Images obtained from electro-optical scanner are corrupted by coupling and amplification of low- level signals in the scanners’ electronic circuitry. The resulting images tend to contain significant, 2D periodic structures superimposed on the scene data.
44 Periodic Noise Reduction by Frequency Domain Filtering (cont.)
Image of the Martian terrain taken by the Mariner 6 spacecraft. The interference pattern is hard to detect. The star-like components were caused by the interference, and several pairs of components are present. The interference components generally are not single-frequency bursts. They tend to have broad skirts that carry information about the interference pattern.
45 Periodic Noise Reduction by Frequency Domain Filtering (cont.)
Optimal Notch filtering minimizes local variances of the restored estimate image. The procedure contains three steps: Extract the principal frequency components of the interference pattern. Subtracting a variable, weighted portion of the pattern from corrupted image.
46 Periodic Noise Reduction by Frequency Domain Filtering (cont.)
The first step is to extract the principal frequency component of the interference pattern Done by placing a notch pass filter, H(u,v) at the location of each spike. The Fourier transform of the interference noise pattern is given by the expression N(u,v) = H (u,v)G(u,v) where G(u,v) denotes the Fourier transform of the corrupted image.
47 Periodic Noise Reduction by Frequency Domain Filtering (cont.)
Formation of H(u,v) requires considerable judgment about what is or is not an interference spike. The notch pass filter generally is constructed interactively by observing the spectrum of G(u,v) on a display. After a particular filter has been selected, the corresponding pattern in the spatial domain is obtained from the expression η(x, y) = ℑ−1{H (u,v)G(u,v)}
48 Periodic Noise Reduction by Frequency Domain Filtering (cont.)
Because the corrupted image is assumed to be formed by the addition of the uncorrupted image f(x,y) and the interference, if η(x,y) were know completely, subtracting the pattern from g(x,y) to obtain f(x,y) would be a simple matter. This filtering procedure usually yields only an approximation of the true pattern. The effect of components not present in the estimate of η(x,y) can be minimized instead by subtracting from g(x,y) a weighted portion of η(x,y) to obtain an estimate of f(x,y). (5.4-13) fˆ(x, y) = g(x, y)− w(x, y)η(x, y)
The function w(x,y) is to be determined, which is called as weighting or modulation function. The objective of the procedure is to select this function so that the result is optimized in some meaningful way. 49 Periodic Noise Reduction by Frequency Domain Filtering (cont.)
To select w(x,y) so that the variance of the estimate f(x,y) is minimized over a specified neighborhood of every point (x,y). Consider a neighborhood of size (2a+1) by (2b+1) about a point (x,y), the local variance can be estimated as
a b 2 2 1 σ (x, y) = ∑ ∑ fˆ(x + s, y + t)− fˆ(x, y) (5.4-14) ( + )( + ) where 2a 1 2b 1 s=−a t=−b
a b ˆ 1 ˆ f (x, y) = ∑ ∑ f (x + s, y + t) (5.4-15) (2a +1)(2b +1) s=−a t=−b
50 Periodic Noise Reduction by Frequency Domain Filtering (cont.)
Substituting Eq(5.4-13) into Eq(5.4-14) yield
1 a b 2 σ 2 (x, y) = ∑ ∑{[g(x + s, y + t)− w(x + s, y + t)η(x + s, y + t)]− [g(x, y) − w(x, y)η(x, y)]} (2a +1)(2b +1) s=−a t=−b (5.4-16) Assuming that w(x,y) remains essentially constant over the neighborhood gives the approximation w(x + s, y + t) = w(x, y) (5.4-17) This assumption also results in the expression
w(x, y)η(x, y) = w(x, y)η(x, y) (5.4-18) in the neighborhood.
51 Periodic Noise Reduction by Frequency Domain Filtering (cont.)
With these approximations Eq(5.4-16) becomes
1 a b 2 σ 2 (x, y) = ∑ ∑{[g(x + s, y + t)− w(x + s, y + t)η(x + s, y + t)]− [g(x, y) − w(x, y)η(x, y)]} (2a +1)(2b +1) s=−a t=−b To minimize variance, we solve (5.4-19) ∂σ 2 (x, y) = 0 (5.4-20) for w(x,y) ∂w(x, y)
The result is g(x, y)η(x, y) − g(x, y)η(x, y) ( ) = w x, y 2 (5.4-21) η 2 (x, y) −η (x, y)
將w(x,y)代回Eq. (5.4-13)可獲得restored image. 52 Periodic Noise Reduction by Frequency Domain Filtering (cont.) The origin was not 圖5-20(a)的傅立 shifted to the center of 葉頻譜 the frequency plane.
53 Periodic Noise Reduction by Frequency Domain Filtering (cont.)
對應的雜訊圖樣 N(u,v)的傅立葉頻 譜
54 Periodic Noise Reduction by Frequency Domain Filtering (cont.)
Original image 處理後的影像 55 Linear, Position-Invariant Degradations
Additivity If H is a linear operator, the response to a sum of two inputs is equal to the sum of the two response.
g(x, y) = H[ f (x, y)]+η(x, y) (5.5-1) assume that η(x, y) = 0 g(x, y) = H[ f (x, y)] H is linear if
H[af1(x, y) + bf2 (x, y)] = aH[ f1(x, y)]+ bH[ f2 (x, y)] (5.5-2) If a = b =1
H[ f1(x, y) + f2 (x, y)] = H[ f1(x, y)]+ H[ f2 (x, y)] (5.5-3)
56 Linear, Position-Invariant Degradations
Homogeneity The response to a constant multiple of any input is equal to the response to that input multiplied by the same constant.
with f2 (x, y) = 0, Eq.(5.5.2) becomes
H[af1(x, y)] = aH[ f1(x, y)] (5.5-4)
57 Linear, Position-Invariant Degradations
Position (space) invariance The response at any point in the image depends only on the value of the input at that point, not on its position.
H[ f (x −α, y − β )]= g(x −α, y − β ) (5.5-5)
f(x,y)可用連續脈衝函數表示成
∞ ∞ (5.5-6) f (x, y) = f (α, β )δ (x −α, y − β )dαdβ ∫−∞ ∫−∞
假設η(x,y)=0,則將Eq(5.5-6)代入Eq(5.5-1)可得
∞ ∞ g(x, y) = H[ f (x, y)]= H f (α, β )δ (x −α, y − β )dαdβ ∫−∞ ∫−∞ (5.5-7)
因為H為線性運算子,利用加成性的性質 ∞ ∞ g(x, y) = H[ f (x, y)]= H[ f (α, β )δ (x −α, y − β )]dαdβ (5.5-8) ∫−∞ ∫−∞ 58 Linear, Position-Invariant Degradations (cont.)
又因為f(α,β)和x,y無關,因此利用Homogeneity可得
∞ ∞ g(x, y) = H[ f (x, y)]= f (α, β )H[δ (x −α, y − β )]dαdβ (5.5-9) ∫−∞ ∫−∞
其中H為脈衝響應(impulse response), h(x,α,y,β)為點展開函數(point spread function, PSF)
h(x,α, y, β ) = H[δ (x −α, y − β )] (5.5-10)
將Eq(5.5-10)代入Eq(5.5-9)可得 ∞ ∞ g(x, y) = f (α, β )h(x,α, y, β )dαdβ (5.5-11) ∫−∞ ∫−∞
如果H對脈衝的響應已知，則其對任何輸入f(α,β)的響應可由上 式計算得到。一個線性系統H，可完全由其脈衝響應來描述。
59 Linear, Position-Invariant Degradations (cont.)
若H為位置不變,則從Eq(5.5-5)可知 H[δ (x −α, y − β )]= h(x −α, y − β ) (5.5-12) 則Eq(5.5-11)可變成 ∞ ∞ g(x, y) = f (α, β )h(x −α, y − β )dαdβ (5.5-13) ∫−∞ ∫−∞ 上式為convolution integral(同Eq(4.2-30)) 若在有加成性雜訊下, Eq(5.5-11)可表示成 ∞ ∞ g(x, y) = f (α, β )h(x,α, y, β )dαdβ +η(x, y) (5.5-14) ∫−∞ ∫−∞
若H是位置不變,則Eq(5.5-14)會變成 ∞ ∞ g(x, y) = f (α, β )h(x −α, y − β )dαdβ +η(x, y) (5.5-15) ∫−∞ ∫−∞
60 Linear, Position-Invariant Degradations (cont.)
Summary 因為雜訊項η(x,y)是隨機,與位置無關的,可將Eq(5.5-15)改 寫成
g(x, y) = h(x, y)* f (x, y) +η(x, y) (5.5-16)
G(u,v) = H (u,v)F(u,v) + N(u,v) (5.5-17)
A linear, spatially invariant degradation system with additive noise can be modeled in the spatially domain as the convolution of the degradation function with an image, followed by the addition of noise.
61 Estimating the Degradation Function
There are three principal ways to estimate the degradation function for use in image restoration: Observation Experimentation Mathematical modeling
62 Estimating the Degradation Function
Estimation by image observation When a given degraded image without any knowledge about the degradation function H. To gather information from the image itself. Look at a small section of the image containing simple structures.
Look for areas of strong signal content. Gs(u,v) Construct an unblurred image as the observed subimage. Fs(u,v) Assume that the effect of noise is negligible, thus the degradation function could be estimated by Hs(u,v)=Gs(u,v)/Fs(u,v) To construct the function H(u,v) by turns out the Hs(u,v) to have the shape of Butterworth lowpass filter.
63 Estimating the Degradation Function
Estimation by experimentation A linear, space-invariant system is described completely by its impulse response. A impulse is simulated by a bright dot of light, as bright as possible to reduce the effect of noise. 把一個已知的函數加以模糊以便得到近似的H(u,v) G(u,v) = H (u,v)F(u,v) the Fourier transform of an impulse is a constant (ℑ[δ (x, y)]=1) G(u,v) ∴H (u,v) = (5.6-2) A A is a constant describing the strength of the impulse.
64 Estimating the Degradation Function
An impulse of Degraded light (光脈衝) impulse(影像脈衝)
65 Estimating the Degradation Function
Estimation by modeling Degradation model based on the physical characteristics of atmospheric turbulence: This model has a familiar form
2 2 5 / 6 H (u,v) = e−k(u +v ) (5.6-3)
where k is a constant that depends on the nature of the turbulence. With the exception of the 5/6 power on the exponent, this equation has the same form as the Gaussian lowpass filter. In fact, the Gaussian LPF is used sometimes to model mild uniform blurring.
66 Estimating the Degradation Function k=0.0025
k=0.001 k=0.00025
67 Remove the degradation of planar motion
An image has been blurred by uniform linear motion between the image and the sensor during image acquisition. Suppose that an image f(x,y) undergoes planar motion and that x0(t) and y0(t) are the time varying components of motion in the x- and y-directions. The total exposure at any point of the recording medium is obtained by integrating the instantaneous exposure over the time internal during which the imaging system shutter is open. Assuming that shutter opening and closing takes place instantaneously, and that the optical imaging process is perfect, isolates the effect of image motion.
68 Remove the degradation of planar motion (cont.)
If T is the duration of the exposure, it follows that
T g(x, y) = f [x − x0 (t), y − y0 (t)]dt (5.6-4) ∫0 where g(x,y) is the blurred image. From Eq(4.2-3), the Fourier transform of Eq(5.6-4) is
∞ ∞ G(u,v) = g(x, y)e− j2π (ux+vy)dxdy ∫−∞ ∫−∞ ∞ ∞ T − j2π (ux+vy) (5.6-5) = f [x − x0 (t), y − y0 (t)]dt e dxdy Reversing the order of ∫−∞ ∫−∞ ∫0 integration T ∞ ∞ − j2π (ux+vy) (5.6-6) = f [x − x0 (t), y − y0 (t)]e dxdy dt ∫0 ∫−∞ ∫−∞
T − π + = F(u,v)e j2 [ux0 (t) vy0 (t)]dt (5.6-7) ∫0
T − π + F(u,v) is = F(u,v) e j2 [ux0 (t) vy0 (t)]dt The Fourier transform of ∫0 independent of t the displaced function.
69 Remove the degradation of planar motion (cont.)
By defining T − π + H (u,v) = e j2 [ux0 (t) vy0 (t)]dt (5.6-8) ∫0
Eq(5.6-7) may be expressed in the familiar form G(u,v) = H (u,v)F(u,v) (5.6-9)
If the motion variables x0(t) and y0(t) are known, the transform function H(u,v) can be obtained from Eq(5.6-8). Suppose that the image in question undergoes uniform linear motion in the x-direction only, at a rate given by
x0 (t) = at /T When t=T, the image has been displaced by a total distance a.
With y0(t)=0, Eq(5.6-8) yields
T − π T − π T − π H (u,v) = e j2 ux0 (t)dt = e j2 uat /T dt = sin(πua)e j ua (5.6-10) ∫0 ∫0 πua 70 Remove the degradation of planar motion (cont.)
If we allow the y-component to vary as well, with the motion given by
y0 (t) = bt /T Then the degradation function becomes
T − π + H (u,v) = sin[π (ua + vb)]e j (ua vb) (5.6-11) π (ua + vb)
71 Remove the degradation of planar motion (cont.)
Example 5.10: Fig. (b) is an image blurred by computing the Fourier transform of the image in Fig. (a), multiply the transform by H(u,v) from Eq.(5.6-11), and taking the inverse transform. The images are of size 688x688 pixels, and the parameters used in Eq(5.6-11) were a=b=0.1 and T=1 將左圖的Fourier Transform 乘上(5.6-11)的H(u,v)後， 取其反Fourier Transform的 結果。 a=b=0.1, T=1
72 Inverse Filtering
Direct inverse filtering Compute an estimate,F ˆ ( u , v ) ,of the transform of the original image simply by dividing the transform of the degraded image, G(u,v), by the degradation function: G(u,v) Fˆ (u,v) = (5.7-1) H (u,v) G(u,v) = H (u,v)F(u,v) + N(u,v) N(u,v) ∴Fˆ (u,v) = F(u,v) + H (u,v) (5.7-2)
Even if we know the degradation function we cannot recover the undegraded image exactly because N(u,v) is a random function whose Fourier transform is not known. If the degradation has zero or very small values, then the ratio N(u,v)/H(u,v) could easily dominate the estimate Fˆ (u,v.) 73 Inverse Filtering (cont.)
One way to get around the zero or small-value problem is to limit the filter frequencies to values near the origin. We know that H(0,0) is equal to the average value of h(x,y) and that this is usually the highest value of H(u,v) in the frequency domain. Thus, by limiting the analysis to frequencies near the origin, we reduce the probability of encountering zero values. In general, direct inverse filtering has poor performance.
74 Cutoff H(u,v) a Inverse Filtering (cont.) radius of 40
直接 G(u,v)/H(u,v)
Cutoff Cutoff H(u,v) a H(u,v) a radius of 70 radius of 85
75 Minimum Mean Square Error (Wiener) Filtering
Incorporated both the degradation function and statistical characteristics of noise into the restoration process. The objective is to find an estimate f of the uncorrupted image f such that the mean square error between them is minimized. The error measure is given by 2 2 e = E{(f − fˆ ) } (5.8-1) The minimum of the error function is given in the frequency domain by H * (u,v)S (u,v) Fˆ (u,v) = f G(u,v) 2 + S f (u,v) H (u,v) Sη (u,v) H * (u,v) = G(u,v) 通常為未知，因此以 2 + K H (u,v) Sη (u,v) / S f (u,v) (5.8-2) 來估計 2 1 H (u,v) = 2 G(u,v) H (u,v) H (u,v) + S (u,v) / S (u,v) η f (5.8-3) 2 1 H (u,v) = G(u,v) 2 76 H (u,v) H (u,v) + K Example 5.12 Fig. (a) is the full inverse-filtered result shown in Fig. 5.27(a). Fig. (b) is the radially limited inverse filter result of Fig. 5.27(a). Fib. (c) shows the result obtained using Eq(5.8-3) with the degradation function used in Example 5.11.
77 Example 5.13 Gaussian noise Wiener filtering zero mean Inverse filtering variance 650 From left to right, the blurred image of Fig. 5.26(b) heavily corrupted by additive Gaussian noise of zero mean and variance of 650. The result of direct inverse filtering The result of Wiener filtering.
78 Constrained Least Squares Filtering
The difficulty of the Wiener filter: The power spectra of the undegraded image and noise must be known A constant estimate of the ratio of the power spectra is not always a suitable solution. Constrained Least Squares Filtering Only the mean and variance of the noise are needed.
79 Constrained Least Squares Filtering
We can express Eq(5.5-16) in vector-matrix form, as
g = Hf + η (5.9-1)
Suppose that g(x,y) is of size M x N, then we can form the first N elements of the vector g by using the image elements in first row of g(x,y), the next N elements from the second row, and so on. The resulting vector will have dimensions MN x 1. these are also the dimensions of f and η. The matrix H then has dimensions MN x MN Its elements are given by the elements of the convolution given in Eq(4.2-30). Central to the method is the issue of the sensitivity of H to noise. To alleviate the noise sensitivity problem is to base optimality of restoration on a measure of smoothness, such as the second derivation of an image.
80 Constrained Least Squares Filtering (cont.)
To find the minimum of a criterion function, C, defined as M −1 N −1 2 C = ∑∑[∇2 f (x, y)] (5.9-2) x=0 y=0 subject to the constraint (5.9-3) 2 2 g − Hfˆ = η 2 where w ≡ w T w is the Euclidean vector norm, and fˆ is the estimate of the undegraded image. The frequency domain solution to this optimization problem is given by the expression
* ˆ H (u,v) F(u,v) = 2 2 G(u,v) (5.9-4) H (u,v) + γ P(u,v)
where γ is a parameter that must be adjusted so that the constraint inEq(5.9-3) is satisfied. 81 Constrained Least Squares Filtering (cont.)
P(u,v) is the Fourier transform of the function.
0 −1 0 p(x, y) = −1 4 −1 (5.9-5) 0 −1 0
This function is the same as the Laplacian operator. Eq.(5.9-4) reduces to inverse filtering if γ is zero.
82 Constrained Least Squares Filtering (cont.)
γ were selected manually to yield the best visual results.
83 Constrained Least Squares Filtering (cont.)
It is possible to adjust the parameter γ interactively until acceptable results are achieved. If we are interested in optimality, the parameter γ must be adjusted so that the constraint in Eq(5.9-3) is satisfied. Define a “residual” vector r as r = g − Hfˆ (5.9-6) Since, from the solution in Eq(5.9-4), f ˆ is a function of γ, then r also is a function of this parameter. It can be shown that 2 φ(γ ) = rT r = r (5.9-7)
is a monotonically increasing function of γ. What we want to do is adjust gamma so that 2 2 r = η ± a (5.9-8) 84 Constrained Least Squares Filtering (cont.)
Because φ(γ) is monotonically increasing function , finding the desired value of γ is not difficult. Step 1: specify an initial value of γ. Step 2: Compute ||r||2 Step 3: Stop if Eq(5.9-8) satisfied; otherwise return to Step 2 after 2 2 increasing γ if r < η − a 2 2 or decreasing γ if r > η + a Use the new value of γ in Eq(5.9-4) to recompute the optimum estimate Fˆ (u,v)
85 Constrained Least Squares Filtering (cont.)
2 In order to use the algorithm, we need the quantities r 2 and η 2. To compute r , from Eq(5.9-6) that R(u,v) = G(u,v)− H (u,v)Fˆ (u,v) (5.9-9) From which we obtain r(x,y) by computing the inverse transform of R(u,v). M −1 N −1 2 2 r = ∑∑ r (x, y) (5.9-10) x=0 y=0
Consider the variance of the noise over the entire image, which we estimate by the sample-average method: M −1 N −1 2 2 1 ση = ∑∑[η(x, y)− mη ] MN x=0 y=0 (5.9-11) where 1 M −1 N −1 mη = ∑∑η(x, y) (5.9-12) MN x=0 y=0 is the sample mean. 86 Constrained Least Squares Filtering (cont.)
With reference to the form of Eq(5.9-10), the double 2 summation in Eq(5.9-11) is equal to η This gives us the expression
2 2 η = MN[ση + mη ] (5.9-13)
We can implement an optimum restoration algorithm by having knowledge of only the mean and variance of the noise.
87 Constrained Least Squares Filtering (cont.)
The initial value used for γ was 10-5, the correction factor for adjusting γ was 10-6, the value for a was 0.25.
88 Geometric Mean Filter The generalized Wiener filter is the form of the so- called geometric mean filter. 1−α α H * (u,v) H * (u,v) (5.10-1) ˆ = F(u,v) 2 G(u,v) H (u,v) 2 Sη (u,v) H (u,v) + β S f (u,v) where α and β being positive, real constants. When α=1 this filter reduces to the inverse filter With α=0, the filter becomes the so-called parametric Wiener filter. If α=0.5, the filter becomes the geometric mean filter. With β=1, as α decreases below 0.5, the filter tend more toward the inverse filter. When α increases above 0.5, the filter will behave more like the Wiener filter. When α=0.5 and β=1, the filter is referred to as spectrum equalization filter. 89 Geometric Transformations
Geometric transformations modify the spatial relationships between pixels in an image. Often be called rubber-sheet transformations. They may be viewed as the process of printing an image on a sheet of rubber and then stretching this sheet according to some predefined set of rules. A geometric transformation consists of two basic operations: Spatial transformation Defines the “rearrangement” of pixels on the image plane Gray-level interpolation Deals with the assignment of gray levels to pixels in the spatially transformed image.
90 Geometric Transformations
Spatial Transformations Suppose that an image f with pixel coordinates (x,y) undergoes geometric distortion to produce an image g with coordinates (x’, y’). This transformation may be expressed as
' x = r(x, y) (5.11-1) y' = s(x, y) (5.11-2) where r(x,y) and s(x,y) are the spatial transformations that produced the geometrically distorted image g(x’,y’). To formulate the spatial relocation of pixels by the use of tiepoints, which are a subset of pixels whose location in the input (distorted) and output (correct) images is known precisely.
91 Geometric Transformations (cont.) Figure 5.32 shows quadrilateral regions in a distorted and corresponding corrected image. The vertices of the quadrilaterals are corresponding tiepoints. Suppose that the geometric distortion process within the quadrilateral regions is modeled by a pair of bilinear equations so that
r(x, y) = c1x + c2 y + c3xy + c4 (5.11-3)
s(x, y) = c5 x + c6 y + c7 xy + c8 (5.11-4)
Then, from Eqs.(5.11-1) and (5.11-2)
x'= c1x + c2 y + c3xy + c4
y'= c5 x + c6 y + c7 xy + c8
92 Geometric Transformations (cont.) Since there are a total of eight known tiepoints, these equations can be solved for the eight coefficients ci, i=1, 2,…,8. The coefficients constitute the geometric distortion model used to transform all pixels within the quadrilateral region defined by the tiepoints used to obtain the coefficients. In general, enough tiepoints are needed to generate a set of quadrilaterals that cover the entire image, with each quadrilateral having its own set of coefficients. If we want to find the value of the undistorted image at any point (x0,y0), we simply need to know where in the distorted image f(x0, y0) was mapped. Substituting (x0,y0) into Eqs.(5.11-5) and (5.11-6) to obtain the geometrically distorted coordinates (x’0,y’0). The restored image value by letting ˆ f (x0 , y0 ) = g(x', y') The procedure continues pixel by pixel and row by row until an array whose size does not exceed the size of image g is obtained.
93 Geometric Transformations (cont.)
Depending on the values of the coefficients ci, Eqs(5.11- 5) and (5.11-6) can yield noninteger values for x’ and y’. Because the distorted image g is digital, its pixel values are defined only at integer coordinates. Thus using noninteger values for x’ and y’ causes a mapping into locations of g for which no gray levels are defined.
What the gray-level values at those locations should be? The technique used to accomplish this is called gray-level interpolation
94 Geometric Transformations (cont.) The simplest scheme for gray-level interpolation is based on a nearest neighbor approach. Zero-order interpolation (1) mapping the integer coordinates (x, y) into fractional coordinates (x’, y’) by Eqs.(5.11-5) and (5.11-6). (2) select the closest integer coordinates neighbor to (x’, y’). (3) assign the gray level of this nearest neighbor to the pixel located at (x,y).
(1) (2) (3)
95 Geometric Transformations (cont.) The drawback of nearest neighbor interpolation is producing undesirable artifacts. Smoother results can be obtained by using more complex techniques, such as Cubic convolution interpolation Which fits a surface of the sin(z)/z type through a much larger number of neighbors to obtain a smooth estimate of the gray level at any desired point. Bilinear interpolation Uses the gray levels of the four nearest neighbors to estimate interpolation value. Because the gray level of each of the four integral nearest neighbors of a nonintegral pair of coordinates (x’, y’) is known, the gray-level value at these coordinates, denoted v(x’, y’) can be interpolated from the values of its neighbors by using
v(x', y') = ax'+by'+cx' y'+d (5.11-7)
96 Geometric Transformations (cont.)
(a) Image with 25 regularly spaced tiepoints. (b) a simple rearrangement of the tiepoints to create geometric distortion.
97 Geometric Transformations (cont.)
98