IMAGE RESTORATION USING A HYBRID COMBINATION OF PARTICLE FILTERING AND WAVELET DENOISING Yan Zhai, Mark Yeary, Victor DeBrunner, Joseph P. Havlicek, and Osama Alkhouli School of Electrical and Computer Engineering University of Oklahoma, Norman, OK USA Corresponding Email: fyan.zhai-1, yeary, vdebrunn, [email protected] ABSTRACT propagates a set of weighted samples (or particles) to simulate the posterior distribution of a system’s trajectory in state space. Since In this paper we propose a novel image restoration method PF inherently supports nonlinear dynamics and non-Gaussian forc- that effectively combines a particle filter with wavelet shrinkage ing and measurement noises, we use it here to suppress the heavily to achieve robust performance against inhomogeneous noise mix- tailed components of the corrupting noise, whereas wavelet shrink- tures. Specifically, the particle filter acts to suppress outlier-rich age is applied in a subsequent step to attenuate the Gaussian noise components of the noise while, in a subsequent step, the wavelet components. domain shrinkage attenuates any remaining, less heavily tailed noise components. We present late breaking preliminary examples demonstrating excellent rejection of salt-and-pepper like Cauchy 2. PROPOSED METHOD: PF-DWT DENOISING noise mixed with additive white Gaussian noise (AWGN). Al- When an image is corrupted by noise comprising both heavily though limited in scope, these preliminary results suggest that tailed and Gaussian components, we have observed that undesir- the combination of particle filters with more traditional restoration able blurring artifacts are often introduced if one attempts to sup- techniques is a powerful approach that can provide a new dimen- press both components simultaneously. Therefore, we introduce a sion of flexibility for addressing noise mixtures involving difficult two-stage hybrid denoising technique called PF-DWT. In the first nonlinear and non-Gaussian components. stage, a spatial PF is used to suppress the heavy tailed component of the noise. In the second stage, wavelet thresholding is applied 1. INTRODUCTION to attenuate the remaining “Gaussian-like” noise. The input of the second stage depends on the output of the first stage, thus realizing The objective of restoration is to recover an image that has been a synergistic hybrid approach that is robust and effective against a corrupted by distortion and noise. In general, the distortion could wide variety of inhomogeneous noise mixtures. be linear or nonlinear and the noise might be Gaussian, heav- ily tailed, or both [1]. Traditional approaches typically involve 2.1. General Form of The Particle Filter a tradeoff between a deconvolution process designed to combat the distortion and a denoising process. In this paper, we focus on In this section, we briefly review the general PF framework de- the denoising part of the problem and specifically consider noise scribed in [6] - [8]. Consider a nonlinear system modeled in state mixtures involving simultaneous heavily tailed and Gaussian com- space according to ponents. Although the median filter is ideal for rejecting impul- xt = f(xt¡1) + vt¡1; (1) sive noise, its performance has been shown to be suboptimal when Gaussian noise components are also present [2]. yt = h(xt) + nt; (2) Shrinkage methods based on thresholding in the discrete where xt and yt denote the hidden states and the observations of wavelet domain have attracted significant attention in image de- the system at time t, respectively. Both f(¢) and h(¢) could be non- noising applications [3]- [5]. The basic idea is to decompose the linear functions. For image processing, the time index t is replaced noisy image using the discrete wavelet transform (DWT) and then by spatial indices i and j representing the row and column of a apply thresholding, where it is assumed that small coefficients in pixel, respectively. The process and observation noises, which are the high frequency (small spatial scale) subbands are associated assumed non-Gaussian, are given by vt and nt, respectively. The with noise and can be set to zero without substantially affecting goal of the PF is to simulate the posterior distribution p(x0:tjy1:t), the main visually important features of the image. which, according to Bayes’ theorem, is given by In this paper we consider the combination of particle filter- p(y1:tjx0:t) p(x0:t) ing (PF) with wavelet shrinkage as a new hybrid approach for p(x0:tjy1:t) = R : (3) dealing with difficult, inhomogeneous noise mixtures containing p(y1:tjx0:t) p(x0:t) dx0:t both Gaussian and heavily tailed components. The PF technique Applying SIS, the state vector posterior distribution is approxi- has emerged as a superior method for treating nonlinear and non- mated by Gaussian problems [6]- [8] and has been applied recently for im- XNs age and video processing [9]- [11]. Also known as sequential im- (i) (i) p(x0:tjy ) ¼ !~ ±(x0:t ¡ x ) ; (4) portance sampling (SIS), PF is a Monte Carlo (MC) method that 1:t t 0:t i=1 0-7803-9134-9/05/$20.00 ©2005 IEEE (i) where the normalized importance weight !~t is given by 2.2.2. The Spatial Particle Filter Algorithm (i) The first step in designing a particle filter is to choose a proposal (i) !t !~t = P (5) distribution. While arbitrary in form, the proposal must be sup- Ns !(j) j=1 t ported by the posterior distribution of the true system state and and where must be easy to sample. For PF-DWT we use a standard Kalman filter to provide the proposal distribution, since the image model is p(y jx(i))p(x(i)jx(i) ) a linear system with non-Gaussian noise. Although the estimates !(i) = !(i) t t t t¡1 : t t¡1 (i) (i) (6) delivered by the Kalman filter are not optimal in this case, they q(x jx ; y ) t 0:t¡1 1:t still provide the required support for the true posterior. The 2-D Equation (6) is the importance weight update equation. The par- Kalman filter formulation is given by (i) ticles (or samples) xt are drawn from the proposal distribution x¹(i; j) =Cx^(i; j ¡ 1) + Eu(i; j) (10) q(x jx(i) ; y ). In addition, p(y jx(i)) and p(x(i)jx(i) ) repre- t 0:t¡1 0:t t t t t¡1 ¹ T T sent the system’s likelihood and the state transition prior, respec- P (i; j) =CP (i; j ¡ 1)C + DQD (11) T T ¡1 tively. The variable Ns denotes the number of particles. The pro- K(i; j) =P¹(i; j)H [HP¹(i; j)H + R] (12) posal distribution is an arbitrary distribution which needs to have x^(i; j) =x¹(i; j) + K(i; j)[y(i; j) ¡ Hx¹(i; j)] (13) at least some support from the posterior distribution of the true state. In addition, it has been shown that the variance of the im- P (i; j) =[I ¡ K(i; j)H]P¹(i; j) ; (14) portance weights increases over time [6]. This implies that after a where R and Q denote the covariance of the process noise and few iterations, all but one weight will converge to zero. When this measurement noise, respectively. Finally, the spatial particle filter occurs, a resampling scheme is introduced to solve this degeneracy algorithm is illustrated in Fig. 1 and is also summarized below. problem. More specifically, in a resampling scheme, a new set of equally weighted particles are generated from the previous parti- cles with large weights, while the particles with small weight are 1. Sequential Importance Sampling (SIS) Step: replaced. In other words, resampling discards the particles with ² At each iteration, calculate x^(i; j) and P (i; j) according small weights and focuses on the particles with more significant to (10)-(14). weights. ² Sample from the proposal distribution: 2.2. Particle Filter For Image Processing (Spatial Case) x(i)(i; j) » N (x^(i; j);P (i; j)) : To implement the PF, we employ a 2-D state space model similar to the ones used for Kalman filtering in [2, 12, 13]. ² Evaluate the importance weights according to (6): calcu- late the transition prior, the likelihood and the proposal. 2.2.1. The Image Model ² Normalize the importance weights using (5). We use the image model proposed in [13] since it is efficient in the 2. Resample and Update Step: sense of using only a small number of pixels: ² Generate a new set of particles xi?(t) from x(i)(i; j), so that I(i; j) =h1I(i; j ¡ 1) + h2I(i ¡ 1; j) ³ ´ i? (j) (j) + h3I(i ¡ 1; j ¡ 1); (7) P r x (i; j) = x (i; j) =! ~ (i; j) : th th where I(i; j) denotes the pixel at the i row and j column of the ² The final estimate of x(i; j) is given by image. In (7), h1, h2, and h3 are image parameters which can be estimated by various methods including, e.g., least-squares. Next, Ns 1 X the following state space model is constructed: x(i; j) ¼ xi?(i; j) : Ns i=1 x(i; j) =Cx(i; j ¡ 1) + Eu(i; j) + Dw(i; j); (8) y(i; j) =Hx(i; j) + v(i; j); (9) ² Update the proposal with the resampled particles. where 2 3 2 3 I(i; j) h1 0 h2 h3 2.3. Wavelet Thresholding for PF-DWT 6 I(i; j ¡ 1) 7 6 1 0 0 0 7 x = C = 4 I(i ¡ 1; j + 1) 5 4 0 0 0 0 5 We begin by applying the DWT to decompose the data into sub- I(i ¡ 1; j) 0 0 1 0 bands.