Shot-Noise Adaptive Bilateral Filter Harold Phelippeau, Hugues Talbot, Stefan Bara, Mohamed Akil

Shot-noise adaptive bilateral filter Harold Phelippeau, Hugues Talbot, Stefan Bara, Mohamed Akil

To cite this version:

Harold Phelippeau, Hugues Talbot, Stefan Bara, Mohamed Akil. Shot-noise adaptive bilateral filter. International Conference on Signal Processing (ICSP’08), IEEE, 2008, Beijing, China. pp.864-867, 10.1109/ICOSP.2008.4697265. hal-00622459

HAL Id: hal-00622459 https://hal-upec-upem.archives-ouvertes.fr/hal-00622459 Submitted on 24 Sep 2013

HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. SHOT NOISE ADAPTIVE BILATERAL FILTERING

H. Phelippeau, H. Talbot, M. Akil H. Phelippeau, S. Bara

Universite´ Paris-Est, NXP Semiconductors Laboratoire A2SI, Groupe ESIEE, 2, Esplanade Anton Philips Campus Efﬁscience, 93162 Noisy-le-Grand Cedex France Colombelles BP 2000 14906, Caen Cedex 9, France {phelipph, talboth, akilm}@esiee.fr {harold.phelippeau, stefan.bara}@nxp.com

ABSTRACT the pixel at the x position in the original image, v(x) is the The bilateral filter [1, 2] has an important place in image estimated pixel at the x position, ρ and h are respectively the denoising. It smooths images while preserving edges using standard deviation of the Gaussian distribution used for the means of nonlinear combination of local pixels values. The geometrical weight calculation and the intensity weight cal- method formulation and implementation are simple. However culation. the set of the bilateral filter parameters has an important influ- The parameters ρ and h define the bilateral filtering be- ence on its filtering behavior. They have to be chosen consid- havior. The geometrical parameter is chosen considering the ering the user application. In the case of noise removing, the size of the convolution kernel. The intensity parameter h has parameters have to be adapted to the noise level, the bilateral to be chosen considering the level of filtering needed for the filter adapting then itself to the image details content. In this application. Indeed, the more the standard deviation is high paper we propose a method to estimate the best bilateral filter the more the filter is low-pass in the intensity space. An il- parameters set for shot noise removing, the dominant noise in lustration of the h influence is shown on Fig.1. In the case of digital imaging. noise removing the parameter h has to be chosen considering the noise level. The best h could be selected among a set of Index Terms — Digital imaging; image sensors; Poisson h by chosing the one that has the lowest mean squarre error noise model; noise filtering; noise reduction; noise estima- measure between the denoised image and the original version. tion. Repeating this task for each image filtering is not acceptable in term of time consumption and computational complexity. 1. INTRODUCTION In order to avoid this task, a solution consists to calibrate the best h for each noise level. Using this calibration, estimating The bilateral filter is a non linear filter used among others for the best h amouts to estimate the noise level in the image. In denoising applications. In this paper we propose a method to the following section we present a new method to estimate estimate automatically this standard deviation using the vari- shot noise level in a poisson processus acquired image. ance and the mean of smooth image region.

2. BILATERAL FILTER CHARACTERISTICS 3. BEST H PARAMETER ESTIMATION BASED ON IMAGE CONTENT The bilateral filter consists to replace a pixel by a weighted mean of its neighbors considering both their geometric close- 3.1. Sensor noise assumption ness and their photometric similarities. In this paper, we fo- cus on the Gaussian bilateral filter because all the practical Many sources are cause of noise generation in CCD and applications use this version. The Gaussian bilateral filter is CMOS sensors. These can be categorized in four main defined as follows: factors [3]: (1) The photon shot noise – associated with a random Poisson process governing the number of incident photons reaching a photosite; (2) the Photon Response Non 2 2 1 −|x−y| − |u(y)−u(x)| v(x) = e ρ2 e h2 .u(y).dy (1) Uniformity – caused by small sensitivity differences between C(x) Zβ photosites; (3) the dark current noise – produced by minor- ity carriers thermically generated in the sensors wells, also Where β represents the sliding window, y represents a set associated with a random Poisson process; and (4) the read- of 2-D pixel positions in the sliding window, x represents the out noise – resulting from thermal noise cause by MOSFET centered pixel in the sliding window, u(x) is the intensity of amplifiers. Considering a Poisson distribution process, we can assume the following relation where T −(I) represents regions of near constant values in the image I.

E(T −(I)) = V ar(T −(I)) (3)

(a) h=0 (b) h=10 From image IG we can write:

− − 2 − 2 V ar(T (I)) = E(T (IG) ) − E(T (IG)) (4)

Using 2 we can write:

− − T (IG) = G × T (I) (5)

− − (c) h=20 (d) h=40 E(T (I)) = G × E(T (IG)) (6) Replacing the expressions 5 and 6 in 4 we obtain an esti- Fig. 1. Bilateral filtering using a 5x5 square window as β and mation of the gain: a variable h E(T −(I)) G = (7) V ar(T −(I)) A sensor noise model complete with respect to these factors is proposed in [3]. According to this model, photon shot The image mean E(I) and variance V ar(I) can be simul- noise has, in standard illumination conditions, the most im- taneously calculated at each point in I by using an efficient portant influence on the output image. sliding-window algorithm [4]. We seek to establish a correlation between all E(I) and V ar(I) at each point where T − 3.2. Automatic gain control is non-zero. Using all the millions of pixels generally avail- able in modern camera sensors is not necessary, so a random In a digital imaging system, an automatic gain control (AGC) sampling of perhaps approximately 104 pixels was found suf- allows to capture images in varying light conditions by ad- ficient. Since the mean and the variance are expected to be justing the average intensity of the output signal (see Fig.2). proportional, a robust linear, intercept-free, least-square cor- By having access to the gain factor and knowing the sensor relation should be performed, the slope coefficient can then be resolution, it is possible to deduce the number of photons that interpreted as the gain factor transforming photon numbers to have reached the sensor and estimating then the noise level. recorded pixel values. In the following section we propose a method to estimate the applied gain by using images statistics and the proporties of a 3.4. Regions of near-constant values poisson distribution process. In order to estimate the gain, we need to select areas T −(I) in the noisy image I that correspond to approximately constant values. These can be readily estimated from the image gradient − T (I) = εw(∇(Gσ ⋆ I) > ϑ), (8) where I is the image, G is a Gaussian convolution of variance σ, εw the morphological erosion on a window of size w × w. Fig. 2. Digital conversion of a photon flow by a camera sen- ϑ is an intensity parameter. sor, the AGC adjusts the average image output intensity 3.5. Best h from photon density The filtering properties of the Gaussian bilateral filter varies 3.3. Photon density from image statitics with the set of its two parameters. Using a fixed sliding win- In the presence of gain manipulation only, we expect the noise dow, the filtering properties vary with the h parameter. In- variance to be proportional to average pixel values in near- tuitively we can assume that a low h should be required in constant regions, due to its Poisson characteristics. Let con- low noise level, wheras a high h is necessary in high-noise sider I the image before AGC, and IG the adjusted image af- conditions. A calibration of the best h for each noise level ter the AGC. Assuming G to be the gain factor, we can write: is necessary to establish a correspondance between the gain factor estimation and the adapted h. In the following we use IG = G × I; (2) a photon shot noise model to do this calibration. 4. EXPERIMENTS AND RESULTS

●

In this section seek to validate the previous assumptions. ●

●

4.1. Photon shot noise simulation ● To do so we simulate sensor aquisition via a regionalized ● cumulative spatial Poisson point process [5] using scene im- ● ages as probability distribution functions. This simulates ● 6.0 6.5 7.0 7.5 8.0 8.5 estimated number of photons (log10 scale) individual photons being recorded at each pixel location, as ● in a Monte-Carlo process. As scene images we used the 5.5 6.0 6.5 7.0 7.5 8.0 8.5 9.0

Kodak PhotoCD database, extracted at the 512x768 resolu- number of simulated photons (log10 scale) tion. These virtually noise-free images were scanned at the 3 full color samples per pixel from ﬁlm original following the Fig. 4. Correlation between simulated and estimated photon Kodak professional PhotoCD procedure. Fig.3 illustrate this numbers. simulation.

4.3. Best h estimation

(a) (b) (c)

Fig. 5. Correlation between simulated and estimated photon numbers. (d) (e) (f)

Fig. 3. Shot noise simulation using a sample representative image as a source and a per-pixel Poisson process. From (a) 5. CONCLUSION to (c) the mean number of photons per pixel over the whole image are 20, 80, 160. From (d) to (f) the corresponding im- In this paper, we have proposed a methodology for digital images with a gain control. age filtering in the presence of photon shot noise and a color filter array. For this, we have studied a subset of known filters that can be efficiently implemented in low-power CPUs and DSPs found in consumer digital cameras. We have shown 4.2. Photon number estimation that modeling noise in this context by a Gaussian distribution over an RGB image is not accurate, especially at high noise We ran trials with a photon density per pixel between 20 and levels. We have also shown that when the noise standard de- 320, from an extremely noisy to a clean image. We estimated viation is sufficiently low, then it is better not to filter at all, the number of photons Φ¯ with the following formula: even with the best filters, using fixed parameters. It is there- fore important to adapt the level of filtering to the level of noise. Among the tested filters, the bilateral filter was shown nbpix(I)E(I) x,y ∈I I(x, y) Φ(¯ I) = = ( ) , (9) G P G to be the one with best performance. In addition, we showed how this filter can be optimised if the noise level is known. where nbpix(I) is the total number of pixels in I. The ex- We have proposed a simple and efficient noise level detection cellent correlation between simulated and estimated photons algorithm. This algorithm was shown to exhibit good correla- is shown on the log-log plot of Fig. 5 (R2 = 0.996). tion between estimated and real noise level. By applying this noise level detection method to set the bilateral filter h parameter, we obtain a novel adaptive bilateral filter, which exhibits best-of-class performance at all noise levels.

6. REFERENCES

[1] Carlo Tomasi and Roberto Manduchi, “Bilateral ﬁltering for gray and color images,” pp. 839–846, 1998. [2] Stephen M. Smith and J. Michael Brady, “Susan a new approach to low level image processing,” Int. J. Comput. Vision, vol. 23, no. 1, pp. 45–78, 1997. [3] Roberto Costantini and Sabine Susstrunk, “Virtual sensor (a) (b) (c) design,” Proc. IS&T/SPIE Electronic Imaging 2004: Sen- sors and Camera Systems for Scientiﬁc, Industrial, and Digital Photography Applications, vol. 5301, pp. 408– 419, 2004. [4] C. Sun, “Fast algorithm for local statistics calculation for n-dimensional images,” Journal of Real-Time Imaging, (d) (e) (f) vol. 7, no. 6, pp. 519–527, December 2001. [5] M. Shinozuka and C.M. Jan, “Digital simulation of random processes and its applications,” Journal of Sound and Vibration, vol. 25-1, pp. 111–128, 1972.

(g) (h) (i)

(j) (k) (l)

(m) (n) (o)

Fig. 6. Shot noise simulation using a sample representative image as a source and a per-pixel Poisson process. From (a) to (c) the mean number of photons per pixel over the whole image are 20, 80, 160. From (d) to (f) the corresponding images with a gain control.