Sparse Norm Filtering

Sparse Norm Filtering Chengxi Ye†, Dacheng Tao‡, Mingli Song§, David W. Jacobs†, Min Wu† †Department of Computer Science, University of Maryland ‡Centre for Quantum Computation and Intelligent Systems, University of Technology Sydney §College of Computer Science, Zhejiang University (a) (b) (c) Figure 1: Examples of filtering results using different norms. (a) Left: original image Middle: smoothed image via minimizing l0 energy. Right: sharpened image. (b) Up: image with pepper and salt noise. Down: smoothed result by minimizing the l1 norm. (c) Left: drag-and-drop editing. Right: seamlessly editing using l2 norm filtering. Abstract studied for decades and have been proven to be critical for a wide variety of applications including blurring, sharpening, Optimization-based filtering smoothes an image by minimiz- stylization and edge detection. ing a fidelity function and simultaneously preserves edges by exploiting a sparse norm penalty over gradients. It has ob- In general, existing edge-preserving filtering techniques can tained promising performance in practical problems, such as be classified into the following two groups: weighted average detail manipulation, HDR compression and deblurring, and filtering and optimization-based filtering. thus has received increasing attentions in fields of graphics, computer vision and image processing. This paper derives Well-known techniques of weighted average filtering includes a new type of image filter called sparse norm filter (SNF) anisotropic diffusion [Perona and Malik 1990; Black and from optimization-based filtering. SNF has a very simple Sapiro 1998] and bilateral filtering [Tomasi and Manduchi form, introduces a general class of filtering techniques, and 1998]. Anisotropic diffusion uses the gradients of each pixel explains several classic filters as special implementations of to guide a diffusion process and avoids blurring across edges. SNF, e.g. the averaging filter and the median filter. It has The bilateral filter can be regarded as a non-local diffusion advantages of being halo free, easy to implement, and low process that uses pixel intensities within a neighborhood to time and memory costs (comparable to those of the bilateral guide the diffusion. Both approaches can be implemented filter). Thus, it is more generic than a smoothing operator using explicit weighted averaging. Acceleration of weighted and can better adapt to different tasks. We validate the filtering has been a research hotspot in recent years [Paris proposed SNF by a wide variety of applications including and Durand 2007; Porikli 2008; Yang et al. 2009; He et al. edge-preserving smoothing, outlier tolerant filtering, detail 2010b; Gastal and Oliveira 2012]. manipulation, HDR compression, non-blind deconvolution, Optimization-based filtering formulates edge preserving fil- arXiv:1305.3971v1 [cs.GR] 17 May 2013 image segmentation, and colorization. tering as an optimization problem that consists of a fidelity term and a penalty term [Rudin et al. 1992; Farbman et al. CR Categories: I.3.3 [Computer Graphics]: Pciture/Im- 2008; Xu and Lu 2011]. Edge preserving is enforced by in- age Generation|Display Algorithms I.4.3 [Image Processing troducing a sparse norm penalty on the gradients, thus the and Computer Vision]: Enhancement|Grayscale Manipula- cost function is usually non-quadratic, and solving the sys- tion I.4.9 [Image Processing and Computer Vision]: Appli- tem is more time consuming [Wang et al. 2008] compared cation; with weighted average filtering. Nevertheless, this framework often produces high quality results. Keywords: sparse norm, image filtering, optimization In this paper, we present a novel type of edge-preserving filter, called sparse norm filter (SNF), derived from a sparse op- 1 Introduction timization problem. For each pixel, the filtering output mini- mizes its difference with its neighboring pixels; the penalty is Image filtering plays a fundamental role in image processing, defined by a sparse norm. Although SNF is closely related computer graphics and computer vision, and has been widely to and produces results as excellent as optimization-based used to reduce noise and extract useful image structures. In filters, it is conceptually and computationally simpler than particular, edge-preserving smoothing operations have been optimization-based filters. SNF naturally preserves edges through the use of the sparse the similarity with the input I. When p = q = 2, the op- norm, and is capable of producing halo-free filtering effects, timization problem is the well-known Tikhonov regulariza- which is a desirable but lacking property of current weighted tion [Tikhonov et al. 1995]. The explicit solution can be average filtering techniques. We demonstrate many of the found by other favorable properties of this simple and versatile ap- proach to filtering via a wide variety of applications. Fig.1 −1 shows some applications of our filtering technique. Fig.1(a) B = Id + l∇T ∇ I: (5) demonstrates our smoothing and sharpening results that ap- proximate the l0 energy. Note that the filtering result pre- Since sparse norms have better tolerance for outliers than serves edges and does not introduce halos. Fig.1(b) shows 2 1 the l norm, the optimization was later extended to to- our l norm filtering effect to remove the pepper and salt tal variation regularization with p = 1 [Rudin et al. 1992] noise. Fig.1(c) shows a new way of seamless editing en- and even sparser versions [Farbman et al. 2008][Xu and 2 abled by the l norm filtering. More detailed discussions on Lu 2011] for edge-preserving purposes. Solving these non- applications will be presented in the Applications section. quadratic optimizations is more time-consuming. Thus, vari- able splitting [Wang et al. 2008] is usually exploited to cast 2 Background the original large optimization problem into several small sub-problems and alternatively minimize each of these sub- One simple and classic way to smooth an image is to min- problems. imize the difference of each pixel with nearby ones, which can be formulated as 3 The Sparse Norm Filter 2 3.1 Definition min Inew − I : (1) new ∑ i j Ii j2N i We propose SNF by generalizing (1) through allowing the original l2 norm to be a fractional-norm. To preserve strong The solution of this optimization can be found by averaging edges, we need to smooth the image while to tolerate the the nearby pixels and is known as the box filter when we outlier pixels by assigning lower weights to them. This type consider a square neighborhood of adaptive weighting ideas have been well explored in robust statistics [Black and Sapiro 1998] and we achieve this by 1 exploiting sparse norms. Then, SNF is defined by Inew = I : (2) i jN j ∑ j i j2Ni min jInew − I jp;0 < p <= 2: (6) new ∑ i j Ii j2N The box filter can be calculated in linear time with the in- i tegral image technique [Porikli 2008]. However, it does not Minimizing this non-quadratic cost function when p < is preserve the salient structures or edges in an image. 2 difficult. Especially when p < 1, the cost function is non- Modern filtering techniques solve this problem by taking the convex and conventional gradient descent-based algorithms weighted average of nearby pixels [Perona and Malik 1990; are easily trapped into local minima. Black and Sapiro 1998; Tomasi and Manduchi 1998]. In the In this paper, we consider two approximation strategies. The anisotropic diffusion framework, the neighborhood consists first strategy iteratively exploits the weighted least square of the adjacent pixels, and the system has to be iterated tens technique of times to produce good smoothing result. Most recent filtering techniques consider a larger neighborhood consisting of tens or hundreds of neighboring pixels and the filtering is new p p−2 new 2 solved in one or a few rounds. Edges are preserved by con- ∑ jIi − Ijj ≈ ∑ jIi − Ijj Ii − Ij structing the weight matrix with the criterion that similar j2Ni j2Ni nearby pixels shall be given higher weights. As an exam- 2 = w Inew − I : (7) ple, the bilateral filter [Tomasi and Manduchi 1998] uses the ∑ i j i j j2N intensity to measure similarity and assigns weights by i By taking the derivative, we find that the solution can be approximated by using weighted average filtering Inew = 2 ! 2 ! i w I I − I w I new ∑ j2Ni i j j ( j − i) j i ∑ j2Ni i j j p−2 I = ;w = exp − exp − : , with wi j = jIi − Ijj . This solution can be under- i i j 2 2 ∑ j2N wi j ∑ j2N wi j 2ss 2sr i Ni stood as one iteration of the anisotropic diffusion process, (3) with the diffusity wi j calculated based at the current pixel Edge-preserving image smoothing can also be achieved by intensity. This way of weighting naturally enforces fidelity solving the following optimization problem with the input image. Similar to the anisotropic diffusion, we can iteratively update the diffusity once we update the image with this weighted average filtering result. In practice, minkB − Ikq + lk∇Bkp: (4) like the bilateral filter, one iteration is usually sufficient be- B cause the diffusion is non-local. It is noteworthy that when Ii = Ij, the weight goes to infinity. In practice, we can avoid The penalty term lk∇Bkp controls the amount of smooth- this by setting a threshold and raising the pixel differences ness of the output B and the fidelity term kB − Ikq controls by/to the threshold. We can also modify the optimization by weighting pixels according to distance using a Gaussian-like The norm in the fidelity term is usually an l2 norm in existing weight. However, we observe that treating all neighboring works. SNF simplifies (8) by integrating the fidelity term pixels equally is good enough in practice. and the sparse norm penalty. By setting l = 1, changing the lq norm in the first term to the l p norm, and defining Another strategy quantizes the solution into a set of discrete the neighborhood to contain the current pixel, we can write values.

Load more