Journal of Mathematical Imaging and Vision manuscript No. (will be inserted by the editor) Two Enhanced Fourth Order Diffusion Models for Image Denoising Patrick Guidotti · Kate Longo Received: date / Accepted: date Abstract This paper presents two new higher order In this paper, only grey scale images are considered, so diffusion models for removing noise from images. The u is a scalar valued function taking on quantized integer models employ fractional derivatives and are modifi- values between 0 and 255. u is an observed image, con- cations of an existing fourth order partial differential sisting of a “true” image v polluted with noise n. To de- equation (PDE) model which was developed by You and noise an image is to recover v from the observed image Kaveh as a generalization of the well-known second or- u, a theoretically impossible task. Indeed fine details of der Perona-Malik equation. The modifications serve to an image can be indistinguishable from noise and all cure the ill-posedness of the You-Kaveh model without known denoising methods can cause various degrees of sacrificing performance. Also proposed in this paper is blurring, staircasing, and other artifacts. a simple smoothing technique which can be used in nu- Noise reduction, in particular PDE-based noise re- merical experiments to improve denoising and reduce duction, has been a subject of much research since a processing time. Numerical experiments are shown for seminal paper by Perona and Malik in 1990 [29] which comparison. introduced the then novel paradigm of using nonlin- ear diffusions for the task of denoising images. Their Keywords Nonlinear Diffusion · Fractional Deriva- method improves upon the technique of linear diffusion tives · Image Denoising · Fourth Order used previously (and introduced by Witkin [33]) by re- ducing the diffusivity at locations of the image where edges are found by an appropriate edge detector. The 1 Introduction latter is typically implemented by measuring the gradi- ent |∇u| of the image. The Perona-Malik model is based Noise is an unavoidable component of digital image on the equation acquisition, and as such, noise removal has become a fundamental task of image processing. The problem is ut − ∇ · (g(|∇u|)∇u) = 0, (1.1) formulated by considering an image as a collection of where g(·) is a diffusivity such as intensity data about N pixels, 1 g(s) = , c > 0 . (1.2) u(i) = v(i) + n(i), i = 1, ..., N. 1 + c2s2 Discretizations of (1.1) have proved to be effective de- The authors of this research were supported by the National Sci- ence Foundation under award number DMS-0712875. noising tools. However, two significant problems have been observed. Firstly, equation (1.1) has been shown Patrick Guidotti to be mathematically ill-posed [22], which makes it im- Department of Mathematics University of California at Irvine possible to prove theoretical results about the behavior Irvine, CA 92691-3875, USA of algorithms based on that equation. Secondly, in nu- E-mail: [email protected] merical experiments, such algorithms tend to produce Kate Longo artifacts at edges, such as staircasing (when false edges E-mail: [email protected] are introduced) [22,32] and blocky, cartoonish effects 2 (when smooth edges are sharpened into corners) [34]. trated with numerical experiments in section 5, and the The literature abounds in attempts to cure these prob- paper is concluded in section 6. lems, including regularizing the edge detector |∇u| [1,2, 5,8,18], and generalizing the ideas of Perona and Malik to higher order equations [30,6,14,31,20,21,34,25]. Of 2 Background note is a fourth order generalization of (1.1) proposed by You and Kaveh which is the focus of this paper. Nu- Use of diffusion PDEs for noise removal can be traced merical experiments in the aforementioned papers and back to the scale space method introduced by Witkin others show that many fourth order denoising models in 1983 [33]. This method smoothes a noisy image by are successful at avoiding the staircasing and cartoon- convolving it with Gaussian kernels on a scale of vari- ishness characteristic of second order models, and excel ances. It is equivalent to considering a smoothed image at preserving a natural look to images such as human as the solution to the linear heat equation faces. This success often comes at the cost of good per- ut = ∇ · (γ∇u), u(0) = u0 (original image), (2.1) formance in flat regions of images, and many fourth or- der denoising methods leave a kind of splotchy artifact where the diffusion coefficient γ is constant. This is an in such regions (see for example experiments in [20,21, effective smoothing method, but it cannot distinguish 25,10]). Additionally, equation (2.7) has been noted to noise from edges, thus blurring the entire image. The produce a speckle artifact, which is removed by the au- method needs to be complemented by a second pro- thors of [34] in a post-processing step. Improvements of cessing step which locates and reintroduces edges. In a models such as (2.7) can be gained through the con- 1990 paper [29], Perona and Malik propose their idea tributions of this paper. Proposed in this paper are to preserve edges in the first step by replacing γ with two new PDE models which transplant ideas from reg- an “edge detector”, a nonlinear function which would ularized second order models [18] to the fourth order inhibit diffusion across edges. They observe that edges You-Kaveh model and which offer analytical as well as in an image correspond to regions of high gradient, and practical benefits. The new models utilize a fractional thus consider γ = g(|∇u|), where g(·) is chosen appro- Laplacian operator and a fractional gradient, respec- priately so as to slow diffusion (become small) when tively, to detect edges and limit blurriness. |∇u| is large. One possibility for such a function sug- Processing time can be reduced and denoising re- gested by Perona and Malik is given in (1.2). In their sults improved through the use of a novel smoothing paper, they propose discrete equations which can be step involving a simple manipulation of parameters. incorporated into a continuous PDE model for image The proposed equations are discretized with a spec- processing. The Perona-Malik model can be viewed as tral scheme. With the combination of this numerical an approximation of equation (1.1), which is therefore scheme and smoothing technique, we are able to ob- commonly referred to as the Perona-Malik equation in tain improved denoising results compared to previous the literature. Analytically, PDE models based on (1.1) experiments shown with (2.7) and its variants. Specif- have been found to be ill-posed [22]. Despite, or rather ically, the splotchy artifact characteristic of fourth or- because of this, the Perona-Malik model preserves sharp der denoising methods can be alleviated without sacri- edges well, but still doesn’t escape some practical draw- ficing edge preservation, and speckling is not observed backs, including the creation of artifacts such as stair- at all, with either (2.7) or the proposed modifications. casing and blocky effects [22,32,34]. The smoothing technique is not particular to the pro- Attempts to overcome the problematic issues of Per- posed fourth order equations; experiments are shown ona-Malik can be broadly classified into two categories: to demonstrate the ability of the smoothing step to im- second order regularizations and relaxations of Perona- prove the performance of a previously proposed second Malik, and higher order diffusions. Spatial regulariza- order PDE for noise removal, as well as to dramatically tions have been considered, which involve smoothing reduce the necessary processing time. the argument in the nonlinear edge detector function ∞ The paper is organized as follows. Key results from g with a C kernel Gσ, typically a Gaussian. See, for the literature about second and fourth order diffusions example, [1,8,27]. This leads to the equation and their regularizations are summarized in section 2. ut − ∇ · g(|∇Gσ ∗ u|)∇u = 0 , in Ω for t > 0. (2.2) In section 3, the behavior of the You-Kaveh model and the effects of various parameters are analyzed, and the Other authors have proposed models in which the non- smoothing technique is proposed and its ability to im- linearity is regularized with a space-time convolution, prove performance is demonstrated. Section 4 details namely [11,12,28]. Purely temporal regularizations are the two proposed models, their performance is illus- considered in [2,4,5]. 3 An irony of the Perona-Malik model is that its great- (conferring a “cartoonish” feel to processed images), est strengths, superior edge detection capabilities, are however, remain. See figures 3 and 9 for examples of made possible by the nonlinearity g(|∇u|), which is also the denoising effects of (2.3). Since |∇1−u| is itself an responsible for its weaknesses, ill-posedness and arti- effective edge detector, blurring is not observed, as it is facts. Consequently, regularizing this nonlinearity can with other regularized models such as (2.2). cure ill-posedness, but often at the cost of good edge Other researchers have opted to modify the Perona- detection. This is particularly true of regularizations of Malik model by generalizing it to higher orders instead the type in (2.2). Reintroducing a smooth kernel Gσ of by applying regularizations. Fourth order PDEs from results in blurring, exactly what Perona and Malik try the literature include those proposed in [31,34,6,25,20, to avoid. 21]. One of most frequently cited is the model derived Guidotti in [18,17,16] proposes two different, “mild- by You and Kaveh in [34] from a variational formula- er” regularizations characterized by the use of fractional tion.