3D Discrete Shearlet Transform and Video Processing Pooran Singh Negi and Demetrio Labate
Total Page:16
File Type:pdf, Size:1020Kb
IEEE TRANSACTIONS ON IMAGE PROCESSING 1 3D Discrete Shearlet Transform and Video Processing Pooran Singh Negi and Demetrio Labate Abstract—In this paper, we introduce a digital implementation Up to the log-like factor, this is the optimal approximation rate, of the 3D shearlet transform and illustrate its application to in the sense that no other orthonormal systems or even frames problems of video denoising and enhancement. The shearlet can achieve a rate better than M −2. By contrast, wavelet representation is a multiscale pyramid of well-localized wave- −1 forms defined at various locations and orientations, which was approximations can only achieve a rate M for functions in introduced to overcome the limitations of traditional multiscale this class [3]. Concerning the topic of sparse approximations, it systems in dealing with multidimensional data. While the shearlet is important to recall that the relevance of this notion goes far approach shares the general philosophy of curvelets and sur- beyond the applications to compression. In fact, constructing facelets, it is based on a very different mathematical framework sparse representations for data in a certain class entails the which is derived from the theory of affine systems and uses shearing matrices rather than rotations. This allows a natural intimate understanding of their true nature and structure, so transition from the continuous to the digital setting and a that sparse representations also provide the most effective tool more flexible mathematical structure. The 3D digital shearlet for tasks such as feature extraction and pattern recognition [6], transform algorithm presented in this paper consists in a cascade [7]. of a multiscale decomposition and a directional filtering stage. Even though shearlets and curvelets share the same phi- The filters employed in this decomposition are implemented as finite-length filters and this ensures that the transform is local losophy of combining multiscale and directional analysis and and numerically efficient. To illustrate its performance, the 3D have similar sparsity properties, they rely on a rather different Discrete Shearlet Transform is applied to problems of video mathematical structure. In particular, the directionality of the denoising and enhancement, and compared against other state-of- shearlet systems is controlled through the use of shearing ma- the-art multiscale techniques, including curvelets and surfacelets. trices rather than rotations, which are employed by curvelets. This offers the advantage of preserving the discrete integer Index Terms—Affine systems, curvelets, denoising, shearlets, lattice and enables a natural transition from the continuous to sparsity, video processing, wavelets. the discrete setting. The contourlets, on the other hand, are a purely discrete framework, with the emphasis in the numerical I. INTRODUCTION implementation rather than the continuous construction. The The shearlet representation, originally introduced in [1], [2], special properties of the shearlet approach have been success- has emerged in recent years as one of the most effective frame- fully exploited in several imaging application. For example, the works for the analysis and processing of multidimensional combination of multiscale and directional decomposition using data. This representation is part of a new class of multiscale shearing transformations is used to design powerful algorithms methods introduced during the last 10 years with the goal for image denoising in [7], [8]; the directional selectivity of the to overcome the limitations of wavelets and other traditional shearlet representation is exploited to derive very competitive methods through a framework which combines the standard algorithms for edge detection and analysis in [9]; the sparsity multiscale decomposition and the ability to efficiently capture of the shearlet representation is used to derive a very effective anisotropic features. Other notable such methods include the algorithm for the regularized inversion of the Radon transform curvelets [3] and the contourlets [4]. Indeed, both curvelets in [10]. We also recall that a recent construction of compactly and shearlets have been shown to form Parseval frames of supported shearlets appears to be promising in PDE’s and other 2 2 L (R ) which are (nearly) optimally sparse in the class of applications [11], [12]. cartoon-like images, a standard model for images with edges While directional multiscale systems such as curvelets and [3], [5]. Specifically, if fM is the M term approximation shearlets have emerged several years ago, only very recently obtained by selecting the M largest coefficients in the shearlet the analysis of sparse representations using these representa- or curvelet expansion of a cartoon-like image f, then the tions has been extended beyond dimension 2. This extension approximation error satisfies the asymptotic estimate is of great interest since many applications from areas such jj − S jj2 ≍ −2 3 ! 1 as medical diagnostic, video surveillance and seismic imaging f fM 2 M (log M) ; as M : require to process 3D data sets, and sparse 3D representations Copyright (c) 2010 IEEE. Personal use of this material is permitted. are very useful for the design of improved algorithms for data However, permission to use this material for any other purposes must be obtained from the IEEE by sending a request to [email protected]. analysis and processing. P. S. Negi is with the Department of Mathematics, University of Houston, Notice that the formal extension of the construction of Houston, TX 77204, USA (e-mail:[email protected]). multiscale directional systems from 2D to 3D is not the D. Labate is with the Department of Mathematics, University of Houston, Houston, TX 77204, USA (e-mail:[email protected]). major challenge. In fact, 3D versions of curvelets have been EDICS: TEC-RST, TEC-MRS introduced in [13], with the focus being on their discrete ξ3 40 ξ3 ξ3 40 40 30 30 20 20 20 10 ξ1 ξ1 10 ξ1 0 0 ξ2 ξ2 0 ξ2 −10 −10 −20 −20 −20 −30 −30 −40 −40 −40 40 40 40 20 40 20 20 0 40 20 40 0 20 20 0 −20 0 0 0 −20 −20 −20 −20 −20 −40 −40 −40 −40 −40 −40 b3 Fig. 1. From left to right, the figure illustrates the pyramidal regions P1, P2 and P3 in the frequency space R . implementations. Another discrete method is based on the to various orthogonal transformations controlled by shearing system of surfacelets that were introduced as 3D extensions matrices. of contourlets in [14]. However, the analysis of the sparsity In dimension D = 3, a shearlet system is obtained by properties of curvelets or shearlets (or any other similar appropriately combining 3 systems of functions associated systems) in the 3D setting does not follow directly from the with the pyramidal regions 2D argument. Only very recently [15], [16] it was shown { } 3 ξ2 ξ3 by one of the authors in collaboration with K. Guo that 3D P1 = (ξ1; ξ2; ξ3) 2 R : j j ≤ 1; j j ≤ 1 ; shearlet representations exhibit essentially optimal approxima- ξ1 ξ1 tion properties for piecewise smooth functions of 3 variables. { } 3 ξ1 ξ3 Namely, for 3D functions f which are smooth away from P2 = (ξ1; ξ2; ξ3) 2 R : j j < 1; j j ≤ 1 ; discontinuities along C2 surfaces, it was shown that the M ξ2 ξ2 S { } term approximation fM obtained by selecting the N largest 3 ξ1 ξ2 coefficients in the 3D Parseval frame shearlet expansion of f P3 = (ξ1; ξ2; ξ3) 2 R : j j < 1; j j < 1 ; satisfies the asymptotic estimate ξ3 ξ3 jj − S jj2 ≍ −1 2 ! 1 Rb3 f fM 2 M (log M) ; as M : (1) in which the Fourier space is partitioned (see Fig. 1). To define such systems, let ϕ be a C1 univariate function Up to the logarithmic factor, this is the optimal decay rate and ≤ ^ ≤ ^ − 1 1 ^ such that 0 ϕ 1, ϕ = 1 on [ 16 ; 16 ] and ϕ = 0 outside the significantly outperforms wavelet approximations, which only interval [− 1 ; 1 ]. That is, ϕ is the scaling function of a Meyer −1=2 8 8 yield a M rate for functions in this class. wavelet, rescaled so that its frequency support is contained the It is useful to recall that optimal approximation properties 1 1 b3 interval [− ; ]. For ξ = (ξ1; ξ2; ξ3) 2 R , define for a large class of images can also be achieved using adaptive 8 8 b b ^ ^ ^ methods by using, for example, the bandelets [17] or the Φ(ξ) = Φ(ξ1; ξ2; ξ3) = ϕ(ξ1) ϕ(ξ2) ϕ(ξ3) (2) grouplets [18]. The shearlet approach, on the other hand, q in non-adaptive. Remarkably, shearlets are able to achieve and let W (ξ) = Φb2(2−2ξ) − Φb2(ξ): It follows that approximation properties which are essentially as good as an X adaptive approach when dealing with the class of cartoon-like Φb2(ξ) + W 2(2−2jξ) = 1 for ξ 2 R3: (3) images. j≥0 The objective of the paper is to present a numerical im- −2j plementation of the 3D Discrete Shearlet Transform which Notice that each function Wj = W (2 ·), j ≥ 0, is takes advantage of the sparsity properties of the corresponding supported inside the Cartesian corona continuous representation. To illustrate the performance of − 2j−1 2j−1 3 n − 2j−4 2j−4 3 ⊂ Rb3 this new numerical algorithm, we consider a number of [ 2 ; 2 ] [ 2 ; 2 ] ; applications to problems of video denoising and enhancement. and the functions W 2; j ≥ 0, produce a smooth tiling of Rb3. As it will become apparent from the results presented below, j Next, let V 2 C1(R) be such that supp V ⊂ [−1; 1] and not only our video processing algorithm based on the 3D Discrete Shearlet Transform outperforms those based on the jV (u − 1)j2 + jV (u)j2 + jV (u + 1)j2 = 1 for juj ≤ 1: (4) corresponding 2D Discrete Shearlet Transform (when applied “slice by slice”), but it is also extremely competitive against In addition, we that V (0) = 1 and that V (n)(0) = 0 for all similar algorithms based on 3D curvelets and surfacelets.