A Shearlet Approach to Edge Analysis and Detection Sheng Yi, Demetrio Labate, Glenn R

IEEE TRANS. IMAGE PROCESSING 1 A Shearlet Approach to Edge Analysis and Detection Sheng Yi, Demetrio Labate, Glenn R. Easley, and Hamid Krim

Abstract—It is well known that the wavelet transform tracting and classifying basic edge features such as cor- provides a very effective framework for analysis of multi- ners and junctions. scale edges. In this paper, we propose a novel approach based on the shearlet transform: a multiscale directional Our shearlet approach has similarities with a num- transform with a greater ability to localize distributed ber of other methods in applied mathematics and en- discontinuities such as edges. Indeed, unlike traditional gineering to overcome the limitations of traditional wavelets, shearlets are theoretically optimal in representing images with edges and, in particular, have the ability wavelets. These methods include contourlets [6], [7] to fully capture directional and other geometrical fea- complex wavelets [8], ridgelets [9] and curvelets [10]. tures. Numerical examples demonstrate that the shear- In contrast to all these methods, the shearlet frame- let approach is highly effective at detecting both the loca- work provides a unique combination of mathematical tion and orientation of edges, and outperforms methods based on wavelets as well as other standard methods. rigidness and computational efficiency when addressing Furthermore, the shearlet approach is useful to design edges, optimal efficiency in dealing with edges, and com- simple and effective algorithms for the detection of cor- putational efficiency. In addition, its continuous formu- ners and junctions. lation is particularly well-suited for designing an imple- Index Terms—Curvelets, edge detection, feature extraction, shearlets, singularities, wavelets. mentation, presented in this work, for the purpose of edge analysis and detection.

I. Introduction A. Edge detection using wavelets. Edges are prominent features in images and their In the classic Canny edge detection algorithm [11], an analysis and detection are an essential goal in computer image u is smoothed by a convolution with a Gaussian vision and image processing. Indeed, identifying and filter: localizing edges are a low level task in a variety of ap- ua = u Ga, (I.1) plications such as 3D reconstruction, shape recognition, ∗ 1 1 image compression, enhancement and restoration. where Ga(x)= a− G(a− x), for a> 0, and G(x) is the In this paper, we apply a novel directional multiscale Gaussian function. Edges are then recognized as the mathematical framework which is especially adapted to local maxima of the magnitude of the gradient of ua. identification and analysis of distributed discontinuities The adjustable scaling factor a determines the amount such as edges occurring in natural images. Multiscale of smoothing: as a increases, the detector’s sensitivity methods based on wavelets, have been successfully ap- to noise decreases; however, as a increases, the local- plied to the analysis and detection of edges [1], [2], ization error in the detection of edges also increases. [3], [4], [5]. Despite their success, wavelets are how- As a result, the performance of the algorithm heavily ever known to have a limited capability in dealing with depends on the scaling factor a [12], [13]. directional information. By contrast, the shearlet ap- It was observed by Mallat et al. [1], [2] that, at a proach, which we propose here, is particularly designed single scale, the Canny edge detector is equivalent to to deal with directional and anisotropic features typi- the detection of the local maxima of the wavelet trans- cally present in natural images, and has the ability to form of u, for some particular choices of the analyzing wavelet. In fact, the function ψ = G is a wavelet accurately and efficiently capture the geometric infor- ∇ mation of edges. As a result, the shearlet framework known as the first derivative Gaussian wavelet. Thus, each image u L2(R2) satisfies: provides highly competitive algorithms, for detecting ∈ both the location and orientation of edges, and for ex-

u(x)= Wψu(a,y) ψa(x y) dy, D. Labate, S. Yi and H. Krim are with North Carolina State − University, Raleigh, NC 27695, USA ([email protected]; Z [email protected]; [email protected]) 1 1 where ψa(x) = a− ψ(a− x), and Wψu(a, x) is the S.Yi and H.Krim were in part funded by AFOSR, AF9550-07- 1-0104.DL acknowledges partial support from NSF DMS 0604561 wavelet transform of u, deﬁned by and DMS (Career) 0746778 G.R. Easley is with System Planning Corporation 1000 Wilson W u(a, x)= u(y) ψ (x y) dy = u ψ (x). Boulevard, Arlington, VA 22209, USA ([email protected]) ψ a − ∗ a Z 2 IEEE TRANS. IMAGE PROCESSING

The significance of this representation is that the This transform, introduced by the authors and their col- wavelet transform provides a space-scale decomposition laborators in [19], [20], is a genuinely multidimensional of the image u, where u L2(R2) is mapped into ∈ version of the traditional wavelet transform, and is es- the coefficients Wψu(a,y) which depend on the loca- pecially designed to address anisotropic and directional tion y R2 and the scaling variable a > 0. Another useful observation∈ is that the wavelet transform of u is information at various scales. Indeed, the traditional proportional to the gradient of the smoothed image ua: wavelet approach, which is based on isotropic dilations, has a very limited capability to account for the geome- ∇ua(x)= u ∗∇Ga(x) = u ∗ ψa(x) = Wψu(a, x). (I.2) try of multidimensional functions. In contrast, the analyzing functions associated to the shearlet transform This shows that the maxima of the magnitude of the are highly anisotropic, and, unlike traditional wavelets, gradient of the smoothed image ua correspond exactly are defined at various scales, locations and orientations. to the maxima of the magnitude of the wavelet trans- As a consequence, this transform provides an optimally form Wψu(a, x); this provides a natural mathematical efficient representation of images with edges [21]. framework for the multiscale analysis of edges [1], [2]. The shearlet transform has similarities to the curvelet In particular, utilizing multiscale wavelet representa- transform, first introduced by Candès and Donoho [10], tion avoids the problem of finding the appropriate scale [22]. Shearlets and curvelets, in fact, are the only two a which produces an improved detection of the clas- systems which were mathematically known to provide sic Canny algorithm. Furthermore, there are very effi- optimally sparse representations of images with edges. cient numerical implementations of the wavelet trans- The spatial-frequency tilings of the two representations form [14]. are completely different, yet the implementations of the The difficulty of edge detection is particularly promi- curvelet transform corresponds to essentially the same nent in the presence of noise, and when several edges are tiling as that of the shearlet transform. In spite of this close together or cross each other, e.g., 2–dimensional similarity, it is not clear how the curvelet transform im- projections of 3–dimensional objects [15]. In such cases, plementations could be modified to do the applications the following limitations of the wavelet approach (and described in this paper. Yet an application of low-level other traditional edge detectors) become evident: vision using curvelets has been suggested in [23]. Difficulty in distinguishing close edges. The • Both systems are related to contourlets [6], [7] and isotropic Gaussian filtering causes edges running steerable filters [24], [25]. Contourlets, however, provide close together to be blurred into a single curve. a purely discrete approach which presents difficulties in Poor angular accuracy. In the presence of sharp • rigorously addressing the edge detection problem. We changes in curvature or crossing curves, the refer to [19] for more details about the comparison of isotropic Gaussian filtering leads to an inaccurate shearlets and other orientable multiscale transforms. detection of edge orientation. This affects the de- In this paper, we combine the shearlet framework tection of corners and junctions. with several well established ideas from the image pro- To address these difficulties one has to account for the cessing literature to obtain improved and computation- anisotropic nature of edge lines and curves. For exam- ally efficient algorithms for edge analysis and detection. ple, in [16], [17], [18] it is proposed to replace the scal- Our approach may be viewed as a truly multidimen- able collection of isotropic Gaussian filters Ga(x1, x2), sional refinement of the approach of Mallat et al., where a > 0 in (I.1) with a family of steerable and scalable the isotropic wavelet transform Wψu(a, x) is replaced by anisotropic Gaussian filters an anisotropic directional multiscale transform. Specif-

1/2 1/2 1 1 ically, for an image u, the shearlet transform is a map- Ga1,a2,θ(x1, x2)= a1− a2− Rθ G(a1− x1,a2− x2), ping u ψu(a,s,x), where a1,a2 > 0 and Rθ is the rotation matrix by θ. → SH Unfortunately, the design and implementation of such depending on the scale a > 0, the orientation s and filters is computationally demanding. In addition, the the location x. This provides a directional scale-space justification for this approach is essentially intuitive, decomposition of u and, as shown below, a theoretical and there is no proper theoretical model to indicate justifiable framework for the identification and analy- how to “optimize” such family of filters to best capture sis of the edges of u. The shearlet transform can be edges. expressed as

B. The Shearlet Approach. u(a,s,x)= u(y) ψ (x y) dy = u ψ (x). SHψ as − ∗ as The approach proposed in this paper is based on a Z new multiscale transform called the shearlet transform. where the analyzing elements ψas are well localized A SHEARLET APPROACH TO EDGE ANALYSIS AND DETECTION 3 waveforms at various scales and orientations. As a re- It is known that the continuous wavelet transform has sult, the shearlet transform acts as a multiscale direc- the ability to identify the singularities of a signal. In tional difference operator and provides a number of very fact, if a function u is smooth apart from a discontinu- useful features: ity at a point x0, then the continuous wavelet transform Improved accuracy in the detection of edge orienta- W u(a,t) decays rapidly as a 0, unless t is near x • ψ 0 tion. Using anisotropic dilations and multiple ori- [26]. This property is useful to→ identify the set of points entations, the shearlet transform precisely captures where u is not regular, and explains the ability of the the geometry of edges. continuous wavelet transform to detect edges. However, Well organized multiscale structure. It is a multi- the isotropic continuous wavelet transform is unable to • scale transform, based on the same affine mathe- provide additional information about the geometry of matical structure of traditional wavelets. the set of singularities of u. In many situations, includ- Computational efficiency. The discretization of the ing edge detection, it is useful to not only identify the • shearlet transform provides a stable and computa- location of edges, but also to capture their geometrical tionally efficient decomposition and reconstruction properties, such as the edge orientation. As shown by algorithm for images. the authors and their collaborators in [27], [28], this can The paper is organized as follows. Section II re- be achieved by employing a non-isotropic version of the calls the basic definitions and properties of the shearlet continuous wavelet transform (II.4) called the continu- transform. Section III presents a new numerical imple- ous shearlet transform. This is defined as the mapping mentation of the shearlet transform consistent with the u(a,s,t)= u, ψ , (II.5) theoretical requirements for the analysis of edges. Sec- SHψ h asti 1 tion IV describes applications of the shearlet approach 2 1 where ψast(x)= det Mas − ψ(Mas− (x t)), and Mas = to the estimation of edge orientation, feature classifica- a √as | | − − for a > 0,s R,t R2. Observe that tion and edge detection. 0 √a ∈ ∈ a 0 1 s II. Shearlet Transform Mas = Bs Aa, where Aa = and Bs = − . 0 √a 0 1 Let G be a subgroup of the group of 2 2 invertible Hence to each matrix Mas are associated two distinct × 2 2 matrices. The affine systems generated by ψ L (R ) actions: an anisotropic dilation produced by the ma- are the collections of functions: ∈ trix Aa and a shearing produced by the non-expansive − 1 − 2 1 2 ψM,t(x) = | det M| ψ(M (x − t)), t ∈ R ,M ∈ G. (II.3) matrix Bs.

If any u L2(R2) can be recovered via the reproducing formula ∈

ξ2 ξ2 u = u, ψM,t ψM,t dλ(M) dt, 1 (a, s) = ( 1 , 0) Rn h i (a, s) = ( , 0) 4 Z ZG 4 HYH @ where λ is a measure on G, then ψ is a continuous @ (a,s) = ( 1 , 1) @R 32 @R wavelet [14]. In this case, the continuous wavelet trans- @ form is the mapping ξ1 @ ξ1 6 @R u Wψu(M,t)= u, ψM,t , (II.4) 1 @I@ → h i (a,s) = ( 32 , 0) (a,s) = ( 1 , 0) for (M,t) G R2. There are a variety of examples 32 of wavelet∈ transforms.× The simplest case is when the matrices M have the form a I, where a > 0 and I is Fig. 1. Frequency support of the horizontal shearlets (left) and the identity matrix. In this situation, one obtains the vertical shearlets (right) for diﬀerent values of a and s. isotropic continuous wavelet transform: The generating function ψ is a well localized function 1 1 Wψu(a,t)= a− u(x) ψ(a− (x t)) dx, satisfying appropriate admissibility conditions [27], [28], R − so that each u L2(R2) has the representation Z ∈ where the dilation factor is the same for all coordi- ∞ ∞ da nate directions. This is the “standard” wavelet trans- u = u, ψast ψast 3 ds dt. R2 0 h i a form used in most wavelet applications (including the Z Z−∞ Z 2 wavelet-based edge detection by Mallat et al. described In particular, for ξ = (ξ1, ξ2) R , ξ1 = 0, we set ˆ ˆ ˆ ξ2 ˆ ∈ ˆ 6 in Section I). ψ(ξ) = ψ1(ξ1) ψ2( ξ1 ), where ψ1, ψ2 are smooth func- 4 IEEE TRANS. IMAGE PROCESSING

1 1 tions with supp ψˆ1 [ 2, ] [ , 2] and supp ψˆ2 ⊂ − − 2 ∪ 2 ⊂ @I 4 4 [ 1, 1]. In addition, to obtain the edge detection results 3 3 − ˆ ˆ 2 2 presented in the next section, ψ1 (respectively, ψ2) is as- 1 1 sumed to be odd (respectively, even). In the frequency 0 0

−1 −1

domain: −2 −2 0 1 0 1 10 10 a10 10 a 3 1 4 2πiξt 2 ξ2 (a) (b) (c) ψâst(ξ1, ξ2)= a e− ψˆ1(a ξ1) ψˆ2(a− ( s)), ξ1 − Fig. 2. Analysis of the edge response. The magnitudes of the ˆ shearlet (respectively, wavelet) transform of an edge point on the and, thus, each function ψast is supported in the set star-shaped figure (a) are plotted on a logarithmic scale as a black (respectively, gray) line. Figure (b) shows the response of the 2 1 1 2 ξ2 √ (ξ1, ξ2) : ξ1 [ a , 2a ] [ 2a , a ], ξ1 s a . shearlet transform when its orientation variable is tangent to the { ∈ − − ∪ | − |≤ } edge. Figure (c) shows the response when the orientation variable Thus each shearlet ψast has frequency support on a pair is normal to the edge. of trapezoids, at various scales, symmetric with respect to the origin and oriented along a line of slope s. Asa result, the shearlets form a collection of well-localized totic decay rate of the continuous shearlet transform u(a,s,t), for a 0 (fine scales), can be used to ob- waveforms at various scales a, orientations s and loca- SHψ → tions t. tain both location and orientation of edges in an image Notice that the shearing variable s corresponds to u. This is a significant refinement with respect to traditional wavelets, which only detect the location. We the slope of the line of orientation of the shearlet ψâst, 1 now present a brief summary of the most relevant re- rather than its angle with respect to the ξ1 axis . It follows that the shearlets provide a nonuniform angular sults which are useful for our analysis. We refer to [27], [28] for more details. covering of the frequency plane when the variable s is 2 discretized, and this can be a disadvantage for the nu- To model an image, let Ω = [0, 1] and consider the partition Ω = L Ω Γ, where: merical implementation of the shearlet transform. To n=1 n ∪ 1. each “object” Ωn, for n =1,...,L, is a connected avoid this problem, the continuous shearlet transform S is modified as follows. In the definition of , given open set (domain); SHψ L by (II.5), the values of s will be restricted to the in- 2. the set of edges of Ω be given by Γ = n=1 ∂ΩΩn, terval [ 1, 1]. Under this restriction, in the frequency where each boundary ∂ΩΩn is a smooth curve of − S plane, the collection of shearlets ψast will only cover finite length. ξ2 Consider the space of images I(Ω) defined as the collec- the horizontal cone (ξ1, ξ2) : ξ1 1 . To compen- sate for this, we add{ a similarly| defined| ≤ } second shear- tion of functions which have the form let transform, whose analyzing elements are the “ver- L tical” shearlets ψ(1) . In the frequency plane they are u(x)= u (x) χ (x) for x Ω Γ ast n Ωn ∈ \ obtained from the corresponding “horizontal” shearlets n=1 X ψ through a rotation by π/2. The frequency supports ast where, for each n =1,...,L, u C1(Ω) has bounded of some representative horizontal and vertical shearlets n 0 partial derivatives, and the sets Ω∈ are pairwise disjoint are illustrated in Figure 1. By combining the two shear- n in measure. For u I(Ω), we have the following results. let transforms, any u L2(R2) can be reproduced with ∈ ∈ Theorem II.1: If t / Γ, then respect to the combination of vertical and horizontal ∈ shearlets. We refer to [27], [28] for additional details 3 lim a− 4 ψu(a,s,t)=0. (II.6) about this construction. In the following, when it will a 0 → SH be needed to distinguish the two transforms we will use If t Γ and, in a neighborhood of t = (t ,t ), the notation (0) (respectively, (1)) for the continu- 1 2 ψ ψ the boundary∈ curve is parametrized as (E(t ),t ), and ous shearlet transformSH associatedSH to the horizontal cone 2 2 s = E′(t ), then also (II.6) holds. Otherwise, if (respectively, the vertical cone). 6 − 2 s = E′(t ), there is a constant C > 0 such that − 2 A. Edge resolution using the shearlet transform 3 lim a− 4 ψu(a,s,t)= C [u]t , The continuous shearlet transform is able to precisely a 0 → SH | | capture the geometry of edges. Indeed, the asymp- where [u]t is the jump of u at t, occurring in the normal 1 The curvelets are indexed by scale, angle and location, where direction to the edge. the angle is the angle of orientation in polar coordinates. The This shows that asymptotic decay of the continuous drawback of this representation, however, is that curvelets do not form an affine system, as in (II.3), and are not obtained from a shearlet transform of u is the slowest for t on the bound- single generating function. ary of Ω, and s corresponding to the normal orientation A SHEARLET APPROACH TO EDGE ANALYSIS AND DETECTION 5 to Ω at t (see Figure 2). Other useful results from [27], amenable to a continuous (non-dyadic) scaling. For its [28] are the following: development, consistent with the theoretical analysis in If u I(Ω) and t is away from the edges, then [28], special properties on the shearlet generating func- • ∈ ψu(a,s,t) decays rapidly as a 0, and the de- tion ψ (see Sec. II) are utilized. SHcay rate depends on the local regularity→ of u. In We start by reformulating the operations associated 2 particular, if u is Lipschitz–α near t0 R , then with the shearlet transform in a way which is suitable ∈ 2 the following estimates hold: for α 0, to its numerical implementation. For ξ = (ξ1, ξ2) R , ≥ a< 1, and s 1, let ∈ 1 3 2 (α+ 2 ) | |≤ ψu(a,s,t0) Ca , as a 0; b |SH |≤ → 1 1 (0) 4 ˆ 2 ξ2 wa,s(ξ) = a− ψ2(a− ( s)) χ 0 (ξ) while for α< 0, ξ1 − D 1 1 (1) ξ1 − 4 ˆ 2 3 wa,s(ξ) = a ψ2(a− ( ξ2 s)) χ 1 (ξ), (α+ 4 ) b − D ψu(a,s,t0) Ca , as a 0. |SH |≤ → 2 ξ2 where b0 = (ξ1, ξ2) R : 1 , 1 = (ξ1, ξ2) We refer to [14] for the definition of Lipschitz reg- D { ∈ | ξ1 |≤ } D { ∈ R2 : ξ1 1 . For a< 1, s 1, t R2, d =0, 1, the ularity. Classification of points by their Lipschitz | ξ2 |≤ } b | |≤ ∈ regularity is important as it can be used to distin- Fourier transform of the shearlets can be expressed as guish true edge points from points corresponding b ˆ(d) (d) (d) 2πiξt to noise [1], [2]. ψa,s,t(ξ)= a V (a ξ) wa,s(ξ) e− , If u contains a corner point, then locally, as a 0, • 3/4 → (0) (1) ψu(a,s,t) decays as a when s is the direction where V (ξ1, ξ2)= ψˆ1(ξ1), V b (ξ1, ξ2)= ψˆ1(ξ2). The normalSH to the edges. It decays as a5/4 otherwise. continuous shearlet transform of u L2(R2) is: Spike-type singularities produce a different behav- ∈ • (d) (d)(a ξ) (d) 2πiξt ior for the decay of the shearlet transform. Con- SH u(a, s, t)= a uˆ(ξ) V wa,s(ξ) e dξ, (III.7) ψ Z sider, for example, the Dirac delta centered at t0. b Then where d =0, 1 correspond to the horizontal and vertical

3/4 transforms, respectively. Hence, from (III.7) we have δ 0 (a,s,t ) a− , as a 0, |SHψ t 0 |≍ → that (d)u(a,s,t)= v(d)u w(d)(t) (III.8) so that the transform actually grows at fine scales. SHψ a ∗ a,s The decay is rapid for t = t . where 6 0 These observations show that the geometric informa- (d) (d) 2πiξt tion about the edges of u can be completely resolved by va u(t)= a uˆ(ξ) V (aξ) e dξ. R2 looking at the values of its continuous shearlet trans- Z form ψu(a,s,t) at fine scales. Additionally, similar To obtain a transform with the ability to detect edges to theSH wavelet case, the behavior of the decay rate across effectively, we choose the functions ψˆ1 to be odd and ψˆ2 scales provides useful information about the regularity to be even. of u. In the following sections, we will take advantage We are now ready to derive an algorithmic procedure of these properties to design improved numerical algo- to compute a discrete version of (III.8). For N N, rithms for the analysis and detection of edges. an N N image can be considered as a finite sequence∈ × III. A Discrete Shearlet Transform for Edge u[n1,n2] : n1,n2 = 0,...,N 1 . Thus, identifying { Z2 − 2 }Z2 Detection the domain with N , we view ℓ ( N ) as the discrete analog of L2(R2). Consistently with this notation, the A numerically efficient implementation of the shear- inner product of the images u1 and u2 is defined as let transform was previously introduced in [19]. It was N 1 N 1 based on the use of a Laplacian pyramid combined with − − appropriate shearing filters. That particular implemen- u1,u2 = u1[n1,n2] u2[n1,n2], h i n1=0 n2=0 tation, however, was specially designed for image de- X X noising. Since its direct application to edge detection and, for N/2 k , k < N/2, its 2D Discrete Fourier suffers from large sidelobes around prominent edges 1 2 Transform− (DFT)≤ u ˆ[k , k ] is given by: (which is the same problem with the curvelet implemen- 1 2 tations), a different implementation will be presented N 1 − n1 n1 1 2πi( k1+ k2) here. This new implementation is based on separately uˆ[k , k ]= u[n ,n ] e− N N . 1 2 N 1 2 calculating the vertical and horizontal shearlets and is n1,n2=0 X 6 IEEE TRANS. IMAGE PROCESSING

We adopt the convention that brackets [ , ] denote arrays of indices, and parentheses (·, ·) denote function evaluations. We shall interpret· · the numbersu ˆ[k1, k2] as samplesu ˆ[k1, k2] =u ˆ(k1, k2) from the trigonometric polynomialu ˆ(ξ1, ξ2) = n1 n1 N 1 2πi( ξ1+ ξ2) − N N n1,n2=0 u[n1,n2] e− . To implement the directional localization associated (a) (b) withP the shearlet decomposition described by the win- (d) dow functions wa,s, we will compute the DFT on a grid Fig. 4. Examples of spline based shearlets. Figure (a) corre- consisting of lines across the origin at various slopes sponds to ℓ = 5 using a support window of size 16. Figure (b) corresponds to ℓ = 2 using a support window of size 8. called the pseudo-polar grid, and then apply a one- dimensional band-pass filter to the components of the signal with respect to this grid. To do this, let us define To discretize the window function, consider a function 2 1 2j 1 the pseudo-polar coordinates (ζ1, ζ2) R by: (d) (d) j ∈ w such that d=0 ℓ=− 2j wˆ [2 k2 ℓ]=1. That ξ2 − − j (ζ , ζ ) = (ξ , ) if (ξ , ξ ) ; is, the shearing variable s is discretized as sj,ℓ = 2− ℓ. 1 2 1 ξ1 1 2 ∈ D0 P P Lettingg ϕP be the mapping functiong from the Carte- (ζ , ζ ) = (ξ , ξ1 ) if (ξ , ξ ) . 1 2 2 ξ2 1 2 ∈ D1 sian grid to the pseudo-polar grid, the discrete shear- Using this change of coordinates, we obtain let transform can thus be expressed in the discrete frequency domain as (d) (d) vâ f(ζ1, ζ2)=v â f(ξ1, ξ2), 1 ](d) 1 j (d) 1/2 (d) ˆ (d) wˆ (a− (ζ s)) =w ˆ (ξ , ξ ). ϕP− (vˆj u[k1, k2]) ϕP− δP [k1, k2] wˆ [2 k2 ℓ] , g 2 − a,s 1 2 − This expression shows that the different directional g where δˆ is the discrete Fourier transformg of the Dirac components are obtained by simply translating the win- P delta function δP in the pseudo-polar grid. Thus, the (d) dow function wˆ . For an illustration of the mapping discrete shearlet transform can be expressed as the dis- from the Cartesian grid to the pseudo-polar grid, see crete convolution Figure 3. g (d)u[j, ℓ, k]= v(d)u w(d)[k], SH j ∗ j,ℓ

ξ2 (d) 1 wherew ˆ [k , k ]= ϕ− δˆ [k , k ] wˆ(d)[2j k ℓ] . ζ2 j,ℓ 1 2 P P 1 2 2 − ϕ The discrete shearlet transform will be computed as P g ζ1 ξ1 follows. Let Hj and Gj be the low-pass and high-pass - filters of a wavelet transform with 2j 1 zeros inserted 1 between consecutive coefficients of the− filters H and G, ϕP− respectively. Given 1-dimensional filters H and G, define u (H, G) to be the separable convolution of the rows and∗ the columns of u with H and G, respectively. Fig. 3. The mapping ϕP from a Cartesian grid to a Notice that G is the wavelet filter corresponding to ψ1 pseudo-polar grid. The shaded regions illustrate the mapping ˆ − where ψ1 must be an odd function and H is the fil- ϕ 1(δˆ [k ,k ]w ˜(d)[a−1/2k − ℓ]), for fixed a, ℓ. P P 1 2 2 ter corresponding to the coarse scale. Finally as indi- (d) 2j cated above, the filtersw ˆ are related to the function At the scale a = 2− , j 0, we will denote ˆ (d) ≥ ψ2, which must be an even function and can be imple- by vj u[n1,n2] the discrete samples of a function mented using a Meyer-type filter. Hence, we have: (d) (d) v −2 u(x , x ), whose Fourier transform is v u(ξ , ξ ). 2 j 1 2 a 1 2 Discrete Shearlet Cascade Algorithm. (d) ](d) 2 Z2 Also, the discrete samples vˆ u[k , k ]= vˆ −2 u(k , k ) Let u ℓ ( ). Define j 1 2 2 j 1 2 ∈ N (d) b are the values of the DFT of va u[n ,n ] on the pseudo- g 1 2 S0u = u polar grid. One may obtain the discrete Frequency val- (d) Sj u = Sj 1u (Hj , Hj ), j 1. ues of va u on the pseudo-polar grid by direct extrac- − ∗ ≥ tion using the Fast Fourier Transform (FFT) with com- For d =0, 1, the discrete shearlet transform is given by plexity O(N 2 log N) or by using the Pseudo-polar DFT (PDFT) with the same complexity. (d)u[j, ℓ, k]= v(d)u w(d)[k], SH j ∗ j,ℓ A SHEARLET APPROACH TO EDGE ANALYSIS AND DETECTION 7 where j 0, 2j ℓ 2j 1, k Z2 and v(0)u = spect to traditional edge detectors. For example, Fig- ≥ − ≤ ≤ − ∈ j S u (G ,δ), v(1)u = S u (δ, G ). ure 2 compares the behavior of the discrete shearlet j ∗ j j j ∗ j For simplicity of notation, it will be convenient to transform of an image at the edge points with that combine the vertical and horizontal transforms (d = of the traditional wavelet transform. The figure dis- 0, 1) by re-labeling the orientation index ℓ as follows: plays the magnitudes of the discrete shearlet transform u[j, ℓ, k] at the edge points of a star-like image u, as SH(0)u[j,ℓ − 1 − 2j ,k], 1 ≤ ℓ ≤ 2j+1; SH SHu[j,ℓ,k]= a function of the scaling index j, for two different values  SH(1)u[j, 3(2j ) − ℓ,k], 2j+1 <ℓ ≤ 2j+2. of the directional variable ℓ. The test shows that the 4 Using the new notation, at the scale a = 2− (j = 2), values u[j, ℓ, k] are much larger when ℓ corresponds the index ℓ of the discrete shearlet transform u[2,ℓ,k] to the normal|SH direction| to the edge, in accordance with SH ranges over ℓ = 1,..., 16. Here the first (respectively, the theoretical predictions from Section II. Since the last) eight indices correspond to the orientations asso- wavelet transform has no variable associated with the ciated with the horizontal (resp. vertical) transform orientation, its magnitudes do not depend on the orien- (0) (resp. (1)). tation. SHIn our implementation,SH we use the finite impulse response filters H and G that correspond to a quadratic A. Shearlet-based Orientation Estimation spline wavelet. A reflexive boundary condition on the The discrete shearlet transform can be applied to pro- borders of the image is assumed for the convolution op- vide a very accurate description of the edge orientations. eration. For an N N image, the numerical complexity We will show that this approach has several advantages × of the shearlet transform is O(N 2 log(N)). In some ex- with respect to traditional edge orientation extraction periments, non-dyadic scaling will be used, i.e. no zeros methods. will be inserted in the filters Hj and Gj .

Figure 4 displays examples of shearlets associated 65 65 θ 85 θ 85 with the discrete shearlet transform. Figure 5 shows the 105 105 125 125 shearlet coeﬃcients u[j, ℓ, k], where u is the charac- 145 145 SH 165 165 teristic function of a disk, at multiple scales, for several 185 185 205 205

225 225 values of the orientation index ℓ. 2 4 6 8 10 12 14 16 2 4 6 8 10 12 14 16 ℓ ℓ (j = 1) (j = 0) Fig. 6. The directional response of the shearlet transform DR(θ,ℓ,j) is plotted as a greyscale value, for j = 1, 0. The orientations θ of the half-planes range over the interval [45, 225] (degrees), ℓ ranges over 1,..., 16 .

Recall that, in the continuous model where the edges of u are identified as local maxima of its gradient, the edge orientation is associated with the orientation of the gradient of u, an idea used, for instance, in the Canny edge detector. Similarly, using the continuous wavelet transform Wψu(a,t), the orientation of the edges of an image u can be obtained by looking at the horizontal Fig. 5. A representation of shearlet coefficients of the character- istic function of a disk (shown above), at multiple scales (ordered and vertical components of Wψu(a,t). In fact, letting x ∂Ga y ∂Ga by rows), for several values of the orientation index ℓ (ordered by ψa = Ga and ψa = ∂x ,ψa = ∂y , the edge orienta- columns). tion of∇u at τ is given u ψy(τ) The directionally oriented shearing filters have the arctan ∗ a . (IV.9) (d) u ψx(τ) special feature that w f = f. As a consequence, a ℓ,d j,ℓ ∗ ∗ a unified map of directionally-captured edge elements According to (I.2), the expression (IV.9) measures the P can be assembled through a simple summation. Other direction of the gradient ua at τ. directionally oriented filters such as steerable Gaussian Unfortunately, the gradient∇ model is not very accu- filters do not typically share this property [29, Ch.2]. rate in the presence of noise or when the edge curve is not regular. Furthermore, in the usual numerical im- IV. Analysis and Detection of Edges ∂ plementations, the operator ∂x is approximated by a As explained above, a main feature of the shearlet finite difference and this becomes a significant source of transform is its superior directional selectivity with re- inaccuracies in edge orientation estimation. 8 IEEE TRANS. IMAGE PROCESSING

18 18 18 16 No noise 16 SNR=16.94dB 16 SNR=4.91dB 14 14 14

12 12 12

10 10 10

error 8 error 8 error 8

6 6 6

4 4 4

2 2 2

0 0 0 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 a a a

Fig. 7. Comparison of the average error in the estimation of edge orientation (expression (IV.11)), for the disk image shown on the left, using the wavelet method (dashed line) versus the shearlet method (solid line), as a function of the scale a, for various SNRs (additive white Gaussian noise).

2j The advantage of the orientation estimation based Due to its directional sensitivity, at a fixed scale 2− , on the shearlet transform is that, by decomposing an most of the energy of the discrete shearlet transform image with respect to scale, location and orientation will be concentrated at a few values of the orientation variables, the orientation information at each scale is index ℓ. This is illustrated in Figure 6 for two different directly available. In addition, we can identify not only values of j (in both cases, 16 values of the directional the ‘dominant’ orientation at a certain point, but a index ℓ have been considered). vector of all orientation ‘components’ present at that The above experiment confirms the theoretical prop- point. This provides a general orientation representa- erties described in Section II and shows that the discrete tion which will be further exploited in Section IV.B to shearlet transform is very effective in indicating the ori- analyze corner points and junctions. entation of the edges of a digital image. Thus, following Other useful ideas to estimate edge orientation have Theorem II.1, we propose to estimate the edge orienta- 2j appeared in the literature. For example, the method tion of an image u, at fine scales (a =2− “small”), by used by Perona in [30] to estimate edge orientations, measuring the index ℓ˜ which maximizes the magnitude which is based on anisotropic diffusion, is very robust of the u[j, ℓ, k]: SH in the presence of noise. In contrast to the shearlet ℓ˜(j, k) = argmax u[j, ℓ, k] . (IV.10) approach, however, this method only yields a single ori- ℓ |SH | entation estimation and is unable to handle the situ- Once this is found, we can compute θS(j, k), the angle ation where multiple orientations are present, such in of orientation associated with the index ℓ˜(j, k). the presence of corner points or junctions. Other re- Notice that, since we are working with a discrete searchers view orientation estimation as a local fitting transform, this procedure only yields a discrete set of problem [31], [32] which, under least square criterion, possible orientations since ℓ ranges over a finite set. For is equivalent to PCA analysis. This method is numer- example, ℓ ranges over 1,..., 16 for j = 2. To improve ically stable and efficient but loses the scaling nature the accuracy of the orientation estimation and reduce of orientation and, again, only estimates a single orien- the quantization error, we will interpolate the values of tation (which could be an average of many orientation the angle orientation as follows. We will assume that, components at the same point). at an edge point t, the continuous shearlet transform As a first experiment, we demonstrate the directional ψu(a,s,t) is essentially a parabolic function with sensitivity of the discrete shearlet transform by measur- respectSH of the s variable, and its maximum occurs for ing its ability to detect edge orientations for a class of smax corresponding to the edge normal orientation. test images. Namely, consider the collection of images Hence, for j and k fixed, we will estimate the angle uθ, representing half-planes at various orientations: θmax associated at smax using the following algorithm. y uθ(x, y)= χDθ (x, y), Dθ = (x, y) : tan(θ) . { x ≤ } Shearlet Orientation Detection Algorithm. ˜ Let E be the set of edge points of u and E be the Compute ℓ1 = ℓ using (IV.10); let ℓ0 = ℓ1 1, θ | | • − number of elements in the set E. We define the direc- ℓ2 = ℓ1 +1 (here the sums are meant to be modulus tional response of shearlet transform at the edge points N, where N is the size of the set of indices ℓ). For i = 0, 1, 2, let θ be the angles of orientation as the function: • i associated with the indices ℓi. 1 2 DR(θ,ℓ,j)= u [j, ℓ, k] Let S(θ)= c1θ + c2θ + c3, where S(θi) is identified E · |SH θ | • k E with u[j, ℓi, k], i = 0, 1, 2. Then c1,c2,c3 is the | | X∈ SH A SHEARLET APPROACH TO EDGE ANALYSIS AND DETECTION 9

2 solution of linear system S(θi)= c1θi +c2θi+c3. So smaller in amplitude than those for the points A and B). θ = c2 (this is the value where the parabolic The same behavior holds, essentially, for more general max − 2c1 function S(θ) achieves its maximum). images, even in the presence of additive white Gaus- sian noise (provided, of course, that the SNR is not too Notice that, by construction, the function S(θ) is al- small). In particular, at corner points, the behavior is ways concave down, and its maximum is close to the ˜ similar to that of point A, with the plot sA(ℓ) showing orientation associated with ℓ(j, k). two main peaks corresponding to the orientations of the Extensive numerical experiments demonstrate the two segments converging to the corner point. ability of this approach to provide a very accurate mea- This suggests to devise a strategy for classifying sure of the edge orientation. An example of application smooth regions, edges, corners and junctions based on of this algorithm is illustrated in Figure 7, using as a the plot patterns of Figure 8(b). We will proceed in test image a smooth function with an edge along the two steps. Let j = j0 be a fixed (fine) scale. First, boundary of a disk. The figure displays the average an- since the magnitude of the discrete shearlet transform gular error in the estimate of the edges orientation, as a 2j coefficients at the points along the boundaries (includ- function of the scale a =2− , where the average angle ing smooth edges, corners and junctions) is significantly error is defined by larger than at the other points, they can be easily sep- 1 arated by looking at the energy function at k: θ˜(t) θ(t) , (IV.11) E · | − | | | t E E(k)= u[j ,ℓ,k] 2. X∈ |SH 0 | ℓ E is the set of edge points, θ is the exact angle and X θ˜ the estimated angle. The average angle error us- We will refer to these points with large amplitude values ing shearlets is compared to that obtained using the as boundary points. wavelet approach, where the edge orientation is esti- Next, once the boundary points have been separated, mated using (IV.9). Recall that this approach, for a we will examine the “form” of the function sk(ℓ) to dis- single scale, is equivalent to the estimation obtained tinguish edge points from corners and junction points. from Canny’s algorithm. Tests reported in Figure 7 That is, a point k is recognized as a regular edge point show that the shearlet approach significantly outper- if the corresponding function sk(ℓ) has a single peak. If forms the wavelet method, especially at finer scales, and more than one peak is present, k will be a corner or a is extremely robust in the presence of noise. Note the junction point. experiments shown in Figure 7 are a base line compar- Thus, for each boundary point k, let P = p (ℓ) : k { k ison between the wavelet and shearlet multiscale repre- ℓ = 1,...N where pk(ℓ) is the normalized peak value sentations. Additional filtering or additional feature ex- in the ℓ orientation} and is given by tractions could be used to improve their noise response. u[j0,ℓ,k] If a pixel k correspond to a junction or a corner point, |SH | , ℓ Lk; P ∈ u[j0,ℓ,k] then the orientation index ℓ˜(j, k), given by (IV.10), will pk(ℓ)= ℓ Lk |SH | ∈ ( 0, ℓ/ Lk. identify the orientation of the ‘strongest’ edge. A more ∈ precise analysis of this situation is discussed in the next and Lk is the set of orientations where local maxima of section. the discrete shearlet transform occur, that is,

B. Feature classification L = ℓ : u[j ,ℓ,k] > u[j ,ℓ +1, k] k { |SH 0 | |SH 0 | Consider a simple image u containing edges and and u[j0,ℓ,k] > u[j0,ℓ 1, k] . smooth regions, like the one illustrated in Figure 8(a), |SH | |SH − |} and examine the behavior of the discrete shearlet trans- We present the following algorithm which uses the K- form u[j, l, k], at a fixed scale j , for some typi- mean clustering algorithm [33, Ch.10]: SH 0 cal locations k0. As Figure 8 shows, we can recog- Feature Classification Algorithm nize four classes of points from the plot of s (ℓ) = Using the Energy E(k) as the feature function, we k0 • u[j ,ℓ,k ] . Namely, at the junction point k = A, apply the K-Mean clustering algorithm, with Eu- |SH 0 0 | 0 sk0 (ℓ) exhibit three peaks corresponding to the orienta- clidean metric, to cluster the image points into tions of the three edge segments converging into A; at three sets I1, I2, I3. Namely, let Ω denote the set the point k0 = B on a smooth edge, sk0 (ℓ) has a single of points Iimax , where peak; at a point k = D in a smooth region, sk0 (ℓ) is essentially flat; finally, at a point k = C “close” to an 1 0 imax = arg max E(k). i Ii edge, sk0 (ℓ) exhibit two peaks (however they are much k Ii | | X∈ 10 IEEE TRANS. IMAGE PROCESSING

coefficient pattern of A coefficient pattern of B 8 8

6 6

4 4 D A B C 2 2 coefficient amplitude coefficient amplitude

0 0 0 5 10 15 20 0 5 10 15 20 l l coefficient pattern of C coefficient pattern of D 1 1

0.5 0.5

0 0 coefficient amplitude coefficient amplitude

−0.5 −0.5 0 5 10 15 20 0 5 10 15 20 l l (a) (b) Fig. 8. (a) Representative points on the test image. (b) Shearlet Transform pattern s(ℓ), as a function of ℓ for the points indicated in (a).

Ω is identiﬁed as the set of boundary points. Let R [29]. Furthermore, even in this case, the wavelet ap-

be the set Iimin , where proach is unable to provide speciﬁc information about the geometry of the corners or junctions because it is 1 based on isotropic ﬁlters (e.g., the 2D Mexican hat [29]). imin = arg min E(k). i Ii k Ii | | X∈ C. Edge Detection Strategies

R is the set of regular points. The remaining set Ii The shearlet transform provides a multiscale and contains the near edge points. multi-directional framework which can be used to devise For a point k in I , sort the entries of the vector several effective edge detection schemes. In the context • imax Pk as of edge detection, it was shown in several studies that one can take advantage of the multiscale decomposition [˜pk(1),..., p˜k(N)], of images to improve the robustness of an edge detector where p˜k(1) = max pk(ℓ) Pk , p˜k(2) = in the presence of noise. In what follows, we will adapt max p (ℓ) P p˜ (1){ , and∈ so on.} Then using these ideas to propose several multiscale edge detection { k ∈ k \{ k }} again the K-Mean clustering algorithm on Iimax , schemes. the set is further classified in two groups corre- As in the wavelet approach, we will select the edge sponding to smooth edge points in one set, corner point candidates of an image u by first identifying and junction points in the other set. the shearlet transform modulus maxima, that is, those It is clear that the algorithm described above can be fur- points (n1, n2) which, at fine scales j, are local maxima ther refined to distinguish corner points from junction of the function points, and different classes of junction points having M u[n ,n ]2 = ( u[j,ℓ,n ,n ])2. different geometric features. j 1 2 SH 1 2 ℓ Figure 9 illustrates the application of the Feature X Classification Algorithm to three test images, at scale According to the results from Section II, we expect that, j = 0, using N = 16 orientations. The results show the if (n1, n2) is an edge point, the shearlet transform of u separation of the image points into four classes: corners will behave as and junctions (Figures b1-b3); smooth edges (Figures βj u[j, ℓ, n , n ] C 2− , c1-c3); near-edge points (Figures d-d3); regular points |SH 1 2 |∼ in smooth regions (Figures e1-e3). where β 0. This shows that the shearlet approach provides a very If, however,≥ β < 0 (in which case the size of u in- simple and computationally efficient framework for clas- |SH | creases at finer scales), then (n1, n2) will be recognized sifying several features of interest in an image. This has as a delta-type singularity and the point will be classi- some advantages with respect to wavelet-based methods fied as noise. Notice that a similar behavior holds for and other existing methods which do not have a specific the wavelet transform and it is similarly used for edge ability to capture the directional features of an image. detection. The sign of β can be estimated by computing For example, using wavelets, in order to identify corner the best linear fit to the data points, one has to design a set of filters which is very different from the one used to detect the smooth edges log u[j, ℓ, n , n ] J , { |SH 1 2 |}j=1 A SHEARLET APPROACH TO EDGE ANALYSIS AND DETECTION 11

(a1) (b1) (c1) (d1) (e1)

(a2) (b2) (c2) (d2) (e2)

(a3) (b3) (c3) (d3) (e3) Fig. 9. (a1-a3) Test images. ((b1-b3) Identification of corners and junctions. ((c1-c3) Identification of smooth edge points. ((d1-d3) Identification of points near the edges. ((e1-e3) Identification of regular points (smooth regions). for the various values of ℓ, where J is the last decom- this reason we suggest the following modification, which position level. follows essentially the same ideas, and uses the cascade Using this procedure, edge point candidates for each algorithm of the discrete shearlet transform to “rein- of the oriented components are found by identifying the force” the true edges and suppress the false ones. points for which β 0. These components can then be Shearlet Edge Detection algorithm. Given a func- ≥ 2 Z2 (d) stitched together by simply adding them up, or using tion u ℓ ( N ), let u[j, ℓ, k] be the discrete shear- additional constraints. Next, a non-maximal suppres- let transform,∈ givenSH by sion routine is applied to these points to trace along (d)u[j, ℓ, k]= v(d)u w(d)[k], the edge in the edge direction and suppress any pixel SH j ∗ j,ℓ value that is not considered to be an edge. The non- for j 0, 2j ℓ 2j 1, k Z2, d = 0, 1, where maximal suppression routine is a standard routine in (0) ≥ − ≤ ≤ (1) − ∈ vj u = Sj u (Gj ,δ), vj u = Sj u (δ, Gj ), and Sj u = the Canny edge detector [11]. Using this routine, at ∗ ∗ Sj 1u (Hj , Hj ). Set each edge point candidate, the magnitude of the shear- − ∗ (d) (d) let transform is compared with the values of its neigh- (d) 1 if u[j, ℓ, k] > u[j 1,ℓ,k] , χℓ [k]= |SH | |SH − | bors along the gradient direction (this is obtained from (0 otherwise, the orientation map of the shearlet decomposition). If the magnitude is smaller, the point is discarded; if it is and R(d)u[k]= (d)u[j, ℓ, k] χ(d)[k]. the largest, it is kept. This yields a set of possible edge j SH ℓ ℓ points denoted by e0. Finally, the set of edge points can X (d) be further refined by Then vj u is modified as (d) (d) (d) (d) [n ,n ] : e [n ,n ] T, e h[n ,n ] 0 , (d) v u + R u if v u R u { 1 2 0 1 2 ≥ 0 ∗ 1 2 ≥ } v u = j j | j | ≤ | j | j R(d)u otherwise. where h is a windowing filter and T > 0 is an appropri- ( j ate threshold. The edge candidates eju at level j are then given by Numerical tests have shown that this procedure is 2 2 very effective at detecting edges. Its computational 2 1 2 ej u[k]= u[j, ℓ, k] + u[j, ℓ, k] . complexity, however, is too high for practical use. For SH ! SH ! Xℓ Xℓ 12 IEEE TRANS. IMAGE PROCESSING

Fig. 10. Results of edge detection methods. From left to right: Original image, noisy image (PSNR=24.61 dB), Sobel result (FOM=0.79), wavelet result (FOM=0.88), shearlet result (FOM=0.92).

Fig. 11. Results of edge detection methods. From left to right: Original image, noisy image (PSNR=24.58 dB), Sobel result (FOM=0.54), wavelet result (FOM=0.84), and shearlet result (FOM=0.94)

Fig. 12. Results of edge detection methods. From left to right: Original image, noisy image (PSNR=24.61 dB), Sobel result (FOM=0.32), wavelet result (FOM=0.60), and shearlet result (FOM=0.89)

old. To avoid this, hysteresis uses 2 thresholds, a high (d) Note that vj u is modified to keep the scales small T1 and a low T2. Any pixel in the image that has a for locations with positive Lipschitz regularity and in- value greater than T1 is presumed to be an edge pixel, crease the scales for locations with negative Lipschitz and is marked as such immediately. Then, any pixels regularity. that are connected to this edge pixel and that have a The edge candidates for the J th level are finally value greater than T2 are also selected as edge pixels. filtered through the application of− a non-maximal sup- We have tested several images using the Shearlet pression routine using the shearlet-based orientation Edge Detection algorithm described above, and com- map followed by the application of hysteresis thresh- pared its performance to other well established edge de- olding. The hysteresis thresholding, which was also in- tectors. For the tests reported in Figures 10-12, each of cluded in the original Canny edge detector [11] (see also the images had an additive white Gaussian noise with [34, Ch.7]) is used as a means of eliminating streaking. a standard deviation of 20. The shearlet decomposi- Streaking is the breaking up of an edge contour caused tion tested consisted of 4 decomposition levels using by the edge detector operator output fluctuating above 16 directional components. We compared the shearlet- and below a fixed threshold. Indeed, if a single thresh- based routine against a similar wavelet-based routine old T1 is applied to an image, then, due to noise, there that was derived by removing the shearlet aspect of the will be instances where the edge dips below the thresh- algorithm given above. Throughout all experiments the A SHEARLET APPROACH TO EDGE ANALYSIS AND DETECTION 13 wavelet and shearlet routines had the same fixed param- V. Conclusion and future direction eters. The comparison against this particular wavelet Several concepts based on the shearlet framework routine highlights how the anisotropic aspect of shear- presented in this work have a great potential for de- lets can improve performance in detection. For a base- vising alternative improved edge analysis and detection line comparison against standard routines, we also used schemes. Since the notion of analyzing edges by looking the Sobel method using its default parameters [34]. The at the maxima of the wavelet transform has appeared, standard Canny method was not used for comparisons a multitude of edge detection algorithms based on this because we found that the default parameters provided concept have been developed. In this work we have only very poor results. By adjusting the variance used in explored a few of these techniques by using the shear- the default parameter for each test, better results can let transform. Several concepts, such as those in [36], be achieved. The issue of how to choose an appropriate [15], could be further explored and adapted for their variance is exactly why multiscale methods, such as the use with the shearlet transform. This study shows that wavelet-based routine tested, have been suggested. the shearlet transform provides a multiscale directional Several attempts have been made in the literature framework which is very competitive for the purpose of to propose an objective measure for the performance edge analysis and detection. This approach is based on of an edge detector. While there is no consensus and a simple and rigorous mathematical theory which ac- each of these measures has their limitations, one of the counts for the geometrical properties of edges. In par- most recognized measure is the Pratt’s Figure of Merit ticular, it provides an accurate method for extracting (FOM) [35], whose definition is based on the combina- the information about edges and their orientations even tion of three factors: non detection of true edges, detec- in presence of noise. This opens the door to a number tion of false edges and edge delocalization error, which of further applications including feature extraction and is defined as shape recognition.

N 1 d 1 VI. Acknowledgment F = 2 , max(Ne,Nd) 1+ α d(k) k=1 The authors would like to thank the referees for their X useful comments and suggestions which significantly im- where Ne is the number of actual edge points, Nd is proved this paper. the number of detected edge points, d(k) is the dis- References tance from the k-th actual edge point to the detected edge point and α is a scaling constant typically set to [1] S. Mallat and W. L. Hwang, Singularity detection and processing with wavelets, IEEE Trans. Inf. Theory, vol. 38, no. 1/9. Hence, the output from Pratt’s Figure of Merit 2, 617-643, Mar. 1992. is a fidelity function ranging from 0 to 1, where 1 is a [2] S. Mallat and S. Zhong, Characterization of signals from perfect edge detector. We computed the Pratt’ Figure multiscale edges, IEEE Trans. Pattern Anal. Mach. Intell., Vol. 14, no. 7, pp. 710-732, Jul. 1992. of Merit using the image shown in Figure 10 based on [3] C. L. Tu, W. L. Hwang, and J. Ho, “Analysis of Singularities the actual edge map that is known analytically. The From Modulus Complex Wavelets”, IEEE Transactions Inf. results reported in the following table confirm the vi- Theory, Vol. 51, n. 3, pp. 1049–1062, 2005. [4] L. Zhang, P. Bao, “A Wavelet-based Edge Detection Method sual impression that the shearlet approach outperforms by Scale Multiplication”, Pattern Recognition, Proceedings. the other edge detectors considered in our tests. The 16th International Conference on, Vol. 3, pp. 501–504, Aug. Pratt’ Figure of Merit was also computed for the im- 2002. [5] J. Li, A Wavelet Approach to Edge Detection, Masters The- ages in Figures 11-12 and the results are reported in sis, Sam Houston State University, 2003. the captions of those figures. Since in these cases the [6] M. N. Do, and M. Vetterli, “The contourlet transform: an exact edge maps were not known, they were computed efficient directional multiresolution image representation”, IEEE Trans. Image Process., vol. 14, no. 12, pp. 2091-2106, using the Canny algorithm on the image without noise. Dec. 2005. The shearlet approach is shown to yield higher values [7] D. D. Po and M. N. Do, “Directional multiscale modeling of for the Figure of Merit. images using the contourlet transform”, IEEE Trans. Image Process., vol. 15, no. 6, pp. 1610-1620, June 2006. [8] I.W.Selesnick, R.G. Baraniuk, and N.G.Kingsbury, “The Dual-Tree Complex Wavelet Transform”, IEEE Signal Pro- Pratt’s FOM for the image of Figure 10 cessing Magazine vol. 22(6), 123–151, Nov. 2005. PSNR Shearlet Wavelet Sobel Prewitt [9] E. J. Candès and D. L. Donoho, “Ridgelets: a key to higher– dimensional intermittency?”, Phil. Trans. Royal Soc. London 34.16 dB 0.94 0.93 0.88 0.87 A, Vol. 357, pp. 2495–2509, 1999. 28.14 dB 0.94 0.91 0.84 0.84 [10] E. J. Candès and D. L. Donoho, “New tight frames of 24.61 dB 0.92 0.88 0.79 0.78 curvelets and optimal representations of objects with C2 singularities”, Comm. Pure Appl. Math., Vol. 56, pp. 219–266, 22.10 dB 0.76 0.67 0.67 0.69 2004. 14 IEEE TRANS. IMAGE PROCESSING

[11] F. J. Canny, A computational approach to edge detection, [36] T. Aydin, Y. Yemes, E. Anarim, B. Sankur, ”Multidirec- IEEE Trans. Pattern Anal. Machine Intell., Vol. 8, no. 6, tional and Multiscale Edge Detection via M-Band Wavelet 679-698, 1986. Transform”, IEEE Trans. Image Process., Vol. 5, pp. 1370– [12] T. Lindeberg, “Edge detection and ridge detection with au- 1377, 1996. tomatic scale selection”, Proceedings of the IEEE Int. Con- ference on Computer Vision and Pattern Recognition, San Francisco, June 1996, pp. 465-470, 1996 [13] T. Lindeberg, “Feature Detection with Automatic Scale Se- Sheng Yi Sheng Yi received the B.E in lection”, International Journal of Computer Vision, vol. electronic and information engineering from 30(2), pp. 79-116, 1998. Huazhong University of Science and Tech- [14] S. Mallat, A Wavelet Tour of Signal Processing, Academic nology, China, in 2006, and currently in Press, San Diego, CA, 1998. Ph.D. program of electrical engineering de- [15] D. Ziou, and S. Tabbone, “Edge Detection Techniques An partment in North Carolina State Univer- Overview”, International Journal of Pattern Recognition sity. His research interesting includes im- and Image Analysis, Vol. 8(4) pp. 537–559, 1998. age representation, shape analysis and high [16] P. Perona, “Steerable-scalable kernels for edge detection and dimensional signal processing. junction analysis”, Image Vis. Comput., Vol. 10, pp. 663- 672, 1992. [17] J. Weickert, Foundations and applications of nonlinear anisotropic diffusion filtering, Zeitschr. Angewandte Math. Demetrio Labate Demetrio Labate re- Mechan., Vol. 76, pp. 283-286, 1996. ceived the Ph.D. in electrical engineering [18] J. Geusebroek, A. W. M. Smeulders, and J. van de Wei- from the Politecnico di Torino, Italy, in jer, “Fast Anisotropic Gauss Filtering”, IEEE Trans. Image 1995, and the Ph.D. degree in Mathematics Process., Vol. 8, pp. 938–943, 2003. from the Georgia Institute of Technology, [19] G. Easley, D. Labate, and W-Q. Lim “Sparse Directional Im- Atlanta, in 2000. Between 2000 and 2003, age Representations using the Discrete Shearlet Transform”, he was a Chauvenet lecturer at Washing- to appear in Appl. Comput. Harmon. Anal. 2008. ton University in St.Louis. Since 2003, he [20] D. Labate, W.-Q. Lim, G. Kutyniok and G. Weiss, “Sparse is an Assistant Professor at North Carolina multidimensional representation using shearlets”, Wavelets State University. His research interests in- XI (San Diego, CA, 2005), SPIE Proc., 5914, pp. 254–262, clude harmonic analysis, with special em- SPIE, Bellingham, WA, 2005. phasis on wavelet theory, time-frequency analysis and image pro- [21] K. Guo and D. Labate, “Optimally sparse multidimensional cessing applications. Prof. Labate received the National Science representation using shearlets”, SIAM J. Math. Anal., Vol. Foundation CAREER Award in 2008. 9, pp. 298–318, 2007. [22] E. J. Candès and D. L. Donoho, “Continuous curvelet trans- Glenn R. Easley Glenn R. Easley received form: I. Resolution of the wavefront set”, Appl. Comput. the B.S. degree (with honors) and the M.A. Harmon. Anal., Vol. 19, pp. 162–197, 2005. 162–197. degree in mathematics from the University [23] S. Dekel and A. Sherman, “Curvelets: A low-level system of Maryland, College Park, in 1993 and for computer vision”, preprint 2008. 1996, respectively, and the Ph.D. degree in computational science and informatics from [24] W. Freeman and E. Adelson, “The design and use of steer- George Mason University in 2000. Since able filters”, IEEE Trans. Patt. Anal. and Machine Intell., 2000, he has been working for System Plan- Vol. 13, pp. 891-906, 1991. ning Corporation in signal and image pro- [25] E. Simoncelli and W. Freeman, “The steerable pyramid: A cessing. He has also been a Visiting Assis- flexible architecture for multi-scale derivative computation”, tant Professor for the Norbert Wiener Cen- in Proc. IEEE ICIP, Washington, DC, 1995. ter in the University of Maryland since 2005. His research inter- [26] M. Holschneider, Wavelets. Analysis tool, Oxford University ests include computational harmonic analysis, with special em- Press, Oxford, 1995. phasis on wavelet theory and time-frequency analysis, synthetic [27] G. Kutyniok and D. Labate, “Resolution of the Wavefront aperture radar, deconvolution and computer vision. Set using Continuous Shearlets”, to appear in Trans. Am. Math. Soc., 2008. Hamid Krim Hamid Krim received his [28] K. Guo, D. Labate and W. Lim, “Edge analysis and identi- degrees in Electrical Engineering. As fication using the continuous shearlet transform”, to appear a member of technical staff at AT&T in Appl. Comput. Harmon. Anal. . Bell Labs, he has worked in the area [29] J. P. Antoine, R. Murenzi, P. Vandergheynst, S. T. Ali, Two- of telephony and digital communication Dimensional Wavelets and their Relatives, Cambridge Uni- systems/subsystems. In 1991 he became versity Press, 2004. a NSF Post-doctoral scholar at Foreign [30] P. Perona, “Orientation Diffusion”, IEEE Transactions on Centers of Excellence (LSS Supelec Univ. Image Processing, Vol.7(3), pp. 457–467 , 1998. of Orsay, Paris, France). He subsequently [31] R. Mester, “Orientation Estimation: Conventional Tech- joined the Laboratory for Information and niques and A New Non-differential Approach”, Proc. 10th Decision Systems, MIT, Cambridge, MA, European Signal Processing Conf, 2000. as a Research Scientist performing and supervising research in [32] J. Bigun, G.H.Granlund, J.Wiklund, “Multidimensional his area of interest, and later as a faculty in the ECE dept. Orientation Estimation with Applications to Texture Analy- at North Carolina State University in 1998. He is an original sis and Optical Flow”, IEEE Transactions on Pattern Anal- contributor and now an affiliate of the Center for Imaging ysis and Machine Intelligence, Vol. 13(8), pp. 775–790, 1991. Science sponsored by the Army. He also is a recipient of the [33] R. O. Duda, P. E. Hart, D. G. Stork, Pattern Classification, NSF Career Young Investigator award. He is a Fellow of IEEE Wiley-Interscience, 2000. and was on the editorial board of the IEEE Trans. on SP and [34] E. Davies, Machine Vision: Theory, Algorithms and Prac- regularly contributes to the society in a variety of ways. He is a ticalities, Academic Press, 1990. member of SIAM and of Sigma Xi. His research interests are in [35] W.K. Pratt, Digital Image Processing, Wiley Interscience statistical Signal Processing and mathematical modeling with a Publications, 1978. keen emphasis on applications.