Edge Orientation Using Contour Stencils Pascal Getreuer

To cite this version:

Pascal Getreuer. Edge Orientation Using Contour Stencils. SAMPTA’09, May 2009, Marseille, France. Special session on sampling and (in)painting. hal-00452291

HAL Id: hal-00452291 https://hal.archives-ouvertes.fr/hal-00452291 Submitted on 1 Feb 2010

HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Edge Orientation Using Contour Stencils

Pascal Getreuer (1)

(1) Department of Mathematics, University of California Los Angeles [email protected]

Abstract: structure tensor J( u) = u u. The structure tensor satisfies ∇J( u)∇ = J⊗( ∇u) and u is an eigenvector of J( u).−∇ The structure∇ tensor takes∇ into Many image processing applications require estimat- ∇ ing the orientation of the image edges. This estimation account the orientation but not the sign of the direc- is often done with a finite difference approximation tion, thus solving the antipodal cancellation problem. of the orthogonal gradient. As an alternative, we ap- As developed by Weickert [9], let ply contour stencils, a method for detecting contours from total variation along curves, and show it more Jρ( uσ) = Gρ J(Gσ u) (1) ∇ ∗ ∗ robustly estimates the edge orientations than several where Gσ and Gρ are Gaussians with standard devia- finite difference approximations. Contour stencils are tions σ and ρ. The eigenvector of Jρ( uσ) associated demonstrated in image enhancement and zooming ap- with the smaller eigenvalue is called the∇ coherence di- plications. rection, and is an effective approximation of edge orientation.

1. Introduction 2. Contour Stencils

A fundamental and challenging problem in image processing is estimating edge orientations. Accurate Numerical implementation of J( u) yet involves es- ∇ edge orientations are important for example in edge- timating u. Since numerical estimates of u are ∇ ∇ oriented inpainting methods [2], and optical character sensitive to noise and unreliable near edges, signiﬁ- recognition features [8]. cant amounts of smoothing is still needed for accept- able results. We abandon u⊥ and approach the estimation of edge orientation∇ from an entirely different 1.1 u⊥ for Estimating Edge Orientation principle. ∇ Given a smooth curve C and a parameterization γ : A starting point to edge orientation estimation is to [0,T ] C, consider measuring the total variation of approximate u⊥ with ﬁnite differences. Finite dif- u along→C, ∇ ference estimation alone is typically too noisy to be T reliable, especially near edges, so the gradient is often TV(C) = ∂tu γ(t) dt. (2) Z regularized by a convolution u (G u) where 0 G is for example a Gaussian.∇ However,≈ ∇ there∗ is a se- Edge orientations can be estimated by comparing rious problem in that u⊥ and u⊥ both describe TV(C) with various candidate curves. Contour sten- the same edge orientation,∇ so linear−∇ smoothing tends cils [4, 5] is a numerical implementation of this idea. to cancel the desired edge information. Let u :Λ R be a discrete image. Denote by ui,j, Introduced by Bigun¨ and Granlund [1] and Forstner (i, j) Λ,→ the value of u at the (i, j)th pixel, and let ∈ 2 and Gulch [3], a better approach is to use the 2 2 xi,j R denote its spatial location. × ∈ +i 1 1 2 1 21 2 2 1 2 1 +j S α, β ( ) = 1 2 1 1 2 2 1 1 − 1 1 (i, j) 1 α = (i, j), β = (i 1, j + 1), 81 α = (i, j), β = (i + 1, j − 1), 1 4 1 > α i, j , β i , j , >4 = ( + 1) = ( + 1 ) 1 2 2 1 <>1 α = (i + 1, j + 1), β = (i, j + 2), 1 2 1 1 2 1 1 1 α = (i + 1, j + 1), β = (i + 2, j), 1 2 1 2 1 1 2 1 1 2 > >0 otherwise :> 1 1 1 1 2 1 2 1 Figure 1: An example contour stencil for detecting 2 2 2 2 ◦ S 1 2 1 2 a 45 orientation. 1 1 1 1

contour stencil R+ A is a function :Λ Λ de- Figure 3: A node-centered stencil set. scribing weighted edges betweenS pixels× (see→ Figure 1). These edges approximate several parallel curves local- ized over a small neighborhood. As a discretization of candidate stencil, and then determining the best-fitting (2), the total variation of is stencil ∗. For efficient implementation, define S S 1 H A TV( ) := (α, β) uα uβ , (3) D = vi,j vi+1,j ,D = vi,j vi+1,j+1 , S |S| S | − | i,j | − | i,j | − | α,βX∈Λ V B D = vi,j vi,j ,D = vi,j vi ,j , i,j | − +1| i,j | +1 − +1 | and := (α, β) xα xβ . For the contour |S| α,β S | − | then the TV( ) can be computed as sums of these dif- stencil in FigureP 1, = (1 + 1 + 4 + 1 + 1)√2 and S |S| ferences, and the differences may be reused between 1 successive cells. For the proposed stencil sets, contour TV( ) = ui,j ui−1,j+1 + ui,j ui+1,j−1 S |S| | − | | − | stencils cost a few dozen operations per pixel [4]. + 4 ui,j+1 ui+1,j | − | Input Estimated Orientations + ui ,j ui,j + ui ,j ui ,j . | +1 +1 − +2| | +1 +1 − +2 |

1 1 121 1 1 1 4 1 1 4 1 121 2 2 1 1 1 1

1 2 1 2 1 1 1 2 1 1 1 1 2 1 1 1 2 1 1 1 2 1

1 1 2 1 1 1 2 1 2 1 1 2 1 1 1 2 1 2 1 1 1 1 Figure 4: Edge orientation estimation with contour Figure 2: The proposed cell-centered contour stencils. stencils (using the cell-centered stencils in Figure 2).

The contours of u are estimated by ﬁnding a stencil Contour stencils extend naturally to nonscalar data by with low total variation, replacing the absolute value in (3) with a metric. On color images for example, a suitable choice is the ℓ1 ∗ = arg min TV( ) (4) vector norm in YCbCr color space. S S∈Σ S where Σ is a set of candidate stencils (see Figures 2 and 3). The best-ﬁtting stencil ∗ provides a model of 3. Comparison the underlying contours. S

In summary, contour stencil orientation estimation is Here we compare contour stencils and several finite done by first computing the TV estimates (3) for each difference methods for estimating edge orientation. As a test image with fine orientations, we use a small square whose corners correspond to ui,j, ui+1,j, image of straw (Figure 5). ui,j+1, ui+1,j+1. Cell-centered methods compute orientation estimates logically located in the center of the u cells. With node-centered methods, the edge orientation estimates are centered on the pixels.

+ Let Dx denote the forward difference operator + Dx ui,j = ui+1,j ui,j and similarly in the other co- + − ordinate Dy . An estimate of u symmetric over the cell is ∇ + + (Dx ui,j + Dx ui,j+1)/2 ui,j + + . (5) ∇ ≈ (Dy ui,j + Dy ui+1,j)/2 Figure 6 compares u⊥ estimated using (5) with contour stencils using∇ the cell-centered stencil set shown in Figure 2.

Figure 5: The test image. Sobel filter (6) As is done with coherence direction (1), any orientation field θ~ can be smoothed by filtering its tensor product: Gρ (θ~ θ~). But for easier comparison, all methods are shown∗ ×without smoothing.

u⊥ with (5) ∇

Contour Stencils (Σ as in Figure 3)

Contour Stencils (Σ as in Figure 2)

Figure 7: Comparison of node-centered methods.

The Sobel filter [7] is a node-centered approximation of u, ∇ 1 0 1 − ∂xu  2 0 2 u (6) ≈ −1 0 1 ∗ Figure 6: Comparison of cell-centered methods. −  and similarly for ∂yu. Figure 7 compares the Sobel fil- We consider two categories of methods: cell-centered ter with contour stencils using the node-centered sten- and node-centered. Define the (i, j)th cell as the cil set from Figure 3. 4. Applications Input Zooming (4×)

Contour stencils are useful in applications where edges are signiﬁcant.

Input Contour Stencil Enhancement

Figure 9: (This is a color image.) Edge-adaptive zooming using contour stencils [5].

References:

[1] J. Bigun¨ and G. H. Granlund. Optimal orientation detection of linear symmetry. In IEEE First Inter- national Conference on Computer Vision, pages 433–438, London, Great Britain, June 1987. [2] Folkmar Bornemann and Tom Marz.¨ Fast image inpainting based on coherence transport. J. Math. Imaging Vis., 28(3):259–278, 2007. [3] W. Forstner¨ and E. Gulch. A fast operator for detection and precise location of distinct points, cor- Figure 8: Simultaneous sharpening and denoising us- ners, and centers of circular features. pages 281– ing contour stencils [4]. 305, 1987. [4] Pascal Getreuer. Contour stencils for edge- Contour stencils can be useful in discretizing image adaptive image interpolation. volume 7257, 2009. diffusion processes. Figure 8 demonstrates image en- [5] Pascal Getreuer. Image zooming with contour hancement using a combination of the Rudin-Osher stencils. volume 7246, 2009. shock filter [6] and TV-flow that has been discretized [6] S. J. Osher and L. I. Rudin. Feature-oriented im- with contour stencils. age enhancement using shock filters. SIAM Jour- nal on Numerical Analysis, 27:919–940, 1990. As another application, Figure 9 shows an image [7] Irwin Sobel and Jerome A. Feldman. A 3x3 zooming result using contour stencils. The method isotropic gradient operator for image process- approaches zooming as an inverse problem using a ing. Presented at a talk at the Stanford Artificial least-squares graph regularization. The regularization Project in 1968. is adapted according to the edge orientations estimated [8] Øivind Due Trier, Anil K. Jain, and Torfinn from the contour stencils. Taxt. Feature-extraction methods for character- recognition: A survey. Pattern Recognition, 29(4):641–662, April 1996. 5. Conclusions [9] Joachim Weickert. Anisotropic diffusion in image processing. ECMI Series, Teubner-Verlag, Stuttgart, Germany, 1998. Contour stencils provide reliable orientation estimates at low computational cost, enabling better results in image processing applications.