MULTIREGION GRAPH CUT IMAGE SEGMENTATION Mohamed Ben Salah1, Ismail Ben Ayed2 and Amar Mitiche3 1,3 Institut National de la Recherche Scientifique, INRS-EMT, 800 De La Gauchetire Ouest , Montral, QC, H5A 1K6, Canada 2 GE healthcare, 268 Grosvenor, E5-137, London, ON, N6A 4V2, Canada Keywords: Combinatorial optimization, graph cuts, parametric modeling, image segmentation. Abstract: The purpose of our study is two-fold: (1) investigate an image segmentation method which combines paramet- ric modeling of the image data and graph cut combinatorial optimization and, (2) use a prior which allows the number of labels/regions to decrease when the number of regions is not known and the algorithm initialized with a larger number. Experimental verification shows that the method results in good segmentations and runs faster than conventional graph cut methods. 1 INTRODUCTION ber of labels to equal the number of regions. Regions can then be described by positive parametric func- Image segmentation occurs in many important ap- tions, in which case, the data term of the objective plications. It consists of partitioning an image into function measures the conformity of the image data regions homogeneous with respect to a given de- to a parametric model in each region. This formula- scription. Energy minimization formulations have tion was used to state the problem of concurrent seg- led to flexible, transparent, and effective algorithms. mentation and estimation of image motion (Schoen- These formulations can be divided into two broad emann and Cremers, 2006). This formulation leads categories: continuous and discrete (Zhu and Yuille, to an optimization which iterates two steps: contin- 1996) (Ayed et al., 2006) (Chan and Vese, 2001) uous optimization with respect to the region parame- (Boykov et al., 2006). Continuous formulations, ters and segmentation by discrete graph cut optimiza- which seek a partition of the image domain by active tion. However, the regularization term in (Schoene- curves via a level set representation, have segmented mann and Cremers, 2006), which estimates the length accurately a variety of difficult images. of the segmentation boundaries assumes the number Discrete formulations use objective functions of regions known. When the number of regions is not which contain terms similar to those in continuous known beforehand, one sensible formulation would formulations, generally a data term which measures be to use a larger number initially, to be decreased by the conformity of a segmentation to the image data, the graph cut optimization algorithm (Boykov et al., and a regularization term. A graph cut formulation 2001). However, and contrary to the priors used in states segmentation as a labeling problem. The graph (Boykov et al., 2001), the length prior does not allow cut algorithm assigns each pixel a grey level label this, in general, because merging two disjoint regions in the set of all possible labels. Thus, the optimiza- does not change the value of the prior. tion of the objective function results in a segmenta- The purpose of our study is two-fold: (1) investi- tion described by a subset of the set of possible labels. gate image segmentation which combines parametric Since no explicit number of regions guides the algo- modeling of the image data and graph cut combinato- rithm, the solution is generally an implicit overseg- rial optimization and (2) use a prior which allows the mentation. Also, the use of a set of all possible labels number of labels/regions to decrease when the num- lengthens the execution time unnecessarily. When the ber of regions is not known and the algorithm initial- number of regions is known, one can reduce the num- ized with a larger number. As in (Schoenemann and 535 Ben Salah M., Ben Ayed I. and Mitiche A. (2008). MULTIREGION GRAPH CUT IMAGE SEGMENTATION. In Proceedings of the Third International Conference on Computer Vision Theory and Applications, pages 535-538 DOI: 10.5220/0001081605350538 Copyright c SciTePress VISAPP 2008 - International Conference on Computer Vision Theory and Applications Cremers, 2006), the method iterates two steps: closed of labels via continuous optimisation. Given the seg- form region parameters update and segmentation by mentation, regions parameters are estimated accord- graph cut combinatorial optimization but our regular- ing to a closed-form solution. Given the parameters, ization term allows region merging. the segmentation is updated by graph cut optimiza- tion. 2 OBJECTIVE FUNCTION 3.1 Labels Updating As in (Boykov et al., 2001), and other studies as well Given a segmentation, a labeling λ is an assignment (Chan and Vese, 2001), the objective function con- of a label λ(p) to each pixel p of the image domain tains two terms: a data term to measure the confor- Ω. Equivalently, it is a partition of Ω into disjoint re- mity of the segmentation to a parametric model, a gions Rl where l is the label assigned to pixels of the piecewise constant model in this case, and a regular- region Rl. The process of updating labels consists in ization term to bias the segmentation toward partitions estimating these parameters in a way to optimize the of the image domain with fewer boundary edges. The functional when the segmentation is given. goal is to segment the image into a fixed but arbitrary To optimize the functional (3) for a given segmenta- number of regions using graph cut optimization of the tion, we proceed by derivation towards the value of a objective function. To do so, we use a labeling func- label µl: tion λ : Ω −→ L so that each pixel p ∈ Ω is assigned λ ∂F ({µ },λ) the label (p) in a finite set of labels L of cardinal- l = 2 ∑ (µ − I )+ ity N equal to the number of regions. Let R be the ∂µ l p reg l l p∈Rl region of pixels with label l. The solution of the min- α imization, or the final labeling, can be represented ei- 2 ∑ (µλ(p) − µλ(q)) (4) λ ther by the function λ or the partition P = {R | l ∈ L}, {p,q}∈N , (p)=l l |µλ(p)−µλ(q)|<const where Rl = {p ∈ Ω | λ(p)= l}. The energy functional to minimize, is formulated using the labeling function We choose a neighborhood system, N , of size 4 and λ , a data term and the regularization term, R : let N p be the set of pixels q neighbors of p which F (λ)= ∑ Dp(λ(p)) + α R (λ) (1) are assigned a label λ(q) different from, λ(p), the la- p∈Ω bel assigned to p and verifying |µλ(p) −µλ(q)| < const. where α is a positive weight. Typically, Dp(λ(p)) is Consequently, equation (4) can be written as 2 (µl − Ip) , where Ip is the intensity at pixel p and µl ∂F is the value of the label assigned to region R for 1 ≤ = 2 ∑ (µl − Ip) + 2α ∑ ∑ (µl − µλ ) (5) l ∂µ (q) l p∈Rl p∈Cl l ≤ Nreg. We use the following regularization term: q∈N p λ ∑ λ λ R ( )= r{p,q}( (p), (q)) (2) where C is the boundary of the region R . The la- {p,q}∈N l l bel value optimizing the functional, given a segmenta- where N is the neighborhood set. Function λ λ tion, is deduced from the last equation and expressed r{p,q}( (p), (q)) is a smoothness regularization in a closed-form as follows function. We consider a regularization function α which is piecewise smooth. More precisely, we ∑ Ip + ∑ ∑ µλ(q) p∈R p∈C consider the truncated squared absolute difference l l q∈N p 2 µ = (6) model r(λ(p),λ(q)) = min(const,|µλ − µλ | ) l (p) (q) ♯Rl + α ∑ ♯N where const is, in all experimentations, the same used p p∈Cl in (Boykov et al., 2001). The objective function can where ♯ designates set cardinality. be written, for 1 ≤ l ≤ Nreg, as follows: λ ∑ ∑ 2 F ({µl }, ) = (µl − Ip) 3.2 Graph Cuts Segmentation Updating l∈L p∈Rl (3) α λ λ + ∑ r{p,q}( (p), (q)) {p,q}∈N Graph cut optimization was introduced by Greig et al. (Greig et al., 1989) for image restoration. We use the algorithm of (Boykov et al., 2001) which has 3 MINIMIZATION been used successfully in several vision applications. The algorithm functions as follows. Let G = hV ,Ei Minimization of functional (1) is done by alternating be a weighted graph where V is the set of vertices graph cut optimization for segmentation and updating (nodes) and E the set of edges. V contains a node 536 MULTIREGION GRAPH CUT IMAGE SEGMENTATION for each pixel and two additional nodes called termi- much larger than the right number of regions: figure nals. There is an edge {p,q} between any two distinct 1(e) is segmented into 98 regions, 1(f) is segmented nodes p and q. A cut C ⊂ E is a set of edges verify- into 51 regions, 1(g) is segmented into 58 regions and ing: 1(h) is segmented into 53 regions. On the other hand, • Terminals are separated in the graph G(C) = our method allows regions parameters to vary jointly hV ,E \ Ci with the segmentation; thus, the initial set of labels is arbitrarily chosen and the number of regions is fixed • No subset of C separates terminals in G(C) equal to the right number. The minimum cut problem consists in finding the cut In addition to that, our method realizes a big improve- C in a given graph with the lowest cost. The cost of a ment in term of running time since we deal with a cut, |C|, is the sum of its edges weights.