Color Mathematical Morphology Using a Fuzzy Color-Based Supervised Ordering
Mateus Sangalli and Marcos Eduardo Valle*
Department of Applied Mathematics Institute of Mathematics, Statistics, and Scientific Computing University of Campinas - Brazil
July 4, 2018
Marcos Eduardo Valle (Unicamp) Fuzzy Color-Based MM July 4, 2018 1 / 19 Introduction
Mathematical morphology (MM) is a powerful non-linear image processing framework based on geometrical and topological concepts.
Applications of MM include edge detection, segmentation, feature extraction, and image reconstruction and decomposition.
Mathematical morphology has been developed by Matheron and Serra in the 1960s for the analysis of binary images.
It has been successfully extended to gray-scale images using the level-sets and the concept of umbra in the 1980s.
Some approaches to gray-scale MM have also been developed using fuzzy logic and fuzzy set theory.
Marcos Eduardo Valle (Unicamp) Fuzzy Color-Based MM July 4, 2018 2 / 19 Motivation
From a mathematical point of view, mathematical morphology can be very well defined on complete lattices.
A complete lattice L is a partially ordered set in which any subset _ ^ X ⊆ L has both a supremum X an infimum X.
Complete lattices allowed for the development of MM to multivalued data including color images.
In contrast to gray-scale approaches, however, there is no natural ordering for colors.
Thus, most research on MM consist on finding an appropriate ordering for a given color image processing tasks.
Marcos Eduardo Valle (Unicamp) Fuzzy Color-Based MM July 4, 2018 3 / 19 Color Images and Mathematical Morphology
A color image is a mapping I : D → C, where D ⊆ R2 or D ⊆ Z2 is the point set and C ⊆ R¯ 3, R¯ = R ∪ {+∞, −∞}, is the color space.
We shall focus on the RGB color space C = [0, 1]3, which is widely used by hardware devices. Let us assume that C is equipped with a partial ordering “≤”.
The dilation and the erosion of I by a structuring element S are defined by _ ^ δS(I(p) = I(p + s) and εS(p) = I(p + s). s∈S s∈S
The structuring element S is used to extract useful information about the geometrical and topological structures on I.
Marcos Eduardo Valle (Unicamp) Fuzzy Color-Based MM July 4, 2018 4 / 19 Marginal Approach
The marginal or component-wise approach is a straightforward extension of the gray-scale MM to color images.
The marginal approach is obtained by ordering colors x = (x1, x2, x3) and y = (y1, y2, y3) as follows:
x ≤marg y ⇐⇒ x1 ≤R y1, x2 ≤R y2 and x3 ≤R y3.
Despite its simplicity, the marginal approach does not take into account the correlations between the color components: certain features can be enhanced in one color channel but not in the others.
Also, there is the possibility of creating “false colors”.
Marcos Eduardo Valle (Unicamp) Fuzzy Color-Based MM July 4, 2018 5 / 19 Example – Marginal Approach
Color image f :
200
400
600
800
1000
1200 200 400 600 800 1000 1200
Marcos Eduardo Valle (Unicamp) Fuzzy Color-Based MM July 4, 2018 6 / 19 Example – Marginal Approach
Dilation:
200
400
600
800
1000
1200 200 400 600 800 1000 1200
Marcos Eduardo Valle (Unicamp) Fuzzy Color-Based MM July 4, 2018 6 / 19 Example – Marginal Approach
Erosion:
200
400
600
800
1000
1200 200 400 600 800 1000 1200
Marcos Eduardo Valle (Unicamp) Fuzzy Color-Based MM July 4, 2018 6 / 19 Lexicographical Approach
In the lexicographical approach, colors x = (x1, x2, x3) and y = (y1, y2, y3) are ranked sequentially as follows: x < y , 1 R 1 x ≤lex y ⇐⇒ x1 = y1, x2 Note that ≤lex is a total ordering! Thus, it prevents false colors. On the downside, it prioritizes excessively the first channel. Marcos Eduardo Valle (Unicamp) Fuzzy Color-Based MM July 4, 2018 7 / 19 Example – Lexicographical Approach Color image f : 200 400 600 800 1000 1200 200 400 600 800 1000 1200 Marcos Eduardo Valle (Unicamp) Fuzzy Color-Based MM July 4, 2018 8 / 19 Example – Lexicographical Approach Dilation: 200 400 600 800 1000 1200 200 400 600 800 1000 1200 Marcos Eduardo Valle (Unicamp) Fuzzy Color-Based MM July 4, 2018 8 / 19 Example – Lexicographical Approach Erosion 200 400 600 800 1000 1200 200 400 600 800 1000 1200 Marcos Eduardo Valle (Unicamp) Fuzzy Color-Based MM July 4, 2018 8 / 19 Reduced Ordering (h-Ordering) In a reduced ordering (h-ordering), colors are ranked according to a mapping h : C → R. In order to avoid ambiguities, we consider ( h(x) Note that ≤h is a total ordering! Thus, it prevents false colors! Marcos Eduardo Valle (Unicamp) Fuzzy Color-Based MM July 4, 2018 9 / 19 Distance-Based Approach In the distance-based approach, the mapping h is determined by the distance to a certain reference color r ∈ C. For example, using the Gaussian kernel, we have d 2(x, r) h (x) = κ(x, r), where κ(x, r) = exp − . r 2σ2 Note that: • x ≤rr y if y is closer to the reference than x. Thus, dilation expands objects of color r. • The least element, however, is the color farthest from the reference r. Hence, erosion doest not have a simple interpretation. Marcos Eduardo Valle (Unicamp) Fuzzy Color-Based MM July 4, 2018 10 / 19 Example – Distance-Based Approach Color image f : 200 400 600 800 1000 1200 200 400 600 800 1000 1200 Marcos Eduardo Valle (Unicamp) Fuzzy Color-Based MM July 4, 2018 11 / 19 Example – Distance-Based Approach Dilation with reference r = (1, 0, 0) (). 200 400 600 800 1000 1200 200 400 600 800 1000 1200 Marcos Eduardo Valle (Unicamp) Fuzzy Color-Based MM July 4, 2018 11 / 19 Example – Distance-Based Approach Dilation with reference r = (1, 0.65, 0) (). 200 400 600 800 1000 1200 200 400 600 800 1000 1200 Marcos Eduardo Valle (Unicamp) Fuzzy Color-Based MM July 4, 2018 11 / 19 Example – Distance-Based Approach Dilation with reference r = (0, 0, 1) (). 200 400 600 800 1000 1200 200 400 600 800 1000 1200 Marcos Eduardo Valle (Unicamp) Fuzzy Color-Based MM July 4, 2018 11 / 19 Example – Distance-Based Approach Erosion with reference r = (0, 0, 1) (). 200 400 600 800 1000 1200 200 400 600 800 1000 1200 Marcos Eduardo Valle (Unicamp) Fuzzy Color-Based MM July 4, 2018 11 / 19 h-Supervised Ordering In many situations, we are interested in objects composed of several colors or we may want to discriminate foreground and background colors. Given sets of foreground and background colors F = {f1, f2,..., fK } and B = {b1, b2,..., bM }, in a h-supervised ordering we expect that h(fi ) = >, ∀i = 1,..., K , and h(bj ) = ⊥, ∀j = 1,..., M, where > = W h(C) and ⊥ = V h(C) denote the largest and the least values attained by h. Morphological operators are interpreted with respect to F and B. Marcos Eduardo Valle (Unicamp) Fuzzy Color-Based MM July 4, 2018 12 / 19 Support Vector Machine-Based Approach The support vector machine-based approach is obtained by considering (Velasco-Forero and Angulo, 2011): K M X X hSVM(x) = αi κ(x, fi ) − βj κ(x, bj ), i=1 j=1 where αi ’s and βj ’s are given by solving the quadratic problem: K M K X X 1 X maximize αi + βj − αi α`κ(fi , f`) 2 i=1 j=1 i,`=1 M K M 1 X 1 X X − βj β`κ(bj , b`) + αi βj κ(fi , bj ), 2 2 j,`=1 i=1 j=1 K M X X subject to αi − βj = 0 and 0 ≤ αi , βj ≤ C, i=1 j=1 Marcos Eduardo Valle (Unicamp) Fuzzy Color-Based MM July 4, 2018 13 / 19 Example – SVM-Based Supervised Approach Color Image f : 200 400 600 800 1000 1200 200 400 600 800 1000 1200 Marcos Eduardo Valle (Unicamp) Fuzzy Color-Based MM July 4, 2018 14 / 19 Example – SVM-Based Supervised Approach Let σ = 0.5 and C = 10. Dilation with F = {, } e B = {, }. 200 400 600 800 1000 1200 200 400 600 800 1000 1200 Marcos Eduardo Valle (Unicamp) Fuzzy Color-Based MM July 4, 2018 14 / 19 Example – SVM-Based Supervised Approach Let σ = 0.5 and C = 10. Erosion with F = {, } e B = {, }. 200 400 600 800 1000 1200 200 400 600 800 1000 1200 Marcos Eduardo Valle (Unicamp) Fuzzy Color-Based MM July 4, 2018 14 / 19 Despite its elegant formulation, the hSVM-supervised ordering may fail to satisfy the condition: h(fi ) = > and h(bj ) = ⊥. Marcos Eduardo Valle (Unicamp) Fuzzy Color-Based MM July 4, 2018 15 / 19 Fuzzy Color-Based Approach Fuzzy colors address the vagueness and imprecision inherent to the description of colors by humans. A fuzzy color is described by a normal fuzzy set on a color space C Chamorro-Martínez et al. (2017). Given “crisp” colors f1,..., fK and b1,..., bM , we can define families of fuzzy colors ˜ ˜ F = {F1,..., FK } and B = {B1,..., BM }, whose membership functions are given by the Gaussian kernel: ϕFi (x) = κ(x, fi ) and ϕBj (x) = κ(x, bj ), ∀x ∈ C. Marcos Eduardo Valle (Unicamp) Fuzzy Color-Based MM July 4, 2018 16 / 19 hfuzzy-Supervised Ordering Given families F˜ and B˜ of foreground and background fuzzy colors, we define hfuzzy : C → [0, 1] as the degree of truth of the proposition “x is a foreground color but it is not a background color”. Formally, we have hfuzzy(x) = ϕF1 (x)O ... OϕFK (x) M η ϕB1 (x)O ... OϕBM (x) , where M, O, and η denote respectively a triangular norm, a triangular co-norm, and a strong fuzzy negation. If ϕFi (fi ) = 1, ϕBj (bj ) = 1, and ϕBj (fi ) = 0, then hfuzzy(fi ) = 1 and hfuzzy(bj ) = 0. Marcos Eduardo Valle (Unicamp) Fuzzy Color-Based MM July 4, 2018 17 / 19 Example – Fuzzy Color-Based Approach Color Image f : 200 400 600 800 1000 1200 200 400 600 800 1000 1200 Marcos Eduardo Valle (Unicamp) Fuzzy Color-Based MM July 4, 2018 18 / 19 Example – Fuzzy Color-Based Approach Let σ = 0.5. Dilation with F = {, } e B = {, }. 200 400 600 800 1000 1200 200 400 600 800 1000 1200 Marcos Eduardo Valle (Unicamp) Fuzzy Color-Based MM July 4, 2018 18 / 19 Example – Fuzzy Color-Based Approach Let σ = 0.5. Erosion with F = {, } e B = {, }. 200 400 600 800 1000 1200 200 400 600 800 1000 1200 Marcos Eduardo Valle (Unicamp) Fuzzy Color-Based MM July 4, 2018 18 / 19 Concluding Remarks We introduced a fuzzy color-based supervised ordering for color mathematical morphology. The fuzzy-based approach can address the vagueness and uncertainty inherent to the description of colors. In the future, we plan to investigate further the theoretical properties of the fuzzy color-based approach. We also intent to study their applications for color image analysis and processing. Thank you very much! Marcos Eduardo Valle (Unicamp) Fuzzy Color-Based MM July 4, 2018 19 / 19 References (1) J. Chamorro-Martínez, J. M. Soto-Hidalgo, P. M. Martínez-Jiménez, and D. Sánchez. Fuzzy color spaces: A conceptual approach to color vision. IEEE Transactions on Fuzzy Systems, 25(5): 1264–1280, 2017. S. Velasco-Forero and J. Angulo. Supervised Ordering in Rp: Application to Morphological Processing of Hyperspectral Images. IEEE Transactions on Image Processing, 20(11):3301–3308, Nov 2011. doi: 10.1109/TIP.2011.2144611. Marcos Eduardo Valle (Unicamp) Fuzzy Color-Based MM July 4, 2018 1 / 1