Morphological Image Processing Morphology • Identification, analysis, and description of the structure of the smallest unit of words • Theory and technique for the analysis and processing of geometric structures – Based on set theory, lattice theory, topology, and random functions – Extract image components useful in the representation and description of region shape such as boundaries, skeletons, and convex hull Preliminaries • Set theory in the context of image processing – Set of all white pixels in a binary image is a complete morphological description of the image • Sets in binary images – Members of the 2D integer space Z2 – Each element of the set is a 2-tuple whose coordinates are the (x, y) coordinates of a white pixel in the image – Gray scale images can be represented as a set of 3-tuples in Z3 – Set reflection Bˆ Bˆ = {w|w = −b, for b ∈ B} ∗ In binary image, Bˆ is the set of points in B whose (x, y) coordinates have been replaced by (−x, −y) ∗ Figure 9.1a – Set translation ∗ Translation of a set B by point z = (z1, z2) is denoted by (B)z (B)z = {c|c = b + z, for b ∈ B} ∗ In binary image, (B)z is the set of points in B whose (x, y) coordinates have been replaced by (x + z1, y + z2) ∗ Figure 9.1c – Set reflection and set translation are used to formulate operations based on so-called structuring elements ∗ Small sets or subimages used to probe an image for properties of interest ∗ Figure 9.2 ∗ Preference for ses to be rectangular arrays – Using ses in morphology ∗ Figure 9.3 – A simple set A and an se B ∗ Convert A to a rectangular array by adding background elements ∗ Make background border large enough to accommodate the entire se when the origin is on the border of original A ∗ Fill in the se with the smallest number of background elements to make it a rectangular array ∗ Operation of set A using se B · Create a new set by running B over A · Origin of B visits every element of A Morphological Image Processing 2 · If B is completely contained in A, mark that location as a member of the new set; else it is not a member of the new set · Results in eroding the boundary of A Erosion and dilation • Erosion – With A and B as sets in Z2, erosion of A by B, denoted by A B is defined as A B = {z | (B)z ⊆ A} – Set of all points z such that B, translated by z, is contained in A – B does not share any common elements with the background c A B = {z | (B)z ∩ A = ∅ – Figure 9.4 – Example: Figure 9.5 ∗ Erosion shrinks or thins objects in a binary image ∗ Morphological filter in which image details smaller than the se are filtered/removed from the image • Dilation – With A and B as sets in Z2, dilation of A by B, denoted by A ⊕ B is defined as A ⊕ B = {z | (Bˆ)z ∩ A 6= ∅} – Reflect B about the origin, and shift the reflection by z – Dilation is the set of all displacements z such that B and A overlap by at least one element – An equivalent formulation is A ⊕ B = {z | [(Bˆ)z ∩ A] ⊆ A} – Grows or thickens objects in a binary image – Figure 9.6 – Example: Figure 9.7 ∗ Bridging gaps in broken characters ∗ Lowpass filtering produces a grayscale image; morphological operation produces a binary image • Duality – Erosion and dilation are duals of each other with respect to set complementation and reflection (A B)c = Ac ⊕ Bˆ (A ⊕ B)c = Ac Bˆ – Allows for erosion of an image by dilating its background (Ac) – Proving duality ∗ Definition for erosion can be written as c c (A B) = {z | (B)z ⊆ A} c ∗ (B)z ⊆ A ⇒ (B)z ∩ A = ∅ Morphological Image Processing 3 ∗ So, the previous expression yields c c c (A B) = {z | (B)z ∩ A = ∅} c c ∗ The complement of the set of z’s that satisfy (B)z ∩ A = ∅ is the set of z’s such that (B)z ∩ A 6= ∅ ∗ This leads to c c (A B) = {z | (B)z ∩ A 6= ∅} = Ac ⊕ Bˆ Opening and closing • Opening smoothes the contours of an object, breaks narrow isthmuses, and eliminates thin protrusions • Closing smoothes sections of contours , fusing narrow breaks and long thin gulfs, eliminates small holes, and fills gaps in the contour • Opening of a set A by se B, denoted by A ◦ B, is defined by A ◦ B = (A B) ⊕ B • Closing of a set A by se B, denoted by A • B, is defined by A • B = (A ⊕ B) B • Geometric interpretation of opening expressed as a fitting process such that [ A ◦ B = {(B)z | (B)z ⊆ A} – Union of all translates of B that fit into A – Figure 9.8 • Similar interpretation of closing inFigure 9.9 • Example – Figure 9.10 • Duality property (A • B)c = (Ac ◦ Bˆ) (A ◦ B)c = (Ac • Bˆ) • Opening operation satisfies the following properties 1. A ◦ B ⊆ A 2. C ⊆ D ⇒ C ◦ B ⊆ D ◦ B 3. (A ◦ B) ◦ B = A ◦ B • Similarly, closing operation satisfies 1. A ⊆ A • B 2. C ⊆ D ⇒ C • B ⊆ D • B 3. (A • B) • B = A • B – In both the above cases, multiple application of opening and closing has no effect after the first application Morphological Image Processing 4 • Example: Removing noise from fingerprints – Figure 9.11 – Noise as random light elements on a dark background Hit-or-miss transformation • Basic tool for shape detection in a binary image – Uses the morphological erosion operator and a pair of disjoint ses – First se fits in the foreground of input image; second se misses it completely – The pair of two ses is called composite structuring element • Figure 9.12 – Three disjoint shapes denoted C, D, and E ∗ A = C ∪ D ∪ E – Objective: To find the location of one of the shapes, say D – Origin/location of each shape given by its center of gravity – Let D be enclosed by a small window W – Local background of D defined by the set difference (W − D) ∗ Note that D and W − D provide us with the two disjoint ses D ∩ (W − D) = ∅ – Compute Ac – Compute A D – Compute Ac (W − D) – Set of locations where D exactly fits inside A is (A D) ∩ (Ac (W − D)) ∗ The exact location of D – If B is the set composed of D and its background, the match of B in A is given by A B = (A D) ∩ [Ac (W − D)] • The above can be generalized to the composite se being defined by B = (B1,B2) leading to c A B = (A B1) ∩ (A B2) • A point z in universe A belongs to the output if (B1)z fits in A (hit) and (B2)z misses A Some basic morphological algorithms • Useful in extracting image components for representation and description of shape • Boundary extraction – Boundary of a set A ∗ Denoted by β(A) ∗ Extracted by eroding A by a suitable se B and computing set difference between A and its erosion β(A) = A − (A B) Morphological Image Processing 5 – Figure 9.13 ∗ Using a larger se will yield a thicker boundary – Figure 9.14 • Hole filling – Hole ∗ Background region surrounded by a connected border of foreground pixels – Algorithm based on set dilation, complementation, and intersection – Let A be a set whose elements are 8-connected boundaries, each boundary enclosing a background (hole) – Given a point in each hole, we want to fill all holes – Start by forming an array X0 of 0s of the same size as A ∗ The locations in X0 corresponding to the given point in each hole are set to 1 – Let B be a symmetric se with 4-connected neighbors to the origin 0 1 0 1 1 1 0 1 0 c – Compute Xk = (Xk−1 ⊕ B) ∩ A k = 1, 2, 3,... – Algorithm terminates at iteration step k if Xk = Xk−1 – Xk contains all the filled holes – Xk ∪ A contains all the filled holes and their boundaries – The intersection with Ac at each step limits the result to inside the roi ∗ Also called conditioned dilation – Figure 9.15 – Example: Figure 9.16 ∗ Thresholded image of polished spheres (ball bearings) ∗ Eliminate reflection by hole filling ∗ Points inside the background selected manually • Extraction of connected components – Let A be a set containing one or more connected components – Form an array X0 of the same size as A ∗ All elements of X0 are 0 except for one point in each connected component set to 1 – Select a suitable se B, possibly an 8-connected neighborhood as 1 1 1 1 1 1 1 1 1 – Start with X0 and find all connected components using the iterative procedure Xk = (Xk−1 ⊕ B) ∩ A k = 1, 2, 3,... – Procedure terminates when Xk = Xk−1; Xk contains all the connected components in the input image – The only difference from the hole-filling algorithm is the intersection with A instead of Ac – Figure 9.17 – Example: Figure 9.18 Morphological Image Processing 6 ∗ X-ray image of chicken breast with bone fragments ∗ Objects of “significant size” can be selected by applying erosion to the thresholded image ∗ We may apply labels to the extracted components (region labeling) • Convex hull – Convex set A ∗ Straight line segment joining any two points in A lies entirely within A – Convex hull H of an arbitrary set of points S is the smallest convex set containing S – Set difference H − S is called the convex deficiency of S – Algorithm to compute convex hull C(A) of a set A ∗ Figure 9.19 ∗ Let Bi, i = 1, 2, 3, 4 represent the four structuring elements in the figure · Bi is a clockwise rotation of Bi−1 by 90◦ ∗ Implement the equation i i Xk = (Xk−1 B ) ∪ A i = 1, 2, 3, 4 and k = 1, 2, 3,..