Morphological Image Processing n Binary dilation and erosion n Set interpretation n Opening, closing, morphological edge detectors n Hit-miss filter n Morphological filters for gray-level images n Cascading dilations and erosions n Rank filters, median filters, majority filters Digital Image Processing: Bernd Girod, Gordon Wetzstein © 2013-2016 Stanford University -- Morphological Image Processing 1 Binary image processing n Binary images are common l Intermediate abstraction in a gray-scale/color image analysis system • Thresholding/segmentation • Presence/absence of some image property l Text and line graphics, document image processing n Representation of individual pixels as 0 or 1, convention: l foreground, object = 1 (white) l background = 0 (black) n Processing by logical functions is fast and simple n Shift-invariant logical operations on binary images: “morphological” image processing n Morphological image processing has been generalized to gray-level images via level sets Digital Image Processing: Bernd Girod, Gordon Wetzstein © 2013-2016 Stanford University -- Morphological Image Processing 2 Shift-invariance n Assume that digital images f [x,y] and g[x,y] have infinite support x, y ∈ ,−2,−1,0,1,2, × ,−2,−1,0,1,2, [ ] { } { } . then, for all integers a and b f [x, y] Shift- g[x, y] invariant system f [x − a, y − b] Shift- g[x − a, y − b] invariant system n Shift-invariance does not imply linearity (or vice versa). Digital Image Processing: Bernd Girod, Gordon Wetzstein © 2013-2016 Stanford University -- Morphological Image Processing 3 Structuring element “structuring element” n Neighborhood “window” operator W f ⎡x, y⎤ = f ⎡x − x′, y − y′⎤ : ⎡x′, y′⎤ ∈Πxy { ⎣ ⎦} { ⎣ ⎦ ⎣ ⎦ } n Example structuring elements Π xy : 5x5 square cross x x y y Digital Image Processing: Bernd Girod, Gordon Wetzstein © 2013-2016 Stanford University -- Morphological Image Processing 4 Structuring element “structuring element” n Neighborhood “window” operator (−) W f ⎡x, y⎤ = f ⎡x − x , y − y ⎤ : ⎡x , y ⎤ ∈Π rotated by 180 degrees { ⎣ ⎦} { ⎣ ′ ′⎦ ⎣ ′ ′⎦ xy } (+) not rotated W f ⎡x, y⎤ = f ⎡x + x′, y + y′⎤ : ⎡x′, y′⎤ ∈Πxy { ⎣ ⎦} { ⎣ ⎦ ⎣ ⎦ } n Example structuring elements Π xy : 5x5 square cross x x y y Digital Image Processing: Bernd Girod, Gordon Wetzstein © 2013-2016 Stanford University -- Morphological Image Processing 5 Binary dilation (expanding foreground) g ⎡x, y⎤ = OR ⎡W (−) f ⎡x, y⎤ ⎤ := dilate( f ,W ) ⎣ ⎦ ⎣ { ⎣ ⎦}⎦ x y f [x, y] Π g[x, y] xy Digital Image Processing: Bernd Girod, Gordon Wetzstein © 2013-2016 Stanford University -- Morphological Image Processing 6 Binary dilation with square structuring element (−) g ⎣⎡x, y⎦⎤ = OR ⎡W { f ⎣⎡x, y⎦⎤}⎤ := dilate( f ,W ) ⎣ ⎦ n Expands the size of 1-valued objects n Smoothes object boundaries n Closes holes and gaps dilation with dilation with Original (701x781) 3x3 structuring element 7x7 structuring element Digital Image Processing: Bernd Girod, Gordon Wetzstein © 2013-2016 Stanford University -- Morphological Image Processing 7 Binary erosion (shrinking foreground) g ⎡x, y⎤ = AND ⎡W (+) f ⎡x, y⎤ ⎤ := erode f ,W ⎣ ⎦ ⎣ { ⎣ ⎦}⎦ ( ) x y f [x, y] Π g[x, y] xy n attention: Matlab function imerode() does not flip the window (imdilate() does)! Digital Image Processing: Bernd Girod, Gordon Wetzstein © 2013-2016 Stanford University -- Morphological Image Processing 8 Binary erosion with square structuring element (+) g ⎣⎡x, y⎦⎤ = AND ⎡W { f ⎣⎡x, y⎦⎤}⎤ := erode( f ,W ) ⎣ ⎦ n Shrinks the size of 1-valued objects n Smoothes object boundaries n Removes peninsulas, fingers, and small objects erosion with erosion with Original (701x781) 3x3 structuring element 7x7 structuring element Digital Image Processing: Bernd Girod, Gordon Wetzstein © 2013-2016 Stanford University -- Morphological Image Processing 9 Relationship between dilation and erosion n Duality: erosion is dilation of the background dilate f ,W NOT ⎡erode NOT f ,W ⎤ ( ) = ⎣ ( ⎣⎡ ⎦⎤ )⎦ erode( f ,W ) = NOT ⎡dilate NOT ⎣⎡ f ⎦⎤,W ⎤ ⎣ ( )⎦ n But: erosion is not the inverse of dilation f ⎣⎡x, y⎦⎤ ≠ erode(dilate( f ,W ),W ) ≠ dilate erode( f ,W ),W ( ) Digital Image Processing: Bernd Girod, Gordon Wetzstein © 2013-2016 Stanford University -- Morphological Image Processing 10 Example: blob separation/detection by erosion Original binary image Erosion by 30x30 Erosion by 70x70 Erosion by 96x96 Circles (792x892) structuring element structuring element structuring element Digital Image Processing: Bernd Girod, Gordon Wetzstein © 2013-2016 Stanford University -- Morphological Image Processing 11 Example: blob separation/detection by erosion Original binary image Erosion by disk-shaped Erosion by disk-shaped Erosion by disk-shaped Circles (792x892) structuring element structuring element structuring element Diameter=15 Diameter=35 Diameter=48 Digital Image Processing: Bernd Girod, Gordon Wetzstein © 2013-2016 Stanford University -- Morphological Image Processing 12 Example: chain link fence hole detection Original grayscale image Fence thresholded Erosion with 151x151 Fence (1023 x 1173) using Otsu’s method “cross” structuring element Digital Image Processing: Bernd Girod, Gordon Wetzstein © 2013-2016 Stanford University -- Morphological Image Processing 13 Set-theoretic interpretation Continuous (x,y). n Set of object pixels Works for discrete [x,y] F ≡ x, y : f x, y = 1 in the same way. {( ) ( ) } n Background: complement of foreground set F c ≡ x, y : f x, y = 0 {( ) ( ) } n Dilation is union of shifted sets (Minkowski set addition) Commutative and associative! G = shift Π ∪ xy ( not−rotated ) (x,y)∈F Digital Image Processing: Bernd Girod, Gordon Wetzstein © 2013-2016 Stanford University -- Morphological Image Processing 14 Set-theoretic interpretation: dilation G F x y Π xy G = shift Π ∪ xy ( not−rotated ) (x,y)∈F Digital Image Processing: Bernd Girod, Gordon Wetzstein © 2013-2016 Stanford University -- Morphological Image Processing 15 Set-theoretic interpretation: erosion F x y Π xy G Not commutative! Not associative! intersection of shifted sets (Minkowski set subtraction) G shift NOT( ) = ∩ xy ( Πrotated ) x,y F c ( )∈ infinite support! Digital Image Processing: Bernd Girod, Gordon Wetzstein © 2013-2016 Stanford University -- Morphological Image Processing 16 Opening and closing n Goal: smoothing without size change n Open filter open( f ,W ) = dilate(erode( f ,W ),W ) n Close filter close( f ,W ) = erode dilate( f ,W ),W ( ) n Open filter and close filter are biased l Open filter removes small 1-regions l Close filter removes small 0-regions l Bias is often desired for enhancement or detection! n Unbiased size-preserving smoothers close − open( f ,W ) = close(open( f ,W ),W ) open − close f ,W = open close f ,W ,W ( ) ( ( ) ) n close-open and open-close are duals, but not inverses of each other. Digital Image Processing: Bernd Girod, Gordon Wetzstein © 2013-2016 Stanford University -- Morphological Image Processing 17 Small hole removal by closing Original binary mask Dilation 10x10 Closing 10x10 Difference to original mask Digital Image Processing: Bernd Girod, Gordon Wetzstein © 2013-2016 Stanford University -- Morphological Image Processing 18 Morphological edge detectors f ⎡x, y⎤ dilate(f ,W )≠ f erode(f ,W )≠ f dilate(f ,W)≠ erode(f ,W) ⎣ ⎦ Digital Image Processing: Bernd Girod, Gordon Wetzstein © 2013-2016 Stanford University -- Morphological Image Processing 19 Recognition by erosion Binary image f open(NOT ⎣⎡ f ⎦⎤,W ) = dilate(erode(NOT ⎣⎡ f ⎦⎤,W ),W ) 1400 2000 Structuring element W 44 34 Digital Image Processing: Bernd Girod, Gordon Wetzstein © 2013-2016 Stanford University -- Morphological Image Processing 20 Recognition by erosion Structuring element W 44 34 Digital Image Processing: Bernd Girod, Gordon Wetzstein © 2013-2016 Stanford University -- Morphological Image Processing 21 Recognition by erosion Binary image f open(NOT ⎣⎡ f ⎦⎤,W ) = dilate(erode(NOT ⎣⎡ f ⎦⎤,W ),W ) 1400 2000 Structuring element W 62 18 Digital Image Processing: Bernd Girod, Gordon Wetzstein © 2013-2016 Stanford University -- Morphological Image Processing 22 Recognition by erosion Structuring element W 62 18 Digital Image Processing: Bernd Girod, Gordon Wetzstein © 2013-2016 Stanford University -- Morphological Image Processing 23 Hit-miss filter Binary image f dilate(erode(NOT ⎣⎡ f ⎦⎤,V ) & erode( f ,W ),W ) 1400 2000 Structuring Structuring 62 element V 62 element W 18 18 Digital Image Processing: Bernd Girod, Gordon Wetzstein © 2013-2016 Stanford University -- Morphological Image Processing 24 Hit-miss filter Structuring element V 62 18 Structuring element W 62 18 Digital Image Processing: Bernd Girod, Gordon Wetzstein © 2013-2016 Stanford University -- Morphological Image Processing 25 Morphological filters for gray-level images n Threshold sets of a gray-level image f [x,y] Τ f ⎡x, y⎤ = ⎡x, y⎤ : f ⎡x, y⎤ ≥ θ , − ∞ < θ < +∞ θ ( ⎣ ⎦) {⎣ ⎦ ⎣ ⎦ } n Reconstruction of original image from threshold sets f ⎡x, y⎤ = sup θ : ⎡x, y⎤ ∈Τ f ⎡x, y⎤ ⎣ ⎦ { ⎣ ⎦ θ ( ⎣ ⎦)} n Idea of morphological operators for multi-level (or continuous-amplitude) signals l Decompose into threshold sets l Apply binary morphological operator to each threshold set l Reconstruct via supremum (maximum) operation l Gray-level operators thus obtained: flat operators ! Flat morphological operators and thresholding are commutative Digital Image Processing: Bernd Girod, Gordon Wetzstein © 2013-2016 Stanford University -- Morphological Image Processing 26 Dilation/erosion for gray-level images n Explicit decomposition into threshold sets not required in practice n Flat dilation operator: local maximum over window