Morphological Image Processing

n Binary dilation and erosion n Set interpretation n Opening, closing, morphological edge detectors n Hit-miss filter n Morphological filters for gray-level images n Cascading dilations and erosions n Rank filters, median filters, majority filters

n Binary images are common l Intermediate abstraction in a gray-scale/color image analysis system • Thresholding/segmentation • Presence/absence of some image property l Text and line graphics, document image processing n Representation of individual pixels as 0 or 1, convention: l foreground, object = 1 (white) l background = 0 (black) n Processing by logical functions is fast and simple n Shift-invariant logical operations on binary images: “morphological” image processing n Morphological image processing has been generalized to gray-level images via level sets

n Assume that digital images f [x,y] and g[x,y] have inﬁnite support x, y ∈ ,−2,−1,0,1,2, × ,−2,−1,0,1,2, [ ] { } { } . . . then, for all integers a and b

f [x, y] Shift- g[x, y] invariant system

f [x − a, y − b] Shift- g[x − a, y − b] invariant system

n Shift-invariance does not imply linearity (or vice versa).

“structuring element” n Neighborhood “window” operator

W f ⎡x, y⎤ = f ⎡x − x′, y − y′⎤ : ⎡x′, y′⎤ ∈Π { ⎣ ⎦} { ⎣ ⎦ ⎣ ⎦ xy }

n Example structuring elements Π xy :

5x5 square cross x x

y y

“structuring element” n Neighborhood “window” operator

(−) W f ⎡x, y⎤ = f ⎡x − x′, y − y′⎤ : ⎡x′, y′⎤ ∈Π rotated by 180 degrees { ⎣ ⎦} { ⎣ ⎦ ⎣ ⎦ xy }

(+) W f ⎡x, y⎤ = f ⎡x + x′, y + y′⎤ : ⎡x′, y′⎤ ∈Π not rotated { ⎣ ⎦} { ⎣ ⎦ ⎣ ⎦ xy }

n Example structuring elements Π xy :

5x5 square cross x x

y y

g ⎡x, y⎤ = OR ⎡W (−) f ⎡x, y⎤ ⎤ := dilate f ,W ⎣ ⎦ ⎣ { ⎣ ⎦}⎦ ( )

y g[x, y] f [x, y] Π xy

g ⎡x, y⎤ = OR ⎡W (−) f ⎡x, y⎤ ⎤ := dilate f ,W ⎣ ⎦ ⎣ { ⎣ ⎦}⎦ ( )

n Expands the size of 1-valued objects n Smoothes object boundaries n Closes holes and gaps

dilation with dilation with Original (701x781) 3x3 structuring element 7x7 structuring element

g ⎡x, y⎤ = AND ⎡W (+) f ⎡x, y⎤ ⎤ := erode f ,W ⎣ ⎦ ⎣ { ⎣ ⎦}⎦ ( )

y g[x, y] f [x, y] Π xy

n attention: Matlab function imerode() does not ﬂip the window (imdilate() does)!

g ⎡x, y⎤ = AND ⎡W (+) f ⎡x, y⎤ ⎤ := erode f ,W ⎣ ⎦ ⎣ { ⎣ ⎦}⎦ ( )

n Shrinks the size of 1-valued objects n Smoothes object boundaries n Removes peninsulas, ﬁngers, and small objects erosion with erosion with Original (701x781) 3x3 structuring element 7x7 structuring element

n Duality: erosion is dilation of the background

dilate f ,W NOT ⎡erode NOT f ,W ⎤ ( ) = ⎣ ( ⎣⎡ ⎦⎤ )⎦ erode f ,W = NOT ⎡dilate NOT ⎡ f ⎤,W ⎤ ( ) ⎣ ( ⎣ ⎦ )⎦

n But: erosion is not the inverse of dilation

f ⎣⎡x, y⎦⎤ ≠ erode(dilate( f ,W ),W ) ≠ dilate erode f ,W ,W ( ( ) )

Original binary image Erosion by 30x30 Erosion by 70x70 Erosion by 96x96 Circles (792x892) structuring element structuring element structuring element

Original binary image Erosion by disk-shaped Erosion by disk-shaped Erosion by disk-shaped Circles (792x892) structuring element structuring element structuring element Diameter=15 Diameter=35 Diameter=48

Original grayscale image Fence thresholded Erosion with 151x151 Fence (1023 x 1173) using Otsu’s method “cross” structuring element

Continuous (x,y). n Set of object pixels Works for discrete [x,y] F ≡ x, y : f x, y = 1 in the same way. {( ) ( ) } n Background: complement of foreground set F c ≡ x, y : f x, y = 0 {( ) ( ) } n Dilation is union of shifted sets (Minkowski set addition) Commutative and associative!

G = shift Π ∪ xy ( not−rotated ) (x,y)∈F

F x

y Π xy

G = shift Π ∪ xy ( not−rotated ) (x,y)∈F

F x

y Π xy G Not commutative! Not associative! intersection of shifted sets (Minkowski set subtraction)

G shift NOT( ) = ∩ xy ( Πrotated ) c (x,y)∈F inﬁnite support! Digital Image Processing: Bernd Girod, Gordon Wetzstein © 2013-2016 Stanford University -- Morphological Image Processing 16 Opening and closing

n Goal: smoothing without size change n Open filter open( f ,W ) = dilate(erode( f ,W ),W ) n Close filter close f ,W = erode dilate f ,W ,W ( ) ( ( ) ) n Open filter and close filter are biased l Open filter removes small 1-regions l Close filter removes small 0-regions l Bias is often desired for enhancement or detection! n Unbiased size-preserving smoothers close − open( f ,W ) = close(open( f ,W ),W ) open − close f ,W = open close f ,W ,W ( ) ( ( ) ) n close-open and open-close are duals, but not inverses of each other.

Original binary mask Dilation 10x10 Closing 10x10 Difference to original mask

f ⎡x, y⎤ dilate f ,W ≠ f erode f ,W ≠ f dilate f ,W ≠ erode f ,W ⎣ ⎦ ( ) ( ) ( ) ( )

open NOT ⎡ f ⎤,W = dilate erode NOT ⎡ f ⎤,W ,W Binary image f ( ⎣ ⎦ ) ( ( ⎣ ⎦ ) ) 1400

2000

Structuring element W 44

open NOT ⎡ f ⎤,W = dilate erode NOT ⎡ f ⎤,W ,W Binary image f ( ⎣ ⎦ ) ( ( ⎣ ⎦ ) ) 1400

2000

Structuring element W 62

dilate erode NOT ⎡ f ⎤,V & erode f ,W ,W Binary image f ( ( ⎣ ⎦ ) ( ) ) 1400

2000

Structuring Structuring 62 element V 62 element W

18 18

Structuring element V 62

Structuring element W 62

n Threshold sets of a gray-level image f [x,y] Τ f ⎡x, y⎤ = ⎡x, y⎤ : f ⎡x, y⎤ ≥ θ , − ∞ < θ < +∞ θ ( ⎣ ⎦) {⎣ ⎦ ⎣ ⎦ } n Reconstruction of original image from threshold sets

f ⎡x, y⎤ = sup θ : ⎡x, y⎤ ∈Τ f ⎡x, y⎤ ⎣ ⎦ { ⎣ ⎦ θ ( ⎣ ⎦)}

n Idea of morphological operators for multi-level (or continuous-amplitude) signals l Decompose into threshold sets l Apply binary morphological operator to each threshold set

l Reconstruct via supremum (maximum) operation l Gray-level operators thus obtained: ﬂat operators ! Flat morphological operators and thresholding are commutative

n Explicit decomposition into threshold sets not required in practice

n Flat dilation operator: local maximum over window W g ⎡x, y⎤ = max W (−) f ⎡x, y⎤ := dilate f ,W ⎣ ⎦ { { ⎣ ⎦}} ( ) n Flat erosion operator: local minimum over window W g ⎡x, y⎤ = min W (+) f ⎡x, y⎤ := erode f ,W ⎣ ⎦ { { ⎣ ⎦}} ( ) n Binary dilation/erosion operators contained as special case

Structuring element length = ——Dilation ——Erosion ——Original

Structuring element: horizontal line

Amplitude

Sample no.

Original Dilation Erosion

Original Diamond Disk 20 degree line

9 points 2 horizontal lines

thresholded dilate after thresholding

Original 20 connected components

dilation thresholded after dilation

Original grayscale image Flat erosion with 151x151 Binarized by Thresholding Fence (1023 x 1173) “cross” structuring element

original f dilation g g − f |gg-f− f| thresholded

n General dilation operator

g ⎣⎡x, y⎦⎤ = sup{ f ⎣⎡x −α, y − β ⎦⎤ + w⎣⎡α,β ⎦⎤} = sup{w⎣⎡x −α, y − β ⎦⎤ + f ⎣⎡α,β ⎦⎤} α ,β α ,β n Like linear convolution, with sup replacing summation, addition replacing multiplication n Dilation with “unit impulse”

⎧0 α = β = 0 d ⎡α,β ⎤ = ⎨ ⎣ ⎦ −∞ else ⎩ does not change input signal:

f ⎣⎡x, y⎦⎤ = sup{ f ⎣⎡x −α, y − β ⎦⎤ + d ⎣⎡α,β ⎦⎤} α ,β

n Find w ⎡ α , β ⎤ such that ⎣ ⎦ f ⎣⎡x, y⎦⎤ = sup{ f ⎣⎡x −α, y − β ⎦⎤ + w⎣⎡α,β ⎦⎤} = dilate( f ,W ) α ,β n Answer: ⎧0 , ⎪ ⎣⎡α β ⎦⎤ ∈Πxy w⎣⎡α,β ⎦⎤ = ⎨ else ⎩⎪−∞

n Hence, write in general

g "!x, y$# = sup f "!x −α, y − β $# + w"!α,β $# α ,β { } dilate f ,w dilate w, f = ( ) = ( )

n General erosion operator

g ⎣⎡x, y⎦⎤ = inf f ⎣⎡x +α, y + β ⎦⎤ − w⎣⎡α,β ⎦⎤ = erode( f ,w) α ,β { } n Dual of dilation

g ⎣⎡x, y⎦⎤ = −dilate(− f ,w)

n Flat erosion contained as a special case

f dilate f ,w dilate f ,w ,w dilate f ,w ,w ,w ( 1 ) ( 1 2 ) ( 1 2 3 )

dilate⎡dilate f ,w ,w ⎤ = dilate f ,w ⎣ ( 1 ) 2 ⎦ ( ) where w = dilate w ,w ( 1 2 )

n Cascaded erosions can be lumped into single erosion

erode⎡erode f ,w ,w ⎤ erode⎡ dilate f ,w ,w ⎤ ⎣ ( 1 ) 2 ⎦ = ⎣− (− 1 ) 2 ⎦ dilate⎡dilate f ,w ,w ⎤ = − ⎣ (− 1 ) 2 ⎦ = −dilate(− f ,w) = erode( f ,w) where w = dilate w ,w ( 1 2 )

n New structuring element (SE) is not the erosion of one SE by the other, but dilation.

n Idea: build larger dilation and erosion operators by cascading simple, small operators n Example: binary erosion by 11x11 window

f x, y f x, y Erosion with … Erosion with [ ] [ ] 1x3 window 1x3 window Erosion with 5 stages 11x11 window Erosion with Erosion with g x, y g x, y … [ ] [ ] 3x1 window 3x1 window

5 stages

120 AND per pixel 2x10 = 20 AND per pixel

n Generalisation of flat dilation/erosion: in lieu of min or max value in window, use the p-th ranked value n Increases robustness against noise n Best-known example: median filter for noise reduction n Concept useful for both gray-level and binary images n All rank filters are commutative with thresholding

n Gray-level median ﬁlter g ⎡x, y⎤ = median⎡W f ⎡x, y⎤ ⎤ := median f ,W ⎣ ⎦ ⎣ { ⎣ ⎦}⎦ ( ) n Binary images: majority ﬁlter g ⎡x, y⎤ = MAJ ⎡W f ⎡x, y⎤ ⎤ := majority f ,W ⎣ ⎦ ⎣ { ⎣ ⎦}⎦ ( ) n Self-duality median( f ,W ) = − ⎣⎡median(− f ,W )⎦⎤ majority f ,W = NOT ⎡majority NOT ⎡ f ⎤,W ⎤ ( ) ⎣ ( ⎣ ⎦ )⎦

Binary image with 3x3 majority ﬁlter 20% ‘Salt&Pepper’ noise 3x3 majority ﬁlter 5% ‘Salt&Pepper’ noise

Original image 5% ‘Salt&Pepper’ noise 3x3 median ﬁltering 7x7 median ﬁltering

Original image Dilation (local max) Rank ﬁlter 1632x1216 pixels 61x61 structuring element 10st brightest pixel 61x61 structuring element

Background – original image After global thresholding