IMAGE FILTERING WITH OPERATORS

DR. – ING. GABRIELE CAVALLARO, JUELICH SUPERCOMPUTING CENTRE (JSC) DEPUTY HEAD - HIGH PRODUCTIVITY DATA PROCESSING RESEARCH GROUP 18.05.2020, IMAGE PROCESSING FOR REMOTE SENSING (INVITED LECTURE), TECHNICAL UNIVERSITY OF BERLIN LECTURER Gabriele Cavallaro

. Since 2013 working in the field of remote sensing and image processing . Holds PhD in Electrical and Computer Engineering (University of Iceland - 2016) − BS and MS Degrees in Telecommunications Engineering (University of Trento – 2011,2013) . Postdoctoral researcher at Forschungszentrum Juelich - Juelich Supercomputing Centre (since 2016) − Deputy Head of the High Productivity Data Processing group − The group focuses on application-driven parallel and scalable machine learning methods that exploit innovative high performance and distributed computing technologies . Research interests − Processing and analysis of high dimensional remote sensing data − Parallel and scalable machine (deep) learning with HPC . Contact − Email: [email protected] − Website: https://www.gabriele-cavallaro.com/

Page 2 FORSCHUNGSZENTRUM JUELICH (FZJ) Multi-Disciplinary Research Centre of the Helmholtz Association in Germany

(Juelich Supercomputing Centre - JSC)

[1] Holmholtz Association Facility on 2.2 . Selected Facts 2 𝐾𝐾𝑚𝑚 − One of EU’s largest inter-disciplinary research centres (~6000 employees) − Special expertise in physics, materials science, nanotechnology, neuroscience and medicine & information technology (HPC & Data) Page 3 JUELICH SUPERCOMPUTING CENTRE (JSC) Evolution of Supercomputers and Research activities

. JSC is part of the Institute for Advance Simulation (IAS) − Operate supercomputers (->TOP500) and cloud systems − Unique support and research environment . R&D work − Algorithms, computational science, performance tools − Scientific big data analytics and data management − Exascale labs (e.g., NVIDIA) and Co-design projects (DEEP) . With General purpose processors . With Accelerators (e.g., GPGPUs ) . Broad range of capabilities . Hundreds of (weak) computing cores . High single thread performance . Specific programming models (CUDA) . High memory per core . Very energy efficient . Standard programming (MPI,OpenMP) . Limited energy efficiency

[2] The DEEP projects

Page 4 RECOMMENDED LITERATURE

. P. Soille, Morphological Image Analysis: Principles and Applications, Second edition, Springer-Verlag Berlin, 2004, ISBN-10: 3540429883, ISBN-13: 978-3540429883.

Page 5 OUTLINE

. Remote Sensing Image Enhancement

. Introduction to Mathematical Morphology

. Filtering with Morphology Operators

− Neighborhood operator,

. Basic Morphological Operators

− , , and

. Geodesic Transformation

− Morphological Reconstruction

. Morphological Profiles

Page 6 HOW TO EXTRACT SPATIAL FEATURES? Very high spatial resolution remote sensing images: large amount of details

. Intuition from Mathematical Morphology − “Recognition of an object simply means that all the rest has been eliminated from the scene.”

[3] J. Serra Page 7 MORPHOLOGY IS A GENERAL CONCEPT From the Greek and meaning "study of shape", may refer to many tasks

. In archaeology: study of the shapes or forms of artifacts

. In astronomy: study of the shape of astronomical objects such as nebulae, galaxies, or other extended objects

. In biology: the study of the form or shape of an organism or part thereof

. In folkloristics: the structure of narratives such as folk tales

. In linguistics: the study of the structure and content of word forms

. Mathematical morphology: a theoretical model based on lattice theory, used for digital image processing

. River morphology: the field of science dealing with changes of river platform

. Urban morphology: study of the form, structure, formation and transformation of human settlements

. Geomorphology: the study of landforms . In architecture and engineering: research which is based on theories of two-dimensional and three- dimensional symmetries, and then uses these geometries for planning buildings and structures Page 8 [4] Wikipedia, Morphology MATHEMATICAL MORPHOLOGY PIONEERS . Originates from the study of the geometry of porous media in the mid-sixties in

. Porous media are binary in the sense that a point of porous medium: − Either belongs to a pore − Or to the matrix surrounding the pores

[5] Fundamentals of Fluid Flow in Porous Media

. This led and Jean Serra to introduce in 1967 a set formalism for analyzing binary images

. [3] J. Serra , Image Analysis and Mathematical Morphology. Volume 2: Theoretical Advances, J. Serra, Ed. London, U.K.: Academic, 1988 . [6] G. Matheron, Eléments Pour une Théorie des Milieux Poreux. Paris, France: Masson, 1967. . [7] G. Matheron, Random Sets and Integral Geometry. New York: Wiley, 1975. . [8] J. Serra, Image Analysis and Mathematical Morphology. New York: Academic, 1982. . [9] J. Serra, The birth of Mathematical Morphology, in H. Talbot & R. Beare (Eds.), Mathematical Morphology and its Applications to Signal and Image Processing (pp. 1–16), Sydney: CSIRO, 2002. . [10] P. Soille, Morphological Image Analysis: Principles and Applications, Second edition, Springer-Verlag Berlin, 2004. . [11] L. Najman and Hugues Talbot (editors), Mathematical Morphology, John Wiley & Sons, Inc., London, 2010. Georges Matheron Jean Serra . Course on mathematical morphology by Jean Serra: *1930, +2000 *1940 . http://www.cmm.mines-paristech.fr/~serra/cours/ Page 9 MATHEMATICAL MORPHOLOGY A theory for the analysis of planar and spatial structures

. Morphology: aims at analyzing the shape and form of objects . Mathematical: analysis is based on: - Set of theory - Integral geometry - Lattice algebra - Successful due to a simple mathematical formalism, which opens a path to powerful image analysis tools

. Search strategy: apply a sequence of morphological operators . Delete all undesirable spatial structures . Until the searched objects are detected

. How: compare local structures with a reference shape (probe) called the Structuring Element (SE)

Page 10 MATHEMATICAL MORPHOLOGY For signal/image processing

. Mathematical Morphology Operators . Nonlinear signal and image processing approach based on minimum and maximum operations . Produce an irreversible, though controlled, loss of information . Do not blur the edges as convolution does

[12] J. Serra, Five decades of Images Analysis and Mathematical Morphology

[13] J. Serra, Courses on Mathematical Morphology: Basics Notions

Page 11 MATHEMATICAL MORPHOLOGY It can be applied to a finite set if

1. We can partially order its elements𝑃𝑃 (where the ordering is denoted by " ") − , , , = , I.e., for all : , , ≤ 2. And, each non𝑎𝑎 𝑏𝑏-empty𝑐𝑐 ∈ 𝑃𝑃 subset𝑎𝑎 ≤ 𝑎𝑎 of𝑃𝑃 𝑎𝑎has≤ 𝑏𝑏 a𝑏𝑏 maximum≤ 𝑎𝑎 ⟹ 𝑎𝑎 and𝑏𝑏 minimum𝑃𝑃 𝑎𝑎 ≤ 𝑏𝑏 𝑏𝑏 ≤ 𝑐𝑐 ⟹ 𝑎𝑎 ≤ 𝑐𝑐 𝑃𝑃 . Example: any finite set of real or integer numbers is a suitable set − " “ 3 4, 4 18 The ordering is defined as in ordinary calculus ( ,etc.)𝑃𝑃 − max 5,3,4 = 5 The maximum and≤ minimum are also defined in the usual≤ sense≤ (e.g., )

. Example: the set of all subsets of a superset is a suitable set − " “ " “ The ordering is defined by the subset relation𝑆𝑆 𝑃𝑃 − The maximum is≤ defined by the union operator ⊂ max , = p , = p o ∪ − 1 2 1 2 2 The minimum𝑝𝑝 𝑝𝑝 is∪defined𝑝𝑝 by the intersection operator [14] W. Burger and M. J. Burge , = p , = p Page 12 o ∩ , . . , 𝑚𝑚𝑚𝑚𝑚𝑚 𝑝𝑝1 𝑝𝑝2 ∩ 1 𝑝𝑝2 1 𝑝𝑝1 ⊂ 𝑝𝑝2 𝑖𝑖 𝑒𝑒 𝑝𝑝1 ≤ 𝑝𝑝2 MATHEMATICAL MORPHOLOGY For image processing

. Mathematical Morphology can be applied to grey-scale images (or subsets of images), because − The collection of grey values can be viewed as a finite set − With ordering, maximum, and minimum well defined 𝑃𝑃

. Partially ordering is extended to digital images by applying the rules of individual pixels

. For two images and , the relation holds if − : ( ) 𝑓𝑓 𝑔𝑔 𝑓𝑓 ≤ 𝑔𝑔 − 𝑓𝑓Where≤ 𝑔𝑔 ⟺ ∀refers𝑥𝑥 𝑓𝑓 𝑥𝑥to ≤all𝑔𝑔 possible𝑥𝑥 locations ∀𝑥𝑥 . The “maximum image” and “minimum image” of two images can also be defined on a pixelwise basis − max , = max{ , } − (min{ , } ) = min{ , } 𝑓𝑓 𝑔𝑔 𝑥𝑥 𝑓𝑓 𝑥𝑥 𝑔𝑔 𝑥𝑥 Page 13 𝑓𝑓 𝑔𝑔 𝑥𝑥 𝑓𝑓 𝑥𝑥 𝑔𝑔 𝑥𝑥 MATHEMATICAL MORPHOLOGY OPERATORS For remote sensing image processing

. Image filtering − For noise reduction, edge enhancement and extraction/suppression of structures − The selection depends on shape, orientation, and size criteria

. Image segmentation: − For delineating the boundary of objects in gray-scale images − For separating connected objects in binary images

. Image measurements: − Compute values for the whole image (or each segment of a segmentation step or even sub-windows of fixed size) − In order to characterize the texture, fragmentation, shape, orientation, or size of the image structures − The output measurements are then used for classification purposes

Page 14 MATHEMATICAL MORPHOLOGICAL FILTERS Noise filtering and selective objects removal

. Restore images corrupted by some type of noise − Interpretation techniques and measurements are strongly hampered by the presence of noisy data

o E.g., remote sensing data − A noisy image must therefore be filtered prior to any further processing

o Such as edge detection, segmentation, and grey scale measurements − There exist also many linear filters for filtering noisy images

o But they usually fail to preserve sharp edges

[15] The Earth-Atmosphere Energy Balance . Selectively remove image structures or objects while preserving the other ones − The selection is based on the geometry and local contrast of the objects − Morphological filters can be considered as a first step towards the interpretation of the image

Page 15 EXAMPLES Noise filtering and selective objects removal

. Task: Detection of the boundaries of the cross − Can be easily achieved if the noise level is reduced beforehand

Cross corrupted by a Gaussian noise

. Task: Extraction of the elevation contour lines − All the objects that do not correspond to contour lines need to be filtered out

Subset of a topographic map [12] P. Soille

Page 16 IMAGE FILTERING Types of Operation . Point image operators − Modify the value of a pixel independently of the values of its neighboring pixels − E.g., Compute the negative view of an image

o I.e., An image having lights and shades reversed

. Neighborhood image operator . Obtain the output value of a given pixel by combining the values which lie within its neighborhood . Mathematica Morphology: look for objects defined as a specific spatial arrangement of pixels rather than single pixels with a specific spectral signature

Multivariate Mathematical Morphology

Page 17 STRUCTURING ELEMENT (SE) – 1 Pattern specified as the coordinates of a number of discrete points relative to some origin

. Structuring Element (SE) . Small set used to analyze locally the image . It serves as a local probe in morphological operators . Its interaction with the image reveals different features of the objects

. Shape and size of SE . A prior knowledge about the geometry of relevant/irrelevant structures . Usually symmetrical, connected, and convex (but not always) . SE that fit into a 3x3 grid with its origin at the center are the most commonly seen type

. Each point of the SE may have a value − Operations on binary images: the elements of the SE only have one value, conveniently represented as a one − Operations on grayscale images: the elements of the SE may have other pixel values

Page 18 STRUCTURING ELEMENT (SE) - 2 Pattern specified as the coordinates of a number of discrete points relative to some origin

. Although a rectangular grid is used to represent the SE, not every point in that grid is part of the SE − The elements that are not part of the SE can be represented as blanks or zero values

[16] Structuring Elements

. A morphological operator is therefore defined by its SE and the applied set operator − The origin of the SE is typically translated to each pixel position in the image in turn − Then the points within the translated SE are compared with the underlying image pixel values − The details of this comparison, and the effect of the outcome depend on which morphological operator is being used

Page 19 BASIC OPERATORS OF MATHEMATICAL MORPHOLOGY Erosion and Dilation

. The two most basic operations in mathematical morphology are: . Erosion and Dilation . All other operators can be defined as their combinations along with set operators (intersection and union)

. Both operators take two input data: − Image to be eroded or dilated − Structuring Element (SE)

. Each input data is treated as representing sets of coordinates − In a way that is slightly different for binary and grayscale images

. Translate the SE to various points in the image − Examine the intersection between the translated SE coordinates and the image coordinates Page 20 MORPHOLOGICAL DILATION For Gray-Scale Images

. Digital image functions − , : input image − b , : 𝑓𝑓 𝑥𝑥 𝑦𝑦 Structuring Element − b , and𝑥𝑥 𝑦𝑦 assign a gray-level value to each distinct pair of coordinates 𝑓𝑓 𝑥𝑥 𝑦𝑦 . Dilation: replace every pixel by the maximum value computed over the neighborhood defined by the SE . Gray-scale dilation of by b, denoted by b, is define as

𝑓𝑓 𝑓𝑓 ⨁ b s, t = max { , + ( , )| , ; ( , ) }

𝑓𝑓 ⨁ 𝑓𝑓 𝑥𝑥 𝑦𝑦 𝑏𝑏 𝑠𝑠 − 𝑥𝑥 𝑡𝑡 − 𝑦𝑦 𝑠𝑠 − 𝑥𝑥 𝑡𝑡 − 𝑦𝑦 ∈ 𝐷𝐷𝑓𝑓 𝑥𝑥 𝑦𝑦 ∈ 𝐷𝐷𝑏𝑏 . is the function that slides past and overlap

𝑓𝑓 𝑏𝑏 [17] R. C. Gonzales et al. . 𝑏𝑏 and D : domains of and b 𝑓𝑓

𝐷𝐷𝑓𝑓 b 𝑓𝑓 . Similar to 2-D convolution (max operation -> sums of convolutions and adding -> products of convolution Page 21 EXAMPLE Dilation on image

. E.g., Look at the group 60, 90, 120 . Place SE with the center on 90 . Calculate the sum corresponding with each SE and each item of the group . 60+10=70 , 90+10=100 , 120+10=130 . Take the max, which is 130 . This is done for each element (while moving the SE around and repeat the process)

𝑏𝑏

Page 22 [18] Dilation - gray-scale image 𝑓𝑓 GENERAL EFFECTS OF DILATION Example on remote sensing image

. If all the SE values are positive, the output image tends to be brighter than the input . Dark details are either reduced or eliminated − Depending on how their values and shapes relate to the SE

. Example − Features that are brighter than their immediate surroundings are enlarged − Features that are darker than their immediate surroundings are shrinked ( ) ( )

𝛿𝛿5 𝑓𝑓 𝛿𝛿15 𝑓𝑓

Effect driven by the size and shape of the SE

SE: disk of radius = 5 SE: disk of radius = 15 Page 23 MORPHOLOGICAL EROSION For Gray-Scale Images

. Erosion: replace every pixel by the minimum value computed over the neighborhood defined by the SE

. Gray-scale erosion of by b, denoted by b, is define as

𝑓𝑓 𝑓𝑓 ⊖ b s, t = min { , ( + , + )| + , + ; ( , ) }

𝑓𝑓 ⊖ 𝑓𝑓 𝑥𝑥 𝑦𝑦 − 𝑏𝑏 𝑠𝑠 𝑥𝑥 𝑡𝑡 𝑦𝑦 𝑠𝑠 𝑥𝑥 𝑡𝑡 𝑦𝑦 ∈ 𝐷𝐷𝑓𝑓 𝑥𝑥 𝑦𝑦 ∈ 𝐷𝐷𝑏𝑏 . is the function that slides past

. 𝑏𝑏 and D : domains of and b 𝑓𝑓

𝐷𝐷𝑓𝑓 b 𝑓𝑓 . Similar to 2-D correlation (min operation -> sums of correlations and subtraction -> products of correlation)

[17] R. C. Gonzales et al.

Page 24 EXAMPLE Erosion on image

. E.g., Look at the group 100,50,40,30,50 . Place SE with the center on 40 . Calculate the subtraction corresponding with each SE and each item of the group . 100-10=90 , 50-10=40 , 40-10=30 , 30-10=20 , 50-10=40 . Take the min, which is 20 . This is done for each element (while moving the SE around and repeat the process)

𝑏𝑏

Page 25 𝑓𝑓 [19] Erosion - gray-scale image GENERAL EFFECTS OF EROSION Example on remote sensing image . If all the SE values are positive, the output image tends to be darker than the input . The effect of bright details in the input image that are smaller in area than the SE is reduced − With the degree of reduction being determined by . The gray level values surrounding the bright detail . By the shape and amplitude values of the SE itself

. Example − Features that are darker than their immediate surroundings are enlarged − Features that are brighter than their immediate surroundings are shrinked ( ) ( )

𝜀𝜀5 𝑓𝑓 𝜀𝜀15 𝑓𝑓

Effect driven by the size and shape of the SE

Page 26 SE: disk of radius = 5 SE: disk of radius = 15 MORPHOLOGICAL OPENING Strategy

. Once an image is eroded, there exists in general no inverse transformation to get the original image back − Erosion removes all structures that cannot contain the SE but it also shrinks all the other ones

. The search for an operator recovering most structures lost by the erosion leads to the definition of the morphological opening operator

. Idea: dilate the image previously eroded using the same SE

. Not all structures are recovered − E.g., objects completely destroyed by the erosion are not recovered at all

Page 27 MORPHOLOGICAL OPENING For gray-scale images

. Denoted , is the erosion of by followed by a dilation of the result by

𝑓𝑓 ∘ 𝑏𝑏 𝑓𝑓 𝑏𝑏 = ( ) 𝑏𝑏

. Geometric interpretation: given a ( , ) in𝑓𝑓 3∘-𝑏𝑏D prospective𝑓𝑓 ⊖ 𝑏𝑏 ⨁ 𝑏𝑏 − Suppose to open by a spherical SE𝑓𝑓 𝑥𝑥 𝑦𝑦(view as a “rolling ball”) − Then is the process𝑓𝑓 of pushing the𝑏𝑏 ball against𝑏𝑏 the underside of the surface o While𝑓𝑓 ∘ at𝑏𝑏 the same time rolling it so that the entire underside of the surface is traversed . Result: surface of the highest points reached by the sphere as it slides over the entire undersurface of

𝑓𝑓 Gray-scale scan line

[16] R. C. Gonzales et al.

Positions of rolling ball All the peaks that are narrow with respect of the diameter of the ball are reduced in amplitude and sharpness Result of opening Page 28 GENERAL EFFECTS OF OPENING Example on remote sensing image . Opening operations usually are applied to remove small light details − While leaving the overall gray levels and larger bright features relatively undisturbed 1. The initial erosion removes the small details, but it also darkens the image 2. The dilation again increases the overall image intensity without reintroducing the details removed by the erosion

( ) ( ) 𝛿𝛿5 𝑓𝑓 5 𝜀𝜀 𝑓𝑓 . Features that are brighter than their immediate surroundings and smaller than the SE disappear . Other features (dark, or bright and ( ) ( ) large) remain “unchanged”

𝜀𝜀15 𝑓𝑓 𝛿𝛿15 𝑓𝑓

( ) Page 29 ( ) 𝛾𝛾5 𝑓𝑓 𝛾𝛾15 𝑓𝑓 MORPHOLOGICAL CLOSING For Gray-Scale Images

. Idea: build an operator tending to recover the initial shape of the structures that have been dilated

. Denoted , is the dilation of by b followed by a erosion of the result by b

𝑓𝑓 𝑏𝑏 𝑓𝑓 = ( ) b . ⋅ The ball slides on top of the surface, and peaks𝑓𝑓 𝑏𝑏 essentially𝑓𝑓 ⨁ 𝑏𝑏 are⊖ left on their original form (assuming that their separation at the narrowest point exceeds the⋅ diameter of the ball)

Gray-scale scan line

Positions of rolling ball [16] R. C. Gonzales et al.

Positions of closing Page 30 GENERAL EFFECTS OF CLOSING Example on remote sensing image

. Generally used to remove dark details, while leaving bright features relatively undisturbed − 1. The initial dilation removes the dark details and brightens the image − 2. The erosion darkens the image without reintroducing the details removed by dilation

( ) ( )

𝛿𝛿5 𝑓𝑓 𝜀𝜀5 𝑓𝑓

. Features that are darker than their immediate surroundings and smaller than the SE disappear . ( ) ( ) Other features (bright, or dark and large) remain “unchanged” 𝛿𝛿15 𝑓𝑓 𝜀𝜀15 𝑓𝑓

( ) Page 31 ( ) 𝜙𝜙5 𝑓𝑓 𝜙𝜙15 𝑓𝑓 GEODESIC TRANSFORMATIONS

. Dilation, Erosion, Opening and Closing involve the combination of an input image with a specific SE − Their interaction reveals different structural features of the objects within the image − However they distort spatial features (e.g., borders, shapes)

. Geodesic transformations require two input images: marker and mask image . Apply a transformation on the marker which is then forced to remain either above or below the mask

. E.g., Geodesic dilation of a binary image within a geodesic mask . Dilate the marker by the isotropic SE𝑌𝑌 [12] P. Soille . Intersect it with the𝑋𝑋 geodesic mask

= . It requires any value in the image to be the minimum of the result and mask value 𝑋𝑋 . It limits / shapes the dilation 𝛿𝛿1 𝑌𝑌 𝛿𝛿1 𝑌𝑌 ⋂𝑋𝑋 . The dilation is never allowed to grow outside of the Page 32 bounds of the masked regions MORPHOLOGICAL RECONSTRUCTION

. Morphological reconstruction: iteration of geodesic erosion and dilation operations until convergence − Convergence is reached when nothing changes between iterations

. Geodesic transformations of bounded images always converge after a finite number of iterations − I.e., until the propagation or shrinking of the marker is totally impeded by the mask

. The image under study is usually used as mask image . A suitable marker image is then determined using: − Some transformations of the mask image itself (most utilized in practice) − Knowledge about the expected result − Known facts about the image or the physics of the object it represents − Other image data if available (i.e., multispectral and multitemporal images) − Interaction with the user (i.e., markers are manually defined)

o One or usually a combination of these approaches is considered Page 33 OPENING BY RECONSTRUCTION Definition

. 1. Erode the input image . 2. Iteratively dilate the result image while maintaining each pixel dilation result is less than the original image

. After the erosion, only objects larger than the SE remain − These objects act as seeds for rounds of dilation to reconstruct the original shapes that passed the erosion

MASK MARKER = [ ] 𝑛𝑛 𝛿𝛿 𝑅𝑅 𝑓𝑓 𝑛𝑛 ( ) ( ) ( ) ( ) ( )𝛾𝛾 𝑓𝑓 𝑅𝑅 𝜀𝜀 𝑓𝑓 . . = = . . … ( . ) 𝛿𝛿 𝑖𝑖 1 2 3 𝑖𝑖 𝑓𝑓 𝑓𝑓 𝑓𝑓 𝑓𝑓 𝑓𝑓 𝑓𝑓 𝑅𝑅 𝛿𝛿 𝛿𝛿 𝛿𝛿 𝛿𝛿 ( times)𝛿𝛿 ( ) . With Idempotence𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼property𝐼𝐼𝐼𝐼 𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 𝑖𝑖 . = ( . ) 𝑖𝑖 𝑖𝑖−1 Page 34 𝛿𝛿𝑓𝑓 𝛿𝛿𝑓𝑓 OPENING BY RECONSTRUCTION Example with binary image

Input Image ( ) marker 5 Erosion𝜀𝜀 with𝑓𝑓 SE Disk, diameter 5 = mask

The dilation is never allowed to grow outside of ( times) the bounds of the masked regions 𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼 𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 𝑖𝑖

…

( ) ( ) ( ) ( ) ( )

1 2 3 4 27 𝛿𝛿𝑓𝑓 𝛿𝛿𝑓𝑓 𝛿𝛿𝑓𝑓 𝛿𝛿𝑓𝑓 𝛿𝛿𝑓𝑓 Page 35 OPENING BY RECONSTRUCTION For Gray-Scale Images

. For grey-scale images, when the mask is the original image . The different intensity value of the mask not only constrains the shape but also the intensity of a region

( ) ( ) Original image: 𝜀𝜀15 𝑓𝑓 𝛿𝛿15 𝑓𝑓 ( ) 𝑅𝑅 ( 𝑓𝑓 ) 𝑖𝑖𝑖𝑖𝑔𝑔1 ≔ 𝜀𝜀min15 (𝑓𝑓 , ) 𝑖𝑖𝑖𝑖𝑔𝑔2 ≔ 𝛿𝛿15 𝑖𝑖𝑖𝑖𝑔𝑔1 𝑖𝑖𝑖𝑖𝑔𝑔3 ≔ 𝑖𝑖𝑖𝑖𝑔𝑔2 𝑓𝑓 𝑖𝑖𝑖𝑖𝑔𝑔1 ≔ 𝑖𝑖𝑖𝑖𝑔𝑔3 ( ) 15 𝛾𝛾𝑅𝑅 𝑓𝑓

Page 36 CLOSING BY RECONSTRUCTION Definition

. 1. Dilate the input image . 2. Iteratively erode the result image while maintaining each pixel erosion result is bigger than the original image

. After the dilation, only objects larger than the SE remain − These objects act as seeds for rounds of erosion to reconstruct the original shapes that passed the dilation

MASK MARKER = [ ] 𝑛𝑛 𝜀𝜀 𝜙𝜙𝑅𝑅 𝑓𝑓 𝑛𝑛 ( ) ( ) ( ) ( ) ( ) 𝑓𝑓 𝑅𝑅 𝛿𝛿 𝑓𝑓 . . = = . . … ( . ) 𝜀𝜀 𝑖𝑖 1 2 3 𝑖𝑖 𝑅𝑅𝑓𝑓 𝜀𝜀𝑓𝑓 𝜀𝜀𝑓𝑓 𝜀𝜀𝑓𝑓 𝜀𝜀𝑓𝑓 𝜀𝜀𝑓𝑓 ( times) ( ) . With Idempotence𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼property𝐼𝐼𝐼𝐼 𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 𝑖𝑖 . = ( . ) 𝑖𝑖 𝑖𝑖−1 𝜀𝜀𝑓𝑓 𝜀𝜀𝑓𝑓 Page 37 FILTERING REMOTE SENSING IMAGES With Very High Spatial Resolution

. It is difficult to identify a single filter parameter suitable to handle all the objects in the image

− Heterogeneous characteristics of the spatial information − Multi-level analysis is required in order to perform a complete modeling

. Morphological Profile (MP) − Multi-scale decomposition by using several values for the filter parameters − Build a stack of images with different degrees of filtering

WorldView-2 Panchromatic Resolution 0.46 [m]

[20] M. Pesaresi et al.

Page 38 MORPHOLOGICAL PROFILE Multi-scale decomposition of the image into a stack of filtered images

. Sequence of opening and closing by reconstruction filters

. Segmentation variables: (1) Number of opening/closing operations (2) Step size

INPUT IMAGE 𝜆𝜆3 𝜆𝜆2 𝜆𝜆1 𝜆𝜆1 𝜆𝜆2 𝜆𝜆3 𝜙𝜙𝑅𝑅 𝑓𝑓 𝜙𝜙𝑅𝑅 𝑓𝑓 𝜙𝜙𝑅𝑅 𝑓𝑓 𝛾𝛾𝑅𝑅 𝑓𝑓 𝛾𝛾𝑅𝑅 𝑓𝑓 𝛾𝛾𝑅𝑅 𝑓𝑓

= 15 = 9 = 3 = 0 = 3 = 9 = 15

𝜆𝜆3 𝜆𝜆2 𝜆𝜆1 𝜆𝜆0 𝜆𝜆1 𝜆𝜆2 𝜆𝜆3 . Morphological Profile: = { , … , , , , … , } 𝜆𝜆𝑛𝑛 𝜆𝜆1 𝜆𝜆1 𝜆𝜆𝑛𝑛 − = [ Closing by reconstruction𝑀𝑀 𝑀𝑀 𝑓𝑓 𝜙𝜙𝑅𝑅 𝑓𝑓 ] 𝜙𝜙𝑅𝑅 𝑓𝑓 𝑓𝑓 𝛾𝛾𝑅𝑅 𝑓𝑓 𝛾𝛾𝑅𝑅 𝑓𝑓 𝑛𝑛 𝜀𝜀 − Opening by reconstruction𝜙𝜙 𝑅𝑅 𝑓𝑓 =𝑅𝑅𝑓𝑓 𝛿𝛿[ 𝑛𝑛 𝑓𝑓 ] Page 39 𝑛𝑛 𝛿𝛿 𝛾𝛾𝑅𝑅 𝑓𝑓 𝑅𝑅𝑓𝑓 𝜀𝜀𝑛𝑛 𝑓𝑓 MORPHOLOGICAL RECONSTRUCTION All reconstruction types are connected operators . Process an image by only merging its flat zones (i.e., connected components) . Either completely remove or entirely preserve a connected component . Partition induced by the flat zones of a reconstructed image is always coarser than that of the input mask image

CLOSING BY RECON. INPUT IMAGE OPENING BY RECON.

Preserves the shape of the objects that are not removed by the erosion

= [ ] = [ ] 𝑛𝑛 𝜀𝜀 𝑛𝑛 𝛿𝛿 . The geometrical characteristics𝜙𝜙𝑅𝑅 𝑓𝑓 𝑅𝑅𝑓𝑓 𝛿𝛿 𝑛𝑛(e.g.,𝑓𝑓 shape, edges) of𝑓𝑓 the structures is not𝛾𝛾𝑅𝑅 distorted𝑓𝑓 𝑅𝑅𝑓𝑓 𝜀𝜀𝑛𝑛 𝑓𝑓 . However, these operators are shape-biased: restricted to SE (such as lines, rectangles and disk) Page 40 REFERENCES (1)

. [1] Helmholtz Association Web Page Online: https://www.helmholtz.de/en

. [2] Deep Projects Online: http://www.deep-projects.eu/

. [3] J. Serra , Image Analysis and Mathematical Morphology. Volume 2: Theoretical Advances, J. Serra, Ed. London, U.K.: Academic, 1988

. [4] Wikipedia, Morphology Online: https://en.wikipedia.org/wiki/Morphology

. [5] Fundamentals of Fluid Flow in Porous Media Online: http://perminc.com/resources/fundamentals-of-fluid-flow-in-porous-media/chapter-2-the-porous-medium/porosity/

. [6] G. Matheron, Eléments Pour une Théorie des Milieux Poreux. Paris, France: Masson, 1967.

. [7] G. Matheron, Random Sets and Integral Geometry. New York: Wiley, 1975.

. [8] J. Serra, Image Analysis and Mathematical Morphology. New York: Academic, 1982.

. [9] J. Serra, The birth of Mathematical Morphology, in H. Talbot & R. Beare (Eds.), Mathematical Morphology and its Applications to Signal and Image Processing (pp. 1–16), Sydney: CSIRO, 2002.

. [10] P. Soille, Morphological Image Analysis: Principles and Applications, Second edition, Springer-Verlag Berlin, 2004.

Page 41 REFERENCES (2)

. [11] L. Najman and Hugues Talbot (editors), Mathematical Morphology, John Wiley & Sons, Inc., London, 2010.

. [12] J. Serra, Five decades of Images Analysis and Mathematical Morphology Online: https://www.isibang.ac.in/~cwjs70/ISI_Serra_conf.pdf

. [13] J. Serra, Courses on Mathematical Morphology: Basics Notions Online: http://www.cmm.mines-paristech.fr/~serra/cours/index.htm

. [14] W. Burger and M. J. Burge, Principles of Digital Image Processing, Mathematical Morphology, Springer-Verlag London Online: http://www.cs.uu.nl/docs/vakken/ibv/reader/chapter6.pdf

. [15] The Earth-Atmosphere Energy Balance Online: http://theatmosphere.pbworks.com/w/page/27058542/The%20Earth-Atmosphere%20Energy%20Balance

. [16] Structuring Elements Online: https://homepages.inf.ed.ac.uk/rbf/HIPR2/strctel.htm

. [17] R. C. Gonzales and R. E. Woods, Digital Image Processing (second edition), Prentice Hall Online: http://web.ipac.caltech.edu/staff/fmasci/home/astro_refs/Digital_Image_Processing_2ndEd.pdf

. [18] Dilation - gray-scale image Online: https://www.youtube.com/watch?v=3PmVBakfqAQ

. [19] Erosion - gray-scale image Online: https://www.youtube.com/watch?v=oef00Opehkk

. [20] M. Pesaresi and J. A. Benediktsson, “A new approach for the morphological segmentation of high-resolution satellite imagery," IEEE Transactions on Geoscience and Remote Sensing, vol. 39, no. 2, pp. 309-320, 2001. Page 42