IMAGE FILTERING WITH MATHEMATICAL MORPHOLOGY OPERATORS DR. – ING. GABRIELE CAVALLARO, JUELICH SUPERCOMPUTING CENTRE (JSC) DEPUTY HEAD - HIGH PRODUCTIVITY DATA PROCESSING RESEARCH GROUP 18.05.2020, IMAGE PROCESSING FOR REMOTE SENSING (INVITED LECTURE), TECHNICAL UNIVERSITY OF BERLIN LECTURER Gabriele Cavallaro . Since 2013 working in the field of remote sensing and image processing . Holds PhD in Electrical and Computer Engineering (University of Iceland - 2016) − BS and MS Degrees in Telecommunications Engineering (University of Trento – 2011,2013) . Postdoctoral researcher at Forschungszentrum Juelich - Juelich Supercomputing Centre (since 2016) − Deputy Head of the High Productivity Data Processing group − The group focuses on application-driven parallel and scalable machine learning methods that exploit innovative high performance and distributed computing technologies . Research interests − Processing and analysis of high dimensional remote sensing data − Parallel and scalable machine (deep) learning with HPC . Contact − Email: [email protected] − Website: https://www.gabriele-cavallaro.com/ Page 2 FORSCHUNGSZENTRUM JUELICH (FZJ) Multi-Disciplinary Research Centre of the Helmholtz Association in Germany (Juelich Supercomputing Centre - JSC) [1] Holmholtz Association Facility on 2.2 . Selected Facts 2 − One of EU’s largest inter-disciplinary research centres (~6000 employees) − Special expertise in physics, materials science, nanotechnology, neuroscience and medicine & information technology (HPC & Data) Page 3 JUELICH SUPERCOMPUTING CENTRE (JSC) Evolution of Supercomputers and Research activities . JSC is part of the Institute for Advance Simulation (IAS) − Operate supercomputers (->TOP500) and cloud systems − Unique support and research environment . R&D work − Algorithms, computational science, performance tools − Scientific big data analytics and data management − Exascale labs (e.g., NVIDIA) and Co-design projects (DEEP) . With General purpose processors . With Accelerators (e.g., GPGPUs ) . Broad range of capabilities . Hundreds of (weak) computing cores . High single thread performance . Specific programming models (CUDA) . High memory per core . Very energy efficient . Standard programming (MPI,OpenMP) . Limited energy efficiency [2] The DEEP projects Page 4 RECOMMENDED LITERATURE . P. Soille, Morphological Image Analysis: Principles and Applications, Second edition, Springer-Verlag Berlin, 2004, ISBN-10: 3540429883, ISBN-13: 978-3540429883. Page 5 OUTLINE . Remote Sensing Image Enhancement . Introduction to Mathematical Morphology . Filtering with Morphology Operators − Neighborhood operator, Structuring Element . Basic Morphological Operators − Dilation, Erosion, Opening and Closing . Geodesic Transformation − Morphological Reconstruction . Morphological Profiles Page 6 HOW TO EXTRACT SPATIAL FEATURES? Very high spatial resolution remote sensing images: large amount of details . Intuition from Mathematical Morphology − “Recognition of an object simply means that all the rest has been eliminated from the scene.” [3] J. Serra Page 7 MORPHOLOGY IS A GENERAL CONCEPT From the Greek and meaning "study of shape", may refer to many tasks . In archaeology: study of the shapes or forms of artifacts . In astronomy: study of the shape of astronomical objects such as nebulae, galaxies, or other extended objects . In biology: the study of the form or shape of an organism or part thereof . In folkloristics: the structure of narratives such as folk tales . In linguistics: the study of the structure and content of word forms . Mathematical morphology: a theoretical model based on lattice theory, used for digital image processing . River morphology: the field of science dealing with changes of river platform . Urban morphology: study of the form, structure, formation and transformation of human settlements . Geomorphology: the study of landforms . In architecture and engineering: research which is based on theories of two-dimensional and three- dimensional symmetries, and then uses these geometries for planning buildings and structures Page 8 [4] Wikipedia, Morphology MATHEMATICAL MORPHOLOGY PIONEERS . Originates from the study of the geometry of porous media in the mid-sixties in France . Porous media are binary in the sense that a point of porous medium: − Either belongs to a pore − Or to the matrix surrounding the pores [5] Fundamentals of Fluid Flow in Porous Media . This led Georges Matheron and Jean Serra to introduce in 1967 a set formalism for analyzing binary images . [3] J. Serra , Image Analysis and Mathematical Morphology. Volume 2: Theoretical Advances, J. Serra, Ed. London, U.K.: Academic, 1988 . [6] G. Matheron, Eléments Pour une Théorie des Milieux Poreux. Paris, France: Masson, 1967. [7] G. Matheron, Random Sets and Integral Geometry. New York: Wiley, 1975. [8] J. Serra, Image Analysis and Mathematical Morphology. New York: Academic, 1982. [9] J. Serra, The birth of Mathematical Morphology, in H. Talbot & R. Beare (Eds.), Mathematical Morphology and its Applications to Signal and Image Processing (pp. 1–16), Sydney: CSIRO, 2002. [10] P. Soille, Morphological Image Analysis: Principles and Applications, Second edition, Springer-Verlag Berlin, 2004. [11] L. Najman and Hugues Talbot (editors), Mathematical Morphology, John Wiley & Sons, Inc., London, 2010. Georges Matheron Jean Serra . Course on mathematical morphology by Jean Serra: *1930, +2000 *1940 . http://www.cmm.mines-paristech.fr/~serra/cours/ Page 9 MATHEMATICAL MORPHOLOGY A theory for the analysis of planar and spatial structures . Morphology: aims at analyzing the shape and form of objects . Mathematical: analysis is based on: - Set of theory - Integral geometry - Lattice algebra - Successful due to a simple mathematical formalism, which opens a path to powerful image analysis tools . Search strategy: apply a sequence of morphological operators . Delete all undesirable spatial structures . Until the searched objects are detected . How: compare local structures with a reference shape (probe) called the Structuring Element (SE) Page 10 MATHEMATICAL MORPHOLOGY For signal/image processing . Mathematical Morphology Operators . Nonlinear signal and image processing approach based on minimum and maximum operations . Produce an irreversible, though controlled, loss of information . Do not blur the edges as convolution does [12] J. Serra, Five decades of Images Analysis and Mathematical Morphology [13] J. Serra, Courses on Mathematical Morphology: Basics Notions Page 11 MATHEMATICAL MORPHOLOGY It can be applied to a finite set if 1. We can partially order its elements (where the ordering is denoted by " ") − , , , = , I.e., for all : , , ≤ 2. And, each non -empty ∈ subset ≤ of has≤ a maximum≤ ⟹ and minimum ≤ ≤ ⟹ ≤ . Example: any finite set of real or integer numbers is a suitable set − " “ 3 4, 4 18 The ordering is defined as in ordinary calculus ( ,etc.) − max 5,3,4 = 5 The maximum and≤ minimum are also defined in the usual≤ sense≤ (e.g., ) . Example: the set of all subsets of a superset is a suitable set − " “ " “ The ordering is defined by the subset relation − The maximum is≤ defined by the union operator ⊂ max , = p , = p o ∪ − 1 2 1 2 2 The minimum is∪defined by the intersection operator [14] W. Burger and M. J. Burge , = p , = p Page 12 o ∩ , . , 1 2 ∩ 1 2 1 1 ⊂ 2 1 ≤ 2 MATHEMATICAL MORPHOLOGY For image processing . Mathematical Morphology can be applied to grey-scale images (or subsets of images), because − The collection of grey values can be viewed as a finite set − With ordering, maximum, and minimum well defined . Partially ordering is extended to digital images by applying the rules of individual pixels . For two images and , the relation holds if − : ( ) ≤ − Where≤ ⟺ ∀refers to ≤all possible locations ∀ . The “maximum image” and “minimum image” of two images can also be defined on a pixelwise basis − max , = max{ , } − (min{ , } ) = min{ , } Page 13 MATHEMATICAL MORPHOLOGY OPERATORS For remote sensing image processing . Image filtering − For noise reduction, edge enhancement and extraction/suppression of structures − The selection depends on shape, orientation, and size criteria . Image segmentation: − For delineating the boundary of objects in gray-scale images − For separating connected objects in binary images . Image measurements: − Compute values for the whole image (or each segment of a segmentation step or even sub-windows of fixed size) − In order to characterize the texture, fragmentation, shape, orientation, or size of the image structures − The output measurements are then used for classification purposes Page 14 MATHEMATICAL MORPHOLOGICAL FILTERS Noise filtering and selective objects removal . Restore images corrupted by some type of noise − Interpretation techniques and measurements are strongly hampered by the presence of noisy data o E.g., remote sensing data − A noisy image must therefore be filtered prior to any further processing o Such as edge detection, segmentation, and grey scale measurements − There exist also many linear filters for filtering noisy images o But they usually fail to preserve sharp edges [15] The Earth-Atmosphere Energy Balance . Selectively remove image structures or objects while preserving the other ones − The selection is based on the geometry and local contrast of the objects − Morphological filters can be considered as a first step towards the interpretation of the image Page 15 EXAMPLES Noise filtering and selective objects removal . Task: Detection of