ISSN: 1402-1544 ISBN 978-91-7583-XXX-X Se i listan och fyll i siffror där kryssen är DOCTORAL T H E SIS Department of Computer Science, Electrical and Space Engineering Division of Signals and Systems StructuringAdaptive Elements for Mathematical Morphology Anders Landström Elliptical ISSN 1402-1544 Elliptical Adaptive Structuring Elements ISBN 978-91-7583-047-6 (print) ISBN 978-91-7583-048-3 (pdf) for Mathematical Morphology Luleå University of Technology 2014 Anders Landström Elliptical Adaptive Structuring Elements for Mathematical Morphology Anders Landstr¨om Dept. of Computer Science, Electrical and Space Engineering Lule˚aUniversity of Technology Lule˚a,Sweden Supervisors: Matthew J. Thurley and H˚akan Jonsson Printed by Luleå University of Technology, Graphic Production 2014 ISSN 1402-1544 ISBN 978-91-7583-047-6 (print) ISBN 978-91-7583-048-3 (pdf) Luleå 2014 www.ltu.se To my mother and father, for all their support, and Elizabeth, for all her patience. Thank you. iii iv Abstract As technological advances drives the evolution of sensors as well as the systems using them, processing and analysis of multi-dimensional signals such as images becomes more and more common in a wide range of applications ranging from consumer products to automated systems in process industry. Image processing is often needed to enhance or suppress features in the acquired data, enabling better analysis of the signals and thereby better use of the system in question. Since imaging applications can be very different, image processing covers a wide range of methods and sub-fields. Mathematical morphology constitutes a well defined framework for non-linear im- age processing based on set relations. It relies on minimum and maximum values over neighborhoods (i.e. regions surrounding the individual points) defined by shapes or func- tions known as structuring elements. Classical morphological operations use a predefined structuring element which is used repeatedly for each point in the image. This is often not ideal, however, which has motivated the evolution of adaptive morphological filter- ing where the structuring element changes from point to point. The field of adaptive mathematical morphology includes many different concepts with different strengths and weaknesses, and the specific choice of method should be made with the specific applica- tion in mind. The main contribution of this thesis is a novel method for adaptive morphological filtering using Elliptical Adaptive Structuring Elements (EASE). The method enhances directional structures in images by orienting the structuring elements along the existing structure, and can be efficiently used to close gaps in such structures. The method is introduced by summarizing underlying theory as well as presenting a practical application motivating it: crack detection in casted steel. Furthermore, it is demonstrated how the method can be extended to allow for filtering of incomplete (i.e. partially missing) image data without need for pre-filtering. The EASE concept is also put in relation to other related work by presenting a survey of the field of adaptive mathematical morphology. In conclusion, EASE allows for fast structure-based adaptive morphological filtering of images based on solid mathematical theory, successfully enhancing directional structures such as lines, borders, etc. in the data. The method is user-friendly, as it does not require more than a few user-defined parameters, and can also be adapted for direct filtering of incomplete data. v vi Contents Part I 1 Chapter 1 { Introduction 3 1.1 Background . 3 1.2 Contribution . 5 1.3 Related work . 5 1.4 Thesis outline . 7 Chapter 2 { The Application 9 2.1 Cracks in casted steel . 9 2.2 Industrial Machine Vision . 10 2.3 3D profile data . 10 2.4 Photometric stereo . 11 Chapter 3 { Underlying Theory 15 3.1 Digital images . 15 3.2 Mathematical morphology . 16 3.3 The Local Structure Tensor . 26 3.4 Normalized Convolution . 26 3.5 Normalized Differential Convolution . 29 Chapter 4 { Elliptical Adaptive Structuring Elements 31 4.1 The EASE concept . 31 4.2 Required parameters . 34 4.3 Implementation and computational issues . 35 4.4 EASE for incomplete Data . 36 4.5 Strengths and weaknesses . 38 Chapter 5 { Contributions 41 5.1 Paper A . 41 5.2 Paper B . 42 5.3 Paper C . 42 5.4 Paper D . 42 5.5 Paper E . 43 5.6 Paper F . 43 Chapter 6 { Conclusions and Future Work 45 6.1 Conclusions . 45 vii 6.2 Future work . 47 Part II 57 Paper A 59 1 Introduction . 61 2 Measurements . 64 3 Segmentation . 65 4 Classification . 73 5 Results . 77 6 Discussion . 78 7 Conclusion . 80 8 Future work . 81 Paper B 85 1 Introduction . 87 2 Method . 90 3 Implementation . 92 4 Results . 93 5 Discussion . 100 6 Conclusion . 101 7 Future work . 101 Paper C 105 1 Introduction . 107 2 Method . 109 3 Results . 117 4 Discussion . 119 5 Conclusion . 120 6 Future work . 120 Paper D 123 1 Introduction . 125 2 Method . 126 3 Experiments and results . 129 4 Discussion . 132 5 Conclusion . 134 Paper E 137 1 Introduction . 139 2 Theory . 141 3 Method . 143 4 Experiments and results . 145 5 Discussion . 150 viii Paper F 155 1 Introduction . 157 2 Overview of adaptive mathematical morphology . 158 3 Theory . 163 4 Selected methods . 168 5 Experimental results . 171 6 Discussion . 175 7 Perspectives and trends . 176 ix x Acknowledgments First of all, I would like to thank my supervisors Matthew Thurley and H˚akan Jonsson for their support and guidance throughout this work. I am very grateful for the amount of freedom in my research, which have encouraged me to venture into new areas. I would also like to thank all colleagues at the department of Computer Science, Elet- rical and Space Engineering at Lule˚aUniversity of Technology. You have all contributed into making my PhD studies a memorable time, in a very positive sense. An extra thank you goes to everyone at the division of Signals and Systems, and in particular to Roland Hostettler and Martin Simonsson for many good discussions on various topics { sometimes work-related, sometimes not. A special thank you goes to Frida Nellros, who has been a very good friend and colleague throughout many years of studies. Another special thank you goes to Vladimir Curi´cand´ Cris Luengo Hendriks at Uppsala University, for very good collaboration. This thesis would not have been the same without our discussions. A thank you also goes to our measurement technology partners, Kemi-Tornio Univer- sity of Applied Sciences (KTUAS). More specifically: Harri Pikkarainen, Jukka Leinonen, Juha Maronen, and Pauli Vaara. Furthermore; I have greatly appriciated the collabo- ration with industry throughout this thesis, which has provided an interesting challenge and a solid motivation for my work. I especially want to thank Robert Johansson and Mats Emmoth at SSAB Lule˚afor their interest and assistance. My intention has always been to do research that can be of practical use, and you have provided me with that connection. I would also like to acknowledge ProcessIT Innovations, and in particular P¨ar-ErikMartinsson, for all invested time and effort. Finally I would like to express my gratitude towards my family: for all support throughout this journey, and for always being understanding when time has been a scarcity. This work was partly supported by the EU Interreg IVA Nord program and Jernkon- toret. Anders Landstr¨om Lule˚a,November 2014 xi xii Part I 1 2 Chapter 1 Introduction \Use a picture. It's worth a thousand words." { Arthur Brisbane 1.1 Background 1.1.1 Automated systems and Machine Vision As technology advances, automated systems become more and more common. These solutions, designed to assist in different ways, often rely on processing of acquired sensor information, i.e. signals, in order to fulfill their task. For instance, consider a few examples such as an automated surveillance system using cameras to track movement [1], a car detecting pedestrians using a radar sensor [2], or an industrial system measuring the dimensions of the produced goods using a laser scanner [3]. The complexity of the signals as well as the complexity of the systems can of course vary greatly, but the common task of many automated systems is to interpret signals in order to behave or react as desired. Many more advanced systems rely on multi-dimensional signals, which do not vary with respect to one variable only (e.g. time or distance) but can vary with respect to multiple dimensions. Images constitute a common type of such signals, where the color values captured by the sensor (i.e. camera) varies in both an x-direction (left{right) and a y-direction (down{up). Other examples of multi-dimensional signals are volumetric data (which varies with respect to x, y, and a third spatial axis z) or video (which varies with respect to x, y, and a time variable t). Automated systems based on multi-dimensional data are often referred to as Machine Vision (MV) or Computer Vision (CV) systems, and rely on image processing and analysis. This thesis will focus on data which can be represented in two dimensions, i.e. can be depicted as an image, but the presented theory can be extended into higher dimensions as well. 3 4 Introduction 1.1.2 Image processing and analysis When an image captured by camera sensors is presented as an input signal to a system, it is merely a collection of numbers in a known order representing their different spatial po- sitions. These positions correspond to the smallest spatial entities in the resulting digital images, which are known as pixels. From these pixel values, which together form digital images, the system must then extract more high-level knowledge.