Surface roughness estimation by 3D stereo SEM reconstruction Juan Camilo Henao Londo~no Universidad Nacional de Colombia Faculty of Exact and Natural Sciences Department of Physics and Chemistry Manizales, Colombia 2015 Surface roughness estimation by 3D stereo SEM reconstruction Juan Camilo Henao Londo~no Thesis submitted as a partial requirement to receive the grade of: Master in Sciences - Physics Advisor: Ph.D. Juan Carlos Ria~noRojas Co-advisor: Ph.D. Juan Bernardo G´omezMendoza Academic Research Group: PCM Computational Applications Universidad Nacional de Colombia Faculty of Exact and Natural Sciences Department of Physics and Chemistry Manizales, Colombia 2015 Estimaci´onde rugosidad superficial por reconstrucci´on3D MEB est´ereo Juan Camilo Henao Londo~no Tesis presentada como requisito parcial para optar al t´ıtulode: Magister en Ciencias - F´ısica Director: Ph.D. Juan Carlos Ria~noRojas Co-director: Ph.D. Juan Bernardo G´omezMendoza Grupo de trabajo acad´emico: PCM Computational Applications Universidad Nacional de Colombia Facultad de Ciencias Exactas y Naturales Departamento de F´ısicay Qu´ımica Manizales, Colombia 2015 A` ma famille. Acknowledgments Foremost, I would like to express my sincere gratitude to my advisor Dr. Juan Carlos Ria~no Rojas and my co-advisor Dr. Juan Bernardo G´omezMendoza for the continuous support during my Master research, for their patience, motivation, enthusiasm, and immense know- ledge. Their guidance helped me in all this time of research and writing of this thesis. I can not imagine having better advisors and mentors for my master. Besides my advisors, I would like to thank my thesis committee for their encouragement, insightful comments, and constructive questions. My sincere thanks also goes to Dr. Elisabeth Restrepo Parra and Professor Pedro Jos´eArango Arango, for supporting me at the beginning of all this way trough the world of research. To Dr. Jean Meunier and Dr. S´ebastienRoy for offering me the internship in their laboratory and giving significant contributions to my work. Furthermore, I recognize that this research would not have been possible without the finan- cial assistance provided by the Universidad Nacional de Colombia and Colciencias through its program “J´ovenes Investigadores e Innovadores" with the projects \Procesamiento Di- gital para la Reconstrucc´on3D de Im´agenesde Microscop´ıaElectr´onicade Barrido" and “An´alisisde recubrimientos por reconstrucci´ontridimensional autom´aticade im´agenesde microscop´ıaelectr´onica",and by the government of Canada through the \Emerging Leaders in the Americas Program (ELAP)" scholarship. I thank my fellow labmates in PCM Computational Applications group and \Laboratorio de F´ısicadel Plasma" group at the Universidad Nacional de Colombia, and my fellow labmates in \Laboratoire de Traitement d'Images" group, for the stimulating discussions and the time we shared together. Last but not least, I would like to thank my family and my closest friends for being always there for me, and supporting me at all times, both the bad ones and the good ones. I would like to make a special acknowledgment to my mother, for giving me the chance to study and having a really good education throughout my life. Juan Camilo Henao Londo~no 2015 xi Abstract Surface roughness is an important parameter to describe materials' topography. This parame- ter has been widely studied and presents important tasks in many engineering applications. The development of non-contact-based roughness measurement techniques for engineering surfaces has received much attention. However, stylus-based equipments are still dominating this measurement task. Stylus techniques have great inherent limitations as they were origi- nally intended to acquire 2D surface topography. Therefore, 3D surface roughness data can only be obtained from stylus equipment executing multiple scans of the surface. This task takes a lot of time to achieve a satisfactory result, may make micro-scratches on surfaces and can only evaluate a small area in a reasonable amount of time. In this work a new automated methodology for obtaining a 3D reconstruction model of sur- faces using scanning electron microscope (SEM) images based on stereo-vision is proposed. The 3D models can then be used to evaluate the surface roughness parameters. The horizon- tal stereo matching step is done with a robust and efficient algorithm based on semi-global matching. Since the brightness change of corresponding pixels is negligible for the small tilt involved in stereo SEM, and the cost function relies on dynamic programming, the matching algorithm uses a sum of absolute differences (SAD) over a variable pixel size window and an occlusion parameter which penalizes large depth discontinuities, that in practice, smooths the disparity map and the corresponding reconstructed surface. This step yields a disparity map, i.e. the differences between the horizontal coordinates of the matching points in the stereo images. The horizontal disparity map is finally converted into heights according to the SEM acquisition parameters: tilt angle, magnification and pixel size. A validation test was first performed using a microscopic grid with manufacturer specifications as reference. Finally, some surface roughness parameters were calculated within the model. Keywords: Roughness, Scanning Electron Microscopy, 3D reconstruction, Stereo-Vision, Dynamic Programming. xii Resumen La rugosidad superficial es un par´ametroimportante para describir la topograf´ıade los ma- teriales. Este par´ametroha sido ampliamente estudiado y es utilizado en importantes tareas en varias aplicaciones de ingenier´ıa.El desarrollo de t´ecnicasde medida de rugosidad basadas en m´etodos de no contacto para superficies han recibido mucha atenci´on.Sin embargo, los equipos basados en t´ecnicasde contacto siguen dominando las tareas de medida. Las t´ecnicas basadas en instrumentos de contacto tienen grandes limitaciones inherentes debido a que en principio fueron dise~nadaspara adquirir superficies topogr´aficasen 2D. As´ı,informaci´onde la rugosidad superficial en 3D solo puede ser obtenida con equipos de contacto ejecutando m´ultiplesbarridos de la superficie. Esta tarea toma mucho tiempo para obtener un resultado satisfactorio, puede producir microrayones sobre la superficie, y solo puede evaluar peque~nas ´areasen un tiempo razonable. En este trabajo se propone una nueva metodolog´ıausando visi´onpor computador. Con ella se busca obtener un modelo de reconstrucci´on3D de superficies usando im´agenesde mi- croscopio electr´onicode barrido (MEB) basadas en visi´onest´ereo,y as´ıevaluar par´ametros de rugosidad superficial. El paso de asociaci´onhorizontal est´ereoes hecho con un algoritmo robusto y eficiente basado en asociaci´onsemiglobal. Debido a que el cambio en el brillo de los pixeles correspondientes es despreciable para las peque~nasinclinaciones utilizadas en MEB est´ereoy la funci´onde costo se basa en programaci´ondin´amica,el algoritmo de asociaci´on usa la suma de diferencias absoluta (SAD en ingl´es)sobre una ventana de tama~novariable en pixeles y un par´ametrode oclusi´onel cual penaliza grandes discontinuidades de profundidad y, en pr´actica,suaviza el mapa de disparidad, y la superficie reconstruida correspondien- te. Este paso produce un mapa de disparidad, es decir, la diferencia entre las coordenadas horizontales de los puntos correspondientes en las im´agenesest´ereo.El mapa de disparidad es finalmente convertido en alturas de acuerdo a los par´ametrosde adquisici´ondel MEB: ´angulode inclinaci´on,magnificaci´ony tama~node pixel. Una prueba de validaci´onfue llevada a cabo usando como referencia una cuadr´ıculamicrosc´opicacon especificaciones de f´abrica. Finalmente, con el modelo son calculados algunos par´ametrosde rugosidad. Palabras Clave: Rugosidad, Microscopio Electr´onicode Barrido, Reconstrucci´on3D, Visi´onEst´ereo, Programaci´onDin´amica. Contents Acknowledgments IX Abstract XI Resumen XII List of Symbols XV List of Figures XVIII List of Tables XIX I. Preliminaries 1 1. Motivation 2 1.1. Materials Science . .2 1.2. Roughness . .2 1.3. Scanning Electron Microscope . .3 1.4. 3D Stereo Reconstruction . .4 2. State of the Art 6 2.1. Contact Methods . .7 2.2. Non-contact Methods . .8 2.2.1. Optical Methods . .8 2.2.2. Scanning Electron Microscopy . .8 2.3. Use of Non-contact Methods versus Contact Methods . .9 3. Objectives 10 3.1. General Objective . 10 3.2. Specific Objectives . 10 xiv Contents II. Materials and Methods 11 4. 3D Stereo Reconstruction 12 4.1. Stereo Pair Acquisition . 12 4.2. Stereo Correspondence . 12 4.2.1. 3D Reconstruction using Optical Flow . 12 4.3. 3D Reconstruction using Dynamic Programming . 15 4.3.1. Preprocessing . 15 4.3.2. Stereo Matching and Dynamic Programming . 18 5. Roughness Estimation 22 5.1. The Amplitude Parameters . 22 5.1.1. Arithmetic Average Height (Ra)..................... 22 5.1.2. Root Mean Square Roughness (Rq)................... 23 5.1.3. Maximum Height of Peaks (Rp)..................... 23 5.1.4. Maximum Depth of Valleys (Rv).................... 23 5.1.5. Mean Height of Peaks (Rpm)....................... 24 5.1.6. Mean Depth of Valleys (Rvm)...................... 24 5.1.7. Maximum Height of the Profile (Rt or Rmax).............. 24 5.1.8. Mean of Maximum Peak to Valley Height (Rtm)............ 24 5.1.9. Largest Peak to Valley Height (Ry)................... 25 5.2. SEM Images . 25 5.3. Measurements . 27 5.3.1. Disparity Maps and 3D Reconstruction . 27 5.3.2. Roughness Measurements . 33 5.3.3. Another Application: Wear Measurements . 37 6. Conclusions 40 6.1. Future Work . 41 A. Appendix A: Affine Transformation 43 B. Appendix B: Horizontal Disparity into Heights 45 Bibliography 47 List of Symbols Symbols with Latin letters Symbol Denomination IS Unit Definition Ra Arithmetic average height µm Eq. (5-1) Ry Largest peak to valley height µm Eq. (5-12) Rv Maximum depth of valleys µm Eq. (5-6) Rp Maximum height of peaks µm Eq. (5-5) Rt Maximum height of the profile µm Eq. (5-9) Rmax Maximum height of the profile µm Eq. (5-9) Rti Maximum peak to valley height µm Eq.