Infrared thermography enables the non-contact measurement of an object’s sur- Close range 3D thermography: face temperature and presents the results in form of thermal images. The analysis of these images provides valuable information about an object’s thermal state. How- real-time reconstruction of ever, the fidelity of the thermal images strongly depends on the pose of the thermo- high fidelity 3D thermograms graphic camera with respect to the surface. 3D thermography offers the possibility to overcome this and other limitations that affect conventional 2D thermography but most 3D thermographic systems developed so far generate 3D thermograms Antonio Rafael Ordóñez Müller from a single perspective or from few noncontiguous points of view and do not operate in real time. As a result, the 3D thermograms they generate do not offer much advantage over conventional thermal images. However, recent technological advances have unlocked the possibility of implementing affordable handheld 3D thermal imaging systems that can be easily maneuvered around an object and that can generate high-fidelity 3D thermograms in real time. This thesis explores vari- ous aspects involved in the real-time generation of high-fidelity 3D thermograms at close range using a handheld 3D thermal imaging system, presents the results of scanning an operating industrial furnace and discusses the problems associated with the generation of 3D thermograms of large objects with complex geometries. Close range 3D thermography: real-time reconstruction of high fidelity 3D thermograms reconstruction real-time 3D thermography: Close range 07 Schriftenreihe Mess- und Regelungstechnik der Universität Kassel ISBN 978-3-7376-0624-0 Band Univ.-Prof. Dr.-Ing. Andreas Kroll 9 783737 606240 07 H.`1` VJ`V1.V V R%JRV$VC%J$ VH.J1@RV`J10V` 1 ? : VC CQ V`:J$V .V`IQ$`:].77 `V:CR 1IV`VHQJ `%H 1QJ Q`.1$.`1RVC1 7 .V`IQ$`:I J QJ1Q:`:VC `RSMV<*CCV` kassel university press !"# $"%%&"'% (%"% " " %)"%*% $ #%"+ , -& . / 1 -& . (/ .23"& 4526 &"% * & "%"7 "$ !"%"7 "$ "& % "%"7 8 &"% ""&. 9 . / / 4526 -736::;5;4<5+&, -736::;5;4=+ $, >."&.? * 25 23422 36::;5;4= @7."&. ...5554<5;4=6 '4526/$ &B* / ### & $ B* Abstract Infrared thermography is a powerful technique that enables the non-contact measure- ment of an object’s surface temperature and presents the results in form of thermal images. The analysis of these images provides valuable information about an object’s thermal behavior and allows in many cases the diagnostic of abnormal conditions. How- ever, the fidelity with which thermal images depict the temperature distribution of an object’s surface is limited because the position of the camera with respect to the sur- face affects the accuracy of the measurements. 3D thermography offers the possibility to overcome this and other limitations that affect conventional 2D thermography but most 3D thermographic systems developed so far generate 3D thermograms from a single per- spective or from few noncontiguous points of view and do not operate in real time. As a result, the 3D thermograms they generate do not offer much advantage over conventional thermal images. But reductions in cost, size and weight of thermographic cameras, the recent introduction to the market of depth cameras that are also small, low-cost and light and the significant increase of computational power of commercial computers over the last years have unlocked the possibility to implement affordable handheld 3D thermal imaging systems that can be easily maneuvered around an object and that can generate high-fidelity 3D thermograms in real time. This thesis exploits these technological advancements and explores the multiple aspects involved in the real-time generation of high-fidelity 3D thermograms at close range using a handheld 3D thermal imaging system. It provides a structure for the knowledge accumulated in the field of 3D thermography, offers new insight into the radiometric, geometric and temporal calibration of the sensors and makes various contributions to improve the design of 3D thermographic systems, the fidelity of the 3D thermograms and the interaction with the user. In regards to the radiometric calibration, it shows through a series of original experiments how the thermal background radiation affects the accuracy of the temperature measurements and provides a plausible explanation for this poorly documented behavior. On the subject of geometric calibration, it presents and analyzes the results obtained with an improved calibration target and demonstrates that the orientation of the thermographic camera cannot be estimated with enough accuracy iii Abstract due of the poor spatial resolution of the thermal images and that manual fine-tuning may be necessary. Concerning the temporal calibration, this work provides a detailed analysis of the problem of assigning timestamps to the depth and thermal data. It also proposes a distinction between the concepts of data fusion and data validation where the former addresses the problem of determining which temperature measurements corresponds to which vertex on the 3D model and the latter addresses the problem of validating the measurement conditions before a temperature value is assigned to the corresponding vertex. In regards to sensor data fusion, it discusses the most effective approach to relate depth and thermal data and presents an effective method to reduce registration errors due to the parallax effect. In regards to data validation, it presents a new approach to determine when temperature values in the thermal texture of the 3D thermogram should be updated. Finally, this work shows that the use of shading and special color palettes helps improve the discernibility of features in renderings of 3D thermograms and proposes the use of additional textures to inform the user during the scan which parts of the model have been scanned under good measurement conditions and which need to be rescanned. iv Contents Abstract iii Contents v List of Figures ix List of Tables xi Acronyms xiii List of Symbols xv 1 Introduction 1 1.1 Motivation ................................... 1 1.2 State of the art ................................. 3 1.3 Objectives and outline ............................. 5 2 Theoretical background 7 2.1 Radiometric quantities ............................. 7 2.2 Electromagnetic radiation ........................... 9 2.2.1 Black body ............................... 10 2.2.2 Real body ................................ 12 2.2.3 Other theoretical bodies ........................ 13 2.2.4 Propagation in a medium ....................... 14 2.2.5 Radiative properties at an interface ................. 15 2.2.6 Radiative properties of opaque bodies ................ 20 2.3 Optical systems ................................. 21 2.3.1 Thin lens ................................ 21 2.3.2 Depth of field .............................. 22 2.3.3 Field of view .............................. 23 2.3.4 Optical aberrations ........................... 24 2.3.5 Pinhole camera model ......................... 24 2.4 Summary .................................... 26 3 2D Thermal imaging 29 3.1 Radiometric chain ............................... 29 3.2 Thermographic camera ............................. 31 3.2.1 Basic components ........................... 31 v Contents 3.2.2 Characterization ............................ 34 3.2.3 Switch-on behavior ........................... 34 3.3 Thermal image ................................. 36 3.3.1 Accuracy ................................ 36 3.3.2 Discernibility .............................. 41 3.4 Summary .................................... 42 4 3D thermal imaging: overview 43 4.1 System ...................................... 43 4.2 Summary .................................... 46 5 3D thermal imaging: calibration 47 5.1 Radiometric calibration ............................ 47 5.1.1 Overview ................................ 47 5.1.2 Related work .............................. 48 5.1.3 Experiments .............................. 49 5.1.4 Analysis ................................. 52 5.1.5 Discussion ................................ 55 5.2 Geometric calibration ............................. 56 5.2.1 Overview ................................ 56 5.2.2 Related work .............................. 58 5.2.3 Calibration procedure ......................... 59 5.2.4 Experimental results and analysis ................... 62 5.2.5 Discussion ................................ 67 5.3 Temporal calibration .............................. 68 5.3.1 Overview ................................ 68 5.3.2 Related work .............................. 69 5.3.3 Proposed approach ........................... 69 5.3.4 Experimental results .......................... 70 5.3.5 Discussion ................................ 72 5.4 Summary .................................... 73 6 3D thermal imaging: generation 75 6.1 3D model generation .............................. 75 6.1.1 Overview ................................ 75 6.1.2 Related work .............................. 75 6.1.3 Original approach and modifications ................. 77 6.1.4 Experimental results .......................... 82 6.1.5 Discussion ................................ 83 6.2 Data fusion ................................... 84 6.2.1 Overview ................................ 84 6.2.2 Related work .............................. 85 6.2.3 Proposed approach ........................... 88 6.2.4 Analytical and experimental