This item was submitted to Loughborough’s Institutional Repository (https://dspace.lboro.ac.uk/) by the author and is made available under the following Creative Commons Licence conditions. For the full text of this licence, please go to: http://creativecommons.org/licenses/by-nc-nd/2.5/ SPATIAL MEASUREMENT WITH CONSUMER GRADE DIGITAL CAMERAS Ren´eWackrow Department of Civil and Building Engineering Loughborough University A Doctoral Thesis Submitted for the Degree of Doctor of Philosophy October 2008 c by Ren´eWackrow (2008) Abstract Key words: photogrammetry, camera calibration, digital camera, camera stability, image configuration, spatial measurement, digital elevation model, lens distortion This thesis develops the potential of consumer-grade digital cameras for accu- rate spatial measurement. These cameras are generally considered unstable but their uncertain geometry can be partially resolved by calibration. The valid- ity of calibration data over time should be carefully assessed before subsequent photogrammetric measurement. The use of such digital cameras for photogram- metric measurement is increasingly accepted in many industrial fields but also in a diverse range of fields including medical and forensic science and architectural work. However, the stability of these cameras is less frequently reported in the literature, which can be attributed to the absence of standards for quantitative analyses of camera stability. The approach used to assess camera stability in this study is based on compar- ing the accuracy in the reconstructed object space, achieved using sets of interior orientation parameters of a sensor, derived in different calibration sessions. This technique was successfully applied to assess the temporal stability and manufac- turing consistency of seven identical Nikon Coolpix 5400 digital cameras. These cameras demonstrated remarkable potential to maintain their internal geometry over a 1-year period. This study also identified residual systematic error surfaces, discernable in digital elevation models (DEMS) derived from image pairs. These ’domes’ are caused by slightly inaccurately estimated lens distortion parameters. A methodology that uses a mildly convergent image configuration removes the systematic error sources. This result is significant for DEM generation us- ing low-cost digital cameras and a series of case studies demonstrated that this methodology can reduce the need for an accurate lens model and effectively in- crease the accuracy achievable with non-metric digital sensors. i Acknowledgements I have really enjoyed the three years of research at Loughborough University. However, the production of this thesis would not have been possible without the support and assistance of so many people. First of all, I would like to thank my supervisor Jim H. Chandler for his expert guidance in the field of Photogrammetry, constant encouragement and for always being accessible, despite other obligations. I also appreciated the help by Paul Bryan (English Heritage) for allowing access to their digital cameras. Special thanks go to Tobi, Nick and Xin for helping to conduct the case studies of a simulated river bed in London, Ontario and a small river in Loughborough. Thanks to all the people I have had the pleasure of working with over my three years in RL.0.17 and RL.0.18, including Uli, Jan, Colin, Masoud, Tingting, Keyur, Ben and Fiona. Also thanks to all members of the Loughborough Academicals and the ”Friday Evening Football Team” for many hours of enjoyable football, probably not at the highest standard but at a very competitive level. My time in Loughborough would never have been so enjoyable without the many great friends that I met in this town, including Jan and Jan, Paul, Kate, Lora, Chris, Richard, Gaiva, Nils, Abi and Lee, Patricia, my housemates from 75 Westfield Drive and many, many others. Thanks for many great nights out and an unforgettable time. My final thanks are to my family, particularly my parents, without their con- sideration and support I would have not got this far. Thanks to Loughborough University and the Engineering and Physical Sciences Research Council for funding the project. ii Contents Contents iii List of Tables vii List of Figures ix Nomenclature xiii 1 Introduction 1 1.1 Aims and Objectives . 2 1.2 Contribution to Knowledge . 4 1.3 Structure of Thesis . 5 2 Review of Relevant Literature 8 2.1 Brief Overview of the History of Photogrammetry . 8 2.2 Mathematical Models in Photogrammetry . 11 2.2.1 Coordinate Transformations . 11 2.2.2 Collinearity Equations . 13 2.2.3 Extended Collinearity Equations . 15 2.3 Fundamentals of Camera Calibration . 17 2.3.1 Radial Lens Distortion . 18 2.3.2 Tangential or Decentring Lens Distortion . 21 2.3.3 Affinity and Shear . 22 2.3.4 Least Squares Estimation (LSE) . 23 iii CONTENTS 2.4 Camera Calibration Methods . 25 2.4.1 Laboratory Calibration . 25 2.4.2 On-the-job Calibration . 27 2.4.3 Self-calibration . 28 2.4.4 Plumb-line Calibration . 29 2.5 Survey of Calibration Software . 30 2.5.1 Leica Photogrammetry Suite (LPS) PRO . 31 2.5.2 A Camera Calibration Toolbox for Matlab . 32 2.5.3 PI-3000 Software (trial version) . 33 2.5.4 General Adjustment Program (GAP) . 35 2.6 Stability analysis of Consumer-grade Digital Cameras . 36 2.6.1 Photogrammetric Capabilities of Consumer-grade Digital Cameras . 36 2.6.2 Stability of Non-metric Digital Cameras . 38 2.7 DEM Extraction using LPS . 39 2.8 Photogrammetric Data Quality . 44 2.8.1 Precision . 45 2.8.2 Reliability . 48 2.8.3 Accuracy . 48 2.9 Summary . 49 3 Methodology 51 3.1 Photogrammetric Processing . 51 3.1.1 External Self-calibrating Bundle Adjustment GAP . 51 3.1.2 Plumb-line Calibration . 56 3.2 Quality Assessment . 65 3.2.1 Truth DEM . 66 3.2.2 Automatically Extracted DEMs Using LPS . 67 3.2.3 DEM of Difference . 68 3.3 Assessment of Camera Stability . 70 iv CONTENTS 3.3.1 Temporal Stability . 72 3.3.2 Consistency of Manufacture . 73 3.4 Residual Systematic Error Surfaces . 74 3.4.1 The Simulation Process . 75 3.4.2 Radial Domes and the Lens Model . 75 3.4.3 Mildly Convergent Configuration . 76 3.4.4 Practical Test Using a Nikon D80 Camera . 77 3.5 Summary . 79 4 Results 80 4.1 Geometric Stability of the Nikon Coolpix 5400 Camera . 81 4.1.1 Criticality of Interior Orientation Parameters . 83 4.1.2 Temporal Stability . 87 4.1.3 Manufacturing Consistency . 92 4.2 Minimising Systematic Error Surfaces . 95 4.2.1 The Simulation . 95 4.2.2 Practical Test Using a Nikon D80 Camera . 102 4.3 Summary . 105 5 Case Studies 107 5.1 Measuring a Flume Bed Surface . 108 5.1.1 Data Accuracy of the Laboratory Flume . 112 5.2 Measurement of a Dynamic Water Surface . 115 5.2.1 Simulating Oblique Image Configurations . 116 5.2.2 Practical Test Measuring a Dynamic Water Surface . 125 5.3 Summary . 134 6 Discussion 136 6.1 Disturbing Effects on Camera Stability . 136 6.2 Role of Primary IO Parameters Using Oblique Imagery . 138 6.2.1 Impact of Varying the Focal Length Using Oblique Imagery 138 v CONTENTS 6.2.2 Impact of Varying the Principal Point Offset Using Oblique Imagery . 148 6.3 Determinability of Camera Parameters . 153 6.4 Guidelines for accurate spatial measurement using consumer-grade digital cameras . 156 6.5 Radial Distortion in Different Colour Channels . 158 6.6 Quantification of Impact of Inaccuracy of the Lens Model . 161 6.7 Cost - Resolution - Accuracy . 164 6.8 Medium versus High Accuracy - What is the Boundary? . 167 6.9 Summary . 169 7 Conclusion 172 7.1 Achievements of this Research Project . 173 7.2 Recommendations for Future Work . 177 7.3 Final Comment . 178 References 179 A Matlab code 188 B Refereed Journal Papers 196 vi List of Tables 2.1 List of camera calibration software . 31 3.1 Verification of the plumb-line program . 58 3.2 Strategy parameters for automatic DEM extraction . 68 3.3 Characteristics of the Nikon Coolpix 5400 camera . 72 3.4 Characteristics of the Nikon D80 camera . 77 4.1 Interior orientation parameters of the Nikon Coolpix 5400 cameras 81 4.2 MDF stability (Residuals of control points) . 82 4.3 DEM accuracy by varying focal length . 84 4.4 DEM accuracy by varying parameter of the principal point offset . 85 4.5 DEM accuracy by varying radial distortion parameter . 86 4.6 Restitution accuracy by using IOP from different dates . 88 4.7 DEM accuracy by using IOP from different dates . 89 4.8 Restitution accuracy by using IOP from different cameras/dates . 92 4.9 DEM accuracy by using IOP from different cameras/dates . 93 4.10 DEM accuracy for the simulation process . 98 4.11 DEM accuracy for the Nikon D80 camera . 104 5.1 Check point rms error (best possible lens model) . 113 5.2 Check point rms error (inaccurate lens model) . 113 5.3 Restitution accuracy achieved for the normal image configuration 129 5.4 Restitution accuracy achieved for the convergent image configuration129 vii LIST OF TABLES 5.5 Estimated focal lengths for the Nikon D80 cameras . 129 6.1 Recovered interior orientation parameters . 154 6.2 Variation in object accuracies for differing lens models - aerial case 163 6.3 Variation in object accuracies for differing lens models - oblique case163 6.4 Calibrated non-metric digital cameras . 164 viii List of Figures 2.1 Camera from Brunner (1859) in Luhmann et al. (2006)...... 9 2.2 Sequential rotations of axes in three dimensional space (Cooper and Robson, 2001).......................... 12 2.3 Central perspective projection (Cooper and Robson, 2001).... 14 2.4 Extended collinearity condition (Cooper and Robson, 2001).... 16 2.5 Image coordinate systems (Luhmann et al., 2006)........