Chapter 4 Photometric Stereo Shape-From-Shading (PS-Sfs) for 3D Modelling of the Lunar Surface
Total Page:16
File Type:pdf, Size:1020Kb
Copyright Undertaking This thesis is protected by copyright, with all rights reserved. By reading and using the thesis, the reader understands and agrees to the following terms: 1. The reader will abide by the rules and legal ordinances governing copyright regarding the use of the thesis. 2. The reader will use the thesis for the purpose of research or private study only and not for distribution or further reproduction or any other purpose. 3. The reader agrees to indemnify and hold the University harmless from and against any loss, damage, cost, liability or expenses arising from copyright infringement or unauthorized usage. IMPORTANT If you have reasons to believe that any materials in this thesis are deemed not suitable to be distributed in this form, or a copyright owner having difficulty with the material being included in our database, please contact [email protected] providing details. The Library will look into your claim and consider taking remedial action upon receipt of the written requests. Pao Yue-kong Library, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong http://www.lib.polyu.edu.hk CONSTRUCTION OF HIGH-RESOLUTION LUNAR DEMS BY FUSING PHOTOGRAMMETRY AND SHAPE-FROM-SHADING Wai Chung LIU PhD The Hong Kong Polytechnic University 2020 The Hong Kong Polytechnic University Department of Land Surveying and Geo-Informatics Construction of High-Resolution Lunar DEMs by Fusing Photogrammetry and Shape-from-Shading Wai Chung LIU A Thesis Submitted in Partial Fulfilment of the Requirements for the Degree of Doctor of Philosophy December 2019 CERTIFICATE OF ORIGINALITY I hereby declare that this thesis is my own work and that, to the best of my knowledge and belief, it reproduces no material previously published or written, nor material that has been accepted for the award of any other degree or diploma, except where due acknowledgment has been made in the text. (Signed) Wai Chung LIU (Name of student) Abstract 3D topographic modelling of the lunar surface is fundamental to lunar and planetary science and exploration. Topographic models such as digital elevation models (DEMs) of the lunar surface have seen extensive use in the study of the lunar surface and geologic processes and for planning of lunar exploration missions. Lunar DEMs are usually generated from remote sensing data acquired by instruments such as cameras and laser altimeters onboard lunar orbiters. The currently available lunar DEMs have the following limitations: (1) lunar DEMs with global or near-global coverage usually have a comparatively low spatial resolution (i.e., 60 m/pixel or lower); (2) regional high-resolution DEMs generated with photogrammetry have limited spatial coverage; and (3) lunar DEMs cannot reach the effective spatial resolution of optical images acquired by lunar orbiters. The performance of photogrammetry, in particular, depends heavily on the robustness of image matching, which directly affects the spatial resolution and accuracy of the resulting lunar DEMs. Photogrammetry may even fail due to the limitations of the image matching approach when the stereo images are poorly textured and/or acquired with large variations in illumination, which are common problems for lunar images. To overcome these limitations, lunar topographic modelling was performed with Shape- from-Shading (SfS; also known as photoclinometry), a technique that generates 3D models based on the image’s photometric content (i.e., intensity value). It can reconstruct lunar DEMs with optimal spatial resolution using single or multiple images, and it is robust to fine-grained details, subtle surface textures and variations in illumination. However, SfS lacks accuracy for large-scale topographic modelling, at which photogrammetry excels. As a result, SfS and photogrammetry are natural complements for the generation of accurate and optimal 3D representations of the lunar surface, which can better support exploration missions (e.g., landing site selection) and scientific research (e.g., lunar geology). This research aims to integrate SfS and photogrammetry to construct accurate lunar DEMs with optimal spatial resolution. This study includes three major developments. First, a novel hierarchical SfS approach was developed. Given a single monocular image, the single-image SfS (SI-SfS) approach refines an initial DEM of lower resolution to an optimal resolution of the image. The corresponding pixel-wise albedo map of the surface is estimated simultaneously in the process and I is used by SI-SfS to regularise the 3D refinement of the initial DEM. A hierarchical architecture was used for effective SfS at multiple resolutions, and an explicit shadow constraint was developed and incorporated into the SI-SfS approach to overcome image regions with shadows. Experiments were carried out using Narrow Angle Camera (NAC) images from the Lunar Reconnaissance Orbiter Camera (LROC), with a spatial resolution of 0.5 to 1.5 m/pixel. The integrated Selenological and Engineering Explorer and LRO Elevation Model (SLDEM), with a spatial resolution of 60 m/pixel, was chosen as the initial DEM for constraint. The results indicate that local topographic details were well recovered by the SI-SfS approach with plausible albedo estimation. The refined DEM achieved geometric accuracies of 2 to 7 m relative to the reference NAC DEMs generated from photogrammetry. The low-frequency topographic consistency depends upon the quality of the low- resolution DEM and the difference in the spatial resolution between the image and the initial DEM. Second, a photometric stereo SfS (PS-SfS) approach was developed. The PS- SfS approach uses two co-registered images acquired under different illumination conditions to generate high-resolution lunar DEMs. In contrast to the previous SI- SfS approach, PS-SfS does not require an initial DEM. A novel formulation was used to effectively factor out the effects of albedo variations, leading to more robust performance. Moreover, a quantitative investigation of the effects of illumination differences on the performance of 3D reconstruction was carried out for the PS-SfS. First, the fundamental process of PS-SfS was mathematically modelled to correlate the numeric solution of PS-SfS and the images’ differences in illumination. Based upon the mathematical model, an error model was then derived to analyse the relationships between the azimuthal and zenith angles of the images’ illumination and the reconstruction qualities. The developed PS-SfS approach and the error model were verified with LROC NAC images. Our experimental analyses reveal that the resulting error in PS-SfS depends upon both the azimuthal and the zenith angles of illumination and upon the general intensity of the images. The predictions from the proposed error model are consistent with the actual slope errors obtained by PS-SfS using LROC NAC images. The proposed error model enriches the theory of PS-SfS and is significant for optimised lunar surface reconstruction based on SfS techniques. Third, an integrated photogrammetric and photoclinometric approach was developed. The integrated approach can generate lunar DEMs with optimal II resolution and is invariant to variations in illumination. The fusion of photoclinometry and photogrammetry involves two main steps. First, a photoclinometry-assisted image matching (PAM) approach is developed by integrating PS-SfS into the image matching stage to create pixel-wise matches, even for images with large differences in illumination. Second, the DEM derived from photogrammetry using the matching results is refined to optimal resolution using the SI-SfS approach. The proposed approach has been validated with high-resolution LROC NAC images acquired in various conditions of illumination at the Chang’E-4 and Chang’E-5 landing sites. The results indicate that the integrated approach is robust to severe inconsistencies in illumination and to subtle textures in cases in which the conventional approaches fail. The integrated approach can achieve geometric accuracies comparable to photogrammetry, while giving more small-scale topographic detail. The presented research and development allows for the generation of high- precision and high-resolution lunar DEMs from a single image or multiple images, which effectively extends the ability of photogrammetry in lunar mapping. The developed approaches have been widely used in China’s Chang’E-4 and Chang’E-5 lunar missions to generate high-resolution DEMs of the landing regions to support optimised landing site selection and surface operation. The developed approaches can also be used for high-resolution topographic mapping of other planetary bodies, such as Mercury and asteroids, and to provide a useful reference for topographic mapping on Earth. III Publications Arising from the Thesis Journal Papers: [1] Liu, W. C. and Wu, B., 2020. An Integrated Photogrammetric and Photoclinometric Approach for Illumination-Invariant Pixel-Resolution 3D Mapping of the Lunar Surface. ISPRS Journal of Photogrammetry and Remote Sensing, 159: 153-168. [2] Liu, W. C., Wu, B., and Wöhler, C., 2018. Effects of Illumination Differences on Photometric Stereo Shape-and-Albedo-from-Shading for Precision Lunar Surface Reconstruction. ISPRS Journal of Photogrammetry and Remote Sensing, 136: 58-72. [3] Wu, B., Liu, W. C., Grumpe, A., and Wöhler, C., 2017. Construction of Pixel- Level Resolution DEMs from Monocular Images by Shape and Albedo from Shading Constrained with Low-Resolution DEM. ISPRS Journal of Photogrammetry and Remote Sensing,