Large Area High-Resolution 3D Mapping of Oxia Planum: the Landing Site for the Exomars Rosalind Franklin Rover

Home , Aurora programme, Department of Space, Kazachok, Mars Express, Next Mars Orbiter, Oxia Planum

remote sensing

Article Large Area High-Resolution 3D Mapping of Oxia Planum: The Landing Site for the ExoMars Rosalind Franklin Rover

Yu Tao 1,* , Jan-Peter Muller 1 , Susan J. Conway 2 and Siting Xiong 3,4

1 Imaging Group, Mullard Space Science Laboratory, Department of Space and Climate Physics, University College London, Holmbury St Mary, Surrey RH5 6NT, UK; [email protected] 2 Laboratoire de Planétologie et Géodynamique, CNRS, UMR 6112, Université de Nantes, 44300 Nantes, France; [email protected] 3 College of Civil and Transportation Engineering, Shenzhen University, Shenzhen 518060, China; [email protected] 4 Ministry of Natural Resources Key Laboratory for Geo-Environmental Monitoring of Great Bay Area & Guangdong Key Laboratory of Urban Informatics & Shenzhen Key Laboratory of Spatial Smart Sensing and Services, Shenzhen University, Shenzhen 518060, China * Correspondence: [email protected]

Abstract: We demonstrate an end-to-end application of the in-house deep learning-based surface modelling system, called MADNet, to produce three large area 3D mapping products from single images taken from the ESA Mars Express’s High Resolution Stereo Camera (HRSC), the NASA Mars Reconnaissance Orbiter’s Context Camera (CTX), and the High Resolution Imaging Science Experiment (HiRISE) imaging data over the ExoMars 2022 Rosalind Franklin rover’s landing site at Oxia Planum on Mars. MADNet takes a single orbital optical image as input, provides pixelwise height predictions, and uses a separate coarse Digital Terrain Model (DTM) as reference, to produce a DTM product from the given input image. Initially, we demonstrate the resultant 25 m/pixel HRSC DTM mosaic covering an area of 197 km × 182 km, providing fine-scale details to the 50 m/pixel Citation: Tao, Y.; Muller, J.-P.; HRSC MC-11 level-5 DTM mosaic. Secondly, we demonstrate the resultant 12 m/pixel CTX MADNet Conway, S.J.; Xiong, S. Large Area DTM mosaic covering a 114 km × 117 km area, showing much more detail in comparison to High-Resolution 3D Mapping of Oxia photogrammetric DTMs produced using the open source in-house developed CASP-GO system. Planum: The Landing Site for the ExoMars Rosalind Franklin Rover. Finally, we demonstrate the resultant 50 cm/pixel HiRISE MADNet DTM mosaic, produced for the Remote Sens. 2021, 13, 3270. https:// first time, covering a 74.3 km × 86.3 km area of the 3-sigma landing ellipse and partially the ExoMars doi.org/10.3390/rs13163270 team’s geological characterisation area. The resultant MADNet HiRISE DTM mosaic shows fine-scale details superior to existing Planetary Data System (PDS) HiRISE DTMs and covers a larger area Academic Editor: Long Xiao that is considered difficult for existing photogrammetry and photoclinometry pipelines to achieve, especially given the current limitations of stereo HiRISE coverage. All of the resultant DTM mosaics Received: 22 July 2021 are co-aligned with each other, and ultimately with the Mars Global Surveyor’s Mars Orbiter Laser Accepted: 16 August 2021 Altimeter (MOLA) DTM, providing high spatial and vertical congruence. In this paper, technical Published: 18 August 2021 details are presented, issues that arose are discussed, along with a visual evaluation and quantitative assessments of the resultant DTM mosaic products. Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in Keywords: mars; 3D mapping; large area mapping; digital terrain model; deep learning; MADNet; published maps and institutional affil- HRSC; CTX; HiRISE; Oxia Planum; ExoMars; Rosalind Franklin rover; landing site; planetary iations. mapping; 3D reconstruction

Copyright: © 2021 by the authors. 1. Introduction Licensee MDPI, Basel, Switzerland. This article is an open access article Large area, high-resolution, three-Dimensional (3D) mapping of the Martian surface is distributed under the terms and not only essential for performing key science investigations of the generation and evolution conditions of the Creative Commons of the planet’s surface, but also critical for supporting existing and future surface robotic Attribution (CC BY) license (https:// missions as well as human exploration. Over the past 20 years, 3D mapping of the Martian creativecommons.org/licenses/by/ surface has only been done for larger areas with lower-resolution data, or for small areas 4.0/). with higher-resolution data.

Remote Sens. 2021, 13, 3270. https://doi.org/10.3390/rs13163270 https://www.mdpi.com/journal/remotesensing Remote Sens. 2021, 13, 3270 2 of 26

The large area lower-resolution 3D mapping work usually uses the Mars Express’s High Resolution Stereo Camera (HRSC) data at 12.5–50 m/pixel [1] producing photogrammetric digital terrain models (DTMs) at 50–150 m/pixel [2–7], or uses the Mars Recon- naissance Orbiter (MRO) Context Camera (CTX) data at 6 m/pixel [8] producing DTMs at 18–24 m/pixel [9–11] where such serendipitous stereo coverage exists. On the other hand, the small area high-resolution 3D mapping work usually employs the MRO High Resolution Imaging Science Experiment (HiRISE) data at 0.25–0.5 m/pixel [12] producing DTMs at 0.75–2 m/pixel using photogrammetry [13–15] or DTMs at 0.25–0.5 m/pixel using photoclinometry [16–20], or uses the ExoMars Trace Gas Orbiter (TGO) Colour and Stereo Surface Imaging System (CaSSIS) data at 4–6 m/pixel [21] producing DTMs at 12–18 m/pixel using photogrammetry [22] or at 4 m/pixel using photoclinometry (at the pixel-level) or at 1 m/pixel using super- resolution restoration assisted photoclinometry [23] (at the sub-pixel level). The limitations of having to trade-off between “large area” and “high-resolution”, for Mars 3D mapping results from three different aspects. Firstly, lower-resolution data such as HRSC has better stereo coverage as it is inherently a stereo mapping instrument, whereas the main high-resolution data to date (HiRISE) is only capable of targeted stereo. Secondly, the photogrammetry and photoclinometry processes are complex and subject to different errors (or artefacts) at each of their different processing stages, which require specialist expertise to deal with these complexities, in order to produce high-quality DTM products, especially over large areas. Thirdly, the expensive computational cost from the existing photogrammetry and photoclinometry pipelines make it extremely difficult to achieve large area high-resolution coverage. In this paper, we experiment with a previously developed single-input-image fast DTM surface modelling system, called MADNet (Multi-scale generative Adversarial u- net with Dense convolutional and up-projection blocks) [24], exploring large area high- resolution DTM production using a set of co-registered HRSC-CTX-HiRISE images that are co-aligned with the global reference DTM from the Mars Global Surveyor’s Mars Orbiter Laser Altimeter (MOLA) [25–27]. In contrast with traditional photogrammetric methods, where two overlapping images (with a suitable stereo angle, typically ≥8◦ [10]) are used as inputs to derive the DTM, MADNet requires only a single image as input to derive a DTM, firstly in relative values (normalised), and then uses a referencing coarse DTM (e.g., the globally available MOLA DTM) to produce the final DTM product. It should be noted that the referencing coarse DTM is also required in photogrammetric methods because the initial surface heights, triangulated from the relative image disparities, are generally considered imprecise and are required to be corrected using a reference DTM (typically being the downsampled 463 m/pixel MOLA DTM for Mars). The above differences are illustrated in Figure1, where we show the simplified flow diagrams of the photogrammetric DTM pipeline and the deep learning-based single-image DTM pipeline (e.g., MADNet [24]). We focus on the Oxia Planum area [28–31], where the joint European Space Agency (ESA) and Russian Roscosmos ExoMars mission will land the ESA “Rosalind Franklin” rover and the Roscosmos landing platform “Kazachok” in 2023. Large area high-resolution DTM mosaics, covering the 3-sigma landing ellipses, are produced here at 25 m/pixel for HRSC, 12 m/pixel for CTX, and at 50 cm/pixel for HiRISE. Cascaded 3D co-alignments (HRSC-to-MOLA, CTX-to-HRSC, and HiRISE-to-CTX) have been achieved to guarantee precise global congruence with respect to MOLA. Part of this area has been used for the Jet Propulsion Laboratory webGIS (web based Geographic Information System) called MMGIS (Multi-Mission Geographic Information System) [32] to assist the ExoMars team’s geological characterisation of the area [33]. Remote Sens. 2021,, 13,, 3270x FOR PEER REVIEW 33 of 2628

Figure 1. Illustration of theFigure major differences1. Illustration between of the themajor photogrammetric differences betw DTMeen the pipeline photogrammetric and the deep DTM learning-based pipeline and the single-image DTM pipelinedeep (e.g., learning-based MADNet) discussed single-image here. DTM pipeline (e.g., MADNet) discussed here.

WeThe focus layout on of the this Oxia paper Planum is as follows.area [28–31], In Section where 2.1 the, the joint processing European core Space (MADNet) Agency (ESA)is reviewed, and Russian followed Roscosmos by a comprehensive ExoMars mission description will land of the the overall ESA “Rosalind processing Franklin” chain in roverSection and 2.2 the. Technical Roscosmos challenges landing are platform discussed “Kazachok” in Section in2.3 2023., followed Large by area an introductionhigh-resolu- tionto the DTM study mosaics, site. Incovering Sections the 3.1 3-sigma–3.3, we landing present ellipses, final are results produced from HRSC,here at CTX,25m/pixel and forHiRISE HRSC, data, 12m/pixel respectively, for CTX, and and these at 50cm/pixel are then followed for HiRISE. by Section Cascaded 3.4 where 3D co-alignments we present (HRSC-to-MOLA,additional assessments CTX-to for-HRSC, the 50 cm/pixel and HiRISE-to-CTX) HiRISE MADNet have DTMbeen mosaic.achieved Product to guarantee access, preciseextensibility global of congruence the proposed with methods, respect to known MOLA issues,. Part of limitations, this area has and been future used work for arethe Jetdiscussed Propulsion in Section Laboratory4 before webGIS conclusions (web are based drawn Geographic in Section Information5. System) called MMGIS (Multi-Mission Geographic Information System) [32] to assist the ExoMars team’s geological2. Materials characterisation and Methods of the area [33]. 2.1. OverviewThe layout of MADNetof this paper and Trainingis as follows. Details In Section 2.1, the processing core (MADNet) is reviewed,The processing followed core by a of comprehensive this work is the description MADNet deepof the learning overall basedprocessing single-image chain in SectionDTM estimation 2.2. Technical system challenges described are in [discussed24]. MADNet in Section is based 2.3, on followed the Generative by an introduction Adversarial toNetwork the study (GAN) site. In framework Section 3.1, [34 Section], and in3.2, particular and Section is based 3.3, we on present a multi-scale final results relativistic from HRSC,GAN architecture CTX, and HiRISE [35,36], data, which respectively, was previously and these developed are then for followed image super-resolution by Section 3.4 wheretasks. MADNetwe present operates additional by training assessments a generative for the model50cm/pixel for relative HiRISE height MADNet prediction, DTM mo- and saic.in an Product alternating access, manner, extensibility updating of the a discriminatorproposed methods, model known to distinguish issues, limitations, the predicted and futureheights work from are the discussed normalised in ground-truthSection 4 before heights. conclusions are drawn in Section 5. The MADNet generator network uses a three-scale U-Net [37] based architecture. 2.Each Materials of the three-scale and Methods U-Nets (the coarse-scale, intermediate-scale, and fine-scale) consists of a dense convolution block (DCB) [38] based encoder arm and an up-projection block 2.1. Overview of MADNet and training details (UPB) [39] based decoder arm. The fine-scale U-Net contains five stacks of convolutional layers,The pooling processing layers, core and of DCBsthis work to encode is the theMADNet input imagedeep learning into a feature based tensor,single-image which DTMis then estimation fed into five system stacks described of UPBs in and [24]. convolutional MADNet is based layers on with the Generative concatenations Adversar- of the ialcorresponding Network (GAN) outputs framework of each pooling[34], and layer in particular to decode is the based feature on avectors multi-scale into therelativistic output GANheight architecture map. The intermediate-scale [35,36], which was and previo coarse-scaleusly developed U-Nets take for two image times super-resolution and four times tasks.downsampled MADNet input operates images by training and use a four generati and threeve model stacks for of relative the convolution-pooling-DCB height prediction, and inand an UPB-convolution alternating manner, layers updating to reconstruct a discriminator the height model maps to in distinguish two times and the fourpredicted times heightscoarser scales,from the respectively. normalised ground-truth heights. TheOutputs MADNet from thegenerator three-scale network U-Net uses networks a three-scale are then U-Net merged [37] with based adaptive architecture. weights Eachto reconstruct of the three-scale the final outputU-Nets height (the coarse-s map (relativecale, intermediate-scale, height). A simplified and network fine-scale) architec- con- siststure ofof thea dense MADNet convolution generator block can be (DCB) found [3 in8] Figure based2 forencoder a detailed arm descriptionand an up-projection (including blockthe discriminator (UPB) [39] based network), decoder please arm. refer The to fine-scale [24]. U-Net contains five stacks of convolutional layers, pooling layers, and DCBs to encode the input image into a feature tensor, which is then fed into five stacks of UPBs and convolutional layers with concatenations of

Remote Sens. 2021, 13, x FOR PEER REVIEW 4 of 28

the corresponding outputs of each pooling layer to decode the feature vectors into the output height map. The intermediate-scale and coarse-scale U-Nets take two times and four times downsampled input images and use four and three stacks of the convolution- pooling-DCB and UPB-convolution layers to reconstruct the height maps in two times and four times coarser scales, respectively. Outputs from the three-scale U-Net networks are then merged with adaptive weights Remote Sens. 2021, 13, 3270 to reconstruct the final output height map (relative height). A simplified network archi-4 of 26 tecture of the MADNet generator can be found in Figure 2 for a detailed description (including the discriminator network), please refer to [24].

Figure 2. Overview of the MADNetFigure 2. generator Overview network of the MADNet architecture. generator The input network image ar haschitecture. a size of The 512 ×input512 pixelsimage andhas thea size of final output height map has512×512 a size of pixels 256 × and256 the pixels final in output a relative height value map range has ofa size (0, 1). of 256×256 pixels in a relative value range of (0, 1). Training of the MADNet model was achieved in two stages. In the first stage, initial trainingTraining was performed of the MADNet on each model of the was three achieved adversarial in two U-Nets, stages. using In the 12,600 first training stage, initial pairs trainingof 4 m/pixel was (down-sampled)performed on each HiRISE of the images three and adversarial 8 m/pixel U-Nets, existing using HiRISE 12,600 DTMs. training In the pairssecond of stage,4m/pixel the adaptive(down-sampled) weighted HiRISE multi-scale images adversarial and 8m/pixel U-Nets existing were HiRISE trained DTMs. jointly withIn the weights second initialised stage, the from adaptive the first weighted stage training multi-scale using adversarial 46,500 training U-Nets pairs were of 2 m/pixel trained jointlyHiRISE with images weights and 4initialised m/pixel from HiRISE the DTMs. first stage The training training using pairs 46,500 contain training a collection pairs of 2m/pixeldifferent MartianHiRISE images surface and features 4m/pixel including HiRISE layers, DTMs. craters, The training cones, peaks,pairs contain dunes, a and collec- flat tionterrains. of different For detailed Martian hyperparameter surface features setups, including please referlayers, to craters, [24]. cones, peaks, dunes, and flat terrains. For detailed hyperparameter setups, please refer to [24]. 2.2. Overall Processing Chain 2.2. OverallIn this processing work, MADNet chain is employed alongside 3D co-alignment and DTM mosaicing methods to produce co-aligned DTM mosaics using a set of co-registered HRSC-CTX- In this work, MADNet is employed alongside 3D co-alignment and DTM mosaicing HiRISE images over the landing site area. The overall processing chain of the work methods to produce co-aligned DTM mosaics using a set of co-registered HRSC-CTX- described here is shown in Figure3. HiRISE images over the landing site area. The overall processing chain of the work de- The inputs of the proposed large area multi-resolution 3D mapping work are the scribed here is shown in Figure 3. 463 m/pixel MOLA areoid DTM (available at https://astrogeology.usgs.gov/search/details/ Mars/GlobalSurveyor/MOLA/Mars_MGS_MOLA_DEM_mosaic_global_463m/cub, accessed on 22 July 2021), the 50 m/pixel HRSC MC-11W (Mars Chart-11 West) level 5 DTM mosaic and 12.5 m/pixel ORI mosaic (available at http://hrscteam.dlr.de/HMC3 0/MC11W/, accessed on 22 July 2021), the 6 m/pixel CTX stereo-view images (accessible through the Arizona State University’s Mars Image Explorer at http://viewer.mars. asu.edu/viewer/ctx, accessed on 22 July 2021), and the 25–50 cm/pixel HiRISE single- view (monocular) images (available through the University of Arizona’s HiRISE site at https://hirise-pds.lpl.arizona.edu/PDS/, accessed on 22 July 2021). The CTX and HiRISE image IDs that cover the landing site area were found through the product coverage shapefile site (https://ode.rsl.wustl.edu/mars/coverage/ODE_Mars_shapefile.html, accessed Remote Sens. 2021, 13, 3270 5 of 26

on 22 July 2021). The final outputs of the mapping work include a 25 m/pixel HRSC DTM mosaic covering about a 197 km × 182 km area of the landing site, a 12 m/pixel CTX DTM mosaic covering about a 114 km × 117 km area of the landing site, and a 50 cm/pixel HiRISE DTM mosaic covering about a 74.3 km × 86.3 km area of the 3-sigma landing ellipses and partially the ExoMars team’s geological characterisation area [33]. It should Remote Sens. 2021, 13, x FOR PEER REVIEW 5 of 28 be noted that all final outputs are 3D co-aligned with the reference MOLA DTM using the B-spline fitting based 3D co-alignment method that was described in [6,23].

Figure 3. Overall processingFigure chain 3. of Overall the large processing area multi-resolution chain of the large (HRSC-CTX-HiRISE) area multi-resolution 3D mapping (HRSC-CTX-HiRISE) work. Inputs are3D map- shown with the light orangeping boxes, work. intermediate Inputs are products shown with with the the light light orange blue boxes, boxes, and intermediate ﬁnal outputs products are shown with withthe light the blue light green boxes. boxes, and final outputs are shown with the light green boxes.

The B-splineinputs of fitting the proposed algorithm large described area mu inlti-resolution [6,23] is based 3D on mapping the calculation work are of thethe nonrigid463m/pixel closest MOLA 3D areoid points DTM of two (available planar B-spline at https://astrogeology.usgs.gov/search/de- surfaces computed from the target andtails/Mars/GlobalSurveyor/MOLA/Mars_ reference DTMs. In particular, the nonrigidMGS_MOLA_DEM_mosaic_global_463m/cub), transformation of each 3D point within the 50m/pixel target DTM HRSC is calculated MC-11W by (Mars minimising Chart-11 the West) weighted level 5 terms DTM ofmosaic the distance and 12.5m/pixel between theORI target mosaic and (available reference at 3Dhttp://hrscteam.dlr.de/HMC30/MC11W/), points, the stiffness values that penalise the transformations 6m/pixel CTX ste- of thereo-view neighbourhood images (accessible of the target through 3D the points, Ariz andona theState distance University’s between Mars the Image 2D tie-points Explorer computedat http://viewer.mars.asu.edu/viewer/ctx), from the matching of image features. and the 25–50cm/pixel HiRISE single-view (monocular)The overall images processing (available chain through includes the 11 University steps (labelled of Arizona’s in Figure 3HiRISE) and cansite be at summarisedhttps://hirise-pds.lpl.arizona.edu/PDS/). as follows: The CTX and HiRISE image IDs that cover the (1)landingB-spline site fittingarea basedwere onfound 3D co-alignment through the of theproduct input (cropped)coverage HRSCshapefile MC-11W site (https://ode.rsl.wustl.edu/mars/coverage/ODE_Mars_shapefile.html).level 5 DTM mosaic with respect to the input MOLA DTM to obtain The an final intermediate outputs of the50 mapping m/pixel MOLAwork correctedinclude a HRSC 25m/pixel DTM mosaic.HRSC DTM mosaic covering about a (2)197km×182kmMADNet DTMarea of production, the landing using site, the a 12m/pixel cropped HRSC CTX DTM MC-11W mosaic level covering 5 ORI mosaic about as a 114km×117kminput to produce area of anthe intermediate landing site, 25 and m/pixel a 50cm/pixel HRSC DTM HiRISE mosaic. DTM mosaic covering (3)about 3D a 74.3km×86.3km co-alignment of thearea intermediate of the 3-sigma 25 m/pixel landing HRSC ellipses DTM and mosaic partially from the step ExoMars (2) and team’sthe geological intermediate characterisation 50 m/pixel MOLA-co-aligned area [33]. It should HRSC be noted DTM that mosaic all final from outputs step (1), are to 3D co-alignedproduce thewith 25 the m/pixel reference MOLA-co-aligned MOLA DTM using HRSC the DTMB-spline mosaic, fitting which basedis 3D the co-align- first of ment themethod three that final was products described of this in work.[6,23]. (4) TheCASP-GO B-spline [6 ,10fitting,40] photogrammetricalgorithm described DTM in production[6,23] is based of CTX on serendipitousthe calculation “stereo” of the nonrigidimages closest to produce 3D points 18 m/pixelof two planar CTX DTMs,B-spline which surfaces are co-alignedcomputed withfrom the the 25 target m/pixel and referenceMOLA-co-aligned DTMs. In particular, HRSC the DTM nonrigid mosaic transformation from step (3), of as each well 3D as point 6 m/pixel within CTX the targetORIs, DTM whichis calculated are co-registered by minimising with the the weighted input 12.5 terms m/pixel of the HRSC distance MC-11W between level the target5 and ORI reference mosaic. 3D points, the stiffness values that penalise transformations of the neighbourhood of the target 3D points, and the distance between the 2D tie-points computed from the matching of image features. The overall processing chain includes 11 steps (labelled in Figure 3) and can be sum- marised as follows: 1) B-spline fitting based on 3D co-alignment of the input (cropped) HRSC MC-11W level 5 DTM mosaic with respect to the input MOLA DTM to obtain an intermediate 50m/pixel MOLA corrected HRSC DTM mosaic. 2) MADNet DTM production, using the cropped HRSC MC-11W level 5 ORI mosaic as input to produce an intermediate 25m/pixel HRSC DTM mosaic.

Remote Sens. 2021, 13, 3270 6 of 26

(5) MADNet DTM production using the intermediate 6 m/pixel HRSC-co-registered CTX ORIs from step (4) to produce intermediate 12 m/pixel CTX DTMs. (6) 3D co-alignment of the intermediate 12 m/pixel CTX DTMs from step (5) and the 18 m/pixel HRSC-co-aligned CTX DTMs from step (4) to produce 12 m/pixel HRSC- co-aligned CTX DTMs. (7) DTM mosaicing (using the ASP [40] “dem_mosaic” function) of the 12 m/pixel HRSC- co-aligned CTX DTMs from step (6) to produce a 12 m/pixel HRSC-co-aligned CTX DTM mosaic, which is the second of the three final products of this work. (8) Image co-registration (using the mutual shape adapted scale invariant features as described in [41,42]) of the input 25–50 cm/pixel HiRISE images with respect to the 6 m/pixel HRSC-co-registered CTX ORIs from step (4). (9) MADNet DTM production using the CTX-co-registered HiRISE images from step (8) to produce intermediate 50 cm/pixel HiRISE DTMs. (10) 3D co-alignment of the intermediate 50 cm/pixel HiRISE DTMs from step (9) and the 12 m/pixel HRSC-co-aligned CTX DTM mosaic from step (7) to produce CTX-co- aligned 50 cm/pixel HiRISE DTMs. (11) DTM mosaicing of the 50 cm/pixel CTX-co-aligned HiRISE DTMs from step (10) to produce a 50 cm/pixel CTX-co-aligned HiRISE DTM mosaic, which is the third of the three final products of this work. It should be noted that the photogrammetric DTMs from HRSC are used as the reference DTM to co-align the HRSC MADNet DTM, because they have a smaller resolution gap in comparison to using the MOLA DTM as the reference. This is also the case for CTX. However, if HRSC and CTX photogrammetric DTMs are not available, the MOLA DTM (for HRSC) and the HRSC MADNet DTM (for CTX) can also be used as the reference datasets. This is demonstrated in the HiRISE case, where photogrammetric processing is far too computationally expensive, and thus not included for HiRISE. In this work, a total of 12 CTX images (6 serendipitous pairs) and 44 HiRISE images are used (both are manually selected using the list from the aforementioned product coverage shapefile site) to cover the 3-sigma landing ellipses and partially the ExoMars team’s geological characterisation area [33]. The input HiRISE images are manually selected for quality, while keeping sufficient overlaps within neighbouring scenes in order to produce a single uniform quality and gap-free DTM mosaic. The 25 cm/pixel images have a priority over 50 cm/pixel images if both cover the same area. Some HiRISE images that contain severe framelet-stitching artefacts or missing data are excluded. New images are also checked to fill remaining DTM gaps at the end of the process. For full lists of the used CTX and HiRISE image IDs, please refer to Sections 3.2 and 3.3 It should be noted that there are many more images that exist and are available other than the selected CTX and HiRISE images for this work. An overview of the area covered, and the input MOLA-HRSC-CTX-HiRISE datasets, are shown in Figure4. The yellow boxes in Figure4 represent the coverage of the available HiRISE images (up until NASA’s last released images on 9 June 2021) within the 3-sigma landing ellipses, where the yellow boxes that are shown as hatched fill are HiRISE images with off-nadir counterparts and the plain yellow boxes are HiRISE images without any repeat observations (i.e., only single views are available). As the proposed method only requires single images as inputs, the 3D mapping area for HiRISE can be greatly enlarged to cover other areas without any targeted or serendipitous observations being available. Remote Sens. 2021, 13, x FOR PEER REVIEW 7 of 28

Remote Sens. 2021, 13, 3270 7 of 26 requires single images as inputs, the 3D mapping area for HiRISE can be greatly enlarged to cover other areas without any targeted or serendipitous observations being available.

Figure 4. An overview of theFigure proposed 4. An 3D overview mapping areaof the at proposed Oxia Planum 3D (centredmapping at ar 18.239ea at ◦OxiaN, 24.368 Planum◦W), (centred showing at the 18.239°N, ESA Rosalind Franklin rover 202224.368°W), 1-sigma (red)showing and the 3-sigma ESA Rosalind (green) landing Franklin ellipse rover areas2022 and1-sigma the ExoMars(red) and team’s3-sigma geological (green) landing ellipse areas and the ExoMars team’s geological characterisation area (blue). Shown in the back- characterisation area (blue). Shown in the background are the co-registered input datasets of MOLA DTM (colourised and ground are the co-registered input datasets of MOLA DTM (colourised and hill-shaded), cropped hill-shaded), cropped HRSCHRSC MC-11W MC-11W level level 5 ORI 5 greyscaleORI greyscale mosaic, mosaic, CTX CTX ORI ORI mosaic, mosaic, and HiRISEand HiRISE image image footprints footprints (in (in yellow; with hatched outlinesyellow; referring with tohatched those availableoutlines refe withrring off-nadir to those stereo; available the footprints with off-n areadir relevant stereo; the to thefootprints NASA are release data of 9 June 2021). relevant to the NASA release data of 09 June 2021).

2.3. Practical issues Issues and and solutions Solutions Two practical issues issues occurred occurred while while achiev achievinging the theproposed proposed large large area MADNet area MADNet pro- processing.cessing. The The first first issue issue refers refers to the to the strength strength of ofthe the 3D 3D co-alignment co-alignment process process that that converts converts thethe MADNetMADNet output height height map map from from relative relative va valueslues to to absolute absolute values values with with respect respect to to a a low-resolutionlow-resolution referencereference DTM.DTM. It It should should be be noted noted that that raw raw MADNet MADNet output output height height map map (in tiles,(in tiles, each each 256 256×256× 256 pixels) pixels) do do not not contain contain any any information information for for the the contextual contextual terrain, terrain, e.g., large-scalee.g., large-scale slopes, slopes, and and such such information information needs needs to beto recoveredbe recovered from from the the low-resolution low-resolu- referencetion reference DTM DTM using using 3D 3D co-alignment. co-alignment. If If one one insufficiently insufficiently correctscorrects the raw raw MADNet MADNet outputs, this will result in inaccurate height height values values or or incorrect incorrect large-scale large-scale topographic topographic variations,variations, and when such errors are large, large, ti tilesles can can no no longer longer be be seamlessly seamlessly mosaiced. mosaiced. On On thethe otherother hand,hand, over-correctingover-correcting the the raw raw MADNet MADNet outputs outputs will will result result in in small-scale small-scale features fea- beingtures being smoothed smoothed out, and out, subsequently, and subsequently, local topographiclocal topographic variations variations becoming becoming extremely ex- low.tremely The low. strength The strength of the correction of the correction is mainly is controlledmainly controlled by two weightedby two weighted terms from terms the B-splinefrom the fitting B-spline algorithm fitting [algorithm6,23], i.e., the[6,23], weighted i.e., the distance weighted between distance the between target and the reference target 3D points and the weighted stiffness term to penalise transformations of neighbouring 3D points. However, in practice, such weights sometimes need to be manually adjusted depending on different situations, e.g., the surface roughness, topographic characteristics, Remote Sens. 2021, 13, x FOR PEER REVIEW 8 of 28

Remote Sens. 2021, 13, 3270 and reference 3D points and the weighted stiffness term to penalise transformations8 of of 26 neighbouring 3D points. However, in practice, such weights sometimes need to be manually adjusted depending on different situations, e.g., the surface roughness, topographic characteristics,image quality andimage effective quality resolutions.and effective As resolutions. supervised As processing supervised of processing each single-strip of each single-stripHiRISE DTM HiRISE is not feasible,DTM is wenot havefeasible, used we a fairly havestrict used set a fairly of 3D strict correction set of settings, 3D correction which settings,has a preference which has towards a preference smoother towards and lesssmoother erroneous and solutionsless erroneous instead solutions of sharper instead but ofmore sharper erroneous but more solutions, erroneous and solutions, consequently, and consequently, some minor smoothing some minor issues smoothing still exist issues for stillthe HiRISEexist for results. the HiRISE results. Figure 55 showsshows anan exampleexample ofof thisthis issue.issue. WeWe cancan seesee thethe differencedifference betweenbetween thethe 50cm/pixel50 cm/pixel HiRISE HiRISE MADNet MADNet DTMs DTMs with with a well-balanced a well-balanced 3D co-alignment 3D co-alignment (B), and (B), with and insufficientwith insufficient 3D co-alignment 3D co-alignment (A), and (A), inordinate and inordinate 3D co-alignment 3D co-alignment (C) with (C) respect with respect to the referenceto the reference DTM (12m/pixel DTM (12 CTX m/pixel MADNet CTX MADNetDTM in this DTM case in - D). this As case—D). shown by As the shown individ- by ualthe DTMs individual and DTMsthe measured and the profiles, measured insufficient profiles, insufficient3D co-alignment 3D co-alignment results in inaccurate results in DTMinaccurate heights DTM as well heights as visible as well artefacts as visible at artefacts the joints at thebetween joints adjacent between tiles adjacent (there tiles are (there nine tilesare nine mosaiced tiles mosaiced together togetherfor each of for the each HiRISE of the DTMs HiRISE that DTMs are shown that are in shownFigure in5), Figure whereas5), inordinatewhereas inordinate 3D co-alignment 3D co-alignment results results in seamless in seamless mosaicing mosaicing but but with with over-smoothed over-smoothed heights. In In large large area area processing, processing, a a good set of 3D co-alignment parameters generally works for most of the images (as the the images images are are geographically geographically close and thus have similar topographictopographic properties, properties, e.g., e.g., the the scales scales of of the the slopes are simi similar),lar), but occasionally, if an imageimage is significantly significantly different to the others,others, thethe parametersparameters willwill needneed toto bebe adjusted.adjusted.

Figure 5. 1st row: ExamplesFigure of a cropped5. 1st row: area Examples of the HiRISEof a cropped MADNet area of DTMs—nine the HiRISE tilesMADNet mosaiced DTMs—nine (A): insufficiently tiles mosaiced corrected against the reference(A: DTM; insufficiently (B): well-corrected corrected against against the the reference reference DT DTM;M; (B:C ):well-corrected over-corrected against against the the reference reference DTM; DTM) and the CTX MADNetC: over-corrected DTM mosaic (againstD): the the reference reference DTM DTM) for A,and B, the and CTX C; 2ndMADNet row: DTM measured mosaic DTM (D: profilesthe reference nd showing the difference betweenDTM A, for B, A, C, B, and and D C); and 2 the row: input measured CTX-co-registered DTM profiles HiRISE showing (ESP_037070_1985_RED) the difference between image. A, B, C, and D and the input CTX-co-registered HiRISE (ESP_037070_1985_RED) image. The second issue refers to the tiling process of MADNet. Due to the nature of the U-NetThe based second architecture issue refers and to the the limitation tiling process of memory of MADNet. of a graphics Due to processing the nature unit, of the each U- Netinput based image architecture needs to be and processed the limitation as small of overlapping memory of tiles, a graphics and the processing input tile size unit, is fixedeach inputat 512 image× 512 pixelsneeds. to For be each processed input image,as small the ov numbererlapping of tilestiles, that and need the input to be processed,tile size is fixeddepends at 512×512 on the pixels. number For of each overlapping input image, pixels the we number choose. of tiles Generally, that need a full-strip to be processed, HiRISE dependsimage needs on the to benumber divided of intooverlapping 6000 to 12,000 pixels tiles we usingchoose. 100 Generally, pixels of overlap,a full-strip whereas HiRISE a imagefull-strip needs CTX to image be divided only requires into 6,000 200 to to 12,000 300 tiles. tiles Having using 100 more pixels overlapping of overlap, pixels whereas results a full-stripin a better CTX full-strip image only mosaic requires (seamless 200 to mosaic) 300 tiles. and Having better more tolerance overlapping of errors pixels produced results inby aindividual better full-strip tiles. mosaic However, (seamless when mosaic) the number and better of overlapping tolerance of pixels errors increases, produced theby number of tiles increases, and consequently the processing time and required storage space also increases. Having insufficient overlapping pixels could result in seamline artefacts around the joints of adjacent tiles. Using 80 to 120 pixels of overlap works for most of the Remote Sens. 2021, 13, x FOR PEER REVIEW 9 of 28

individual tiles. However, when the number of overlapping pixels increases, the number Remote Sens. 2021, 13, 3270 9 of 26 of tiles increases, and consequently the processing time and required storage space also increases. Having insufficient overlapping pixels could result in seamline artefacts around the joints of adjacent tiles. Using 80 to 120 pixels of overlap works for most of the images, butimages, in steep but slope in steep areas, slope this areas, number this needs number to be needs increased to be to increased achieve seamless to achieve mosaicing. seamless Currently,mosaicing. we Currently, do not have we do an not automated have an automated way of selecting way of the selecting optimised the optimised number of number over- lappingof overlapping pixels, so pixels, a default so a number default numberof 100 pixels of 100 is used, pixels which is used, results which in resultsmostly inseamless mostly mosaicingseamless mosaicing but occasional but occasional seamline artefacts. seamline artefacts. FigureFigure 66 showsshows anan exampleexample ofof thisthis issueissue aroundaround aa slopedsloped areaarea inin thethe HiRISEHiRISE imageimage (ESP_037703_1980_RED).(ESP_037703_1980_RED). Although Although the the tiling tiling artefacts artefacts are are comparably comparably minor minor (causing a ~10cm~10 cm height height variation variation according according to to the the measured measured profiles) proﬁles) and and almost almost invisible invisible in the DTMs,DTMs, they they are are picked picked up up in the hill-shaded im images.ages. We We can observe that increasing the numbernumber of of overlapping overlapping pixels pixels of of adjacent adjacent tiles tiles (from (from 80 80 pixels to 100 pixels) in MADNet processingprocessing helps in reducing the impact of this artefact.

Figure 6. Examples of the tilingFigure artefact 6. Examples of the HiRISE of the tiling (ESP_037703_1980_RED) artefact of the HiRISE MADNet (ESP_037703_1980_RED) DTM and the effect MADNet (including DTM a and measured profile crossing thethe tiling effect artefact) (including of using a measured a larger profile number crossing of overlapping the tiling pixelsartefact) between of using each a larger tile. It number should of be over- noted that the colourised DTMslapping and pixels hill-shaded between images each show tile. anIt should identical bearea noted of interest.that the colourised The profile DTMs location and that hill-shaded is shown im- st nd in the 1st row/2nd columnages is identical show an to theidentical location area of of the interest. measured The profile profile in location the 2nd that row/2nd is shown column. in the 1 row/2 column is identical to the location of the measured profile in the 2nd row/2nd column. 2.4. Study Site 2.4. Study site All results shown here are made for the Rosalind Franklin ExoMars 2022 rover’s All results shown here are made for the Rosalind Franklin ExoMars 2022 rover’s land- landing site at Oxia Planum [28–31]. This site is a 200 km wide clay bearing plain centred ing site at Oxia Planum [28–31]. This site is a 200km wide clay bearing plain centred on on 18.239◦ N, 24.368◦ W (see Figure4) inside the United States Geological Survey (USGS)’s 18.239°N,Oxia Palus 24.368°W (MC-11) (see quadrangle Figure 4) of inside Mars. the The United landing States site area Geological straddles Survey the dichotomy (USGS)’s Oxiaboundary Palus of(MC-11) Mars, whichquadrangle separates of Mars. the northern The landing lowlands site area from straddles the southern the dichotomy highlands. boundaryThe site slopes of Mars, towards which theseparates north andthe northern hosts mineralogical lowlands from (iron-magnesium the southern highlands. rich clay Theminerals) site slopes and geomorphologicaltowards the north evidenceand hosts of mineralogical liquid water in (iron-magnesium the ancient past [rich28–31 clay], which min- erals)motivated and geomorphological the choice of this site evidence for this of mission liquid whosewater in principal the ancient objective past [28–31], is to search which for motivatedpotential biomarkers the choice of indicating this site for past this life mission on Mars. whose principal objective is to search for potential biomarkers indicating past life on Mars. 3. Results 3. ResultsIn this work, three main sets of 3D products are produced. These are: (1) The 25 m/pixel HRSC MADNet DTM mosaic (3D co-aligned with the 463 m/pixel MOLA DTM) covering an area of about 197 km × 182 km of the landing site.

Remote Sens. 2021, 13, 3270 10 of 26

(2) The 12 m/pixel CTX MADNet DTM mosaic (3D co-aligned with the 25 m/pixel HRSC MADNet DTM mosaic) and 6 m/pixel CTX ORIs (co-registered with the 12.5 m/pixel HRSC MC-11W level 5 ORI mosaic — “HMC_11W24_nd5”), covering an area of about 114 km × 117 km of the landing site. (3) The 50 cm/pixel HiRISE MADNet DTM mosaic (3D co-aligned with the 12 m/pixel CTX MADNet DTM mosaic) and 25 cm/pixel HiRISE ORIs (co-registered with the 6 m/pixel CTX ORIs), covering about a 74.3 km × 86.3 km area of the 3-sigma landing ellipses and part of the ExoMars team’s geological characterisation area [33]. In order to maintain a high level of geospatial accuracy of all 3D mapping products produced from this work, precise co-registrations are required between the different resolutions. We achieve cascaded image co-registration and 3D co-alignments with respect to the MOLA DTM and the HRSC MC-11W level 5 ORI mosaic [3], which are considered the most precise and globally consistent topographic geospatial reference of Mars to date, respectively, as our baseline references. In the following three subsections, a qualitative assessment of the MADNet DTM mosaics, alongside the cascaded image co-registration and 3D co-alignment accuracies, is presented for each dataset.

3.1. HRSC Results Example tiles of the resultant 25 m/pixel HRSC MADNet DTM mosaic are shown in Figure7. We can observe improved quality, apparent improvement of effective resolution, and reduced noise/artefact in the HRSC MADNet DTM mosaic in comparison to the original HRSC MC-11W level 5 DTM mosaic. From the provided zoom-in views, ﬁne-scale 3D features, e.g., craters and peaks, are better resolved and have more realistic shapes in the HRSC MADNet DTM mosaic compared to the original HRSC MC-11W level 5 DTM mosaic. In particular, the zoom-in view A of Figure7 shows that the MADNet process is robust against issues in the input image mosaic, i.e., the seamline artefact and obvious resolution changes (caused by mosaicing two HRSC orbits that have different inherent resolutions). The MADNet DTM prediction was not affected by these issues because we cannot observe any artefacts at the location of the seam line. While the HRSC MC-11W level 5 ORI and DTM mosaics are considered to be precisely co-aligned with the MOLA data, there are still some remaining vertical residuals. In order to achieve a high-standard of MOLA congruence/co-alignment for the HRSC MADNet DTM mosaic and to avoid the propagation of systematic errors from HRSC to the subsequent CTX and HiRISE MADNet DTM mosaics, we perform a further 3D co-alignment process on top of the HRSC MC-11W level 5 DTM mosaic. The difference maps (at the scale of 500 m/pixel) and scatter plots of the HRSC MC- 11W level 5 DTM mosaic (before 3D co-alignment), the HRSC MC-11W level 5 DTM mosaic (after 3D co-alignment), and the HRSC MADNet DTM mosaic, in comparison with the reference MOLA DTM, are shown in Figure8. We can observe that the HRSC MC-11W level 5 DTM mosaic (after 3D co-alignment) and the HRSC MADNet DTM mosaic shows better overall 3D agreements with the MOLA DTM and reduced systematic errors in comparison to the original HRSC MC-11W level 5 DTM mosaic (before 3D co-alignment). The mean and StdDev (the standard deviation of height differences–provided alongside the difference maps) values have also been reduced slightly. Remote Sens. 2021, 13, 3270 11 of 26 Remote Sens. 2021, 13, x FOR PEER REVIEW 11 of 28

Figure 7. The 25 m/pixel HRSCFigure MADNet 7. The DTM25m/pixel mosaic HRSC (bottom MADNet row) comparedDTM mosaic to the (bottom 12.5 m/pixel row) compared HRSC MC-11W to the 12.5m/pixel level 5 ORI mosaic (HMC_11W24_ND5,HRSC top MC-11W row) and level the 5 50ORI m/pixel mosaic HRSC (HMC_11W24_N MC-11W levelD5, 5top DTM row) mosaic and the (HMC_11W20_DA5, 50m/pixel HRSC MC- middle row). The 1st column11W shows level the 5 overview DTM mosaic area, and(HMC_11W20_DA5, the 2nd to 5th columns middle show row). the The zoom-in 1st column views shows of four the different overview nd th areas (locations are labelledarea, in top-left and the subﬁgure). 2 to 5 columns Please refershowto the Supplementary zoom-in views Materialsof four different for each areas individual (locations ﬁgure are labelled at full resolution. in top-left subfigure). Please refer to Supplementary Materials for each individual figure at full resolution.

3.2. CTXWhile Results the HRSC MC-11W level 5 ORI and DTM mosaics are considered to be pre- ciselyThe co-aligned image IDs with of thethe inputMOLA CTX data, images there are are listed still some in Table remaining1. The resultant vertical 12 residuals. m/pixel CTXIn order MADNet to achieve DTM a mosaichigh-standard is shown of inMOLA Figure congruence/co-alignment9. We can observe improved for the quality,HRSC apparentMADNet resolution,DTM mosaic and and reduced to avoid noise/artefacts the propagation in of the systematic CTX MADNet errors DTMfrom mosaicHRSC to in comparisonthe subsequent to theCTX CTX and CASP-GO HiRISE MADNet DTMs (produced DTM mosaics, using we photogrammetric perform a further methods). 3D co- Fromalignment the provided process on zoom-in top of the views, HRSC we MC-11W can observe level more5 DTM ﬁne mosaic. scale 3D details of small- scaleThe features difference and reduction maps (at the of small-scalescale of 500m/pixel) noise in theand CTXscatter MADNet plots of DTMthe HRSC mosaic. MC- It should11W level be noted5 DTM that mosaic in zoom-in (before view3D co-alig D ofnment), Figure9 ,the the HRSC bumpy MC-11W “features” level shown 5 DTM in mo- the CTXsaic (after CASP-GO 3D co-alignment), DTM are actually and the matching HRSC MADNet artefacts. DTM If we mosaic, compare in thecomparison same location with withinthe reference the ORI, MOLA such DTM, “features” are shown at their in scaleFigure do 8. notWe exist.can observe In addition, that the the HRSC two smallMC- and11W connected level 5 DTM craters mosaic at the (after western 3D co-alignment) rim of the central and the crater HRSC are notMADNet discernible DTM frommosaic the CASP-GOshows better DTM. overall In contrast, 3D agreements the CTX with MADNet the MOLA DTM mosaicDTM and shows reduced much systematic better quality errors for thein comparison central crater to andthe theoriginal two smallHRSC craters MC-11W at the level western 5 DTM rim mosaic are identiﬁable. (before 3D Moreover,co-align- zoom-inment). The view mean D of and Figure StdDev9 shows (the thatstandard the MADNet deviation process of height is robustdifferences against – provided issues of smallalongside amounts the difference of missing maps) data withinvalues have the input also been image, reduced e.g., severe slightly. shading effects, where the photogrammetric methods are likely to fail or produce artefacts. It should also be noted that the input CTX ORIs (for MADNet DTM prediction) sometimes contain small-scale artefacts, which could either come from the original CTX data (e.g., patterned stripe lines– see CTX J05_046934_1985_XN_18N025W in Supplementary Materials) or come from the photogrammetric process (e.g., small gaps, local distortions due to image co-registration and transformation), and MADNet has shown moderate robustness to such issues.

Remote Sens. 2021, 13, 3270 12 of 26

In order to achieve a high standard of spatial and vertical accuracy for the CTX MADNet DTM mosaic with respect to MOLA, the CTX CASP-GO ORIs (CTX MADNet input images) are ﬁrstly co-registered with the HRSC MC-11W level 5 ORI mosaic, and then the CTX CASP-GO DTMs (CTX MADNet input reference DTMs) are 3D co-aligned with the HRSC MADNet DTM mosaic, which itself is co-aligned with MOLA. Examples showing the image co-registration accuracy between CTX ORIs and the HRSC MC-11W level 5 ORI mosaic or other neighbouring CTX ORIs are presented in Figure 10. We can observe fairly good agreements for both the CTX-to-HRSC and CTX-to-CTX comparisons after the image co-registration process, whereas 50–100 m local displacements are observable before the image co-registration process. The CTX-to-HRSC image co-registration process guarantees Remote Sens. 2021, 13, x FOR PEER REVIEW 12 of 28 a high level of geospatial accuracy of the CTX MADNet DTM processing, as well as being a useful base for subsequent HiRISE processing.

Figure 8. Difference maps andFigure scatter 8. plotsDifference of the maps HRSC-to-MOLA and scatter plots comparisons of the HRSC-to-MOLA (averaged) at the comparisons scale of 500 (averaged) m/pixel for at the the original HRSC MC-11Wscale level of 5 DTM500m/pixel mosaic for (HMC_11W20_DA5), the original HRSC MC the-11W HRSC level MC-11W 5 DTM levelmosaic 5 DTM (HMC_11W20_DA5), mosaic after 3D the co-alignment, and the HRSC MADNetHRSC MC-11W DTM mosaic.level 5 DTM It should mosaic be notedafter 3D that co-alignment, the mean and and standard the HRSC deviation MADNet (StdDev) DTM mosaic. are shown on the difference mapsIt (1stshould row). be noted that the mean and standard deviation (StdDev) are shown on the difference maps (1st row).

Table 1. Input CTX3.2. CTX image results IDs (12 in total) that are co-registered and used as MADNet inputs. F03_037070_1980_XN_18N024WThe image J01_045101_1981_XN_18N023W IDs of the input CTX images are listed in J05_046934_1985_XN_18N025W Table 1. The resultant 12m/pixel F03_037136_1977_XN_17N023WCTX MADNet J02_045378_1981_XN_18N023WDTM mosaic is shown in Figure 9. We can F17_042622_1981_XN_18N024W observe improved quality, apparent resolution, and reduced noise/artefacts in the CTX MADNet DTM mosaic in com- F23_044811_1985_XN_18N024Wparison to the J01_045167_1983_XN_18N024WCTX CASP-GO DTMs (produced using photogrammetric J14_050138_1986_XN_18N025W methods). From F23_044956_1984_XN_18N024Wthe provided zoom-in J03_045800_1983_XN_18N024W views, we can observe more fine K05_055518_1986_XN_18N025Wscale 3D details of small-scale features and reduction of small-scale noise in the CTX MADNet DTM mosaic. It should be noted that in zoom-in view D of Figure 9, the bumpy “features” shown in the CTX CASP- GO DTM are actually matching artefacts. If we compare the same location within the ORI, such “features” at their scale do not exist. In addition, the two small and connected craters at the western rim of the central crater are not discernible from the CASP-GO DTM. In contrast, the CTX MADNet DTM mosaic shows much better quality for the central crater and the two small craters at the western rim are identifiable. Moreover, zoom-in view D of Figure 9 shows that the MADNet process is robust against issues of small amounts of missing data within the input image, e.g., severe shading effects, where the photogrammetric methods are likely to fail or produce artefacts. It should also be noted that the input CTX ORIs (for MADNet DTM prediction) sometimes contain small-scale artefacts, which could either come from the original CTX data (e.g., patterned stripe lines – see CTX J05_046934_1985_XN_18N025W in Supplementary Materials) or come from the photogrammetric process (e.g., small gaps, local distortions due to image co-registration and transformation), and MADNet has shown moderate robustness to such issues.

Table 1. Input CTX image IDs (12 in total) that are co-registered and used as MADNet inputs.

Remote Sens. 2021, 13, x FOR PEER REVIEW 13 of 28

F03_037070_1980_XN_18N02 J01_045101_1981_XN_18N02 J05_046934_1985_XN_18N02 4W 3W 5W F03_037136_1977_XN_17N02 J02_045378_1981_XN_18N02 F17_042622_1981_XN_18N02 3W 3W 4W F23_044811_1985_XN_18N02 J01_045167_1983_XN_18N02 J14_050138_1986_XN_18N02 4W 4W 5W F23_044956_1984_XN_18N02 J03_045800_1983_XN_18N02 K05_055518_1986_XN_18N02 Remote Sens. 2021, 13, 3270 13 of 26 4W 4W 5W

Remote Sens. 2021, 13, x FOR PEER REVIEW 14 of 28

Figure 9. The resultant 12 m/pixelFigure CTX9. The MADNet resultant DTM12m/pixel mosaic CTX in MADNet comparison DTM to mosa theic 18 in m/pixel comparison CTX to CASP-GO the 18m/pixel DTMs CTX (photogrammetric results) andCASP-GO the 6 m/pixel DTMs CTX (photogrammetric CASP-GO ORIs. results) The 1st and column the 6m/pixel shows CTX overviews CASP-GO of the ORIs. ORIs The and 1st DTMs, column the image co-registration process. The CTX-to-HRSCnd imageth co-registration process guar- and the 2nd to the 5th columnsshows show overviews the zoom-in of the views ORIs of fourand DTMs, different and areas the (locations2 to the 5 are columns labelled show in the the top-left zoom-in subﬁgure views of fourantees different a high areas level (locations of geospatial are labelled accuracy in the of top-left the CTX subfigure MADNet from DTMleft-to-right). processing, Please as refer well to from left-to-right). Please refer to Supplementary Materials for examination of each individual ﬁgure at full resolution. Supplementaryas being a useful Materials base for for su examinationbsequent HiRISEof each individual processing. figure at full resolution.

In order to achieve a high standard of spatial and vertical accuracy for the CTX MAD- Net DTM mosaic with respect to MOLA, the CTX CASP-GO ORIs (CTX MADNet input images) are firstly co-registered with the HRSC MC-11W level 5 ORI mosaic, and then the CTX CASP-GO DTMs (CTX MADNet input reference DTMs) are 3D co-aligned with the HRSC MADNet DTM mosaic, which itself is co-aligned with MOLA. Examples showing the image co-registration accuracy between CTX ORIs and the HRSC MC-11W level 5 ORI mosaic or other neighbouring CTX ORIs are presented in Figure 10. We can observe fairly good agreements for both the CTX-to-HRSC and CTX-to-CTX comparisons after the image co-registration process, whereas 50–100m local displacements are observable before

Figure 10. Examples of the inputFigure CTX 10. ORIs, Examples before of CTX-to-HRSC the input CTX imageORIs, co-registrationbefore CTX-to-HRSC (1st row) image and co-registration after CTX-to-HRSC (1st row) nd image co-registration (2nd row).and The after example CTX-to-HRSC CTX ORIs image are co-registration in side-by-side (2 views row). with The example the corresponding CTX ORIs are HRSC in side-by-side MC-11W views with the corresponding HRSC MC-11W level 5 ORI mosaic (1st and 3rd columns) and in side- level 5 ORI mosaic (1st and 3rd columns) and in side-by-side views with the neighbouring CTX ORIs (2nd and 4th columns). by-side views with the neighbouring CTX ORIs (2nd and 4th columns).

The difference maps (at the scale of 50m/pixel) and scatter plots of the CTX CASP- GO (photogrammetric) DTM mosaic (before 3D co-alignment), the CTX CASP-GO DTM mosaic (after 3D co-alignment), and the CTX MADNet DTM mosaic, in comparison to the reference HRSC MADNet DTM mosaic (MOLA co-aligned), are shown in Figure 11. We can observe that the CTX CASP-GO DTM mosaic (after 3D co-alignment) and the CTX MADNet DTM mosaic, show much better overall 3D agreements with the HRSC MADNet DTM mosaic (MOLA co-aligned), in comparison to the CTX CASP-GO DTM mosaic (before 3D co-alignment). The mean and StdDev values, provided alongside the difference maps, have also been significantly reduced. It should be noted that the photogrammetric DTMs from HRSC and CTX are used as the “intermediate” reference DTMs to co-align the HRSC and MADNet DTM in this work, considering the smaller resolution gap (in comparison to using the MOLA DTM as reference) should in theory achieve better 3D co-alignment accuracy. However, lower-resolution reference DTMs (without photogrammetric processing) could also be used as the inputs of the MADNet processing – this is not demonstrated with CTX and HRSC but is demonstrated with HiRISE in this work, where no photogrammetric processing of HiRISE images is involved.

Remote Sens. 2021, 13, 3270 14 of 26

Figure 11. Difference maps and scatter plots of the CTX-to-HRSC comparisons (averaged) at the scale of 50 m/pixel for (1st column) the CTX CASP-GO (photogrammetric) DTM mosaic (before 3D co-alignment), (2nd column) the CTX CASP-GO DTM mosaic (after 3D co-alignment), and (3rd column) the CTX MADNet DTM mosaic. It should be noted that the mean and standard deviation (StdDev) are shown on the difference maps (1st row).

It should be noted that the photogrammetric DTMs from HRSC and CTX are used as the “intermediate” reference DTMs to co-align the HRSC and MADNet DTM in this work, considering the smaller resolution gap (in comparison to using the MOLA DTM as reference) should in theory achieve better 3D co-alignment accuracy. However, lower- resolution reference DTMs (without photogrammetric processing) could also be used as the inputs of the MADNet processing–this is not demonstrated with CTX and HRSC but is demonstrated with HiRISE in this work, where no photogrammetric processing of HiRISE images is involved.

3.3. HiRISE Results The image IDs of the input HiRISE images are listed in Table2. The resultant 50 cm/pixel HiRISE MADNet DTM mosaic is shown in Figure 12. In order to demonstrate the quality of the DTM, HiRISE MADNet DTM mosaic (made with 44 single- strip DTMs) are compared against the available HiRISE PDS DTMs (5 in total; image IDs are DTEEC_003195_1985_002694_1985_L01; DTEEC_036925_1985_037558_1985_L01; DTEEC_037070_1985_037136_1985_L01; DTEEC_039299_1985_047501_1985_L01; DTEEC_ Remote Sens. 2021, 13, 3270 15 of 26

042134_1985_053962_1985_L01). From the overview, we can observe that there is no systematic bias for the HiRISE MADNet DTM mosaic. From the zoom-in views, we can observe superior DTM quality, more details, higher effective resolution, and reduced noise/artefacts from the HiRISE MADNet DTM mosaic in comparison to the HiRISE PDS DTMs. It should also be noted that some of the input HiRISE images (MADNet input) contain artefacts, such as striping, seamlines, and missing data, however, MADNet shows fairly good robustness to such issues if they appear to be minor (approximately less than 1000 pixels within a 512 × 512 pixels tile). In order to achieve a high standard of spatial and vertical accuracy for the HiRISE MADNet DTM mosaic with respect to MOLA and HRSC, the input HiRISE images are firstly orthorectified and co-registered with the overlapping CTX CASP-GO ORIs, which themselves are co-registered with the HRSC MC-11W level 5 ORI mosaic. Secondly, the HiRISE MADNet DTM mosaic is spatially corrected with respect to the co-registered HiRISE ORIs. Finally, the HiRISE MADNet DTM mosaic is 3D co-aligned with the CTX MADNet DTM mosaic, which itself is co-aligned with the HRSC MADNet DTM mosaic, which itself is co-aligned with MOLA. Examples showing the image co-registration accuracy between HiRISE ORIs and the CTX ORIs or other neighbouring HiRISE ORIs, are presented in Figure 13. We can observe significant improvements between “before image co-registration” and “after image co-registration”, for both HiRISE-to-CTX and HiRISE-to-HiRISE comparison examples, as local displacements have been reduced from ~100–200 m to subpixel level of the referencing CTX image (≤3 m). The HiRISE-to-CTX image co-registration process guarantees a high level of geospatial accuracy of the resultant HiRISE MADNet DTM mosaic. The difference maps (at 20 m/pixel) and scatter plots of the HiRISE PDS DTMs and the HiRISE MADNet DTM mosaic, in comparison to the reference CTX MADNet DTM mosaic (HRSC/MOLA co-aligned), are shown in Figure 14. We can observe that the HiRISE MADNet DTM mosaic shows much better overall 3D agreement with the CTX reference in comparison to the HiRISE PDS DTMs. The mean and StdDev values, provided alongside the difference maps, also demonstrate significant improvements. It should be noted that the HiRISE and CTX DTMs are down-sampled and compared at the scale of 20 m/pixel, thus eliminating fine-scale variations, and only show large-scale errors that are greater than 20 m within the difference maps and scatter plots.

Table 2. Input HiRISE image IDs that are co-registered and used as MADNet inputs (44 images resulting in 44 HiRISE MADNet DTMs that are subsequently mosaiced to produce the ﬁnal HiRISE MADNet DTM mosaic). N.B. left-to-right and top-to-bottom shows the order of the mosaicing priority.

ESP_037558_1985_RED ESP_037703_1980_RED ESP_039154_1985_RED ESP_039721_1980_RED ESP_039932_1980_RED ESP_040433_1985_RED ESP_040921_1985_RED ESP_042134_1985_RED ESP_042701_1980_RED ESP_044178_1985_RED ESP_044257_1980_RED ESP_044679_1985_RED ESP_047079_1985_RED ESP_047435_1985_RED ESP_047501_1985_RED ESP_049927_1980_RED ESP_051272_1985_RED ESP_052828_1980_RED ESP_055175_1980_RED ESP_055241_1980_RED ESP_057747_1985_RED ESP_058090_1985_RED ESP_058169_1985_RED ESP_059659_1980_RED ESP_059725_1985_RED ESP_060714_1985_RED ESP_066833_1980_RED ESP_067044_1990_RED ESP_067585_1990_RED ESP_068258_1980_RED ESP_068456_1985_RED PSP_003195_1985_RED PSP_007019_1980_RED PSP_009735_1985_RED ESP_037070_1985_RED ESP_046512_1980_RED ESP_051206_1985_RED ESP_019084_1980_RED ESP_043268_1980_RED ESP_062402_1985_RED ESP_065448_1985_RED ESP_067717_1985_RED PSP_002694_1985_RED ESP_045167_1980_RED Remote Sens. 2021, 13, 3270 16 of 26 Remote Sens. 2021, 13, x FOR PEER REVIEW 17 of 28

Figure 12. The resultant 50 cm/pixelFigure 12. HiRISE The resultant MADNet 50cm/pixel DTM mosaic HiRISE (consisting MADNet of 44 DT HiRISEM mosaic MADNet (consisting DTMs) of in 44 comparison HiRISE MAD- with the 1 m/pixel HiRISENet PDS DTMs) DTMs in (photogrammetric comparison with the results) 1m/pixel and HiRI theSE 25 PDS cm/pixel DTMs HiRISE (photogrammetric ORIs. 1st rowresults) shows and the 25cm/pixel HiRISE ORIs. 1st row shows overviews of the ORIs and DTMs, and the 2nd to the 4th rows overviews of the ORIs and DTMs, and the 2nd to the 4th rows show the zoom-in views of six different areas (locations are show the zoom-in views of six different areas (locations are labelled in the 2nd subfigure of the 1st labelled in the 2nd subﬁgure of the 1st row). Please refer to Supplementary Materials for details of each individual ﬁgure at row). Please refer to Supplementary Materials for details of each individual figure at full resolution. full resolution. In order to achieve a high standard of spatial and vertical accuracy for the HiRISE MADNet DTM mosaic with respect to MOLA and HRSC, the input HiRISE images are firstly orthorectified and co-registered with the overlapping CTX CASP-GO ORIs, which themselves are co-registered with the HRSC MC-11W level 5 ORI mosaic. Secondly, the HiRISE MADNet DTM mosaic is spatially corrected with respect to the co-registered HiRISE ORIs. Finally, the HiRISE MADNet DTM mosaic is 3D co-aligned with the CTX MADNet DTM mosaic, which itself is co-aligned with the HRSC MADNet DTM mosaic, which itself is co-aligned with MOLA. Examples showing the image co-registration accuracy between HiRISE ORIs and the CTX ORIs or other neighbouring HiRISE ORIs, are presented in Figure 13. We can observe significant improvements between “before image co-registration” and “after image co-registration”, for both HiRISE-to-CTX and HiRISE- to-HiRISE comparison examples, as local displacements have been reduced from ~100– 200m to subpixel level of the referencing CTX image (≤3m). The HiRISE-to-CTX image co-

Remote Sens. 2021, 13, x FOR PEER REVIEW 18 of 28

Remote Sens. 2021, 13, 3270 17 of 26 registration process guarantees a high level of geospatial accuracy of the resultant HiRISE MADNet DTM mosaic.

Figure 13. Examples of the inputFigure HiRISE 13. Examples ORIs, before of the HiRISE-to-CTX input HiRISE image ORIs, co-registrationbefore HiRISE-to-CTX (1st row), image and after co-registration HiRISE-to- (1st CTX image co-registration (2ndrow), row). and Theafter exampleHiRISE-to-CTX HiRISE image ORIs areco-registration in side-by-side (2nd viewsrow). withThe example the corresponding HiRISE ORIs CTX are in st rd ORIs (HRSC co-registered; 1stside-by-side and 3rd columns) views with and the in corresponding side-by-side views CTX withORIs the(HRSC neighbouring co-registered; HiRISE 1 and ORIs 3 columns) (2nd and and in side-by-side views with the neighbouring HiRISE ORIs (2nd and 4th columns). 4th columns). The difference maps (at 20m/pixel) and scatter plots of the HiRISE PDS DTMs and 3.4.the AdditionalHiRISE MADNet Assessments DTM for mosaic, the Resultant in compar HiRISEison and to the CTX reference MADNet CTX DTM MADNet Mosaics DTM mosaicIn Figure(HRSC/MOLA 15, we show co-aligned), the resultant are 50shown cm/pixel in Figure HiRISE 14. MADNet We can DTMobserve mosaic that using the hill-shadedHiRISE MADNet images DTM (with mosaic a vertical shows exaggeration much be factortter overall of 3) using 3D agreement different lighting with the sources, CTX i.e.,reference 45◦, 135 in◦ ,comparison 225◦, and 315 to ◦theof solarHiRISE azimuth PDS DTMs. angles The and mean all at and 30◦ ofStdDev solar elevation,values, pro- for fourvided selected alongside areas the covering difference a large maps, crater, also small-sized demonstrate craters, significant TARs (Transverse improvements. Aeolian It Ridges),should be and noted small that peaks. the HiRISE In comparison and CTX withDTMs the are 25 down-sampled cm/pixel HiRISE andORIs, compared we observe at the thatscale the of 20m/pixel, MADNet DTMthus eliminating mosaic has fine-scale captured variations, fine scale and topographic only show details large-scale at a similarerrors effectivethat are greater resolution than to20m the within image the itself. difference Hill-shaded maps and images scatter from plots. four different angles demonstrate that there are no obvious errors or artefacts for the selected features. We have included the full-resolution hill-shaded image, a DTM roughness map, and a slope image in the Supplementary Materials. In Figure 16, we show profile measurements of an example area with a variety of small-sized craters (from 10 m to 30 m diameter). The profiles are measured from the resultant 50 cm/pixel HiRISE MADNet DTM, the 1 m/pixel HiRISE PDS DTM (DTEEC_003195_1985_002694_1985_L01), the 12 m/pixel CTX MADNet DTM, and the 25 m/pixel HRSC MADNet DTM (co-aligned with MOLA). In general, we observe fairly good correlations between all four measured DTMs. The maximum difference between the HiRISE MADNet DTM and the HRSC MADNet DTM is ±10 m (in Profile-1), and the maximum difference between the HiRISE MADNet DTM and the CTX MADNet DTM is ±7 m (in Profile-3). In addition, the HiRISE MADNet DTM shows the best effective resolution compared to the other DTM products and are able to capture many fine-scale features, such as small craters (in all profiles) down to ~5 m in diameter and small peaks (e.g., Profile-5) for sizes down to ~5 m.

Remote Sens. 2021, 13, 3270 18 of 26

Figure 14. Difference maps and scatter plots of the HiRISE-to-CTX elevation comparisons (averaged) at the scale of 12 m/pixel for the HiRISE PDS (photogrammetric) DTM mosaic and the HiRISE MADNet DTM mosaic. It should be noted that the mean and standard deviation (StdDev) are shown on the difference maps (1st row). Remote Sens. 2021, 13, 3270 19 of 26

Figure 15. Examples of the 25 cm/pixel HiRISE ORI (1st row), 50 cm/pixel HiRISE MADNet DTM mosaic (2nd row), and hill-shaded images (3rd–6th rows), using 45◦, 135◦, 225◦, and 315◦, of azimuth angles and 30◦ of illumination elevation with a vertical exaggeration factor of 3, of the resultant HiRISE MADNet DTM mosaic for four cropped areas (1st–4th columns) showing a large crater and small craters, layers, and hills. Remote Sens. 2021, 13, 3270 20 of 26

Figure 16. Profile measurements of the resultant 50 cm/pixel HiRISE MADNet DTM mosaic, the 1 m/pixel HiRISE PDS DTM (DTEEC_003195_1985_002694_1985_L01), and the 12 m/pixel CTX MADNet DTM (co-aligned with HRSC/MOLA), using an example area with small-sized craters (from 10 m to 30 m diameter). The units of the y axis of all profile figures are in metres and the units of the x axis of all profile figures are normalised latitude and longitude degrees (counting from 0◦). It should be noted that the MADNet DTM (2nd row) is colourised using the same colour key shown in the HiRISE PDS DTM (1st row). It should also be noted that the range of heights for Profiles 1–5 is a maximum of 14 m and that of Profiles 6–10 is 6 m. Remote Sens. 2021, 13, 3270 21 of 26

On the other hand, we can also observe two types of issues. Firstly, there are some large-scale height differences (of around 3 m to 8 m) between the CTX and HRSC MADNet DTMs. In particular, the behaviour of the higher resolution DTMs (CTX and HiRISE), which display opposing large-scale trends compared with the lower resolution DTM (HRSC), could either be the result of local mis-co-alignment, an inherited photogrammetric error from the HRSC MC-11 DTM, or simply a consequence of the resolution differences between the HRSC and CTX images. In this case, the features from the CTX image are not being properly recorded by HRSC. Although we have achieved fairly good 3D co-alignment accuracy between the CTX and HRSC MADNet DTMs at a much larger scale, as shown in Section 3.2, when zooming into local areas, like Figure 16, height differences still exist. Such height differences (less than 24 m for HRSC-MOLA according to Figure8 and less than 8 m for CTX-HRSC according to Figure 11) between the baseline DTMs can lower the absolute height accuracy of the HiRISE MADNet DTM even though the HiRISE MADNet DTM has shown “perfect” agreements with the CTX MADNet DTM. Secondly, we can observe some degradation of features at an intermediate scale (between 120 m and 240 m). For example, the larger crater (in Proﬁle-4 and Proﬁle-5) has more realistic height variations from the HiRISE PDS DTM. Although the HiRISE MADNet DTM has captured much more small-sized craters (with diameter less than 100 m), its large-scale height information is purely dependent on the reference DTM. This is because within each MADNet inference process, the usable input information for MADNet is limited to a tile of 512 × 512 HiRISE pixels (equal to 128 m × 128 m) squared area, which means for craters that are larger than 128 m × 128 m (equal to ~20 × 20 CTX pixels), their reconstruction quality is actually based on the MADNet processing of the CTX images. From Figure 16, we can observe that the smallest craters we can retrieve through MADNet applied to HiRISE are about 2 m in diameter, and for a good quality retrieval, the diameter is probably about 10 m. This approximates to an area of 40 × 40 pixels of the HiRISE image. As the same rule also applies to CTX images, we should therefore have good reconstructions of features (e.g., craters) that are larger than 240 m × 240 m from CTX (i.e., 40 × 40 CTX pixels). Counting the HiRISE tile size being 128 m × 128 m, HiRISE MADNet DTM theoretically has a weakness for detecting features at this intermediate scale, i.e., between 128 m × 128 m and 240 m × 240 m. This is generally not an issue for MADNet processing of CTX and CaSSIS images as the resolution gap between the CTX/CaSSIS images and the reference HRSC DTMs is much smaller compared to the resolution gap between the HiRISE images and the reference CTX DTMs. However, this could be an issue for MADNet processing of HRSC images if HRSC photogrammetric DTMs are not available, and where MOLA has to be used. We are exploring potential solutions to this issue (see discussions in Section 4.4).

4. Discussion 4.1. Product Access The resultant HRSC, CTX, and HiRISE DTM mosaics and ORIs for the Oxia Planum landing site will be made available through the ESA Guest Storage Facility (GSF; see https: //www.cosmos.esa.int/web/psa/ucl-mssl_meta-gsf accessed on 22 July 2021). Before this becomes available, please contact the authors for access to the products.

4.2. Extensibility with Other Satasets or Area The original MADNet work [24] was demonstrated using the 4 m/pixel CaSSIS images. With only minor changes and parameter tuning, the same DTM production system has been applied to the 12.5 m/pixel HRSC, 6 m/pixel CTX, and 25–50 cm/pixel HiRISE images for a much larger area in this work. Typical barriers in planetary 3D mapping, such as ﬁnd suitable stereo pairs, potential complication of surface albedos, and expensive computational costs, are avoided using the demonstrated method. The extensibility is thus considered high and applications to other Mars datasets, e.g., the 50 cm/pixel Tianwen- 1 High Resolution Imaging Camera (HiRIC) images [43], lunar datasets (retraining can Remote Sens. 2021, 13, 3270 22 of 26

be achieved using the Lunar Reconnaissance Orbiter Camera Narrow Angle Camera images [44] and existing PDS DTM products), and/or other planetary datasets, could be straightforwardly explored in the future.

4.3. Known Artefact The resultant 50 cm/pixel HiRISE MADNet DTM mosaic has some known artefacts. These include the aforementioned tiling artefacts and smoothing issues that were discussed in Sections 2.3 and 3.4, respectively, DTM gaps due to insufﬁcient overlap of HiRISE images, and linear artefacts due to missing data within the HiRISE image (only found in strips). Figure 17 provides an example for each of the four known artefacts within the hill-shaded images and show how much they would affect the DTMs. The affected areas are considered minor (less than 0.2% of the total number of pixels) within the mosaic. It should also be noted that the HRSC and CTX MADNet DTM mosaics do not have any of these artefacts.

Figure 17. Examples of known artefacts of the HiRISE MADNet DTM mosaic, i.e., 1st column: tiling artefacts; 2nd column: smoothing issues; 3rd column: missing columns in the original HiRISE image; 4th column: linear artefacts due to the use of JPEG2000 HiRISE images that do not have the correct radiometric de-calibration. Hill-shaded image is created using the HiRISE MADNet DTM mosaic with illumination azimuth 330◦, elevation 30◦, and a vertical exaggeration factor of three times.

4.4. Limitations and Future Work Currently, the proposed MADNet based DTM processing chain has two main limitations. The ﬁrst limitation is the introduction of tiling artefacts to the ﬁnal product. Although we do not observe sharp transitions between each MADNet tile (a typical height difference across the tile boundaries is ~10 cm), sometimes two adjacent tiles may have Remote Sens. 2021, 13, 3270 23 of 26

different inference performance that results in different levels of detail, and subsequently results in different appearance in a full-strip DTM mosaic (see issues raised in Section 2.3). We are exploring potential solutions to this issue, for example, adapting the MADNet architecture to focus on fine-scale reconstructions only, using “flattened” training DTMs (large-scale slopes being removed), and leaving out the coarse-scale reconstruction with the low-resolution reference DTM. The second limitation is regarding the reference data. While introducing photogrammetric DTMs as reference data gives better 3D accuracy to the resultant MADNet DTMs, in comparison to using the very low-resolution MOLA DTM as reference (see issues discussed in Section 3.4), we are aware that there are some photogrammetric DTM artefacts that may get propagated into the MADNet DTMs through the 3D co-alignment process. In this work, we use a multi-stage strategy to gradually produce a 50 cm/pixel HiRISE DTM when propagating coarse-to-fine information from 463 m/pixel MOLA DTM to 50 m/pixel HRSC DTM, to 25 m/pixel HRSC MADNet DTM, to 18 m/pixel CTX DTM, to 12 m/pixel CTX MADNet DTM, and then to 50 cm/pixel HiRISE MADNet DTM. Each step has inevitably inherited photogrammetric DTM artefacts, but these get reduced with the MADNet processing. However, such inherited photogrammetric DTM artefacts still cannot be fully eliminated despite our best efforts to date. In the future, we will explore potential solutions like the use of a pyramidal coarse-to- fine approach on the full-sized input image itself on top of the existing pyramidal scheme within individual tiles. This will allow incremental corrections of large-scale inherited photogrammetric errors and also address the issue of the resolution gaps between target images and reference DTMs as mentioned in Section 3.4. Finally, finding automated ways to solve the issues described in Section 2.3 will also be part of future work. Combining MADNet with image super-resolution restoration [36] or shape-from-shading techniques [23] will also be explored in the future.

5. Conclusions In this paper, we demonstrate an end-to-end application using the MADNet DTM surface modelling system [24] to create large area, multi- and high-resolution 3D mapping products from HRSC, CTX, and HiRISE single images that are co-aligned with each other and MOLA baseline DTM over the ExoMars 2022 Rosalind Franklin rover’s landing site, at Oxia Planum. Technical issues are discussed and assessments are provided. We believe the demonstrated DTM mosaic products have state-of-the-art DTM quality and geospatial accuracy with respect to MOLA. Moreover, the resultant HiRISE MADNet DTM mosaic covers the whole 3-sigma landing ellipses for the ﬁrst time at 50 cm/pixel resolution with uniform quality and consistency. All resultant DTM products will be made publicly available alongside this paper.

Supplementary Materials: The following are available online. DTM and ORI results, derived products, and figures at full resolution are available online at https://liveuclac-my.sharepoint.com/:f:/g/personal/ ucasyta_ucl_ac_uk/EnrEek4jUppMkh5c7mBPAEwBwLUsO_MgoJWyLpQaLmN1Jw?e=jBcwEo, accessed on 22 July 2021. Author Contributions: Conceptualization, Y.T., J.-P.M. and S.J.C.; methodology, Y.T.; software, Y.T.; validation, Y.T., J.-P.M. and S.J.C.; formal analysis, Y.T.; investigation, Y.T. and J.-P.M.; resources, Y.T., J.-P.M., S.J.C. and S.X.; data curation, Y.T., J.-P.M. and S.X.; writing—original draft preparation, Y.T.; writing—review and editing, Y.T., J.-P.M. and S.X.; visualization, Y.T. and S.X.; supervision, Y.T. and J.-P.M.; project administration, Y.T. and J.-P.M.; funding acquisition, Y.T., J.-P.M. and S.J.C. All authors have read and agreed to the published version of the manuscript. Funding: The research leading to these results is receiving funding from the UKSA Aurora programme (2018–2021) under grant no. ST/S001891/1, as well as partial funding from the STFC MSSL Consolidated Grant ST/K000977/1. S.C. is grateful to the French Space Agency CNES for supporting her HiRISE related work. S.X. has received funding from the Shenzhen Scientiﬁc Research and Remote Sens. 2021, 13, 3270 24 of 26

Development Funding Programme (grant No. JCYJ20190808120005713) and China Postdoctoral Science Foundation (grant No. 2019M663073). Institutional Review Board Statement: Not applicable. Informed Consent Statement: Not applicable. Data Availability Statement: The output results will all be publicly available and open access. See https://www.cosmos.esa.int/web/psa/ucl-mssl_meta-gsf, accessed on 22 July 2021. Acknowledgments: The research leading to these results is receiving funding from the UKSA Aurora programme (2018–2021) under grant ST/S001891/1, as well as partial funding from the STFC MSSL Consolidated Grant ST/K000977/1. S.C. is grateful to the French Space Agency CNES for supporting her HiRISE related work. Support from SGF (Budapest), the University of Arizona (Lunar and Planetary Lab.), and NASA are also gratefully acknowledged. Operations support from the UK Space Agency under grant ST/R003025/1 is also acknowledged. S.X. has received funding from the Shenzhen Scientific Research and Development Funding Programme (grant No. JCYJ20190808120005713) and China Postdoctoral Science Foundation (grant No. 2019M663073). The authors would like to thank the HRSC, CTX, and HiRISE teams for making these images and derived products publicly available. Conflicts of Interest: The authors declare no conflict of interest.

References 1. Neukum, G.; Jaumann, R. HRSC: The high resolution stereo camera of Mars Express. Sci. Payload 2004, 1240, 17–35. 2. Gwinner, K.; Scholten, F.; Preusker, F.; Elgner, S.; Roatsch, T.; Spiegel, M.; Schmidt, R.; Oberst, J.; Jaumann, R.; Heipke, C. Topography of Mars from global mapping by HRSC high-resolution digital terrain models and orthoimages: Characteristics and performance. Earth Planet. Sci. Lett. 2010, 294, 506–519. [CrossRef] 3. Gwinner, K.; Jaumann, R.; Hauber, E.; Hoffmann, H.; Heipke, C.; Oberst, J.; Neukum, G.; Ansan, V.; Bostelmann, J.; Dumke, A.; et al. The High Resolution Stereo Camera (HRSC) of Mars Express and its approach to science analysis and mapping for Mars and its satellites. Planet. Space Sci. 2016, 126, 93–138. [CrossRef] 4. Gwinner, K.; Scholten, F.; Spiegel, M.; Schmidt, R.; Giese, B.; Oberst, J.; Heipke, C.; Jaumann, R.; Neukum, G. Derivation and validation of high-resolution digital terrain models from Mars Express HRSC data. Photogramm. Eng. Remote. Sens. 2009, 75, 1127–1142. [CrossRef] 5. O’Hara, R.; Barnes, D. A new shape from shading technique with application to Mars Express HRSC images. ISPRS J. Photogramm. Remote. Sens. 2012, 67, 27–34. [CrossRef] 6. Tao, Y.; Michael, G.; Muller, J.P.; Conway, S.J.; Putri, A.R. Seamless 3 D Image Mapping and Mosaicing of Valles Marineris on Mars Using Orbital HRSC Stereo and Panchromatic Images. Remote. Sens. 2021, 13, 1385. [CrossRef] 7. Putri, A.R.D.; Sidiropoulos, P.; Muller, J.P.; Walter, S.H.; Michael, G.G. A new south polar digital terrain model of Mars from the High-Resolution Stereo Camera (HRSC) onboard the ESA Mars Express. Planet. Space Sci. 2019, 174, 43–55. [CrossRef] 8. Malin, M.C.; Bell, J.F.; Cantor, B.A.; Caplinger, M.A.; Calvin, W.M.; Clancy, R.T.; Edgett, K.S.; Edwards, L.; Haberle, R.M.; James, P.B.; et al. Context camera investigation on board the Mars Reconnaissance Orbiter. J. Geophys. Res. Space Phys. 2007, 112, 112. [CrossRef] 9. Wang, Y.; Wu, B. Investigation of boresight offsets and co-registration of HiRISE and CTX imagery for precision Mars topographic mapping. Planet. Space Sci. 2017, 139, 18–30. [CrossRef] 10. Tao, Y.; Muller, J.P.; Sidiropoulos, P.; Xiong, S.T.; Putri, A.R.D.; Walter, S.H.G.; Veitch-Michaelis, J.; Yershov, V. Massive stereo-based DTM production for Mars on cloud computers. Planet. Space Sci. 2018, 154, 30–58. [CrossRef] 11. Chen, Z.; Wu, B.; Liu, W.C. Mars3DNet: CNN-Based High-Resolution 3D Reconstruction of the Martian Surface from Single Images. Remote. Sens. 2021, 13, 839. [CrossRef] 12. McEwen, A.S.; Eliason, E.M.; Bergstrom, J.W.; Bridges, N.T.; Hansen, C.J.; Delamere, W.A.; Grant, J.A.; Gulick, V.C.; Herkenhoff, K.E.; Keszthelyi, L.; et al. Mars reconnaissance orbiter’s high resolution imaging science experiment (HiRISE). J. Geophys. Res. Space Phys. 2007, 112, E05S02. [CrossRef] 13. Kirk, R.L.; Howington-Kraus, E.; Rosiek, M.R.; Cook, D.; Anderson, J.; Becker, K.; Archinal, B.A.; Keszthelyi, L.; King, R.; McEwen, A.S.; et al. Ultrahigh resolution topographic mapping of Mars with HiRISE stereo images: Methods and ﬁrst results. In Proceedings of the Seventh International Conference on Mars, Moscow, Russia, 9 July 2007; p. 3381. 14. Kim, J.R.; Muller, J.P. Very high resolution stereo DTM extraction and its application to surface roughness estimation over Martian surface. Int. Arch. Photogramm. Remote. Sens. Spat. Inf. Sci. 2008, 37, 993–998. 15. Kim, J.R.; Muller, J.P. Multi-resolution topographic data extraction from Martian stereo imagery. Planet. Space Sci. 2009, 57, 2095–2112. [CrossRef] 16. Beyer, R.A.; McEwen, A.S.; Kirk, R.L. Meter-scale slopes of candidate MER landing sites from point photoclinometry. J. Geophys. Res. Planets 2003, 108, E12. [CrossRef] Remote Sens. 2021, 13, 3270 25 of 26

17. Douté, S.; Jiang, C. Small-Scale Topographical Characterization of the Martian Surface With In-Orbit Imagery. IEEE Trans. Geosci. Remote. Sens. 2019, 58, 447–460. [CrossRef] 18. Jiang, C.; Douté, S.; Luo, B.; Zhang, L. Fusion of photogrammetric and photoclinometric information for high-resolution DEMs from Mars in-orbit imagery. ISPRS J. Photogramm. Remote. Sens. 2017, 130, 418–430. [CrossRef] 19. Hess, M.; Wohlfarth, K.; Grumpe, A.; Wöhler, C.; Ruesch, O.; Wu, B. Atmospherically compensated shape from shading on the martian surface: Towards the perfect digital terrain model of mars. Int. Arch. Photogramm. Remote. Sens. Spat. Inf. Sci. 2019, XLII-2/W13, 1405–1411. [CrossRef] 20. Hess, M. High Resolution Digital Terrain Model for the Landing Site of the Rosalind Franklin (ExoMars) Rover. Adv. Space Res. 2019, 53, 1735–1767. 21. Thomas, N.; Cremonese, G.; Ziethe, R.; Gerber, M.; Brändli, M.; Bruno, G.; Erismann, M.; Gambicorti, L.; Gerber, T.; Ghose, K.; et al. The colour and stereo surface imaging system (CaSSIS) for the ExoMars trace gas orbiter. Space Sci. Rev. 2017, 212, 1897–1944. [CrossRef] 22. Simioni, E.; Re, C.; Mudric, T.; Cremonese, G.; Tulyakov, S.; Petrella, A.; Pommerol, A.; Thomas, N. 3DPD: A photogrammetric pipeline for a PUSH frame stereo cameras. Planet. Space Sci. 2021, 198, 105165. [CrossRef] 23. Tao, Y.; Douté, S.; Muller, J.P.; Conway, S.J.; Thomas, N.; Cremonese, G. Ultra-High-Resolution 1 m/pixel CaSSIS DTM Using Super-Resolution Restoration and Shape-from-Shading: Demonstration over Oxia Planum on Mars. Remote. Sens. 2021, 13, 2185. [CrossRef] 24. Tao, Y.; Xiong, S.; Conway, S.J.; Muller, J.-P.; Guimpier, A.; Fawdon, P.; Thomas, N.; Cremonese, G. Rapid Single Image-Based DTM Estimation from ExoMars TGO CaSSIS Images Using Generative Adversarial U-Nets. Remote. Sens. 2021, 13, 2877. [CrossRef] 25. Smith, D.E.; Zuber, M.T.; Frey, H.V.; Garvin, J.B.; Head, J.W.; Muhleman, D.O.; Pettengill, G.H.; Phillips, R.J.; Solomon, S.C.; Zwally, H.J.; et al. Mars Orbiter Laser Altimeter—Experiment summary after the ﬁrst year of global mapping of Mars. J. Geophys. Res. 2001, 106, 23689–23722. [CrossRef] 26. Neumann, G.A.; Rowlands, D.D.; Lemoine, F.G.; Smith, D.E.; Zuber, M.T. Crossover analysis of Mars Orbiter Laser Altimeter data. J. Geophys. Res. 2001, 106, 23753–23768. [CrossRef] 27. Albee, A.L.; Arvidson, R.E.; Palluconi, F.; Thorpe, T. Overview of the Mars Global Surveyor mission. J. Geophys. Res. 2001, 106, 23291–23316. [CrossRef] 28. Quantin-Nataf, C.; Carter, J.; Mandon, L.; Thollot, P.; Balme, M.; Volat, M.; Pan, L.; Loizeau, D.; Millot, C.; Breton, S.; et al. Oxia Planum: The Landing Site for the ExoMars “Rosalind Franklin” Rover Mission: Geological Context and Prelanding Interpretation. Astrobiology 2021, 21, 345–366. [CrossRef][PubMed] 29. Favaro, E.A.; Balme, M.R.; Davis, J.M.; Grindrod, P.M.; Fawdon, P.; Barrett, A.M.; Lewis, S.R. The Aeolian Environment of the Landing Site for the ExoMars Rosalind Franklin Rover in Oxia Planum, Mars. J. Geophys. Res. Planets 2021, 126, 2020JE006723. [CrossRef] 30. Ivanov, M.A.; Slyuta, E.N.; Grishakina, E.A.; Dmitrovskii, A.A. Geomorphological analysis of ExoMars candidate landing site Oxia Planum. Sol. Syst. Res. 2020, 54, 1–14. [CrossRef] 31. Mandon, L.; Parkes Bowen, A.; Quantin-Nataf, C.; Bridges, J.C.; Carter, J.; Pan, L.; Beck, P.; Dehouck, E.; Volat, M.; Thomas, N.; et al. Morphological and Spectral Diversity of the Clay-Bearing Unit at the ExoMars Landing Site Oxia Planum. Astrobiology 2021, 21, 464–480. [CrossRef] 32. Calef, F.J., III; Gengl, H.E.; Soliman, T.; Abercrombie, S.P.; Powell, M.W. MMGIS: A Multi-Mission Geographic Information System for in situ Mars Operations. In Proceedings of the Liquid Propulsion Systems Centre 2017, The Woodlands, TX, USA, 20–24 March 2017; Volume XLVIII, p. 2541. 33. Sefton-Nash, E.; Fawdon, P.; Orgel, C.; Balme, M.; Quantin-Nataf, C.; Volat, M.; Hauber, E.; Adeli, S.; Davis, J.; Grindrod, P.; et al. Exomars RSOWG. Team Mapping of Oxia Planum for The Exomars 2022 Rover-Surface Platform Mission. In Proceedings of the Liquid Propulsion Systems Centre 2021, 19–30 April 2021; Volume 52, p. 1947. 34. Goodfellow, I.J.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; Bengio, Y. Generative adversarial networks. arXiv 2014, arXiv:1406.2661. 35. Jolicoeur-Martineau, A. The relativistic discriminator: A key element missing from standard GAN. arXiv 2018, arXiv:1807.00734. 36. Tao, Y.; Conway, S.J.; Muller, J.-P.; Putri, A.R.D.; Thomas, N.; Cremonese, G. Single Image Super-Resolution Restoration of TGO CaSSIS Colour Images: Demonstration with Perseverance Rover Landing Site and Mars Science Targets. Remote. Sens. 2021, 13, 1777. [CrossRef] 37. Ronneberger, O.; Fischer, P.; Brox, T. U-net: Convolutional networks for biomedical image segmentation. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany, 18 May 2015; pp. 234–241. 38. Huang, G.; Liu, Z.; Van Der Maaten, L.; Weinberger, K.Q. Densely connected convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Donostia, Spain, 5–8 June 2017; pp. 4700–4708. 39. Laina, I.; Rupprecht, C.; Belagiannis, V.; Tombari, F.; Navab, N. Deeper depth prediction with fully convolutional residual networks. In Proceedings of the 2016 IEEE Fourth international Conference on 3D Vision (3DV), Stanford, CA, USA, 25 October 2016; pp. 239–248. 40. Beyer, R.; Alexandrov, O.; McMichael, S. The Ames Stereo Pipeline: NASA’s Opensource Software for Deriving and Processing Terrain Data. Earth Space Sci. 2018, 5, 537–548. [CrossRef] Remote Sens. 2021, 13, 3270 26 of 26

41. Tao, Y.; Muller, J.-P.; Poole, W.D. Automated localisation of Mars rovers using co-registered HiRISE-CTX-HRSC orthorectiﬁed images and DTMs. Icarus 2016, 280, 139–157. [CrossRef] 42. Cai, G.R.; Jodoin, P.M.; Li, S.Z.; Wu, Y.D.; Su, S.Z.; Huang, Z.K. Perspective-SIFT: An efﬁcient tool for low-altitude remote sensing image registration. Signal. Process. 2013, 93, 3088–3110. [CrossRef] 43. Meng, Q.; Wang, D.; Wang, X.; Li, W.; Yang, X.; Yan, D.; Li, Y.; Cao, Z.; Ji, Q.; Sun, T.; et al. High Resolution Imaging Camera (HiRIC) on China’s First Mars Exploration Tianwen-1 Mission. Space Sci. Rev. 2021, 217, 1–29. [CrossRef] 44. Robinson, M.S.; Brylow, S.M.; Tschimmel, M.; Humm, D.; Lawrence, S.J.; Thomas, P.C.; Denevi, B.W.; Bowman-Cisneros, E.; Zerr, J.; Ravine, M.A.; et al. Lunar reconnaissance orbiter camera (LROC) instrument overview. Space Sci. Rev. 2010, 150, 81–124. [CrossRef]