Investigating Lunar Boulders at the Apollo 17 Landing Site Using Photogrammetry and Virtual Reality

remote sensing

Article Investigating Lunar Boulders at the Apollo 17 Landing Site Using Photogrammetry and Virtual Reality

Stéphane Le Mouélic 1,* , Pauline Enguehard 1, Harrison H. Schmitt 2, Gwénaël Caravaca 1 , Benoît Seignovert 3 , Nicolas Mangold 1, Jean-Philippe Combe 4 and François Civet 5

1 Laboratoire de Planétologie et Géodynamique, CNRS UMR6112, Université de Nantes, 44322 Nantes, France; [email protected] (P.E.); [email protected] (G.C.); [email protected] (N.M.) 2 Department of Engineering Physics, University of Wisconsin-Madison, P.O. Box 90730, Albuquerque, NM 87199, USA; [email protected] 3 Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA 91109, USA; benoit.a.seignovert@jpl.nasa.gov 4 Bear Fight Institute, 22 Fiddler’s Road, Winthrop, WA 98862, USA; jean-philippe_combe@bearﬁghtinstitute.com 5 VR2Planets, 44322 Nantes, France; [email protected] * Correspondence: [email protected]; Tel.: +33-2-5-12-55-69

Received: 11 May 2020; Accepted: 9 June 2020; Published: 11 June 2020

Abstract: The Taurus-Littrow valley on the Moon was the location of intensive geologic fieldwork during three days in December 1972. In situ activities at sampling stations were systematically documented by the astronauts using a series of overlapping images taken with their Hasselblad cameras. We investigated how this Apollo image archive can be used to perform 3-D reconstructions of several boulders of interest using close-range photogrammetry. We specifically focused on seven different boulders located at Stations 2, 6, and 7, at the foot of South and North Massifs, respectively. These boulders represent samples from highland materials, which rolled down the slopes of the surrounding hills. We used the Agisoft Metashape software to compute 3-D reconstructions of these boulders, using 173 scanned images as input. We then used either a web-based platform or a game engine to render the models in virtual reality. This allowed the users to walk around the boulders and to investigate in detail their morphology, fractures, vesicles, color variations, and sampling spots, as if standing directly in front of them with the astronauts. This work suggests that many features can be reconstructed in other sites of the Apollo missions, so as other robotic landing sites. Virtual reality techniques coupled to photogrammetry is thus opening a new era of exploration, both for past and future landing sites.

Keywords: sample documentation; remote sensing; photogrammetry; moon; 3-D; Apollo; geology; virtual reality

1. Introduction The constant development of computer hardware (notably graphic computing units used in game engines) and the availability since 2016 of technologically mature virtual reality (VR) headsets to the general public opens a new era of geological exploration. In planetary exploration, the use of simulated visual experience, commonly known as virtual reality,is quite an old concept [1], but the technique is only starting to become mature and progressively widespread in our community. It can be coupled with the possibility of reconstructing high-resolution 3-D data sets, such as digital outcrop models (DOMs), from a

Remote Sens. 2020, 12, 1900; doi:10.3390/rs12111900 www.mdpi.com/journal/remotesensing Remote Sens. 2020, 12, 1900 2 of 18 set of overlapping photographs using structure-from-motion photogrammetry (e.g., [2–8]). By doing so, VR enables geoscientists to complement, quantitatively evaluate, and analytically enhance field observations, and also to eventually perform analysis at multiple scales and optical manipulation, which is not directly possible in the field with conventional methods [9–12]. This is particularly important for remote locations that are not easily accessible, such as planets, moons, and small bodies of the solar system. VR also allows the possibility of easily extracting and visualizing the scientific content of a set of images, and sharing the results with the community through web-based platforms or other digital systems, such as game engines, thanks to their optimized 3-D rendering capabilities [13,14]. This new immersive technique potentially allows the users to explore 3-D reconstructions of planetary landscapes as if they were physically present “on the field”. We investigated here how the in situ lunar imagery can be coupled to the orbital imagery to provide the user the possibility of exploring the lunar surface in VR. We focused on the reconstruction of the Apollo 17 lunar landing site, with a particular emphasis on the reproduction of the geological settings related to the melt-breccia contacts in boulders documented in photographs at Stations 6 and 7 at the foot of the North Massif in the Taurus-Littrow valley, and on boulders at Station 2 at the foot of the South Massif [8]. The location of these three sites is shown in Figure1a on the traverse map produced from [ 15], building on the initial work of [16], together with panoramas taken in situ near the three areas of boulders. The topic of this paper is to describe the reconstruction process of the main boulders, from the input data up to the visualization of the products. The results of the 3-D reconstructions of the boulders are shared through a public web-based platform. The Apollo 17 crew spent three days at the Taurus-Littrow site in December 1972. The southernmost part of the traverse was reached during the second extra vehicular activity (EVA), at Station 2 located at the foot of South Massif (Figure1a,b). During the third EVA, the sampling sites Station 6 and Station 7 were selected as a suitable location to collect possible ancient highland material from the North Massif (Figure1c,d). North Massif rises about 1400 m above Station 6, and probably formed as the result of the giant impact that created the Serenitatis basin [8]. Sampling impact melts are particularly useful for the dating of major events, the radio-isotopic age of the sampled rock being reset by the melting during the impact. Radio-isotopic dating, however, continues to evolve, depending on the ability to precisely identify what is being dated. Whereas other locations can be envisaged in further works, we focused our efforts here on the Station 6 and Station 7 boulders at the base of North Massif, located on the northernmost portion of the EVA-3 traverse, and the Station 2 boulders at the base of the South Massif. In the first section, we present the orbital and in situ data used as input, as well as the software solution used for the photogrammetric process. In the second part, we detail the 3-D reconstructions of the boulders at these key sampling sites of the Apollo 17 field campaign: Station 2, 6, and 7. We then present the integration of the products into VR in the third section, before discussing the results and concluding the study. Remote Sens. 2020, 12, 1900 3 of 18 Remote Sens. 2020, 12, 1900 3 of 19

Figure 1. (a) Apollo 17 traverse map (adapted from [15]) showing the localization of Stations 2, 6, Figure 1. (a) Apollo 17 traverse map (adapted from [15]) showing the localization of Stations 2, 6, and and 7 (red circles). (b) Panorama taken by E. Cernan at Station 2 (frames AS17-137-20926 to 20956). 7 (red circles). (b) Panorama taken by E. Cernan at Station 2 (frames AS17‐137‐20926 to 20956). (c) (c) Panorama taken by at H. H. Schmitt at Station 6 (frames AS17-141-21575 to 21603), showing several Panorama taken by at H. H. Schmitt at Station 6 (frames AS17‐141‐21575 to 21603), showing several boulder fragments to the left of the lunar roving vehicle (LRV). (d) Panorama taken by E. Cernan at boulder fragments to the left of the lunar roving vehicle (LRV). (d) Panorama taken by E. Cernan at station 7 (frames AS17-146-22339 to 22363). The main boulder is on the right of the LRV. station 7 (frames AS17‐146‐22339 to 22363). The main boulder is on the right of the LRV. 2. Materials and Methods 2. Materials and Methods 2.1. Context Orbital Imagery 2.1. Context Orbital Imagery We ﬁrst used the available orbital imagery to map the Taurus-Littrow valley using the ArcGIS GeographicWe first Information used the available System. orbital The purpose imagery was to tomap give the a broaderTaurus‐Littrow regional valley context using of the the sampling ArcGIS Geographicsites once integrated Information into System. virtual reality, The purpose and to preciselywas to give locate a broader the sites regional of interest. context The of general the sampling context wassites computedonce integrated using ainto subset virtual of the reality, SELenological and to precisely and Engineering locate the Explorersites of interest. (SELENE) The “Kaguya” general Terraincontext Camerawas computed (TC) global using orthomosaic a subset of atthe 4096 SELenological pixels per degreesand Engineering [17], available Explorer on the(SELENE) United “Kaguya”States Geological Terrain SurveyCamera (USGS) (TC) global astrogeology orthomosaic science at 4096 center pixels website. per degrees With a [17], resolution available of ~10on the m perUnited pixel, States this Geological data set bridges Survey the (USGS) gap between astrogeology Clementine science and center the website. LROC WAC With (~100 a resolution m/pixel) of and~10 mthe per LRO pixel, NAC this (~1 data m/pixel) set bridges and therefore the gap provides between relevantClementine contextual and the information. LROC WAC The (~100 Kaguya-TC m/pixel) andmosaic the wasLRO draped NAC (~1 over m/pixel) a digital and elevation therefore model provides produced relevant from contextual the merging information. of LRO Lunar The Kaguya Orbiter‐ TCLaser mosaic Altimeter was draped (LOLA) over data a and digital SELENE elevation Kaguya model TC produced data, available from atthe a horizontalmerging of resolution LRO Lunar of Orbiter512 pixels Laser per degreeAltimeter (corresponding (LOLA) data to ~59and m SELENE per pixel Kaguya at the equator) TC data, and available a typical verticalat a horizontal accuracy resolution of 512 pixels per degree (corresponding to ~59 m per pixel at the equator) and a typical

Remote Sens. 2020, 12, 1900 4 of 18 Remote Sens. 2020, 12, 1900 4 of 19

~3vertical to 4 m accuracy [17,18]. ~3 A globalto 4 m color[17,18]. ratio A global composite color from ratio Clementine composite from UVVIS Clementine is used to UVVIS provide is spectral used to informationprovide spectral at the information regional scale at the [19 regional,20]. scale [19, 20]. For the the highest highest spatial spatial resolution, resolution, we we over over imposed imposed an Apollo an Apollo 17 LROC 17 LROC NAC NAC orthomosaic orthomosaic at 50 atcm 50 per cm pixel per and pixel the andcorresponding the corresponding associated associated 1.5‐m‐scale 1.5-m-scale LROC DEM, LROC both DEM,also being both available also being on availablethe USGS astrogeology on the USGS Center astrogeology website [15]. Center This websitemosaic was [15 ].produced This mosaic from images was producedM137353046 from‐L/‐ imagesR, M152669024 M137353046-L‐L/‐R, /M134985003-R, M152669024-L‐R, M168000580/-R, M134985003-R,‐L/‐R, M119652859 M168000580-L‐L/‐R,/ -R,M109032389 M119652859-L‐R, and/-R, M109032389-R,orthorectified using and orthorectiﬁedthe 1.5 m DEM. using Figure the 2 1.5 shows m DEM. a selection Figure2 of shows global a selectionviews of these of global cartographic views of theseproducts cartographic once integrated products into once a 3‐ integratedD rendering into system. a 3-D rendering system.

Figure 2. (a) Perspective view of a Kaguya-TC mosaic draped over a LOLA-Kaguya TC merged digital Figure 2. (a) Perspective view of a Kaguya‐TC mosaic draped over a LOLA‐Kaguya TC merged digital elevation model, providing the context of the Taurus-Littrow valley (north is pointing on the upper right elevation model, providing the context of the Taurus‐Littrow valley (north is pointing on the upper corner, and the tile dimension is 55 55 km). The Lunar Reconnaissance Orbiter NAC 50 cm mosaic right corner, and the tile dimension× is 55 × 55 km). The Lunar Reconnaissance Orbiter NAC 50 cm appears in a lighter shade in the center of the frame. (b) Clementine UVVIS Color ratio composite mosaic appears in a lighter shade in the center of the frame. (b) Clementine UVVIS Color ratio draped over the same DEM. (c)–(f) display progressive close-up views showing the location of the composite draped over the same DEM. (c) to (f) display progressive close‐up views showing the reconstructed boulders at Station 6 and 7 at the foot of North Massif that is discussed in Section3. location of the reconstructed boulders at Station 6 and 7 at the foot of North Massif that is discussed The distance between Station 6 and 7 is 490 m. in Section 3. The distance between Station 6 and 7 is 490 m. 2.2. The Hasselblad Surface Image Data Set 2.2. The Hasselblad Surface Image Data Set During their EVAs, the astronauts used two Hasselblad 500 EL data cameras attached to the chestDuring of their their spacesuits. EVAs, the Cameras astronauts were calibratedused two Hasselblad and equipped 500 with EL data a reseau cameras plate attached in front to of the imagechest of plane their tospacesuits. provide aCameras photogrammetric were calibrated reference and [equipped21]. The gridwith consists a reseau in plate 5 in5 mmfront crosses of the × atimage 10-mm plane intervals, to provide imaged a photogrammetric in each frame ofreference the 70-mm [21]. film. The grid The consists cameras in were 5 × 5 fitted mm crosses with a at Zeiss 10‐ Biogonmm intervals, lens with imaged a 60-mm in each focal frame length, of thus the 70 providing‐mm film. a moderately The cameras wide were field fitted of view. with a The Zeiss astronauts Biogon systematicallylens with a 60‐ acquiredmm focal single length, frames, thus providing full panoramas, a moderately overlapping wide images, field of and view. stereo The pairs astronauts along theirsystematically traverse path acquired and at single sampling frames, stations full topanoramas, document overlapping the samples’ images, collection and and stereo allow pairs post-flight along geologicaltheir traverse analysis path and [22]. at sampling stations to document the samples’ collection and allow post‐flight geologicalWe mainly analysis used [22]. two different online resources to compile the relevant imagery database for our researchWe mainly project. used Both two are different based on online original resources film scans to compile performed the relevant at the Johnson imagery Space database Center for [ our23]. research project. Both are based on original film scans performed at the Johnson Space Center [23]. We first used the “Apollo in real time” (https://apollo17.org) website, which provides a very user‐

Remote Sens. 2020, 12, 1900 5 of 18 Remote Sens. 2020, 12, 1900 5 of 19

Wefriendly first used possibility the “Apollo to evaluate in real time” how ( https:many// apollo17.orgimages are )available website, whichat each provides location, a very and user-friendly when (and possibilitywhere) they to were evaluate sequentially how many taken images during are available the mission. at each Images location, directly and accessible when (and from where) this they website were havesequentially a resolution taken of during 2100 × the 2100 mission. pixels. Images While being directly generally accessible well from contrasted this website and havecolor a balanced, resolution we of rather2100 2100obtained pixels. a Whilebetter beingaccuracy generally in our well photogrammetric contrasted and color process balanced, when we using rather the obtained same images a better × downloadedaccuracy in our photogrammetricfrom the process“Project when usingApollo the sameArchive” images downloaded page fromon the “ProjectFlickr Apollo(https://www.flickr.com/p Archive” page on Flickrrojectapolloarchive), (https://www.flickr.com which complements/projectapolloarchive the comprehensive), which complements Apollo Lunarthe comprehensive Surface Journal Apollo source Lunar of Surfaceinformation. Journal The source highest of information. pixel resolution The highestprovided pixel on resolutionthe Flickr providedwebsite corresponds on the Flickr to website a size of corresponds 4175 × 4175 topixels. a size An of example 4175 4175 of the pixels. images An used example as input of the is imagesshown × inused Figure as input 3. These is shown black in and Figure white3. Theseand color black images and white correspond and color to images1800 dpi correspond Hasselblad to film 1800 scans dpi producedHasselblad in film 2005 scans at the producedJohnson Space in 2005 Center at the [23]. Johnson Considering Space that Center the Hasselblad [23]. Considering film is 70 that mm the on theHasselblad diagonal, film this is corresponds 70 mm on the to diagonal,an equivalent this corresponds pixel size of to ~12 an μ equivalentm. It should pixel be sizenoted of ~12that µthem. originalIt should oversampled be noted that scans the original with a oversampledresolution up scans to 14,000 with × a 16,000 resolution pixels up by to 14,000photo are16,000 also available pixels by × photoat the Apollo are also Image available Archive at the (https://tothemoon.ser.asu.edu/gallery/Apollo) Apollo Image Archive (https://tothemoon.ser.asu.edu from the/gallery Arizona/Apollo State) University.from the Arizona In this State study, University. we decided In thisto work study, with we the decided 4175 to× 4175 work pixel with resolution the 4175 images4175 pixel for × optimumresolution handling images for with optimum our current handling computing with our resources. current computing resources.

Figure 3. ExampleExample of of the the images images of of H. H. H. H. Schmitt Schmitt taking taking rock rock samples samples during during the the third third extravehicular activity at Station 7 (images labeled AS17-146-22335AS17‐146‐22335 to AS17-146-22338).AS17‐146‐22338). There There is sufficient suﬃcient overlap between these kind of sample images to allow a photogrammetric 3-D3‐D reconstruction ofof thethe boulder.boulder. The cameras were calibrated before flight to evaluate their internal orientation and lens distortion [24]. The cameras were calibrated before flight to evaluate their internal orientation and lens In the photogrammetric process that we describe in further sections, this information was directly derived distortion [24]. In the photogrammetric process that we describe in further sections, this information from the set of pictures themselves by the structure-from-motion optimization algorithm. was directly derived from the set of pictures themselves by the structure‐from‐motion optimization algorithm.2.3. Structure from Motion Principle and Software Choice

2.3. StructureThe structure-from-motion from Motion Principle (SfM) and photogrammetry Software Choice principle relies on the identification of tie-points on a set of overlapping images taken from different points of views to reconstruct a shape in 3-D [2]. The structure‐from‐motion (SfM) photogrammetry principle relies on the identification of tie‐ When a sufficient number of overlapping images of a given scene is available as input, it is possible to points on a set of overlapping images taken from different points of views to reconstruct a shape in compute the position of tie points in a 3-D space, together with optical parameters linked to the camera 3‐D [2]. When a sufficient number of overlapping images of a given scene is available as input, it is used to take the pictures. Using a dense point cloud, a 3-D mesh can be reconstructed to provide possible to compute the position of tie points in a 3‐D space, together with optical parameters linked to the camera used to take the pictures. Using a dense point cloud, a 3‐D mesh can be reconstructed

Remote Sens. 2020, 12, 1900 6 of 18 the shape of the elements constituting the landscape. Individual images are finally mosaicked and wrapped over the mesh as a texture to provide a photorealistic final 3-D model that can be rendered by various common 3-D processing software. Several commercial or freeware software can be used to perform all the processing steps. An example of photogrammetric reconstruction made with the iWitness ProTM 3.6 software [25] on a series of six overlapping images of the 70019 sample area is shown in [8]. Another example of photogrammetric reconstruction is presented in [26], using Agisoft PhotoScan Professional (recently renamed “Metashape”) on descent images of the Apollo 15 landing site. In this work, we also relied on Agisoft Metashape (v. 1.6.2, [27]), which is one of the most widely used SfM solutions in the field of planetary science for geological applications (e.g., [3,5–7,11,12]).

3. Results

3.1. Three-Dimensional Reconstruction of the Station 7 Boulder We started our analysis with the Station 7 boulder, which consists in a single ~2-m-high block that had rolled down from several hundreds of meters higher on the Massif (Figures1a,d and2, and [ 8]). The objectives of the Apollo 17 mission at this location were to characterize and sample both Massif and dark mantle materials [16], although eventually no dark mantle materials were identified at Station 7. A first panorama was acquired south of the rover (images AS17-141-21646 to AS17-141-21664) and a second panorama was acquired north of the rover (images AS17-146-22339 to AS17-146-22363). This boulder stands on a 13.6◦ slope, just above the break in the slope between the Massif and the valley floor. The block was originally described during the in situ exploration as a vesicular anorthositic gabbro, but it was later found to be a fine-grained impact melt-breccia [8,28]. We used the set of 41 images labeled AS17-146-22298 to AS17-146-22338 as input to reconstruct the main boulder using the photogrammetric process. First, a mask was applied to remove 75 lines and columns of pixels at the edge of each image, corresponding to the diffuse limit of the scanned film. All 41 images were then successfully automatically aligned with Agisoft Metashape using ~450,000 tie points, allowing for the generation of a sparse point cloud. In the next step, a dense cloud of 30 million points was generated, allowing the computation of a detailed 3-D mesh. Considering the relatively simple shape of the boulder, we produced a mesh of 300,000 polygons, draped with two textures of 8192 8192 pixels. This represents a good tradeoff between the level of detail and the computer × resources’ optimization of the 3-D model, for further handling into VR systems. During the texture computation, the mosaicking of the 41 original images was performed using an average of the values on each polygon of the 3-D model. Figure4 shows di fferent views of the textured 3-D model reconstructed by the photogrammetric software. The astronauts documented approximatively one-half of the boulder. The 41 positions and orientations of the camera computed by Metashape are indicated at their exact location (annotated squares) around the perspective view in the lower left of Figure4. The inset (white rectangle) shows the level of detail achieved during the reconstruction process. Arrows point, for example, to a veinlet of dark very fine-grained melt-breccia that stands out on the model and that can be traced throughout various parts of the boulder. It presents potential similarities with impact pseudotachylites [29]. The light patch (arrow) in the middle of the left view corresponds to the markings left when collecting sample 77115. Sample 77135 was taken above this area on the left. In the final processing step, the scaling can be determined using a known reference such as the gnomon (ideally) or any piece of hardware, when visible in the original images. Any footprint can also be useful for this purpose. However, this step can prove to be problematic when no known reference is available on the scene. Another important scaling indication is given by the position of the cameras with respect to the surrounding ground, provided that enough ground is present in the 3-D model, and considering that the cameras were systematically attached to the chest of the two 1.83 m- (Cernan) Remote Sens. 2020, 12, 1900 7 of 18 and 1.75 m-tall (Schmitt) astronauts. In this specific case at Station 7, we relied on the astronaut’s suited heightRemote Sens. on the2020 original, 12, 1900 pictures to evaluate the scale. 7 of 19

Figure 4. PhotogrammetricPhotogrammetric reconstruction reconstruction (textured (textured 3‐ 3-DD model) model) of of the the 2‐ 2-m-highm‐high boulder boulder at atStation Station 7. 7.The The position position and and orientation orientation of of the the 41 41 entry entry Apollo Apollo images images (AS17 (AS17-146-22298‐146‐22298 to to AS17 AS17-146-22338)‐146‐22338) as computed by Metashape are indicated on the left perspective view (annotated squares), together with the location of samples 77135 and 77115. The inset inset shows a close close-up‐up of the area located in the white rectangle on the front view, toto illustrate thethe level of details allowed by the 3-D3‐D reconstruction. Arrows point to a 3–5 cm vein. 3.2. Three-Dimensional Reconstruction of the Station 6 Boulders In the final processing step, the scaling can be determined using a known reference such as the gnomonStation (ideally) 6 is located or any at 490piece m of west hardware, of Station when 7, north visible of the in Henry the original crater (Figure images.1a). Any Station footprint 6 o ffered can thealso opportunity be useful for to explorethis purpose. a collection However, of five this large step boulder can prove fragments to be and problematic regolith [8 ],when lying no at the known base ofreference a long boulder is available trail on descending the scene. from Another the North important Massif scaling (Figures indication1c and2). is The given fragments by the originatedposition of fromthe cameras a single with boulder, respect which to the rolled surrounding down the hill,ground, leaving provided a 10-m-wide that enough and 1-km-long ground is trail, present and in broke the apart3‐D model, about and 30 m considering above the valley that the floor, cameras at the were transition systematically between the attached ~26◦ slope to the of chest the massif of the to two a 15.4 1.83◦ slopem‐ (Cernan) [15] near and the 1.75 massif’s m‐tall contact (Schmitt) with astronauts. the valley floor. In this Activities specific at case this at site Station included 7, we the relied collection on the of samples,astronaut’s a tube suited core height and a on traverse the original gravimeter pictures measurement, to evaluate allthe these scale. activities being documented by many photographs, of which we found 82 that were suitable for a 3-D reconstruction. Two additional sets3.2. Three of panoramas‐Dimensional were Reconstruction acquired to provide of the Station a comprehensive 6 Boulders view of the site. H. H. Schmitt recorded the South Pan panorama with frames AS17-141-21575 to AS17-141-21603. E. Cernan recorded the Station 6 is located at 490 m west of Station 7, north of the Henry crater (Figure 1a). Station 6 second panorama while being north of the boulder, with frames 140-21483 to AS17-140-21509. offered the opportunity to explore a collection of five large boulder fragments and regolith [8], lying To reconstruct this complex area from the available imagery, and considering the incomplete at the base of a long boulder trail descending from the North Massif (Figures 1c and 2). The fragments spatial overlap between images of the main boulder fragments, we had to split the photogrammetric originated from a single boulder, which rolled down the hill, leaving a 10‐m‐wide and 1‐km‐long project for this scene into three separate chunks, allowing an independent treatment of each fragment. trail, and broke apart about 30 m above the valley floor, at the transition between the ~26° slope of The first chunk, corresponding to the northern most fragment referred to as Boulder 1 in [8,15,16], the massif to a 15.4° slope [15] near the massif’s contact with the valley floor. Activities at this site was reconstructed from a subset of 42 images labeled AS17-140-21441 to As-140-21482 (Figure5). included the collection of samples, a tube core and a traverse gravimeter measurement, all these The entry images were aligned, generating a sparse point cloud of about 300,000 tie-points. A dense activities being documented by many photographs, of which we found 82 that were suitable for a 3‐ cloud of 46 million points was computed during the SfM process, allowing the generation of a D reconstruction. Two additional sets of panoramas were acquired to provide a comprehensive view of the site. H. H. Schmitt recorded the South Pan panorama with frames AS17‐141‐21575 to AS17‐141‐ 21603. E. Cernan recorded the second panorama while being north of the boulder, with frames 140‐ 21483 to AS17‐140‐21509.

Remote Sens. 2020, 12, 1900 8 of 19

To reconstruct this complex area from the available imagery, and considering the incomplete spatial overlap between images of the main boulder fragments, we had to split the photogrammetric project for this scene into three separate chunks, allowing an independent treatment of each fragment. The first chunk, corresponding to the northern most fragment referred to as Boulder 1 in [8,15,16], was reconstructed from a subset of 42 images labeled AS17‐140‐21441 to As‐140‐21482 (Figure 5). The Remote Sens. 2020, 12, 1900 8 of 18 entry images were aligned, generating a sparse point cloud of about 300,000 tie‐points. A dense cloud of 46 million points was computed during the SfM process, allowing the generation of a mesh of 9 meshmillion of 9polygons, million polygons, that we thatdownsampled we downsampled down downto 1 tomillion 1 million polygons polygons for for better better computer performances duringduring the the visualization visualization process. process. The The computed computed location location and orientationand orientation of the 42of imagesthe 42 isimages shown is on shown the perspective on the perspective view on the view right on panel the ofright Figure panel5. The of insetFigure shows 5. The an enlargementinset shows onan theenlargement locations on where the sampleslocations 76295 where and samples 76275 76295 were extracted.and 76275 were extracted.

Figure 5.5. PhotogrammetricPhotogrammetric reconstruction reconstruction (textured (textured 3-D model)3‐D model) of the of northern the northern most boulder most fragmentboulder atfragment Station at 6, Station informally 6, informally named Boulder named Boulder 1 by previous 1 by previous authors. authors. The position The position and orientation and orientation of the 42of entrythe 42 Apollo entry imagesApollo AS17-140-21441 images AS17‐140 to‐ As-140-2148221441 to As‐ computed140‐21482 fromcomputed Metashape, from Metashape, are indicated are in theindicated perspective in the viewperspective on the right.view on the right.

The second chunkchunk coverscovers the the fragments fragments 2 2 and and 3 3 of of the the boulders. boulders. It It was was produced produced using using a subset a subset of sevenof seven images images labeled labeled AS17-140-21434 AS17‐140‐21434 to to AS17-140-21440 AS17‐140‐21440 (Figure (Figure6). 6). The The alignment alignment step step generated generated a sparsea sparse point point cloud cloud of of about about 40,000 40,000 tie-points. tie‐points. A A dense dense point point cloudcloud ofof 77 millionmillion pointspoints thenthen allowed the generation of a mesh of 500,000 polygons. A significant signiﬁcant part of the soil was reconstructed in the 3-D3‐D model, thanks to the overlap between images. This proved to be useful to give a reference for the orientation and the scaling of the model. Finally, a third chunk using a subset of 33 images labeled AS17-140-21401 to AS17-140-21433 was used to reconstruct the fragments 4 and 5 (Figure7). The alignment step generated a sparse point cloud of about 250,000 tie-points. A dense point cloud of 30 million points then allowed the generation of a mesh that we downsampled to 2 million polygons. This third chunk is of particular interest as the gnomon used by Apollo astronauts was fully reconstructed in the 3-D model, thanks to the fact that it appears in multiple overlapping images. The gnomon is 53 cm when stowed, and 62 cm high when deployed. A partial footprint is also visible in the soil just left of the gnomon, allowing for a secondary scaling of the model, in addition to the camera’s height. This part of the boulder is a vesicular tan-gray breccia, with scattered distinctive light-colored clasts [30] that were accurately represented in the 3-D model, so as the vesicles.

Remote Sens. 2020, 12, 1900 9 of 18 Remote Sens. 2020, 12, 1900 9 of 19

Figure 6. PhotogrammetricPhotogrammetric reconstruction reconstruction (textured (textured 3‐D 3-D model) model) of fragments of fragments 2 and 2 and 3 at 3Station at Station 6. The 6. Thelocation location and andorientation orientation of the of theseven seven entry entry Apollo Apollo images images AS17 AS17-140-21434‐140‐21434 to AS17 to AS17-140-21440‐140‐21440 as as computed by Metashape are indicated by annotated squares. A signiﬁcant part of the ground Remotecomputed Sens. 2020, 12by, 1900Metashape are indicated by annotated squares. A significant part of the ground10 of 19 underneath the boulders was reconstructed thanks to signiﬁcantsignificant overlapoverlap inin thethe entryentry images.images.

Finally, a third chunk using a subset of 33 images labeled AS17‐140‐21401 to AS17‐140‐21433 was used to reconstruct the fragments 4 and 5 (Figure 7). The alignment step generated a sparse point cloud of about 250,000 tie‐points. A dense point cloud of 30 million points then allowed the generation of a mesh that we downsampled to 2 million polygons. This third chunk is of particular interest as the gnomon used by Apollo astronauts was fully reconstructed in the 3‐D model, thanks to the fact that it appears in multiple overlapping images. The gnomon is 53 cm when stowed, and 62 cm high when deployed. A partial footprint is also visible in the soil just left of the gnomon, allowing for a secondary scaling of the model, in addition to the camera’s height. This part of the boulder is a vesicular tan‐gray breccia, with scattered distinctive light‐colored clasts [30] that were accurately represented in the 3‐D model, so as the vesicles.

Figure 7. 7. PhotogrammetricPhotogrammetric reconstruction reconstruction (textured (textured 3‐D 3-D model) model) of the of fragments the fragments 4–5 at 4–5 Station at Station 6. The 6.position The position and orientation and orientation of the 33 of Apollo the 33 images Apollo labeled images AS17 labeled‐140‐ AS17-140-2140121401 to AS17‐140 to‐21433 AS17-140-21433 computed computedby Metashape by Metashape is indicated is indicatedby annotated by annotated squares on squares the bottom on the perspective bottom perspective view. The view. gnomon The gnomon is fully isreproduced fully reproduced in 3‐D, in as 3-D, is a asfootprint, is a footprint, allowing allowing for a precise for a precise scaling scaling of the of model. the model. The Theinset inset shows shows the themarkings markings (white (white arrow) arrow) left leftby astronauts by astronauts when when hammering hammering the theboulder boulder for forsampling. sampling.

3.3. Three‐Dimensional Reconstruction of the Station 2 Boulders The rocks of the South Massif were one of the main geological objectives of the Apollo 17 mission. Station 2 is located 7.4 km southwest of the lunar module, at the foot of the South Massif, at the massif slope’s intersection with the southeast rim of the Nansen moat (Figure 1), the latter probably formed by a pull‐apart due to thrust faulting [8]. The boulders at the feet of the hill slopes may contain clues about the origination of the Serenitatis basin‐forming event. Several sets of images taken at Station 2 at the edge between the plain and the South Massif hill appear suitable for a photogrammetric processing, in particular on three main boulders. Boulder 1 is an interesting ~2 m‐diameter foliated and layered fragment‐rich breccia, lying right at the break of the slope. Four samples were collected across the layering (samples 72255, 72215, 72235, and 72275). We used a set of 20 images labeled AS17‐137‐20900 to AS17‐137‐20909 and AS17‐ 138‐21029 to AS17‐138‐21042 to generate the textured 3‐D model shown in Figure 8. The second part of the data set, covering the right part of the boulder, is in black and white with slightly overexposed images that we had to homogenize. This does not seem to affect the photogrammetric treatment. The subtle difference in color between the left and right parts of the boulder should, however, not be considered. The alignment generated a sparse point cloud of 88,000 tie‐points. A dense point cloud of about 13 million points allowed the generation of a mesh downsampled to 400,000 polygons. This boulder appears much less coherent than the tan‐gray or blue‐gray breccia types [30]. It contains large clasts of both dark‐ and light‐colored breccias in a generally light‐colored matrix.

Remote Sens. 2020, 12, 1900 10 of 18

3.3. Three-Dimensional Reconstruction of the Station 2 Boulders The rocks of the South Massif were one of the main geological objectives of the Apollo 17 mission. Station 2 is located 7.4 km southwest of the lunar module, at the foot of the South Massif, at the massif slope’s intersection with the southeast rim of the Nansen moat (Figure1), the latter probably formed by a pull-apart due to thrust faulting [8]. The boulders at the feet of the hill slopes may contain clues about the origination of the Serenitatis basin-forming event. Several sets of images taken at Station 2 at the edge between the plain and the South Massif hill appear suitable for a photogrammetric processing, in particular on three main boulders. Boulder 1 is an interesting ~2 m-diameter foliated and layered fragment-rich breccia, lying right at the break of the slope. Four samples were collected across the layering (samples 72255, 72215, 72235, and 72275). We used a set of 20 images labeled AS17-137-20900 to AS17-137-20909 and AS17-138-21029 to AS17-138-21042 to generate the textured 3-D model shown in Figure8. The second part of the data set, covering the right part of the boulder, is in black and white with slightly overexposed images that we had to homogenize. This does not seem to aﬀect the photogrammetric treatment. The subtle diﬀerence in color between the left and right parts of the boulder should, however, not be considered. The alignment generated a sparse point cloud of 88,000 tie-points. A dense point cloud of about 13 million points allowed the generation of a mesh downsampled to 400,000 polygons. This boulder appears much less coherent than the tan-gray or blue-gray breccia types [30]. It contains large clasts of bothRemote dark- Sens. and 2020 light-colored, 12, 1900 breccias in a generally light-colored matrix. 11 of 19

Figure 8. Photogrammetric reconstruction (textured 3-D model) of boulder 1 at Station 2. The location Figure 8. Photogrammetric reconstruction (textured 3‐D model) of boulder 1 at Station 2. The location and orientation of the entry Apollo images labelled AS17-137-20900 to AS17-137-20909 and and orientation of the entry Apollo images labelled AS17‐137‐20900 to AS17‐137‐20909 and AS17‐138‐ AS17-138-2102921029 to AS17‐ to138 AS17-138-21042‐21042 computed computed by Metashape by is Metashape indicated by is annotated indicated rectangles by annotated in the rectanglesbottom in theperspective bottom perspective view. The view.inset (white The inset rectangle) (white rectangle)shows the shows quality the of quality the textured of the textured3‐D model, 3-D with model, witharrows arrows pointing pointing to tothe the blue blue-gray‐gray markings markings left at left the at location the location of samples of samples 72255, 72255, 72215, 72215, and 72235. and 72235.

The second place of interest at Station 2 is a ~3‐m‐large boulder labeled “Boulder 2”, which rests on the Massif’s 14.9° slope (Figure 1). Figure 9 presents the 3‐D reconstruction that we obtained using 19 images labeled AS17‐137‐20912 to AS17‐137‐20956. The alignment generated a sparse point cloud of about 136,000 tie‐points. The dense point cloud of 8 million points allowed the generation of a mesh of 500,000 polygons. The gnomon is present in several images, allowing its 3‐D reconstruction together with the boulder. The boulder is similar in texture to the ones observed at intrusive melt‐ breccias at Stations 6 and 7.

Remote Sens. 2020, 12, 1900 11 of 18

The second place of interest at Station 2 is a ~3-m-large boulder labeled “Boulder 2”, which rests on the Massif’s 14.9◦ slope (Figure1). Figure9 presents the 3-D reconstruction that we obtained using 19 images labeled AS17-137-20912 to AS17-137-20956. The alignment generated a sparse point cloud of about 136,000 tie-points. The dense point cloud of 8 million points allowed the generation of a mesh of 500,000 polygons. The gnomon is present in several images, allowing its 3-D reconstruction together with the boulder. The boulder is similar in texture to the ones observed at intrusive melt-breccias at Remote Sens. 2020, 12, 1900 12 of 19 Stations 6 and 7.

Figure 9. Photogrammetric reconstruction (textured 3-D model) of boulder 2 at Station 2. The location Figure 9. Photogrammetric reconstruction (textured 3‐D model) of boulder 2 at Station 2. The location and orientation of the entry Apollo images labeled AS17-137-20912 to AS17-137-20925 as computed by and orientation of the entry Apollo images labeled AS17‐137‐20912 to AS17‐137‐20925 as computed Metashape are indicated in the bottom perspective view. The inset shows the location where samples by Metashape are indicated in the bottom perspective view. The inset shows the location where 72335 and 72315 were extracted. samples 72335 and 72315 were extracted. Finally, Boulder 3 at Station 2 was particularly interesting, as it contained crushed crystalline dunite,Finally, which Boulder turned 3 outat Station to be possibly2 was particularly ~4.55 billion interesting, years old as [ 31it ,contained32] and may crushed have crystalline originally crystallizeddunite, which near turned the base out of to the be lunar possibly magma ~4.55 ocean billion [33]. years We used old 11[31,32] images and labeled may AS17-137-20963have originally tocrystallized AS17-137-20973 near the to base reconstruct of the lunar this ~70-cmmagma brecciaocean [33]. boulder We used (Figure 11 images10). The labeled alignment AS17 generated‐137‐20963 a to AS17‐137‐20973 to reconstruct this ~70‐cm breccia boulder (Figure 10). The alignment generated a sparse point cloud of about 67,000 tie-points. The dense point cloud of about 13 million points was sparse point cloud of about 67,000 tie‐points. The dense point cloud of about 13 million points was used to generate a mesh of 200,000 polygons, taking into account the simple shape of the area. Only a used to generate a mesh of 200,000 polygons, taking into account the simple shape of the area. Only part of the gnomon was reproduced in 3-D. Several 2- to 4-cm clasts and one 10-cm light-gray clast a part of the gnomon was reproduced in 3‐D. Several 2‐ to 4‐cm clasts and one 10‐cm light‐gray clast clearly appeared in the 3‐D model, as two planar perpendicular fractures [30]. Samples 72415‐72418 were chipped from a large dunite clast (bright white spot in Figure 10). A sample (72435) of the boulder melt‐breccia matrix was chipped from the blue‐gray patch. It contains lithic and mineral clasts in a finely crystalline deep bluish‐gray matrix.

Remote Sens. 2020, 12, 1900 12 of 18 clearly appeared in the 3-D model, as two planar perpendicular fractures [30]. Samples 72415-72418 were chipped from a large dunite clast (bright white spot in Figure 10). A sample (72435) of the boulder melt-breccia matrix was chipped from the blue-gray patch. It contains lithic and mineral clasts in a ﬁnelyRemote Sens. crystalline 2020, 12, deep1900 bluish-gray matrix. 13 of 19

Figure 10. Photogrammetric reconstruction (textured (textured 3 3-D‐D model) model) of of the the 40 40-cm‐cm boulder boulder 3 3 at at Station Station 2. 2. The location andand orientation orientation of of the the 11 11 entry entry Apollo Apollo images images labeled labeled AS17-137-20963 AS17‐137‐20963 to to AS17-137-20973 AS17‐137‐20973 as computedas computed by Metashapeby Metashape are indicatedare indicated by annotated by annotated squares squares in the bottomin the bottom view. Sample view. Sample 72435 (matrix) 72435 was(matrix) extracted was extracted from the blue-grayfrom the blue patch.‐gray Samples patch. 72415-72418 Samples 72415 were‐72418 extracted were from extracted a large from dunite a large clast (whitedunite patch).clast (white These patch). sampling These areas sampling are well areas visible are on well the visible texture on of the the texture reconstructed of the boulder.reconstructed boulder. 3.4. Integration into Virtual Reality of the 3-D Models 3.4. IntegrationOnce accurately into Virtual reconstructed Reality of the using 3‐D SfM Models photogrammetry, the boulders’ 3-D models could be integrated into an immersive visualization system in order to be manipulated and analyzed, as if the Once accurately reconstructed using SfM photogrammetry, the boulders’ 3‐D models could be user were standing by himself in front of the boulders. Beyond the scientific analyses itself, the objective integrated into an immersive visualization system in order to be manipulated and analyzed, as if the was also to easily share interpretations, enhance scientific communication, or allow student education. user were standing by himself in front of the boulders. Beyond the scientific analyses itself, the For this step, we used two approaches. The first relied on a web-based public platform. The second objective was also to easily share interpretations, enhance scientific communication, or allow student made use of a game engine. A comparison of the respective pros and cons of each approach for a education. For this step, we used two approaches. The first relied on a web‐based public platform. terrestrial test case is discussed in [14]. The second made use of a game engine. A comparison of the respective pros and cons of each 3.4.1.approach Publication for a terrestrial of the Models test case on is a discussed Web-Based in Platform [14].

3.4.1.The Publication Sketchfab of 3-D the meshes Models repository on a Web‐ platformBased Platform (www.sketchfab.com ) provides a digital access and intuitive basic controls to interact with online 3-D objects of various scales, from centimeter-scale samples The Sketchfab 3‐D meshes repository platform (www.sketchfab.com) provides a digital access to large digital outcrop models, without the need for speciﬁc software and hardware. The visualization and intuitive basic controls to interact with online 3‐D objects of various scales, from centimeter‐scale can be done on either a traditional computer screen, on a smartphone, or using a virtual reality headset, samples to large digital outcrop models, without the need for specific software and hardware. The visualization can be done on either a traditional computer screen, on a smartphone, or using a virtual reality headset, directly online. When using a smartphone or a tablet, the 3‐D models can also be viewed in augmented reality. Contributors are allowed to upload models using a web‐based interface, and to define custom rendering options, such as lighting, material properties, and the size and orientation of the model with respect to the VR rendering. Annotations can be added to the model, in order to highlight areas of interest (such as sampling areas) and give relevant information.

Remote Sens. 2020, 12, 1900 14 of 19

This makes the exchange of data very user friendly. Snapshots of the full 3‐D models of the Station 2 Remote Sens. 2020, 12, 1900 13 of 18 and Station 6 boulders integrated in Sketchfab are displayed in Figure 11, together with a character showing the scaling of the objects in VR. The models are available online (e.g., directlyhttps://sketchfab.com/LPG online. When using‐3D). a smartphone or a tablet, the 3-D models can also be viewed in augmented reality.The Contributors rendering in are VR allowed allows toin uploadparticular models the user using to ainvestigate web-based the interface, morphology and to of deﬁne the boulders custom renderingat real scales. options, It provides such as a lighting, very user material‐friendly properties, way to visualize and the size and and analyze orientation the scientific of the model content with of respectthe whole to theset VRof integrated rendering. images. Annotations For example, can be added the distribution to the model, and in geometry order to highlightof the small areas dark of interestveins, which (such are as samplingintruding areas) noritic and breccia give clasts relevant in the information. Station 7 boulder This makes comprising the exchange angular of data minerals very userand lithic friendly. clasts, Snapshots are clearly of thevisible full in 3-D VR. models The latest of the generations Station 2 and of VR Station headsets 6 boulders have a integrated screen pixel in Sketchfabresolution, are which displayed is now in good Figure enough 11, together to distinguish with a character the fine showing details theon the scaling textures. of the Similarly, objects in VR.the Therenderings models could are available potentially online allow (e.g., quantitative https://sketchfab.com evaluation/ LPG-3Dof the vesicle). patterns in the boulders at Station 6.

Figure 11. SnapshotsSnapshots of the the reconstructed reconstructed boulder models that can be viewed online on the web web-based‐based Sketchfab platform (e.g., https:https://sketchfab.com/LPG//sketchfab.com/LPG-3D‐3D).). Any Any user user equipped equipped with with a virtual reality headset can stand in front of these boulders and investigate them in detail at a true scale in virtual headset can stand in front of these boulders and investigate them in detail at a true scale in virtual reality). The character is 1.80 m tall. reality). The character is 1.80 m tall. The rendering in VR allows in particular the user to investigate the morphology of the boulders at The size limitation on the uploadable meshes and textures in Sketchfab (50 MB in total for free real scales. It provides a very user-friendly way to visualize and analyze the scientiﬁc content of the accounts and 200 MB for pro accounts), and the availability of basic tools only with limited online whole set of integrated images. For example, the distribution and geometry of the small dark veins, computing resources, prevent the user from navigating in a complete environment. In particular, this which are intruding noritic breccia clasts in the Station 7 boulder comprising angular minerals and lithic web‐based solution is not well suited to the integration of orbital imagery with multiple digital clasts, are clearly visible in VR. The latest generations of VR headsets have a screen pixel resolution, outcrop models. For this more evolved purpose, we rather used a game engine, which optimizes the which is now good enough to distinguish the ﬁne details on the textures. Similarly, the renderings use of computing resources thanks to the graphical processing unit. could potentially allow quantitative evaluation of the vesicle patterns in the boulders at Station 6. 3.4.2.The Integration size limitation into a Game on the Engine uploadable meshes and textures in Sketchfab (50 MB in total for free accounts and 200 MB for pro accounts), and the availability of basic tools only with limited online computingCompared resources, to web prevent‐based theplatforms, user from game navigating engines in such, a complete as Unity environment. 3D or Unreal, In particular,developed this for web-basedand by the solutiongame industry, is not well provide suited a tomuch the integration more versatile of orbital way to imagery integrate, with manipulate, multiple digital and merge outcrop a models.large quantity For this of moreplanetary evolved orbital purpose, and in we situ rather data used(e.g., a [9,10,14]). game engine, For our which project, optimizes we selected the use the of computingUnity 3D platform resources (version thanks to2019.2.15 the graphical f1), which processing is thoroughly unit. documented through user manuals,

Remote Sens. 2020, 12, 1900 14 of 18

3.4.2. Integration into a Game Engine Compared to web-based platforms, game engines such, as Unity 3D or Unreal, developed for and byRemote the Sens. game 2020 industry,, 12, 1900 provide a much more versatile way to integrate, manipulate, and merge a15 large of 19 quantity of planetary orbital and in situ data (e.g., [9,10,14]). For our project, we selected the Unity 3D platformextensive (versiontutorials, 2019.2.15 and forums. f1), which We designed is thoroughly a simple documented scene using through the Unity user terrains manuals, asset, extensive which tutorials,allows the and generation forums. Weof designedreal terrains a simple based scene on 16 using‐bit DEM the Unity files terrains for height asset, values, which draped allows theby generationcorresponding of real texture terrains images. based on The 16-bit Kaguya DEM ﬁles TC, for Clementine height values, UVVIS, draped and by correspondingLRO NAC mosaics texture images.discussed The in Section Kaguya 2.1 TC, were Clementine used as input UVVIS, to andgenerate LRO these NAC various mosaics terrains’ discussed renderings in Section (Figures 2.1 were 2 usedand 12). as inputThe advantages to generate of these using various the built terrains’‐in Unity renderings terrains generator (Figures2 is and that 12 optimization). The advantages functions of usingare already the built-in implemented Unity terrains to reduce generator the load is on that computer optimization resources functions when are displaying already implemented data at various to reducescales. The the loadlevel onof details computer depends resources mostly when on the displaying distance data of the at player various (the scales. rendering The level camera) of details with dependsrespect to mostly the ground. on the distance of the player (the rendering camera) with respect to the ground.

Figure 12. IntegrationIntegration in in a game a game engine, engine, showing showing a scene a scene composed composed using using 3‐D products 3-D products derived derived from fromboth bothorbital orbital imagery imagery and andphotogrammetric photogrammetric reconstructions reconstructions based based on on in in situ situ photographs photographs of of the boulders at Station 6 (up) and Station 7 (bottom). This solution provides insights toward a “VR ﬁeldfield geology” characterization of the impact melt boulders in Taurus Littrow.Littrow.

The 3-D3‐D models of the outcrops can then be directly added to the scene by using a simple drag and drop function on the scene hierarchy and placed at the relevant location with the right orientation and scaling. We We used the Autodesk FilmBoX (.fbx) ﬁlefile export format from Metashape,Metashape, associated with corresponding jpeg jpeg textures, textures, for for this this purpose. purpose. The The advantage advantage of ofthe the .fbx .fbx format format is to is export to export the thecamera camera positions positions together together with with the 3 the‐D 3-Dmodel. model. Navigation Navigation within within the scene the scene and interactions and interactions with withelements elements were programmed were programmed in C#. In in order C#. In to order provide to providethe user thewith user the possibility with the possibility of freely exploring of freely exploringthe entire scene, the entire we used scene, a first we‐ usedperson a ﬁrst-person movement script, movement allowing script, either allowing to walk either at the to surface walk at or the to fly over the landscape. Various components can be added to the scene, such as a skybox with stars, a 3‐D model of the lunar module, in situ 360° panoramas, or sound recordings of the actual astronauts’ conversations, which are triggered when the user is approaching relevant elements, such as the boulders. A user interface control panel is available to switch between different textures (i.e., Kaguya TC versus Clementine) or to directly go to points of interest.

Remote Sens. 2020, 12, 1900 15 of 18 surface or to ﬂy over the landscape. Various components can be added to the scene, such as a skybox with stars, a 3-D model of the lunar module, in situ 360◦ panoramas, or sound recordings of the actual astronauts’ conversations, which are triggered when the user is approaching relevant elements, such as the boulders. A user interface control panel is available to switch between diﬀerent textures (i.e., Kaguya TC versus Clementine) or to directly go to points of interest.

4. Discussion and Future Work One of the main interests of the VR reconstruction is to give a real sense of scales and shapes, without the deformations induced by reprojections on a conventional computer screen. The surface of the Moon lacks atmospheric scattering (inducing distance effects on Earth) and known features, such as trees, roads, or any known reference, that could help the brain to figure out the distances, and therefore the size (and real shape) of landscapes elements. This problem is easily overcome by VR. One main advantage of the 3-D reconstruction compared to single images is it also provides a synoptic view of the rocks and/or outcrops, including their complex real shape, without the loss of any scientific information from the original set of data. In the case of the boulders at Station 6, the detailed vesicular nature of the rock clearly appears when viewed in VR, while it is not so obvious when seen on a conventional flat screen. The exact emplacement of several lunar rock samples also appears readily when viewed in VR (ex. inset of Figure8). Further work using the 3-D scans of lunar samples themselves might allow in some cases to replace the samples directly in their exact position in VR, which might, for example, be useful for paleo magnetism studies that require the exact position of the sample to be of any relevance [8]. The “astromaterials 3D” initiative at the Astromaterials Research and Exploration Science division at the Johnson Space Center is currently aiming at providing research-grade photogrammetric exterior 3-D models of several tens of lunar samples. The detailed comparison of these 3-D meshes of lunar samples with 3-D meshes of originating rocks, when available and sufficiently detailed, could, for example, be performed using the open source “CloudCompare” (www.cloudcompare.org) software, to look for matching correspondences on the common faces of the biggest samples. This search for correspondences can be envisaged in further studies, when a sufficiently high number of samples will be digitized and available. Sampling activities during extra-vehicular activities were extremely limited in time, in order to maximize the scientific return of the mission. Using the VR reconstruction of the main places of interest allows users to virtually “come back” to the sites, to analyze them without any time (and space) constraints. Whereas it is obviously not the same as being there, it can help to bridge the gap between a simple computer screen experience and reality. The photogrammetry technique can also eventually be applied to descent images when available to compute a local high-resolution digital elevation model [26] and to retrieve the descent spacecraft trajectory [34]. Future surface photogrammetric work could also be carried out with the full-resolution scans of the Apollo Image Archive. However, with ~300 MB per image, this would require a very powerful computing system, for an improvement which is still to be evaluated, depending often on the focus quality of the views. These original high-resolution scans also need to be centered and cropped in a consistent manner before being usable in an automated photogrammetric pipeline. It could also be interesting to further work on the lighting conditions in the reconstructed virtual world, in order to merge orbital data with ground-based data taken with a solar azimuth and elevation as close as possible. Figure 12 uses the LRO image M129086118LR for this purpose. Another optimization of the final products could be the generation of normal maps to realistically render the micro relief on the boulders without overloading the graphic processor. Whereas this falls beyond the scope of this work, game engines provide the possibility to fine-tune all the sources of light and their mutual interactions, including multiple scattering effects, which should help in providing an even more realistic experience. This will be particularly interesting with the next generation of graphic cards, which take into account the ray tracing. Remote Sens. 2020, 12, 1900 16 of 18

5. Conclusions We reconstructed in 3-D and integrated in virtual reality the boulders explored by Apollo 17 astronauts during their EVA3 at Station 6 and Station 7 and the three main boulders observed at Station 2. Whereas recent photogrammetric projects are now mainly carried out with images taken from digital cameras, we observed that the reconstruction process can still work easily with Apollo images, thanks to the quality and homogeneity of the scans of the original films performed at the Johnson Space Center performed in the last decade. We provide reconstructed 3-D models that can be viewed online either on a traditional computer screen, or directly in virtual and augmented reality, using the web-based Sketchfab platform. The rendering in the virtual world can eventually be merged with orbital imagery and in situ 360◦ panoramas to provide the general context. This approach could ultimately allow the placement of the collected lunar samples in their original context, and help to precisely characterize their sampling area. Particular care has to be taken to handle the lighting conditions in the virtual world to provide a realistic experience. It is very likely that geological fieldtrips will be available in virtual reality in the near future in many places of interest, either for scientific investigations or for student education, as well as public outreach and historic conservation. This is particularly useful in the case of remote (and almost inaccessible) places, such as moons and planets, where only 12 human beings have experienced a physical field trip so far. The work that we carried out here suggest that the versatility of this approach allows similar reproductions of the geological settings documented in many photographs of the Apollo collection. In the future, similar reconstructions could also be applied to the imagery acquired by the ongoing and forthcoming lunar robotic missions, preparing the next generation of exploration. Realistic 3-D surface models can, for example, be used in simulations dedicated to the training of future astronauts taking an active role in the forthcoming Artemis lunar exploration program.

Author Contributions: Conceptualization, methodology, S.L.M., P.E.; software, S.L.M., G.C., P.E., F.C.; validation, H.H.S.; image resources, H.H.S., B.S., J.-P.C.; writing—original draft preparation, review and editing, S.L.M.; supervision, N.M.; project administration, funding acquisition, N.M. All authors have read and agreed to the published version of the manuscript. Funding: Part of this research (G.C.) benefited from the support of the European Commission Research Executive Agency under the Horizon 2020 “PlanMap” project (grant n◦ 776276). J.-Ph. Combe was funded by the NASA Lunar Data Analysis Program 17-LDAP17_2-0067. Acknowledgments: The authors thank the NASA/JSC/Arizona State University for providing the scanned imagery of the Apollo missions. We are grateful to three anonymous reviewers for their constructive remarks, and we also would like to thank Carolyn Van Der Bogert for useful discussions. Conflicts of Interest: The authors declare no conflict of interest.

References

1. McGreevy, M.W. Virtual reality and planetary exploration. In Virtual Reality; Elsevier: Amsterdam, The Netherlands, 1993; pp. 163–197, ISBN 0-12-745045-9. 2. Ullman, S. The interpretation of structure from motion. Proc. R. Soc. Lond. Ser. B Biol. Sci. 1979, 203, 405–426. [CrossRef] 3. Verhoeven, G. Taking computer vision aloft-archaeological three-dimensional reconstructions from aerial photographs with photoscan. Archaeol. Prospect. 2011, 18, 67–73. [CrossRef] 4. Favalli, M.; Fornaciai, A.; Isola, I.; Tarquini, S.; Nannipieri, L. Multiview 3D reconstruction in geosciences. Comput. Geosci. 2012, 44, 168–176. [CrossRef] 5. Arbuées, P.; García-Sellés, D.; Granado, P.; Lopez-Blanco, M.; Muñoz, J. A method for producing photorealistic digital outcrop models. In Proceedings of the 74th EAGE Conference and Exhibition Incorporating EUROPEC, Copenhagen, Denmark, 4–7 June 2012; p. cp-293-00121, Abstract #D029. 6. Tavani, S.; Granado, P.; Corradetti, A.; Girundo, M.; Iannace, A.; Arbués, P.; Muñoz, J.A.; Mazzoli, S. Building a virtual outcrop, extracting geological information from it, and sharing the results in Google Earth via OpenPlot and Photoscan: An example from the Khaviz Anticline (Iran). Comput. Geosci. 2014, 63, 44–53. [CrossRef] Remote Sens. 2020, 12, 1900 17 of 18

7. Ostwald, A.; Hurtado, J. 3D models from structure-from-motion photogrammetry using Mars science laboratory images: Methods and implications. In Proceedings of the 48th Lunar and Planetary Science Conference, The Woodlands, TX, USA, 20–24 March 2017. LPI Contribution No. 1964, id.1787. 8. Schmitt, H.H.; Petro, N.E.; Wells, R.A.; Robinson, M.S.; Weisss, B.P.; Mercer, C.M. Revisiting the field geology of Taurus-Littrow. Icarus 2017, 298, 2–33. [CrossRef] 9. Civet, F.; Le Mouélic, S.; Le Menn, E.; Beaunay, S. Using Virtual Reality for Outreach Purposes in Planetology. In Proceedings of the American Astronomical Society, DPS meeting #48, Washington, DC, USA, 16–21 October 2016. id.419.10. 10. Le Mouélic, S.; L’Haridon, J.; Civet, F.; Mangold, N.; Triantafyllou, A.; Massé, M.; Le Menn, E.; Beaunay, S. Using virtual reality to investigate geological outcrops on planetary surfaces. In Proceedings of the 20th EGU General Assembly, EGU2018, Conference held, Vienna, Austria, 4–13 April 2018; p. 13366. 11. Triantafyllou, A.; Watlet, A.; Le Mouélic, S.; Camelbeeck, T.; Civet, F.; Kaufmann, O.; Quinif, Y.; Vandycke, S. 3-D digital outcrop model for analysis of brittle deformation and lithological mapping (Lorette cave, Belgium). J. Struct. Geol. 2019, 120, 55–66. [CrossRef] 12. Caravaca, G.; Le Mouélic, S.; Mangold, N.; L’Haridon, J.; Le Deit, L.; Massé, M. 3D digital outcrop model reconstruction of the Kimberley outcrop (Gale crater, Mars) and its integration into Virtual Reality for simulated geological analysis. Planet. Space Sci. 2020, 182, 104808. [CrossRef] 13. Mat, R.C.; Shariff, A.R.M.; Zulkifli, A.N.; Rahim, M.S.M.; Mahayudin, M.H. Using game engine for 3D terrain visualisation of GIS data: A review. In IOP Conference Series: Earth and Environmental Science; IOP Publishing: Bristol, UK, 2014; Volume 20, p. 012037. 14. Nesbit, P.R.; Boulding, A.D.; Hugenholtz, C.H.; Durkin, P.R.; Hubbard, S.M. Visualization and Sharing of 3D Digital Outcrop Models to Promote Open Science. GSA Today 2020, 30, 4–10. [CrossRef] 15. Haase, I.; Wählisch, M.; Gläser, P.; Oberst, J.; Robinson, M. Coordinates and Maps of the Apollo 17 Landing Site. Earth Space Sci. 2019, 6, 59–95. [CrossRef] 16. Wolfe, E.W.; Bailey, N.G.; Lucchitta, B.K.; Muehlberger, W.R.; Scott, D.H.; Sutton, R.L.; Wilshire, H.G. Geologic investigation of the Taurus-Littrow Valley: Apollo 17 landing site. U.S. Geol. Surv. Prof. Pap. 1981, 1080, 225–280. 17. Haruyama, J.; Matsunaga, T.; Ohtake, M.; Morota, T.; Honda, C.; Yokota, Y.; Torii, M.; Ogawa, Y. LISM Working Group. Global lunar-surface mapping experiment using the Lunar Imager/Spectrometer on SELENE. Earth Planets Space 2008, 60, 243–255. [CrossRef] 18. Barker, M.K.; Mazarico, E.; Neumann, G.A.; Zuber, M.T.; Haruyama, J.; Smith, D.E. A new lunar digital elevation model from the Lunar Orbiter Laser Altimeter and SELENE Terrain Camera. Icarus 2016, 273, 346–355. [CrossRef] 19. McEwen, A.S.; Robinson, M.S. Mapping of the Moon by Clementine. Adv. Space Res. 1997, 19, 1523–1533. [CrossRef] 20. Lucey, P.G.; Blewett, D.T.; Taylor, G.J.; Hawke, B.R. Imaging of lunar surface maturity. J. Geophys. Res. Planets 2000, 105, 20377–20386. [CrossRef] 21. Kammerer, J.; Zeiss, C. The moon camera and its lenses. Opt. Eng. 1972, 11, 73–78. [CrossRef] 22. Batson, R.M.; Larson, K.B.; Tyner, R.L. Apollo 17 lunar surface photography, in Geologic Investigation of the Taurus-Littrow Valley: Apollo 17 Landing Site. U.S. Geol. Surv. Prof. Pap. 1981, 1080, 225–280. 23. Lawrence, S.J.; Robinson, M.S.; Broxton, M.; Stopar, J.D.; Close, W.; Grunsfeld, J.; Ingram, R.; Jefferson, L.; Locke, S.; Mitchell, R.; et al. The Apollo digital image archive: New research and data products. In Proceedings of the NLSI Lunar Science Conference, California, CA, USA, 22–23 July 2008; p. 2066. Available online: https://www.lpi.usra.edu/meetings/nlsc2008/pdf/2066.pdf (accessed on 11 May 2020). 24. Borgeson, W.T.; Batson, R.M. Photogrammetric Calibration of Apollo Film Cameras; Usgs Open-File Report N69-N27911; U.S. Geological Survey: Reston, VA, USA, 1969. 25. iWitnessPro. 2015. Available online: https://www.photometrix.com.au/iwitness/ (accessed on 11 May 2020). 26. Manheim, M.; Wagner, R.; Klem, S.; Robinson, M. Photoscan DEMs from Apollo 15 Hasselblad photographs. In Proceedings of the European Planetary Science Congress, Berlin, Germany, 16–21 September 2018; Volume 12. EPSC2018-996. 27. Agisoft LLC. Metashape Professional. 2020. Available online: https://www.agisoft.com (accessed on 30 January 2020). Remote Sens. 2020, 12, 1900 18 of 18

28. Meyer, C. Lunar Sample Compendium. In Proceedings of the 41st Lunar and Planetary Science Conference, The Woodlands, TX, USA, 1–5 March 2010; p. 1016, LPI Contribution No. 1533. 29. Reimold, W.U. Pseudotachylite in impact structures—generation by friction melting and shock brecciation?: A review and discussion. Earth Sci. Rev. 1995, 39, 247–265. [CrossRef] 30. Schmitt, H.H.; Cernan, E.; Lyndon, B.; Johnson Space Center. Apollo 17: Preliminary Science Report; Scientific and Technical Information Office: Washington, DC, USA, 1973. 31. Papanastassiou, D.A.; Wasserburg, G.J. Rb-Sr study of a lunar dunite and evidence for early lunar differentiates. In Proceedings of the Lunar and Planetary Science Conference, New York, NY, USA, 17–21 March 1975; pp. 1467–1489. 32. Ryder, G. Chemical variation and zoning of olivine in lunar dunite 72415: Near-surface accumulation. In Proceedings of the 22nd Lunar and Planetary Science Conference, Houston, TX, USA, 18–22 March 1991; Lunar Planetary Institute: Houston, TX, USA, 1992; pp. 373–380. 33. Schmitt, H.H. Symplectites in dunite 71415 and troctolite 76535 indicate mantle overturn beneath lunar near-side. In Proceedings of the 47th Lunar and Planetary Science Conference, The Woodlands, TX, USA, 21–25 March 2016; p. 2339, LPI Contribution No. 1903. 34. Binet, R.; Grizonnet, M.; Torres, A.; Malapert, J.-C.; Jocteur-Bronzier, F. Lunar Landing Site Localization,

Trajectory Inversion, and DTM Update from CHANG0E-3 Descent Images. In Proceedings of the Lunar and Planetary Science Conference, The Woodlands, TX, USA, 18–22 March 2019. LPI Contribution No. 2132, id.2433.

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).