50th LPSC Houston, Texas, March 18–22, 2019 #2226 Christian Koeberl1, Gerhard Paar2, Robert Barnes3, Sanjeev Gupta3, Christoph Traxler4 , Thomas Ortner4 1Natural History Museum, Burgring 7, 1010 , , and Department of Lithospheric Research, , 1090 Vienna, Austria, [email protected]; 2JOANNEUM RESEARCH, Graz, Austria, [email protected]; 3Imperial College London, UK 4VRVis Zentrum für Virtual Reality und Visualisierung Forschungs-GmbH, Vienna, Austria;

Introduction & Rationale PRo3D for training AI-based science assessment from Planetary rover imagery

• Planetary rover missions use panoramic stereo camera systems to image rock outcrops along rover The presented idea primarily targets the support of ground traverses. The Mars 2020 rover will also carry a Panoramic Camera System (Mastcam-Z, [2]) to operations and the prospects for categorization of existing obtain multi-spectral stereoscopic panoramic images. mission archives. Its building blocks shown in Figure 1: • Mastcam-Z will provide 3D vision capabilities from which the bedrock geology, sedimentary • Training uses planetary images from past and present stratigraphy, impact cratering history, and paleoenvironmental information can be reconstructed missions &relevant field campaigns (MER, MSL, ExoFit, from Digital Outcrop Models (DOMs, [4]).using the 3-D vision processing framewok PRoViP [1] and PRoViScout). the visualization software PRo3D [3]. • Training is foreseen using three approaches: • As the Mars-2020 Landing Site Jezero crater is an , it can be expected that impact – manual annotation by experts lithologies are encountered, such as shatter cones. The study of planetary impact products, – known targets from existing catalogues including the detection and understanding of impact breccias, shatter cones, and other materials of – simulation (i.e. virtual placement of known targets in true context impact origin is important to understanding impact processes. environment) • Here we investigate how to recognize materials of impact origin in planetary datasets by (1) placing • Cues are both 2D (multi-spectral/monochrome) and 3D shatter cones in a Mars analog environment (Danakil depression) and taking multiple images used • Cross-instrument cues are being investigated for 3D reconstructions of virtual Digital Outcrop Models, and (2) placing reference objects therein, • Special emphasis is laid on strategic objects of interest such which is an important validation component of validation of automatic recognition: as , shatter cones, or bedrock structures to keep • In such way we investigate how we can recognize such features on Mars in rover-derived 2D and 3D variability on a reasonable level and rather approach an end- Figure 1: Logic of an AI-based training scheme for Planetary Science datasets. We intend to realize a deep-learning based AI scheme that is able to support scientists to to-end result for specific classes. assessment primarily based on PRo3D simulations find scientifically relevant regions and objects on images captured along Planetary Rover routes. Danakil Depression – a Mars Analog Environment Planetary Robotics 3D Viewer (PRo3D)

The Danakil Depression is located in North-Eastern Ethiopia, near the border with Eritrea. It sits around 125 m The interactive 3D viewing tool PRo3D [3] allows virtual exploration of reconstructed Martian terrain and geologic below sea level and is at the junction of three of the Earth's lithospheric plates (Arabian, Nubian and Somalian). analysis of 3D datasets. Main functionality for the activity described here is virtual placement of objects within (real- Danakil has a large number of extreme environments that measured) 3D environment form an intricate complex geological and biological setting with volcanoes, hydrothermal systems, salt flats and deposits, and extreme microbial communities, which have been used in the past 2 years as Mars analog sites through Training Data by PRo3D: Placing Reference Objects in Virtual Views from Real Observations

2 Europlanet expeditions (Credits: Google Earth). 100 m 100 km Mastcam-Z instrument Figure 5: Shatter Cones Shatter Cones’ Placement in Danakil Partial data and PRo3D screenshot of shatter cones • created during high- In January 2019 one of us (C.K., Figure 2) participated in an expedition within possibly allow the pressure identification and study placed in impacts. the framework of Europlanet to the Mars analog site at the Danakil Depression Mars-like • Shatter cones take on in Ethiopia. A shatter cone was placed in a variety of geological setting that of impact breccias and environment a variety of sizes and their clast population, (middle of shapes depending on might be similar to those encountered in eroded impact crater central uplifts the image) in the pressure at impact as well as the possible or rims, and the shatter cone (from the Boysen Shale at the Vredefort impact the PRo3D and the rock being detection (for the first impacted structure in South Africa, 2 billion years old; Figure 3) was photographed in a software suite. Field • distinctive fan-shaped variety of such environments and light conditions. Two such shatter cone time) of shatter cones, “horsetail” structures with implications for tests should that resemble images in Figure 4 are shown as examples. These and others will be used to help to striations. validate PRO3D. shock metamorphic provide more • naturally sought and realistic identified in the studies on Mars. simulation vicinity of impact To train scientists and structures and serve as conditions critical proof that an future automatic and validate impact has occurred in systems that search for the a given area. simulation • Criteria to distinguish: scientifically interesting method • recognizing the distinctive horsetail, targets in rapidly fan-like structure; growing imagery from • recognizing the orientation of the Planetary rovers, PRo3D Figure 6: striations; and offers the possibility to Further cases gathering data in the place virtual reference of virtual vicinity of a find to shatter cone indicate a past impact objects in truly event (i.e., the placement presence of shocked Figures 4: shatter cone placed in geological measured environment. (left) and 3D quartz) Figure 2: Author CK with shatter cone setting at Danakil, Ethiopia, similar to what can Here, shatter cones renderings of • Mapping field criteria in the field at Danakil, Ethiopia be expected at impact crater lithologies. (Photos were reconstructed in the shatter and semantics to taken from different distances). cones Machine Vision is 3D before being placed zoomed problematic (right) • “horsetail” color, in PRo3D renderings texture/structural (Figure 5 and Figure 6). trends, structural and textural frequency, diameters of super- structures etc. has much variation • The “orientation of striations” - can be Acknowledgements References analyzed by computer vision techniques, only This work receives ESA-PRODEX funding, supported by the Austrian [1] Paar, G, et al. Vision and Image Processing. In Gao, Yang, ed. Contemporary Planetary Robotics: An Approach Toward Autonomous Systems (2016). John Wiley & Sons, 2016. given well-defined Research Promotion Agency under ESA PEA Grants 4000105568 & data acquisition / [2] Bell, J. F. et al (2016). Mastcam-Z: Designing a Geologic, Stereoscopic, and Multispectral Pair of Zoom Cameras for the NASA Mars 2020 Rover. 4000117520. RB and SG were funded by UKSA grant ST/P002064/1. Thanks LPI Contributions, 1980. illumination also to the Mastcam-Z and PanCam PI-s and Teams for their kind support in [3] R. Barnes, S. Gupta, C. Traxler, T. Ortner, A. Bauer, G. Hesina, G. Paar, B. Huber, K. Juhart, L. Fritz, B. Nauschnegg, J.-P. Muller, Y. Tao (2018) conditions, proper terms of requirements setting, data provision and valuable comments to Geological analysis of Martian rover‐derived Digital Outcrop Models using the 3D visualisation tool, Planetary Robotics 3D Viewer – PRo3D. Earth and Space Science - An Open Access AGU Journal resolution, knowledge tools, results and validation plans. The field work by CK was sponsored by about scale, and better [4] Banham, S.G., Gupta, S., Rubin, D., Watkins, J., Edgett, K.S., Sumner, D., Grotzinger, J., Lewis, K., Edgar, L., Stack, K., Barnes, R., Bell, J.F., Day, M., Figure 3: The same shatter Europlanet, with support from International Research School of Planetary Ewing, R., Lapotre, M.G., Rivera-Hernandez, F. and Vasavada, A.R. (2018): Ancient Martian aeolian processes and palaeomorphology reconstructed in 3D cone in a studio setting Sciences in Pescara, Italy , Europlanet 2020 from the Stimson formation on the lower slope of Aeolis Mons, Gale crater, Mars. Sedimentology. V. 65, 993-1042. Doi: 10.1111/sed.12469. https://www.hou.usra.edu/meetings/lpsc2019/eposter/2226.pdf MSL Mastcam Sol 538, Data Courtesy NASA/JPL-Caltech/MSSS, data processing JOANNEUM RESEARCH / PRoViDE www.PRoViDE-space.eu