i-SAIRAS2020-Papers (2020) 5064.pdf ENABLING AUTONOMY IN COMMERCIAL-CLASS LUNAR MISSIONS Virtual Conference 19–23 October 2020 Kaizad Raimalwala1, Michele Faragalli1, Evan Smal1, Melissa Battler1, Ewan Reid1, Braden Stefanuk2, Krzysztof Skonieczny2 1Mission Control Space Services, 162 Elm St. West, Ottawa, ON K1R 6N5, Canada E-mail: [email protected] 2Concordia University, 1455 De Maisonneuve Blvd. W., Montreal, QC, Canada, Email: [email protected] ABSTRACT nominal operations at mid/low latitudes will likely be 10-12 Earth days. Payloads must also share a con- Early Lunar micro-rover missions will be short in du- strained downlink capacity. Payload operators gener- ration and have constrained downlink capacity. To ating high volumes of data face the problem of not re- maximize the scientific return of these missions, Mis- ceiving data in a timely fashion to influence their op- sion Control is developing technologies to autono- erations or worse, leaving valuable data on the Moon. mously classify geological features and detect novel These constraints are motivating the need for innova- features in rover camera imagery, which can be used tive concept of operations and technologies to ensure to support intelligent decision-making for prioritizing customer satisfaction, and the viability of this new data for downlink and instrument targeting. In a re- model of exploration. cently completed concept study, a trade-off analysis and performance evaluation were conducted for the 1.2 Mobility and Science Operations terrain classifier and novelty detector algorithms Several factors are driving the need for autonomy in across multiple datasets. The terrain classifier devel- mobile science operations. In traditional Mars rover oped achieved accuracies of 77%-86% and Intersec- operations, visual surface characterization and subse- tion over Union (IoU) scores of 0.667-0.680 across 10 quent analysis and decision-making takes place in day- different terrain type, on 3 distinct data sets (totalling long tactical cycles [1]. Upcoming Lunar rover mis- 928 images), demonstrating the robustness of the ap- sions, however, will have reduced latency, short life- proach to varying illumination conditions. In ongoing times, and constrained bandwidth. This will result in a work, a comprehensive Lunar analogue dataset is be- need for rapid tactical decision-making processes with ing developed to continue prototyping, and the algo- limited data, leaving little time to analyze data, iden- rithms are being developed on an embedded processor tify features of interest, and make decisions. for a flight demonstration opportunity. The highly anticipated NASA VIPER rover that will 1 INTRODUCTION fly to the south polar region is a large rover (~300kg) In this section, we begin with highlighting the chal- but will have a constrained direct-to-Earth communi- lenges facing Lunar rover operations and the motiva- cations channel of 230 kbps [2]. Small-scale commer- tions for including technologies that enable autono- cial Lunar rovers will also be constrained; a 10 kg mous perception and decision-making. In Section 2, rover deployed in Astrobotic’s Mission One will be al- we explain intended onboard and offboard use cases located 200 kbps according to standard payload data and User Experience (UX) considerations for how im- rate allocation advertised in their Payload User Guide age annotations that are autonomously created by the (PUG) [3]. As per their CubeRover PUG, a 6kg pay- terrain classifier can be used in a Lunar rover opera- load will be allocated 60 kbps [4]. tions tactical cycle. In Section 3, we provide an over- Sensing capabilities are growing increasingly power- view of technologies with recent results from trade-off ful but data transfer rates are not sufficiently high to analysis of the terrain classifier on datasets. In Section downlink high volume data in short decision-making 4, we highlight open challenges and next steps. timescales. To maximize scientific return, it will be 1.1 Challenges in Commercial Lunar Missions important to have methods to intelligently compress or select data to downlink in real-time or to select key ge- Between the harsh Lunar environment and economic ological features to measure. pressures on companies, early Lunar surface missions will be limited to a Lunar day (14 Earth days). and i-SAIRAS2020-Papers (2020) 5064.pdf The nature of scientific discovery makes onboard au- The Mission Control terrain classifier was first devel- tonomy compelling. It increases the chances of detect- oped under the CSA-funded Autonomous Soil Assess- ing valuable novel/sparse features that may otherwise ment System [14]. In 2019, it was used onboard a rover be missed in scenarios that prioritize driving and other to classify eight Mars-relevant terrain types in real- mission needs. For example, NASA’s Opportunity time at ~15 FPS as the rover drove at 20cm/s at a high- rover was driven 600 ft past the Block Island meteor- fidelity analogue site in Iceland (see Figure 1). This ite, one of its biggest discoveries, before the science was a part of SAND-E (Semi-Autonomous Navigation team discovered it and decided to drive back to inves- for Detrital Environments), a NASA PSTAR funded tigate it [5]. project to inform Mars2020 operations [13]. With tactical cycles a few minutes long and pressure to achieve science objectives, missions will benefit from autonomy in data processing and decision-mak- ing. The ASAS-CRATERS (Autonomous Soil Assess- ment System: Contextualizing Rocks, Anomalies and Terrains in Exploratory Robotic Science) system de- veloped by Mission Control offers such capabilities, with the goal of maximizing scientific return in up- Figure 1: Result from field-testing the deep-learning coming missions [6]. based terrain classifier in Iceland. 1.3 State of the Art in Autonomous Perception In a previous paper [7], we offered a detailed survey Recent work by Kerner et al. [15], [16] have demon- of modern perception and modeling technologies for strated the capability to detect novel geological fea- planetary surface robotics. The state-of-the-art in ter- tures in multispectral images of the Martian surface. rain classification leverages high performance Convo- They show that a spatial-spectral error map can enable lutional Neural Networks (CNNs) that find natural fea- both accurate classification of novelty in multispectral tures and complex patterns in the image. images as well as human-comprehensible explanations of the detection. For example, Soil Property and Object Classification (SPOC) [8] has a terrain classifier that uses Fully 2 USE CASES AND USER EXPERIENCE IN CNNs (FCNNs). Gonzalez and Iagnemma [9] re- LUNAR ROVER OPERATIONS cently published a comparative analysis of CNNs, In this section, we focus on the use cases of the per- Deep Neural Networks, and classical algorithms such ception algorithms (terrain classification and novelty as Support Vector Machines. These and other works detection) in onboard applications to enable autono- have focused on classifying Mars surface images to mous behaviour to benefit science and navigation improve autonomy for Mars rovers. alike, and offboard applications to support science For Lunar applications, terrain classification moti- backroom tasks. vated by scientific research has focused on crater de- 2.1 Onboard Applications tection using orbital data. Stepinski et al. and Chung et al. offer a review of traditional machine learning tech- In Lunar rover operations, we expect two types of data niques, including SVMs [10], [11]. More recently, Sil- collection modes for onboard science instruments, in burt et al. [12] explored the use of CNNs to detect cra- particular those that are scanning the surface: ters using a DEM merged from LRO and Kaguya data. Continuous Mode: The instrument is set to capture While these studies have successfully demonstrated data at a set time or distance-based rate. This mode is the use of deep learning to improve terrain classifica- relevant during scouting operations or driving between tion of images from Mars rover datasets or from a la- destinations. Instrument measurements are desired at boratory setting, only recent work by Mission Control whatever rate is feasible to downlink given other pay- has holistically studied terrain classification in a real- load and telemetry allocation. time system for a science-driven rover mission and its We assume a stronger constraint on downlink capacity implications on mission operations [13]. Additional than on available power for data capture, i.e. more data work, as presented in this paper, has demonstrated the can be captured and stored onboard than is possible to use of this technology on Lunar datasets. downlink in real-time. In this case, we must maximize the scientific value of this data. i-SAIRAS2020-Papers (2020) 5064.pdf Targeted Mode: The instrument is triggered to capture • Efficient evaluation and selection of a feature to re- data at specific instances due to higher power con- quest a navigation stop or instrument targeting, fol- straint that does not make continuous measurements lowed by communication of the request internally feasible. It is desired to target the instrument to scan within the operations team. specific and high-priority features of interest. • Features can be catalogued in a database, enabling In both modes, autonomous perception can support feature-based query in real-time which can be onboard decision-making, i.e. intelligent downlink of highly beneficial in short-duration missions. E.g., continuous data in the first mode, or intelligent target- an operator can quickly retrieve all fresh craters of ing of the instrument in the second mode. size 3-4m in a specific geographic area. With a semantic understanding of the nearby terrain, • Features can be projected on a map frame, and and using science operator defined rules, the onboard map-based data products can be easily integrated system can decide whether the instrument data is suit- into GIS tools for rapid analysis with the context of able for immediate downlink. Similar to the AEGIS scale and other information layers derived in situ system pioneered for Mars rovers [17], we intend to or from orbit.