Advances in Deep Space Exploration via Simulators & Deep Learning James Bird1,∗, Linda Petzold1, Philip Lubin2, Julia Deacon3 Abstract The NASA Starlight and Breakthrough Starshot programs conceptualizes fast interstellar travel via small relativistic spacecraft that are propelled by directed energy. This process is radically different from traditional space travel and trades large and slow spacecraft for small, fast, inexpensive, and fragile ones. The main goal of these wafer satellites is to gather useful images during their deep space journey. We introduce and solve some of the main problems that accompany this concept. First, we need an object detection system that can detect planets that we have never seen before, some containing features that we may not even know exist in the universe. Second, once we have images of exoplanets, we need a way to take these images and rank them by importance. Equipment fails and data rates are slow, thus we need a method to ensure that the most important images to humankind are the ones that are prioritized for data transfer. Finally, the energy on board is minimal and must be conserved and used sparingly. No exoplanet images should be missed, but using energy erroneously would be detrimental. We introduce simulator-based methods that leverage artificial intelligence, mostly in the form of computer vision, in order to solve all three of these issues. Our results confirm that simulators provide an extremely rich training environment that surpasses that of real images, and can be used to train models on features that have yet to be observed by humans. We also show that the immersive and adaptable environment provided by the simulator, combined with deep learning, lets us navigate and save energy in an otherwise implausible way. Keywords: computer vision, simulator, deep learning, space, universe, exoplanet, object detection, novelty detection 1. Introduction ∗ arXiv:2002.04051v2 [astro-ph.IM] 6 Jun 2020 Corresponding author Space travel, up until recently, was Email addresses: [email protected] constrained by chemical propulsion, large (James Bird), [email protected] (Linda Petzold), [email protected] (Philip Lubin), of California Santa Barbara, Santa Barbara, CA [email protected] (Julia Deacon) 93106-9530 1Department of Computer Science, University of 3Department of Computing and Mathematical California Santa Barbara, 2104 Harold Frank Hall, Sciences, California Institute of Technology, 1200 E. Santa Barbara, CA 93106-9530 California Blvd., MC 305-16, Pasadena, California 2Department of Physics, Broida Hall, University 91125-2100 Preprint submitted to Advances in Space Research June 9, 2020 spacecraft, and therefore, relatively slow 1.1. Previous Work speeds. Since the main objective has been The effectiveness of machine learning, exploration of our solar system, these meth- specifically deep learning via TensorFlow ods were sufficient. In contrast, the recent and cuDNN, has been indisputably demon- Starlight program (Kulkarni et al., 2017) strated in the last decade (Abadi et al. has introduced methods for deep space (2016), Chetlur et al. (2014), Canziani travel that utilize small discs, which travel et al. (2016)). The fight over the best at approximately one-fourth of the speed of model and the most accurate results, espe- light via directed energy. cially between the most popular models like ResNets, DenseNets (Huang et al. (2018)), Alongside the prospect of fast deep Inception(Szegedy et al. (2015)), Masks space travel comes many new challenges. (He et al. (2018)), and models that com- The normal model for space travel in- bine some of these together(Szegedy et al. cludes spacecraft capable of housing instru- (2016)), is one that has produced a plethora ments, propulsion and navigational equip- of potent options to choose from. Models ment, telescopes, energy banks, and much that are more accurate than human beings more. Since the Starlight program will be at doing extremely difficult tasks are still utilizing small wafersats that are approxi- being discovered (Rajpurkar et al. (2017)). mately the size of a coffee can lid, all of The areas of deep learning and astron- these features need to be reworked or dis- omy have come together in recent years carded. (Ruffio et al. (2018), Morad et al. (2018), Schaefer et al. (2018), Pearson et al. Besides physical constraints, this new (2017)), mostly in the form of light curves model of space travel introduces feasibility (Shallue & Vanderburg (2017), Zucker et constraints as well. The star of interest is al. (2018), Carrasco-Davis et al. (2018)). beyond four light-years away, meaning that The results and general concepts promote transmission of data and response command a healthy symbiosis between deep learn- transmissions are a combined eight years or ing and the problems that arise in astron- more. Thus, the wafersats need to be able omy. Yet, the processes are carried out from to make decisions without human interven- Earth, not space, and do not address real tion, and for that, artificial intelligence (AI) images, two big issues that create a gap in is paramount. comparability. Outside of astronomy, simulators have The major hurdles that we will discuss been used to train data in specific instances are those concerning computer vision via where the benefits outweigh the drawbacks. planetary detection, data and storage block- Smyth et al. (2018) outlines some major ages via novelty detection and ranking, and drawbacks, namely that the process takes a energy management via combining simula- lot of time and knowledge, as well as a note tor features with subtraction-algorithm-fed that simulator-based training may not gen- computer vision. For all of these issues, tak- eralize well to real images. Alongside those ing advantage of a universe simulator will concerns, McDuff et al. (2018) begins with a introduce solutions that were otherwise in- common issue in machine learning models, effective or impossible to find. which is that training data sets are often 2 biased. This bias arises when there are mi- decomposed into red, green, and blue chan- norities in the training set, which in turn nels and vectors were constructed of length produces poor results when the model is 320 × 180 × 3 = 172; 800. asked to evaluate a similar entity in the pop- Our challenge is to label each image as ulation. These issues are handled through- novel or regular. That is, we wish to gener- out this paper and are shown to not be an ate a classification n s.t. for each input x, issue with the specific problem at hand. n(x) 2 f0; 1g, where a 0 indicates regular- Simulators also introduce a lot of ben- ity and a 1 indicates novelty. The algorithm efits. One large one, also seen in Con- should hopefully yield a large number of 1's nor & Leeuwen (2018), is that "the small when a planet or the Sun is clearly in view catalogue of real events is probably not and should yield very few 1's when the im- yet a representative sample of the under- age is mostly distant stars. lying .. population, nor is it big enough to build a meaningful training set for machine 1.2.2. GWR Algorithm learning, deep or otherwise." An important Define A as the set of nodes in our net- theme throughout this paper, and an ex- work and C as the set of edges between tremely useful aspect of simulators, is that these nodes. We denote our inputs as ξ and they provide an untold amount of training the weight vector for any node n as wn. data, assuming that one can create realistic Each node n has a habituation hn which simulations. represents how familiar that node is to the system. 1.2. Unsupervised Learning for Planetary Detection 1. Initialize two nodes that represent two random samples from the dataset. We The intuition behind object detection, in set their habituations each to 1. The particular planetary detection, might point set of edges between nodes begins toward an unsupervised learning technique. empty. After all, one might reasonably think that detecting a nearby planet after months of 2. Iterate through the dataset. For each traveling through deep space would be easy. data sample ξ: We test this idea using an unsupervised (a) For each node i in the net- technique called a Grow When Required work, calculate its distance from (GWR) Network (Marsland et al., 2002). ξ, which is jjξ − wijj. (b) Find the node s with the smallest 1.2.1. GWR Setup distance and the node t with the Using the worldwide telescope, we gen- second-smallest distance. erated a 9,000 frame series of solar system (c) Add an edge between s and t if it images. It begins with Neptune, then it ex- does not already exist. plores Mercury, the Sun, and finally Mars. (d) Calculate the activity P 2 The majority of images contain only back- a = exp(− j(ξ[j] − ws[j]) =C), ground stars. where C = 29,203,200 was The images were down-scaled to a chosen to prevent an integer 320x180 resolution in order to improve com- overflow. There are 172,800 fields putational speed. For learning, they were in each data vector, and since 3 the average of the quantities in each vector is close to 13, and 132 = 169, we divide by 172; 800 × 169 = 29; 203; 200. (e) If a < aT and s's habituation hs < hT (where aT is some inser- tion threshold and hT is some ha- bituation threshold), then add a ws+ξ new node r. Set wr = 2 . Insert edges between s and r and r and t and remove the edge between s and t. (f) Otherwise, update the weight and Figure 1: A scatter plot of the detected novelty of habituation of s as follows: δws = the data.