Lunar Orbiter State Estimation Using Neural Network-based Crater Detection by Lena Downes B.S., University of New Hampshire (2018) Submitted to the Department of Aeronautics and Astronautics in partial fulfillment of the requirements for the degree of Master of Science in Aeronautics and Astronautics at the MASSACHUSETTS INSTITUTE OF TECHNOLOGY May 2020 ○c Massachusetts Institute of Technology 2020. All rights reserved. Author................................................................ Department of Aeronautics and Astronautics May 19, 2020 Certified by. Jonathan P. How Richard C. Maclaurin Professor of Aeronautics and Astronautics, MIT Thesis Supervisor Certified by. Theodore J. Steiner Senior Member of the Technical Staff, Draper Thesis Supervisor Accepted by . Sertac Karaman Associate Professor of Aeronautics and Astronautics, MIT Chair, Graduate Program Committee 2 Lunar Orbiter State Estimation Using Neural Network-based Crater Detection by Lena Downes Submitted to the Department of Aeronautics and Astronautics on May 19, 2020, in partial fulfillment of the requirements for the degree of Master of Science in Aeronautics and Astronautics Abstract Terrain relative navigation can improve the precision of a spacecraft’s state estimate by providing supplementary measurements to correct for drift in the inertial naviga- tion system. This thesis presents a crater detector, LunaNet, that uses a convolutional neural network and image processing methods to detect craters from imagery taken by a spacecraft’s on-board camera. These detections are matched with known lu- nar craters, and these matches can be used as features that are input to a extended Kalman filter. Our results show that, on average, LunaNet detects approximately twice the number of craters in an intensity image as two other successful intensity image-based crater detectors, and detects more accurate crater centers and diameters than the other two detectors as well. One of the challenges of using cameras for this task is that they can generate imagery with differences in image qualities and noise levels. These differences can occur for reasons such as changes in irradiance ofthe lunar surface, heating of camera electronic elements, or the inherent fluctuation of discrete photons. These image noise effects are difficult to compensate for, makingit important for a crater detector to be robust to noise. When trained on diverse data, convolutional neural networks are able to generalize over varied imagery. Similarly, LunaNet is shown to be robust to four types of image manipulation that result in changes to image qualities and noise levels of the input imagery. LunaNet also pro- duces more repeatable crater detections from frame to frame throughout a trajectory, and that enables more reliable state estimation over a trajectory. A LunaNet-based EKF experiences fewer spikes in estimation error and has lower average estimation error than EKFs using other successful crater detectors. Thesis Supervisor: Jonathan P. How Title: Richard C. Maclaurin Professor of Aeronautics and Astronautics, MIT Thesis Supervisor: Theodore J. Steiner Title: Senior Member of the Technical Staff, Draper 3 4 Acknowledgments I would like to thank my advisors, Professor Jonathan How of MIT and Ted Steiner of Draper. I have learned immensely from all of the careful guidance and instruction that I have received from you both. Ted, your generosity with your time both to help me work through technical problems and to be a welcoming ear when I needed more general support has been truly appreciated. Your advice, from which classes to take to what direction to develop my career towards, has given me structure and clarity and has helped me to grow into the researcher that I am becoming. Jon, your sharp eye and deep knowledge of the field have molded my technical capabilities and honed my ability to focus on the important questions. You have pushed me harder than I have ever been pushed, and as a result I have achieved more than I ever thought that I could within these two years. I am extremely grateful to Draper in their support of my work. Four years ago I heard about Draper and the Draper Fellowship program, and I set my sights on it. Draper gave me a chance by offering me a software engineering internship when I was an undergraduate mechanical engineering student who did not want to be a mechanical engineer. In that internship I found my passion in visual navigation. I do not believe that I would be where I am today without that internship and the subsequent guidance and support of Draper, and for that I cannot thank Draper enough. Thank you to my family and friends, many of whom could not verbalize what I research beyond “landing on the Moon,” but who have been so incredibly supportive and have endured far too many impromptu technical talks on my part. To my parents and siblings, who have never questioned my decision or my ability to pursue an advanced degree, I am so grateful. Thinking deeply, questioning things, and reading have always been encouraged in our home, and a childhood surrounded by books has prepared me for a life of learning. To my friends, for commiserating, sharing in the struggles, and lifting each other up, thank you. You have kept me mentally afloat and given my life balance. To Carl, who has listened to one million complaints and who 5 always strikes the perfect balance of listening, offering advice, and consoling, thank you. You have made these challenging years so much sweeter. I would like to acknowledge the support of Amazon Web Services, which enabled much of this research. Thank you for your generous donation of computational re- sources. 6 Contents 1 Introduction 13 1.1 Precise State Estimation . 13 1.2 Terrain Relative Navigation . 15 1.3 Crater Detection . 16 1.4 LunaNet . 17 2 Background and Related Work 21 2.1 Terrain Relative Navigation . 21 2.2 Crater Detection and Matching . 26 2.2.1 Crater Detection . 26 2.2.2 Machine Learning . 28 2.2.3 Deep Learning Crater Detection . 30 2.2.4 Crater Identification . 33 2.3 Summary . 34 3 Methods 37 3.1 Crater Detection with LunaNet . 38 3.1.1 LunaNet’s Neural Network Subsystem . 38 3.1.2 LunaNet’s Image Processing Subsystem . 44 3.2 Crater Identification . 47 3.3 Extended Kalman Filter . 51 3.3.1 State Space . 51 3.3.2 State Propagation . 52 7 3.3.3 Measurement Update . 54 3.3.4 Kalman Update . 55 3.4 Summary . 55 4 Results 57 4.1 Crater Detection . 58 4.2 Extended Kalman Filter . 70 4.2.1 LunaNet Precision and Accuracy Analysis . 70 4.2.2 LunaNet Detection over a Trajectory . 73 4.2.3 EKF Monte Carlo Simulations . 78 4.3 Summary . 86 5 Conclusion 87 8 List of Figures 3-1 LunaNet system overview. 38 3-2 DeepMoon training image sets. 40 3-3 Crater database overlaid on Moon. 41 3-4 LunaNet training image sets. 43 3-5 LunaNet prediction image processing. 46 3-6 Convex hull example. 48 3-7 RANSAC crater matching example. 49 3-8 Crater matching process. 50 4-1 Example comparison of crater detectors. 60 4-2 Comparison of LRO image with and without different kinds of noise. 62 4-3 Comparison of number of matched craters from different crater detectors. 65 4-4 Example comparison of crater detectors on bright images. 66 4-5 Comparison of accuracy of different crater detectors. 68 4-6 Comparison of detection speed of different crater detectors. 68 4-7 Normalized comparison of detection speed of different crater detectors. 69 4-8 LunaNet detection accuracy and precision. 71 4-9 Close-up of Fig. 4-8 . 71 4-10 Histogram of LunaNet detection accuracy and precision. 72 4-11 Close-up of Fig. 4-10 . 72 4-12 Example test trajectory. 74 4-13 Example test trajectory, overhead view. 74 4-14 Example image persistence of a crater detection. 76 9 4-15 Average image persistence over a trajectory. 77 4-16 Average number of craters detected over a trajectory. 78 4-17 EKF position error for different crater detectors. 79 4-18 EKF velocity error for different crater detectors. 80 4-19 Effect of crater matching on estimation error. 82 4-20 EKF state error for different crater detectors with noise. 83 4-21 Position plot of Fig. 4-20 separated by noise. 84 4-22 Close-up of position plot of Fig. 4-21. 85 10 List of Tables 3.1 Parameters Used to Train LunaNet with Intensity Images . 44 4.1 Crater Detector EKF Monte-Carlo Performance . 81 11 12 Chapter 1 Introduction 1.1 Precise State Estimation In the days of Apollo, the mission was challenging but straightforward: land a man on the Moon by the end of the 1960s. This meant that it was possible to fulfill the mission goals while landing in the safest, least hazardous location possible on the lunar surface. The future of lunar exploration and development expands upon these goals. The return to the Moon focuses not just on landing on its surface, but on finding ways to build infrastructure and extract resources from the Moon after wedo so. These new goals require higher precision navigation in order to enable safely and precisely landing on the Moon. Precision landing enables the mining of a resource, like water, on the Moon. The presence of frozen water was detected in craters around the lunar northern and south- ern poles by NASA’s Moon Mineralogy Mapper instrument, which flew aboard India’s Chandrayaan-1 spacecraft in 2018 [30]. Frozen water is a useful resource because it is possible to combine nano-scale aluminum with water and use it as rocket fuel [25]. This means that the Moon could serve as a refueling location for spacecraft to stop at on their way to deeper space destinations.