MODELING AND MATCHING OF LANDMARKS FOR AUTOMATION OF MARS ROVER LOCALIZATION DISSERTATION Presented in Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy in the Graduate School of The Ohio State University By JUE WANG, M.S., B.S. ***** The Ohio State University 2008 Dissertation Committee: Approved by Dr. Rongxing Li, Advisor Dr. Tony Schenk _________________________________ Dr. Alper Yilmaz Advisor Graduate Program in Geodetic Science and Surveying © Copyright by Jue Wang 2008 ABSTRACT The Mars Exploration Rover (MER) mission, begun in January 2004, has been extremely successful. However, decision-making for many operation tasks of the current MER mission and the 1997 Mars Pathfinder mission is performed on Earth through a predominantly manual, time-consuming process. Unmanned planetary rover navigation is ideally expected to reduce rover idle time, diminish the need for entering safe-mode, and dynamically handle opportunistic science events without required communication to Earth. Successful automation of rover navigation and localization during the extraterrestrial exploration requires that accurate position and attitude information can be received by a rover and that the rover has the support of simultaneous localization and mapping. An integrated approach with Bundle Adjustment (BA) and Visual Odometry (VO) can efficiently refine the rover position. However, during the MER mission, BA is done manually because of the difficulty in the automation of the cross-site tie points selection. This dissertation proposes an automatic approach to select cross-site tie points from multiple rover sites based on the methods of landmark extraction, landmark modeling, and landmark matching. The first step in this approach is that important landmarks such as craters and rocks are defined. Methods of automatic feature extraction and landmark modeling are ii then introduced. Complex models with orientation angles and simple models without those angles are compared. The results have shown that simple models can provide reasonably good results. Next, the sensitivity of different modeling parameters is analyzed. Based on this analysis, cross-site rocks are matched through two complementary stages: rock distribution pattern matching and rock model matching. In addition, a preliminary experiment on orbital and ground landmark matching is also briefly introduced. Finally, the reliability of the cross-site tie points selection is validated by fault detection, which considers the mapping capability of MER cameras and the reason for mismatches. Fault detection strategies are applied in each step of the cross-site tie points selection to automatically verify the accuracy. The mismatches are excluded and localization errors are minimized. The method proposed in this dissertation is demonstrated with the datasets from the 2004 MER mission (traverse of 318 m) as well as the simulated test data at Silver Lake (traverse of 5.5 km), California. The accuracy analysis demonstrates that the algorithm is efficient at automatically selecting a sufficient number of well-distributed high-quality tie points to link the ground images into an image network for BA. The method worked successfully along with a continuous 1.1 km stretch. With the BA performed, highly accurate maps can be created to help the rover to navigate precisely and automatically. The method also enables autonomous long-range Mars rover localization. iii To my families, who have supported me all the time iv ACKNOWLEDGMENTS First of all, I would like to express my sincere gratitude to my advisor, Dr. Rongxing (Ron) Li, for his outstanding guidance, constant encouragement and patience. The many opportunities that he gave me to participate in various projects have stimulated my interests in different research fields and enabled me to make a contribution to the prestigious Mars project. I would also like to extend my sincere appreciation to Dr. Tony Schenk and Dr. Alper Yilmaz for serving on my dissertation committee as reviewers and examiners. Moreover, I would like to thank them for their valuable comments and suggestions. I wish to thank the current and previous Mars team members at the OSU Mapping and GIS Laboratory: Dr. Kaichang Di, Dr. Bo Wu, Dr. Fengliang Xu, Dr. Xutong Niu, Shaojun He, Ju Won Hwangbo, Lin Yan, Yunhang Chen, Wei Chen, Jeremiah Glascock, Sanchit Agarwal, Evgenia Brodyagina, Charles Serafy, and Eric Oberg, who have encouraged me and helped me with their insights. Many of the maps and results in this dissertation would not be possibly produced without teamwork. My appreciation should also be extended to other colleagues who have worked in this lab: Dr. Ruijin Ma, Dr. Tarig A. Ali, Leslie Smith, Sagar Deshpande, I-Chieh Lee, and Alok Srivastava. We always have interesting and good-spirited discussions in the lab. v In addition, I am grateful to the faculty, staff, and students in the Department of Civil and Environmental Engineering and Geodetic Science for creating and promoting a unique atmosphere of academic excellence. I benefited enormously from the advanced courses offered by the faculty members in this department. I wish to thank all my friends. No matter where I am, they always encourage me when I am in adversity. I would also like to thank Ms. Eve A. Baker, Karla Edwards, Lisya Seloni, Dr. Di and Dr. Wu for the proofreading of my dissertation. Finally, I would like to express my deepest thanks to my parents, who contributed the way I am. They have supported me unconditionally all the time. Special thanks also go to my husband, Feng, and my lovely son, Michael. Without their support and tremendous love, this study could never have been completed. I conducted my dissertation at the Mapping and GIS Laboratory of The Ohio State University. The research was supported by NASA/JPL. vi VITA November, 1977…… Born in Zhejiang Province, P.R. China July, 1999………….. B.S., Surveying Engineering Tongji University, Shanghai, P.R. China March, 2002 ………. M.S., GIS and Mapping Tongji University, Shanghai, P.R. China December, 2006 …... M.S., Geodetic and Geo-information Science, The Ohio State University 1999 – 2001……....... Student Tutor, Tongji University, Shanghai, P.R. China 1999 – 2002……....... Graduate Research Assistant Tongji University, Shanghai, P.R. China September 2004 – Graduate Teaching Assistant, The Ohio State University December 2005…..... 2002 – present …….. Graduate Research Assistant, The Ohio State University PUBLICATIONS Research Publications 1. Di, K., F. Xu, J. Wang, S. Agarwal, E. Brodyagina, R. Li, L. Matthies. 2007. Photogrammetric Processing of Rover Imagery of the 2003 Mars Exploration Rover Mission. ISPRS Journal of Photogrammetry and Remote Sensing, doi:10.1016/j.isprsjprs.2007.07.007, available online 12 September 2007. 2. Di, K., J. Wang, R. Ma, and R. Li. 2003. Automatic Shoreline Extraction from IKONOS Satellite Imagery. EOM (Earth Observation Magazine), Vol.12, No.7, pp. 14-18. vii 3. Li, R., K. Di, J. Wang, X. Niu, S. Agarwal, E. Brodyagina, E. Oberg, and J.W. Hwangbo. 2007. A WebGIS for Spatial Data Processing, Analysis, and Distribution for the MER 2003 Mission. Journal of Photogrammetric Engineering and Remote Sensing, Vol.73, No.6, pp.671-680. 4. Li, R., K. Di, A. Howard, L. Matthies, J. Wang, and S. Agarwal. 2007. Rock Modeling and Matching for Autonomous Long-Range Mars Rover Localization. Journal of Field Robotics, Vol.24, No.3, pp.187-203. Published online in Wiley InterScience (www.interscience.wiley.com). DOI: 10.1002/rob.20182. 5. Li, R., S. W. Squyres, R. E. Arvidson, B. A. Archinal, J. Bell, Y. Cheng, L. Crumpler, D. J. Des Marais, K. Di, T. A. Ely, M. Golombek, E. Graat, J. Grant, J. Guinn, A. Johnson, R. Greeley, R. L. Kirk, M. Maimone, L. H. Matthies, M. Malin, T. Parker, M. Sims, L. A. Soderblom, S. Thompson, J. Wang, P. Whelley, and F. Xu. 2005. Initial Results of Rover Localization and Topographic Mapping for the 2003 Mars Exploration Rover Mission. Journal of Photogrammetric Engineering and Remote Sensing, Special issue on mapping Mars, Vol.71, No.10, pp.1129-1142. 6. Wang, J., K. Di, and R. Li. 2005. Evaluation and Improvement of Geopositioning Accuracy of IKONOS Stereo Imagery. ASCE Journal of Surveying Engineering, Vol.131, No.2, pp.35-42. 7. Wang, J., and Chen, Y. 2001. The production of Digital Orthophoto Map and its further application. Remote Sensing Information (in Chinese), No. 2, Sum, No.62. FIELDS OF STUDY Major Field: Geodetic Science Studies in: GIS Mapping & Cartography Photogrammetry Pattern Recognition and Computer Vision viii TABLE OF CONTENTS Page Abstract …………………………………………………………………………. ii Dedication ………………………………………………………………………. iv Acknowledgments ………………………………………………………………. v Vita …………………………………………………………………...………..... vii List of Tables…………………………………………………………………….. xii List of Figures …………………………………………………………………... xiv List of Abbreviations .……………………………………..………….…………. xvii Chapters: 1 Introduction ………………………………………………………..……… 1 1.1 Background information……………………………………..…..……. 1 1.2 Literature review……………………………………..…..…..…..……. 9 1.3 Issues and significance of this research…………………..…..…..…… 16 1.4 Overview of the dissertation…………………..…..…..………….…… 20 2 Landmark Extraction and Modeling ……………………….……….…….. 22 2.1 Landmark definition…………………………………..….……….…… 22 2.2 Landmark extraction……………..…..…..………..…..…..…………... 31 2.2.1 Crater detection…………………..…..…..…………………….. 32 2.2.2 Rock extraction……..…..…..…….………..…..…..…………... 37 2.2.3 Limitations in landmark extraction…………………..…..…….. 41 2.3 Landmark modeling