Integration of Orbital and Ground Data for Martian Crater Mapping
A Methodological Study at Santa Maria Crater
Thesis
Presented in Partial Fulfillment of the Requirements for the Degree Master of Science
in the Graduate School of The Ohio State University
By
Rui Wu, M.S.
Graduate Program in Civil Engineering
The Ohio State University
2012
Thesis Committee:
Rongxing Li, Advisor
Carolyn Merry
Alper Yilmaz
Copyright by
Rui Wu
2012 Abstract
High-quality mapping products for Martian craters are helpful data sources for scientists to explore the red planet in many fields. This thesis focuses on integrating orbital imagery and ground imagery to generate high-quality Martian crater mapping products. The usability of the proposed method is validated mainly at one crater –
Santa Maria Crater. The orbital imagery comes from the High Resolution Instrument
Scientific Experiment (HiRISE) camera onboard the Mars Reconnaissance Orbiter
(MRO) satellite. The ground imagery comes from the Navigation cameras (Navcams) and Panoramic cameras (Pancams) images taken by the Opportunity rover in Mars
Exploration Rover (MER) mission.
In this thesis, important processes during the mapping will be introduced, discussed, and analyzed, including interest point extraction, image network construction, bundle adjustment (BA), dense matching, and product generation. Some new methods during these processes are proposed to improve the quality of the final products. The wide-baseline method is used in an unprecedented way in the tie point selection to guarantee the accuracy of the distant tie points. An integrated bundle adjustment replaces the former incremental bundle adjustment used in the MER mission for large crater mapping. Matching in the featureless areas is also discussed in this paper. The theoretical analysis and the improved results of these proposed ii
methods are highlighted with details.
The mapping products at Santa Maria Crater are listed to illustrate the feasibility of the proposed mapping methods. A reasonable conclusion is that the methods mentioned in this thesis work smoothly at Santa Maria Crater.
iii
Dedication
To my parents –for giving me your eyes
iv
Acknowledgements
Foremost, I want to thank my advisor, Dr. Rongxing Li, for leading me and guiding me in the Master study and research. His thorough knowledge in the research and endless enthusiasm for the job support and encourage me in this two-year adventure. Without his guidance, I could never take the progress in the area of photogrammetry and mapping, let alone writing this thesis to get the Master degree.
Besides my advisor, I would give my sincere thank to the rest of my thesis committees: Dr. Carolyn Merry and Dr. Yilmaz, for their continuous encouragement and advising.
My gratitude also goes to Dr. Kaichang Di, Dr. Shaoming Zhang, and Dr. Xuelian
Meng in the Mapping and GIS Lab, who helped me not only in the research with their expertise, but also in my life with their rich experience.
I also want to thank my coworkers and friends in the Mapping and GIS Lab:
Liwen Lin, Wei Wang, Onur Karahayit, Ding Li, Shaojun He, I-Chee Lee, Jiangye
Yuan, Justin Crawford, Weishu Gong, and Leslie Smith, for the inspiring tutorials, for the sleepless nights we worked together for the project, and for the laughter scattered in every single day of the last two years.
And I want to express my deepest thank to my friend Min Wang. Her comfort is the life saver for me when I feel hopelessness and desperation. I could not be the current me without her encouragement. v
Last but not least, I must thank my greatest parents. Their selfless support has been, and will always be my source of spiritual strength.
The research is supported by the Mars Data Analysis Program (MDAP) from
NASA.
vi
Vita
2004...... Luhe First High School
2008...... B.S. GIS, Peking University
2010...... M.E. Photogrammetry and Remote Sensing,
Peking University
2010 to present ...... Graduate Research Associate, Department of
Civil Engineering, The Ohio State University
Publications Li, R., R. Wu, and X. Meng 2012. Wide-Baseline Mapping of Martian Craters: A
Comparison Study at Santa Maria Crater. ASPRS 2012 Annual Conference, 19-23
March, Sacramento, CA. Abstract no. 345409 (1 p.) and presentation.
Fields of Study
Major Field: Civil Engineering
vii
Table of Contents
Abstract ...... ii
Dedication ...... iv
Acknowledgements ...... v
Vita ...... vii
List of Tables ...... ix
List of Figures ...... x
Chapter 1: Introduction ...... 1
Chapter 2: Background of Orbital and Ground Data ...... 8
Chapter 3: Construction of Image Network ...... 22
Chapter 4: Integrated Bundle Adjustment...... 36
Chapter 5: Mapping Product Generation ...... 56
Chapter 6: Conclusions ...... 70
References ...... 72
viii
List of Tables
Table 2-1. Important parameters of the camera system...... 11
Table 2-2. An example of RMC sequence...... 17
Table 4-1. The statistics of the inconsistencies between features ...... 51
Table 4-2. The statistics of the consistency of features and rover positions ...... 53
Table 5-1. The comparison on the dimension of Santa Maria Crater...... 68
ix
List of Figures
Figure 1-1. The workflow of Martian crater mapping...... 6
Figure 2-1. HiRISE focal plane assembly layout...... 9
Figure 2-2. Mars body-fixed reference system...... 13
Figure 2-3. Landing site cartographic coordinate system...... 14
Figure 2-4. Site frame...... 15
Figure 2-5. Multiple instances of site frame...... 16
Figure 2-6. Calculation of current position...... 20
Figure 3-1. Three types of tie points...... 23
Figure 3-2. Parallaxes of all candidate matching points and the parallax curve...... 26
Figure 3-3. The same feature observed from two different rover positions...... 27
Figure 3-4. Finding the feature correspondence by overlaying orthoimages...... 28
Figure 3-5. The effect of baseline length on the accuracy ...... 30
Figure 3-6. The distribution of rover positions near ...... 31
Figure 3-7. The result of rigid transformation when hard-baseline ...... 33
Figure 3-8. The construction of wide-baseline tie point selection...... 34
Figure 3-9. The result of rigid transformation when wide-baseline ...... 34
Figure 4-1. Geometry of the collinearity...... 37
Figure 4-2. Typical landforms in Gusev Crater and Meridiani Planum...... 43
Figure 4-3. The histograms of traverse in Gusev Crater and Meridiani Planum...... 44 x
Figure 4-4. Initialization of a rover position through feature comparison ...... 46
Figure 4-5. Shadows and shadings in HiRISE images of Martian craters...... 48
Figure 4-6. The comparison between the DTMs generated from HiRISE images. ...49
Figure 4-7. Inconsistencies between features among multiple rover positions ...... 51
Figure 4-8. The consistency of rover positions and features between ...... 53
Figure 5-1. Rules for choosing dense matching method...... 58
Figure 5-2. Geographe Crater with its blocked area...... 59
Figure 5-3. The dense matching points in Santa Maria Crater...... 61
Figure 5-4. Feature Comparison between the DTM of Santa Maria Crater ...... 62
Figure 5-5. The detailed DTM at the sand dune area of Santa Maria Crater...... 63
Figure 5-6. The blocked areas in crater mapping...... 64
Figure 5-7. Using the nearest pixels causes wrong filling in blocked areas...... 64
Figure 5-8. The brightness adjustment in sand dune area of Santa Maria Crater...... 66
Figure 5-9. Orthoimage of Santa Maria Crater before and after ...... 66
Figure 5-10. Comparison between orthoimages generated via different methods. ...67
Figure 5-11. The product set at Santa Maria Crater...... 69
ii
Chapter 1: Introduction
Mars, the nearest planet to the Earth, has been observed by human beings for thousands of years. It is named after the Roman god of war, probably because its passionate color seen from the Earth reminds people of wars. Nowadays, it attracts more and more attention for the reason that it is still the only planet having the possibility of life existence except for the Earth. Astronomers, geologists, meteorologists, and biologists spend their lives on searching for evidence of life, or the possibility of habitat on Mars. Telescopes, satellites, and rovers point to the Mars trying to find any clue. Although no certain evidence has been found until now, people don’t plan to stop their efforts.
Mars is the fourth planet in the solar system. It has the most similar environment with the Earth. For example, its rotation period, also called solar day or sol, is 24 hours, 39 minutes, and 35.244 seconds, only a little longer than an Earth day. And the surface of Mars is also covered with rocks, soil, sand, which form different landforms such as craters, canals and mountains. Meanwhile, there are also differences between the Mars and the Earth. For instance, the atmosphere is much thinner than the one of the Earth and is composed mostly of carbon dioxide instead of nitrogen and oxygen.
But the difference that matters most to the scientists is that no liquid water exists on
Mars, which is the most basic element to support life. This is also the reason that most life hunting missions focus on looking for water instead of looking directly for life. As long as the evidence of water exists now or in the past, it gives great
1
opportunity to prove that there can be life on Mars.
A regular way to find the evidence of water is to recognize the chemical substances in rocks, sand, and soil, as well as to analyze the surrounding geologic structure. Based on the experience on the Earth, certain chemical substances in certain geologic structures are very strong indicators for fluvial processes. If similar environment is found on Mars, it can be a solid evidence of water existence. Although people have explored Mars for so many years, high-quality data for the aforementioned method is still absent. No previous images or data can provide such a resolution that the small rocks can be observed in detail, until the Mars Rover
Exploration (MER) mission and the High Resolution Instrument Experiment (HiRISE) began.
Twin rovers, Spirit and Opportunity, were launched in the MER mission to acquire close observation of Mars. They landed separately on Gusev Crater and
Meridiani Planum in 2004 and surprisingly survived after 90 sols of exploration as planned, continuing their journey for over seven years so far. They have taken various pictures of Mars with their camera systems. This system contains Navigation Cameras
(Navcam), Panoramic Cameras (Pancam), and Hazard Avoidance Cameras (Hazcam).
Navcam and Pancam can both provide high-resolution panorama images for Mars surface mapping, which is essential for geologists to analyze the geologic structures.
This thesis will focus on utilizing Navcam and Pancam images to generate high-quality mapping products. On one hand, the ground images taken by rovers are critical and quite useful. On the other hand, they cannot be used directly as they are downloaded from the rovers without any pre-processing. This is not only about the image quality itself, but also about the positioning information that records where those images are taken, and in what attitudes they are taken. The sandy surface of
1
Mars often causes slippage of the rover wheels, and then affects the accuracy of the positioning information recorded by the rovers’ Inertial Measurement Unit (IMU).
Although the star tracking technique is also used for rover positioning, the accuracy may be affected by the precision of timing. Therefore, the attitudes of these images, or the rover localization, must be adjusted at first to recover the correct positioning information when the images are taken. The most popular and effective way to fulfill this image processing task is to construct the image network using tie points among multiple rover positions, and then to bundle adjust these positions to get consistency.
This topic will be discussed in detail in Chapter 3 and Chapter 4.
HiRISE is the camera onboard the Mars Reconnaissance Orbiter (MRO) satellite, which was launched in 2005. It has 14 Charged-Coupled Devices (CCDs) working together, providing high-resolution pictures of Mars surface in different spectrum channels. By 2010, HiRISE has taken about 13000 images covering about 1% of Mars
(NASA, 2010) and the number keeps growing fast. The unprecedented spatial resolution of the 0.25 m - 0.3 m imagery helps in discovering small interesting features and landform patterns, such as Cape St. Vincent at Victoria Crater. Before
HiRISE, the highest resolution in satellite data is about 1.4 m in horizontal direction
(Malin, et al., 1992) by the Mars Orbiter Camera (MOC) narrow-angle and 1.5 m in vertical direction by Mars Orbiter Laser Altimeter (MOLA) data (Zuber, et al., 1992).
Although MOLA data is still the best data that covers the whole Mars, and can work as a good reference in large scale, its horizontal resolution of about 1 km (Zuber, et al.,
1992) is too coarse for distinguishing small craters and canals, let alone analyzing the attributes of rocks. Therefore, combining HiRISE images and MOLA data is the most popular way to raise the resolution in both directions. Hwangbo (2010) discussed this issue in detail. With the techniques in that dissertation, HiRISE images controlled by
2
MOLA data can provide very good Digital Terrain Model (DTM) and orthoimage products for large area mapping at high quality.
It is easy to understand that ground images taken by rovers are proper for small area mapping, such as craters, or canals, while orbital images taken by HiRISE are proper for large areas, such as the planum, giant craters and mountains. Combining the advantages of these two imaging systems is promising to produce the best mapping products for both levels. However, these two systems have very different reference frames and imaging mechanisms, and the images are also taken in very different angles of view, oblique in ground images and near-ortho in orbital images.
These problems block the way to combine them easily. To conquer the difficulties, the first thing to do is to understand both imaging systems deeply and find the connection between them. HiRISE images are taken from a satellite which is about 300 km above
Mars surface (Taylor, et al., 2006) without going through thick atmosphere, and then controlled by MOLA data. It is reasonable to predict that the distortion in these images is much smaller than the one in ground images, considering the blurring and the inaccuracy of distant features seen commonly in ground images. In this prerequisite, HiRISE images controlled by MOLA data can be defined as a good reference for ground images, and can provide relatively accurate and precise positioning information, not only for rover positions, but also for features. This connection is used as a guideline in this thesis and the practical analysis is elaborated in Chapter 4.
Since the accuracy of features, or tie points, are so important for recovering the relationship among multiple rover positions, the whole Chapter 3 is dedicated to introducing the method of selecting high-quality tie points, especially the usage of the wide-baseline method in the process. Wide-baseline method has been used in so many
3
fields, especially computer vision and photogrammetry. The Mapping and GIS Lab in
The Ohio State University has also used this method for several crater mapping tests.
Those tests only utilized wide-baseline in the dense matching phase of DTM generation, but never involved wide-baseline in the tie point selection phase. Chapter
3 will illustrate how the wide-baseline method can improve the quality of tie points for bundle adjustment and how this improvement can affect the results.
With the tie points connecting all the ground images taken from multiple positions, it is possible to adjust biased rover positions and get consistent results of the 3D coordinates of those tie points via bundle adjustment, thus to recover the true positioning data. Previously, the Mapping and GIS Lab used incremental bundle adjustment to guarantee the timeline required by the MER mission. It is in a step-wise manner that only the latest position is considered as adjustable, which makes that only two positions need to be considered every time. But it is predictable that error will be accumulated as the traverse gets longer and longer. The integrated bundle adjustment proposed in this thesis, on the contrary, performs bundle adjustment on multiple positions simultaneously, so that the accumulated error in the whole image network within these positions will be kept to a very low level. The theory of incremental and integrated bundle adjustment and the comparison between them will be set forth in
Chapter 4.
Based on the results of bundle adjustment, all the features in the images should be projected to a 3D model so that they can be observed in a simulated environment.
This process is done by dense matching and back-projection. Unlike the Earth full of features in the environment, such as trees, buildings and other artificial features, Mars is so desolated that the only features that can be used are rocks. In some sandy areas in the Meridiani Planum, even rocks are so rare that the feature matching is far from
4
the standard definition of dense. Different matching methods need to be used to automate this process as much as possible, while onerous manual work is also inevitable to supplement the result and ensure the matching quality. For Martian crater mapping, another common issue is the blocked area matching. Wide-baseline, again, is known and tested to be the best method to solve the problem. The content of dense matching and other relevant methods will be discussed in Chapter 5.
These above steps are all necessary for a good-quality DTM and other related mapping products. The final products would not reflect the true terrain or would affect the decision making by scientists if any above step were wrong. In order to evaluate these methods, mapping of Santa Maria Crater in the landing site of Opportunity, the
Meridiani Planum, is executed. Intermediate data are illustrated to be analyzed, compared with previous methods, and final products are compared to the HiRISE products to ensure the consistency. These tests show that the proposed methods in this thesis are quite feasible for mapping most small-to-medium scale Martian crater.
The workflow of Martian crater mapping can be seen in Figure 1-1.
5
Figure 1-1. The workflow of Martian crater mapping.
A discussion of the above workflow is provided. Chapter 2 provides an introduction of the background of two image sources that will be used: orbital images taken by HiRISE and controlled by MOLA data, and ground images taken by rovers.
A brief introduction will be given about the reference frames and coordinate systems, and important imaging parameters. Chapter 3 will provide theories, methods, and practical steps in the image network construction, especially the usage of wide-baseline method in tie point selection. Different tie point types will be introduced. The factors that affect the quality of the tie points will also be discussed.
The role of orbital data as a reference to the tie point selection will be a big part of this chapter. Chapter 4 will provide a description of the theory behind the bundle adjustment, and then the difference between the incremental bundle adjustment and
6
the integrated bundle adjustment. Their advantages and disadvantages will be put forward so that they can be better used in different applications in the future. Chapter
5 is about the generation of the DTM and other mapping products. Dense matching methods used in the tests will be introduced. The difficulty of complete autonomy in matching will be emphasized. The principle behind back-projection is also explained.
Last but not least, the mapping results at Santa Maria Crater and other craters will be given. Chapter 6 will provide a conclusion of the whole mapping process and discuss the relevant future work.
7
Chapter 2: Background of Orbital and Ground Data
This chapter will provide an introduction of the information of the data sources used in Martian crater mapping. The primary data source is the ground images taken by the two rovers, Spirit and Opportunity, along their traverses in Gusev Crater and
Meridiani Planum over seven years. The second data source comes from the orbital images taken by HiRISE onboard the MRO satellite controlled by MOLA data.
Necessary information about their imaging mechanisms and the reference frames will be introduced.
2.1 High Resolution Imaging Scientific Experiment HiRISE camera onboard the MRO satellite is designed to take very detailed orbital images of Mars. This camera has 14 CCDs fixed to a focal plane, in which ten are in red channel, two are in blue-green channel, and two are in the near infrared
(Figure 2-1). Rather than a frame imaging sensor, HiRISE takes images in a pushbroom manner. Each CCD consists of 2048 pixels in the across-track direction and 128 pixels in the along-track direction. In the across-track direction, the average width of the overlap between adjacent CCDs is about 48 pixels. After aligning those overlapping areas by adjustment, HiRISE can still have up to 20,264 pixels in the across-track direction, which is equivalent to 6 km at 300 km altitude (Delamere et al.,
2003; McEwen et al., 2007).
8
Figure 2-1.HiRISE focal plane assembly layout (McEwen et al., 2010).
Two issues block the way to utilize HiRISE images directly. The first one comes from the complexity of the sensor geometry among multiple CCDs. Although they are all fixed to a focal plane and should produce consistent images with each other, small systematic errors and certain random errors always bring in unexpected distortion, bias, and offset. These disagreements among CCDs need to be removed to achieve coherence in the exterior orientation parameters, and in the orbital images.
The second issue is the inaccurate geopositioning of HiRISE images caused by no ground truth. Unlike on the Earth, that various approaches can be used to measure the ground control points for reference, the only measurements close to the ground truth of Mars come from the MOLA data, which is still not dense enough. Without the ground truth, the registration of HiRISE images can be largely affected by various
9
measurement errors.
A systematic approach has been developed by Hwangbo (2010) on solving these issues by utilizing MOLA data and photogrammetry methods to register HiRISE images. Since this is not the major topic of this thesis, the HiRISE images processed by her method will be directly used here without detailed technical discussion.
2.2 Navigation Cameras and Panoramic Cameras As twin sisters, Spirit and Opportunity have the same camera systems including six engineering cameras and two science panoramic cameras. The primary goal for the engineering cameras is to take necessary pictures for keeping the rover on track and avoiding obstacles when doing blind drive, while the goal for the panoramic cameras is to support high resolution imaging for scientific exploration of Mars.
The engineering cameras consist of two navigation cameras, two front Hazard
Avoidance Cameras (Hazcam) and two rear Hazcams. These four Hazcams are mounted to the titanium alignment bracket under the solar panel with the baseline of
10 cm. They use visible light to capture images in black and white. The over 120 field of view (FOV) helps Hazcams to map out the shape of the terrain as far as 3 m in front of the rovers, and the images work with built-in software to keep rovers from crashing into unexpected obstacles (Maki, et al., 2003a).
All mounted on the Panorama Mast Assembly (PMA), Navcams and Pancams are good data sources for rover navigation, surface mapping and scientific research. The
PMA is capable of a rotation of 370 in the horizontal direction and a tilt of 194 in the vertical direction, which makes it possible for Pancams and Navcams to take panoramas of the surroundings. Because of the difference in the field of view, the
Navcams take a panorama using 10 stereo pairs, while the Pancams need 27 pairs.
The reliable distances of these two camera systems are also different as a result of 10
their configurations. Generally, Pancam is a better data source for mapping large craters since the reliable distance is about 60 m, while Navcam is more appropriate for mapping smaller craters within its 30 m reliable distance. More detailed parameters of the camera systems onboard the twin rovers are listed in Table 2-1.
Camera Type Pancam Navcam Hazcam
FOV (Degree) 16*16 45*45 124*124
Baseline (cm) 30 20 10
Focal Length (mm) 43 14.67 5.58
Reliable Distance (m) 60 30 3
Angular Resolution (mrad/pixel) 0.28 0.82 2.1
Table 2-1. Important parameters of the camera system (Maki, et al., 2003a).
The capability of the science panoramic cameras is reflected not only in the high resolution, but also in the multispectral imaging filters. 13 filters are installed to take images in different spectral bands, from near infrared to visible blue light. This is essential for scientists when the characteristics of the materials on the Martian surface can be learned based upon their spectral performances. However, for the purpose of high-resolution crater mapping in this topic, a single spectral band is enough for us during most of the time to distinguish different objects in the image for feature extraction. According to the empirical experience, the red band (labeled as L2 and R2) is used primarily in our experiments. When the red band is not available, the blue band (labeled as L7 and R1) is used as an alternative (Bell III et al., 2004).
11
2.3 Mars Exploration Rover Coordinate Systems It is not difficult to understand that there are a number of reference frames and coordinate systems for different applications in the MER mission. Understanding these coordinate systems is the very first beginning to deal with the integration of orbital data and the ground data. Considering the main topic of the thesis, this section will only introduce the coordinate systems that are relevant to the Martian crater mapping (Maki, 2003b).
2.3.1 Mars Body-Fixed Reference Frame The Mars body-fixed (MBF) reference frame is defined essentially by the
International Astronomical Union/International Association of Geodesy (IAU/IAG)
Working Group on Cartographic Coordinates and Rotational Elements of the Planets and Satellites in their most recent report (Seidelmann et al., 2002). The Mars body-fixed reference axes have their origin at the Mars center-of-mass and are aligned with the spin axis and prime meridian. This frame is described as the following and shown in Figure 2-2:
+XMBF=Vector lies in the Mars equatorial plane and intersects the prime meridian.
+YMBF=Vector lies in the Mars equatorial plane and completes a right handed
coordinate system.
+ZMBF =Mars spin axis, pointing toward Martian North Pole.
12
Figure 2-2. Mars body-fixed reference system.
2.3.2 Landing Site Cartographic Coordinate System More than one local reference system is defined for satellites and rovers for various purposes. Among these reference systems the landing site cartographic (LSC) coordinate system is particularly useful in navigation and imaging. The LSC coordinate system is an East-North-Zenith (X-Y-Z) frame, whose origin is coincident with the landing site, also called lander. Therefore, Spirit and Opportunity both have their own LSC origins since the landers are located differently. The LSC coordinate system is defined relative to the Mars body-fixed frame using radius r, aerocentric latitude , and aerocentric longitude , as shown in Figure 2-3:
13
Figure 2-3. Landing site cartographic coordinate system.
Locating two landers in the Mars body-fixed reference system is critical for planning science and engineering activities in the initial stages of rover exploration, because the landers are starting points and important references to all the calculations in the future mission. Multiple methods were used to determine the lander locations, such as fitting direct-to-Earth two-way X-band Doppler signals, using two passes of
UHF two-way Doppler between rovers and the satellite of Mars Odyssey, and triangulation based on the lander panoramas and the Descent Image Motion
Estimation System (DIMES) descent and MOC images (Li, et al., 2005). The defined lander locations are slightly different from each other according to the methods and the goals of different researches. In this thesis, the locations that were determined by triangulation to features will be used. Translated to the MBF system, the Spirit lander location was determined as 14.5692S, 175.4729E, later unofficially named
“Columbia Memorial Station”, and the Opportunity lander location was determined as
1.9462S, 354.4734E (Golombek and Parker, 2004; Parker et al., 2004), later 14
nicknamed “Eagle Crater”.
2.3.3 Site Frame For the convenience of continuous navigation and imaging tasks in the MER mission, multiple instances of site frames are defined. The site frame is described as a
North-East-Nadir (X-Y-Z) frame whose origin is initially coincident with the LSC coordinate system. This initial instance of the site frame is expressed as S0. After the lander petals were deployed and rover orientation was initially estimated, the site frame was fixed with respect to the Mars body-fixed frame (Figure 2-4).
Figure 2-4. Site frame.
If the rover stays in a very small area, an entire surface mapping could be conducted using only one instance of the site frame. However, the MER mission requires the rovers and their cameras to move regularly relative to the initial site
15
frame. Besides, the knowledge of the absolute position of the rovers can degrade over time, and thus misalign the acquired image data over time. If there are multiple instances of the site frame, it is helpful to prevent the accumulated rover positioning error from propagating into the image data through resetting the origin of the site frame. As the rovers traverse across the Martian surface, the site frame is reset to zero when the rovers stop at strategic locations, declared as a new site. Other rover locations are defined as positions within certain site frames. As Figure 2-5 indicates, there is always one site and multiple positions in one site frame.
Figure 2-5. Multiple instances of site frame.
The rover motion counter (RMC) is used to record these sites and positions so that instrument data for operations can be downlinked, identified, and processed easily.
16
It is a monotonically increasing counter which increments as the rover moves. The
RMC is composed of five indices (JPL, 2004):
1. Site – Declared by operations personnel, this is a major coordinate frame from
which all activities in a local region are referenced.
2. Drive – Incremented by the rover whenever it intentionally moves to a new
position.
3. IDD – Incremented by the rover whenever the instrument deployment device
(IDD) moves.
4. PMA – Incremented by the rover whenever the PMA moves.
5. HGA – Incremented by the rover whenever the high gain antenna (HGA)
moves.
There are two basic categories in the above indices. Site and Drive increments represent cases where the rover is expected to move. When they increase, all lower-priority RMC values are reset to 0. However, when IDD, PMA, and/or HGA increase, the rover is not supposed to move, therefore, the Site and Drive will not have increment (Table 2-2 for the example). This strategy of RMC provides a systematic method for building and maintaining traversal trees across the entire instrument data set.
Site Drive IDD PMA HGA Movement
3 0 0 0 0 New site
3 4 0 0 0 Drive to new position
Continued
Table 2-2. An example of RMC sequence.
17
Table 2-2 continued
3 4 0 3 0 Acquire panorama
3 4 3 3 0 Conduct IDD operations
3 4 3 3 4 Conduct HGA communication session
3 4 6 3 4 Retract IDD
3 23 0 0 0 Drive to new position
4 0 0 0 0 Declare a new site
It is important to register all sites and positions to the same reference frame, so all the data can be processed together. Therefore, the offset vectors among different coordinate systems are recorded in the RMC files. An RMC file is nothing more than a collection of coordinate system definitions containing a list of coordinate frames, the reference frame for each rover movement, and the offset and orientation between the frames. Among various types of RMC files, the Master Site Vector File (Master
SVF) and daily image files are inevitable for our purpose of crater mapping.
A Master SVF is a central file which contains all project-approved solutions that have been generated for their respective coordinate frames. A Master SVF naturally contains the telemetry solution for all relevant coordinate frames (all Sites, and all
Rover frames where the rover intentionally moved). Each new site is defined relative to the immediately previous site with an offset vector. Thus the file defines a continuous chain of sites from Site 0 up to the current site. An example of the Master
SVF is as follows:
add_date="2004-10-22T21:28:49Z" index1="37"> 18
In this example,
(-21.003497m, 7.577715m, -3.067073m) in East-North-Nadir directions from Site 36 to Site 37. The location of the current rover site in the LSC coordinate system can be calculated by adding all the offsets from Site 0 to the current site.
The Master SVF only records the offsets between sites. To get the coordinates of each rover position within one instance of site frame, the distance of each drive in the
LSC coordinate system is absolutely necessary. This drive is recorded in everyday image files as a part of the file header. This file header is also used as the Planetary
Data System (PDS) Label for the intention of query and archival. The following example shows the format of the drive in the PDS label:
GROUP = ROVER_COORDINATE_SYSTEM
COORDINATE_SYSTEM_INDEX = (36,20,15,129,58)
COORDINATE_SYSTEM_INDEX_NAME = (SITE,DRIVE,IDD,PMA,HGA)
COORDINATE_SYSTEM_NAME = ROVER_FRAME
ORIGIN_OFFSET_VECTOR = (-2.89207,4.47442,0.645726) 19
ORIGIN_ROTATION_QUATERNION =
(0.440945,-0.0552843,-0.152523,0.88275)
POSITIVE_AZIMUTH_DIRECTION = CLOCKWISE
POSITIVE_ELEVATION_DIRECTION = UP
QUATERNION_MEASUREMENT_METHOD = FINE
REFERENCE_COORD_SYSTEM_INDEX = 36
REFERENCE_COORD_SYSTEM_NAME = SITE_FRAME
END_GROUP = ROVER_COORDINATE_SYSTEM
GROUP defines which coordinate system is used as the reference frame for all the indices in the group. ROVER_COORDINATE_SYSTEM indicates that it is an instance of the site frame. Therefore, the ORIGIN_OFFSET_VECTOR records the offset from the current position to the previous position in the same instance of site frame. To get the coordinates of current position in the LSC coordinate system, all these offsets in the site frame needs to be added up to the coordinates of the site. The detailed process is shown in Figure 2-6.
Figure 2-6. Calculation of current position. Ai is the offset between Site i-1 and i. Bni is the drive between Position i-1 and i in the current instance of site frame n.
The calculation result of the telemetry data provides the fundamental relationship
20
among multiple positions. Based on this telemetry data all the subsequent adjustment and mapping processes are performed in a uniform reference frame.
This chapter introduced several reference frames that are closely relevant to the
Martian crater mapping. Site frame is the most important one because that all the offsets between daily sites and positions are defined directly under it. Landing Site
Cartographic coordinate system is indispensable when the relationship of multiple rover positions needs to be adjusted in the same reference frame to map the crater.
The Site frame and the LSC system both consider Mars as a flat plane because of the relatively small mapping area. On the contrary, the Mars body-fixed reference system considers Mars as a sphere and is appropriate when mapping large area in a global level.
21
Chapter 3: Construction of Image Network
A successful construction of an image network is of vital importance to Martian crater mapping, because it indicates the attitude of each image and therefore holds the key to connecting all images. The hinge to the successful construction of image network is the tie points that link images taken from multiple rover positions. This chapter will first introduce the classification of tie points based on the types of stereo pairs, and then the process of tie point selection under each category. Focus will be put on the theory and implement of wide-baseline tie point selection.
As discussed in the last chapter, the data downloaded directly from rovers is called telemetry data. Although JPL/NASA uses different ways intending to control the accuracy of telemetry data, it is mostly based on the movement of rovers’ wheels
(Ali, et al., 2005). On the Earth, the mileage of a car is calculated by multiplying the rotation of the tire with its perimeter. Similar with cars, Spirit and Opportunity use wheels to move upon Mars, and JPL/NASA uses the same method above to obtain the drive distance of the rovers. However, different from the roads made of pitch and asphalt, the Martian surface is covered mostly by sand and rocks, which sometimes cannot provide enough static friction to grab the rovers’ wheels. When slippage happens, wheels continue rotating but the rover stays still. It can lead to blunder in the drive of telemetry data. This condition happens so often that it is very risky to trust the telemetry data over a long drive. The real relationship among different rover positions needs to be recovered so that images from multiple positions can be linked
22
together. And the key factor to the successful construction is sufficient number of high-quality and well-distributed tie points.
As the name implies, tie points are points that can be observed in more than one images and then tie them up. According to what images are used to form the image pair, tie points can be classified into three categories: intra-site, inter-site and cross-site. Intra-site tie points are selected within a stereo pair in one rover position, also named as intra-stereo (Xu, 2004). This stereo pair is composed of the left and right images taken by Navcams or Pancams. The overlap is around 90% for Navcam images and 70% for Pancam images. Inter-site tie points are selected in neighboring image pairs in one rover position that have overlapping area, also named as inter-stereo. The overlay between inter-stereo is around 10%. Cross-site tie points are selected in image pairs taken from different rover positions. Theoretically, any image pairs that have overlap can be used for cross-site tie point selection. Figure 3-1 illustrates these three types of tie points.
Figure 3-1. Three types of tie points.
Intra- and inter-site tie points can be selected automatically using a systematic 23
approach developed by the Mapping & GIS Lab. This tie point selection method consists of four steps: (1) interest point extraction using the Förstner operator
(Förstner and Guelch, 1987), (2) interest point matching, (3) parallax verification, and
(4) tie point selection by gridding (Xu et al, 2002).
The Förstner operator measures the cornerness and uses local statistics to calculate the selection threshold. Compared with other common-used feature extraction algorithms, Förstner operator has fairly good localization and noise robustness, which makes it an excellent choice for applications in photogrammetry and computer vision over decades. The algorithm identifies interest points, edges and regions using the autocorrection matrix A . The derivatives of Matrix A are computed on the smoothed image, and are then summed over a Gaussian window.
Contrary to Harris, Förstner takes the two eigenvalues of the inversion of to define the size and shape of the error ellipse. The size is determined by:
1 det(A) w , w 0 trace(A) 1 2 (3-1)
And the shape of the ellipse is determined by:
4 det(A) q 1 ( 1 2 )2 , 0 q 1 trace(A)2 1 2 (3-2)
The values of w and q determine the characteristics of the feature as follows
(Rodehorst and Koshan, 2006):
Small circular ellipses define a salient point;
Elongated error ellipses suggest a straight edge;
Large ellipses mark a homogeneous area.
In practice, Förstner operator is often used as it is easily extended to detect the center of circular features along with corners.
24
Usually, about 5000 interest points can be extracted by Förstner operator from each Navcam/Pancam image. These points are first matched from the left image to the right image using the normalized cross-correlation coefficient (NCCC) method. The
NCCC between a reference image r(x, y) and a scene image f (x, y) is defined as
m / 2 n / 2 f (x i, y j) r(x i, y j) m n f r im./ 2 jn / 2 1/ 2 2 2 2 2 ( f (x i, y j) m n f ) (r (x i, y j) m n r ) i j i j (3-3)
Where m n is the size of the template window; f and r are the gray-level averages of the window subimages from the scene and the reference, respectively (Tsai and Lin, 2003).
However, it slows down the matching process a lot if the template window slides through all interest points. To accelerate the process, for each interest point in the left image, only points close to its epipolar line in the right image are considered and checked by the NCCC method. The typical size of the template window is 15×15.
Only the point that has the highest NCCC greater than a threshold will be kept as a candidate. Then points are matched from the right image to the left image for cross verification. When a pair of points proves to be the best match in both directions, they are considered as matching points, otherwise they will be discarded.
Although this cross verification reduces the chance of mismatch, it is still possible to have outliers left. A parallax curve verification method is developed and used to filter the remaining mismatch points. Parallax consistency is one type of spatial consistency. When all candidates of the matching points are sorted in the row direction from image top to image bottom, their parallaxes can generate a wave in a monotonic decreasing trend, with some abnormal points deviate from the trend. Small variations represent the continuous terrain changes and landmarks, such as rocks. 25
Large variations may represent peaks or valleys. Therefore, the terrain can be modeled as a parallax curve by applying a median filter on the original parallax wave.
The outliers can be identified if their distances to the parallax curve are greater than a terrain roughness threshold. Figure 3-2 gives an example of the original parallaxes and the parallax curve after passing a filter.
Figure 3-2. Parallaxes of all candidate matching points and the parallax curve.
The above procedure proves its success in selecting tie points within the same rover position. At most of the rover positions (>95%), intra- and inter-site tie points can be selected automatically through this procedure.
The attitude and position information from the telemetry data within one rover position is generally consistent, because the rover stays still and only the PMA and certain arms move. This consistency is further guaranteed under the control of intra- and inter-site tie points. To some small craters observed in a single rover position,
26
intra- and inter-site tie points are already sufficient to construct the image network for the crater mapping. However, ground points of the same feature derived from the positioning information of different rover positions usually have significant inconsistencies caused by factors such as wheel slippage and IMU angular drift.
Cross-site tie points are necessary in this case.
The selection of cross-site tie points has been, and is still extremely challenging as a result of the difficulty from the significant differences in viewing angles, resolutions, and distances associated with images from different rover positions.
Therefore, selection of cross-site tie points is often conducted manually instead of automatically. Figure 3-3 shows an example of the same feature in images taken from two different rover positions. At the first glance, it is even difficult for human eyes to identify the same feature.
Figure 3-3. The same feature observed from two different rover positions.
To overcome the difficulty, the Mapping and GIS Lab developed a number of
27
interactive tools to assist manual selection. For example, one big issue about choosing the same feature is its unknown spatial relationship with rover positions. But the orthoimage tool can generate two local orthoimages of the two positions using the telemetry data, and overlay them to identify the corresponding features. Demonstrated in Figure 3-4, the spatial relationship between the two rover positions can be recovered by overlaying these two orthoimages based on the position of Feature 1.
Then the inconspicuous correspondence of Feature 2 can be noticed.
Figure 3-4. Finding the feature correspondence by overlaying orthoimages. Obvious features are used as indicators for spatial relationship between rover positions, and then inconspicuous features can be matched.
Another common issue is that feature may look widely different when the illumination changes. The anaglyph stereo tool is particularly helpful in this case. It can identify the corresponding feature by reproduce its 3D shape in the scene, and the shape is irrelevant with the illumination. These tools are used separately or in
28
combination to identify features depending on the characteristics of the specific terrain.
Since the cross-site tie point selection involves more than one rover position, the image combination has wide options. If there are only two rover positions, both positions will use their own stereo pairs to calculate the coordinates of tie points, so that the offset between two sets of coordinates can be eliminated in the bundle adjustment. But if there are more than two positions, any two positions can be combined to generate stereo pairs as long as the images have overlapping areas. Then the bundle adjustment may adjust these two positions as an analogous rigid body to other positions. The primary difference between these two patterns is the length of the baseline between the left camera and the right camera. The baseline in the first condition is fixed since the cameras are installed on the PMA, so it is named as hard baseline. On the contrary, the baseline in the second condition is flexible depending on the image combination, so it is named as wide baseline, or soft baseline.
The baseline length is an important factor to decide the accuracy of the coordinates of cross-site tie points. Figure 3-5 below illustrates how the baseline length can affect the accuracy of a feature in Pancam images. Assume that the matching point in the right image gets an offset of 1 pixel (10 µm × 10µm) in the across-photograph axis, and the ground point is about 50 m away from cameras. If a stereo pair with hard-baseline is used to observe this point, the 1 pixel offset can cause
2.6 m offset in the along-photograph direction. However, if two images from two rover positions that are 5 m from each other are used as a stereo pair, the offset in the object space can be reduced to 0.15 m. This fact determines what type of baseline length should be used under different conditions.
29
Figure 3-5. The effect of baseline length on the accuracy of coordinates of tie points.
In the rover localizations for MER mission, cross-site tie points are always selected using hard-baseline stereo pairs. With the limit of manpower and required timeliness of the mission, an incremental bundle adjustment is performed in a stepwise manner, which only involves two adjacent rover positions each time.
Therefore, the 3D coordinates of tie points must be extracted using stereo pairs from both rover positions separately. The same condition happens to the mapping of some craters until the Opportunity decided to study Santa Maria Crater.
Santa Maria is a relatively young impact crater, but old enough to collect sand dunes in its interior. The Compact Reconnaissance Imaging Spectrometer for Mars
(CRISM) data shows indications of hydrated sulfates on the southeast edge of the
Santa Maria Crater (Kremer, 2010). And the sand dunes at the bottom also hold clues to the past and present climate processes on Mars. Its scientific value cannot be dug out without a detailed terrain model. Therefore, the traverse around Santa Maria was
30
specially designed to fulfill the mapping task.
Before Santa Maria, only one large crater, Endurance, was mapped completely by the Mapping and GIS Lab. Another mapping that involves multiple rover positions is for the Duck Bay at Victoria Crater. By comparing these three mapping cases, it is not difficult to observe that the rover positions around Santa Maria are grouped into the western part and the eastern part, in which the positions of Site 1, 2, 4, and 5 are planned specifically for wide-baseline mapping. This special distribution provides an unprecedented opportunity for us to study the wide-baseline tie point selection.
(a) (b)
Continued
Figure 3-6. The distribution of rover positions near (a) Endurance, (b) Duck Bay at
Victoria, and (c) Santa Maria.
31
Figure 3-6 continued
(c)
In the case of mapping Santa Maria, the key to success is to link the western positions with the east positions by cross-site tie points. The best option is to select these points in the middle of the crater so that they are not too far or too close to either side. However, if the traditional hard-baseline selection is used, it may cause a systematic bias among sets of tie points because of the reliable distance of
Pancam/Navcam images. And this bias will be carried to the initial rover positions through rigid transformation, which will be discussed later, and then affect the rover localization, as illustrated in the following Figure 3-7.
32
Figure 3-7. The result of rigid transformation when hard-baseline cross-site tie points are used. The dots in four different colors are tie points observed from those four rover positions. The black crosses are telemetry rover positions. The red crosses are the true rover positions. The green crosses are the rover positions following the pattern of the hard-baseline tie points.
The solution to this issue is to construct stereo pairs with wide baseline between two rover positions and use their camera parameters to calculate the 3D coordinates of tie points. For example, the left images in Site 1 and Site 2 that have overlaps are selected to form stereo pairs, so do the left images in Site 4 and Site 5. Instead of having four sets of hard-baseline cross-site tie points from four rover positions, this wide-baseline method only generates two sets of coordinates for the tie points, and they are with much more accuracy. Figure 3-8 and 3-9 present the structure and its advantage over the hard-baseline method. As can be seen, the pattern between two sets of points is consistent with the pattern between the telemetry rover positions and the true positions.
33
Figure 3-8. The construction of wide-baseline tie point selection.
Figure 3-9. The result of rigid transformation when wide-baseline cross-site tie points are used. The red dots are observed from Site 1 and 2 in the western side, and the blue dots are from Site 4 and 5 in the eastern side.
Although the wide-baseline method was used in dense matching in previous crater mappings, it has never been used in the tie point selection phase. The successful experience at Santa Maria Crater demonstrates the feasibility and advantage of wide-baseline tie point selection in mapping large craters in the future. 34
After the selection of intra-, inter-, and cross-site tie points, either by hard-baseline method or wide-baseline method, it is ready to link all images taken from different rover position together to generate the image network. In this network, each image is a node in a connected graph, and the tie points function as paths among nodes.
This chapter introduced three types of tie points for constructing image network.
Intra- and inter-site tie points can be selected automatically with a systematic process, but cross-site tie points are often selected manually due to the significant differences in viewing angles, resolutions, and distances between images and rover positions.
With the help of interactive tools, the difficulty of tie point selection may be simplified. To ensure the accuracy of coordinates of tie points, different combinations of image are used to compose soft baseline, or wide baseline. The experiment at Santa
Maria Crater is the first successful attempt of this method, and the experience can be used for reference in the future mapping.
35
Chapter 4: Integrated Bundle Adjustment
Bundle adjustment is almost always used as a step in every feature-based 3D reconstruction algorithm. It is the problem of refining a visual reconstruction to produce jointly optimal 3D structure and viewing parameter (camera pose and/or calibration) estimates (Trigg, et al, 1999). Bundle adjustment is the most important step in the Martian crater mapping. All the following processes must be built on the success of bundle adjustment. In the previous MER mission operation, a step-wise incremental bundle adjustment was used because of the timeliness and the lack of manpower. In recent studies, a new integrated bundle adjustment that utilizing both orbital and ground data is developed. Chapter 4 is going to discuss the general theory behind bundle adjustment and the practice of both incremental and integrated manners, and then to explore their strengths and weaknesses.
4.1 Theory of bundle adjustment Bundle adjustment was originally conceived in the field of photogrammetry during 1950s and has been applied increasingly to computer vision, industrial metrology, surveying, geodesy and many other fields. Bundle adjustment is a problem about geometric parameter estimation including the coordinates of ground features, the interior and exterior orientation parameters of cameras. The purpose of bundle adjustment is to find a set of the parameters that minimizing the model fitting error described by certain cost functions, and thus to give good prediction of the location of the observed points in a set of images. Like other adjustment computations, classical
36
bundle adjustment is formulated as a nonlinear least square problem. The cost function is assumed to be quadratic in the feature re-projection errors, and robustness is provided by explicit outlier filtering.
Mathematically, the core of bundle adjustment is the collinearity equations, which describes the geometry among the projection center, the coordinates of a point in the object space, and its coordinates in the image plane. As the name implies, these three elements are connected collinearly by a bundle of rays. An optical camera system can help illustrate the principle as shown in Figure 4-1 below.
Figure 4-1. Geometry of the collinearity.
There are three coordinate systems in the above figure:
37
(1) The image space coordinate system. Its origin is the projection center S (the camera). The z axis coincides with the principal optic axis while the positive direction is against the photographic direction. The x and y axes are parallel to the x and y axes in the image plane coordinate system. This system is represented by S xyz .
(2) The object space coordinate system. The origin is at an arbitrarily specified location M . The Z axis coincides with the zenith directions at the origin of the coordinate system. Then the X and Y axes form a horizontal plane, whose directions can be determined by a certain ground coordinate system, for example, the direction of the base line between two cameras, or the flight direction. This system is denoted by M XYZ .
(3) The image auxiliary coordinate system. Its origin is also S , but the axes are rotated to be parallel to those of object space coordinate system. Angle rotates around the axis SY ; Angle rotates around the axis SX ; Angle rotates around the SZ . This coordinate system is denoted by S uvw.
Given a ground point A , its coordinates in M XYZ is (X A ,YA ,Z A ) , and its image point has the coordinates of (x, y, f ) with respect to S xyz and
(X,Y,Z) with respect to S uvw. The coordinates of the projection center S in
M XYZ is (X S ,YS ,Z S ) . The following equations can be deduced from the similar triangles in Figure 4-1:
X Y Z 1 (4-1) X X Y Y Z Z A S A S A S in which is a constant scaling. The equations can also be written in the matrix format as:
38
X X A X S 1 Y Y Y (4-2) A S Z Z Z A S
Besides that, the rotation between the image space coordinate system and the image auxiliary coordinate system can be used to establish the following equation:
X x a1 a2 a3 x Y R y b b b y (4-3) 1 2 3 Z f c c c f 1 2 3 in which matrix R is the rotation matrix whose elements are functions of the three rotation angles:
cos sin 0 1 0 0 cos 0 sin R R R R sin cos 0 0 cos sin 0 1 0 (4-4) 0 0 1 0 sin cos sin 0 cos
Combining equations (4-1) to (4-4), the collinearity equation is given as follows:
a (X X ) b (Y Y ) c (Z Z ) x x x f 1 A S 1 A S 1 A S a 0 a (X X ) b (Y Y ) c (Z Z ) 3 A S 3 A S 3 A S (4-5) a (X X ) b (Y Y ) c (Z Z ) y y y f 2 A S 2 A S 2 A S a 0 a (X X ) b (Y Y ) c (Z Z ) 3 A S 3 A S 3 A S in which (xa , ya ) and (x0 , y0 ) are the image coordinates of Point A and the principle point S in a coordinate system whose origin is the image center.
As can be seen from Equation (4-5), each point can construct two collinear equations with variables. According to different applications, certain variables are known, while some others need to be calculated. However, the interior orientation
parameters, including f , x0 , and y0 , are considered as known in most conditions since they are measured in the laboratory calibration process and are rarely changed.
The image coordinates of points can also be known through direct measurement from images. In the application of Martian crater mapping, the unknowns include the
39
exterior orientation parameters of all the Pancam / Navcam images in the image network, and the ground coordinates of all the tie points.
Good initial values are essential to get the optimal solution to the collinearity equations. The attitude information of images from the telemetry data within one rover position is generally consistent and can be used directly as initial values.
However, the coordinates of same features observed from different positions are often inconsistent with each other and thus must be pre-processed for bundle adjustment.
This inconsistency comes from the factors such as wheel slippage, IMU angular drift, and other navigation errors. Since these factors exist through the whole traverse in a systematic manner, a rigid transformation (translation and rotation without scaling) is applied to all relevant rover positions except for the reference position with the help of cross-site tie points. A least-square-based algorithm is iterated to estimate the optimal transformation model and screen the outliers in tie points. The transformation parameters are determined only by tie points with residual errors less than 1 m in all
X-Y-Z directions. The exterior orientation parameters and the ground coordinates of features will be much more consistent and can be used as initial values after this transformation.
As can be seen from Equation (4-5), the collinearity equations are non-linear functions with respect to most of the parameters and variables. The Linearization is a good solution for simplifying computation, solving unknowns simultaneously and enabling the least squares to minimize the re-projection errors. The classical linearization algorithm is to use the Taylor series expansion to derivate the linear form.
4.2 Incremental Bundle Adjustment
40
The incremental bundle adjustment was designed for the demand of daily localization task in the MER mission. The rovers move along their traverses and take images incrementally sol by sol. These images generate a special chain rather than a network. For a long traverse, the amount of images involved is extremely huge, which causes two issues:
(1) The tie points between positions cannot provide strong connection in recovering the relative orientation between rover positions. Tie points can only be selected within the reliable distance of cameras if hard-baseline method is used for selection, which limits their ability of binding multiple positions at the same time.
Therefore, it is difficult to find an optical solution for all the positions at the same time, let alone that the tie points may not be as precise as required because of the lack of identifiable features or the long distance between locations.
(2) It is too computationally expensive to involve the entire image network for the localization of each rover position. The time is highly restricted for everyday data acquisition, data processing, rover localization and map submission. In addition, the manual cross-site tie point selection is quite time consuming even for an experienced operator. As the traverse expands sol by sol, it is more and more impossible to take the entire network into account.
In an ideal condition, a sequential computing process allows adding/deleting variables, known or unknown, at any node of the image network, and updates the solution with the alterations, so that the final results are the same with those of a simultaneous process (Gruen, 1985). In consideration of the nature of daily operation and the advantage of sequential computing techniques, an incremental bundle adjustment model was built up for the special application in MER mission (Li et al.,
2002; Ma et al., 2001).
41
This bundle adjustment model is implemented using the Kalman form. At each new rover position, the unknown parameters are estimated with respect to its previous position. For the bundle adjusted positions, the parameters will be re-calculated if new images are available, otherwise the parameters keep unchanged. As long as there are plenty images with identifiable features to guarantee the quality of tie points, this incremental bundle adjustment can obtain a similar solution with the result of a simultaneous bundle adjustment.
In the operation of MER mission, incremental bundle adjustment is always performed in a stepwise manner to locate the Spirit rover, and the result demonstrates that the incremental BA is able to correct significant localization errors (Li, et al.,
2011). However, this success cannot be reproduced in localizing the Opportunity rover.
The reason of this distinct result generally falls into two categories:
(1) Landform diversity. The land covers are quite disparate in Gusev Crater and
Meridiani Planum. Gusev Crater is covered by plenty of rocks, which are perfect candidates for tie points. On the contrary, Meridiani Planum is full of smooth sand dunes without many significant features (Figure 4-2). In addition, the terrain of
Meridiani Planum is much plainer than Gusev Crater. Therefore the number and quality of tie points in MERB operation are hard to predict.
42
Figure 4-2. Typical landforms in Gusev Crater and Meridiani Planum. The left image was taken by Spirit at Site 3100 in Sol 105. The right image was taken by Opportunity at Site 5000 in Sol 399.
(2) Connectivity of image network. Statistical data in Figure 4-3 shows that the average drive distance of Spirit rover within each sol is 8.13 m with the standard deviation of 13.83 m. This length is within the reliable distance of both Navcam and
Pancam, and hence is beneficial to cross-site tie point selection. Meanwhile, the mean drive distance of Opportunity rover within each sol is 22.78 m with the standard deviation of 35.77 m. This flexible range is very risky for tie point selection when
Navcam is the only data source, sometimes even risky for Pancam images. If few cross-site tie points are found between two adjacent rover positions, the connectivity of image network is weak, and the bundle adjustment may derive wrong results.
43
Figure 4-3. The histograms of traverse in Gusev Crater and Meridiani Planum.
In summary, as a bundle adjustment algorithm that heritages thoughts of sequential algorithms, incremental bundle adjustment is effective and accurate for rover localization in MER mission. However, the precision is highly subject to the quality of tie points, especially when the connectivity of image network is weak. The errors in the BA of one single position will be accumulated in all the following positions, until new data is available to update the BA of the first position. In the application of Martian crater mapping, the biggest problem this error accumulation may cause is to distort the crater in the along-photography direction. 44
4.3 Integrated Bundle Adjustment To eliminate significant error propagation experienced in incremental BA due to the insufficient number of features and lengthy drives, an integrated bundle adjustment method has been developed to bring orbital data in the BA process as well as to adjust multiple rover positions simultaneously.
As introduced in Chapter 2, HiRISE imagery has resolution of 0.25 m – 0.3 m, which is the highest until now in orbital images on Mars. This resolution is sufficient to recognize big rocks and other features on the Martian surface. Meanwhile, the nature of orbital imagery decides small distortion in a relatively small area that covers a crater. This distortion is further removed in the orthoimages generated with stereo images. With the nature of high resolution and small distortion, HiRISE orthoimages are commendable reference for initial values in the bundle adjustment.
The contribution of HiRISE orthoimage to the initial values comes from two aspects: the rover localization and the ground positions of tie points. As introduced in
Section 4.2, adjacent rover positions are often linked weakly by insufficient tie points in Meridiani Planum, which means that the number of cross-site tie points is not large enough to ignore the disturbance from outliers. This disturbance is going to affect the result of rigid transformation and provide bad initial rover positions. This phenomenon is mostly determined by the nature of point-based matching. However, if an area-based matching is used instead of point-based matching, the situation can be improved much. The solution is to compare the features, such as ridges, rocks, and rims, as a whole in HiRISE orthoimage and the ground-based local orthoimage. By registering local features, a HiRISE orthoimage and a ground-based local orthoimage within one rover position can be precisely overlaid, and then the rover position at the
45
imaging center on the ground-based orthoimage can be determined on the HiRISE orthoimage as Figure 4-4 shows below.
Figure 4-4. Initialization of a rover position through feature comparison between
HiRISE orthoimage and ground-based orthoimage.
The second contribution of HiRISE orthoimage is to provide good initial values of the coordinates of tie points. Using wide-baseline method could lower the risk of getting bad distant cross-site tie points, but errors can still exist in the coordinates of those points because of the subjectivity in the manual selection, the residual error in the adjusted camera parameters, or other factors. To further improve the accuracy of these coordinates, the HiRISE orthoimage is used as a reference. Only features that can be seen in the orbital images are selected as cross-site tie points, so the small distortion in the HiRISE orthoimage guarantees that tie points are precise enough to play the role of ground control points in the following bundle adjustment. For any distant tie points whose image coordinate are already verified as accurate, if there is still a large difference between the 3D coordinates calculated from the ground images 46
and those measured from the orbital orthoimage, the difference is regarded as the result of inconsistent relationship among rover positions, and the orbital coordinates are considered to be more reliable and are used as the initial values in the following bundle adjustment.
By the contribution of HiRISE orthoimage, the link within the image network is highly strengthened. With good initial values of rover positions and coordinates of tie points, a bundle adjustment is performed over all related rover positions simultaneously, so that the whole image network can achieve the optimal solution as a whole. Experimental results show that this simultaneous bundle adjustment with integrated data input can provide much better attitude and position revision than the incremental bundle adjustment with only ground data.
However, the current integration of orbital data is still in the initial phase with a number of issues under continuous research. One primary issue is the extraction of elevations of tie points from orbital products. In the current experiment, only the horizontal coordinates of tie points measured from the orbital orthoimages are used in the integrated bundle adjustment. Ideally, the elevations of tie points from orbital products would also contribute much to the strong connection of image network because that they are relatively consistent with each other in a large area without a lot fluctuation, especially when the elevation is further controlled by MOLA data.
However, they are not extracted from the orbital DTM or utilized in the bundle adjustment. The crucial reason is the unreliability of the quality of present DTM in small areas such as a crater. As can be observed in Figure 4-5, shadows and shadings are quite common in HiRISE images of Martian craters, especially the ones with certain depth. Image matching process is almost impossible to be fulfilled in these dark areas. And without matching features, the stereo pair cannot provide valid 3D
47
information, let alone act as reference. Figure 4-6 shows an example of the invalid orbital DTM at Santa Maria crater comparing with the DTM generated in the proposed method. The inconsistency is quite inescapable in the shading area. A failure is expectable if the bundle adjustment employs the elevations of tie points from this
DTM.
(a) (b)
Continued
Figure 4-5. Shadows and shadings in HiRISE images of Martian craters. They are clipped from the HiRISE images: (a) TRA_000873_1415, (b) ESP_025680_1350; (c)
TRA_000873_1780. (Image Source: HiRISE)
48
Figure 4-5 continued
(c)
Figure 4-6. The comparison between the DTMs generated from HiRISE images (the above one) and the one generated with proposed method (the bottom one).
4.4 Evaluation of Bundle Adjustment Results It is risky to generate digital terrain models and other topographic mapping products before evaluating the results of the bundle adjustment because of the
49
possibility of remaining inconsistencies. Since no absolute ground control is available on Mars, it is impossible to evaluate the accuracy of the bundle adjustment using the conventional method of comparing the adjusted positions with ground truth. Instead, the accuracy was evaluated by looking at two aspects: the level of consistency among multiple rover positions, and the comparison of bundle adjustment localization results with orbital products.
The consistency among multiple rover positions is checked by comparing the 3D coordinates of points which are calculated using positioning information of different rover positions. These points may not be seen in the orbital image, but they need to be observed in more than one rover position. Then their 3D coordinates are calculated using either the hard-baseline method or the wide-baseline method, according to different circumstances. The offsets among sets of coordinates from different positions indicate the quality of bundle adjustment. Figure 4-7 shows one of these comparison results at Santa Maria Crater. The red dots come from two sites in the west using wide-baseline method. Blue dots come from two sites in the east also using wide-baseline method. Apparently, the inconsistency before bundle adjustment in the left image is reduced significantly after bundle adjustment in the right image. Detailed statistic information can be found in Table 4-1.
50
(a) Features before bundle adjustment. (b) Features after bundle adjustment.
Figure 4-7. Inconsistencies between features among multiple rover positions before and after bundle adjustment. Black crosses are the telemetry rover positions. Red triangles are the rover positions based on the bundle-adjustment results.
Offset Before BA Offset After BA
Point ID X Y Z X Y Z
0 -2.34 5.21 1.58 -0.56 -0.66 -0.33
1 -2.57 5.55 1.36 0.17 -0.29 -0.19
2 -2.63 5.41 2.11 -0.39 -0.46 0.39
3 -3.57 5.46 1.92 -0.94 -0.33 0.31
4 -2.06 5.36 1.82 0.40 -0.08 -0.01
5 -0.63 5.91 1.88 0.77 0.14 0.05
7 -2.79 5.83 1.06 0.13 -0.28 -0.16
Continued
Table 4-1. The statistics of the inconsistencies between features among multiple rover positions before and after bundle adjustment.
51
Table 4-1 continued
8 -1.14 5.91 1.57 -0.24 0.28 -0.08
9 -0.98 5.65 1.99 1.61 -0.24 0.24
11 -1.71 5.60 1.51 -0.11 0.16 -0.25
12 -3.55 4.37 1.45 -0.33 -1.37 -0.23
13 -3.14 4.37 1.70 -0.58 -1.05 0.01
In the experiment at Santa Maria Crater, the 3D coordinates of 151 manually-chosen tie points were measured from all images that contain the features, and then the offset for each point after bundle adjustment was calculated. Result shows that 88.7% of the points have an offset of less than 1 m. The minimum of those offsets is 0.05 m, the maximum is 1.52 m, and the average offset is 0.53 m.
Another evidence of the dependability of the bundle adjustment is that the rover positions and features measured from the adjusted ground images are consistent with the same positions and features observed from the orbital orthoimage. This evaluation requires features that can be observed in both orbital image and ground images.
Figure 4-8 illustrates a good comparison result at Santa Maria Crater. Red dots represent ground-image-based features after bundle adjustment. Yellow dots represent the same features measured from the orbital orthoimage. Meanwhile, the red triangles are the rover positions based on bundle adjustment results, and the yellow triangles are the rover positions measured from HiRISE orthoimage. After bundle adjustment, the minimum of those offsets is 0.31 m, the maximum is 1.66 m, and the average offset is 0.74 m. More detailed statistic information about the compared features is listed in Table 4-2.
52
Figure 4-8. The consistency of rover positions and features between bundle adjusted ground images and orbital orthoimage.
Positions in HiRISE Positions in Ground Offset
Point ID X Y X Y X Y
0 4,491.79 -12,546.28 4491.71 -12546.23 -0.08 0.05
1 4,517.97 -12,537.74 4516.99 -12538.54 -0.98 -0.80
2 4,494.57 -12,556.36 4494.70 -12556.16 0.13 0.20
3 4,532.82 -12,543.54 4532.32 -12543.09 -0.50 0.45
4 4,532.62 -12,554.14 4532.52 -12552.50 -0.10 1.64
Continued
Table 4-2. The statistics of the consistency of features and rover positions between adjusted ground images and orbital orthoimage.
53
Table 4-2 continued
5 4,556.77 -12,565.65 4555.66 -12565.58 -1.11 0.07
6 4,545.27 -12,610.18 4544.50 -12610.85 -0.77 -0.67
7 4,532.96 -12,594.37 4532.54 -12594.72 -0.42 -0.35
8 4,507.60 -12,600.89 4506.92 -12600.45 -0.68 0.44
9 4,519.64 -12,609.50 4520.05 -12608.74 0.41 0.76
10 4,513.59 -12,617.78 4513.55 -12617.18 -0.04 0.60
11 4,493.14 -12,610.78 4494.01 -12610.31 0.87 0.47
12 4,471.24 -12,559.56 4470.94 -12559.51 -0.30 0.05
Site ID X Y X Y X Y
1 4648.10 -13301.33 4648.10 -13301.33 0 0
2 4649.75 -13306.61 4650.07 -13307.48 0.32 -0.87
3 4744.96 -13332.90 4743.44 -13333.31 -1.52 -0.38
4 4743.96 -13322.58 4743.93 -13322.13 -0.03 0.45
5 4745.07 -13317.64 4744.92 -13317.37 -0.15 0.27
The above criteria can be used alone or in combination to check the quality of the bundle adjustment before it was applied to crater mapping.
In this chapter, the basic theory of bundle adjustment and its two practical implementation manners for different circumstances are introduced. The incremental bundle adjustment is designed mainly to meet the timely limitation in the rover localization task in the MER mission. It works well when sufficient, well-distributed tie points can be found between adjacent rover positions. However, due to the different landforms and other factors, this successful example in Gusev Crater cannot
54
be repeated in Meridiani Planum. To eliminate the error accumulation that may be caused by incremental bundle adjustment, an integrated bundle adjustment is designed to involve orbital data in the process. Despite of the lack of features in Meridiani
Planum, the initial values of rover positions can still be ensured by HiRISE orthoimage. In addition, the coordinates of tie points can also be supervised by orbital images. Although this integrated method is still under research, the evaluation reveals its great potential for Martian crater mapping.
55
Chapter 5: Mapping Product Generation
After the bundle adjustment is performed and evaluated, the attitudes and positions of all cameras agree with each other and get prepared to extract dense matching points within the craters to build the terrain model. Besides the DTM, which is the most fundamental mapping products, other products will also be derived, including the orthoimage, the slope map, the contour map, and so on. However, these mapping products cannot be generated without the basic DTM. Therefore, this chapter will devote itself to the detailed process of DTM generation since it is the bed stone for all other mapping products. Some discussion will be put forward about interest point extraction in featureless area. A brief introduction will also be given to the generation of the orthoimage, including the back-projection and other image processing methods. Since other derivatives such as the contour map and the slope map can be easily generated by commercial software such as ArcMap, few discussions will be put on them.
5.1 Generation of DTM DTM generation includes: (1) dense interest point matching, (2) 3D coordinate calculation of the matched points, and (3) DTM interpolation. In these three steps, dense interest-point matching and 3D coordinate calculation of the matched points are related with each other very closely, and they form the fundamental part of the DTM generation. Based on the theoretic research discussed before and the empirical
56
experience accumulated in these years, hard-baseline and wide-baseline techniques are both used to implement these two tasks under different conditions.
There is no strict law on which method should be used in which conditions, but some rules may help for the decision: (1) the dimension of the crater, (2) the distance between the crater and the camera, and (3) the available image type. All the three rules are defined in a relative way. For example, the definition of a small crater in this research varies depending on what type of rover images were taken around the crater.
If both Pancam images and Navcam images are available to map the crater, a small crater could be as large as about 60 m in diameter. But if Navcam images are the only data source, the diameter should not be over 30 m for the purpose of accurate mapping. These diameters are determined by the reliable distances of cameras. In addition, the distance from the crater to the camera is also critical to determine the dense matching method. Even if the crater itself is a small one, if it is far from the rover position, a hard-baseline method may not have a good performance at the far end of the crater since it is outside the reliable distance. In a summary, hard-baseline method is a simple but effective choice for craters within the reliable distances of the cameras, while wide-baseline mapping is used mainly to cover areas that are blocked by crater walls near sites or when points are more distant than the reliable mapping distance found for hard-baseline mapping. Figure 5-1 gives a simple illustration on this principle.
57
Figure 5-1. Rules for choosing dense matching method.
However, a phenomenon has been more and more noticeable in our research these days, which cannot be classified using the above rules. Following the drive plan, the rover may pass by an interesting crater, taking a few images of the crater in one position without making a close observation. As a result, the near end of the crater will be blocked by its own rim while no images from other positions were available to observe this blocked area. Figure 5-2 shows an example at Geographe Crater in
Meridiani Planum. The diameter of this crater is about 13 m, but the distance to the rover is about 23 m, which makes part of it invisible. However, this crater was only observed in Site 4700, so this crater cannot be mapped using the wide-baseline method according to the above rules. With the help of orbital image, the blocked area may be completed. However, the resolution will not be as high as the one using ground images. A more critical issue is that a precise bundle adjustment must be implemented simultaneously on both orbital images and ground images to guarantee a smooth mosaic, which is still very challenging so far. This topic is our priority in the future work.
58
Figure 5-2. Geographe Crater with its blocked area.
Similar with tie point selection, the dense interest point matching is also divided into the automatic process and the inevitable manual process. Most of the automatic process is implemented by the same method used in tie point selection: Förstner operator and NCCC. This combination works well in the areas with rich features, but it performs poor in the relatively smooth areas, such as the sand dunes in the bottom of many Martian craters. Yet the sand dune is an important subject in the research of
Mars. Scientists believe that those unusual ripples hold clues to the past and present climate processes on Mars. Since they are an outer layer composed of granule-sized material ranging from about 2 mm - 5 mm (Williams, et al., 2002), they look so smooth even from the high-resolution rover images that very few features can be found in the troughs. Another factor affecting the performance of the Förstner operator and NCCC is the very different viewing angles at sand dune area. As mentioned in previous chapters, sandy areas are the most dangerous area for rovers because of the possibility of stuck of wheels. With that principle in mind, the rover almost never took the risk of going into craters, not to mention the sand dunes in the middle of the craters. Therefore, the sand dune is always far from rover positions and
59
cannot be mapped in the hard-baseline way. Only one option is left in this condition, to use wide-baseline mapping and to live with the different viewing angles.
One beneficial solution to this dilemma is to use Scale Invariant Feature
Transform (SIFT) algorithm at the sandy areas to find as many matching point candidates as possible. This algorithm was developed by Lowe (1999; 2004) and has been demonstrated by numerous papers as superior to other feature matching algorithms in scale and affine variant regions. Since this algorithm is already known to everyone in the computer vision related field, the technical details will not be introduced again here. With the help of SIFT algorithm, the features can be extracted more from the troughs rather than along the ridges, which is necessary to map the detailed shape of the sand dune area. Otherwise, if all features are extracted on the ridges, the elevation of the sand dune will lose the small variations.
Although SIFT algorithm is known for its accuracy in feature extraction, it will still have many outliers left because the smoothness of the sandy area, and much manual editing must be involved to delete the mismatches and add new matching points. It is very onerous even for a very experienced operator, but it is also very necessary step before any better automatic matching algorithm appears. Figure 5-3 plots the dense matching points over the HiRISE orthoimage in the Santa Maria
Crater, including automatic ones and manual ones. Manual points are added as many as possible to make sure that all obvious features in the crater are covered. However, some smooth areas on the wall are still not covered by any points. Considering that these smooth areas have a relatively simple slope, the mapping quality will not be affected too much.
60
Figure 5-3. The dense matching points in Santa Maria Crater.
With sufficient matching points, a DTM can be generated easily using interpolation methods, such as Kriging, Natural Neighbor, and Spline provided in
ArcMap. In this study, the Natural Neighbor method is used to get a relatively smooth surface. The DTM of Santa Maria Crater is shown in Figure 5-4 with a resolution of
0.1 m. Figure 5-5 gives a detailed DTM in the sand dune area. Compared with
HiRISE orthoimage, it is easy to see that the ridges and troughs in the DTM are very close to “the ground truth”.
61
Figure 5-4. Feature Comparison between the DTM of Santa Maria Crater and the
HiRISE orthoimage. The above image is the HiRISE orthoimage of Santa Maria. The bottom image is the DTM generated from the proposed method.
62
Figure 5-5. The detailed DTM at the sand dune area of Santa Maria Crater.
5.2 Other Topographic Product Mapping Although the DTM is good at presenting the terrain, it is not good at reflecting the ground materials, from which an experienced researcher can dig out much information about the geological structure, climate change, and even evidence of water existence. Therefore, an orthoimage is necessary for the research purpose.
Points in the DTM are back-projected to the ground images taken from multiple positions to get corresponding pixel values. The adjusted camera parameters are used to calculate the coordinates in image plane from the 3D coordinates in the object space. In principle, the image pixel taken from the nearest rover position should be used for the pixel value, because the calculation accuracy increases when the distance gets smaller and smaller. To a crater with certain depth, most part of the crater can be processed following this rule. But unlike flat areas, the rim of a crater can block the rover from observing the nearby features (Figure 5-6). If following the same rule in this condition, pixels on the boundary of the blocked area will be used as substitutes
63
and cause blunder in the orthoimage, as Figure 5-7 shows. The red polygon marks a blocked area which cannot be observed from western positions, but the software still gives pixel values from images of the rocks on the western cliff just because those pixels are closest to the blocked area.
Figure 5-6. The blocked areas in crater mapping.
Figure 5-7. Using the nearest pixels causes wrong filling in blocked areas.
64
To eliminate these exceptions, the image pixels taken from the opposite sides are used for the final pixel values. First of all, the program generates the orthoimage automatically without any human intervention. Then the images taken close to the blocked areas are removed from the database, so that the software is forced to use the images from distant cameras. Take Santa Maria Crater for an example, images from
Site 4 and 5 are used to map the blocked areas in the western part of the crater, while images from Site 1 and 2 are used for the blocked areas in the eastern part.
Another issue in generating orthoimage is the coherence of the brightness among different ground images. As can be seen in Figure 5-7, the sand dune area is obviously brighter than other areas of this crater. The rocks on the cliff are also always brighter than their surroundings. This is an inevitable phenomenon caused by the exposure characteristics of the cameras onboard the rovers, which may not affect the scientific research, but cannot give observers a good visualization because it splits the crater in pieces. It is not realistic to pre-process all those images to get a similar brightness, especially when they are separated from each other, but a post-processing is quite simple and effective. The Photoshop provides the function of adjusting brightness with real-time preview, so the distinguished areas can be chosen carefully and be adjusted until the operator sees different pieces merging together. Figure 5-8 shows the brightness at sand dune area before and after this adjustment.
65
Figure 5-8. The brightness adjustment in sand dune area of Santa Maria Crater. The left image is before adjustment, and the right image is after adjustment.
The complete orthoimage will look quite smooth after all those circumspect image processing steps. Figure 5-9 illustrates how the final orthoimage is improved from the raw version. This improvement also demonstrates the necessity of the post-processing in orthoimage generation.
Figure 5-9. Orthoimage of Santa Maria Crater before and after image post-processing.
The resolution is 0.1 m. 66
The quality of the above orthoimage can be evaluated by comparing it to the
HiRISE orthoimage. In Figure 5-10, features in both orthoimages are quite matching with each other.
Figure 5-10. Comparison between orthoimages generated via different methods.
Since Santa Maria Crater is not in a regular shape, the dimension is compared with HiRISE orthoimage by measuring the diameter in northing and easting directions using the same features as the end points. Table 5-1 lists the result of comparison. The orthoimage/DTM generated by the proposed method have very similar dimension with the HiRISE product.
67
Northing Easting
HiRISE 95.12 m 91.83 m
New Method 95.33 m 91.95 m
Table 5-1. The comparison on the dimension of Santa Maria Crater.
Other mapping products, such as the contour map and the slope map can be derived easily using commercial GIS software. As per the Mars Data Analysis
Program (MDAP) proposal, the positions of Alpha Particle X-Ray Spectrometer
(APXS) data and Microscopic Imager (MI) images are also adjusted using the bundle-adjusted rover positions and then plotted to the terrain models. Figure 5-11 is a summary of the product set at Santa Maria Crater. No MI images were taken around
Santa Maria, so there is no legend in this summary.
68
Figure 5-11. The product set at Santa Maria Crater.
69
Chapter 6: Conclusions
Surrounding the topic of Martian crater mapping, this thesis followed the workflow in Chapter 1 and introduced all the important steps leading to a successful mapping and the theories behind those steps.
Mars draws more and more attention in these years because it is the only planet that may support lives, except for the Earth. High-quality surface mapping can provide detailed data for scientists to analyze the geological environment of Mars, and to look for the evidence of water existence, which is the most direct proof for the possibility of life on Mars.
With the help of HiRISE data, it is now possible to integrate the orbital data and the ground data for high-resolution mapping. HiRISE data contributes in the image network construction by providing reference on the position of tie points. HiRISE acts as the ground truth in the bundle adjustment phase by helping in deciding the initial values of the rover positions. This integration is still in the initial stage, but more and more study will be done in the future to utilize HiRISE data in a deeper level. As mentioned in Chapter 5, our next goal is to bundle adjust the orbital images and ground images simultaneously. There are plenty of issues to be fixed, but the integration will definitely reform the whole crater mapping process.
As to the ground data processing, the wide-baseline method is used in the tie point selection phase for the first time, and proves to be very effective for controlling the accuracy of distant tie points in the case of Santa Maria Crater mapping.
Experiments in other craters need to be designed to make sure that this method is 70
robust under various conditions. Theoretical analysis is still in demand to support the realistic practices.
Another important evolution is that the bundle adjustment is not in the step-wise manner any more in the crater mapping process. Instead, a simultaneous manner with orbital reference is put forward to acquire more accurate rover position adjustment.
Along with better-controlled tie points, this adjustment turns to be very successful at
Santa Maria Crater. This method will continue the fulfillment in other large craters listed in the MDAP proposal.
Besides the big improvements above, some algorithms and programs are added in to the process for better automation and visualization. For instance, the SIFT algorithm helps automate the dense interest point matching in the featureless areas, which saves manual work. The Photoshop program is used in the orthoimage generation to solve the issues in mosaicking ground images with different brightness.
Without the help of these useful tools, the high-resolution Martian crater mapping could have been a much more difficult task.
On one hand, the proposed method proves successful in the case of Santa Maria
Crater and shows great potential in Martian crater mapping. On the other hand, there are still many questions about its robustness and common adaptability. More experiences need to be accumulated to give more convincing conclusions. The future work will focus on the aforementioned issues in this chapter, and try to involve more scientific analysis upon the mapping products for data mining.
71
References
Ali, K. S., C. A. Vanelli, J. J. Biesiadecki, et al., 2005. Attitude and Position
Estimation on the Mars Exploration Rovers. 2005 IEEE Conference on
Systems, Man, and Cybernetics, Kona, Hawaii, October, 2005.
Bell III, J. F., S. W. Squyres, R. E. Arvidson, et al., 2004. Pancam Multispectral
Imaging Results from the Spirit Rover at Gusev Crater. Science, 305:
800-806.
Delamere, A., I. Becker, J. Bergstrom, et al., 2003. MRO High Resolution Imaging
Science Experiment (HiRISE): Instrument Development. Sixth International
Conference on Mars, Houston, TX, No. 3287.
Förstner, W., and E. Guelch, 1987. A Fast operator for detection and precise location
of distinct points, corners and centers of circular features. ISPRS
Intercommission Workshop on Fast Processing of Photogrammetric Data,
Interlaken, Switzerland, pp. 281-305.
Golombek, M., and T. Parker, 2004. Lander Localization, MER Project Memorandum
(One for Spirit, One for Opportunity). JPL/NASA.
Gruen, A., 1985. Algorithmic Aspects in On-line Triangulation. Photogrammetric
Engineering & Remote Sensing, 51(4): 419-436.
Hwangbo, J. W., 2010. Integration of Orbital and Ground Imagery for Automation of
Rover Localization. PhD dissertation, The Ohio State University.
JPL, 2004. Mars Exploration Rover Project Software Interface Specification (SIS)
Rover Motion Counter (RMC) Master File Version 1.0. JPL D-22854. 72
Kremer, K., 2010. Powerful Mars Orbiter Directs Opportunity to Clays and Hydrated
Minerals. University Today. Accessed July 31, 2012,
http://www.universetoday.com/81789/powerful-mars-orbiter-directs-opportu
nity-to-clays-and-hydrated-minerals/
Li, R., F. Ma, F. Xu, et al., 2002. Localization of Mars Rovers Using Descent and
Surface-based Image Data. Journal of Geophysical Research, Planets FIDO
Special Issue, Vol. 107 (E11), pp. FIDO 4.1 – FIDO 4.8.
Li, R., S. W. Squyres, R. E. Arvidson, et al., 2005. Initial Results of Rover
Localization and Topographic Mapping for the 2003 Mars Exploration Rover
Mission. Photogrammetric Engineering & Remote Sensing, 71(10):
1129-1142.
Li, R., S. He, Y. Chen, et al., 2011. MER Spirit Rover Localiztion: Comparison of
Ground Image- and Orbital Image-based Methods and Science Applications.
Journal of Geophysical Research, 116(E00F16), doi: 10.1029/2010JE003773.
Lowe, D. G., 1999. Object Recognition from Local Scale-Invariant Features.
Proceedings of the International Conference on Computer Vision, 2:
1150-1157.
Lowe, D. G., 2004. Distinctive Image Features from Scale-Invariant Keypoints.
International Journal on Computer Vision, 60(2): 91-110.
Ma, F., K. Di, R. Li, et al., 2001. Incremental Mars Rover Localization Using Descent
and Rover Imagery. ASPRS 2001 Annual Conference, St. Louis, MO, April
23-27, 11 pp. CD-ROM.
Maki, J. N., J. F. Bell III, K. E. Herkenhoff, et al., 2003a. Mars Exploration Rover
Engineering Cameras. Journal of Geophysical Research, 108(E12), 24 pp.,
doi: 10.1029/2003JE002077.
73
Maki, J., 2003b. Mars Exploration Rover Coordinate Systems Relevant to Imaging &
Rover Motion Counter. MER Analyst’s Notebook Producted by NASA’s PDS
Geosciences Node. Accessed July 30, 2012,
http://an.rsl.wustl.edu/mer/merbrowser/browserFr.aspx?tab=res&m=MERB
Malin, M. C., G. E. Danielson, A. P. Ingersoll, et al., 1992. Mars Observer Camera.
Journal of Geophysical Research, 97(E5): 7699-7718.
McEwen, A. S., E. M. Eliason, J. W. Bergstrom, et al., 2007. Mars Reconnaissance
Orbiter’s High Resolution Imaging Science Experiment (HiRISE). Journal of
Geophysical Research, 112(E05S02), 40 pp., doi: 10.1029/2005JE002605.
McEwen, A. S., M. E. Banks, N. Baugh, et al., 2010. The High Resolution Imaging
Science Experiment (HiRISE) during MRO’s Primary Science Phase (PSP).
Icarus 205 (2010): 2-37.
NASA, 2010. Microsoft and NASA Bring Mars Down to Earth Through the
WorldWide Telescope. NASA. Accessed July 29, 2012,
http://www.nasa.gov/topics/nasalife/features/microsoft_ww_telescope.html.
Parker, T., M. Malin, M. Golombek, et al., 2004. Localization, localization,
localization, Report, 35th Lunar and Planetary Science Conference, League
City, Texas, March 15-19, 2 pp.
Rodehorst, V. and A. Koschan, 2006.Comparison and Evaluation of Feature Point
Detectors. In Proceedings of 5th International Symposium Turkish-German
Joint Geodetic Days, Technical University of Berlin, Germany, March, 2006;
ISBN 3-9809030-4-4.
Seidelmann, P. K., V. K. Abalakin, M. Bursa, et al., 2002. Report of the IAU/IAG
Working Group on Cartographic Coordinates and Rotational Elements of the
Planets and Satellites: 2000. Celestial Mechanics and Dynamical Astronomy,
74
82(1): 83-110.
Taylor, J., D. K. Lee, and S. Shambayati, 2006. Mars Reconnaissance Orbiter
Telecommunications. Article 12, DESCANSO Design and Performance
Summary Series.
Triggs, B., P. F. McLauchlan, R. I. Hartley, et al., 1999. Bundle Adjustment – A
Modern Synthesis. ICCV 1999 Proceedings of the International Workshop on
Vision Algorithms: Theory and Practice, September 21-22, 1999, pp.
298-372.
Tsai, D., and C. Lin, 2003. Fast Normalized Cross Correlation for Defect Detection.
Pattern Recognition Letters, 24(15): 2625-2631.
Williams, S. H., J. R. Zimbelman, and A. W. Ward, 2002. Large Ripples on Earth and
Mars. Lunar and Planetary Science XXXIII, The Woodlands, TX, March
11-15, Abstract No. 1508, 2 pp.
Xu, F., K. Di, R. Li, et al., 2002. Automatic Feature Registration and DEM Generation
for Martian Surface Mapping. International Achieves of Photogrammetry,
Remote Sensing and Spatial Information Sciences, Vol. 34, Part 2, Comm. II,
Xi’an, Aug. 20-23, 2002, pp. 549-554.
Xu, F., 2004. Automation in Mars Landing-Site Mapping and Rover Localization.
ISPRS XXth Congress 2004, Istanbul, Turkey, July 12-23, 2004.
Zuber, M. T., D. E. Smith, S. C. Solomon, et al., 1992. The Mars Observer Laser
Altimeter Investigation. Journal of Geophysical Research, 97(E5):
7781-7797.
75