Targeting and Localization for Mars Rover Operations

Mark W. Powell, Thomas Crockett, Jason M. Fox, Joseph C. Joswig, Jeffrey S. Norris, Kenneth J. Rabe, Jet Propulsion Laboratory, Michael McCurdy, NASA Ames Research Center, Guy Pyrzak, San Jose State University Foundation, NASA Ames Research Center, M/S 262-4

Abstract— In this work we discuss how the quality of system can observe the Sun position, it combines that with its localization knowledge impacts the remote operation of rovers knowledge of the expected position of the Sun at a given time on the surface of Mars. We look at the techniques of localization of sol (Martian day) to more accurately refine its estimate of estimation used in the Mars Pathfinder and Mars Exploration vehicle attitude. Visual odometry [2] is a localization Rover missions. We examine the motivation behind the modes technique that has also been proven effective at producing of targeting for different types of activities, such as navigation, remote science, and in situ science. We discuss the virtues and more accurate position and attitude knowledge than shortcomings of existing approaches and new improvements in previously used methods, although it requires significant the latest operations tools used to support the Mars Exploration additional onboard computing time. Rover missions and rover technology development tasks at the Quality of localization knowledge has a direct impact on Jet Propulsion Laboratory. We conclude with future directions how rover activities are planned and refined into commands we plan to explore in improving the localization knowledge for execution on the vehicle. Stereo image pairs returned from available for operations and more effective targeting of rovers and their instrument payloads. the vehicle are correlated into 3D XYZ maps of the visible local terrain around the spacecraft. These XYZ maps are collections of accurate 3D position data used for pointing I. INTRODUCTION science instruments at very specific points of interest, or HIS document describes recent advances in the state of targets. However, as the rover traverses on the surface and Tp ractice of targeting activities in the operation of Mars accumulates error in localization knowledge, these targets rovers. Teleoperation of robotic rovers over a very long become less and less relevant. It is therefore necessary to plan distance such as from Earth to Mars where one-way data activities that refer to any specific targets prior to any driving transfer times vary from 6 to 20 minutes is a challenging activity. The only exceptions to this rule are when a specific problem. Due to this delay, conventional real-time target is not visible to a particular instrument from the teleoperation techniques are not applicable and instead we vehicle’s current position, requiring a drive to bring the target employ onboard vehicle autonomy. Control software on the into the field of view of that instrument. In this case, there are spacecraft can use images from the stereo camera pairs to two mitigating strategies available: either articulating the track landmarks, identify obstacles, and compute a safe instrument in a rastering fashion to increase its effective field traverse path to a location of scientific interest on the surface. of view to compensate for localization error, or acquiring new However, in order to perform these tasks effectively the stereo imagery from the new position, sending it back to vehicle control system must have knowledge of localization Earth and waiting until the next planning cycle (the next sol) (the position and orientation of the vehicle). to command the science accurately. Both of these strategies Several approaches have been used on Mars surface have a cost in onboard execution time and bandwidth, and so vehicles to make localization estimates. During the Mars great care must be taken in operational planning to avoid this Pathfinder mission (MPF) the Sojourner rover used a heading cost whenever possible. sensor and wheel encoder counts to estimate changes in In the next section we discuss how spacecraft activities are position and localization [1]. The Mars Exploration Rovers targeted differently depending on their aims. We also (MER) also uses inertia measurement unit (IMU) and describe how a large group of collaborating scientists who are mobility motor encoder information to estimate position and geographically distributed can effectively define targets for orientation. Further, the Pancam instrument is equipped with operations planning. We then discuss recent improvements Sun filters, and the control software can perform an we have made in the operational processing of defining and autonomous Sun-finding activity. When the vehicle control using targets. Finally, we conclude with a discussion of promising future directions in localization and targeting.

The work described in this paper was carried out at the Jet Propulsion II. TARGETING ROVER ACTIVITIES Laboratory, California Institute of Technology and funded by the Mars Technology Program, the JPL Research and Technology Development Program, the Mars Exploration Rover project and the Mars Science A. Coordinate Frame Types Laboratory project. The conventions for coordinate frame descriptions that capture the history of rover positions on the surface come in

several types. The frame that describes the rover-centric view 0. As previous discussed, the Local Level frames all share a of the world is called Rover frame. Rover frame is defined as common orientation of X-north, Y-east, Z-nadir while the X-forward, Y-starboard, Z-down with an origin defined at a Rover frame are relative to the orientation of the vehicle at fixed position on the rover body, according to the SAE J670E that position. standard for vehicle coordinate frame. Although not N required, it is advantageous to choose an origin point on a holonomic vehicle (a vehicle that is turn-in-place capable) (4, 0) W E N such that the origin point remains fixed in space as the vehicle (4, 1) turns in place. A related frame to Rover frame is Local Level N S frame, which shares the same origin as Rover frame, but (4, 2) rotates the axes to an orientation of X-north, Y-east, Z-nadir. N This rotation is approximated by the estimated pitch, roll, and yaw of the vehicle, and as such it is subject to error in attitude estimation. Accurate vehicle position estimation after a traverse is well-known as a challenging problem, and because of this a third frame convention was defined for surface operations to compensate for the problem. In the absence of Fig. 1. Coordinate frames used to represent the position localization error, the naïve approach of defining a single and attitude of a 6-wheeled rover. Local Level frames (in world reference frame on the surface of the planet and solid black) are surface-relative and Rover frames (in referencing all subsequent locations on the surface to that dotted red) are vehicle relative. The labels above each frame would be reasonable. In practice where this error does frame denote its Site and Drive indices, respectively, which uniquely identify each frame. exist, this is not a viable approach. To wit, for a relatively

distant target—one which represents a medium-to-long term As a labeling mechanism for the history of frames of the goal for navigation—the originally specified XYZ coordinate spacecraft, the control system maintains a Rover Motion of the target becomes increasingly inaccurate the more the Counter. This counter has several integer index values that vehicle traverses on the surface and accumulates more and increase as the vehicle moves. The Drive index is increased more localization error. It is therefore regularly necessary by one every time the vehicle performs a part of a drive, such disregard the history of all vehicle position information to as a turn-in-place or a single segment of an automated date and define a new “world frame”, called Site frame. This traverse with onboard path planning and obstacle avoidance. Site frame then becomes the surface-relative (as opposed to There is also a Site index in the counter which is incremented rover-relative) frame of reference for all operations planning whenever the operations planners need to declare a new Site and targeting. The definition of a Site frame is identical to that to mitigate localization error. When the Site index increases of a Local Level frame, and indeed it is correct to say that a by one, the Drive index is reset to zero and then increases Site frame is a particular Local Level frame that is denoted as with each new drive segment as before. The Rover Motion also being a Site frame. The process for defining which Counter is a useful tool to correlate the telemetry returned frames are Site frames is managed on the ground. In from the vehicle to the position of the vehicle. It is also operations planning, when it is observed that cumulative possible to use image data returned from the spacecraft to localization error has risen to the degree that targeting refine the localization knowledge of the vehicle during accuracy may suffer, a new Site is defined. Another way to operations planning since the frames and images are think of a Site is as a local surface region with the vehicle as correlated by the counter. We will discuss this possibility its center. New Sites may or may not intersect with previous further in Section III.B.1). Sites, depending on how much localization error the rover accumulated during a traverse, which itself depends on B. Navigation Activities factors such as wheel slippage on the soil. Navigating rovers on Mars involves two modes of driving: We can connect all of the Site frames, Local Level frames, commanded and semi-autonomous. Commanded driving is and Rover frames from the landing site of the spacecraft all used when there are enough images available from the the way to the frame at the most recent vehicle position in a spacecraft to assess the topography of the terrain to determine chain. This chain helps to visualize the relationship between a safe path by inspection during operations planning. If a the various types of frames that we define that are of rover driver who is operating the spacecraft determines that a operational relevance. Figure 1 illustrates these different particular path is safe to drive, she may choose to command a types of coordinate frames and their inter-relations. Local drive directly by programming a sequence of drive arcs and level frames are shown in solid black and Rover frames are in turns-in-place to drive the vehicle to a new position. This dotted red. The black dashed line connects the pre-drive (4,0), mode of driving is advantageous in that it requires minimal mid-drive (4,1), and end-of-drive (4,2) positions. The Local computing time from the vehicle’s onboard controller and Level frame (4,0) is a Site frame, since its drive index value is power consumption, but has the disadvantage that it can only

be used when an operator has sufficient information to executed before a drive, it is generally targeted in Rover determine a safe path. If the terrain safety cannot be frame or the current Local Level frame with targets specified determined by inspection, the rover driver may command a in the latest image data returned from the spacecraft at the semi-autonomous drive to a waypoint (drive target), which time. If imaging must occur after a drive, it is targeted in Site employs onboard path planning, obstacle avoidance, and frame since as previously noted the knowledge of the Rover localization estimation to navigate to the target. For frame before the drive is lost but the knowledge of Site frame commanded driving, waypoints are generally not used, since is retained. the drive commands are simulated on terrain models derived D. In-situ Science Activities from spacecraft imagery to ensure their safety and effectiveness. Targets are always used for semi-autonomous Targeting in-situ science instrumentation calls for the drives as goal specification for the onboard navigation highest positioning accuracy of all of the types of activities control system. Since it is often the case that a planning cycle we have discussed. In-situ instruments are mounted on a may contain a combination of commanded drives followed by robotic arm or manipulator, generally at its outermost end or semi-autonomous drives, targets for semi-autonomous drives end effector. Depending on the instrument, to obtain science must be specified in Local Level frame or Site frame. In data the manipulator must either place either in contact with practice, the Mars Exploration Rovers only retain knowledge or very close to a particular surface. For the in-situ of the most recent Site frame, Local Level frame, and Rover instruments on the Mars Exploration Rover, targeting frame. Thus, after any drive command is executed, targets accuracy of 0.5 cm is required for safe manipulator operation that are specified in Rover frame are irrelevant. As a result, and effective science data acquisition. To achieve this, in-situ drive commands use waypoints designated in Site frame. activities are conventionally commanded prior to any driving and on targets defined in Rover frame. C. Remote Science Activities Activities that point remote science instruments such as III. TARGETING PARADIGMS mast-mounted high resolution cameras and spectrometers are targeted in a variety of ways. Many articulated masts used on A. Improving Localization in Operations rovers such as the Mars Exploration Rovers have two degrees As previously discussed, the onboard control system on the of freedom to pan and tilt the head of the mast and thus point Mars Exploration Rover retains knowledge of only the most the science instruments, so let us assume this is the case for recent Rover frame, Local Level frame, and Site frame. This this discussion. For targeting a body of air, such as is done strategy has the benefit of lowering the complexity of the when measuring the amount of dust in the atmosphere, it is onboard localization and navigation control system. Even in often sufficiently accurate to point the mast at a particular the presence of localization error, activities can be accurately azimuth (pan angle) and elevation (tilt angle). This mode of targeted when image data from the vehicle’s current position targeting is called pointing in azimuth and elevation or Az/El, is available. which is the simplest method of targeting remote science. Operationally, localization error creates a number of Az/El targeting is also used in other commonly used challenges. For instance, consider a multi-sol campaign to activities, such as acquiring a new set of stereo navigation map a particular section of crater wall that requires a series of images after a long traverse into new terrain or very distant drives and image acquisitions. The desired end result is to features such as hills that lay far out toward the horizon. For construct an integrated map of the wall, bringing together remote science surface features that are relatively close to the image data taken from different vehicle positions as it moves vehicle, such as within 20 meters for example, XYZ targeting from one end of the wall to the other. In order to register each is required for greatest accuracy. Such accuracy is needed to adjacent XYZ map with the next, it is necessary to correct for capture a specific, small location such as on a small rock or a localization error on the ground. Once localization error is specific region of interest such as an interesting segment of removed, the entire wall and its collection of XYZ maps can crater rim or wall. High target accuracy can also result in be combined into one integrated map in one common greater conservation of communication bandwidth for coordinate frame of reference. However, the vehicle’s returning science data to Earth. If targeting is sufficiently localization knowledge still contains the errors, so it must be accurate, images or sets of images can be subframed or updated. Further, each image product that was produced by cropped to a fraction of their normal size to return only the the telemetry processing pipeline that supports operations region of interest. Subframing is conventionally used to must be updated to store the improved localization capture images of the calibration target, which is mounted on information in its metadata. One might just as well say why the rover deck at a precisely known position, and it is also bother making the updates, since the goal of mapping the wall used whenever possible for imaging small targets on the was achieved by computing better localization on the ground, surface to reduce data volume. Because of localization error, removing the errors, and building an integrated map. But image capture operations are planned to execute before what if we then want to drive the rover back to a specific driving whenever possible. When targeted imaging is target on the map? If the vehicle is updated with the new localization information and the image products for

operations used by the planning tools are also updated, then requirement to acquire and downlink new imagery for planning the route back to the particular target on the wall is navigation when existing data is sufficient to the task. straightforward, but without that it would require multiple To take full advantage of localization updates during the sols to execute and new navigation imagery of the ground that operations planning process, operations planning tools need was already covered in order to return. to be made more flexible to adapt to localization updates. Not all of the tools used for Mars Exploration Rover operations B. Improving Targeting have sufficient flexibility to handle this sort of localization 1) Updating Localization in Operations update effectively. Enabling localization updates requires a When the vehicle traverses over a surface that is sloped or coordinated effort across telemetry processing tools, has high slippage, the onboard localization estimation may operations planning tools, and onboard control system. have significant error. Although it is possible to use Telemetry processing tools produce image product files and techniques like visual odometry to reduce the error onboard, other products for operational use that include localization onboard computing time may not be available. Thus, it is information in their metadata. It is this metadata that is parsed sometimes necessary to correct localization errors manually and used in operations planning tools used for targeting after spacecraft images are transmitted to Earth. This manual activities. If the metadata in product files was updated with process involves correlating points on the surface that appear better localization, the operations tools could re-parse the in stereo images taken before and after the drive. If the same information, and the internal representation of the coordinate point appears in both the pre-drive and post-drive image frames of the vehicle could be updated to reflect the new pairs, then the XYZ coordinate of the point should be information. identical—any difference is due in part to localization and in However, operations tools publish targets into shared part to errors in identifying one particular surface point documents and databases where operators read them and use accurately from the two different points of view (pre-drive them for a number of purposes. One common scenario is that and post-drive). To reduce the latter error, usually 3 or more a scientist may create a target that designates the desired drive points are manually correlated and the Euclidean distance position for the next day’s operations. The rover driver who is between them is computed. If time permits, many point-pair responsible for commanding a safe drive to that location distances can be compared and the median of their distances needs to run lengthy simulations to evaluate alternatives and taken as an accurate estimate of localization error that is ensure a safe drive within time and power constraints, so she relatively unaffected by outliers. In practice, an estimate sees the target soon after it is designated by the scientists, so using as few as 3 points is computed manually. There are she begins her work, using a tool to plan a drive that reads the many studies in machine vision that address the problem of target that was published by the scientist. Meanwhile, a correlating images taken from a mobile robot platform as it mobility engineer has produced a localization update. Let us moves from one place to another such as [1]. What makes this now assume that the data products have been updated to problem particularly challenging in this case is to apply a include the improved localization. The target that was method that is general enough to take advantage of whatever originally designated by the scientist has coordinates in Site images are available, which may be some heterogeneous frame as discussed in Section II.B, but the rover’s current combination of relatively low-resolution Hazard Avoidance position relative to that Site frame is now updated. As a result, Camera images and medium resolution Navcam images and the position of the target, the position of the rover, and the ideally be able to find correlations automatically. However, XYZ map product—in which 3D coordinates are stored in although the method must be general, it must also be fast Site frame—must now all be updated. This issue becomes enough to produce accurate estimates in no more than several more significant if, as is often the case, multiple candidate minutes to meet the time constraints of rover operations targets are designated for navigation to compare alternative planning. drives, as well as targets that are designated for in situ science 2) Impact of Localization Updates and remote science. Indeed, it is not uncommon for a group of It is arguable that having the additional flexibility to scientists on any given day to have designated twenty, thirty, respond to localization updates during an operations planning or more targets for use in planning. cycle would be a worthwhile goal. Of course, it also is The complexity of localization updates could be reduced arguable that the current localization strategy used in significantly for operations products and planning tools by operations is sufficient, although it is the authors’ opinion changing the convention of using Site frame to Rover frame. that the added flexibility would add significant value in terms If XYZ map products used Rover frame to store their 3D of spacecraft resource usage, quantity, and quality of science coordinates, they would not need to have their content data that would be worth the cost to add capability to updated since the 3D positions of the points on the surface do operations tools and onboard software. To wit, if localization not change after localization updates if they are could be corrected for accuracy shortly after telemetry is rover-relative. To use this information to create targets in Site received and then factored into operational data products, frame which is required by the vehicle control system, we can then operations planners could target activities in older image easily transform the Rover frame target coordinates to Site products as well as new ones. Further, this relaxes the

frame when the command sequence to be executed on the which the vehicle will begin a command cycle. For any given vehicle is generated. Using Rover frame is used as the native sol, the planning position is the final position of the vehicle of representation for all targeting in operations planning is by far the previous sol, which is the same as the vehicle’s position at the approach of least complexity because the number the beginning of the sol when it begins to execute its information updates needed is minimal. The only information command sequence. It is important to use the final position of updates that are needed to synchronize all representations of the vehicle as the planning position, since the image products 3D coordinates are coordinate frame definitions. In other taken from that position are the most accurate for defining words, when there is a localization update, only the origin and targets. We denote targets that are created from data acquired orientation of Rover frame axes need to be updated. Since at current planning position as “safe” and any targets that are every product and target are defined natively in Rover frame, defined in other data as “unsafe”. Safe targets will be very none of the information in the product or target need to be accurate since the vehicle’s onboard representation of the updated. Only the definition of where the Rover frame is Rover frame is consistent with the Rover frame of the image oriented with respect to Site frame needs be updated to reflect products, regardless of how much localization error has been the new localization. In order to synchronize the localization accumulated. Often, the vehicle will remain stationary for update with the vehicle’s localization knowledge, the two or more days, such as when a lengthy in-situ science plan onboard control system’s knowledge of its current position is executing, or when a lengthy mosaic of images is being and attitude with respect to the Site (in other words the acquired. In these situations, the planning position will onboard notion of its current Rover frame) must be updated at remain the same for several sols, and in Maestro it is the beginning of the command sequence so that the target straightforward to reuse targets for this entire duration. coordinates in the command sequence are accurate. When the rover does drive to a new location, there is a new planning position for planning the next sol, and targets from C. Temporal vs. Spatial Persistence the previous planning position may still be reused by defining The Science Activity Planner (SAP) [3] is an operations a new version of the target. When new images from the planning tool that is used in Mars Exploration Rover post-drive planning position are viewed for the first time, all operations to create targets. The tool allows operators to existing targets (which all were made the previous day or define any number of named targets by selecting a pixel in a earlier) are denoted unsafe. In the user interface, unsafe stereo image from the spacecraft and assigning a name to the targets are drawn as yellow crosshair icons and safe targets underlying XYZ coordinates—placing a label on a point on are green crosshair icons. In new data, an unsafe target is the the surface of Mars. These targets are stored in a database that projection of an XYZ coordinate in older data onto a newly all SAP users share, letting them see each other’s targets as acquired image. Since there is localization error between the they are created. Targets are used not only for designating pre-drive and post-drive vehicle positions, the location of the specific placement and pointing of the vehicle and its payload unsafe target’s projection in the new image will be off of the of science instruments, but also as points for collaborative target by some amount. If the operator wants to reuse the science discussion. Since SAP allows every mission target and update its position to be accurate relative to the participant to view the spacecraft images and shared target new data, she clicks and drags the target to the correct repository, it effectively allows the group to turn these data location, and a new version of the target is created for the into a shared document of Mars that is updated daily with new current planning position. A target can be reused in this way images and information added by scientists. indefinitely, for as long as the terrain location remains visible Recently the SAP tool was upgraded to a new version to the vehicle. In Maestro when the user views images that dubbed “Maestro” [4] that delivered shared targeting support span a number of sols, the versions of targets that are most which addresses two major shortcomings of the original appropriate for (i.e. the version defined at the planning tool’s implementation: target reuse and version tracking. position closest to the vehicle position at the time of image The strategy for target reuse that was implemented in SAP acquisition) is displayed on the image. This strategy of was temporal. It allowed operators to define targets in images version tracking allows the same named location to be from the rover’s current position and reuse them during that specified consistently and accurately for an object on the sol. On the next sol, all existing targets that remained relevant Martian surface even in the presence of localization had to be re-designated. Since targets can remain relevant for estimation error. as long as several weeks, the repetitive process of target re-designation is onerous. D. Targeting in Distributed Operations To address this issue in Maestro, the shared targeting A centralized mission activity planning system is not a strategy uses a reuse strategy that is spatial rather than viable option for long-term missions due to budget and temporal. Instead of creating a target that is reusable only for geographic constraints. The architecture must provide secure, a sol, we instead defining a target that is reusable for a reliable channels for distributing both downlink and uplink planning position. A planning position is defined as a vehicle planning data, including targets. For targets to be a useful position (indicated by a Site and Drive index pair) from collaboration tool, they must be available to every Maestro

user regardless of geographic location. For targets there are three relevant database tables: SAP was originally designed to support distributed mission • The primary table that holds the actual target data operations for MER, but was limited to distribution among all (TARGETS) SAP instances behind the JPL flight operations firewall. • A table for recent changes which relates a Maestro was designed from the ground up to operate in a truly message identifier with a target ID distributed environment [5]. Maestro retains its full (RECENT_CHANGES) functionality regardless of location – the only requirement is • A table containing a running counter to be used as a network connection. To realize this new approach to the message identifier for the next JMS message distributed mission operations, the architecture supporting (MESSAGE_COUNTER) Maestro depends upon a central database. This database is To change a target, a client performs the following actions one of the necessary components for ensuring in a single database transaction: synchronization and accessibility for all Maestro clients. The 1. Save the target modification to the TARGETS central database is only part of the equation, however. To table maintain synchronization without employing an inefficient 2. Retrieve (and then increment) the message polling technique, a messaging service is used. In the case of identifier from the MESSAGE_COUNTER table Maestro, we use Java Message Service (JMS). 3. Insert the message identifier and target ID into a 1) JMS Notification new row of the RECENT_CHANGES table The JMS API is a messaging standard that allows Upon successful completion of the database transaction, application components to create, send, receive, and read the client can broadcast the change using the associated messages. It enables distributed communication that is message identifier. The key to this protocol is the database loosely coupled, reliable, and asynchronous. The JMS transaction. The three steps listed above must be done by a specification does not guarantee a consistent arrival order for single client without any other client interleaving its messages sent from multiple sessions [6]. However, Maestro modifications. For a given attempt by a single client, either all must be able to guarantee that all messages are processed in three steps will complete successfully, or none of them will. the same order they are generated. For example, consider a This invariant guarantees the uniqueness of the message case where a message to delete target X is received before a identifier used for the JMS message and the message to create target X…if messages are not processed in RECENT_CHANGES table. a deterministic fashion, then synchronization between clients Due to uniqueness of the message identifier, clients will be becomes intractable. expecting to receive a sequential list of change broadcasts via To avoid this problem, Maestro implements a simple JMS. If this constraint is violated because a client crashes message ordering layer on top of JMS. This layer's sole after updating the database but before publishing to JMS, concern is the message identifier (an integer value stored in then there will be a sequential list of changed targets in the the message header) which is used to determine global order. database. A client can then ask the database to retrieve the If any message arrives out of order, a timeout mechanism is modified targets associated with the message identifiers that put into place. The layer waits on the expected message for a it missed. There is also the case where a client tries to perform configurable duration. If the message does not arrive, the an update and finds the next message identifier is not the one ordering layer will notify the processing layer of the timeout it was expecting. This case is handled in a similar fashion. and move on to the next available message. Once the messages have been properly ordered, they are IV. FUTURE DIRECTIONS sent to the message processing layer. Message processors register with the messaging ordering layer and receive A. Single-cycle instrument placement notifications for each message under the following Currently Mars rover operations planning often requires 3 conditions: sols to place an in-situ instrument on a target that is a 1. A message has been received in the proper order significant distance (more than 2-3 meters): One sol to and is ready to be processed approach the target, covering most of the distance to the 2. A message has been received out of order target, a second sol to refine the vehicle’s attitude to bring the therefore the ordering layer will begin the timeout target into the work area of the manipulator, and third to phase precisely target the in-situ observation. Single Cycle 3. The ordering layer has timed out waiting for a Instrument Placement (SCIP) technology [7] is being particular message to arrive evaluated for possible use in the 2009 Mars Science For this system to work, the message identifier must be Laboratory mission. SCIP can track a target from its initial both unique and monotonically increasing. In order to designation in high-resolution imagery taken from a distance maintain these invariants, the Maestro client follows a strict and visually track it as the vehicle approaches it protocol when attempting to publish a modification to the autonomously, first using mast camera images and then target database. passing the tracking off to the Hazard Avoidance cameras

(Hazcams) used to map the manipulation workspace. This process can be completed in a one sol instead of three, potentially tripling the amount of science return from a rover. Target designation for SCIP in its current implementation is subtly different from conventional targeting. The system takes as input a target in 2D image coordinates rather than 3D coordinates. Internally, SCIP uses the same images that the operator uses to designate the target image coordinates to compute the 3D position of the target and also performs visual tracking of the target during navigation by taking new images at each navigation step. Sending the XYZ coordinate of the target to track to SCIP may be a slight improvement over image coordinate designation, since stereo correlation may be manually refined to higher accuracy in operations compared to the correlation that can be achieved automatically by the onboard control system. Operations planning may benefit from the products generated by the SCIP process as well: the tracked target locations that are detected by SCIP may be used to create new versions of the original target that can be overlaid onto images to show the vehicle operators where SCIP found the target in each image taken throughout the traverse. The version tracking capability in Maestro could be directly applied to this task.

REFERENCES [1] K. Di, F. Xu, R. Li, L. H. Matthies, C. F. Olson, “High Precision Landing Site Mapping and Rover Localization by Integrated Bundle Adjustment of MPF Surface Images,” in Symposium on Geospatial Theory, Processing and Applications, Ottawa 2002. [2] Y. Cheng, M. Maimone, L. H. Matthies, "Visual Odometry on the Mars Exploration Rovers," IEEE Conference on Systems, Man and Cybernetics, The Big Island, Hawaii, USA, October 2005. [3] J. S. Norris, M. W. Powell, M. A. Vona, P. G. Backes, J. V. Wick., "Mars Exploration Rover Operations with the Science Activity Planner," IEEE International Conference on Robotics and Automation, April 2005. [4] J. S. Norris, M. W. Powell, J. M. Fox, K. J. Rabe, I. Shu, "Science Operations Interfaces for Mars Surface Exploration," 2005 IEEE Conference on Systems, Man, and Cybernetics, October 15-17, Big Island, HI., October 15, 2005. [5] J. M. Fox, J. S. Norris, K. J. Rabe, M. W. Powell, “Advances in Distributed Operations and Mission Activity Planning for Mars Surface Exploration”, accepted to SpaceOps Conference, Rome, Italy, June 2006. [6] Java Message Service Specification, http://java.sun.com/products/jms/docs.html [7] R. Madison, “Improved Target Handoff for Single Cycle Instrument Placement,” in IEEE Aerospace, Big Sky, MT, 2006.