On-Orbit Testing of Target-less TriDAR 3D Rendezvous and Docking Sensor Stephane Ruel, Tim Luu, Andrew Berube* *Neptec Design Group, Canada e-mail: [email protected] Abstract shape of the target object and calculate its position and orientation. The innovative computer vision algorithms TriDAR is a vision system developed by Neptec developed by Neptec allow this process to be performed for autonomous rendezvous and docking in space. The in real-time on TriDAR’s onboard flight computer, while system is entirely model based and does not require the achieving the necessary robustness and reliability use of cooperative markers, such as retro-reflectors, on expected for mission critical operations [4]. TriDAR’s the target spacecraft. The system integrates a TriDAR onboard thermal imager is used to provide bearing and 3D active sensor, a thermal imager, and embedded model range information beyond the reach of the active 3D based tracking software. It provides full 6 degree of sensor. freedom (6 DOF) relative pose information in real-time. Fast data acquisition has been achieved by In partnership with the Canadian Space Agency implementing a smart scanning strategy referred to as (CSA) and NASA, Neptec has space qualified the “More Information Less Data” (MILD). Using this TriDAR proximity operations vision system and strategy, only the data that will be directly used to integrated it onboard the Space Shuttle Discovery for perform the pose estimation (~1000 3D measurements) is two test flights on the STS-128 and STS-131 missions to acquired by the sensor [5]. As opposed to reflector the International Space Station (ISS). The objectives of based solutions, all the data acquired by the sensor is the test flights were to demonstrate the system’s ability to used to compute the relative pose. This results in better perform autonomous acquisition and tracking of a known sensor noise averaging and a more stable and accurate target in space and provide real-time relative navigation relative pose measurement. It also has the advantage of cues. Autonomous operations involved automatic relaxing the requirements on sensor data acquisition acquisition of the ISS, real-time tracking as well as speed as well as onboard processing, memory and detection and recovery from system malfunctions and/or bandwidth. loss of tracking. Neptec has also developed the necessary tools and This paper presents an overview of the TriDAR processes to configure, test and qualify the system for system as well as results from the first two test flights of various targets and missions. Neptec’s TriDAR the technology on board the Space Shuttle. simulator can produce highly realistic synthetic data that allow testing of the flight system under scenarios that 1 Introduction would be difficult to test on the ground. These include large targets, fly around and complex trajectories, long Traditionally, relative navigation vision systems for range, different material properties, and failed states. rendezvous and docking in space have relied on This paper presents an overview of TriDAR’s first cooperative targets such as retro-reflectors installed on two test flights on STS-128 and STS-131. Section 2 the spacecraft to capture [1][2]. While reliable, target introduces the TriDAR system. Section 3 summarizes based systems have operational limitations, as targets the mission concept and objectives. Section 4 presents must be installed on the targeted payload. This is not a summary of results obtained during ISS rendezvous always practical or even possible [3]. This approach is and docking operations and Section 5 presents results also inefficient, especially when active sensors are from undocking and fly around operations. required to operate at long distances and require a wide field of view. Cooperative target based approaches 2 TriDAR Relative Navigation Vision typically need to acquire and process data over the entire System field of view to find its targets but only use a few 3D coordinates to calculate the relative pose. Neptec’s TriDAR vision system was designed to Neptec’s TriDAR (triangulation + LIDAR) sensor provide full 6 degree of freedom (DOF) relative state results from over 6 years of research and development vectors in real-time without requiring cooperative funded by the Canadian Space Agency (CSA) and NASA. TriDAR brings a novel, lighting immune targets. To achieve this, the system uses a laser based capability to space vision systems and provides the 3D sensor and a thermal imager. TriDAR’s proprietary software uses geometric information contained in ability to automatically rendezvous and dock with successive 3D point clouds to match against the known vehicles that were not designed for servicing. The vision i-SAIRAS 2010 August 29-September 1, 2010, Sapporo, Japan 15 system combines an active 3D sensor, a thermal imager, Materials in space tend to be very dark, white or and Neptec’s model-based tracking software. Using only very shiny. Being able to image these surfaces over a knowledge about the target spacecraft’s geometry and wide operating range is critical for operations in space. 3D data acquired from the sensor, the system can The combination of auto-synchronous triangulation and automatically acquire and track an object and output the LIDAR gives TriDAR a very wide dynamic range that 6 DOF relative pose directly. Innovative computer vision allows the system to image such surfaces at the same algorithms developed in house at Neptec allow this time without having to adjust sensor parameters. This process to perform in real-time on TriDAR’s embedded makes the TriDAR technology ideal for multiple space flight computer. [4][5] Initialization of the tracking applications such as: rendezvous & docking, planetary process is performed by automatic acquisition algorithms navigation and landing, site inspection, material also developed at Neptec [6][7]. classification, and vehicle inspection. The TriDAR active three dimensional sensor (Figure 1) is a hybrid scanner that combines 3 Mission Overview auto-synchronous laser triangulation technology with Time of Flight ranging (LIDAR) in a single set of TriDAR was first tested in space on board Space scanning optics. This configuration takes advantage of Shuttle Discovery during the STS-128 mission to the the complementary nature of these two ranging International Space Station (ISS). The objective of this technologies to provide 3D data at both short and long first test flight was to demonstrate the capability of the range without compromising performance [8]. The laser system to autonomously acquire and track the ISS triangulation subsystem is largely based on the Laser without using retro-reflectors or other cooperative Camera System (LCS) used to inspect the Space targets. To reproduce the conditions of an unmanned Shuttle’s thermal protection system following each docking, the system was configured to perform the entire launch [9]. By combining the two ranging subsystem’s mission autonomously, automatically acquiring the ISS optical paths, the TriDAR can provide the functionalities and producing 6 degree of freedom state vectors in of two 3D sensors into a single package. Both the real-time. The system was designed to self-monitor its triangulation and LIDAR ranging capabilities can be tracking solution and was capable of automatically used during short range operations (<20m) thus reacquiring the ISS if tracking was lost. providing an extra level of fault tolerance in the system TriDAR was installed in the Space Shuttle’s for the last critical part of docking. The two subsystems payload bay on the Orbiter Docking System (ODS) next share the same control and processing electronics thus to the Trajectory Control System (TCS) rendezvous providing further savings (mass, power, size etc.) sensor (Figure 2). compared to using two separate 3D sensors to cover the same operational range. A thermal imager is also included to allow the system to provide bearing and range information for operations well beyond LIDAR operating ranges. Figure 2: TriDAR flight unit installed in the Space Figure 1: TriDAR sensor flight hardware Shuttle payload bay 16 The sensor was connected to a laptop computer real-time during undocking and fly around of the Space inside Discovery’s crew cabin. The laptop was used to Shuttle. This provided a scenario similar to that of a command the system, log the data and to display the tumbling target that would be encountered during tracking solution and other system statuses in real-time servicing of non-cooperative spacecrafts. New to the crew. All data processing and state vector objectives also included long range acquisition from calculations were performed locally on the sensor in TriDAR’s onboard thermal imager. real-time. The specific configuration of the TriDAR laser 4 Rendezvous and Docking Operations hardware used for these Space Shuttle missions was selected to meet NASA eye safety requirements and For both missions, TriDAR was activated during ensure safe operations for both Shuttle and Station crews. rendezvous operations on Flight Day 3. TriDAR’s The ISS crew performs inspection of the Space Shuttle thermal imager detected the ISS at a range of 43.4km heat shield during the back flip (aka RPM) maneuver immediately after activation. The thermal imager performed before docking (~600’ away along the detected the ISS in both night and day conditions and R-BAR). This inspection is performed using high acquired over 3000 high quality images of the ISS magnification binoculars and reflex cameras with throughout the rendezvous and docking operations. telephoto lenses. This imposes serious limitations on Figure 3 presents sample images at both long and short laser based systems in order to ensure safety of the range with color mapped to temperature. crews. For TriDAR, this resulted in a system configured with less than 10% of the laser power nominally available in the system. This significantly reduced the operational range of the system to be flight tested. This configuration was nevertheless sufficient to achieve mission objectives and demonstrate target-less tracking on-orbit.