Proceedings of Australasian Conference on Robotics and Automation, 7-9 Dec 2011, Monash University, Melbourne Australia. Image-Based Visual Servoing for the Super-Orbital Re-Entry of Hayabusa Spacecraft Razmi Khan* University of Queensland, Brisbane, Queensland, 4072, Australia [email protected] Troy Eichmann† University of Queensland, Brisbane, Queensland, 4072, Australia [email protected] David Buttsworth‡ University of Southern Queensland, Toowoomba, Queensland, 4350, Australia [email protected] Ben Upcroft§ University of Queensland, Brisbane, Queensland, 4072, Australia [email protected] Abstract This paper presents an image-based visual servoing system that was used to track the atmospheric Earth re-entry of Hayabusa. The primary aim of this ground based tracking platform was to record the emission spectrum radiating from the superheated gas of the shock layer and the surface of the heat shield during re- entry. To the author’s knowledge, this is the first time that a visual servoing system has successfully tracked a super-orbital re-entry of a spacecraft and recorded its spectral signature. Furthermore, we improve the system by including a simplified dynamic model for feed- forward control and demonstrate improved Fig. 1. PTU and camera equipment used for the Hayabusa re-entry tracking performance on the International Space Station (ISS). We present comparisons between simulation 1 Introduction and experimental results on different target Hayabusa is an unmanned Japanese spacecraft that was trajectories including tracking results from launched as a scientific mission to collect samples from Hayabusa and ISS. The required performance for asteroid Itokawa [Uesugi 2003] Although many tracking both spacecraft is demanding when difficulties were experienced the ambitious mission combined with a narrow field of view (FOV). achieved many ‘firsts’ and the capsule successfully re- We also briefly discuss the preliminary results entered the Earth at super-orbital velocities. These high obtained from the spectroscopy of the velocities, in the order of 12km/s provided a rare Hayabusa’s heat shield during re-entry. opportunity to experimentally measure radiation emitted * Postgraduate Student, School of Mechanical and Mining Engineering † Postgraduate Student, School of Mathematics and Physics ‡ Professor, Faculty of Engineering and Surveying § Senior Lecturer, School of Mechanical and Mining Engineering Page 1 of 10 Proceedings of Australasian Conference on Robotics and Automation, 7-9 Dec 2011, Monash University, Melbourne Australia. cθ t r θ cθ x z cθ x rθ y cθ y f Fig. 2. The PTU (left) is shown with 2-DOF rotational movement and camera mountings. Figures on the right top and bottom demonstrate the camera coordinate system in terms of which, all modelling will be shown and a typical target position on the image plane with an error condition respectively. from the shock-compressed air in front of the spacecraft red transmission grating attached before the camera lens. and from the capsule heat shield under true flight The equipment was spatially and spectrally calibrated to conditions. record the desired range of wavelengths on the camera Previous attempts to acquire re-entry radiation CCD. signatures on the Stardust [Winter et al, 2007], [McHarg The tracking of the Hayabusa spacecraft re-entry et al., 2008], [Jenniskens, 2008] and Genesis [Jenniskens was automated by utilizing visual feedback control. This et al., 2006] spacecraft capsules and the heat distribution method was found to be sufficient to track and obtain on the Space Shuttle [Horvath et al., 2010] were carried spectral information during the re-entry using low out by manually pointing the cameras at the surface of the magnification wide FOV cameras which provided limited re-entering vehicle. During such observations, acquiring spatial resolution of the SRC. To obtain temperature of and tracking the target can be difficult because of human the SRC shock layer as a function of surface area, a much errors. Loss of valuable emissions data for all or part of higher spatial resolution would be required resulting in the re-entry event has occurred on some of these the implementation of large focal length lens. This observations due to reliance on manual tracking. To effectively reduces the observed FOV of the cameras and address this problem, we take an automated image-based requires much smaller margins of error in tracking the approach that robustly maintains the alignment of the object. To meet this demanding requirement we improve instruments with the target. the control by adding a simplified dynamic model of the We implemented a visual feedback control system. strategy, based on concepts described by The model enables feed-forward control which is Papanikolopoulos [Papanikolopoulos et al., 1993] and implemented post re-entry and tested on various target Hutchinson [Hutchinson et al, 1996] to track the re-entry. trajectories based on work undertaken by Corke et al. The approach required that the target be initially [Corke et al., 1996]. This control strategy is one of only a identified manually in an image sequence which is small number of applications in which feed-forward is subsequently automatically tracked using a 2 axis pan and implemented. Song [Song et al., 2009] demonstrates feed- tilt unit The image based tracking is done by forward compensation for high performance tracking on a implementing Lucas Kanade pyramidal optical flow six DOF robotic arm. Bernardino [Bernardino et al., 1999] approach [Bouguet, 1981]. Image intensity contrasts are demonstrates a binocular visual servoing framework that used to visually track the front edge of the spacecraft’s uses both kinematic and dynamic control strategies for sample return capsule (SRC). This allowed the SRC to be high performance tracking of objects. reliably tracked without the possibility of tracking We undertake an additional field experiment deviations on the plasma emissions tail or the fragments which involved the tracking of the ISS post the Hayabusa of the main spacecraft* debris. re-entry to show the improvements in the tracking To obtain the spectrum of the radiation emitted performance using feed-forward as compared to feedback from the surface of the re-entering vehicle, a spectrometer alone. The use of a narrow FOV camera lens resulted in a co-aligned to the tracking camera was implemented. A challenging target trajectory across the image plane with a basic spectrometer was set up which includes a near infra- relative observed velocity between the optics and the ISS similar to a super-orbital re-entry observation. The results from both visual kinematic and dynamic control strategies * Spacecraft or main spacecraft here refers to the space probe are discussed. following separation of the SRC Page 2 of 10 Proceedings of Australasian Conference on Robotics and Automation, 7-9 Dec 2011, Monash University, Melbourne Australia. The layout of the paper is as follows; a brief where u, v are the velocities of the image feature with outline of the equipment setup and formation of an Image target coordinates on the image plane u and v. Tx, Ty, Tz Jacobian will be discussed in Section 2. In Sections 3 and ω ω ω 4 we model the feedback and feedforward control and x , y , z are translational and rotational velocities of strategies and present the tracking performance results for the end-effector with respect to the camera frame in a different target trajectories. Section 5 details the fixed camera system respectively. f is the focal length of experiments conducted in simulation and with the actual the camera lens in pixels. system. Section 6 describes the visual tracking For a 2 DOF pure rotational motion in the pitch performance and emission spectra acquired during the and yaw axes of the PTU the above matrix can be reduced Hayabusa spacecraft re-entry. The paper concludes with a to the following equations, discussion on the use of automated vision based tracking 2 2 f2+ v 2 f+ u and techniques for future hypersonic Earth re-entries. u = ω y v = ω x f f In forming the equations above, a few 2 Visual Servoing Architecture approximations need to be addressed. The off-axis In this section we introduce the equipment and setup for mounting of the tracking camera leads to the introduction the re-entry. We also present the image Jacobian and the of the two offset terms Xo and Yo when transforming the approximations applied to it for this particular application. PTU coordinate system to the camera coordinates. The yawing and pitching motions about the y and x axes 2.1 Equipment setup introduce PTU translation velocities in the x-z and y-z planes respectively. Assuming that the coordinates of the The actuation unit is a Directed Perception PTU-47-17 object being tracked are cO =[,,]x y ∞ and the offset high speed 2 axes pan and tilt unit. This unit hosts two identical ‘Flea 2’ black and white Point Grey cameras terms Xo, Yo ≈ 0, the translational motion induced can be capable of running at 30Hz. The tracking camera with a approximated as zero. 6mm lens and the spectrometer with a 25mm lens combined with a 300l/mm grating are mounted on the pan 3 Visual Feedback Control and tilt unit (PTU) payload bracket as shown in Figure 2. The spectrometer setup is briefly discussed in Section 6.3. In this section we present the model of the system that A typical desktop CPU interfaces between the camera and was used to implement both the visual feedback and PTU using an IEEE 1394b standard interface and serial dynamic feed-forward control strategies as shown in communication respectively. Two additional hard drive Figures 3 and 6 respectively. For simplicity, all system disks were included in the host computer to save modelling and analysis will assume a single axis system. simultaneous streaming of images from both cameras. All Target movement is assumed only about the y-axis that vision processing and PTU movement tasks were handled corresponds to the horizontal line on the u-v image plane.