Rob ot Lo calization using a Computer Vision Sextant Fabio Cozman Eric Krotkov Rob otics Institute, Carnegie Mellon University 5000 Forb es Ave., Pittsburgh PA 15213 Abstract of Sun altitude, but the basic ideas could b e applied for any celestial b o dies: satellites, planets, stars. Our primary target is navigation in cloudless atmospheres This paper explores the possibility of using Sun alti- not Earth during long-duration missions. tude for localization of a robot in total ly unknown ter- In this pap er, we describ e a system in which these ritory. A set of Sun altitudes is obtainedbyprocessing ideas were implemented and tested for a demonstra- asequence of time-indexed images of the sky. Each al- tion of feasibility. This system is based on a camera titude constrains the viewer to a circle on the surface with telephoto lens and a digital inclinometer for mea- ofacelestial body, cal led the circle of equal altitude. surement of attitude of camera. Results of real runs A set of circles of equal altitude can be intersectedto are shown in the pap er. yield viewer position. We use this principle to obtain position on Earth. Since altitude measurements are corrupted by noise, a least-square estimate is numer- ical ly calculatedfrom the sequence of altitudes. The 2 Celestial Navigation paper discusses the necessary theory for Sun-based lo- calization, the technical issues of cameracalibration This section summarizes basic concepts used in the and image processing, and presents preliminary results celestial navigation literature. An extensive treatment with real data. is given by Hobbs [3]. We take p osition on Earth to b e represented by a latitude/longitude pair. For calculations, we take 1 Intro duction south latitude as negative zero at the equator and west longitude as negative zero at Greenwich. This work addresses the problem of rob ot lo caliza- Every celestial b o dy can b e pro jected onto the ter- tion: how can a rob ot make precise measurements of restrial surface by considering the line that go es from p osition in totally unknown territory? We fo cus on the center of Earth to the b o dy. The p ointofinter- the use of celestial cues in order to obtain lo caliza- section b etween the terrestrial surface and this line is tion. Techniques for celestial navigation are funda- the geographic position, denoted GP, of the b o dy. The mental for any sea navigator [3]. Celestial information latitude of the GP of a b o dy in the celestial sphere is is also used in spacecraft and satellite applications for denoted declination. attitude control and accurate p ositioning [5]. Yet ce- The Sun maintains an almost constant declination lestial navigation has not b een heavily mentioned in on a given day. Sun declination varies slowly b etween connection to autonomous rovers. This comes as a sur- +23 and 23 degrees, during a year. Sun declination prise since planetary rovers will need reliable means of is tabulated in the Astronomical Almanac [1] for every checking dead reckoning data. dayofagiven year. We pursue an automatic solution of the lo caliza- The longitude of the GP of the Sun, contrary to its tion problem. Instead of using a sextant to obtain the declination, changes continuously during a day since altitude of a celestial b o dy,we use a camera and a dig- the Sun moves from East to West. We can recover ital inclinometer. The measurements are pro cessed by longitude of the GP of the Sun if we know Greenwich a non-linear least-squares optimization that replaces Mean Time GMT, since time is prop ortional to the the tables used by mariners [3]. We use measurements motion of the Sun during the day. GMT is a mean value; there are drifts and variations during the year. This research is supp orted in part by NASA under Grant For this reason, a small correction must b e subtracted NAGW-1175. Fabio Cozman is supp orted under a scholarship from GMT in order to obtain the longitude of the GP from CNPq, Brazil. of the Sun. The correction is called equation of time and it is tabulated in the Astronomical Almanac for North Pole every dayofagiven year. Once we obtain declination and equation of time, we can calculate the GP of the Sun for every given t. instan Earth A measurement of Sun altitude constrains the ob- server to lie on a circle on the terrestrial surface, called the circle of equal altitude. As the name suggests, from any p oint on the circle, the celestial b o dy will app ear to have the same altitude. A circle of equal altitude y a measurement of Sun altitude and the is de ned b GP2 GP1 exact time of measurement. Two circles of equal alti- tude, arising from temp orally distinct measurements, constrain the observer's p osition to two p oints the p oints of intersection of the circles. Three circles of er's p osition to one equal altitude constrain the observ Circles of Equal Altitude p oint. Four or more measurements of altitude over- constrain the observer's p osition and can b e used to check or correct previous measurements. Figure 1 illustrates a situation where two stars are measured. Each measurement gives rise to one circle of equal altitude, so the two measurements give rise to two circles of equal altitude, whichintersect at two Figure 1: Two Measurements of Altitude: GPs and p oints, thus yielding two p ossible solutions. In general Circles of Equal Altitude the solutions are very far from each other and one can b e discarded based on other information, for example, from dead reckoning. The camera, attached to a horizontal adjustable Our system solves the lo calization problem in the platform, is aimed at the Sun. A series of neutral den- following four steps: sity lters is used to eliminate excessive solar bright- ness. In order to reduce the in uence of camera cali- 1. Take a sequence of Sun images. bration, a small eld of view is desired: with a large eld of view, it would b e necessary to know precisely 2. Obtain Sun altitude and the GP of the Sun for the center of the image and the fo cal length. Tele- each image in the sequence. photo lens fo cal length 300mm are used to pro duce 3. Use Sun altitude and the GP of the Sun to gen- a small eld of view Figure 2 depicts the hardware. erate a set of circles of equal altitude. Sun images are senttoaworkstation, which p erforms the following op erations: 4. Intersect the circles of equal altitude numerically in a least-squares pro cedure. 1. A histogram is made with the pixel values, and This scheme is describ ed in detail in the next sections. the image is thresholded so that only the bright- est 20 pixels remain valid. At this p oint the image is binary, but it contains several small re- 3 Obtaining Circles of Equal Altitude gions one or two pixels around the main region the Sun. We need to measure Sun altitude and obtain the 2. A grass re transform is p erformed in order to GP of the Sun. Solutions to these two problems are destroy noisy features in the thresholded image. discussed in turn. The grass re transform is a fast algorithm for exe- cution of morphological op erations on images for 3.1 Obtaining Altitude a discussion of this and other algorithms, see Xia's Sun altitude is measured using a camera and a dig- pap er [6]. In a binary image, the grass re trans- ital inclinometer. form calculates, for each pixel, the distance b e- Figure 3: Typical Images in Altitude Measurement SUN Figure 2: Camera, Digital Inclinometer, Neutral Den- sity Filters and Adjustable Platform tween the pixel and the background. In an image, background can b e taken either as the \0" pixels or the \1" pixels. Initially the grass re transform is used to shrink regions. Regions with less than Optical Axis 4 pixels of radius are then deleted by deleting all pixels with distances smaller than 4. The grass re transform is then applied again to grow touched, but the regions. Large regions remain un CAMERA smaller regions disapp ear in the pro cess. 3. A region coloring algorithm is used to connect all Figure 4: Geometry of Sun altitude measurements remaining regions in the thresholded image. The algorithm scans the image and grows every region where: as much as p ossible. v Row of the centroid of the Sun in pixels. 4. Each connected region is analyzed. A region has its area and asp ect ratio calculated; the largest v Row of the center of the image in pixels. o region that ob eys given constraints in asp ect ratio f Fo cal length in pixels. v is declared to corresp ond to the Sun. The parameters v and f are intrinsic parameters of o v 5. The centroid of the chosen region is calculated the camera and their values were obtained through a and rep orted as the p osition of the Sun in the calibration pro cedure section 5. image. Figure 4 illustrates the geometry of altitude mea- surement: it is necessary to obtain the angle b etween Figure 3 shows a typical result: the region that the Sun and the optical axis, and the angle b etween corresp onds to the Sun is white; the centroid of this the optical axis and horizontal.