Rob ot Lo calization using a Computer Vision Sextant
Fabio Cozman Eric Krotkov
Rob otics Institute, Carnegie Mellon University
5000 Forb es Ave., Pittsburgh PA 15213
Abstract of Sun altitude, but the basic ideas could b e applied
for any celestial b o dies: satellites, planets, stars. Our
primary target is navigation in cloudless atmospheres
This paper explores the possibility of using Sun alti-
not Earth during long-duration missions.
tude for localization of a robot in total ly unknown ter-
In this pap er, we describ e a system in which these
ritory. A set of Sun altitudes is obtainedbyprocessing
ideas were implemented and tested for a demonstra-
asequence of time-indexed images of the sky. Each al-
tion of feasibility. This system is based on a camera
titude constrains the viewer to a circle on the surface
with telephoto lens and a digital inclinometer for mea-
ofacelestial body, cal led the circle of equal altitude.
surement of attitude of camera. Results of real runs
A set of circles of equal altitude can be intersectedto
are shown in the pap er.
yield viewer position. We use this principle to obtain
position on Earth. Since altitude measurements are
corrupted by noise, a least-square estimate is numer-
ical ly calculatedfrom the sequence of altitudes. The
paper discusses the necessary theory for Sun-based lo-
calization, the technical issues of cameracalibration
This section summarizes basic concepts used in the
and image processing, and presents preliminary results
celestial navigation literature. An extensive treatment
with real data.
is given by Hobbs [3].
We take p osition on Earth to b e represented by
a latitude/longitude pair. For calculations, we take
1 Intro duction
south latitude as negative zero at the equator and
west longitude as negative zero at Greenwich .
This work addresses the problem of rob ot lo caliza-
Every celestial b o dy can b e pro jected onto the ter-
tion: how can a rob ot make precise measurements of
restrial surface by considering the line that go es from
p osition in totally unknown territory? We fo cus on
the center of Earth to the b o dy. The p ointofinter-
the use of celestial cues in order to obtain lo caliza-
section b etween the terrestrial surface and this line is
tion. Techniques for celestial navigation are funda-
the geographic position, denoted GP, of the b o dy. The
mental for any sea navigator [3]. Celestial information
latitude of the GP of a b o dy in the celestial sphere is
is also used in spacecraft and satellite applications for
denoted declination.
attitude control and accurate p ositioning [5]. Yet ce-
The Sun maintains an almost constant declination
lestial navigation has not b een heavily mentioned in
on a given day. Sun declination varies slowly b etween
connection to autonomous rovers. This comes as a sur-
+23 and 23 degrees, during a year. Sun declination
prise since planetary rovers will need reliable means of
is tabulated in the Astronomical Almanac [1] for every
checking dead reckoning data.
dayofagiven year.
We pursue an automatic solution of the lo caliza-
The longitude of the GP of the Sun, contrary to its
tion problem. Instead of using a sextant to obtain the
declination, changes continuously during a day since
altitude of a celestial b o dy,we use a camera and a dig-
the Sun moves from East to West . We can recover
ital inclinometer. The measurements are pro cessed by
longitude of the GP of the Sun if we know Greenwich
a non-linear least-squares optimization that replaces
Mean Time GMT , since time is prop ortional to the
the tables used by mariners [3]. We use measurements
motion of the Sun during the day. GMT is a mean
value; there are drifts and variations during the year.
This research is supp orted in part by NASA under Grant
For this reason, a small correction must b e subtracted
NAGW-1175. Fabio Cozman is supp orted under a scholarship
from GMT in order to obtain the longitude of the GP from CNPq, Brazil.
of the Sun. The correction is called equation of time
and it is tabulated in the Astronomical Almanac for North Pole
every dayofagiven year.
Once we obtain declination and equation of time,
we can calculate the GP of the Sun for every given t.
instan Earth
A measurement of Sun altitude constrains the ob-
server to lie on a circle on the terrestrial surface, called
the circle of equal altitude. As the name suggests, from
any p oint on the circle, the celestial b o dy will app ear
to have the same altitude. A circle of equal altitude
y a measurement of Sun altitude and the is de ned b GP2
GP1
exact time of measurement. Two circles of equal alti-
tude, arising from temp orally distinct measurements,
constrain the observer's p osition to two p oints the
p oints of intersection of the circles . Three circles of er's p osition to one
equal altitude constrain the observ Circles of Equal Altitude
p oint. Four or more measurements of altitude over-
constrain the observer's p osition and can b e used to
check or correct previous measurements.
Figure 1 illustrates a situation where two stars are
measured. Each measurement gives rise to one circle
of equal altitude, so the two measurements give rise
to two circles of equal altitude, whichintersect at two
Figure 1: Two Measurements of Altitude: GPs and
p oints, thus yielding two p ossible solutions. In general
Circles of Equal Altitude
the solutions are very far from each other and one can
b e discarded based on other information, for example,
from dead reckoning.
The camera, attached to a horizontal adjustable
Our system solves the lo calization problem in the
platform, is aimed at the Sun. A series of neutral den-
following four steps:
sity lters is used to eliminate excessive solar bright-
ness. In order to reduce the in uence of camera cali-
1. Take a sequence of Sun images.
bration, a small eld of view is desired: with a large
eld of view, it would b e necessary to know precisely
2. Obtain Sun altitude and the GP of the Sun for
the center of the image and the fo cal length. Tele-
each image in the sequence.
photo lens fo cal length 300mm are used to pro duce
3. Use Sun altitude and the GP of the Sun to gen-
a small eld of view Figure 2 depicts the hardware .
erate a set of circles of equal altitude.
Sun images are senttoaworkstation, which p erforms
the following op erations:
4. Intersect the circles of equal altitude numerically
in a least-squares pro cedure.
1. A histogram is made with the pixel values, and
This scheme is describ ed in detail in the next sections.
the image is thresholded so that only the bright-
est 20 pixels remain valid. At this p oint the
image is binary, but it contains several small re-
3 Obtaining Circles of Equal Altitude
gions one or two pixels around the main region
the Sun .
We need to measure Sun altitude and obtain the
2. A grass re transform is p erformed in order to
GP of the Sun. Solutions to these two problems are
destroy noisy features in the thresholded image.
discussed in turn.
The grass re transform is a fast algorithm for exe-
cution of morphological op erations on images for
3.1 Obtaining Altitude
a discussion of this and other algorithms, see Xia's
Sun altitude is measured using a camera and a dig- pap er [6] . In a binary image, the grass re trans-
ital inclinometer. form calculates, for each pixel, the distance b e-
Figure 3: Typical Images in Altitude Measurement
SUN
Figure 2: Camera, Digital Inclinometer, Neutral Den-
sity Filters and Adjustable Platform
tween the pixel and the background. In an image,
background can b e taken either as the \0" pixels or the \1" pixels. Initially the grass re transform
is used to shrink regions. Regions with less than Optical Axis
4 pixels of radius are then deleted by deleting
all pixels with distances smaller than 4 . The
grass re transform is then applied again to grow
touched, but
the regions. Large regions remain un CAMERA
smaller regions disapp ear in the pro cess.
3. A region coloring algorithm is used to connect all
Figure 4: Geometry of Sun altitude measurements
remaining regions in the thresholded image. The
algorithm scans the image and grows every region
where:
as much as p ossible.
v Row of the centroid of the Sun in pixels .
4. Each connected region is analyzed. A region has
its area and asp ect ratio calculated; the largest
v Row of the center of the image in pixels .
o
region that ob eys given constraints in asp ect ratio
f Fo cal length in pixels .
v
is declared to corresp ond to the Sun.
The parameters v and f are intrinsic parameters of
o v
5. The centroid of the chosen region is calculated
the camera and their values were obtained through a
and rep orted as the p osition of the Sun in the
calibration pro cedure section 5 .
image.
Figure 4 illustrates the geometry of altitude mea-
surement: it is necessary to obtain the angle b etween
Figure 3 shows a typical result: the region that
the Sun and the optical axis, and the angle b etween
corresp onds to the Sun is white; the centroid of this
the optical axis and horizontal. The latter is measured
region is marked with a cross.
with a digital inclinometer. Raw altitude is given by:
Determination of the centroid of the Sun deter-
mines the angle b etween the optical axis and the ray
h = +
a
that emanates from the center of the image to the Sun.
where is the inclinometer reading.
This angle is:
There are additional factors that must b e taken into
v v
account in this typ e of measurement[3]. The grad-
o
= arctan
ual b ending of an incoming lightray in the Earth's
f v
atmosphere is called atmospheric refraction;itmust We set the following non-linear least squares problem:
"
b e corrected. Atmospheric refraction dep ends on the
n
X
raw altitude, b ecause rays b end in di erentways when
^ ^
sin l sin l + l; = arg min
GP i
l;
they reach the atmosphere at di erent angles. The re-
i=1
i
fraction correction R in degrees is given by:
2
: 2 cos l cos l cos + sin h
GP i GP i oi
0:0167
R = :
The minimization starts with a grid of starting
7:31
tan h +
a
p oints. The user de nes a rectangular region in terms
h +4:4
a
of latitude/longitude and spacings for a grid that cov-
The di erence in the apparent altitude of a b o dy
ers a region on Earth. The size of the initial grid is
within the solar system as viewed from the surface
arbitrary; the larger the grid, the longer the minimiza-
of the Earth and from its center is called paral lax;
tion will take. Wehave observed that the minimiza-
it must b e corrected since the images of the Sun are tion, for a xed starting grid, increases linearly with
taken from its surface, not from its center. The par- the numb er of images. This can b e exp ensive after a
allax correction referred to as PA [3] for the Sun
great numb er of images has b een acquired. We use
is:
the following pro cedure instead.
PA =0:0024 cos h :
a
1. Minimize expression 2 for the rst two images,
for all p oints in the starting grid.
The nal observed altitude is h = h R + PA.
o a
2. Combine the solutions that are to o close; use the
solutions as new starting p oints.
3.2 Obtaining GP of the Sun
3. Take a new image and augment the summation
The astronomical p osition of the Sun is xed for a
in expression 2 .
given Greenwich Mean Time value. A set of formulae
4. Minimize expression 2 , for the new starting
is given in the Astronomical Almanac for automatic
p oints.
calculation of astronomical p osition of the Sun.
5. Go to the second step while images are avail-
able .
4 Computation of Lo calization Esti-
The remaining solutions are the result of the whole
mates
pro cedure.
The result of the measurement pro cess previously
outlined pro duces:
5 System Calibration
A sequence of altitudes, indexed by time.
The system is sensitive to errors due to miscalibra-
tion. Three measurements are e ected every time a
A sequence of GPs of the Sun, indexed by time.
pair altitude/GP is obtained:
An image is acquired. Camera parameters were
From each pair altitude/GP, it is p ossible to de-
obtained through a calibration metho d prop osed
ne a circle of equal altitude on the surface of the
by Rob ert [4]. The metho d generates the p ersp ec-
Earth. If we parameterize the Earth surface in lon-
tive pro jection matrix [2] which contains all the
gitude/latitude co ordinates, a circle of equal altitude
relevant information ab out the imaging system.
corresp onds to the lo ci of solutions of:
An angle is obtained through a digital incli-
sin l sin l + cos l cos l cos + = sin h ; 1
GP GP GP o
nometer. We use a digital inclinometer with re-
calibration, but limited to 0:2 degrees of accu-
where l; isapoint in latitude/longitude space,
racy reduced to 0:4 for large inclinations .
l ; is the GP of the Sun in this space and
GP GP
h is the measured altitude.
Greenwich Mean Time is stored. Measurement
o
Since wehave n pairs altitude/GP,wehave a series of time is provided by the computer network and
of n equations in two unknowns. The intersection of can b e easily calibrated through a network call to
all circles of equal altitude is p erformed numerically. a NIST server. 50
47.5
45
6 Exp eriments
42.5
40
A preliminary version of the system was imple-
37.5
mented and tested in real environments. A platform
35
as built to which the camera/lens/ lter apparatus w 32.5
-82 -81 -80 -79 -78
was attached. The platform was leveled using the dig-
ital inclinometer.
Measurements were taken at di erent times of
day, in p erio ds of one hour. In each hour, six
Measurements June 3 94
or eight measurements were taken. All exp er-
0 0
iments were p erformed at 40 26 ; 79 59 lati-
50 tude/longitude ground truth obtained with GPS .
47.5
Two runs of the system are presented b elow.
45
wo measurements Figure 5, top were The rst t 42.5
40
taken resp ectively at 22:50:24 and 23:0:41 GMT ,
37.5
June 3 1994. The temp oral distance b etween measure-
35
ments was small, so the circles of equal altitude are 32.5
almost sup erimp osed. Minimizing equation 2 with -82 -81 -80 -79 -78
0 0
b oth measurements yields 40 22 ; 79 26 maxi-
mum relative error of 0.68 ; each degree corresp onds
roughly to 100km . The second two measurements
Measurements May2394
Figure 5, b ottom were taken May 23 1994, resp ec-
tively at 22:11:23 and 22:37:59 GMT . Result of b oth
0 0
measurements is 40 37 ; 79 52 maximum relative
Figure 5: Data from Sun Measurements: Latitude
error of 0.44 . These measurements are indicative
Longitude values in degrees .
of the accuracy we are exp eriencing with our system,
still in the order of minutes of arc. In its currentver-
for improvement of our system are: building a more
sion, the system is not practical for real navigation,
stable platform and using a set of accurate inclinome-
since such errors in latitude/longitude translate into
ters for measurement of altitude. The principal di-
large p ositional errors. The disturbances come from
culty is to maintain the Sun within the eld of view.
the fact that our platform is not completely rigid, suf-
We are considering two approaches: 1 physical track-
fering from p erturbations from the wind. A heavier,
ing with telephoto lens; 2 providing a hemispherical
more stable platform is necessary to increase accu-
eld of view to the camera with more sophisticated
racy. But the most critical comp onent of our system
calibration , so that the Sun is always in sight.
is the digital inclinometer. For the measurements we
The system do es o er the p ersp ective of a new
describ e, we had to use the inclinometer in its less
source of information into an autonomous agent. The
sensitive range, achieving angular accuracy of 0:4
sky is almost always observable in outdo or areas, its
degrees. This alone is enough to corrupt our p osition
prop erties are known with extraordinary accuracy,
calculations. A new, very precise inclinometer is the
and it is indep endent from facts that o ccur in the
very next step, but in order to use it prop erly, a heavy
vicinity of the agent. So celestial information can b e a
and safe platform must b e in place.
most valuable source of information in scenarios where
the autonomous rob ot is lost, either b ecause the terri-
tory is new or b ecause traditional sensors are confused
7 Conclusion
or broken.
The use of celestial cues for rob ot lo calization in
unknown territory is a useful aid to traditional ap-
References proaches such as dead reckoning. This work demon-
strates the feasibility of this idea, by combining old
[1] Astronomical Almanac, issued by the United ideas from the marine navigation community with im-
States Naval Observatory and Gt. Brit. Nauti- age pro cessing algorithms.
cal Almanac Oce, Washington, US Govt. Print. Further work will b e required to deploy a system
O ., 1994. based on the computer vision sextant idea. Next steps
[2] O. Faugeras, Three-Dimensional Computer Vi-
sion: a Geometric Viewpoint, The MIT Press,
Cambridge, Mass., 1993.
[3] Hobbs, Richard R., Marine Navigation, Third
Edition, Naval Institute Press, Maryland, 1974.
[4] Rob ert, Luc, Camera Calibration without Feature
Extraction, Pro ceed. Int. Conf. Pattern Recogni-
tion, Jerusalem, Israel, 1994.
[5] J. R. Wertz, Spacecraft Attitude Determination
and Control, Reidel, Boston, Mass., 1978.
[6] Y. Xia, Skeletonization Via the Realization of the
FireFront Propagation and Extinction in Digital
Binary Shapes, IEEE Trans. on Pattern Analysis
and Machine Intelligence, 11 10 :1076-1086, Oct. 1989.