An Intro duction to Pro jective Geometry
for computer vision
Stan Birch eld
1 Intro duction
We are all familiar with Euclidean geometry and with the fact that it describ es our three-
dimensional world so well. In Euclidean geometry, the sides of ob jects have lengths, inter-
secting lines determine angles b etween them, and two lines are said to b e parallel if they
lie in the same plane and never meet. Moreover, these prop erties do not change when the
Euclidean transformations translation and rotation are applied. Since Euclidean geome-
try describ es our world so well, it is at rst tempting to think that it is the only typ e of
geometry. Indeed, the word geometry means \measurement of the earth." However, when
we consider the imaging pro cess of a camera, it b ecomes clear that Euclidean geometry is
insucient: Lengths and angles are no longer preserved, and parallel lines mayintersect.
Euclidean geometry is actually a subset of what is known as projective geometry.In
fact, there are two geometries b etween them: similarity and ane.To see the relationships
between these di erent geometries, consult Figure 1. Pro jective geometry mo dels well the
imaging pro cess of a camera b ecause it allowsamuch larger class of transformations than
just translations and rotations, a class which includes p ersp ective pro jections. Of course,
the drawback is that fewer measures are preserved | certainly not lengths, angles, or
parallelism. Pro jective transformations preservetyp e that is, p oints remain p oints and
lines remain lines , incidence that is, whether a p oint lies on a line , and a measure known
as the cross ratio, which will b e describ ed in section 2.4.
Pro jective geometry exists in anynumb er of dimensions, just like Euclidean geometry.
1
For example the pro jective line, whichwe denote by P , is analogous to a one-dimensional
2
Euclidean world; the pro jective plane, P , corresp onds to the Euclidean plane; and pro-
3
jective space, P , is related to three-dimensional Euclidean space. The imaging pro cess is
3 2
a pro jection from P to P , from three-dimensional space to the two-dimensional image
plane. Because it is easier to grasp the ma jor concepts in a lower-dimensional space, we
2
will sp end the bulk of our e ort, indeed all of section 2, studying P , the pro jective plane.
That section presents many concepts which are useful in understanding the image plane and
3
whichhave analogous concepts in P . The nal section then brie y discusses the relevance
of pro jective geometry to computer vision, including discussions of the image formation
equations and the Essential and Fundamental matrices.
1
March 12, A.D. 1998
Euclidean similarity ane pro jective
Transformations
rotation X X X X
translation X X X X
X X X uniform scaling
nonuniform scaling X X
X X shear
p ersp ective pro jection X
X comp osition of pro jections
Invariants
length X
angle X X
ratio of lengths X X
X X X parallelism
incidence X X X X
cross ratio X X X X
Figure 1: The four di erent geometries, the transformations allowed in each, and the mea-
sures that remain invariant under those transformations.
The purp ose of this monograph will b e to provide a readable intro duction to the eld
of pro jective geometry and a handy reference for some of the more imp ortant equations.
The rst-time reader may nd some of the examples and derivations excessively detailed,
but this thoroughness should prove helpful for reading the more advanced texts, where the
details are often omitted. For further reading, I suggest the excellent book byFaugeras [2]
and app endix by Mundy and Zisserman [5].
2 The Pro jective Plane
2.1 Four mo dels
There are four ways of thinking ab out the pro jective plane [3]. The most imp ortant of these
for our purp oses is homogeneous co ordinates, a concept which should b e familiar to anyone
who has taken an intro ductory course in rob otics or graphics. Starting with homogeneous
co ordinates, and pro ceeding to each of the other three mo dels, we will attempt to gain
intuition on the nature of the pro jective plane, whose concise de nition will then emerge
from the fourth mo del. 2
2.1.1 Homogeneous co ordinates
Supp ose wehaveapoint x; y in the Euclidean plane. To represent this same p ointin
1
the pro jective plane, we simply add a third co ordinate of 1 at the end: x; y ; 1 . Overall
scaling is unimp ortant, so the p oint x; y ; 1 is the same as the p oint x; y ; , for any
nonzero . In other words,
X; Y ; W = X ; Y ; W
for any 6= 0 Thus the p oint 0; 0; 0 is disallowed . Because scaling is unimp ortant,
the co ordinates X; Y ; W are called the homogeneous coordinates of the p oint. In our
discussion, we will use capital letters to denote homogeneous co ordinates of p oints, and
we will use the co ordinate notation X; Y ; W interchangeably with the vector notation
T
[X; Y ; W ] .
To represent a line in the pro jective plane, we b egin with a standard Euclidean formula
for a line
ax + by + c =0;
and use the fact that the equation is una ected by scaling to arrive at the following:
aX + bY + cW = 0
T T
u p = p u = 0; 1
T T
where u =[a; b; c] is the line and p =[X; Y ; W ] is a p oint on the line. Thus we see that
p oints and lines have the same representation in the pro jective plane. The parameters of
a line are easily interpreted: a=b is the slop e, c=a is the x-intercept, and c=b is the
y -intercept.
To transform a p oint in the pro jective plane backinto Euclidean co ordinates, we sim-
ply divide by the third co ordinate: x; y = X=W;Y=W . Immediately we see that the
pro jective plane contains more p oints than the Euclidean plane, that is, p oints whose third
co ordinate is zero. These p oints are called ideal points,orpoints at in nity. There is a sep-
arate ideal p oint asso ciated with each direction in the plane; for example, the p oints 1; 0; 0
and 0; 1; 0 are asso ciated with the horizontal and vertical directions, resp ectively. Ideal
2
p oints are considered just likeany other p ointinP and are given no sp ecial treatment.
All the ideal p oints lie on a line, called the ideal line, or the line at in nity, which, once
again, is treated just the same as any other line. The ideal line is represented as 0; 0; 1 .
Supp ose wewant to nd the intersection of two lines. By elementary algebra, the
two lines u = a ;b ;c and u = a ;b ;c are found to intersect at the p oint p =
1 1 1 1 2 2 2 2
b c b c ;a c a c ;a b a b . This formula is more easily rememb ered as the cross
1 2 2 1 2 1 1 2 1 2 2 1
pro duct: p = u u . If the two lines are parallel, i.e., a =b = a =b , the p ointof
1 2 1 1 2 2
intersection is simply b c b c ;a c a c ; 0 , which is the ideal p oint asso ciated with
1 2 2 1 2 1 1 2
the direction whose slop e is a =b . Similarly, given two p oints p and p , the equation of
1 1 1 2
the line passing through them is given by u = p p .
1 2
1
In general, a p ointinann-dimensional Euclidean space is represented as a p ointinan n + 1 -dimensional
pro jective space. 3
p oint p = X; Y ; W line u = a; b; c
T T
incidence p u =0 incidence p u =0
collinearity j p p p j =0 concurrence j u u u j =0
1 2 3 1 2 3
join of 2 u = p p intersection p = u u
1 2 1 2
p oints of 2 lines
ideal p oints X; Y ; 0 ideal line 0; 0;c
a b
Figure 2: Summary of homogeneous co ordinates: a p oints, and b lines.
Now supp ose wewant to determine whether three p oints p , p , and p lie on the same
1 2 3
line. The line joining the rst two p oints is p p . The third p oint then lies on the line
1 2
T
if p p p = 0, or, more succinctly, if the determinant of the 3 3 matrix containing
1 2
3
the p oints is zero:
det [ p p p ]= 0:
1 2 3
Similarly, three lines u , u , and u intersect at the same p oint i.e., they are concurrent ,
1 2 3
if the following equation holds:
det [ u u u ]= 0:
1 2 3
The concepts of homogeneous co ordinates are summarized in Figure 2. For further reading,
consult the notes by Guibas [3].
Example 1. Given two lines u = 4; 2; 2 and u = 6; 5; 1 , the p ointofintersection is
1 2
given by:
i j k
4 2 2 = 2 10 i + 12 4 j + 20 12 k = 8; 8; 8 = 1; 1; 1 :
6 5 1
Example 2. Consider the intersection of the hyp erb ola xy = 1 with the horizontal
line y =1.To convert these equations to homogeneous co ordinates, recall that X = Wx
2
and Y = Wy, yielding XY = W for the hyp erb ola and Y = W for the line. The
solution to these two equations is the p oint W;W;W , which is the same as the p oint
1; 1 in the Euclidean plane, the desired result. Now let us consider the intersection of
the same hyp erb ola with the horizontal line y =0,anintersection which do es not exist in
the Euclidean plane. In homogeneous co ordinates the line b ecomes Y = 0 which yields the
solution X; 0; 0 , the ideal p oint asso ciated with the horizontal direction.
2.1.2 Ray space
2
Wehave just seen that, in going from Euclidean to pro jective, a p ointin R b ecomes a
3
set of p oints in R which are related to each other by means of a nonzero scaling factor. 4 point W u line p Y
X
Figure 3: Ray space.
2 2
Therefore, a p oint p = X; Y ; W in P can b e visualized as a \line" in three-dimensional
space passing through the origin and the p oint p Technically sp eaking, the line do es not
include the origin . This three-dimensional space is known as the ray space among other
2
names and is shown in Figure 3. Similarly, a line u = a; b; c in P can b e visualized as a
\plane" passing through the origin and p erp endicular to u. The ideal line is the horizontal
W = 0 \plane", and the ideal p oints are \lines" in this \plane."
2
Because the co ordinates are una ected by scalar multiplication, P is two-dimensional, even
though its p oints contain three co ordinates. In fact, it is top ologically equivalent to a sphere.
Each p oint p = X; Y ; W , represented as a \line" in ray space, can b e pro jected onto the
1
p
unit sphere to obtain the p oint X; Y ; W Notice that the denominator is never
2 2 2
X +Y +W
zero, since the p oint 0; 0; 0 is not allowed . Thus, p oints in the pro jective plane can b e
visualized as p oints on the unit sphere, as shown in gure 4 Since each \line" in ray space
pierces the sphere twice, b oth these intersections represent the same p oint; that is, antip o dal
p oints are identi ed . Similarly, the \planes" that represent lines in ray space intersect the
unit sphere along great circles, so lines are visualized as great circles p erp endicular to u.
The ideal line is the great circle around the horizontal midsection of the sphere, and the
ideal p oints lie on this circle.
2.1.4 Augmented ane plane
2
To complete our geometrical tour of P , let us pro ject the unit sphere onto the plane W =1.
Y X
; ; 1 which lies at Each p oint X; Y ; W on the sphere is thus mapp ed to the p oint
W W
the intersection of the W = 1 plane with the \line" representing the p oint. Similarly, lines
are mapp ed to the intersection of the W = 1 plane with the \plane" representing the line.
Ideal p oints and the ideal line are pro jected, resp ectively, to p oints at in nity and the line
2
Since it can b ecome confusing to read statements such as, \A p oint is represented as a `line,' " we will
always enclose in quotation marks the entities whose sole purp ose is visualizati on in n + 1 dimensions. 5 point line p u ideal ideal
point line Figure 4: The unit sphere.
point line
ideal point ideal line W Y
u X
Figure 5: The ane plane plus the ideal line and ideal p oints.
at in nity, as shown in gure 5. Thus wehave returned to a representation in which p oints
are p oints and lines are lines. A concise de nition of the pro jective plane can now b e given:
2
De nition 1 The projective plane, P , is the ane plane augmented by a single ideal line
and a set of ideal points, one for each direction, where the ideal line and ideal points are
not distinguishable from regular lines and points.
The ane plane contains the same p oints as the Euclidean plane. The only di erence is
that the former also allows for nonuniform scaling and shear.
2.2 Duality
Lo oking once again at gure 2, the similarities b etween p oints and lines are striking. Their
representations, for example, are identical, and the formula for the intersection of two lines
is the same as the formula for the connecting line b etween two p oints. These observations
are not the result of coincidence but are rather a result of the duality that exists b etween
p oints and lines in the pro jective plane. In other words, any theorem or statement that is
true for the pro jective plane can b e reworded by substituting p oints for lines and lines for
p oints, and the resulting statement will b e true as well. 6
2.3 Pencil of lines
2
A set of concurrent lines in P , that is, a set of lines passing through the same p oint, is a
one-dimensional pro jective space called a pencil of lines. That the space is one-dimensional
is the obvious result of applying the principle of duality: a set of concurrent lines is the
same as a set of collinear p oints. We will say no more ab out a p encil of lines other than to
mention that it exists and that it has several applications in computer vision.
2.4 The cross ratio
As mentioned b efore, pro jective geometry preserves neither distances nor ratios of distances.
However, the cross ratio, which is a ratio of ratios of distances, is preserved and is therefore
2
a useful concept. Given four collinear p oints p , p , p , and p in P , denote the Euclidean
1 2 3 4
distance b etween two p oints p and p as . Then, one de nition of the cross ratio is the
i j ij
following:
13 24
Cr p ; p ; p ; p = : 2
1 2 3 4
14 23
In other words, select one of the p oints, say p , to b e a reference p oint. Compute the ratio
1
of distances from that p ointtotwo other p oints, say p and p . Then compute the ratio of
3 4
distances from the remaining p oint, in this case p , to the same two p oints. The ratio of
2
these ratios is invariant under pro jective transformations.
T T
The Euclidean distance b etween two p oints p =[X ;Y ;W ] and p =[X ;Y ;W ] is
i i i i j j j j
computed from the 2D Euclidean p oints obtained by dividing by the third co ordinate, as
mentioned in section 2.1.1:
v
! !
u
2 2
u
Y X Y X
i j j i
t
+ : =
ij
W W W W
i j i j
Actually, the cross ratio is the same no matter which co ordinate is used as the divisor as
long as the same co ordinate is used for all the p oints ; thus, if all the p oints lie on the ideal
line W = 0 for all i , then we can divide by X or Y instead. For a set of collinear p oints,
i i i
we can always select a co ordinate such that at least three of the p oints have nonzero entries
for that co ordinate. If one of the p oints has a zero entry, simply cancel the terms containing
the p oint b ecause it lies at in nity ; for example, if the second p oint is the culprit W =0;
2
W ;W ;W 6= 0 , then = = 1, which cancel each other:
1 3 4 23 24
13
Cr p ; p ; p ; p = :
1 2 3 4
14
Although the cross ratio is invariant once what the order of the p oints has b een chosen,
its value is di erent dep ending on that order. Four p oints can b e chosen 4! = 24 ways, but
in fact only six distinct values are pro duced, which are related by the set
1 1 1
; 1 ; ; ; g: f ;
1 1 7
a b
Figure 6: The cross ratio can b e used with ve noncollinear p oints.
As we hinted b efore, there are other measures of the cross ratio, all of which are also
invariant under pro jective transformations. Not surprisingly, duality leads to a cross ratio
for four concurrent lines by replacing the Euclidean distance b etween two p oints with the
sine of the angle b etween two lines I have not con rmed whether the cosine also works .
Another less obvious way to measure the cross ratio b etween four concurrent lines is to use
a new, arbitrary line that intersects them; the cross ratio of the lines is then de ned as the
cross ratio of the four p oints of intersection The cross ratio will b e the same no matter
which line is used .
As a nal comment on the cross ratio, it is worth noting that it do es not require that the
original p oints b e collinear. For example, given ve p oints in a star con guration, as shown
in gure 6, we can connect the dots as shown in a to yield lines containing four collinear
p oints, the p oints of intersection, whose cross ratio can b e used. Another p ossibilityisto
draw lines from one of the p oints to the other four, as shown in b , thus yielding four
concurrent lines whose cross ratio can b e used.
2.5 Conics
In Euclidean geometry, the second-order conic sections ellipses, parab olas, and hyp erb olas
are imp ortant phenomena, b eyond the rst-order curves such as lines and planes. Ellipses,
parab olas, and hyp erb olas lose their distinction in pro jective geometry b ecause they are all
pro jectively equivalent, that is, any form can b e pro jected into any other form. Collectively,
these curves are referred to as conics, with no distinction b etween the di erent forms.
Just as a circle in Euclidean geometry is de ned as a lo cus of p oints with a constant
distance from the center, so a conic in pro jective geometry is de ned as a lo cus of p oints
with a constant cross ratio to four xed p oints, no three of which are collinear. Note that
in b oth cases the shap e of the curve is de ned with resp ect to an invariant of the particular
geometry, distance in the case of Euclidean, and cross ratio in the case of pro jective.
The equation of a conic is given by:
T
p C p =0;
or
2 2 2
c X + c Y + c W +2c XY +2c XW +2c YW =0;
11 22 33 12 13 23 8
where p isa3 1vector and C is a symmetric 3 3 matrix.
Since p oints and lines are dual concepts, it is not surprising that a conic is a self-dual
gure. That is, it can b e considered as a lo cus of p oints as wehave just done , or it can b e
considered as an envelop e of tangent lines the set of lines that are tangent to the conic .
T 1
The equation for the envelop e of lines is u jC jC u.
2.6 Absolute p oints
A surprising prop erty of conics is that every circle intersects the ideal line, W =0,attwo
xed p oints. To see this, note that a circle is a conic with all o -diagonal elements c ,
12
c , and c set to zero and all diagonal elements equal:
13 23
2 2 2
X + Y + W =0;
which therefore intersects the ideal line W =0 at
2 2
X + Y =0:
This equation has two complex ro ots, known as the absolute points : i = 1;i;0 and j =
1; i; 0 . Although wehave, for simplicity, assumed that homogeneous co ordinates are
real, they can in general b e the elements of any commutative eld in which1+16= 0 [1,
p. 112]. It will b e shown in the next two subsections that the absolute p oints remain
invariant under similarity transformations, which makes them useful for determining the
angle b etween two lines.
2.7 Collineations
2
A col lineation of P is de ned as a mapping from the plane to itself such that the collinearity
of any set of p oints is preserved. Such a mapping can b e achieved with matrix multiplication
0
bya3 3 matrix T . Each p oint p is transformed into a p oint p :
0
p = T p:
We will use the terms transformation and collineation interchangeably. Since scaling is
unimp ortant, only eight elements of T are indep endent. Therefore, since each p oint contains
two indep endentvalues, four pairs of corresp onding p oints are necessary to determine T .
0
To transform a line u into a line u ,we note that collinearitymust b e preserved, that is,
0
if a p oint p lies on the line u, then the corresp onding p oint p must lie on the corresp onding
0
line u . Therefore,
T 1 0 T 0 T T
p u =0= T p u = p T u ;
which indicates that
0 T
u = T u:
T 1
From these results, it is not hard to show that a p oint conic C transforms to T CT ,
1 1 T
and a line conic jC jC transforms to T jC jC T . 9
Regarding transformations, recall that projective ane similarity Euclidean.
Let's study the matrix T to uncover the relationships b etween these various geometries.
First we will write out the elements of T , for reference:
3 2
t t t
11 12 13
5 4
: T = t t t
pr oj ectiv e 21 22 23
t t t
31 32 33
The ane plane is just the pro jective plane minus the ideal line. Therefore, ane
T
transformations must preserve the ideal line and the ideal p oints, that is, any p oint[X; Y ; 0]
T
must b e transformed into [ X ; Y ; 0] for some arbitrary scaling :
2 3 2 3
X X
4 5 4 5
; = T Y Y
0 0
which implies that t = t = 0. The matrix for ane transformation, then, is
31 32
3 2
t t t
11 12 13
5 4
; T = t t t
aff ine 21 22 23
0 0 t
33
where once again only six of these parameters are indep endent, since scale is unimp ortant.
Unlike ane transformations, similarity transformations preserve angles and ratios of
lengths. Delaying the derivation for a moment, we simply state the result:
3 2
cos sin t
13
5 4
; 3 T = sin cos t
similar ity 23
0 0 t
33
where is an arbitrary angle.
Under Euclidean transformation, scale is imp ortant, and therefore the p oint p must rst
b e converted to Euclidean co ordinates by dividing by its third element. The transformation
then is
0
x cos sin x t
x
= + :
0
y sin cos y t
y
In closing this section, we o er one nal prop osition, along with its pro of:
Prop osition 1 Atransformation is a similarity transformation if and only if it preserves
the absolute points, [1; i; 0].
T
The \only if " is rather easy to see: The absolute p oint[1; i; 0] is transformed through
i T
equation 3 to the p oint e [1; i; 0] , which is equivalent b ecause the scale factor is
ignored. 10
The \if " is a little more complicated, but still rather straightforward. Starting with the
unrestricted equation for T,
2 3
t t t
11 12 13
4 5
t t t T = ;
21 22 23
t t t
31 32 33
T
the fact that [1;i;0] is preserved yields the following two equations:
t + it 1
11 12
=
t + it i
21 22
t + it = 0:
31 32
Since the elements of T are constrained to b e real, this leads to the following three con-
straints on the elements of T :
t = t
11 22
t = t
12 21
t = t = 0:
31 32
So the matrix of T lo oks like this:
2 3
t t t
11 12 13
4 5
T = t t t :
12 11 23
0 0 t
33
Given two arbitrary numb ers t and t ,we can always reparameterize them as t = k cos
11 12 11
and t = k sin , where is an angle and k is a scalar. Multiplying the previous equation
12
by1=k which is legal b ecause we are working in homogeneous co ordinates , we then get
3 2
cos sin t =k
13
5 4
; sin cos t =k T =
23
0 0 t =k
33
which is seen to b e the equation of a similarity transformation when compared with equation
T
3 . note: Using the p oint[1; i; 0] yields the same result.
2.8 Laguerre formula
Absolute p oints have a surprising but imp ortant application: they can b e used to determine
the angle b etween two lines. To see how this works, let us assume that wehavetwo lines
u and u whichintersect the ideal line at two p oints, say p and p . Then, the cross ratio
1 2 1 2
between these two p oints and the two absolute p oints i and j yields the directed angle
from the second line to the rst:
1
= log Cr p ; p ; i; j ;
1 2
2i 11
whichisknown as the Laguerre formula.
To gain some intuition on why this formula is true, let us consider a simple example.
Supp ose wehavetwo lines
a x y = 0
1
a x y = 0
2
in the ane plane. It is clear that these two lines can b e represented as twovectors
T T
v =[1;a ] and v =[1;a ] in the Euclidean plane. The directed angle b etween the two
1 1 2 2
lines is the directed angle b etween the twovectors and is given by:
v v a a
2 1 1 2
tan = = :
v v 1+ a a
1 2 1 2
T T
Now in the pro jective plane these lines are represented as [a ; 1; 0] and [a ; 1; 0] ,
1 2
T T
which are found by mapping p oints [x; y ] in the ane plane to p oints [x; y ; 1] in the
T T
pro jective plane. The ideal line passing through i =[1;i;0] and j =[1; i; 0] is given by
T
i j =[0; 0; 1] . The two p oints of intersection b etween this line and the two original lines
T T
are given by[1;a ; 0] and [1;a ; 0] . The cross ratio of the four p oints is then given by:
1 2
a + i a i
2 1
Cr p ; p ; i; j =
1 2
a + i a i
1 2
1+a a + i a a
1 2 1 2
= :
1+a a + i a a
1 2 2 1
Converting the complex numb ers from rectangular to p olar co ordinates yields:
a a
1 1 2
i tan
1+a a
1 2
e
=
a a
2 1
1
i tan
1+a a
1 2
e
a a
1
1 2
2i tan
1+a a
1 2
; = e
from which it follows that
v v a a 1
2 1 1 2
1 1
= tan ; log Cr p ; p ; i; j = tan
1 2
2i 1+a a v v
1 2 1 2
which is the desired result.
3 Pro jective Space
2
All of the concepts that wehave discussed for the pro jective plane, P ,have analogies
3
in pro jective space, P .For example, there is a dualitybetween p oints and planes, lines
3
are self-dual, a p encil of planes is a two-dimensional pro jective space, the cross ratio
3
Do not b e confused by the term space, which can refer to either three dimensions or an arbitrary number
of dimensions. 12
between planes is invariant, quadrics play the same role as conics, the absolute conic remains
invariant under similarity transformations, and the Laguerre formula can b e used to nd
the angle b etween two pro jection rays. For more details, see [2].
3
A p ointin P is represented by a 4-tuple p = X; Y ; Z; W , and similarly for a plane
T
n. Not surprisingly, a p oint lies in a plane if and only if p n = 0. Slightly more dicult
are the tasks of nding the plane which passes through three given p oints or of nding the
intersections of planes. To answer these questions, wemust rst de ne a representation for
lines.
3.1 Representing lines: The Pluc ker relations
Recall from the Section 2 that the co ordinates of the line passing through two p oints p =
1
X ;Y ;W and p = X ;Y ;W is given by
1 1 1 2 2 2 2
u = Y W W Y ;W X X W ;X Y Y X :
1 2 1 2 1 2 1 2 1 2 1 2
Notice that these three co ordinates are just the determinants of the three 2 2 submatrices
of the following matrix:
3 2
X X
1 2
5 4
; Y Y [ p p ]=
1 2 1 2
W W
1 2
taken in the appropriate order and given the appropriate sign.
3
The pro cedure is similar in P . The co ordinates of the line u passing through two p oints
p = X ;Y ;Z ;W and p = X ;Y ;Z ;W is given by the determinants of the six 3 3
1 1 1 1 1 2 2 2 2 2
submatrices of the following matrix:
2 3
X X
1 2
6 7
Y Y
1 2
6 7
[ p p ]= :
1 2
4 5
Z Z
1 2
W W
1 2
In other words, u = l ;l ;l ;l ;l ;l , where
41 42 43 23 31 12
l = W X X W
41 1 2 1 2
l = W Y Y W
42 1 2 1 2
l = W Z Z W
43 1 2 1 2
l = Y Z Z Y
23 1 2 1 2
l = Z X X Z
31 1 2 1 2
l = X Y Y X :
12 1 2 1 2
These co ordinates l are called the Plucker coordinates of the line. It is fairly easy to show
ij
that, if the p oints p and p are not ideal that is, W and W are not zero , then the
1 2 1 2
co ordinates have a nice Euclidean interpretation:
l ;l ;l = p p
41 42 43 2 1
l ;l ;l = p p ;
23 31 12 1 2 13
1
where p = X ;Y ;Z , i =1; 2, are the co ordinates of the corresp onding Euclidean
i i i i
W
i
p oints. That is, the rst three Pluc ker co ordinates describ e the direction of the line, and
the last three co ordinates describ e the plane containing the line and the origin and the
distance from the origin to the line. Therefore the six Pluc ker co ordinates are sucient
to describ e the line. The co ordinates are not indep endent, however, b ecause they always
satisfy
l l + l l + l l =0;
41 23 42 31 43 12
which can b e derived by noting that the 4 4 determinant jp ; p ; p ; p j is identically zero.
1 2 1 2
Where do es the magic numb er six come from? That is, whydowe need six parameters
n+1
3
to represent a line in P ? Interestingly, it turns out that it takes parameters to
k
representanentity de ned by k p oints in a space requiring n + 1 parameters for each p oint
2
To see this, count the numb er of submatrices in the matrix ab ove . For example, in P a
3
p oint requires = 3 parameters, and a line which is de ned bytwo p oints also requires
1
3 3 4 4
= 3 parameters. In P , a p oint requires = 4 parameters, a line = 6 parameters,
2 1 2
4
and a plane = 4 parameters.
3
3.2 Intersections and unions of p oints, lines, and planes
Now that wehave a representation for lines, we can pro ceed to more complex relations. It
should not surprise the reader that the co ordinates of the plane passing through the three
p oints p = X ;Y ;Z ;W , i =1; 2; 3, is obtained from the determinants of the four 3 3
i i i i i
submatrices of
2 3
X X X
1 2 3
6 7
Y Y Y
1 2 3
6 7
:
4 5
Z Z Z
1 2 3
W W W
1 2 3
Paying careful attention to the order of the submatrices then yields the plane's co ordinates:
Y Y Y X X X X X X X X X
1 2 3 1 2 3 1 2 3 1 2 3
n = W W W ; Z Z Z ; W W W ; Y Y Y :
1 2 3 1 2 3 1 2 3 1 2 3
Z Z Z W W W Y Y Y Z Z Z
1 2 3 1 2 3 1 2 3 1 2 3
Note that, as a result of duality, the same formula can b e used to nd the p oint de ning
the intersection of three planes.
A p oint p = X; Y ; Z; W lies on a line u = l ;l ;l ;l ;l ;l if and only if the
41 42 43 23 31 12
vectors p , p , and p are collinear, that is, the determinants of the four 3 3 submatrices
1 2
of the following matrix
2 3
X X X
1 2
6 7
Y Y Y
1 2
6 7
4 5
Z Z Z
1 2
W W W
1 2
are zero. In terms of the Pluc ker co ordinates, this concurrence can b e expressed as follows:
Ap = 0; 14
where
3 2
l l l 0
23 31 12
7 6
0 l l l
43 42 23
7 6
: A =
5 4
l 0 l l
43 41 31
l l 0 l
42 41 12
An intuitiveway to think ab out A is to realize that a line can b e de ned as the intersection
of two planes. Therefore, a p oint lies on the line if it lies in the two planes. The equation
ab ovesays that a p oint lies on the line if it lies in four planes. Only twoof A's rows are
imp ortant for any given line indeed, A is of rank two , but all four rows are necessary to
ensure that degenerate cases are handled prop erly.
0
Two lines u and u intersect if and only if their Pluc ker co ordinates satisfy the equation
0 0 0 0 0 0
l l + l l + l l + l l + l l + l l = 0;
41 23 42 31 43 12
23 41 31 42 12 43
0 0
which arises from the fact that the 4 4 determinant j p p p p j is zero, where p
1 2 1
1 2
0 0 0
and p lie on u and p and p lie on u .
2
1 2
In this section wehave shown how to nd the line containing two p oints and the plane
containing three p oints. Symmetrically,wehave shown how to nd the p oint at the inter-
section of three planes but have not shown how to nd the line at the intersection of two
planes. This latter problem can b e solved by computing the determinants of the submatri-
ces, but it app ears according to my calculations that a di erentchoice of submatrices will
have to b e made. Wehave shown how to determine whether a p oint lies on a line or in a
plane, and wehave shown whether two lines intersect, though wehave not calculated their
intersection p oint. Other remaining problems include nding the intersection of a line and
a plane, calculating the plane de ned bytwo lines, and computing the plane de ned bya
p oint and a line.
4 Pro jective Geometry Applied to Computer Vision
Pro jective geometry is a mathematical framework in which to view computer vision in gen-
eral, and esp ecially image formation in particular. The main areas of application are those in
which image formation and/or invariant descriptions b etween images are imp ortant, such
as camera calibration, stereo, ob ject recognition, scene reconstruction, mosaicing, image
synthesis, and the analysis of shadows. This latter application arises from the fact that the
comp osition of two p ersp ective pro jections is not necessarily a p ersp ective pro jection but
is de nitely a pro jective transformation; that is, pro jective transformations form a group,
whereas p ersp ective pro jections do not. Many areas of computer vision have little to do
with pro jective geometry, such as texture analysis, color segmentation, and edge detection.
And even in a eld such as motion analysis, pro jective geometry o ers little help when the
rigidity assumption is lost b ecause the relationship b etween pro jection rays in successive
images cannot b e describ ed by such simple and elegant mathematics. 15
The following three sections contain the image formation equations, detailed derivations
of the Essential and Fundamental matrices, and an interesting discussion of the interpreta-
tion of vanishing p oints.
4.1 Image formation
3 2
Image formation involves the pro jection of p oints in P the world to p oints in P the
image plane . The p ersp ective pro jection equations with whichwe are familiar,
X
x = f
Z
Y
y = f
Z
where the p oint X; Y ; Z in the world is pro jected to the p oint x; y on the image plane,
are inherently nonlinear. Converting to homogeneous co ordinates, however, makes them
linear:
0
p = T p;
per spectiv e
0 T T
where p =[x; y ; w ] , p =[X; Y ; Z; W ] , and the p ersp ective pro jection matrix T is given
by:
2 3
f 0 0 0
4 5
T = 0 f 0 0 :
per spectiv e
0 0 1 0
The entire image formation pro cess includes p ersp ective pro jection, along with matrices
for internal and external calibration:
2 3 2 3
k k u f 0 0
u c 0
~
4 5 4 5
P = T T T = 0 k v 0 f 0 [ R t ]
inter nal per spectiv e exter nal v 0
0 0 1 0 0 1
2 3
cot u
u u 0
4 5
= 0 = sin v [ R t ]
v 0
0 0 1
= AD ; 4
where and are the scale factors of the image plane in units of the fo cal length f ,
u v
is the skew = =2 for most real cameras , the p oint u ;v is the principal p oint, R is
0 0
the 3 3 rotation matrix, and t is the 3 1 translation vector. The matrix A contains the
internal parameters and p ersp ective pro jection, while D contains the external parameters.
~
It is sometimes convenient to decomp ose the 3 4 pro jection matrix P into a 3 3
matrix P anda3 1vector p
~
P =[P p ]
so that
P = AR and p = At: 5 16
4.2 Essential and fundamental matrices
Supp ose wehave a stereo pair of cameras viewing a p oint M in the world which pro jects onto
the two image planes at m and m Since we are dealing with homogeneous co ordinates,
1 2
M is 4 1, and m and m are each3 1 . If we assume the cameras are calibrated,
1 2
then m and m are given in normalizedcoordinates, that is, each is given with resp ect
1 2
to its camera's co ordinate frame. The epip olar constraintsays that the vector from the
rst camera's optical center to the rst imaged p oint, the vector from the second optical
center to the second imaged p oint, and the vector from one optical center to the other are
all coplanar. In normalized co ordinates, this constraint can b e expressed simply as
T
m t Rm = 0;
1
2
where R and t capture the rotation and translation b etween the two cameras' co ordinate
frames. The multiplication by R is necessary to transform m into the second camera's
1
4
co ordinate frame. By de ning [t] as the matrix such that [t] y = t y for anyvector y ,
x x
we can rewrite the equation as a linear equation:
T T
m [t] Rm = m E m =0;
x 1 1
2 2
where E =[t] R is called the Essential matrix and has b een studied extensively over the
x
last two decades.
Now supp ose the cameras are uncalibrated. Then the matrices A and A from 4
1 2
containing the internal parameters of the two cameras are needed to transform the normal-
ized co ordinates into pixel co ordinates:
m = A m
1 1 1
m = A m :
2 2 2
This yields the following equation:
1 1
T
A m t RA m = 0
2 1
2 1
T T 1
m A t RA m = 0 6
1
2
2 1
T
m F m = 0; 7
1
2
T 1
where F = A EA is the more recently discovered Fundamental matrix.
2 1
Thus b oth the Essential and Fundamental matrices completely describ e the geometric
relationship b etween corresp onding p oints of a stereo pair of cameras. The only di erence
between the two is that the former deals with calibrated cameras, while the latter deals with
uncalibrated cameras. The Essential matrix contains ve parameters three for rotation and
two for the direction of translation | the magnitude of translation cannot b e recovered due
" "
a 0 c b
4
If t = b , then [t] = c 0 a .
x
c b a 0 17
to the depth/sp eed ambiguity and has two constraints: 1 its determinant is zero, and
2 its two non-zero singular values are equal. The Fundamental matrix contains seven
parameters two for each of the epip oles and three for the homographybetween the two
p encils of epip olar lines and its rank is always two [4].
There are several other ways to derive the Essential and Fundamental Matrices, eachof
which presents a little more insightinto their nature. In the next few subsections, we will
lo ok at these metho ds and then summarize our ndings.
4.2.1 Alternate derivation: algebraic
To describ e the relationship b etween R, t, A , and A more exactly, and to connect the
1 2
ab ove equations with those found in [6], we o er the following algebraic derivation.
~
Recall that a p oint M pro duces an image m through the equation m = P M. Without
loss of generality,we can assume that M is given with resp ect to the rst camera's co ordinate
frame to yield the following two imaging equations:
m = A [ I 0 ] M
1 1 1
m = A [ R t ] M;
2 2 2
where and are scale factors, I is the 3 3 identity matrix and 0 is the 3 1null
1 2
T
T
^ ^
vector. By letting M =[M 1] M is 3 1 , weachieve the following relation:
m = A [ R t ] M
2 2 2
^
= A RM + t
2
1
= A RA m + A t 8
1 2 1 2
1
1 1
A m = RA m + t: 9
2 2 1 1
2 1
Geometrically, this equation says that the vector on the left is a linear combination of the
1
m twovectors on the right. Therefore, they are all coplanar, and the vector v = t RA
1
1
is p erp endicular to that plane:
1 T 1 T T
A m v = RA m v + t v
2 2 1 1
2 1
= 0
T T 1
m t R A m = 0; A
1
2
2 1
which is identical to 6 .
1
Similarly, the vector w = A t A RA m is p erp endicul ar to the vectors in 8 :
2 2 1
1
T 1 T T
m w = A RA m w + A t w
2 1 2 1 2
2
1
= 0
T 1
m = 0: m A t A RA
1 2 2
2
1
This is a surprising result b ecause it gives us a new and equivalent expression for F :
1
F =[A t] A RA ; 10
2 x 2
1 18
which shows that F can b e written as the pro duct of an anti-symmetric matrix [A t] and
2 x
1
an invertible matrix A RA [4].
2
1
4.2.2 Alternate derivation: from the epip olar line
Faugeras [2] approaches the problem from a slightly di erent direction by using the fact
that the p oint m must lie on the epip olar line corresp onding to m :
2 1
T
m l =0: 11
2
That line contains two p oints, the epip ole e the pro jection of the rst camera's optical
center into the second camera and the p oint at in nity asso ciated with m :
1
l = e m :
1
In [2, pp. 40-41] it is shown that the epip ole is given by
1
C P p
1
1
~ ~
e = P = P ;
2 2
1 1
and the p oint at in nityby
1
m = P P m :
1 2 1
1
Therefore, the epip olar line is:
l = e m
1
1
P p
1
1
1
~
= P P P m
2 2 1
1
1
0
1
= [ A R A t ] A R A m
2 2 2 1
1
1
1
= A t A RA
2 2
1
1
= A t A RA m ;
2 2 1
1
where wehave used the substitutions in 5 . Combining with 11 , we get the desired result:
1
F =[A t] A RA :
2 x 2
1
4.2.3 Summary
For reference, wenow summarize the equation for the Essential matrix and the two equations
for the Fundamental matrix:
E = [t] R
x
1 T
EA F = A
1 2
1
= [A t] A RA :
2 x 2
1 19
4.3 Vanishing p oints
Anyone who has taken a course in p ersp ective drawing is familiar with the notion that lines
on the pap er which represent parallel lines in the world intersect on the pap er at a p oint
known as the vanishing p oint. Each set of lines has a di erentvanishing p oint. But, just
what is a vanishing p oint? Pro jective geometry sheds light on this issue.
Because image formation is the pro jection from a 3D world to a 2D surface, each p oint
on the image plane is the pro jection of an in nite numberofpoints in the world. Usually,
the closest p oint is opaque and therefore we think of the p oint on the image plane as b eing
the pro jection of only one p oint in the world. However, ideal p oints in the world i.e., ideal
3
p oints in P , always pro ject onto the image plane regardless of the opacity of other p oints.
Each p oint on the image plane is the pro jection of an ideal p oint. To see this, consider the
following p ersp ective pro jection equation a general pro jective transformation is used for
T T
simplicity from an ideal p oint[X; Y ; Z; 0] in the world to a p oint[x; y ; w ] in the image
plane:
2 3
3 X 2 3 2
x t t t t
11 12 13 14
6 7
Y
6 7
5 4 5 4
: y t t t t =
21 22 23 24
4 5
Z
t t t t w
31 32 33 34
0
Because the matrix is full rank, each ideal p oint pro jects to a di erent p oint on the image
3
plane. Since parallel lines in 3D space intersect at an ideal p ointinP , their pro jections
in the image plane must still intersect at a p oint. But now, through the pro jection matrix,
the ideal p oint has b ecome a \real" p oint, in the sense that it is no longer ideal. However,
some ideal p oints do not b ecome \real" p oints. If the ideal p oint represents a direction that
is parallel to the image plane, then the dot pro duct of the ideal p oint with the third rowof
the matrix which is the z axis of the plane is zero, and the pro jected p oint is still ideal.
2 1
Just as the ideal p oints of P have a one-to-one corresp ondence with all the p oints in P ,
3 2
so the ideal p oints of P have a one-to-one corresp ondence with all the p oints in P .Thus,
we see that the fact that parallel lines in the world intersect when drawn on a piece of pap er
follows naturally from pro jective geometry.
1
A Demonstration of Cross Ratio in P
Let p = X ; 1 ;i =1;:::;4 b e four p oints on the pro jective line. In this demonstration,
i i
we will only consider nite p oints, although the cross ratio holds for in nite p oints as well.
De ne the distance between two p oints i and j as = jX X j. What wewantto
ij ij i j
show is that the cross ratio
13 24
Cr p ; p ; p ; p =
1 2 3 4
14 23
is preserved under pro jective pro jection of the p oints. 20
0
A p oint p is pro jected through a 2 2 transformation matrix T to a new p oint p =
i
i
0 0
t X + t ;t X + t . Therefore, the new co ordinate p = X ; 1 is de ned by:
11 i 12 21 i 22
i i
t X + t
11 i 12
0
: X =
i
t X + t
21 i 22
0 0
Then, the distance b etween two p oints X and X is
i j
0 0 0
= jX X j
ij i j
t X + t t X + t
11 i 12
11 j 12
=
t X + t t X + t
21 i 22 21 j 22
det T X X
i j
; =
t X + t t X + t
21 i 22 21 j 22
0
where det T = t t t t . The ratio b etween two distances, one from a p oint X to
11 22 12 21
i
0 0 0
another p oint X , and another from the same p oint X to a third p oint X ,is
j i
k
0 0 0
jX X j
ij i j
=
0 0 0
jX X j
i
ik k
det T X X t X + t t X + t
i j 21 i 22 21 k 22
=
t X + t t X + t det T X X
21 i 22 21 j 22 i k
t X + t X X
21 k 22 i j
; =
X X t X + t
i k 21 j 22
which is the original ratio = ,multiplied by a constant that is dep endent only up on
ij ik
the co ordinates X and X . A similar ratio = , taken with resp ect to another p oint
j k lj lk
X , has this same constant, and therefore dividing the two ratios causes the constants to
l
cancel:
0 0
13 24
Cr p ; p ; p ; p =
1 2 3 4
0 0
14 23
t X + t X X t X + t X X
21 4 22 2 4 21 3 22 1 3
=
X X t X + t X X t X + t
1 4 21 3 22 2 3 21 4 22
X X X X
2 4 1 3
=
X X X X
1 4 2 3
13 24
= ;
14 23
showing that the cross ratio is una ected by pro jection.
Supp ose that one of the p oints, say p , is at in nity i.e., its second co ordinate is zero .
1
Then, dividing by the second co ordinate which is what we normally do to transform the
p ointinto the required form yields X = 1. Substituting into the ab ove formula yields:
1
24
; Cr p ; p ; p ; p =
1 2 3 4
23 21
since the terms with 1 cancel each other. Technically sp eaking, wemust take the limit of
the equation as X tends to 1, but the result is the same. Similarly,ifany of the other
1
p oints are at in nity,we simply cancel the terms containing the p oint, and the result is the
cross ratio. Rememb er that at most one p ointmay b e at in nity, b ecause the p oints must
b e distinct, and there is only one p oint at in nity on the pro jective line.
Acknowledgment
Many thanks to Olaf Hall-Holt for helping me understand Pluc ker co ordinates.
References
[1] H. S. M. Coxeter. Projective Geometry.Toronto: UniversityofToronto, second edition,
1974.
[2] O. Faugeras. Three-Dimensional Computer Vision. Cambridge, MA: MIT Press, 1993.
[3] L. Guibas. Lecture notes for CS348a: Computer Graphics { Mathematical Foundations.
Stanford University, Autumn 1996.
[4] Q.-T. Luong and O. D. Faugeras. Fundamental matrix: Theory, algorithms, and stability
analysis. International Journal of Computer Vision, 17 1 :43{75, 1996.
[5] J. L. Mundy and A. Zisserman. Geometric Invariance in Computer Vision. Cambridge,
MA: MIT Press, 1992.
[6] Z. Zhang and G. Xu. Epipolar Geometry in Stereo, Motion and Object Recognition: A
Uni ed Approach. Kluwer Academic Publishers, 1996. 22