2 Projective Geometry and Transformations of 2D 1

2.1 Planar geometry The classical approach of Euclid studied geometry from a “geometric” or coordinate- free viewpoint. Since Descartes, it has been seen that geometry may be algebraicized. In the algebraic approach, geometric entities are described in terms of coordinates and algebraic entities. A signiﬁcant advantage of the algebraic approach to geometry is that results derived in this way may more easily be used to derive algorithms and practical computational methods.

2.2 The 2D projective plane 2.2.1 Points and lines Homogeneous representation of lines. A line in the plane is represented by an equation such as ax + by + c = 0 or the vector (a, b, c)⊤. Since (a, b, c) and k(a, b, c) (k = 0) represent the same line, an equivalence class ⊤ ⊤ ∕ of vectors is known as a homogeneous vector. The set of equivalence classes of vectors in R3 (0, 0, 0) forms the projective space P2. − ⊤

Homogeneous representation of points. Since (x, y, 1)⊤ and k(x, y, 1)⊤ (k = 0) represent the same point, points as homogeneous vectors are also elements of ∕P2. We distinguish between the homogeneous coordinates x = (x1, x2, x3)⊤ of a point and the inhomogeneous coordinates (x, y)⊤.

Result 2.1. The point x lies on the line l if and only if x⊤l = 0.

Degrees of freedom (dof). In order to specify a point (or a line), two values must be provided. Namely it has two degrees of freedom.

Result 2.2. The intersection of two lines l and l is the point x = l l . ′ × ′ Given x = l l , we have × ′ x⊤l = l⊤x = l⊤l l′ = 0 ×

Result 2.4. The line through two points x and x is l = x x . ′ × ′ Given l = x x , we have × ′ x⊤l = x⊤x x′ = 0 ×

1The notes in this document are taken by Wufei Ma from Multiple View Geometry in Computer Vision by Richard Hartley and Andrew Zisserman. I added some comments and calculations to those steps that are not quite “obvious” to me.

1 x 2

ideal point

O x x π 3

x 1 Figure 1: A model of the projective plane.

2.2.2 Ideal points and the line at inﬁnity

Intersection of parallel lines. Consider two lines l = (a, b, c)⊤ and l′ = (a, b, c′)⊤. Using Result 2.2, the intersection is

l l′ = (bc′ cb, ca ac′, ab ab) = (c′ c)(b, a, 0)⊤ × − − − −

Ideal points and the line at infinity. One may augment R2 with last coordinate x3 = 0. The resulting space is the set of all homogeneous 3-vectors, namely the 2 projective space P . The points with last coordinate x3 = 0 are known as ideal points, or points at infinity. The set of ideal points, given by (x1, x2, 0)⊤, lies on a single line, the line at infinity, denoted by l∞ = (0, 0, 1)⊤. In the projective plane P2, one may state without qualification that two distinct lines meet in a single point.

The study of the geometry of P2 is known as projective geometry.

A model for the projective plane. A fruitful way of thinking of P2 is as a set of rays 3 3 in R , as shown in Figure 1. A ray in R , k(x1, x2, x3)⊤, represent a single point in P2. The lines in P2 are planes passing through the origin.

Result 2.6. Duality principle. To any theorem of 2-dimensional projective geometry there corresponds to a dual theorem, which may be derived by interchanging the roles of points and lines in the original theorem.

2.2.3 Conics and dual conics A conic is a curve described by a second-degree equation in the plane. In Euclidean geometry conics are of three main types: hyperbola, ellipse, and parabola (apart from degenerate conics).

2 The equation of a conic in inhomogeneous coordinates is

ax2 + bxy + cy2 + dx + ey + f = 0

“Homogenizing” by the replacements: x x /x and y x /x gives → 1 3 → 2 3 2 2 2 ax1 + bx1x2 + cx2 + dx1x3 + ex2x3 + f x3 = 0 or in matrix form

x⊤Cx = 0 where a b/2 d/2 C = b/2 c e/2   d/2 e/2 f   The conic has ﬁve degress of freedom.

Five points deﬁne a conic. Each point (xi, yi) on the conic gives a constraint

2 2 axi + bxiyi + cyi + dxi + eyi + f = 0 which can be written as

2 2 xi xiyi yi xi yi 1 C = 0 Stacking the constraints we% obtain &

AC = 0 and the conic is the null vector of the matrix A. With ﬁve points, A is a 5 6 matrix. This shows that a conic is determined uniquely (up to scale) by ﬁve po×ints.

Result 2.7. The line l tangent to C at a point x on C is given by l = Cx. To show that the point x is on l, we have

x⊤l = x⊤Cx = 0

To show that the line l is tangent to C, assume that l intersects with C at another point y (besides x). We have

y⊤l = y⊤Cx = 0

x⊤Cx = 0

Since C = C⊤, it follows that

2 (y + αx)⊤C(y + αx) = y⊤Cy + 2y⊤Cx + α x⊤Cx = 0, α ∀

3 Figure 2: Conic envelope: the lines l satisfying l⊤C∗l = 0.

This means the whole line l is on the conic C, and C is degenerate.

Dual Conics. The conic deﬁned above is known as a point conic. By duality principle, there is a conic that deﬁnes an equation on lines, known as a dual conic, or line conic. The dual conic is given by C∗, which is the adjoint matrix of C. 1 From Result 2.7, we have l = Cx, so x = C− l. Since x⊤Cx = 0, we have 1 1 (C− l)⊤C(C− l) = 0 which is

l⊤C−⊤l = l⊤C∗l = 0

Dual conics are also known as conic envelopes, as illustrated in Figure 2.

Cofactor and adjoing matrix

Given a square matrix M, the cofactor matrix M∗ is given by

i+j M∗ = ( 1) det(Mˆ ) ij − ij

where Mˆ ij is the (i, j) minor of M. The adjoint matrix adj(M) is the transpose of M∗. It is known that

1 adj(M) = det(M)M−

Degenerate conics. Degenerate line conics include two lines (rank 2), and a repeated line (rank 1). For example, C = lm⊤ + ml⊤. Degenerate point conics include two points (rank 2), and a repeated point (rank 1). For example, C = xy⊤ + yx⊤.

2.3 Projective transformations Deﬁnition 2.9. A projectivity is an invertible mapping h from P2 to itself such that three points x1, x2, and x3 lie on the same line if and only if h(x1), h(x2), and h(x3) do. A projectivity is also called a collinearation, a projective transformation, or a homography.

4 Theorem 2.10. A mapping h : P2 P2 is a projectivity if and only if there exists a → non-singular 3 3 matrix H such that for any point in P2 represented by a vector x × it is true that h(x) = Hx.

Deﬁnition 2.11. Projective transformation. A planar projective transformation is a linear transformation on homogeneous 3-vector represented by a non-singular 3 3 matrix: ×

x′ = Hx

Note that the matrix H is a homogeneous matrix, and it follows that the projective transformation has eight degrees of freedom.

2.3.1 Transformations of lines and conics

Transformation of lines. Under the point transformation x′ = Hx, we have

1 1 0 = l⊤x = l⊤H− Hx = l⊤H− Hx = (H−⊤l)⊤Hx

Hence a line transforms as

l′ = H−⊤l

1 Points transform according to H, whereas lines transform according to H− . One says that points transform contravariantly and lines transform covariantly.

Transformation of conics. Under the point transform x′ = Hx, we have

1 0 = x⊤Cx = x′⊤H−⊤CH− x′

Result 2.13. Under a point transformation x′ = Hx, a conic C transforms to C′ = 1 H−⊤CH− . 1 The presence of H− in this equation may be express by saying that a conic transforms covariantly.

Result 2.14. Under a point transform x′ = Hx, a dual conic C∗ transforms to C∗′ = HC∗H⊤.

2.4 A hierarchy of transformations The projective transformations form a group, known as the projective linear group. The group of invertible n n matrices with real elements is the (real) general × linear group on n dimensions, or GL(n). The projective linear group is identiﬁed by a scalar multiplier, giving PL(n) (this is a quotient group of GL(n)).

5 2.4.1 Class I: Isometries Isometries are transformations of the plane R2 that preserve Euclidean distance, represented as x ε cos θ sin θ t x ′ − x y = ε sin θ cos θ t y  ′  y   1 0 0 1 1       where ε = 1. ± If ε = 1, the isometry is orientation-preserving and is a Euclidean transformation. If ε = 1, the isometry is orientation-reversing. A plan−ar Euclidean transformation can be written more precisely as R t x = H x = x ′ E 0 1 + ⊤ ,

A planar Euclidean transformation has three degrees of freedom and can be computed from two point correspondences.

2.4.2 Class II: Similarity transformations A similarity transformation (or similarity) is an isometry composed with an isotropic scaling, also known as equi-form transformation. With no reﬂection, the similarity has matrix representation x s cos θ s sin θ t x ′ − x y = s sin θ s cos θ t y  ′  y   1 0 − 0 1 1       which can be written in block form as sR t x = H x = x ′ S 0 1 + ⊤ ,

A planar similarity has four degrees of freedom and can be computed from two point correspondences.

Metric structure. The description metric structure implies that the structure is deﬁned up to a similarity.

2.4.3 Class III: Affine transformation An affine transformation (or affinity) is a non-singular linear transformation followed by a translation, given by

x′ a11 a12 tx x y = a a t y  ′  21 22 y   1 0 0 1 1       6 or in block form A t x = H x = x ′ A 0 1 + ⊤ ,

A planar afﬁnity has six degrees of freedom and can be computed from three point correspondences.

With SVD, A can be decomposed as

A = UDV⊤ = UV⊤VDV⊤ = R(θ)R( φ)DR(φ) − Hence the distortion of A can be seen as a rotation by R(θ) followed by a deformation R( φ)DR(φ). − Invariants. Three important invariants are parallel lines ◦ ratio of lengths of parallel line segments ◦ ratio of areas ◦ 2.4.4 Class IV: Projective transformations A projective transformation is given by a general non-singular linear transformation of homogeneous coordinates, represented as

x1′ h11 h12 h13 x1 x = h h h x  2′   21 22 23  2 x3′ h31 h32 h33 x3       or in the block form A t x = H x = x ′ P v v + ⊤ ,

A projective transformation has eight degrees of freedom and can be computed from four point correspondences.

Invariants. The most fundamental projective invariant is cross ratio.

2.4.5 Summary and comparison For a given projective transformation, area scaling varies with position; and the orientation of a transformed line depends on both the orientation and position of the source line.

7 Group Invariant properties Projective (8 dof) Concurrency; collinearity; order of contact: intersection (1 pt); tangency (2 pt); inﬂections (3 pt); tangent discontinuities and cusps; cross ratio Alline (6 dof) Parallelism, ratio of areas, ratio of lengths on collinear or parallel lines, linear combination of vectors. The line at inﬁnity, l∞ Similarity (4 dof) Ratio of lengths, angle. The circular points, I, J. Euclidean (3 dof) Length, area.

Table 1: Geometric properties invariants to planar transformations. A transformation lower in the table inherits the invariants of those above.

2.4.6 Decomposition of a projective transformation A projective transformation can be decomposed as

sR t K 0 I 0 A t H = H H H = = S A P 0 1 0 1 v v v v + ⊤ , + ⊤ , + ⊤ , + ⊤ , where A = sRK + tv⊤, and K an upper-triangular matrix normalized as det(K) = 1.

This decomposition is valid provided v = 0 and s is unique if s is chosen positive. ∕ 2.4.7 The number of invariants Result 2.16. The number of functionality independent invariants is equal to, or greater than, the number of degrees of freedom of the conﬁguration less the number of degrees of freedom of the transformation.

The 2D transformation groups and their invariant properties are summarized in Ta- ble 1.

2.5 The projective geometry of 1D

We use x¯ to represent the homogeneous vector (x1, x2)⊤ and a projective transformation of a line is represented by

x¯ ′ = H2 2x¯ × and has three degrees of freedom.

The cross ratio. Given four points x¯ i, the cross ratio is deﬁned as x¯ x¯ x¯ x¯ Cross(x¯ , x¯ , x¯ , x¯ ) = | 1 2|| 3 4| 1 2 3 4 x¯ x¯ x¯ x¯ | 1 3|| 2 4|

8 Figure 3: Concurrent lines.

Figure 4: Removing perspective distortion. where

xi1 xj1 x¯ ix¯ j = det | | x 2 x 2 + i j ,

Concurrent lines. A conﬁguration of concurrent lines is dual to collinear points on a line, as shown in Figure 3 .

2.6 Topology of the projective plane There is a two-to-one correspondence between the unit sphere 2 in R3 and the projective plane P2. S In the language of topology, the sphere 2 is a 2-sheeted covering space of P2. S Skipped: Page 47

Topology and orientation of P1 and P2.

2.7 Recovery of afﬁne and metric properties from images The goal is to remove projective distortion such that similarity properties (angles, ratios of lengths) could be measured on the original plane, as shown in Figure 4.

9 The projective transformation can be completely removed by explicitly computing the transformation given 4 reference points.

In the following, it is shown that the projective distortion may be removed once the image of l∞ is specified, and the affine distortion removed once the image of the circular points is specified. Then the remaining distortion is a similarity.

2.7.1 The line at inﬁnity

Result 2.17. The line at infinity, l∞, is a fixed line under the projective transformation H if and only if H is an affinity.

2.7.2 Recovery of affine properties from images Once the imaged line at infinity is identified in an image of a plane, we can transform the identified l∞ to its canonical position (0, 0, 1)⊤. Affine measurements can be made directly from the rectified image.

If the imaged line at afﬁnity is l = (l , l , l ) , then provided l = 0, a suitable 1 2 3 ⊤ 3 ∕ projective point transformation which will map l back to l∞ is

1 0 0 H = H 0 1 0 A   l1 l2 l3   since 1 0 l /l l 0 − 1 3 1 H−⊤l = 0 1 l /l l = 0  − 2 3  2   0 0 1/l3 l3 1       Thus we have shown how to recover afﬁne properties by specifying a line (2 dof).

Now we show that if affine properties are known, we can also determine the line at infinity. More specifically, we can find the vanishing point using the cross ratio as follows:

Identify three collinear points a, b, c and the length ratio d(a, b) : d(b, c) = a : b ◦ is known.

Measure the distance ratio in the image: d(a , b ) : d(b , c ) = a : b . ◦ ′ ′ ′ ′ ′ ′ Represent points a, b, and c with homogeneous 2-vectors (0, 1) , (a, 1) , and ◦ ⊤ ⊤ (a + b, 1)⊤. Similarly, a′, b′, and c′ have coordinates 0, a′, and a′ + b′.

Compute a 1D projective transformation H2 2 mapping a a′, b b′, and ◦ × → → c c . → ′

10 o v/

c / b/

a/ a a b b c l Figure 5: Determine the vanishing point given a known length ratio.

The image of the point at inﬁnity (1, 0)⊤ under H2 2 is the vanishing point on ◦ × the line a , b , c . 〈 ′ ′ ′〉

The vanishing point can also be computed from a purely geometric approach (in Figure 5:

Given three collinear points a , b , and c corresponding to collinear world ◦ ′ ′ ′ points with interval ratio a : b.

Draw a line through a and mark off points a = a , b, and c from the ratio a : b. ◦ ′ ′ Join bb′ and cc and intersect at o. ◦ ′ The line through o parallel to l meets the line a c in the vanishing point v . ◦ ′ ′ ′

2.7.3 The circular points and their dual

Under any similarity transformation, there are two points on l∞ which are ﬁxed, which are the circular points (or absolute points) I and J with coordinates

I = (1, i, 0)⊤, J = (1, i, 0)⊤ − where s cos θ s sin θ t 1 − x I′ = H I = s sin θ s cos θ t i S  y   0 0 1 0  s(cos θ i sin θ)    1 − iθ = s(cos θ i sin θ)i = se− i     −0 0    

Result 2.21. The circular points, I and J, are ﬁxed points under the projective transformation H if and only if H is a similarity.

11 The name “circular points” arises because every circle intersects I∞ at the circular points.

It will be shown that identifying the circular points (or their dual) allows the recovery of similarity properties (angles, ratios of lengths).

The conic dual to the circular points. The conic

C∞∗ = IJ⊤ + JI⊤ is dual to the circular points.

Result 2.22. The dual conic C∞∗ is ﬁxed under the projective transformation H if and only if H is a similarity.

Two properties of C∞∗ in any projective frame: C has 4 degrees of freedom (symmetric, det C = 0). ◦ ∞∗ ∞∗ l is the null vector of C . ◦ ∞ ∞∗ 2.7.4 Angles on the projective plane For the lines l and m, the angle between the two lines is

l C∗ m cos θ = ⊤ ∞ (l⊤C∗∞l)(m⊤C∞∗ m) -

Result 2.23. Once the conic C∞∗ is identiﬁed on the projective plane then Euclidean angles may be measured by the equation above. In a Euclidean coordinate system, we have

l C∗ m cos θ = ⊤ ∞ (l⊤C∗∞l)(m⊤C∞∗ m)

- l1m1 + l2m2 = 2 2 2 2 (l1 + l2)(m1 + m2) Notice that

l⊤C∞∗ m = (H−⊤l)⊤HC∞∗ H⊤(H−⊤m) 1 = l⊤H− HC∞∗ H⊤H−⊤m

= l⊤C∞∗ m

Hence the equation for cos θ is invariant under point transformation H.

12 Result 2.24. Lines l and m are orthogonal if l⊤C∞∗ m = 0. Geometrically, l and m are conjugate with respect to C∞∗ .

Length ratios. Length ratios can also be measured once C∞∗ is identiﬁed:

d(b, c) : d(a, c) = sin α : sin β

2.7.5 Recovery of metric properties from images Suppose the circular points are identified in an image, then the image is rectified by a projective transformation H that maps the imaged circular points to their canonical position on l∞. The transformation between the world plane and the rectified image is then a similarity.

Metric rectification using C∞∗ . The dual conic C∞∗ packages all the information re- quired for a metric rectification of both the projective and affine transformations.

Result 2.25. Once the conic C∞∗ is identiﬁed on the projective plane then projective distortion may be rectiﬁed up to a similarity. Given a point transformation H, we have

C∞∗′ = (HPHAHS)C∞∗ (HPHAHS)⊤

= (HPHA)C∞∗ (HAHP)⊤ K 0 I 0 K K v = ⊤ ⊤ v K v 0 0 0 v + ⊤ , + , + ⊤ , K 0 K K v = ⊤ ⊤ v K 0 0 v + ⊤ , + ⊤ , KK KK v = ⊤ ⊤ v KK v KK v + ⊤ ⊤ ⊤ ⊤ , It is clear that the projective (v) and afﬁne (K) are determined from the image of C∞∗ .

A suitable rectifying homography may be obtained from the identiﬁed C∞∗ in an image using the SVD:

C∞∗′ = UC∞∗ U⊤

Then the rectifying projectivity is H = U up to a similarity.

Example 2.26. Metric rectification I. Given an image that has been affinely rectified, we can determine a metric rectification by obtaining 2 degrees of freedom of the

13 y

l x

Figure 6: The pole-polar relationships. circular points from two imaged right angles on the world plane.

m KK 0 1′ l l l ⊤ m = 0 1′ 2′ 3′ 0 0  2′  + ⊤ , m . / 3′   so K can be determined up to scale by Cholesky decomposition.

Alternatively, the two constraints may be obtained from an imaged circle or two known length ratios.

Example 2.27. Metric rectiﬁcatin II. Given an original perspective image, we can obtain C∞∗ from 5 constraints given by 5 pairs of orthogonal lines on the world plane.

Stratification. The two-step approach to remove affine and projective distortions is termed stratified.

2.8 More properties of conics 2.8.1 The pole-polar relationship A point x and a conic C deﬁne a line l = Cx. The line l is called the polar of x with respect to C, and the point x is the pole of l with respect to C. See Figure 6.

Deﬁnition 2.29. A correlation is an invertible mapping from points of P2 to lines of P2. It is represented by a 3 3 non-singular matrix A as l = Ax. ×

Conjugate points. If the point y is on the line l = Cx then y⊤l = y⊤Cx = 0. If x is on the polar of y, then y is on the polar of x.

2.8.2 Classiﬁcation of conics Projective normal form for a conic. Since C is a symmetric matrix, it has real eigenvalues and can be decomposed as a C = U⊤DU, where U is orthogonal and D is diagonal. By applying a projective transformation U, we have C′ = D.

14 Diagonal Equation Conic type (1, 1, 1) x2 + y2 + w2 = 0 Improper conics (no real points) (1, 1, 1) x2 + y2 w2 = 0 Circle − 2 −2 (1, 1, 0) x + y = 0 (0, 0, 1)⊤ (1, 1, 0) x2 y2 = 0 x = y − − ± (1, 0, 0) x2 = 0 x = 0 counted twice

Table 2: Projective transformation of point conics.

H e e 1 1

e 3 e 3 / e 2 x e 2 x

Figure 7: Fixed points and lines of a plane projective transformation.

Further, D can be transformed to diag(1, 2, 3), where i = 1, 0. Various types of conics may now be enumerated, as shown in Table 2. ±

Affine classification of conics. The classification of non-degenerate conics, hyperbola, ellipse, and parabola depends on the relation of l∞ to the conic. A conic is an (a) ellipse, (b) a parabola, or (c) a hyperbola according to whether it (a) has no real intersections, (b) is tangent to, or (c) has 2 real intersections with l∞.

2.9 Fixed points and lines The key idea is that an eigenvector corresponds to a ﬁxed point of the transformation, since

He = λe

Often the eigenvector and eigenvalue have physical or geometric signiﬁcance in computer vision applications.

A Euclidean matrix. The two ideal ﬁxed points are the complex conjugate pair of iθ circular points, I and J, with eigenvalues e± . The third eigenvector, which has unit eigenvalue, is called the pole.

A similarity matrix. The two ﬁxed points are again the circular points. The eigen- iθ values are 1, se± .

15 An affine matrix. The two ideal fixed points can be real or complex conjugates, but the fixed line l∞ through these points is real in either case.

2.10 Closure 2.10.1 Literature 2.10.2 Notes and exercises

Last Updated: Jun 01, 2020