Lecture 1.3 Basic projective geometry

Thomas Opsahl Motivation

• For the pinhole camera, the correspondence between observed 3D points in the world and 2D points in the captured image is given by straight lines through a common point (pinhole)

• This correspondence can be described by a mathematical model known as “the perspective camera model” or “the pinhole camera model”

• This model can be used to describe the imaging geometry of many modern cameras, hence it plays a central part in computer vision

2 Motivation

• Before we can study the perspective camera model in detail, we need to expand our mathematical toolbox

• We need to be able to mathematically describe the position and orientation of the camera relative to the world coordinate frame

• Also we need to get familiar with some basic elements of projective geometry, since this will make it MUCH easier to describe and work with the perspective camera model

3 Introduction

𝐴𝐴 𝑇𝑇𝐵𝐵

𝐵𝐵 𝐴𝐴 𝐴𝐴 𝜉𝜉𝐵𝐵

• We have seen that the pose of a coordinate frame relative to a coordinate frame , denoted , can be represented as a homogeneous transformation in 2D 𝐴𝐴 𝐵𝐵 𝐴𝐴 𝐴𝐴 𝜉𝜉𝐵𝐵 𝑇𝑇𝐵𝐵 rrA t AAR t 11 12 Bx AAξ = BB= A∈ B  TB r21 r 22 tBx SE (2) 0 1  00 1 Introduction

𝐴𝐴 𝑇𝑇𝐵𝐵

𝐵𝐵 𝐴𝐴 𝐴𝐴 𝜉𝜉𝐵𝐵

• We have seen that the pose of a coordinate frame relative to a coordinate frame , denoted , can be represented as a homogeneous transformation in 2D and 3D 𝐴𝐴 𝐵𝐵 𝐴𝐴 𝐴𝐴 𝜉𝜉𝐵𝐵 rrrA t 𝑇𝑇𝐵𝐵 11 12 13 Bx AARt rrrA t AAξ  T = BB = 21 22 23 By ∈ SE (3) BBA 0 1 rrr31 32 33 tBz  000 1 Introduction

• And we have seen how they can transform points from one reference frame to another if we represent points in homogeneous coordinates x x x  x y  =  =  pp=    = y ppy   y  z 1 z  1 • The main reason for representing pose as homogeneous transformations, was the nice algebraic properties that came with the representation Introduction

AAB AABA • Euclidean geometry p=ξB⋅= p pRBB pt+ , AAB A A AB AB A – ξξξCBC=⊕=+ ( RC,,t C) ( RR B C R BCtt B) 𝐴𝐴 𝐴𝐴 𝐴𝐴 – Complicated𝐵𝐵 𝐵𝐵 algebra𝐵𝐵 A AT ATA 𝜉𝜉 ⟼ 𝑅𝑅 𝒕𝒕  ξB  ( RRC,− CCt )

• Projective geometry AAB AAB p=⋅=ξBB p ppT – = AABξξξ=⊕=AT AB TT 𝐴𝐴 𝐴𝐴1 CBC C BC 𝐴𝐴 𝐴𝐴 𝑅𝑅𝐵𝐵 𝒕𝒕𝐵𝐵 AAξ −1 – Simple𝜉𝜉𝐵𝐵 ⟼ algebra𝑇𝑇𝐵𝐵  BB T 𝟎𝟎

• In the following we will take a closer look at some basic elements of projective geometry that we will encounter when we study the geometrical aspects of imaging – Homogeneous coordinates, homogeneous transformations The projective plane Points How to describe points in the plane? The projective plane Points How to describe points in the plane?

Euclidean plane • Choose a 2D coordinate2 frame • Each point correspondsℝ to a unique pair of Cartesian coordinates y x = , = x 2 𝑥𝑥 𝒙𝒙 𝑥𝑥 𝑦𝑦 ∈ ℝ ⟼ 𝒙𝒙 𝑦𝑦 2 The projective plane Points How to describe points in the plane?

Euclidean plane x 2 • Choose a 2D coordinate2 frame w  • Each point correspondsℝ to a unique pair of Cartesian coordinates y x = , = 2 𝑥𝑥 x Projective𝒙𝒙 plane𝑥𝑥 𝑦𝑦 ∈ℝ ⟼ 𝒙𝒙 • Expand coordinate2 frame to𝑦𝑦 3D 2 y • Each point corresponds to a triple of homogeneous coordinates

1 = , , = x 2 𝑥𝑥� s.t.𝒙𝒙� 𝑥𝑥� 𝑦𝑦� 𝑤𝑤� ∈ ℝ ⟼ 𝒙𝒙� 𝑦𝑦� , , = , , \𝑤𝑤� 0

𝑥𝑥� 𝑦𝑦� 𝑤𝑤� 𝜆𝜆 𝑥𝑥� 𝑦𝑦� 𝑤𝑤� ∀𝜆𝜆 ∈ ℝ The projective plane Points Observations 1. Any point = , in the Euclidean plane has a corresponding x 𝒙𝒙 𝑥𝑥 𝑦𝑦 = , , 1 2 homogeneous point in the w  projective plane 𝒙𝒙� 𝑥𝑥 𝑦𝑦 2. Homogeneous points of the form , , 0 y x does not have counterparts in the x Euclidean plane 𝑥𝑥� 𝑦𝑦� 2  y They correspond to points at infinity and are called ideal points

1 x The projective plane Points Observations 3. When we work with geometrical problems in the plane, we can swap x 2 between the Euclidean representation w  and the projective representation

x y x 22x  ∋=xx  =y ∈ x y  1 2 y x x 22 w ∋=xx y  =   y 1 w w x Example

1. These homogeneous vectors are different numerical representations of the same point in the plane 3  6 − 30  = =  =−∈   2 x 2  4  20   1  2 − 10 

2. The homogeneous point 1,2,3  represents the same point as , 2 1 2 2 ∈ 3 3 ∈ ℝ The projective plane Lines How to describe lines in the plane? The projective plane Lines How to describe lines in the plane?

Euclidean plane • 3 parameters ,2 , = , ℝ | + + = 0 𝑎𝑎 𝑏𝑏 𝑐𝑐 ∈ ℝ 𝑙𝑙 𝑥𝑥 𝑦𝑦 𝑎𝑎𝑎𝑎 𝑏𝑏𝑏𝑏 𝑐𝑐 y l x 2 The projective plane Lines How to describe lines in the plane?

Euclidean plane • Triple , , 2 \ 2 ℝ w  = , | 3 + + = 0 𝑎𝑎 𝑏𝑏 𝑐𝑐 ∈ ℝ 𝟎𝟎 𝑙𝑙 𝑥𝑥 𝑦𝑦 𝑎𝑎𝑎𝑎 𝑏𝑏𝑏𝑏 𝑐𝑐 y Projective plane  l • Homogeneous vector2 = , , x = | = 0 𝑇𝑇  ̃ 2 2 𝑻𝑻𝒍𝒍 𝑎𝑎 𝑏𝑏 𝑐𝑐  ̃ y 𝑙𝑙 𝒙𝒙� ∈ 𝒍𝒍 𝒙𝒙�

1 l x The projective plane Lines Observations 1. Points and lines in the projective plane have the same representation, we say

2 that points and lines are dual objects in w   2 2. All lines in the Euclidean plane have a y l corresponding line in the projective x plane 2  y 3. The line = 0,0,1 in the projective plane does not have𝑇𝑇 an Euclidean counterpart𝒍𝒍̃ 1 l

x This line consists entirely of ideal points, and is know as the line at infinity

The projective plane Lines Properties of lines in the projective plane 1. In the projective plane, all lines intersect, parallel lines intersect at

2 infinity w  Two lines and intersect in the point = × y ̃1 ̃2 l 𝒍𝒍 𝒍𝒍 1 2 x 2. The line passing𝒙𝒙� 𝒍𝒍̃ through𝒍𝒍̃ points and 2 is given by  1 y = × 𝒙𝒙� 𝒙𝒙�2 𝒍𝒍̃ 𝒙𝒙�1 𝒙𝒙�2 1 l x Example

Determine the line passing through the two points , and ,

𝟐𝟐 𝟒𝟒 𝟓𝟓 𝟏𝟏𝟏𝟏 Homogeneous representation of points 25   22  Matrix representation xx=∈=∈4 13 12   of the cross product 11     × × Homogeneous representation of line where𝒖𝒖 𝒗𝒗 ⟼ 𝒖𝒖 𝒗𝒗 0− 14  5 −− 9 3 0 −uu32   def lxx =×= xx  =1 0 − 2 13 = 3 = 1  12[ 12]×   [u] = uu0 − × 31 −42 0  1 6 2 −uu 0 21 Equation of the line −3xy ++= 20 ⇔ y = 3 x − 2 Example

A point at infinity

https://en.wikipedia.org/wiki/Projective_plane

21 The projective plane Transformations • Some important transformations – like the action of a pose on points in the plane – happen to be linear in the projective plane and non-linear in the Euclidean plane 𝜉𝜉 • The most general invertible transformations of the projective plane are known as homographies – or projective transformations / linear projective transformations / projectivities / collineations

Definition A homography of  is a linear transformation on homogeneous 3-vectors represented by a homogeneous, non2-singular 3 × 3 matrix

= 𝐻𝐻 𝑥𝑥�′ ℎ11 ℎ12 ℎ13 𝑥𝑥� 21 22 23 𝑦𝑦�′ ℎ ℎ ℎ 𝑦𝑦� ℎ31 ℎ32 ℎ33 So is unique up to scale, i.e. 𝑤𝑤�=′ \ 0 𝑤𝑤�

𝐻𝐻 𝐻𝐻 𝜆𝜆𝜆𝜆 ∀ 𝜆𝜆 ∈ ℝ The projective plane Transformations • One characteristic of homographies is that they preserve lines, in fact any invertible transformation of  that preserves lines is a homography 2 • Examples – Central projection from one plane to another is a homography Hence if we take an image with a perspective camera of a flat surface from an angle, we can remove the perspective distortion with a homography

Perspective distortion Without distortion

𝒙𝒙� 𝒙𝒙

Images from http://www.robots.ox.ac.uk/~vgg/hzbook.html The projective plane Transformations • One characteristic of homographies is that they preserve lines, in fact any invertible transformation of  that preserves lines is a homography 2 • Examples – Central projection from one plane to another is a homography – Two images, captured by perspective cameras, of the same planar scene is related by a homography

Image 1 Image 2 The projective plane Transformations • One characteristic of homographies is that they preserve lines, in fact any invertible transformation of  that preserves lines is a homography 2 • Examples – Central projection from one plane to another is a homography – Two images, captured by perspective cameras, of the same planar scene is related by a homography

• One can show that the product of two homographies also must be a homography We say that the homographies constitute a group – the projective linear group 3

𝑃𝑃𝑃𝑃 • Within this group there are several more specialized subgroups Transformations of the projective plane

Transformation of  Matrix #DoF Preserves Visualization 2 Translation 2 Orientation

1 + all below 𝐼𝐼 𝒕𝒕 Euclidean 𝑻𝑻 3 Lengths 𝟎𝟎 1 + all below 𝑅𝑅 𝒕𝒕 Similarity 𝑻𝑻 4 Angles 𝟎𝟎 1 + all below 𝑠𝑠𝑅𝑅𝑻𝑻 𝒕𝒕 Affine 11 𝟎𝟎 12 13 6 Parallelism, 21 22 23 line at infinity 𝑎𝑎0 𝑎𝑎0 𝑎𝑎1 + all below 𝑎𝑎 𝑎𝑎 𝑎𝑎 Homography 11 12 13 8 Straight lines /projective 21 22 23 ℎ31 ℎ32 ℎ33 ℎ ℎ ℎ ℎ ℎ ℎ 26 The projective space

• The relationship between the Euclidean space and the projective space  is much like the relationship between and  3 3 2 2 ℝ ℝ x • In the projective space x  – We represent points in homogeneous coordinates 33 y ∋=xxy  = ∈  z  = = \ 0 z  𝑥𝑥� 𝜆𝜆𝑥𝑥� 1 𝑦𝑦� 𝜆𝜆𝑦𝑦� 𝒙𝒙� ∀𝜆𝜆 ∈ ℝ x 𝑧𝑧̃ 𝜆𝜆𝑧𝑧̃ x  w 𝑤𝑤� – Points at infinity have last homogeneous 𝜆𝜆coordinate𝑤𝑤� equal to zero 33y y  ∋=xx  = ∈ z w  – Planes and points are dual objects  z w w =  | = 0 3 𝑇𝑇 � � � � – The plane at infinity are made upΠ of all𝒙𝒙 points∈ at𝝅𝝅 infinity𝒙𝒙

27 Transformations of the projective space

Transformation of  Matrix #DoF Preserves 𝟑𝟑 Translation 3 Orientation

1 + all below 𝐼𝐼 𝒕𝒕 Euclidean 𝑻𝑻 6 Volumes, volume ratios, lengths 𝟎𝟎 1 + all below 𝑅𝑅 𝒕𝒕 Similarity 𝑻𝑻 7 Angles 𝟎𝟎 1 + all below 𝑠𝑠𝑅𝑅 𝒕𝒕 Affine 𝑻𝑻 12 Parallelism of planes, 𝟎𝟎 The plane at infinity 11 12 13 14 𝑎𝑎 𝑎𝑎 𝑎𝑎 𝑎𝑎 + all below 𝑎𝑎021 𝑎𝑎022 𝑎𝑎023 𝑎𝑎124 𝑎𝑎31 𝑎𝑎32 𝑎𝑎33 𝑎𝑎34 Homography 15 Intersection and tangency of /projective surfaces in contact, straight lines 11 12 ℎ13 ℎ14 ℎ ℎ ℎ21 ℎ22 ℎ23 ℎ24 ℎ31 ℎ32 ℎ33 ℎ34 ℎ41 ℎ42 ℎ43 ℎ44 Summary

• The projective plane  • Additional reading – Homogeneous coordinates2 – Szeliski: 2.1.2, 2.1.3, – Line at infinity – Points & lines are dual

• The projective space  – Homogeneous coordinates3 – Plane at infinity – Points & planes are dual

• Linear transformations of  and  – Represented by homogeneous2 matrices3 – Homographies Affine Similarities Euclidean Translations ⊃ ⊃ ⊃ ⊃ 29 Summary

• The projective plane  • Additional reading – Homogeneous coordinates2 – Szeliski: 2.1.2, 2.1.3, – Line at infinity – Points & lines are dual MATLAB WARNING When we work with linear transformations, we • The projective space  represent them as matrices that act on points by – Homogeneous coordinates3 right multiplication nn – Plane at infinity T : →

– Points & planes are dual x yx= M R Matlab seem to prefer left multiplication instead • Linear transformations of and nn   T : → – Represented by homogeneous2 matrices3 T TT x yx= M L – Homographies Affine Similarities Euclidean Translations So if you use built in matlab functions when you ⊃ ⊃ ⊃ work with transformations, be careful!!! ⊃ 30