Computer Vision I - Projective Geometry and a Camera Carsten Rother 22/12/2014 Computer Vision I: Image Formation Process Roadmap for next five lecture • Appearance based matching (sec. 4.1) • How do we get an RGB image? (sec 2.2-2.3) • The Human eye • The Camera and Image formation model • Projective Geometry - Basics (sec. 2.1.1-2.1.4) • Geometry of a single camera (sec 2.1.5, 2.1.6) • Pinhole camera • Lens effects • Geometry of two cameras (sec. 7.2) • Robust Geometry estimation for two cameras (sec. 6.1.4) • Multi-View 3D reconstruction (sec. 7.3-7.4) Computer Vision I: Image Formation Process 22/12/2014 2 Some Basics 2 1 • Real coordinate space 푅 example: 2 3 1 • Real coordinate space 푅 example: 2 3 • Operations we need are: scalar product: 풕 푥1 풙 풚 = 풙 풚 = 푥1푦1 + 푥2푦2 + 푥3푦3 where x = 푥2 푥3 cross/vector product: 풙 × 풚 = 풙 × 풚 0 −푥3 푥2 풙 × = 푥3 0 −푥1 −푥2 푥1 0 Computer Vision I: Image Formation Process 22/12/2014 3 Euclidean Space • Euclidean Space 푅2 or 푅3 has angles and distances defined • Angle defined as: • Length of the vector 푥: = 푥 푥 푥 • Distance: 푥 − 푦 Θ 푦 표푟푖푔푖푛 Computer Vision I: Image Formation Process 22/12/2014 4 Projective Space • 2D Point in a real coordinate space: 1 2 2 ∈ 푅 has 2 DoF (degrees of freedom) • 3D Point in a real coordinate space: 1 3 2 ∈ 푅 has 3 DoF 3 • Definition: A point in 2-dimensional projective space 푃2 is defined as 푥 2 푘푥 푝 = 푦 ∈ 푃 , such that all vectors 푘푦 ∀ 푘 ≠ 0 푤 푘푤 define the same point 푝 in 푃2 (equivalent classes) 1 2 • Sometimes written as: 2 ~ 4 1 2 1 2 • We write as: 2 = 4 1 2 Computer Vision I: Image Formation Process 22/12/2014 5 Projective Space - visualization Definition: A point in 2-dimensional projective space 푃2 is defined as 푥 2 푘푥 푝 = 푦 ∈ 푃 , such that all vectors 푘푦 ∀ 푘 ≠ 0 푤 푘푤 define the same point 푝 in 푃2 (equivalent classes) A point in 푃2 is a ray in 푅3 that goes through the origin: 푥 w-axis 푦 푤 All rays go through (0,0,0) define a point in 푃2 Plane w=1 Plane w=0 푥 푦 0 Computer Vision I: Image Formation Process 22/12/2014 6 Projective Space 2 3 0 • All points in 푃 are given by: 푅 \ 0 0 푥 2 • A point 푦 ∈ 푃 has 2 DoF (3 elements but norm of vector can be set to 1) 푤 푥 w-axis 푦 푤 All rays go through (0,0,0) define a point in 푃2 Plane w=1 Plane w=0 푥 푦 0 Computer Vision I: Image Formation Process 22/12/2014 7 From 푅2 to 푃2 and back • From 푅2 to 푃2: 푥 푥 푝~ = ∈ 푅2 푝 = 푦 ∈ 푃2 푦 1 - a point in inhomogeneous coordinates - a point in homogeneous coordinates - we soemtimes write ~푝 for inhomogeneous coordinates • From 푃2 to 푅2: 푥 푥/푤 푦 ∈ 푃2 ∈ 푅2 푤 푦/푤 푓표푟 푤 ≠ 0 what does it mean if w=0? We can do this transformation with all primitives Computer Vision I: Image Formation Process 22/12/2014 8 From 푅2 to 푃2 and back: Example • From 푅2 to 푃2: 푥 푥 2 푝~ = ∈ 푅2 푝 = 푦 ∈ 푃 푦 1 - a point in inhomogeneous coordinates - a point in homogeneous coordinates 3 ~ 2 3 4.5 6 2 6/2 푝 = ∈ 푅 푝 = 2 = 3 = 4 ∈ 푃 푝~ = ∈ 푅2 2 1 1.5 2 4/2 w-axis (6,4,2) (3,2,1) Plane w=1 (Space 푹ퟐ) Plane w=0 Computer Vision I: Image Formation Process 22/12/2014 9 Things we would like to have and do We look at these operations in: 푅2/푅3, 푃2/푃3 Primitives: • Points • Lines • Conics (Quadric in 3D) • (Planes in 3D) Operations with Primitives: • intersection • tangent Transformations: • Rotation • Translation • projective • …. Computer Vision I: Image Formation Process 22/12/2014 10 Why bother about 푃2? • All Primitives, operations and transformations are defined in 푅2 and 푃2 • Advantage of 푃2: • Many transformation and operations are written more compactly (e.g. linear transformations) • We will introduce new special “primitives” that are useful when dealing with “parallelism” This example will come 2 later in detail. In 푃 they meet in a „point at inifinty“ Two parallel lines in 푅2 Computer Vision I: Image Formation Process 22/12/2014 11 Points at infinity 푥 w-axis 푦 푤 All rays go through (0,0,0) define a point in 푃2 Plane w=1 Plane w=0 푥 푦 0 푥 Points with coordinate 푦 are ideal points or points at infinity 0 푥 푦 ∈ 푃2 Not defined in 푅2 푠푖푛푐푒 푤 = 0 0 Computer Vision I: Image Formation Process 22/12/2014 12 Lines in 푅2 • For Lines in coordinate space 푅2 we can write 푡 푙 = (푛푥, 푛푦, 푑) with 푛 = 푛푥, 푛푦 is normal vector and ||푛|| = 1 • A line has 2 DoF • A point (푥, 푦) lies on 푙 if: 푛푥 푥 + 푛푦 푦 + 푑 = 0 • Normal can also be encoded with an angle 휃: 푛 = cos 휃 , sin 휃 푡 Computer Vision I: Image Formation Process 22/12/2014 13 Lines in 푃2 • Points in 푃2: 풙 = (푥, 푦, 푤) • Lines in 푃2: 풍 = (푎, 푏, 푐) (again equivalent class: 푎, 푏, 푐 = 푘푎, 푘푏, 푘푐 ∀ 푘 ≠ 0 ) Hence also 2 DoF • All points 푥, 푦, 푤 on the line (푎, 푏, 푐) satisfy: 푎푥 + 푏푦 + 푐푤 = 0 this is the equation of a plane in 푅3 with normal (a,b,c) going through (0,0,0) Computer Vision I: Image Formation Process 22/12/2014 14 Converting Lines between 푃2 and 푅2 • Points in 푃2: 풙 = (푥, 푦, 푤) • Lines in 푃2: 풍 = (푎, 푏, 푐) From 푃2 to 푅2: 풍 = (푘푎, 푘푏, 푘푐) chose 푘 such that ||(푘푎, 푘푏)|| = 1 From 푅2 to 푃2: 2 풍 = 푛푥, 푛푦, 푑 is already a line in 푃 Computer Vision I: Image Formation Process 22/12/2014 15 Line at Infininty • There is a “special” line, called line at infinity: (0,0,1) • All points at infinity 푥, 푦, 0 lie on the line at infinity (0,0,1): 푥 ∗ 0 + 푦 ∗ 0 + 0 ∗ 1 = 0 0 0 vector for line at infinity w-axis 1 Plane w=1 Plane w=0 푥 푦 A point at infinity (w=0) 0 Computer Vision I: Image Formation Process 22/12/2014 16 A Line defined by two points in 푃2 • The line through two points 풙 and 풙′ is given as: 풍 = 풙 × 풙’ • Proof: It is: 풙 풙 × 풙’ = 풙′ 풙 × 풙′ = ퟎ vectors are orthogonal This is the same as: 풙 풍 = 풙′풍 = ퟎ Hence, the line 풍 goes through points 풙 and 풙′ 풙 풙′ Computer Vision I: Image Formation Process 22/12/2014 17 The Intersection of two lines in 푃2 • Intersection of two lines 풍 and 풍’ is the point 풙 = 풍 × 풍’ • Proof: It is: 풍 풍 × 풍’ = 풍′ 풍 × 풍’ = ퟎ vectors are orthogonal This is the same as: 풍풙 = 풍′풙 = ퟎ Hence, the point 풙 lies on the lines 풍 and 풍′ Note the „Theorem“ and Proofs have been very similiar, we only interchanged meaning of points and lines Computer Vision I: Image Formation Process 22/12/2014 18 Duality of points and lines • Note 풍풙 = 풙풍 = ퟎ (풙 and 풍 are “interchangeable”) • Duality Theorem: Two any theorem of 2D projective geometry there corresponds a dual theorem, which may be derived by interchanging the roles of points and lines in the original theorem. The intersection of two lines 풍 and 풍’ is the point 풙 = 풍 × 풍’ The line through two points 풙 and 풙′ is the line 풍 = 풙 × 풙’ Computer Vision I: Image Formation Process 22/12/2014 19 Parallel lines meet in a point at Infinity 풍 1 2 풍 = 0 ; 풍′ = 0 풍’ 1 1 풍 풍’ In 푅2 (Plane 푤 = 1) intersection 푦 0 −1 0 2 0 ′ (-1,1,1) (-1/2,1,1) 풍 × 풍 = 1 0 −1 0 = 1 0 1 0 1 0 (-1,0,1) (-1/2,0,1) 푥 Point at infinty Computer Vision I: Image Formation Process 22/12/2014 20 2D conic “Kegelschnitt” • Conics are shapes that arise when a plane intersects a cone • In compact form: 풙풕푪 풙 = ퟎ where C has the form: 푎 푏/2 푑/2 C = 푏/2 푐 푒/2 푑/2 푒/2 푓 • This can be written in in-homogenous coordinates: 푎푥2 + 푏푥푦 + 푐푦2 + 푑푥 + 푒푦 + 푓 = 0 where 풙~ = (푥, 푦) • 푪 has 5DoF since unique up to scale: 풙풕푪 풙 = 푘풙풕푪 풙 = 풙풕푘푪 풙 = ퟎ • Properties: 풍 is tangent to 푪 at a point 풙 if 풍 = 푪풙 Computer Vision I: Image Formation Process 22/12/2014 21 Example: 2D Conic y A circle: 2 2 2 r 푥 + 푦 − 푟 = 0 x Parabola: −푥2 + 푦 = 0 Computer Vision I: Image Formation Process 22/12/2014 22 Define a conic with five points Given 5 points (푥푖, 푦푖, 1) we can write: as with That gives: This is a 5 × 6 matrix. The 1D null-space gives the conic up to scale. Compute null-space with Gaussian elimination or SVD (to come later) Computer Vision I: Image Formation Process 22/12/2014 23 2D Transformations 2D Transformations in 푅2 Definition: • Euclidean: translation + rotation • Similarity (rigid body transform): Euclidean + scaling • Affine: Similarity + shearing • Projective: arbitrary linear transform in homogenous coordinates Computer Vision I: Image Formation Process 22/12/2014 24 2D Transformations of points • 2D Transformations in homogenous coordinates: 푥′ 푎 푏 푑 푥 푦′ = 푒 푓 ℎ 푦 1 푤′ 푖 푗 푙 Transformation matrix • Example: translation 1 0 푡푥 푥 푥′ 푦 푥′ 푥 푡 푦′ = 0 1 푡푦 = + 푥 1 푦′ 푦 푡푦 1 0 0 1 homogeneous coordinates inhomogeneous coordinates Advantage of homogeneous coordinates (going into 푃2) Computer Vision I: Image Formation Process 22/12/2014 25 2D Transformations of points (of a square) (two special points on the line at infity ) Here 푟푖푗 is a 2 x 2 rotation matrix with 1 DoF cos Θ −sin Θ which can be written as: sin Θ cos Θ [from Hartley Zisserman Page 44] Computer Vision I: Image Formation Process 22/12/2014 26 2D transformations of lines and conics All points move: 풙‘ = 푯풙 then: 풙′ 1) Line (defined by points) moves: 풙 ′ −1 t 풍 = (푯 ) 풍 푯 풍 풍′ 2) conic (defined by points) moves: 푪′ = (푯−1)t 푪 푯−1 Proof: 1) Assume 풙1 and 풙2 lie on 풍.