Projective 2D geometry course 2 Multiple View Geometry Comp 290-089 Marc Pollefeys Content • Background: Projective geometry (2D, 3D), Parameter estimation, Algorithm evaluation. • Single View: Camera model, Calibration, Single View Geometry. • Two Views: Epipolar Geometry, 3D reconstruction, Computing F, Computing structure, Plane and homographies. • Three Views: Trifocal Tensor, Computing T. • More Views: N-Linearities, Multiple view reconstruction, Bundle adjustment, auto- calibration, Dynamic SfM, Cheirality, Duality Multiple View Geometry course schedule (tentative) Jan. 7, 9 Intro & motivation Projective 2D Geometry Jan. 14, 16 (no course) Projective 2D Geometry Jan. 21, 23 Projective 3D Geometry Parameter Estimation Jan. 28, 30 Parameter Estimation Algorithm Evaluation Feb. 4, 6 Camera Models Camera Calibration Feb. 11, 13 Single View Geometry Epipolar Geometry Feb. 18, 20 3D reconstruction Fund. Matrix Comp. Feb. 25, 27 Structure Comp. Planes & Homographies Mar. 4, 6 Trifocal Tensor Three View Reconstruction Mar. 18, 20 Multiple View Geometry MultipleView Reconstruction Mar. 25, 27 Bundle adjustment Papers Apr. 1, 3 Auto-Calibration Papers Apr. 8, 10 Dynamic SfM Papers Apr. 15, 17 Cheirality Papers Apr. 22, 24 Duality Project Demos Projective 2D Geometry • Points, lines & conics • Transformations & invariants • 1D projective geometry and the Cross-ratio Homogeneous coordinates Homogeneous representation of lines ax+ by+ = 0 c (a,b,c)T ka( ) x (+ kb )+ y= kc 0∀ ,≠ a,b,c 0 k( )T ~ k( )T a,b,c equivalence class of vectors, any vector is representative Set of all equivalence classes in R3−(0,0,0)T forms P2 Homogeneous representation of points T T x=x( , y) on l = (a,b,c) if and onlyax if + by+ = 0 c x,y,()1 ( a,b,c)T=(x,y,1 ) l =,x 0 , y() 1 ~T k( x, ,) yT∀ 1 , ≠ k 0 The point x lies on the line l if and only if xTl=lTx=0 T Homogeneous coordinatesx()1,, x 2 x 3 but only 2DOF Inhomogeneous coordinates x(), yT Points from lines and vice-versa Intersections of lines The intersection of two lines l and l' isx = l× l' Line joining two points The line through two points x and x' lis = x× x' Example y =1 x =1 Ideal points and the line at infinity Intersections of parallel lines T T T l ,=a( , b) cand =l'a( b ,) c ,l '× l' =b( , a− ,) 0 Example b( ,− a) tangent vector a( , b) normal direction x =1 x = 2 T Ideal points x(,1 x , 2 0) T Their inner product is zero Line at infinityl∞ 0= ( , 0) , 1 2 2 Note that in P2 there is no distinction PR= ∪l∞ between ideal points and others In projective plane, two distinct in a single point andlines meet ne Two distinct points lie on a single liÎ not true in R^2 A model for the projective plane Points in P^2 are represented by rays passing through origin in R^3. Lines in P^2 are represented by planes passing through origin Points and lines obtained by intersecting rays and planes by plane x3 = 1 Lines lying in the x1 – x2 plane are ideal points; x1-x2 plane is l_{infinity} Duality Roles of points and lines can be interchanged x l xT l= 0 lT x= 0 x= l× l' l= x× x' Duality principle: To any theorem of 2-dimensional projective geometry there corresponds a dual theorem, which may be derived by interchanging the role of points and lines in the original theorem Conics Curve described by 2nd-degree equation in the plane ax+ bxy2 + cy2 + + dx +0 ey = f x x or homogenized x a 1 , y a 2 x3 x3 2 2 2 ax bx+1 x1 + 2 cx + 2 1 dx 3 + x 2 3 ex + 30 x = fx or in matrix form ⎡a/ b 2 d/⎤ 2 T with ⎢ ⎥ xC x= 0 C =b/⎢ 2 c / e ⎥ 2 d/⎣⎢ 2 e / 2 f⎦⎥ 5DOF:a :::::{ b c d e} f Five points define a conic For each point the conic passes through 2 2 ax+ bxi y i + i cy + i i + dx i +0 ey = f or 2 2 T x,i( x i y , i y , i , i x ,i ) yc = f 0 ac =,,,,,( b c d) e f stacking constraints yields 2 2 x⎡ 1 x1 y 1 y 1 1 x 1 1⎤ y ⎢ 2 2 ⎥ x⎢ 2 x2 y 2 y 2 2 x 2 1⎥ y ⎢ 2 2 ⎥ x3 x3 y 3 y 3 3 x 3 1 yc = 0 ⎢ 2 2 ⎥ x⎢ 4 x4 y 4 y 4 4 x 4 1⎥ y ⎢ 2 2 ⎥ x⎣ 5 x5 y 5 y 5 5 x 5 1⎦ y Tangent lines to conics The line l tangent to C at point x on C is given by l=Cx l x C Dual conics Conic C, also called, “point cosdefines an equation on pointnic” Apply duality: dual conic or line conic defines an equation on lines A line tangent to the conic C satisfiesl T C* l= 0 C* is the adjoint of Mtrix C; defined in Appendix 4 of H&Z In general (C full rank): CC*= − 1 Dual conics = line conicsx = conic envelopes Points x T Cx = 0 lie on a point linesl T C * = l0 are tangent conic to the point conic C; conic C is the envelope of lines l Degenerate conics A conic is degenerate if matrix C is not of full rank m e.g. two lines (rank 2) l C=lmTT + ml e.g. repeated line (rank 1) T C = ll l Degenerate line conics: 2 points (rank 2), double point (rank1) * Note that for degenerate conicsCC ()* ≠ Projective transformations Definition: A projectivity is an invertible mapping h from P2 to itself such that three points x1 ,x2 ,x3 lie on the same line if and only if h(x1 ),h(x2), h(x3) do. ( i.e. maps lines to lines in P2) Theorem: A mapping h:P2→P2 is a projectivity if and only if there exist a non-singular 3x3 matrix H such that for any point in P2 reprented by a vector x it is true that h(x)=Hx Definition: Projective transformation: linear transformation on homogeneous 3 vectors represented by a non singular matrix H ⎛ x' ⎞ h⎡ h h⎤⎛ x ⎞ ⎜ 1 ⎟ 11 12⎜ 13 1 ⎟ x' =h⎢ h h⎥ x ⎜ 2 ⎟ 21⎢ 22⎥⎜ 23 2 ⎟ or x'= H x ⎜ ⎟ ⎜ ⎟ ⎝ x'3 ⎠ h31⎣⎢ h 32 h⎦⎥⎝ 33x3 ⎠ 8DOF •projectivity=collineation=projective transformation=homography •Projectivity form a group: inverse of projectivity is also a projectivity; so is a composition of two projectivities. Projection along rays through a common point, (center of projection) defines a mapping from one plane to another Central projection maps points on one plane to points on another plane Projection also maps lines to lines : consider a plane through projection Center that intersects the two planes Î lines mapped onto lines Î Central projection is a projectivity Î central projection may be expressed by x’=Hx (application of theorem) Removing projective distortion Central projection image of a plane is related to the originial plane via a projective transformation Î undo it by apply the inverse transformation select four points in a plane with know coordinates xh' x+ h+ y h xh' x+ h+ y h x=' 1 = 11 12 13 y=' 2 = 21 22 23 xh'3 x31 + h32 y + 33 h xh'3 x31 + h32 y + 33 h xhxhyh'( 31 + 32 + 33 ) = 11 hxhyh+ 12 + 13 (linear in hij ) yhxhyh'( 31 + 32 + 33 ) = 21 hxhyh+ 22 + 23 (2 constraints/point, 8DOF ⇒ 4 points needed) Remark: no calibration at all necessary, better ways to compute (see later) Sections of the image of the ground are subject to another projective distortion Î need another projective transformation to correct that. More examples Projective transformation between two images Induced by a world plane Î concatenation of two Pojective transformations is also a proj. trans. Proj. trans. Between the image of a plane (end of the building) and the image of its Shadow onto another plane (ground plane) Proj. trans. Between two images with the same camera center e.g. a camera rotating about its center Transformation of lines and conics For points on a line l, the transformed points under proj. trans. also lie on a line; if point x is on line l, then transforming x, transforms l For a point transformation x'= H x Transformation for lines l'= H-T l Transformation for conics C'= H-T CH -1 Transformation for dual conics C'* = HC* T H A hierarchy of transformations Group of invertible nxn matrices with real element Î general linear group GL(n); Projective linear group: matrices related by a scalar multiplier PL(n) • Affine group (last row (0,0,1)) • Euclidean group (upper left 2x2 orthogonal) • Oriented Euclidean group (upper left 2x2 det 1) Alternative, characterize transformation in terms of elements or quantities that are preserved or invariant e.g. Euclidean transformations (rotation and translation) leave distances unchanged Similarity Affine projective Similarity: circle imaged as circle; square as square; parallel or perpendicular lines have same relative orientation Affine: circle becomes ellipse; orthogonal world lines not imaged as orthogonoal; But, parallel lines in the square remain parallel Projective: parallel world lines imaged as converging lines; tiles closer to camera larger image than those further away. Class I: Isometries: preserve Euclidean distance (isoe, m=sametric=measure) ⎛ x'⎞ εcos⎡ θ− sin θt ⎤⎛ x ⎞ ⎜ ⎟ x ⎜ ⎟ y' =εsin⎢ θ cos θ t ⎥ y ⎜ ⎟ ⎢ y ⎥⎜ ⎟ ε = ±1 ⎜ ⎟ ⎜ ⎟ ⎝ 1 ⎠ 0⎣⎢ 0 1⎦⎥⎝ ⎠ 1 •orientation preserving: ε = 1 Î Euclidean transf. i.e. composition of translation and rotation Î forms a group •orientation reversing: ε = −1 Î reflection Î does not form a group R t ⎡ ⎤ T x'=H E x =⎢ T ⎥ x RRI= ⎣0 1⎦ on matrix; (ortR is 2x2 rotatihogonal, t is translation 2-veorl 2-vectctor, 0 is a nul 3DOF (1 rotation, 2 translation) Î trans.