2: Vanishing points and horizons. Applications of projective transformations. Lecture 1: Euclidean, similarity, affine and projective transformations. Homo- geneous coordinates and matrices. Coordinate frames. Perspective projection and its matrix representation. Lecture 2: Vanishing points. Horizons. Applications of projective transfor- mations. Lecture 3: Convexity of point-sets, convex hull and algorithms. Conics and quadrics, implicit and parametric forms, computation of intersections. Lecture 4: Bezier curves, B-splines. Tensor-product surfaces. 1 Recall: Perspective (central) projection — 3D to 2D 2.1 Y The camera model Mathematical ideal- ized camera 3D ¡ 2D X X y Image coordinates ¢¤£ x Camera frame ¥§¦©¨ (origin at optical x centre) O f ¨ Focal length , image plane is at Z . image Similar triangles plane ¥ £ ¦ ¢ Y y ¨ ¨ ¨ f where and are 3-vectors, with Z ¢£ ¥¦ ¨! , . Vanishing Points 2.2 2 Vanishing Points in 1D and 2D 2.3 x 1D image O Vanishing point Point at Infinity 1D world image plane vanishing point v horizon O line vanishing point parallel lines Ground plane All parallel lines meet at the same vanishing point 2.4 A line of 3D points isrepresented as -, ¨ Using the vanishing point of ()"+* #"$% '& its image is & (:"¤* . ;=< < /1032 #"$9 4576+8 (:"$> λ λ X( ) D @ GH = 0 ? < , @ H @ H >CB > < , * < >ED > A A I F > V O J. depends only on the direction * , not on & . Parallel lines have the same vanishing point. 3 The Homogeneous KMLON Projection Matrix 2.5 Y X X y x x ¨ O f F Z Choose P from now on. -, Q¢SRTT¢VUWT¢¤XT ¨ Homogeneous image coordinates correctly represent if GH ?@ H @ GH ?@ [\ _a` F ^ ^ ^ H @ ¥ H @ \ ` H ? G @ H @ \ ` ¢YR H @ ^ F ^ ^ H @ ¦ A I F U ced©fhgji ¢ I A Z ] b ^ ^ F ^ ¨ I A F X ¢ because then R U ¢ ¥ ¢ ¦ ¢P £k ¢¤X ¨ ¢VX ¨ Then perspective projection is a linear map, represented by a l§mMn projection matrix, from 3D to 2D. Vanishing points using homogeneous notation 2.6 * A line of points in 3D through the point & with direction is Qo & (poY* Z Writing this in homogeneous notation ? G @ H @ H @ H R ¥ Qo @ ? ? G ? G ? H G G F @ H @ H U ¥ Qo & & * * A A I A I A I I F F ^ ^ (vo ( Z Z ¥qXrQo o A I ¥tsuQo ?@ GH ¡ w Atxzy I In the limit o the point on the line is ^ So, homogeneous vectors with ¥qs{ represent points “at infinity”. Points at infinity are equivalent to directions 4 Vanishing points using homogeneous notation: Example 2.7 w The vanishing point of a line with direction * is the image of the point at ... [\ _a` ?@ GH F ^ ^ ^ \ ` @ H ? G \ ` @ H >CB ^ F ^ ^ * . A I ^ >|D A I ] b < ^ ^ F ^ > Exercise: Compute the vanishing points of lines on an ¥}¨ plane: n~ ¨ ¥ (1) parallel to the ¨ axis; (2) at to the axix; (3) parallel to the axis. ?@ GH @ H ?@ GH @ H A 1: I I 1 A y D = 0 GH @ 3 0 ? GH H ?@ x @ H Z @ 1 I 2 A I D 2 = 0 A 1 GH ?@ G ? H H @ 0 @ H X @ D = 0 Y 1 I 1 3: A I A Z The advantages of homogeneous notation ... 2.8 Jump you daft cat. There are two advantages of using homogeneous notation to represent perspective projection: 1. Non-linear projections equations are turned into linear equations. The tools of linear algebra can then be used. 2. Vanishing points are treated naturally, and awkward limiting procedures are then avoided. 5 Plane to Plane Projective Transformations 2.9 ? G @ H @ H ? G @ H @ H R ¥ @ H @ H @ H R ¢ H O @ @ H U Z ¥ U ¢ ' I Z=0 plane A ¥qX A I X ¢¤X Y ¥ts Choose the world coordinate system such that the world plane has zero ¨ co- lmpl ordinate. Then the lmn matrix reduces to a plane to plane projective transformation. ?@ GH @ H GH [\ _a` [\ _a` ?@ GH ?@ @ H ¥ H \ ` \ ` @ H @ @ H H \ ` \ ` @ H @ RR RU RQX Rs RR RU Rs R ¥ ¢ @ H @ H ¦ ^ UR UU UX U#s UR UU U#s U ¦ ¢ I A I A ] b ] b F A I F XR XU XX Xs XR XU Xs X ¢ This is the most general transformation between the world plane and image plane under imaging by a perspective camera. A projective transformation is also called a “homography” and a “collineation”. 6 Computing a projective transformation 2.10 G ? [ _ ? G H @ \ ` @ H H @ \ ` @ H H @ \ ` @ H ¢+ RR RU RX ¢YR R x ¢ UR UU UX ¢VU U Z I A A I ] b ¢+ XR XU XX ¢¤X x X ?@ GH @ H @ H O ¢YR ¢VU Z A y y I x ¢¤X x lml where is a non-singular homo- geneous matrix with EIGHT degrees of freedom. Each point correspondence gives two constraints ¢+ ¢+ RR¢©( RU£( RQX UR¢( UU£( UX R U ¢ £ XR XU XX XR XU XX ¢ ¢©( £( ¢ ¢( £( X X and multiplying out give two equations linear in the elements of XR XU XX RR RU RQX ¢ ¢( £( ¢©( £( XR XU XX UR UU UX £ ¢( £( ¢©( £( Simple Example 2.11 Q¢V1£ Suppose the correspondences Q¢£+ are known for four points (no three collinear), then is determined uniquely. (0,1) (1,1) (0,1) (2,1) H (0,0) (1,0) (0,0) (1,0) ^ ^ ^ ^ ¡ F ^ F ^ ¡ First correspondence Second correspondence ?@ GH [\ _a` ?@ GH ?@ GH ^ ^ @ H \ ` @ H @ H ?@ GH [\ _a` ?@ GH ?@ GH ¡ ¢ r £ ¢ @ H \ ` @ H @ H RR RU RX RQX ^ ^ r rh¢ h u A IP A IP A I ] b "R UR UU UX UX A I A I A I ] b r¤ rh¤¢r¤¢¤ h¤ ¦¥qr¤¢¤ F F XR XU XX XX ^©¨¡ª+« ^ RR% XRS( XX Whence UR§ RX UX Whence . 7 Simple example Continued 2.12 ^ F ^ F ^ ¡ RU¬ UU¬ XU­( XX Third correspondence gives and . F F F ¡ #® Fourth correspondence ? G [ _ ? G ? G @ H \ ` @ H @ H ^ ^ F @ H \ ` @ H @ H @ H \ ` @ H @ H XR XX XR XX ® ( ( F ^ ^ F s XU XX XU XX " ( ( A I A I A I ] b F F XR XU XX XRS( XU­( XX ¯ Take ratios ¯ 2 equations in 3 unknowns solve for ratio of matrix elements only. [ _ [ _ \ ` \ ` ^ ^ ^ ^ \ ` \ ` \ ` \ ` ® ® ^ F ^ ^ F ^ k " Z ] b ] b ^ °EF ^ °F ® ® Computational Algorithm 2.13 The equations, XR XU XX RR RU RQX ¢ ¢( £( ¢©( £( XR XU XX UR UU UX £ ¢( £( ¢©( £( can be rearranged as _ [ F ^ ^ ^ ° ° ° ¢ £ ¢ ¢ ¢ £ ¢ b9± ] ^ ^ ^ F ° ° ° g ¢ £ £ ¢ £ £ £ ± RRT RUW RX² URT UUW UX² XRT XUW XX³ where is the matrix written as a 9- vector. 8 continued ... 2.14 For 4 points, [\ _a` F ^ ^ ^ ° ° ° \ ` \ ` ¢YR£¦R ¢ ¢YR ¢ £¦R ¢ R R R \ ` ^ ^ ^ F ° ° ° \ ` \ ` ¢SR´£µR £¦ ¢YR £ £¦R £ R R R \ ` F ^ ^ ^ ° ° ° \ ` \ ` ¢VU £U ¢ ¢VU ¢ £U ¢ U U U \ ` ^ ^ ^ F ° ° ° \ ` \ ` U U U U ¢ £ £ ¢ £ £ £ U U U ± \ ` F ^ ^ ^ ° ° ° g \ ` \ ` X X X X ¢ £ ¢ ¢ ¢ £ ¢ X X X \ ` ^ ^ ^ F ° ° ° \ ` \ ` X X X X ¢ £ £ ¢ £ £ £ X X X F ^ ^ ^ ° ° ° s s s s ¢ £ ¢+ ¢ ¢+ £ ¢+ s s s ] b ^ ^ ^ F ° ° ° ¢¤s £¶s £ ¢+s £ £us £ s s s ± ± g · ¸mp¹ which has the form · , with a matrix. The solution is the (one dimensional) null space of · . If using many points, one can use least squares. Solution best found then using ± º»¡¼j½)¾ · ¼ SVD of · — ie Then is the column of corresponding to smallest singular value. (The smallest singular value would be zero of all the data were exact ...) Some Matlab 2.15 npoints = 4 (or 5 later -- 5th point is noisy) x = [0,1,0,1, 1.01]; y = [0,0,1,1, 0.99]; xd = [0,1,0,2, 2.01]; yd = [0,0,1,1, 1.01]; A = zeros(2*npoints,9); for i=1:npoints, A(2*i-1,:)= [x(i),y(i),1,0,0,0, -x(i)*xd(i),-xd(i)*y(i),-xd(i)]; A(2*i, :)= [0,0,0,x(i),y(i),1, -x(i)*yd(i),-yd(i)*y(i),-yd(i)]; end; if npoints==4 h = null(A); else [U,S,V] = svd(A); h=V(:,9); end; H=[h(1),h(2),h(3);h(4),h(5),h(6);h(7),h(8),h(9);]; With the 4 exact points ... Adding the fifth noisy point ... [\ _a` [\ _a` ^ °Á^ ^¡^À^À^ ^ ^À^¡^À^ ^ °Á^ ^À^¡^À^ °=^ ^À^À^À^ \ ` \ ` \ ` \ ` l¦®À~ ®¹¦~ ^ ^À^À^À^ F ^¡^À^À^ °Á^ ^À^¡^À^ °Á^ ^¡^À^¤F ^ ^ ^À^À^¤F Zz¿ Z Z Z¿ Z Z k k ¹À¹¡¹À¹ ] b ] b ^ ^À^À^À^ °Á^ F ^ °Á^ ^¡^ ^ °Á^ F ^ Z Z Z Z Z Z l ® l®À~ ~ l ~À~ lnÀn Z Z ¿ Z¿ Z Z Zz¿ 9 Example 1: Removing Perspective Distortion 2.16 Objective: Back project to world plane 1. Find Euclidean coordinates of four points on the flat object plane Q¢Ã#£¶ÃÄ . Q¢ £ à 2. Measure the corresponding image coordinates of these four points à . à à Q¢ £ Q¢ £ à 3. Compute from the four à . F F R Å¤Æ ¢+1£Ç 4. Euclidean coords of any image point are ¢£ . The image can be warped onto the world plane using . How? Moving the image plane 2.17 An image is the intersection of a plane with the cone of rays between points in 3-space and the optical centre. Any two such “images” (with the same optical centre) are related by a planar projective transformation. X x x As the camera is rotated the points of intersection of the rays with the image plane are related by a planar projective transformation. Image points and correspond to the same scene point . 10 Relationship in terms of rotations 2.18 U For corresponding points R and in two views 1 and 2, ?@ GH @ H [\ _a` ?@ GH ?@ GH F ^ ^ ^ @ H ¥ \ ` @ H @ H [ _ @ H \ ` @ H @ H ¥ ¥ @ H ^ F ^ ^ @ H R g ¦ ]=È b F ^ R U ¦ R ¦ U Z Z È A I È A I ] b ^ ^ F ^ F F g A I F Hence R Æ U U R R È È The cameras could have different focal lengths — so one can do all of this while rotationing and zooming.