Geometry of a single view (a single camera case) Václav Hlaváč Czech Technical University in Prague Czech Institute of Informatics, Robotics and Cybernetics 160 00 Prague 6, Jugoslávských partyzánů 1580/3, Czech Republic http://people.ciirc.cvut.cz/hlavac, [email protected] also Center for Machine Perception, http://cmp.felk.cvut.cz Courtesy: T. Pajdla Outline of the talk: Projectivity Homography Camera calibration Projective space Projective camera Radial distortion Perspective transformation, motivation 2/53 Parallel lines do not look like parallel lines under the perspective projection. Pin-hole camera model 3/53 (straight line) ray pinhole three-dimensional scene image plane Image function 4/53 The image function is abstracted mathematically as f(x, y), f(x, y, t). It is the result of the perspective projection encompassing geometric aspects. X=[x,y,z]T Y u=[u,v]T X y x v u Z -u -v f f reflected image plane image plane x f y f Considering similar triangles: u = z , v = z . Instead of our derived 2D image function f(u, v), it is usually denoted f(x, y). The value of the image function matches color/intensity of a 3D point (a red dot in the figure above) in the scene, which is projected. Basics of projective geometry 5/53 Pinhole model - the simplest geometrical model of human eye, photographic and TV camera. Perspective projection, also central projection. Parallel lines in the world do not remain parallel in the image (e.g., view along the straight section of a railroad). Image plane Horizon Optical axis Focal point 1 vanishing point 2 vanishing points Vanishing point 3 vanishing points Base or ground plane Multiple view geometry 6/53 3D points in the scene (and, more generally, lines and other simple geometric objects), their camera projections, and relations among multiple camera projections of a 3D scene. Projective space 7/53 Consider (d + 1)-dimensional vector space without its origin, Rd+1 − {(0,..., 0)}. Define the equivalence relation > 0 0 > [x1, . , xd+1] ≡ [x1, . , xd+1] > 0 0 > iff ∃ α 6= 0 : [x1, . , xd+1] = α [x1, . , xd+1] d Projective space P is the quotient space of this equivalence relation. Points in the projective space are expressed in homogeneous co-ordinates (called also 0 0 > projective coordinates) x = [x1, . , xd, 1] . Relation between Euclidean and projective spaces 8/53 d d Consider Euclidean space R . Non-homogeneous coordinates represent a point in R d+1 occupying the plane with the equation xd+1 = 1 in R . d d > > There is a one-to-one mapping from the R into P , i.e. [x1, . , xd] → [x1, . , xd, 1] . > Projective points [x1, . , xd, 0] do not have the Euclidean counterpart and represent points at infinity in a particular direction. > > Consider [x1, . , xn, 0] as a limiting case of [x1, . , xn, α] that is projectively equivalent > to [x1/α, . , xn/α, 1] , and assume that α → 0. d This corresponds to a point in R going to infinity in the direction of the radius vector d [x1/α, . , xd/α] ∈ R . Example of a finite line in 2D (image) plane with coordinates (u, v): a u + b v + c = 0. The line corresponds to a (homogeneous) vector l (pronounced el), l ' (a, b, c). There is an equivalence class for α ∈ R, α 6= 0, (α a, α b, α c) ' (a, b, c). Homogeneous coordinates of hyperplanes in Pd 9/53 d > A hyperplane in P is represented by the (d + 1)-vector a = [a1, . , ad+1] such that all points x lying on the hyperplane satisfy a>x = 0 (where a>x denotes the scalar product). > Considering the points in the form x = [x1, . , xd, 1] yields the familiar formula a1x1 + ··· + adxd + ad+1 = 0. The hyperplane defined by d distinct points represented by vectors x1,..., xd lying on it is represented by a vector a orthogonal to vectors x1,..., xd. This vector a can be computed, e.g., by SVD. Symmetrically, the point of intersection of d distinct hyperplanes a1,..., ad is the vector x orthogonal to them. The first useful hyperplane in computer vision: 2 the projective plane P 10/53 2 > 2 We will denote points in P by u = [u, v, w] , lines (a special case of hyperplanes) in P by l (pronounced el). The symbol × stands for the vector product here. The line passing through two points x, y (also called point join) is l = x × y. The point as intersection of two lines l, m is x = l × m. x3 2 p Projective space P , illustration Points and lines in P2 are represented by rays and pla- 1 nes, respectively, which pass through the origin in the corresponding Euclidean space R3. origin O x1 x2 The second useful hyperplane in computer vision: P3 11/53 3 > We will denote points in P by X = [X, Y, Z, W ] . 3 In P , hyperplanes become planes and one more entity occurs that has no counterpart in the projective plane: a 3D line. 3 The elegant homogeneous representation by 4-vectors, available for points and planes in P , does not exist for lines. A 3D line can be represented either by a pair of points lying on it but this representation is not unique, or by a (Grassmann-)Plücker matrix. Homography 12/53 In computer vision, any two (pin-hole camera) images of the same planar surface in 3D are related by homography. Said more generally from projective geometry standpoint: Homography is an isomorphism of projective spaces that maps lines to lines. Called also projective transformation or collineation. d d d+1 Co-lineation is any mapping P → P linear in the embedding space R . 0 Co-lineation is defined up to unknown scale as u ' H u, where H is a (d + 1) × (d + 1) matrix. The transformation maps any triplet of collinear points to a triplet of collinear points (hence one of its names—collineation). If H is regular then distinct points are mapped to distinct points. 2 0 In P , homography is the most general transformation which maps lines to lines. u ' H u, where H is a regular 3×3 matrix. Matrix H has 9 parameters. 8 parameters are independent. The scale is arbitrary. Example of two images mapped by a 2D homography 13/53 Different form of homography for hyperplanes and points 14/53 It can be derived from the fact that if the original point u and a hyperplane a are incident then a>u = 0. The point u and the hyperplane a have to remain incident after the transformation too, a0>u0 = 0. 0 0 −> −> Using equation u ' H u, we obtain that a ' H a, where H denotes the transposed inverse of H. Two simple homographies useful in computer vision 15/53 1.A planar scene and its projection by one pinhole camera are related by a 2D homography. This can be used to rectify images of planar scenes (e.g., building facades) to a frontoparallel view. 2. Two images of a 3D scene (planar or non-planar) by two pinhole cameras sharing a single center of projection is a 2D homography. This can be used for stitching panoramic images from photographs. Homography vs. non-homography (1) 16/53 > Let us illustrate how the non-homogeneous 2D point [u, v] (e.g., a point in an image) is actually mapped to the non-homogeneous image point [u0, v0]> by H using u0 ' H u. With the components and the scale written explicitly, the equation reads 0 u h11 h12 h13 u 0 α v = h21 h22 h23 v , α 6= 0 . 1 h31 h32 h33 1 Homography vs. non-homography (2) 17/53 Writing 1 in the third coordinate of u0, we tacitly assume that u0 is not a point at infinity, that is, α 6= 0. To compute [u0, v0]>, we need to eliminate the scale α. This yields the expression h u + h v + h h u + h v + h u0 = 11 12 13 , v0 = 21 22 23 , h31u + h32v + h33 h31u + h32v + h33 familiar to people who do not use homogeneous coordinates. Note that compared to this, the expression u0 ' H u is simpler, linear, and can handle the case when u0 is a point at infinity. These are the practical advantages of homogeneous coordinates. Cross ratio 18/53 A’ A a’ a B B’ 0 d b a a b’ 0 C d’ b b c C’ c = c0 D d d0 c’ D’ Subgroups of homographies 19/53 Decomposition of homographies 20/53 Any homography can be uniquely decomposed as H = HP HA HS, where I 0 K 0 R −Rt H = ,H = ,H = , P a> b A 0> 1 S 0> 1 Matrix K is upper triangular. Matrices of the form of HS represent Euclidean transformations. Matrices HAHS represent affine transformations; thus matrices HA represent the ‘purely affine’ subgroup of affine transformations, i.e., what is left of the affine group after removing from it (more exactly, factorizing it by) the Euclidean group. Matrices HP HAHS represent the whole group of projective transformations; thus matrices HP represent the ‘purely projective’ subgroup of the projective transformation. 2D homography 21/53 a ny plane 2D homography maps a plane to a plane. i u’ Recall two views of the Prague Castle at the slide 13. k j v’ w’ 1 u0 u α v0 = H v , α 6= 0 , 1 1 u where H – [3 × 3] is the homography matrix. v 1 camera camera center Example: Distance measurement in a plane 22/53 π u’ 1 0 u’4 We know coordinates of four points ui, i = 1,..., 4 in a plane in which we intend to measure distances.