DOCSLIB.ORG

Lecture 16: Planar Homographies Robert Collins CSE486, Penn State Motivation: Points on Planar Surface

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Lecture 16: Planar Homographies Robert Collins CSE486, Penn State Motivation: Points on Planar Surface Robert Collins CSE486, Penn State Lecture 16: Planar Homographies Robert Collins CSE486, Penn State Motivation: Points on Planar Surface y x Robert Collins CSE486, Penn State Review : Forward Projection World Camera Film Pixel Coords Coords Coords Coords U X x u M M V ext Y proj y Maff v W Z U X Mint u V Y v W Z U M u V m11 m12 m13 m14 v W m21 m22 m23 m24 m31 m31 m33 m34 Robert Collins CSE486, PennWorld State to Camera Transformation PC PW W Y X U R Z C V Rotate to Translate by - C align axes (align origins) PC = R ( PW - C ) = R PW + T Robert Collins CSE486, Penn State Perspective Matrix Equation X (Camera Coordinates) x = f Z Y X y = f x' f 0 0 0 Z Y y' = 0 f 0 0 Z z' 0 0 1 0 1 p = M int ⋅ PC Robert Collins CSE486, Penn State Film to Pixel Coords 2D affine transformation from film coords (x,y) to pixel coordinates (u,v): X u’ a11 a12xa'13 f 0 0 0 Y v’ a21 a22 ya'23 = 0 f 0 0 w’ Z 0 0z1' 0 0 1 0 1 Maff Mproj u = Mint PC = Maff Mproj PC Robert Collins CSE486, Penn StateProjection of Points on Planar Surface Perspective projection y Film coordinates x Point on plane Rotation + Translation Robert Collins CSE486, Penn State Projection of Planar Points Robert Collins CSE486, Penn StateProjection of Planar Points (cont) Homography H (planar projective transformation) Robert Collins CSE486, Penn StateProjection of Planar Points (cont) Homography H (planar projective transformation) Punchline: For planar surfaces, 3D to 2D perspective projection reduces to a 2D to 2D transformation. Punchline2: This transformation is INVERTIBLE! Robert Collins CSE486, Penn State Special Case : Frontal Plane What if the planar surface is perpendicular to the optic axis (Z axis of camera coord system)? Then world rotation matrix simplies: Robert Collins CSE486, Penn State Frontal Plane So the homography for a frontal plane simplies: Similarity Transformation! Robert Collins CSE486, Penn State Convert to Pixel Coords Internal camera a11 a12 a13 v pixels parameters a21 a22 a23 0 0 1 u Robert Collins CSE486, Penn State Planar Projection Diagram Here’s where transformation groups get useful! Robert Collins CSE486, Penn State General Planar Projection H1 H A1 Robert Collins CSE486, Penn State General Planar Projection H1 H-1 A1 Robert Collins CSE486, Penn State Frontal Plane Projection S1 A A1 Robert Collins CSE486, Penn State Frontal Plane Projection S1 A-1 A1 Robert Collins CSE486, Penn State General Planar Projection H1 H2 H Robert Collins CSE486, Penn State Summary: Planar Projection Pixel coords Internal u params Perspective v projection y Homography x Point on plane Rotation + Translation Robert Collins CSE486, Penn StateApplying Homographies to Remove Perspective Distortion from Hartley & Zisserman 4 point correspondences suffice for the planar building facade Robert Collins CSE486, Penn State Homographies for Bird’s-eye Views from Hartley & Zisserman Robert Collins CSE486, Penn State Homographies for Mosaicing from Hartley & Zisserman Robert Collins CSE486, Penn State Two Practical Issues How to estimate the homography given four or more point correspondences (will derive L.S. solution now) How to (un)warp image pixel values to produce a new picture (last class) Robert Collins CSE486, Penn State Estimating a Homography Matrix Form: Equations: Robert Collins CSE486, Penn State Degrees of Freedom? There are 9 numbers h11,…,h33 , so are there 9 DOF? No. Note that we can multiply all hij by nonzero k without changing the equations: Robert Collins CSE486, Penn State Enforcing 8 DOF One approach: Set h33 = 1. Second approach: Impose unit vector constraint Subject to the constraint: Robert Collins CSE486, Penn State L.S. using Algebraic Distance Setting h33 = 1 Multiplying through by denominator Rearrange Robert Collins CSE486, Penn State Algebraic Distance, h33=1 (cont) 2N x 8 8 x 1 2N x 1 Point 1 Point 2 Point 3 Point 4 additional points Robert Collins CSE486, Penn State Algebraic Distance, h33=1 (cont) Linear 2Nx8 8x1 2Nx1 equations A h = b 8x2N 2Nx8 8x1 8x2N 2Nx1 Solve: AT A h = AT b 8x8 8x1 8x1 (AT A) h = (AT b) -1 h = (AT A) (AT b) Matlab: h = A \ b Robert Collins CSE486, Penn State Caution: Numeric Conditioning R.Hartley: “In Defense of the Eight Point Algorithm” Observation: Linear estimation of projective transformation parameters from point correspondences often suffer from poor “conditioning” of the matrices involves. This means the solution is sensitive to noise in the points (even if there are no outliers). To get better answers, precondition the matrices by performing a normalization of each point set by: • translating center of mass to the origin • scaling so that average distance of points from origin is sqrt(2). • do this normalization to each point set independently Robert Collins CSE486, Penn State Hartley’s PreConditioning -1 -1 PointSet1 H = T2 S2H? Hnorm S1 T1 PointSet2 Translate T center of mass T2 1 to origin Scale so average S1 point dist is sqrt(2) S2 Estimate Hnorm Robert Collins CSE486, Penn State A More General Approach What might be wrong with setting h33 = 1? If h33 actually = 0, we can’t get the right answer. Robert Collins CSE486, Penn State Algebraic Distance, ||h||=1 ||h|| = 1 Multiplying through by denominator Rearrange = 0 = 0 Robert Collins CSE486, Penn StateAlgebraic Distance, ||h||=1 (cont) 2N x 9 9 x 1 2N x 1 4 P O I N T S additional points Robert Collins CSE486, Penn StateAlgebraic Distance, ||h||=1 (cont) Homogeneous 2Nx9 9x1 2Nx1 equations A h = 0 Solve: 9x2N 2Nx9 9x1 9x2N 2Nx1 AT A h = AT 0 9x9 9x1 9x1 (AT A) h = 0 SVD of ATA = U D UT Let h be the column of U (unit eigenvector) associated with the smallest eigenvalue in D. (if only 4 points, that eigenvalue will be 0).
Load more

Recommended publications
  • Globally Optimal Affine and Metric Upgrades in Stratified Autocalibration
    Globally Optimal Affine and Metric Upgrades in Stratified Autocalibration Manmohan Chandrakery Sameer Agarwalz David Kriegmany Serge Belongiey [email protected] [email protected] [email protected] [email protected] y University of California, San Diego z University of Washington, Seattle Abstract parameters of the cameras, which is commonly approached by estimating the dual image of the absolute conic (DIAC). We present a practical, stratified autocalibration algo- A variety of linear methods exist towards this end, how- rithm with theoretical guarantees of global optimality. Given ever, they are known to perform poorly in the presence of a projective reconstruction, the first stage of the algorithm noise [10]. Perhaps more significantly, most methods a pos- upgrades it to affine by estimating the position of the plane teriori impose the positive semidefiniteness of the DIAC, at infinity. The plane at infinity is computed by globally which might lead to a spurious calibration. Thus, it is im- minimizing a least squares formulation of the modulus con- portant to impose the positive semidefiniteness of the DIAC straints. In the second stage, the algorithm upgrades this within the optimization, not as a post-processing step. affine reconstruction to a metric one by globally minimizing This paper proposes global minimization algorithms for the infinite homography relation to compute the dual image both stages of stratified autocalibration that furnish theoreti- of the absolute conic (DIAC). The positive semidefiniteness cal certificates of optimality. That is, they return a solution at of the DIAC is explicitly enforced as part of the optimization most away from the global minimum, for arbitrarily small .
    [Show full text]
  • Projective Geometry: a Short Introduction
    Projective Geometry: A Short Introduction Lecture Notes Edmond Boyer Master MOSIG Introduction to Projective Geometry Contents 1 Introduction 2 1.1 Objective . .2 1.2 Historical Background . .3 1.3 Bibliography . .4 2 Projective Spaces 5 2.1 Definitions . .5 2.2 Properties . .8 2.3 The hyperplane at infinity . 12 3 The projective line 13 3.1 Introduction . 13 3.2 Projective transformation of P1 ................... 14 3.3 The cross-ratio . 14 4 The projective plane 17 4.1 Points and lines . 17 4.2 Line at infinity . 18 4.3 Homographies . 19 4.4 Conics . 20 4.5 Affine transformations . 22 4.6 Euclidean transformations . 22 4.7 Particular transformations . 24 4.8 Transformation hierarchy . 25 Grenoble Universities 1 Master MOSIG Introduction to Projective Geometry Chapter 1 Introduction 1.1 Objective The objective of this course is to give basic notions and intuitions on projective geometry. The interest of projective geometry arises in several visual comput- ing domains, in particular computer vision modelling and computer graphics. It provides a mathematical formalism to describe the geometry of cameras and the associated transformations, hence enabling the design of computational ap- proaches that manipulates 2D projections of 3D objects. In that respect, a fundamental aspect is the fact that objects at infinity can be represented and manipulated with projective geometry and this in contrast to the Euclidean geometry. This allows perspective deformations to be represented as projective transformations. Figure 1.1: Example of perspective deformation or 2D projective transforma- tion. Another argument is that Euclidean geometry is sometimes difficult to use in algorithms, with particular cases arising from non-generic situations (e.g.
    [Show full text]
  • Robot Vision: Projective Geometry
    Robot Vision: Projective Geometry Ass.Prof. Friedrich Fraundorfer SS 2018 1 Learning goals . Understand homogeneous coordinates . Understand points, line, plane parameters and interpret them geometrically . Understand point, line, plane interactions geometrically . Analytical calculations with lines, points and planes . Understand the difference between Euclidean and projective space . Understand the properties of parallel lines and planes in projective space . Understand the concept of the line and plane at infinity 2 Outline . 1D projective geometry . 2D projective geometry ▫ Homogeneous coordinates ▫ Points, Lines ▫ Duality . 3D projective geometry ▫ Points, Lines, Planes ▫ Duality ▫ Plane at infinity 3 Literature . Multiple View Geometry in Computer Vision. Richard Hartley and Andrew Zisserman. Cambridge University Press, March 2004. Mundy, J.L. and Zisserman, A., Geometric Invariance in Computer Vision, Appendix: Projective Geometry for Machine Vision, MIT Press, Cambridge, MA, 1992 . Available online: www.cs.cmu.edu/~ph/869/papers/zisser-mundy.pdf 4 Motivation – Image formation [Source: Charles Gunn] 5 Motivation – Parallel lines [Source: Flickr] 6 Motivation – Epipolar constraint X world point epipolar plane x x’ x‘TEx=0 C T C’ R 7 Euclidean geometry vs. projective geometry Definitions: . Geometry is the teaching of points, lines, planes and their relationships and properties (angles) . Geometries are defined based on invariances (what is changing if you transform a configuration of points, lines etc.) . Geometric transformations
    [Show full text]
  • Chapter 3 Image Formation
    This is page 44 Printer: Opaque this Chapter 3 Image Formation And since geometry is the right foundation of all painting, I have de- cided to teach its rudiments and principles to all youngsters eager for art... – Albrecht Durer¨ , The Art of Measurement, 1525 This chapter introduces simple mathematical models of the image formation pro- cess. In a broad figurative sense, vision is the inverse problem of image formation: the latter studies how objects give rise to images, while the former attempts to use images to recover a description of objects in space. Therefore, designing vision algorithms requires first developing a suitable model of image formation. Suit- able, in this context, does not necessarily mean physically accurate: the level of abstraction and complexity in modeling image formation must trade off physical constraints and mathematical simplicity in order to result in a manageable model (i.e. one that can be inverted with reasonable effort). Physical models of image formation easily exceed the level of complexity necessary and appropriate for this book, and determining the right model for the problem at hand is a form of engineering art. It comes as no surprise, then, that the study of image formation has for cen- turies been in the domain of artistic reproduction and composition, more so than of mathematics and engineering. Rudimentary understanding of the geometry of image formation, which includes various models for projecting the three- dimensional world onto a plane (e.g., a canvas), is implicit in various forms of visual arts. The roots of formulating the geometry of image formation can be traced back to the work of Euclid in the fourth century B.C.
    [Show full text]
  • Efficient Learning of Simplices
    Efficient Learning of Simplices Joseph Anderson Navin Goyal Computer Science and Engineering Microsoft Research India Ohio State University [email protected] [email protected] Luis Rademacher Computer Science and Engineering Ohio State University [email protected] Abstract We show an efficient algorithm for the following problem: Given uniformly random points from an arbitrary n-dimensional simplex, estimate the simplex. The size of the sample and the number of arithmetic operations of our algorithm are polynomial in n. This answers a question of Frieze, Jerrum and Kannan [FJK96]. Our result can also be interpreted as efficiently learning the intersection of n + 1 half-spaces in Rn in the model where the intersection is bounded and we are given polynomially many uniform samples from it. Our proof uses the local search technique from Independent Component Analysis (ICA), also used by [FJK96]. Unlike these previous algorithms, which were based on analyzing the fourth moment, ours is based on the third moment. We also show a direct connection between the problem of learning a simplex and ICA: a simple randomized reduction to ICA from the problem of learning a simplex. The connection is based on a known representation of the uniform measure on a sim- plex. Similar representations lead to a reduction from the problem of learning an affine arXiv:1211.2227v3 [cs.LG] 6 Jun 2013 transformation of an n-dimensional ℓp ball to ICA. 1 Introduction We are given uniformly random samples from an unknown convex body in Rn, how many samples are needed to approximately reconstruct the body? It seems intuitively clear, at least for n = 2, 3, that if we are given sufficiently many such samples then we can reconstruct (or learn) the body with very little error.
    [Show full text]
  • Feature Matching and Heat Flow in Centro-Affine Geometry
    Symmetry, Integrability and Geometry: Methods and Applications SIGMA 16 (2020), 093, 22 pages Feature Matching and Heat Flow in Centro-Affine Geometry Peter J. OLVER y, Changzheng QU z and Yun YANG x y School of Mathematics, University of Minnesota, Minneapolis, MN 55455, USA E-mail: [email protected] URL: http://www.math.umn.edu/~olver/ z School of Mathematics and Statistics, Ningbo University, Ningbo 315211, P.R. China E-mail: [email protected] x Department of Mathematics, Northeastern University, Shenyang, 110819, P.R. China E-mail: [email protected] Received April 02, 2020, in final form September 14, 2020; Published online September 29, 2020 https://doi.org/10.3842/SIGMA.2020.093 Abstract. In this paper, we study the differential invariants and the invariant heat flow in centro-affine geometry, proving that the latter is equivalent to the inviscid Burgers' equa- tion. Furthermore, we apply the centro-affine invariants to develop an invariant algorithm to match features of objects appearing in images. We show that the resulting algorithm com- pares favorably with the widely applied scale-invariant feature transform (SIFT), speeded up robust features (SURF), and affine-SIFT (ASIFT) methods. Key words: centro-affine geometry; equivariant moving frames; heat flow; inviscid Burgers' equation; differential invariant; edge matching 2020 Mathematics Subject Classification: 53A15; 53A55 1 Introduction The main objective in this paper is to study differential invariants and invariant curve flows { in particular the heat flow { in centro-affine geometry. In addition, we will present some basic applications to feature matching in camera images of three-dimensional objects, comparing our method with other popular algorithms.
    [Show full text]
  • A Geometric Correction Method Based on Pixel Spatial Transformation
    2020 International Conference on Computer Intelligent Systems and Network Remote Control (CISNRC 2020) ISBN: 978-1-60595-683-1 A Geometric Correction Method Based on Pixel Spatial Transformation Xiaoye Zhang, Shaobin Li, Lei Chen ABSTRACT To achieve the non-planar projection geometric correction, this paper proposes a geometric correction method based on pixel spatial transformation for non-planar projection. The pixel spatial transformation relationship between the distorted image on the non-planar projection plane and the original projection image is derived, the pre-distorted image corresponding to the projection surface is obtained, and the optimal view position is determined to complete the geometric correction. The experimental results show that this method can achieve geometric correction of projective plane as cylinder plane and spherical column plane, and can present the effect of visual distortion free at the optimal view point. KEYWORDS Geometric Correction, Image Pre-distortion, Pixel Spatial Transformation. INTRODUCTION Projection display technology can significantly increase visual range, and projection scenes can be flexibly arranged according to actual needs. Therefore, projection display technology has been applied to all aspects of production, life and learning. With the rapid development of digital media technology, modern projection requirements are also gradually complicated, such as virtual reality presentation with a high sense of immersion and projection performance in various forms, which cannot be met by traditional plane projection, so non-planar projection technology arises at the right moment. Compared with traditional planar projection display technology, non-planar projection mainly includes color compensation of projected surface[1], geometric splicing and brightness fusion of multiple projectors[2], and geometric correction[3].
    [Show full text]
  • Estimating Projective Transformation Matrix (Collineation, Homography)
    Estimating Projective Transformation Matrix (Collineation, Homography) Zhengyou Zhang Microsoft Research One Microsoft Way, Redmond, WA 98052, USA E-mail: [email protected] November 1993; Updated May 29, 2010 Microsoft Research Techical Report MSR-TR-2010-63 Note: The original version of this report was written in November 1993 while I was at INRIA. It was circulated among very few people, and never published. I am now publishing it as a tech report, adding Section 7, with the hope that it could be useful to more people. Abstract In many applications, one is required to estimate the projective transformation between two sets of points, which is also known as collineation or homography. This report presents a number of techniques for this purpose. Contents 1 Introduction 2 2 Method 1: Five-correspondences case 3 3 Method 2: Compute P together with the scalar factors 3 4 Method 3: Compute P only (a batch approach) 5 5 Method 4: Compute P only (an iterative approach) 6 6 Method 5: Compute P through normalization 7 7 Method 6: Maximum Likelihood Estimation 8 1 1 Introduction Projective Transformation is a concept used in projective geometry to describe how a set of geometric objects maps to another set of geometric objects in projective space. The basic intuition behind projective space is to add extra points (points at infinity) to Euclidean space, and the geometric transformation allows to move those extra points to traditional points, and vice versa. Homogeneous coordinates are used in projective space much as Cartesian coordinates are used in Euclidean space. A point in two dimensions is described by a 3D vector.
    [Show full text]
  • Paraperspective ≡ Affine
    International Journal of Computer Vision, 19(2): 169–180, 1996. Paraperspective ´ Affine Ronen Basri Dept. of Applied Math The Weizmann Institute of Science Rehovot 76100, Israel [email protected] Abstract It is shown that the set of all paraperspective images with arbitrary reference point and the set of all affine images of a 3-D object are identical. Consequently, all uncali- brated paraperspective images of an object can be constructed from a 3-D model of the object by applying an affine transformation to the model, and every affine image of the object represents some uncalibrated paraperspective image of the object. It follows that the paraperspective images of an object can be expressed as linear combinations of any two non-degenerate images of the object. When the image position of the reference point is given the parameters of the affine transformation (and, likewise, the coefficients of the linear combinations) satisfy two quadratic constraints. Conversely, when the values of parameters are given the image position of the reference point is determined by solving a bi-quadratic equation. Key words: affine transformations, calibration, linear combinations, paraperspective projec- tion, 3-D object recognition. 1 Introduction It is shown below that given an object O ½ R3, the set of all images of O obtained by applying a rigid transformation followed by a paraperspective projection with arbitrary reference point and the set of all images of O obtained by applying a 3-D affine transformation followed by an orthographic projection are identical. Consequently, all paraperspective images of an object can be constructed from a 3-D model of the object by applying an affine transformation to the model, and every affine image of the object represents some paraperspective image of the object.
    [Show full text]
  • CS 4204 Computer Graphics 3D Views and Projection
    CS 4204 Computer Graphics 3D views and projection Adapted from notes by Yong Cao 1 Overview of 3D rendering Modeling: * Topic we’ve already discussed • *Define object in local coordinates • *Place object in world coordinates (modeling transformation) Viewing: • Define camera parameters • Find object location in camera coordinates (viewing transformation) Projection: project object to the viewplane Clipping: clip object to the view volume *Viewport transformation *Rasterization: rasterize object Simple teapot demo 3D rendering pipeline Vertices as input Series of operations/transformations to obtain 2D vertices in screen coordinates These can then be rasterized 3D rendering pipeline We’ve already discussed: • Viewport transformation • 3D modeling transformations We’ll talk about remaining topics in reverse order: • 3D clipping (simple extension of 2D clipping) • 3D projection • 3D viewing Clipping: 3D Cohen-Sutherland Use 6-bit outcodes When needed, clip line segment against planes Viewing and Projection Camera Analogy: 1. Set up your tripod and point the camera at the scene (viewing transformation). 2. Arrange the scene to be photographed into the desired composition (modeling transformation). 3. Choose a camera lens or adjust the zoom (projection transformation). 4. Determine how large you want the final photograph to be - for example, you might want it enlarged (viewport transformation). Projection transformations Introduction to Projection Transformations Mapping: f : Rn Rm Projection: n > m Planar Projection: Projection on a plane.
    [Show full text]
  • • Rotations • Camera Calibration • Homography • Ransac
    Agenda • Rotations • Camera calibration • Homography • Ransac Geometric Transformations y 164 Computer Vision: Algorithms andx Applications (September 3, 2010 draft) Transformation Matrix # DoF Preserves Icon translation I t 2 orientation 2 3 h i ⇥ ⇢⇢SS rigid (Euclidean) R t 3 lengths S ⇢ 2 3 S⇢ ⇥ h i ⇢ similarity sR t 4 angles S 2 3 S⇢ h i ⇥ ⇥ ⇥ affine A 6 parallelism ⇥ ⇥ 2 3 h i ⇥ projective H˜ 8 straight lines ` 3 3 ` h i ⇥ Table 3.5 Hierarchy of 2D coordinate transformations. Each transformation also preserves Let’s definethe properties families listed of in thetransformations rows below it, i.e., similarity by the preserves properties not only anglesthat butthey also preserve parallelism and straight lines. The 2 3 matrices are extended with a third [0T 1] row to form ⇥ a full 3 3 matrix for homogeneous coordinate transformations. ⇥ amples of such transformations, which are based on the 2D geometric transformations shown in Figure 2.4. The formulas for these transformations were originally given in Table 2.1 and are reproduced here in Table 3.5 for ease of reference. In general, given a transformation specified by a formula x0 = h(x) and a source image f(x), how do we compute the values of the pixels in the new image g(x), as given in (3.88)? Think about this for a minute before proceeding and see if you can figure it out. If you are like most people, you will come up with an algorithm that looks something like Algorithm 3.1. This process is called forward warping or forward mapping and is shown in Figure 3.46a.
    [Show full text]
  • Math 535 - General Topology Additional Notes
    Math 535 - General Topology Additional notes Martin Frankland September 5, 2012 1 Subspaces Definition 1.1. Let X be a topological space and A ⊆ X any subset. The subspace topology sub on A is the smallest topology TA making the inclusion map i: A,! X continuous. sub In other words, TA is generated by subsets V ⊆ A of the form V = i−1(U) = U \ A for any open U ⊆ X. Proposition 1.2. The subspace topology on A is sub TA = fV ⊆ A j V = U \ A for some open U ⊆ Xg: In other words, the collection of subsets of the form U \ A already forms a topology on A. 2 Products Before discussing the product of spaces, let us review the notion of product of sets. 2.1 Product of sets Let X and Y be sets. The Cartesian product of X and Y is the set of pairs X × Y = f(x; y) j x 2 X; y 2 Y g: It comes equipped with the two projection maps pX : X × Y ! X and pY : X × Y ! Y onto each factor, defined by pX (x; y) = x pY (x; y) = y: This explicit description of X × Y is made more meaningful by the following proposition. 1 Proposition 2.1. The Cartesian product of sets satisfies the following universal property. For any set Z along with maps fX : Z ! X and fY : Z ! Y , there is a unique map f : Z ! X × Y satisfying pX ◦ f = fX and pY ◦ f = fY , in other words making the diagram Z fX 9!f fY X × Y pX pY { " XY commute.
    [Show full text]