DOCSLIB.ORG

Homography, Transforms, Mosaics

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Homography, Transforms, Mosaics Homography, Transforms, Mosaics Credits: S. Seitz, R. Collins, J. Hays, K. Grauman, C. Choi, C. Brunner, R. Szeliski, L. Zitnick Robert Collins CSE486, Penn StateApplying Homographies to Remove Perspective Distortion from Hartley & Zisserman 4 point correspondences suffice for the planar building facade Robert Collins CSE486, Penn State Homographies for Bird’s-eye Views from Hartley & Zisserman Robert Collins CSE486, Penn State Homographies for Mosaicing from Hartley & Zisserman What are 2D geometric transformations? Translation Preserves: Orientaon Translation and rotation Scale Similarity transformations Similarity transform (4 DoF) = translaon + rotaon + scale Preserves: Angles Aspect ratio Shear Affine transformations Affine transform (6 DoF) = translaon + rotaon + scale + aspect rao + shear Preserves: Parallelism What is missing? Canaleo Are there any other planar transformaons? General affine We already used these How do we compute projecve transformaons? Homogeneous coordinates One extra step: Projective transformations a.k.a. Homographies “keystone” distorons Preserves: Straight Lines Finding the transformation Translaon = 2 degrees of freedom Similarity = 4 degrees of freedom Affine = 6 degrees of freedom Homography = 8 degrees of freedom How many corresponding points do we need to solve? Image warping with homographies image plane in front image plane below Richard Szeliski 17 Finding the transformation - How can we find the transformaon between these images? - How many corresponding points do we need to solve? Planar Homography • A projective mapping from one plane to another • e.g. mapping of points on a ground plane to the image of the camera camera xi xw center image plane world (ground) plane xi Hxw 1 ' H− xi xw '44 Homography Projective – mapping between any two PPs with the same center of projection • rectangle should map to a quadrilateral. – straight lines are conserved. – parallel lines are not conserved. PP2 • same as: project, rotate, reproject. called Homography PP1 p’ H p To apply a homography H • Compute p’ = Hp (regular matrix multiplication) • Convert p’ from homogeneous to image coordinates Planar Homography z key idea: Z = 0 xi xw X x f sc 1000 x x Rt Y y fy cy 0100 2 3 2 3 ' 2 3 2 3 0| 1 Z 1 1 0010 6 1 7 4 5 4 5 4 5 6 7 r r 4 r 5 t X f sc 1000 11 21 31 1 x x r r r t Y = f c 0100 12 22 32 2 y y 2r r r t 3 2Z 3 2 1 3 200103 13 23 33 3 6 00017 6 1 7 4 5 4 5 6 7 6 7 4r r t 5 4 5 f sc 1000 11 21 1 X x x r r t = f c 0100 12 22 2 Y y y 2r r t 3 2 1 3 200103 13 23 3 2 1 3 6 0017 4 5 4 5 6 7 4 5 4 5 homography fx scx r11 r21 t1 X = f c r r t Y matrix 2 y y3 2 12 22 23 2 3 1 r13 r23 t3 1 4 5 4 5 4 5 h11 h21 h31 X = h h h Y 2 12 22 323 2 3 h13 h23 h33 1 4 5 4 5 45 Robert Collins CSE486, Penn State Projection of Planar Points Fitting a homography • Recall: homogeneous coordinates Converting to homogeneous Converting from homogeneous image coordinates image coordinates • Equation for homography: Robert Collins CSE486, Penn State Estimating a Homography Matrix Form: Equations: Robert Collins CSE486, Penn State Degrees of Freedom? There are 9 numbers h11,…,h33 , so are there 9 DOF? No. Note that we can multiply all hij by nonzero k without changing the equations: 10/14/2015 Homography x, y x1, y1 1 1 x2 , y2 x2 , y2 … … xn , yn xn , yn To compute the homography given pairs of corresponding points in the images, we need to set up an equation where the parameters of H are the unknowns… Solving for homographies p’ = Hp wx' a b c x wy' d e f y w g h i 1 Can set scale factor i=1. So, there are 8 unknowns. Set up a system of linear equations: Ah = b where vector of unknowns h = [a,b,c,d,e,f,g,h]T Need at least 8 eqs, but the more the better… Solve for h. If overconstrained, solve using least-squares: min Ah b 2 >> help lmdivide BOARD 13 Image Stitching, Alignment, Blending Image from http://graphics.cs.cmu.edu/courses/15-463/2010_fall/ 10/14/2015 Mosaics . imageSeitz from S. Obtain a wider angle view by combining multiple images. Main questions Alignment: Given two T images, what is the transformation between them? Warping: Given a T source image and a transformation, what does the transformed output look like? 7 Why Mosaic? Are you getting the whole picture? • Compact Camera FOV = 50 x 35° Slide from Brown & Lowe Why Mosaic? Are you getting the whole picture? • Compact Camera FOV = 50 x 35° • Human FOV = 200 x 135° Slide from Brown & Lowe Why Mosaic? Are you getting the whole picture? • Compact Camera FOV = 50 x 35° • Human FOV = 200 x 135° • Panoramic Mosaic = 360 x 180° Slide from Brown & Lowe Image Mosaics + + + virtual wide-angle camera Goal: Stitch together several images into a seamless composite A pencil of rays contains all views real synthetic camera camera Can generate any synthetic camera view as long as it has the same center of projection! Image reprojection mosaic PP The mosaic has a natural interpretation in 3D • The images are reprojected onto a common plane • The mosaic is formed on this plane • Mosaic is a synthetic wide-angle camera How to do it? Basic Procedure 1. Take a sequence of images from the same position. – Rotate the camera about its optical center. 2. Compute transformation between second image and first. 3. Transform the second image to overlap with the first. 4. Blend the two together to create a mosaic. 5. If there are more images, repeat. Full-view Panorama .
Load more

Recommended publications
  • Projective Geometry: a Short Introduction
    Projective Geometry: A Short Introduction Lecture Notes Edmond Boyer Master MOSIG Introduction to Projective Geometry Contents 1 Introduction 2 1.1 Objective . .2 1.2 Historical Background . .3 1.3 Bibliography . .4 2 Projective Spaces 5 2.1 Definitions . .5 2.2 Properties . .8 2.3 The hyperplane at infinity . 12 3 The projective line 13 3.1 Introduction . 13 3.2 Projective transformation of P1 ................... 14 3.3 The cross-ratio . 14 4 The projective plane 17 4.1 Points and lines . 17 4.2 Line at infinity . 18 4.3 Homographies . 19 4.4 Conics . 20 4.5 Affine transformations . 22 4.6 Euclidean transformations . 22 4.7 Particular transformations . 24 4.8 Transformation hierarchy . 25 Grenoble Universities 1 Master MOSIG Introduction to Projective Geometry Chapter 1 Introduction 1.1 Objective The objective of this course is to give basic notions and intuitions on projective geometry. The interest of projective geometry arises in several visual comput- ing domains, in particular computer vision modelling and computer graphics. It provides a mathematical formalism to describe the geometry of cameras and the associated transformations, hence enabling the design of computational ap- proaches that manipulates 2D projections of 3D objects. In that respect, a fundamental aspect is the fact that objects at infinity can be represented and manipulated with projective geometry and this in contrast to the Euclidean geometry. This allows perspective deformations to be represented as projective transformations. Figure 1.1: Example of perspective deformation or 2D projective transforma- tion. Another argument is that Euclidean geometry is sometimes difficult to use in algorithms, with particular cases arising from non-generic situations (e.g.
    [Show full text]
  • 2D and 3D Transformations, Homogeneous Coordinates Lecture 03
    2D and 3D Transformations, Homogeneous Coordinates Lecture 03 Patrick Karlsson [email protected] Centre for Image Analysis Uppsala University Computer Graphics November 6 2006 Patrick Karlsson (Uppsala University) Transformations and Homogeneous Coords. Computer Graphics 1 / 23 Reading Instructions Chapters 4.1–4.9. Edward Angel. “Interactive Computer Graphics: A Top-down Approach with OpenGL”, Fourth Edition, Addison-Wesley, 2004. Patrick Karlsson (Uppsala University) Transformations and Homogeneous Coords. Computer Graphics 2 / 23 Todays lecture ... in the pipeline Patrick Karlsson (Uppsala University) Transformations and Homogeneous Coords. Computer Graphics 3 / 23 Scalars, points, and vectors Scalars α, β Real (or complex) numbers. Points P, Q Locations in space (but no size or shape). Vectors u, v Directions in space (magnitude but no position). Patrick Karlsson (Uppsala University) Transformations and Homogeneous Coords. Computer Graphics 4 / 23 Mathematical spaces Scalar field A set of scalars obeying certain properties. New scalars can be formed through addition and multiplication. (Linear) Vector space Made up of scalars and vectors. New vectors can be created through scalar-vector multiplication, and vector-vector addition. Affine space An extended vector space that include points. This gives us additional operators, such as vector-point addition, and point-point subtraction. Patrick Karlsson (Uppsala University) Transformations and Homogeneous Coords. Computer Graphics 5 / 23 Data types Polygon based objects Objects are described using polygons. A polygon is defined by its vertices (i.e., points). Transformations manipulate the vertices, thus manipulates the objects. Some examples in 2D Scalar α 1 float. Point P(x, y) 2 floats. Vector v(x, y) 2 floats. Matrix M 4 floats.
    [Show full text]
  • 2-D Drawing Geometry Homogeneous Coordinates
    2-D Drawing Geometry Homogeneous Coordinates The rotation of a point, straight line or an entire image on the screen, about a point other than origin, is achieved by first moving the image until the point of rotation occupies the origin, then performing rotation, then finally moving the image to its original position. The moving of an image from one place to another in a straight line is called a translation. A translation may be done by adding or subtracting to each point, the amount, by which picture is required to be shifted. Translation of point by the change of coordinate cannot be combined with other transformation by using simple matrix application. Such a combination is essential if we wish to rotate an image about a point other than origin by translation, rotation again translation. To combine these three transformations into a single transformation, homogeneous coordinates are used. In homogeneous coordinate system, two-dimensional coordinate positions (x, y) are represented by triple- coordinates. Homogeneous coordinates are generally used in design and construction applications. Here we perform translations, rotations, scaling to fit the picture into proper position 2D Transformation in Computer Graphics- In Computer graphics, Transformation is a process of modifying and re- positioning the existing graphics. • 2D Transformations take place in a two dimensional plane. • Transformations are helpful in changing the position, size, orientation, shape etc of the object. Transformation Techniques- In computer graphics, various transformation techniques are- 1. Translation 2. Rotation 3. Scaling 4. Reflection 2D Translation in Computer Graphics- In Computer graphics, 2D Translation is a process of moving an object from one position to another in a two dimensional plane.
    [Show full text]
  • Projective Geometry Lecture Notes
    Projective Geometry Lecture Notes Thomas Baird March 26, 2014 Contents 1 Introduction 2 2 Vector Spaces and Projective Spaces 4 2.1 Vector spaces and their duals . 4 2.1.1 Fields . 4 2.1.2 Vector spaces and subspaces . 5 2.1.3 Matrices . 7 2.1.4 Dual vector spaces . 7 2.2 Projective spaces and homogeneous coordinates . 8 2.2.1 Visualizing projective space . 8 2.2.2 Homogeneous coordinates . 13 2.3 Linear subspaces . 13 2.3.1 Two points determine a line . 14 2.3.2 Two planar lines intersect at a point . 14 2.4 Projective transformations and the Erlangen Program . 15 2.4.1 Erlangen Program . 16 2.4.2 Projective versus linear . 17 2.4.3 Examples of projective transformations . 18 2.4.4 Direct sums . 19 2.4.5 General position . 20 2.4.6 The Cross-Ratio . 22 2.5 Classical Theorems . 23 2.5.1 Desargues' Theorem . 23 2.5.2 Pappus' Theorem . 24 2.6 Duality . 26 3 Quadrics and Conics 28 3.1 Affine algebraic sets . 28 3.2 Projective algebraic sets . 30 3.3 Bilinear and quadratic forms . 31 3.3.1 Quadratic forms . 33 3.3.2 Change of basis . 33 1 3.3.3 Digression on the Hessian . 36 3.4 Quadrics and conics . 37 3.5 Parametrization of the conic . 40 3.5.1 Rational parametrization of the circle . 42 3.6 Polars . 44 3.7 Linear subspaces of quadrics and ruled surfaces . 46 3.8 Pencils of quadrics and degeneration . 47 4 Exterior Algebras 52 4.1 Multilinear algebra .
    [Show full text]
  • • Rotations • Camera Calibration • Homography • Ransac
    Agenda • Rotations • Camera calibration • Homography • Ransac Geometric Transformations y 164 Computer Vision: Algorithms andx Applications (September 3, 2010 draft) Transformation Matrix # DoF Preserves Icon translation I t 2 orientation 2 3 h i ⇥ ⇢⇢SS rigid (Euclidean) R t 3 lengths S ⇢ 2 3 S⇢ ⇥ h i ⇢ similarity sR t 4 angles S 2 3 S⇢ h i ⇥ ⇥ ⇥ affine A 6 parallelism ⇥ ⇥ 2 3 h i ⇥ projective H˜ 8 straight lines ` 3 3 ` h i ⇥ Table 3.5 Hierarchy of 2D coordinate transformations. Each transformation also preserves Let’s definethe properties families listed of in thetransformations rows below it, i.e., similarity by the preserves properties not only anglesthat butthey also preserve parallelism and straight lines. The 2 3 matrices are extended with a third [0T 1] row to form ⇥ a full 3 3 matrix for homogeneous coordinate transformations. ⇥ amples of such transformations, which are based on the 2D geometric transformations shown in Figure 2.4. The formulas for these transformations were originally given in Table 2.1 and are reproduced here in Table 3.5 for ease of reference. In general, given a transformation specified by a formula x0 = h(x) and a source image f(x), how do we compute the values of the pixels in the new image g(x), as given in (3.88)? Think about this for a minute before proceeding and see if you can figure it out. If you are like most people, you will come up with an algorithm that looks something like Algorithm 3.1. This process is called forward warping or forward mapping and is shown in Figure 3.46a.
    [Show full text]
  • Chapter 12 the Cross Ratio
    Chapter 12 The cross ratio Math 4520, Fall 2017 We have studied the collineations of a projective plane, the automorphisms of the underlying field, the linear functions of Affine geometry, etc. We have been led to these ideas by various problems at hand, but let us step back and take a look at one important point of view of the big picture. 12.1 Klein's Erlanger program In 1872, Felix Klein, one of the leading mathematicians and geometers of his day, in the city of Erlanger, took the following point of view as to what the role of geometry was in mathematics. This is from his \Recent trends in geometric research." Let there be given a manifold and in it a group of transforma- tions; it is our task to investigate those properties of a figure belonging to the manifold that are not changed by the transfor- mation of the group. So our purpose is clear. Choose the group of transformations that you are interested in, and then hunt for the \invariants" that are relevant. This search for invariants has proved very fruitful and useful since the time of Klein for many areas of mathematics, not just classical geometry. In some case the invariants have turned out to be simple polynomials or rational functions, such as the case with the cross ratio. In other cases the invariants were groups themselves, such as homology groups in the case of algebraic topology. 12.2 The projective line In Chapter 11 we saw that the collineations of a projective plane come in two \species," projectivities and field automorphisms.
    [Show full text]
  • Homogeneous Representations of Points, Lines and Planes
    Chapter 5 Homogeneous Representations of Points, Lines and Planes 5.1 Homogeneous Vectors and Matrices ................................. 195 5.2 Homogeneous Representations of Points and Lines in 2D ............... 205 n 5.3 Homogeneous Representations in IP ................................ 209 5.4 Homogeneous Representations of 3D Lines ........................... 216 5.5 On Plücker Coordinates for Points, Lines and Planes .................. 221 5.6 The Principle of Duality ........................................... 229 5.7 Conics and Quadrics .............................................. 236 5.8 Normalizations of Homogeneous Vectors ............................. 241 5.9 Canonical Elements of Coordinate Systems ........................... 242 5.10 Exercises ........................................................ 245 This chapter motivates and introduces homogeneous coordinates for representing geo- metric entities. Their name is derived from the homogeneity of the equations they induce. Homogeneous coordinates represent geometric elements in a projective space, as inhomoge- neous coordinates represent geometric entities in Euclidean space. Throughout this book, we will use Cartesian coordinates: inhomogeneous in Euclidean spaces and homogeneous in projective spaces. A short course in the plane demonstrates the usefulness of homogeneous coordinates for constructions, transformations, estimation, and variance propagation. A characteristic feature of projective geometry is the symmetry of relationships between points and lines, called
    [Show full text]
  • Math 535 - General Topology Additional Notes
    Math 535 - General Topology Additional notes Martin Frankland September 5, 2012 1 Subspaces Definition 1.1. Let X be a topological space and A ⊆ X any subset. The subspace topology sub on A is the smallest topology TA making the inclusion map i: A,! X continuous. sub In other words, TA is generated by subsets V ⊆ A of the form V = i−1(U) = U \ A for any open U ⊆ X. Proposition 1.2. The subspace topology on A is sub TA = fV ⊆ A j V = U \ A for some open U ⊆ Xg: In other words, the collection of subsets of the form U \ A already forms a topology on A. 2 Products Before discussing the product of spaces, let us review the notion of product of sets. 2.1 Product of sets Let X and Y be sets. The Cartesian product of X and Y is the set of pairs X × Y = f(x; y) j x 2 X; y 2 Y g: It comes equipped with the two projection maps pX : X × Y ! X and pY : X × Y ! Y onto each factor, defined by pX (x; y) = x pY (x; y) = y: This explicit description of X × Y is made more meaningful by the following proposition. 1 Proposition 2.1. The Cartesian product of sets satisfies the following universal property. For any set Z along with maps fX : Z ! X and fY : Z ! Y , there is a unique map f : Z ! X × Y satisfying pX ◦ f = fX and pY ◦ f = fY , in other words making the diagram Z fX 9!f fY X × Y pX pY { " XY commute.
    [Show full text]
  • Math 395: Category Theory Northwestern University, Lecture Notes
    Math 395: Category Theory Northwestern University, Lecture Notes Written by Santiago Can˜ez These are lecture notes for an undergraduate seminar covering Category Theory, taught by the author at Northwestern University. The book we roughly follow is “Category Theory in Context” by Emily Riehl. These notes outline the specific approach we’re taking in terms the order in which topics are presented and what from the book we actually emphasize. We also include things we look at in class which aren’t in the book, but otherwise various standard definitions and examples are left to the book. Watch out for typos! Comments and suggestions are welcome. Contents Introduction to Categories 1 Special Morphisms, Products 3 Coproducts, Opposite Categories 7 Functors, Fullness and Faithfulness 9 Coproduct Examples, Concreteness 12 Natural Isomorphisms, Representability 14 More Representable Examples 17 Equivalences between Categories 19 Yoneda Lemma, Functors as Objects 21 Equalizers and Coequalizers 25 Some Functor Properties, An Equivalence Example 28 Segal’s Category, Coequalizer Examples 29 Limits and Colimits 29 More on Limits/Colimits 29 More Limit/Colimit Examples 30 Continuous Functors, Adjoints 30 Limits as Equalizers, Sheaves 30 Fun with Squares, Pullback Examples 30 More Adjoint Examples 30 Stone-Cech 30 Group and Monoid Objects 30 Monads 30 Algebras 30 Ultrafilters 30 Introduction to Categories Category theory provides a framework through which we can relate a construction/fact in one area of mathematics to a construction/fact in another. The goal is an ultimate form of abstraction, where we can truly single out what about a given problem is specific to that problem, and what is a reflection of a more general phenomenom which appears elsewhere.
    [Show full text]
  • 9.2.2 Projection Formula Definition 2.8
    9.2. Intersection and morphisms 397 Proof Let Γ ⊆ X×S Z be the graph of ϕ. This is an S-scheme and the projection p :Γ → X is projective birational. Let us show that Γ admits a desingularization. This is Theorem 8.3.44 if dim S = 0. Let us therefore suppose that dim S = 1. Let K = K(S). Then pK :ΓK → XK is proper birational with XK normal, and hence pK is an isomorphism. We can therefore apply Theorem 8.3.50. Let g : Xe → Γ be a desingularization morphism, and f : Xe → X the composition p ◦ g. It now suffices to apply Theorem 2.2 to the morphism f. 9.2.2 Projection formula Definition 2.8. Let X, Y be Noetherian schemes, and let f : X → Y be a proper morphism. For any prime cycle Z on X (Section 7.2), we set W = f(Z) and ½ [K(Z): K(W )]W if K(Z) is finite over K(W ) f Z = ∗ 0 otherwise. By linearity, we define a homomorphism f∗ from the group of cycles on X to the group of cycles on Y . This generalizes Definition 7.2.17. It is clear that the construction of f∗ is compatible with the composition of morphisms. Remark 2.9. We can interpret the intersection of two divisors in terms of direct images of cycles. Let C, D be two Cartier divisors on a regular fibered surface X → S, of which at least one is vertical. Let us suppose that C is effective and has no common component with Supp D.
    [Show full text]
  • Random Projections for Manifold Learning
    Random Projections for Manifold Learning Chinmay Hegde Michael B. Wakin ECE Department EECS Department Rice University University of Michigan [email protected] [email protected] Richard G. Baraniuk ECE Department Rice University [email protected] Abstract We propose a novel method for linear dimensionality reduction of manifold mod- eled data. First, we show that with a small number M of random projections of sample points in RN belonging to an unknown K-dimensional Euclidean mani- fold, the intrinsic dimension (ID) of the sample set can be estimated to high accu- racy. Second, we rigorously prove that using only this set of random projections, we can estimate the structure of the underlying manifold. In both cases, the num- ber of random projections required is linear in K and logarithmic in N, meaning that K < M ≪ N. To handle practical situations, we develop a greedy algorithm to estimate the smallest size of the projection space required to perform manifold learning. Our method is particularly relevant in distributed sensing systems and leads to significant potential savings in data acquisition, storage and transmission costs. 1 Introduction Recently, we have witnessed a tremendous increase in the sizes of data sets generated and processed by acquisition and computing systems. As the volume of the data increases, memory and processing requirements need to correspondingly increase at the same rapid pace, and this is often prohibitively expensive. Consequently, there has been considerable interest in the task of effective modeling of high-dimensional observed data and information; such models must capture the structure of the information content in a concise manner.
    [Show full text]
  • Lecture 16: Planar Homographies Robert Collins CSE486, Penn State Motivation: Points on Planar Surface
    Robert Collins CSE486, Penn State Lecture 16: Planar Homographies Robert Collins CSE486, Penn State Motivation: Points on Planar Surface y x Robert Collins CSE486, Penn State Review : Forward Projection World Camera Film Pixel Coords Coords Coords Coords U X x u M M V ext Y proj y Maff v W Z U X Mint u V Y v W Z U M u V m11 m12 m13 m14 v W m21 m22 m23 m24 m31 m31 m33 m34 Robert Collins CSE486, PennWorld State to Camera Transformation PC PW W Y X U R Z C V Rotate to Translate by - C align axes (align origins) PC = R ( PW - C ) = R PW + T Robert Collins CSE486, Penn State Perspective Matrix Equation X (Camera Coordinates) x = f Z Y X y = f x' f 0 0 0 Z Y y' = 0 f 0 0 Z z' 0 0 1 0 1 p = M int ⋅ PC Robert Collins CSE486, Penn State Film to Pixel Coords 2D affine transformation from film coords (x,y) to pixel coordinates (u,v): X u’ a11 a12xa'13 f 0 0 0 Y v’ a21 a22 ya'23 = 0 f 0 0 w’ Z 0 0z1' 0 0 1 0 1 Maff Mproj u = Mint PC = Maff Mproj PC Robert Collins CSE486, Penn StateProjection of Points on Planar Surface Perspective projection y Film coordinates x Point on plane Rotation + Translation Robert Collins CSE486, Penn State Projection of Planar Points Robert Collins CSE486, Penn StateProjection of Planar Points (cont) Homography H (planar projective transformation) Robert Collins CSE486, Penn StateProjection of Planar Points (cont) Homography H (planar projective transformation) Punchline: For planar surfaces, 3D to 2D perspective projection reduces to a 2D to 2D transformation.
    [Show full text]