DOCSLIB.ORG

Relative Orientation, Fundamental and Essential Matrix

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text

Load more

Photogrammetry & Robotics Lab A Camera Pair Relative Orientation, Fundamental and Essential Matrix Cyrill Stachniss The slides have been created by Cyrill Stachniss. 1 2 Camera Pair § A stereo camera § One camera that moves Orientation Parameters Camera pair = two configurations for the Camera Pair and from which images have been taken Relative Orientation 3 4 Orientation Orientation § The orientation of the camera pair can § The orientation of the camera pair can be described using independent be described using independent orientations for each camera orientations for each camera How many parameters are needed? How many parameters are needed? § Calibrated cameras: ? parameters § Calibrated cameras: 12 parameters (angle preserving mapping) (angle preserving mapping) § Uncalibrated cameras: ? parameters § Uncalibrated cameras: 22 parameters (straight-line preserving mapping) (straight-line preserving mapping) 5 6 Can We Estimate the Which Parameters Camera Motion without Can We Obtain and Which Not? Knowing the Scene? 7 8 Cameras Measure Directions What We Can Compute § We cannot obtain the (global) § The rotation of the second camera translation and rotation (if the w.r.t. the first one (3 parameters) cameras maintain their relative § The direction of the line connecting transformation) as well as the scale the to centers of projection (2 params) § We do not know their distance (the length of ) Image courtesy: Förstner & Wrobel 9 Image courtesy: Förstner & Wrobel 10 For Calibrated Cameras Photogrammetric Model § We need 2x6=12 parameters for two § Given two cameras images, we can calibrated cameras for the orientation reconstruct the object only up to a § With a calibrated camera, we obtain similarity transform an angle-preserving model of the § Called a photogrammetric model object § The orientation of the § Without additional information, we can photogrammetric model is called only obtain 12-7 = 5 parameters the absolute orientation (not 7=translation, rotation, scale) § For obtaining the absolute orientation, we need at least 3 points in 3D first camera distance between cameras (to estimate the 7 parameters) 11 12 What Is Needed for Computing For Uncalibrated Cameras an 3D Model of a Scene? § Straight-line preserving but not angle preserving § Object can be reconstructed up to a straight-line preserving mapping § Projective transform (15 parameters) § Thus, for uncalibrated cameras, we can only obtain 22-15=7 parameters given two images § We need at least 5 points in 3D (15 coordinates) for the absolute o. 13 14 Relative Orientation Summary Cameras #params #params #params #params min #P /img /img pair for RO for AO Coplanarity Constraint calibrated 6 12 5 7 3 for Straight-Line Preserving not calibrated 11 22 7 15 5 (Uncalibrated) Cameras to Obtain the RO = relative orientation Fundamental Matrix AO = absolute orientation min #P = min. number of control points 15 16 Which Parameters Can We Coplanarity Constraint Compute Without Additional § Consider perfect orientation and the Information About the Scene? intersection of two corresponding rays We start with a perfect orientation and § The rays lie within one plane in 3D the intersection of two corresponding rays 17 18 Coplanarity Constraint for Scalar Triple Product Uncalibrated Cameras § The operator is the triple product Coplanarity can be expressed by § Dot product of one of the vectors with the cross product of the other two and refer to the same point in space § It is the volume of the parallelepiped of three vectors § means that the vectors lie in one plane 19 Image courtesy: Wikipedia (Niabot) 20 Coplanarity Constraint for Coplanarity Constraint for Uncalibrated Cameras Uncalibrated Cameras Coplanarity can be expressed by § The directions of the vectors and can be derived from the image coordinates § with the projection matrices Reminder: 21 22 Directions to a Point Base Vector § The normalized directions of the § The base vector directly results vectors and are from the coordinates of the projection image coord. centers § as the normalized projection world coord. § provides the direction to from the center of projection to the point in 3D § Analogous: 23 24 Coplanarity Constraint Derivation § Using the previous relations, § Why is this correct? the coplanarity constraint § can be rewritten as skew-symmetric matrix 25 26 Derivation Coplanarity Constraint § Why is this correct? § By combining and § we obtain § Results from the cross product as § with being a skew-symmetric matrix 27 28 Coplanarity Constraint Fundamental Matrix § By combining § The matrix is the fundamental and matrix (for uncalibrated cameras): § we obtain § It allow for expressing the coplanarity constraint by 29 30 Fundamental Matrix Fundamental Matrix From the Camera Projection Matrices § The fundamental matrix is the matrix that fulfills the equation § If the projection matrices are given, we can derive the fundamental matrix? for corresponding points § The fundamental matrix contains the all the available information about the relative orientation of two § Let the projection matrices be images from uncalibrated cameras partitioned into a left 3×3 matrix and a 3-vector as . 31 32 Fundamental Matrix From the Fundamental Matrix From the Camera Projection Matrices Camera Projection Matrices § We have § We have § and can recover the projection center § and § and thus can compute the F § so that the base line is given by § with 33 34 Alternative Definition § In the context of many images, we will call that fundamental matrix which yields the constraint § Thus in our case, we have . Essential Matrix § Our definition of is not the same as in the book by Hartley and Zisserman (for Calibrated Cameras) § The definition in Hartley and Zisserman is based on , i.e. § The transposition needs to be taken into account when comparing expressions 35 36 Using Calibrated Cameras Coplanarity Constraint § Most photogrammetric systems rely § For calibrated cameras the coplanarity on calibrated cameras constraint can be simplified § Calibrated cameras simplify the § Based on the calibration matrices, we orientation problem obtain the directions as § Often, we assume that both cameras have the same calibration matrix direction in coordinates § Assumption here: no distortions or camera frame in the image other imaging errors § This relation results from 37 38 Coplanarity Constraint Coplanarity Constraint § Exploiting the fundamental matrix § Exploiting the fundamental matrix 39 40 Coplanarity Constraint Essential Matrix § Exploiting the fundamental matrix § From to the essential matrix same form as the fundamental essential matrix matrix but for calibrated cameras (DE: Essentielle Matrix) 41 42 Essential Matrix Essential Matrix § We derived a specialization of the § The essential matrix as five degrees fundamental matrix of freedom § For the calibrated cameras, it is called § There are five parameters that the essential matrix determine the relative orientation of the image pair for calibrated cameras § There are 4=9-5 constraints to its 9 elements (3 by 3 matrix) § We can write the coplanarity constraint for calibrated cameras as § The essential matrix is homogenous and singular 43 44 Five Parameters – How? § Five parameters that determine the relative orientation of the image pair Popular Parameterizations for the Relative Orientation How to parameterize the essential matrix? 45 46 The Popular Parameterizations The Popular Parameterizations § Five parameters that determine the 1. General parameterization of relative orientation of the image pair dependent images § Three popular parameterizations 2. Photogrammetric parameterization of dependent images 3. Parameterization with independent images Image courtesy: Förstner 47 Image courtesy: Förstner 48 General Parameterization of Photogrammetric parametrizat. Dependent Images of Dependent Images The general parameterization of Photogrammetric parameterization of dependent images uses a dependent images uses § normalized direction vector b § two components BY and BZ of the § rotation matrix R base direction (BX=const) § a rotation matrix R DE: Allgemeine Parametrisierung des Folgebildanschluss DE: Klassich-photogrammetrische Parametrisierung des Folgebildanschluss 49 50 Parameterization with Independent Images The parameterization with independent images uses Parameterization of § a rotation matrix Dependent Images (DE: Parametrisierung des Folgebildanschluss ) § a rotation matrix § a fixed basis of constant length DE: Parametrisierung mit Bilddrehungen 51 52 For the Parameterizations of General Parameterization of Dependent Images Dependent Images § The reference frame is the frame of § The orientation of the second camera the first camera is and we obtain from the § Describe the second camera relative to coplanarity constraint the first one § Rotation mat. of the first cam is § The rotation of the R.O. is then § 6 parameters + 1 constraint 53 54 General Parameterization of Summary Dependent Images § Parameters of image pairs § The resulting 5 degree of freedom are § Relative orientation § Fundamental matrix F § Coplanarity constraint § Essential matrix E (F for the calibrated camera pair) § Coplanarity constraint § Parameterization of the relative orientation 55 56 Literature Slide Information § The slides have been created by Cyrill Stachniss as part of the § Förstner, Wrobel: Photogrammetric photogrammetry and robotics courses. Computer Vision, Ch. 12.2 § I tried to acknowledge all people from whom I used images or videos. In case I made a mistake or missed someone, please let me know. § The photogrammetry material heavily relies on the very well written lecture notes by Wolfgang Förstner and the Photogrammetric Computer Vision book by Förstner & Wrobel. § Parts of the robotics material stems from the great Probabilistic Robotics book by Thrun, Burgard and Fox. § If you are a university lecturer, feel free to use the course material. If you adapt the course material, please make sure that you keep the acknowledgements to others and please acknowledge me as well. To satisfy my own curiosity, please send me email notice if you use my slides. 57 Cyrill Stachniss, [email protected], 2014 58 .
Recommended publications
  • Lecture 6: Camera Computation and the Essential Matrix

    Lecture 6: Camera Computation and the Essential Matrix

    Computer Vision Lecture 6 2020-02-05 Lecture 6: Camera Computation and the Essential Matrix 1 Computing Cameras From the Fundamental Matrix In Lecture 5 we considered the two-view structure from motion problem, that is, given a number of measured points in two images we want to compute both camera matrices and 3D points such that they project to the measurements. We showed that the 3D points can be eliminated from the problem by considering the fundamental matrix F . If x is an image point belonging to the fist image and x¯ belongs to the second then there is a 3D point that projects to to these if and only if the epipolar constraint x¯T F x = 0 (1) is fulfilled. Using the projections of 8-scene points we can compute the fundamental matrix by solving a homo- geneous least squares problem (the 8-point algorithm). What remains in order to find a solution to the two-view structure from motion camera is to compute cameras from F and finally compute the 3D-points. In general we may assume (see Lecture 5) that the cameras are of the form P1 = [I 0] and P2 = [A e2] where T e2 is the epipole in the second image. Since we know that F e2 = 0 we can find e2 by computing the null space T of F . In what follows we will show that A = [e2]×F gives the correct epipolar geometry and therefore solution for the second camera is given by P2 = [[e2]×F e2] : (2) The Fundamental matrix of the camera pair P1 and P2 is according to Lecture 5 given by [t]×A = [e2]×[e2]×F we need to show that this expression reduces to F .
  • Table of Contents Stereopsis

    Table of Contents Stereopsis

    CS-9645 Introduction to Computer Vision Techniques Winter 2020 Table of Contents Stereopsis................................................................................................................................................... 1 The Correspondence Problem............................................................................................................... 1 Epipolar Geometry................................................................................................................................ 2 Estimating the Essential and Fundamental Matrices.............................................................................5 Algorithm.............................................................................................................................................. 5 Locating Epipoles..................................................................................................................................6 Rectification.......................................................................................................................................... 7 3D Reconstruction.................................................................................................................................8 Stereopsis In stereopsis, two visual sensors are used in order to obtain the depth of scene points, as an attempt the reconstruct the observed scene. Image features from one image must correlate with the features observed in the other image. This is commonly known as the correspondence problem. Once
  • Monitoring Mountain Cryosphere Dynamics by Time Lapse Stereo Photogrammetry

    Monitoring Mountain Cryosphere Dynamics by Time Lapse Stereo Photogrammetry

    ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume V-2-2020, 2020 XXIV ISPRS Congress (2020 edition) MONITORING MOUNTAIN CRYOSPHERE DYNAMICS BY TIME LAPSE STEREO PHOTOGRAMMETRY ∗ G. Marsy1,2,3,, F. Vernier1, X. Bodin2, D. Cusicanqui2,4, W. Castaings3, E. Trouve´1 1 Univ. Savoie Mont Blanc, LISTIC, 74000 Annecy, France 2 CNRS, EDYTEM, Univ. Savoie Mont Blanc, 73000 Chambery,´ France 3 TENEVIA 38240 Meylan, France 4 Universite´ Grenoble Alpes, Institut de Geosciences´ de l’Environement (IGE, UMR 5001), 38000, Grenoble France Commission II, WG II/4,5 KEY WORDS: Photogrammetry, Cryosphere monitoring, Stereo time lapse, Image registration ABSTRACT: In this paper, we propose a method to monitor surface dynamics in mountain cryosphere environments, based on a device with two fixed cameras at two different locations and with a convergent angle. Computer vision methods are used to derive pixel displace- ments between images taken at different times and to reconstruct the 3D geometry of the scene to convert pixel displacements into meter displacements. The proposed methods overcome the drawbacks of traditional methods such as lack of time repeatability or lack of spatial resolution. We present the results on two study sites instrumented with the device located in the French Alps: a temperate glacier and a rock glacier. We propose different ways to assess the accuracy of each processing steps solely based on stereo images. The method is validated using traditional measurements (GPS and LiDAR) and shows results comparable or even more robust than these traditional methods. 1. INTRODUCTION Rasmussen, 1986). Nowadays, the increase in sensor resol- ution and the digital nature of the images, coupled with new The methods to monitor mountain cryosphere, such as glaciers algorithms, allow movements of a few centimeters to be detec- and rock glaciers dynamics rely on two classical approaches: ted (Roncella et al., 2014).
  • Essential' Matrix

    Essential' Matrix

    Recovering Baseline and Orientation from ‘Essential’ Matrix Berthold K.P. Horn January 1990 Abstract: Certain approaches to the problem of relative orientation in binocular stereo (as well as long-range motion vision) lead to an encoding of the baseline (translation) and orientation (rotation) in a single 3 × 3 matrix called the “essential” matrix. The essential matrix is defined by E = BR, where B is the skew-symmetric matrix that satisfies Bv = b × v for any vector v, with b being the baseline and R the orientation. Shown here is a simple method for recovering the two solutions for the baseline and the orientation from a given essential matrix using elementary matrix op- erations. The two solutions for the baseline b can be obtained from the equality 1 bbT = Trace(EET )I − EET , 2 where I is the 3 × 3 identity matrix. The two solutions for the orientation can be found using (b · b) R = Cofactors(E)T − BE, where Cofactors(E) is the matrix of cofactors of E. There is no need to perform a singular value decomposition, to transform coordinates, or to use circular functions, and it is easy to see that there are exactly two solutions given a particular essential matrix. If the sign of E is reversed, an additional pair of solutions is obtained that are related the two already found in a simple fashion. This helps shed some light on the question of how many solutions a given relative orientation problem can have. 1. Coplanarity Condition in Relative Orientation Relative orientation is the well-known photogrammetric problem of recov- ering the position and orientation of one camera relative to another from five or more pairs of corresponding ray directions [Zeller 52] [Ghosh 72] [Slama et al.
  • Generalized Essential Matrix: Properties of the Singular Value Decomposition

    Generalized Essential Matrix: Properties of the Singular Value Decomposition

    Generalized Essential Matrix: Properties of the Singular Value Decomposition Pedro Miraldo1,2, Helder Araujo2 Abstract When considering non-central imaging devices, the computation of the relative pose requires the estimation of the rotation and translation that transform the 3D lines from one coordinate system to the second. In most of the state-of- the-art methods, this transformation is estimated by the computing a 6 × 6 matrix, known as the Generalized Essential Matrix. To allow a better understanding of this matrix, we derive some properties associated with its singular value decomposition. Keywords: Generalized epipolar geometry, relative pose, singular value decomposition, rigid transformation of lines. 1 1. Introduction 2 Relative pose estimation is one of the main problems in computer vision, which has been studied for more than a 3 century [1]. The goal is to estimate the rigid transformation between two cameras (or the same camera in two different 4 positions) using matching between pixels that are images of the same 3D point in the world. The cameras (or camera) 5 are considered calibrated. As a result, for each image pixel, we know the corresponding 3D projection line in the 6 world. Thus, by computing the 3D projection lines associated to each match of pixels, the problem can be seen as 7 finding the rotation and translation that align the 3D projection lines to ensure that they intersect in the world, as 8 shown in Fig. 1. One of the most important applications is its use in robotics navigation, in methods such as visual 9 odometry [2]. 10 When considering conventional perspective cameras there are several solutions for the relative pose.
  • Mathematical Methods for Camera Self-Calibration in Photogrammetry and Computer Vision

    Mathematical Methods for Camera Self-Calibration in Photogrammetry and Computer Vision

    Mathematical Methods for Camera Self-Calibration in Photogrammetry and Computer Vision A thesis accepted by the Faculty of Aerospace Engineering and Geodesy of the Universität Stuttgart in partial fulfillment of the requirements for the degree of Doctor of Engineering Sciences (Dr.-Ing.) by Rongfu Tang born in Guangdong, China P. R. Committee Chair: Prof. Dr.-Ing habil. Dieter Fritsch Committee member: Prof. Dr.-Ing habil. Christian Heipke Date of defence: 28.05.2013 Institute of Photogrammetry University of Stuttgart 2013 Deutsche Geodätische Kommission bei der Bayerischen Akademie der Wissenschaften Reihe C Dissertationen Heft Nr. 703 Rongfu Tang Mathematical Methods for Camera Self-Calibration in Photogrammetry and Computer Vision München 2013 Verlag der Bayerischen Akademie der Wissenschaften in Kommission beim Verlag C. H. Beck ISSN 0065-5325 ISBN 978-3-7696-5115-7 Deutsche Geodätische Kommission bei der Bayerischen Akademie der Wissenschaften Reihe C Dissertationen Heft Nr. 703 Mathematical Methods for Camera Self-Calibration in Photogrammetry and Computer Vision Von der Fakultät Luft- und Raumfahrttechnik und Geodäsie der Universität Stuttgart zur Erlangung der Würde eines Doktors der Ingenieurwissenschaften (Dr.-Ing.) genehmigte Abhandlung Vorgelegt von M.Sc. Rongfu Tang München 2013 Verlag der Bayerischen Akademie der Wissenschaften in Kommission beim Verlag C. H. Beck ISSN 0065-5325 ISBN 978-3-7696-5115-7 Adresse der Deutschen Geodätischen Kommission: Deutsche Geodätische Kommission Alfons-Goppel-Straße 11 ! D – 80 539 München Telefon +49 – 89 – 23 031 1113 ! Telefax +49 – 89 – 23 031 - 1283 / - 1100 e-mail [email protected] ! http://www.dgk.badw.de Hauptberichter: Prof. Dr.-Ing. habil. Dieter Fritsch Mitberichter: Prof.
  • Epipolar Geometry and the Essential Matrix

    Epipolar Geometry and the Essential Matrix

    Epipolar Geometry and the Essential Matrix Carlo Tomasi The epipolar geometry of a pair of cameras expresses the fundamental relationship between any two corresponding points in the two image planes, and leads to a key constraint between the coordinates of these points that underlies visual reconstruction. The first Section below describes the epipolar geometry. The Section thereafter expresses its key constraint algebraically. 1 The Epipolar Geometry of a Pair of Cameras Figure 1 shows the main elements of the epipolar geometry for a pair of cameras. P projection ray projection ray epipolar plane epipolar line of q J p I center of projection q epipole e baseline epipolar line of p camera C camera D center of epipole f projection Figure 1: Essential elements of the epipolar geometry of a camera pair. The world point P and the centers of projection of the two cameras identify a plane in space, the epipolar plane of point P. The Figure shows a triangle of this plane, delimited by the two projection rays and by the baseline of the camera pair, that is, the line segment that connects the two centers of projection.1 If the image planes are thought of extending indefinitely, the baseline intersects the two image planes at two points called the epipoles of the two images. In particular, if the cameras are arranged so that the baseline is parallel to an image plane, then the corresponding epipole is a point at infinity. The epipoles are fixed points for a given camera pair configuration. With cameras somewhat tilted towards each other, and with a sufficiently wide field of view, the epipoles would be image points.
  • Variational 3D Reconstruction from Stereo Image Pairs and Stereo Sequences

    Variational 3D Reconstruction from Stereo Image Pairs and Stereo Sequences

    Variationelle 3D-Rekonstruktion aus Stereobildpaaren und Stereobildfolgen Dissertation zur Erlangung des Grades des Doktors der Naturwissenschaften der Naturwissenschaftlich-Technischen Fakultäten der Universität des Saarlandes vorgelegt von Levi Valgaerts Saarbrücken, 2011 R S V E I T I A N S U S S A I R S A V I E N Mathematische Bildverarbeitungsgruppe, Fakultät für Mathematik und Informatik, Universität des Saarlandes, 66041 Saarbrücken Vision and Image Processing Group, Cluster of Excellence MMCI, Universität des Saarlandes, 66041 Saarbrücken ii Tag des Kolloquiums 29.09.2011 Dekan Prof. Dr. Holger Hermanns Prüfungsausschuss Prof. Dr. Bernt Schiele (Vorsitz) Universität des Saarlandes Prof. Dr. Joachim Weickert (1. Gutachter) Universität des Saarlandes Prof. Dr. Thomas Brox (2. Gutachter) Universität Freiburg Dr. Andrés Bruhn Universität des Saarlandes Kurzzusammenfassung – Short Abstract Kurzzusammenfassung – Deutsch Diese Arbeit befasst sich mit der 3D Rekonstruktion und der 3D Bewegungsschätzung aus Stereodaten unter Verwendung von Variationsansätzen, die auf dichten Verfahren zur Berechnung des optischen Flusses beruhen. Im ersten Teil der Arbeit untersuchen wir ein neues Anwendungsgebiet von dichtem optischen Fluss, nämlich die Bestimmung der Fun- damentalmatrix aus Stereobildpaaren. Indem wir die Abhängigkeit zwischen der geschätzten Stereogeometrie in Form der Fundamentalmatrix und den berechneten Bildkorresponden- zen geeignet ausnutzen, sind wir in der Lage, im zweiten Teil der Arbeit eine gekop- pelte Bestimmung der Fundamentalmatrix und des optischen Flusses vorzuschlagen, die zur einer Erhöhung der Genauigkeit beider Schätzungen führt. Im Gegensatz zu vielen existierenden Verfahren berechnet unser gekoppelter Ansatz dabei die Lage der Kameras und die 3D Szenenstruktur nicht einzeln, sonderen bestimmt sie in einem einzigen gemein- samen Optimierungsschritt. Dem Prinzip der gemeinsamen Schätzung weiter folgend kop- peln wir im letzten Teil der Arbeit die dichte 3D Rekonstruktion der Szene zusätzlich mit der Bestimmung der zugehörigen 3D Bewegung.
  • Lecture 9: Epipolar Geometry

    Lecture 9: Epipolar Geometry

    Lecture 9: Epipolar Geometry Professor Fei‐Fei Li Stanford Vision Lab Fei-Fei Li Lecture 9 - 1 21‐Oct‐11 What we will learn today? •Why is stereo useful? • Epipolar constraints • Essential and fundamental matrix •Estimating F (Problem Set 2 (Q2)) • Rectification Reading: [HZ] Chapters: 4, 9, 11 [FP] Chapters: 10 Fei-Fei Li Lecture 9 - 2 21‐Oct‐11 Recovering structure from a single view Pinhole perspective projection P p Ow Scene C Calibration rig Camera K Why is it so difficult? Intrinsic ambiguity of the mapping from 3D to image (2D) Fei-Fei Li Lecture 9 - 3 21‐Oct‐11 Recovering structure from a single view Pinhole perspective projection Lazebnik S. slide Courtesy Intrinsic ambiguity of the mapping from 3D to image (2D) Fei-Fei Li Lecture 9 - 4 21‐Oct‐11 Two eyes help! Fei-Fei Li Lecture 9 - 5 21‐Oct‐11 Two eyes help! ? x1 x2 K =known K =known R, T O2 O1 This is called triangulation Fei-Fei Li Lecture 9 - 6 21‐Oct‐11 Triangulation 2 2 • Find X that minimizes d ( x1 , P1 X ) d ( x 2 , P2 X ) X x2 x1 O1 O2 Fei-Fei Li Lecture 9 - 7 21‐Oct‐11 Stereo‐view geometry • Correspondence: Given a point in one image, how can I find the corresponding point x’ in another one? • Camera geometry: Given corresponding points in two images, find camera matrices, position and pose. • Scene geometry: Find coordinates of 3D point from its projection into 2 or multiple images. This lecture (#9) Fei-Fei Li Lecture 9 - 8 21‐Oct‐11 Stereo‐view geometry Next lecture (#10) • Correspondence: Given a point in one image, how can I find the corresponding point x’ in another one? • Camera geometry: Given corresponding points in two images, find camera matrices, position and pose.
  • Inaugural-Dissertation

    Inaugural-Dissertation

    Inaugural-Dissertation zur Erlangung der Doktorw¨urde der Naturwissenschaftlich-Mathematischen Gesamtfakult¨at der Ruprecht–Karls–Universit¨at Heidelberg vorgelegt von Master of Mathematics and Computer Science Andreas Neufeld aus Ramenskoje (Russland) Tag der m¨undlichen Pr¨ufung: Variational Approaches for Motion and Structure from Monocular Video Betreuer: Prof. Dr. Christoph Schn¨orr Prof. Dr. Bj¨orn Ommer Zusammenfassung Das Sch¨atzen von Bewegung und r¨aumlicher Struktur aus Bildfol- gen sind grundlegende Probleme in der Bildverarbeitung. Diese Prob- lemstellungen sind noch aktuell, obwohl die ersten Methoden bereits vor mehreren Jahrzehnten ver¨offentlicht wurden. Wir pr¨asentieren neue Verfahren zur Bewegungs- und Struktursch¨atzung f¨ur das au- tonome Fahren. Ein autonomes Auto braucht genaue Kenntnis ¨uber seine Umgebung, da eine Fehleinsch¨atzung gravierende Konsequenzen haben kann. Speziell behandeln wir monokulare Verfahren, bei denen nur eine Kamera auf dem Fahrzeug verf¨ugbar ist. Bildfolgen aus dem Straßenverkehr sind f¨ur das Sch¨atzen von Be- wegung besonders herausfordernd. Durch die hohe Geschwindigkeit ist die Bewegung sehr groß, die Lichtverh¨altnisse sind nicht stabil und es kann Verf¨alschungen geben durch Reflektionen und wetterbedingte St¨orungen. Wir stellen neue diskrete Verfahren zur Berechnung des optischen Flusses vor, welche probabilistische graphische Modelle f¨ur den optischen Fluss definieren. In dem ersten Ansatz w¨ahlen wir einige Stellen im Referenzbild aus, und vergleichen diese mit dem zweiten Bild. Die besten Korresponden- zen, welche auch zu einer monokularen Bewegung passen, werden als Kandidaten f¨ur ein graphisches Modell ausgew¨ahlt. In einem weit- eren Verfahren, vergleichen wir alle Stellen im Referenzbild, l¨osen das graphische Modell jedoch nicht direkt, sondern approximieren es mit einer Sequenz von kleineren Modellen.
  • Fundamental Matrix / Image Rectification

    Fundamental Matrix / Image Rectification

    Fundamental Matrix / Image Rectification COMPSCI 773 S1 T VISION GUIDED CONTROL A/P Georgy Gimel’farb Epipolar Geometry • Ol,Or - projection centres – Origins of the reference frames – fl, fr - focal lengths of cameras • πl, πr - image planes – 3D reference frame for each camera: Z-axis = the optical axis T T Pl=[Xl,Yl,Zl] , Pr=[Xr,Yr,Zr] - the same 3D point P in the reference frames T T pl=[xl ,yl, zl =fl] , pr=[xr,yr, zr =fr] - projections of P onto the image planes Epipoles COMPSCI 773 1 Basics of Epipolar Geometry • Reference frames of the left and right cameras - related via the extrinsic parameters – Translation vector T = (Or - Ol) and a rotation matrix R defining a rigid transformation in 3-D space, given a 3-D point P, between Pl and Pr: Pr = R(Pl - T) • Epipoles el and er - the points at which the line through the centres of projection intersects the image planes – Left epipole - the image of the right projection centre – Right epipole - the image of the left projection centre – Canonical geometry: the epipole is at infinity of the baseline COMPSCI 773 2 Basics of Epipolar Geometry T • 3-D point P = [X,Y,Z] ⇔ its projections pl and pr: flPl f rPr pl = ; pr = Zl Zr • Epipolar plane: the plane through P, Ol, and Or – Epipolar line: its intersection with each image plane – Conjugated€ lines: both the lines for an epipolar plane – Given pl, the 3-D point P can lie anywhere on the ray pl Ol depicted by the epipolar line through the corresponding pr – Epipolar constraint: the true match lies on the epipolar line COMPSCI 773 3 Basics of Epipolar Geometry • All epipolar lines go through the epipole – With the exception of the epipole, only one epipolar line goes through any image point – Mapping between points on one image and corresponding epipolar lines on the other image ⇒ the 1-D search region – Rejection of false matches due to occlusions – Corresponding points must lie on conjugated epipolar lines • The obvious question: how to estimate the epipolar geometry, i.e.
  • Epipolar Geometry: Essential Matrix

    Epipolar Geometry: Essential Matrix

    CS4501: Introduction to Computer Vision Epipolar Geometry: Essential Matrix Various slides from previous courses by: D.A. Forsyth (Berkeley / UIUC), I. Kokkinos (Ecole Centrale / UCL). S. Lazebnik (UNC / UIUC), S. Seitz (MSR / Facebook), J. Hays (Brown / Georgia Tech), A. Berg (Stony Brook / UNC), D. Samaras (Stony Brook) . J. M. Frahm (UNC), V. Ordonez (UVA), Steve Seitz (UW). Last Class • Stereo Vision – Dense Stereo • More on Epipolar Geometry Today’s Class • More on Epipolar Geometry • Essential Matrix • Fundamental Matrix Es#ma#ng depth with stereo • Stereo: shape from “motion” between two views • We’ll need to consider: • Info on camera pose (“calibration”) • Image point correspondences scene point image plane optical center Key idea: Epipolar constraint X X X x x ’ x ’ x ’ Potential matches for x have to lie on the corresponding line l’. Potential matches for x’ have to lie on the corresponding line l. Epipolar geometry: notation X x x ’ • Baseline – line connecting the two camera centers • Epipoles = intersec9ons of baseline with image planes = projec9ons of the other camera center • Epipolar Plane – plane containing baseline (1D family) Epipolar geometry: notation X x x ’ • Baseline – line connecting the two camera centers • Epipoles = intersections of baseline with image planes = projections of the other camera center • Epipolar Plane – plane containing baseline (1D family) • Epipolar Lines - intersections of epipolar plane with image planes (always come in corresponding pairs) Epipolar Geometry: Another example Credit: William