Complete Endomorphisms in Computer Vision Applied to Multiple View Geometry
Total Page:16
File Type:pdf, Size:1020Kb
Complete Endomorphisms in Computer Vision Applied to multiple view geometry Finat, Javier · Delgado-del-Hoyo, Francisco July 26, 2021 Abstract Correspondences between k-tuples of points 1 Introduction are key in multiple view geometry and motion analy- sis. Regular transformations are posed by homographies A classical issue regarding 3D reconstruction and mo- between two projective planes that serves as structural tion analysis concerns the preservation of the continuity models for images. Such transformations can not in- of the scene or the flow, although small changes in input clude degenerate situations. Fundamental or essential occurs. There are a lot of answers where different con- matrices expand homographies with structural infor- straints have been introduced from the early eighties. mation by using degenerate bilinear maps. The projec- The most common constraint is the structural bilinear tivization of the endomorphisms of a three-dimensional - or multilinear - tensor created with k-tuples of corre- vector space includes all of them. Hence, they are able sponding elements (points and/or lines) for the camera to explain a wider range of eventually degenerate trans- pose. All these constraints are very sensitive to noise. formations between arbitrary pairs of views. To include Classical approaches are based on minimal solutions ex- these degenerate situations, this paper introduces a com- tracted from noise measurements following a RANSAC1 pletion of bilinear maps between spaces given by an scheme. However, indeterminacies persist for some de- equivariant compactification of regular transformations. generate situations - created by low rank matrices - that This completion is extensible to the varieties of fun- can arise for mobile cameras. damental and essential matrices, where most methods Classic literature in computer vision [4,9] display based on regular transformations fail. The construction a low attention to degenerate cases in structural mod- of complete endomorphisms manages degenerate pro- els. These cases arise when independence conditions for jection maps using a simultaneous action on source and features are not fulfilled, including situations where the target spaces. In such way, this mathematical construc- camera turns around its optical axis or it is in fronto- tion provides a robust framework to relate correspond- parallel position w.r.t. a planar surface (a wall or the ing views in multiple view geometry. ground, e.g.). Then, the problem is ill-posed, and con- ventional solutions consists in performing a \small per- arXiv:2002.09003v1 [cs.CV] 20 Feb 2020 turbation" or reboot the process. Both strategies dis- Keywords Epipolar Geometry · Essential Matrix · play issues concerning the lack of control about the Fundamental Matrix · Degeneracies · Secant Varieties · perturbation to be made that generate undesirable dis- Adjoint Representation continuities. Thus, it is important to develop alterna- tive strategies which can maintain some kind of \coher- MoBiVAP research group ence" by reusing the \recent history" of the trajectory. Universidad de Valladolid History is continuously modeled in terms of generically Campus Miguel Delibes regular conditions for tensors in previously sampled im- Paseo de Bel´en,11 ages with a discrete approach of a well-defined path in 47010, Valladolid, Spain Tel.: +34-983-184-398 the space of structural tensors. Unfortunately, degener- https://www.mobivap.es E-mail: franciscojavier.fi[email protected] 1 Random Sample Consensus 2 Finat, Javier, Delgado-del-Hoyo, Francisco acy conditions for typical features give indeterminacy containing their degenerate cases, which are usually for limits of structural tensors, which must be removed. excluded from the analysis. They are recovered by in- Our approach consists of considering Kinematic infor- troducing a \compactification" where degenerate cases mation of the matrix version of the gradient field for are managed in terms of successive envelopes by linear indeterminacy loci. subspaces W of V . All arguments can be extended to Less attention has been paid to preserve the \conti- higher dimensions and even to hypermatrices represent- nuity" of eventually singular trajectories in the space of ing more sophisticated tensors. However, for simplifica- bilinear maps linked to the automatic correspondence tion purposes, we constraint ourselves only to endomor- between pairs of views. In this case, singular maps are phisms extending planar or spatial homographies to the the responsible for indeterminacies in tensors and lie on singular case. singular strata of the space of bilinear maps[24]. In this The rest of the paper is organized as follows. Section work, we develop a more down-to-earth approach using 2 provides the mathematical background to understand some basic properties of the projectivization of spaces the rest of the paper and frames our approach in the of endomorphisms, including homographies H, funda- state of the art. Section 3 analyzes the simplest cases mental F and essential E matrices. All of them can be including regular transformations defined by homogra- described in terms of orbits by a group action on the phies for the planar case that relate 2D views using the space of endomorphisms End(V ), i.e. linear maps of a fundamental variety. Section 4 extends the approach vector space V in itself. Their simultaneous algebraic to rigid transformations in the third dimension, includ- treatment allows to extend the algebraic completion to ing metric aspects in terms of the essential variety in- the space of eventually degenerated central projection volving source and target spaces in P3. Section 5 stud- maps MC with center C. Intuitively, the key for the ies the structural connection between them using the control of degenerate cases is to select appropriate lim- simultaneous action on source and target spaces of a its of tangents in a \more complete" space. variable projection linked to the camera pose; left-right Therefore, the main goal of our work is to model A and contact K-equivalences are explained. Section 6 a continuous solution that also considers degenerated provides additional insight concerning the details and cases for simplest tensors (fundamental and essential practical considerations for implementing this approach 2 matrices, e.g.) such those appearing in structural mod- in VO systems. Finally, Section 7 concludes the paper els for 3D Reconstruction. In order to achieve this goal, with a summary of the main results and guidelines for we introduced an \equivariant compactification” of the further research. space of matrices w.r.t. group actions linked to kine- matic properties visualized in the dual space. This dou- ble representation (positions and \speed") stores the 2 Background \recent history" represented by a path in the tangent bundle τEnd(V ) to End(V ) for the trajectory of a mobile Local symmetries are ubiquitous in a lot of problems camera C(t). Our approach is based on a dual presen- in Physics and Engineering involving propagation phe- tation for the rank stratification of matrices. This dual nomena. Most approaches in applied areas consider only representation encodes tangential information at each regular regions, by ignoring any kind of degenerations point represented by the adjoint matrix, a 3 × 3-matrix linked to rank deficient matrices linked to linearization whose entries represent the gradient r(det(A)) of the of phenomena. To include them, we develop a \locally determinant of A. symmetric completion" of eventually singular transfor- In the simplest case, after fixing a basis BV of V , en- mations for involved tensors such those appearing in domorphisms of a 3D vector space V ' R3 are given by Reconstruction issues. Our approach is not quite origi- arbitrary 3×3-matrices; they are naturally stratified by nal; a similar idea can be found in [26], which introduces the rank giving three orbits with rank r ≤ 3. In partic- a locally symmetric structure in a differential frame- ular, from the differential viewpoint, sets of homogra- work concerning geodesics on the essential manifold. phies H and regular (i.e. rank 2) fundamental matrices Nevertheless, the initial geometric description as sym- F can be considered as two G-orbits of the Lie algebra metric space (union of orbits linked to the rank preser- g = End(V 3) of G = GL(3) = Aut(V 3) by the action of vation) can not be extended to include a differential the projectivization PGL(3) of the general linear group approach to degenerate cases. Due to the occurrence of corresponding to rank 3 and rank 2 matrices, respec- singularities, the support given as a quotient variety is tively. More generally, the description of End(V n+1) as not a smooth manifold but a singular algebraic variety a union of orbits by the action of GL(n + 1; R) gives a structure as an \orbifold", i.e. a union of G-orbits 2 Visual Odometry Complete Endomorphisms in Computer Vision 3 where the methods for Riemannian manifolds no longer double conjugacy classes from the algebraic viewpoint. apply. The A-action is very useful for decoupled models (im- A larger description of a locally symmetric struc- plicit in KLT algorithms or SfM4, e.g.), and conse- ture may be performed by extending the ordinary al- quently very useful by computational reasons. Despite gebraic approach. Roughly speaking, it suffices to add the wide interest for the above approaches, the A-action limits of tangent subspaces along different \branches" is less plausible than the K-action, which incorporates at singularities and \extend the action" to obtain a the graph preservation (corresponding to quadratic con- more complete description including kinematic aspects. tact between a manifold M and its tangent space TpM In this way, one obtains a \local replication" compati- at each contact point p 2 M) as the structural con- ble with the presence of singularities in the adherence of straint. \augmented" orbits by tangent spaces at regular points. In our case, contact equivalence is based on a cou- So, for the \subregular" case - codimension one orbit pling between images and scene representations.