Grouping, Matching and Reconstruction in Multiple View Geometry F. Schaffalitzky Robotics Research Group Department of Engineering Science University of Oxford, UK April 2002 Contents 1 Introduction 6 1.1 Objective and Motivation . 6 1.2 Overview . 8 2 Grouping with geometric constraints 12 2.1 Overview . 12 2.2 Objective and background . 12 2.3 Theory and geometry . 14 2.3.1 Parallel lines . 14 2.3.2 Equally spaced parallel lines . 15 2.3.3 Elations . 18 2.3.4 Regular grids . 21 2.4 Algorithms . 21 2.4.1 Vanishing point detection . 22 2.4.2 Computing the equally spaced line geometry . 24 2.4.3 Computing elations . 27 2.4.4 Computing grids . 28 2.5 Results . 30 2.5.1 Vanishing points . 30 2.5.2 Equally spaced lines . 30 2.5.3 Elations . 35 2.5.4 Grids . 40 1 2.6 Assessment . 40 3 Projective reconstruction 43 3.1 Overview . 43 3.2 Motivation and objective . 43 3.3 Theory . 45 3.3.1 Problem formulation and notation . 46 3.3.2 Pencils of cameras . 47 3.3.3 Derivation of quadratic constraints . 47 3.3.4 Five irrelevant solutions . 48 3.3.5 Factoring the constraints . 48 3.3.6 Cubic constraint . 50 3.3.7 Quasi-linear method for reconstruction . 51 3.3.8 Scaling the constraints . 53 3.3.9 Geometric error . 53 3.3.10 Related Work . 55 3.4 Algorithm details . 55 3.4.1 Computing camera pencils . 55 3.4.2 Inverting . 58 3.4.3 Robust Reconstruction Algorithm . 59 3.5 Results . 60 3.5.1 Synthetic data . 60 3.5.2 Real data I . 62 3.5.3 Real data II . 64 3.6 Assessment . 66 4 Auto-calibration 67 4.1 Overview . 67 4.2 Motivation . 67 4.3 Modulus constraints . 70 2 4.3.1 Motivation . 71 4.3.2 Algebraic nullspaces . 72 4.3.3 Horopters . 73 4.3.4 Characteristic equation of ¢¡¤£¦¥¨§ © . 76 4.3.5 Two-view (strong) modulus constraint . 77 4.3.6 Two-view (algebraic) quartic modulus constraint . 78 4.3.7 Three-view (algebraic) cubic modulus constraint . 79 4.3.8 Solving the equations . 79 4.3.9 Experimental Results. 87 4.3.10 Partial constraint . 89 4.3.11 Conclusion and further work . 90 4.4 Square pixels . 91 4.4.1 Notation . 93 4.4.2 Co-conicity constraint on § . 95 4.4.3 Pascal's theorem . 95 4.4.4 Collinearity in 3D . 97 4.4.5 Octic constraint . 98 4.4.6 Special case of 3D collinearity bracket . 99 4.4.7 Formula for sextic . 100 4.4.8 Formula for quintic . 100 4.4.9 Forty solutions . 103 4.5 Algorithms . 108 4.5.1 Recovering calibration from the plane at infinity . 108 4.5.2 Numerical Computation of Ideal Saturation . 109 4.5.3 Computing the Multiplication Operators . 111 4.5.4 Solving the Generalized Eigensystem . 111 4.6 Results . 112 4.7 Assessment . 113 3 5 Matching images 114 5.1 Objective and motivation . 114 5.1.1 What can one hope for? . 114 5.1.2 Direct methods . 116 5.1.3 Wide versus short base-line. 117 5.2 Framework of invariant descriptors . 118 5.2.1 Interest point neighbourhood descriptors . 120 5.2.2 Intensity profiles descriptors . 127 5.2.3 Region descriptors . 127 5.2.4 Texture descriptors . 128 5.2.5 Geometric features . 128 5.3 Local (affine) transformations . 129 5.4 Algorithms . 131 5.4.1 Corner detector . 132 5.4.2 Shape adaptation . 133 5.4.3 Invariants used . 134 5.4.4 Registration . 135 5.4.5 Growing new matches from old . 136 5.4.6 Constraints on multi-view geometry from local affine transformation 136 5.5 Results . 138 5.6 Assessment . 138 6 Texture 153 6.1 Objective . 153 6.2 Background . 153 6.2.1 Texture description and models . 155 6.3 Theory . 161 6.3.1 Segmentation . 161 6.3.2 The 2nd moment matrix . 162 4 6.3.3 Behaviour under affine transformations . 162 6.3.4 Weak (gradient) isotropy . 165 6.3.5 Previous use of the 2nd moment matrix . 165 6.3.6 Description in normalized frame . 166 6.4 Algorithms . 169 6.4.1 Normalized Cut Segmentation . 169 6.4.2 Practical calculation of 2nd moment matrix . ..