Kenichi Kanatani

Group-Theoretical Methods in Image Understanding

With 138 Figures

Springer-Verlag Berlin Heidelberg NewYork London Paris Tokyo Hong Kong Contents

1. Introduction 3 1.1 What is Image Understanding? 3 1.2 Imaging of Perspective Projection 5 1.3 Geometry of Camera 9 1.3.1 Projective Transformation 12 1.4 The 3D Euclidean Approach 13 1.5 The 2D Non-Euclidean Approach 16 1.5.1 Relative Geometry and Absolute Geometry 18 1.6 Organization of This Book 19

2. Coordinate Rotation Invariance of Image Characteristics 21 2.1 Image Characteristics and 3D Recovery Equations 21 2.2 Parameter Changes and Representations ? 22 2.2.1 Rotation of the Coordinate System 25 2.3 Invariants and Weights 25 2.3.1 Signs of the Weights and Degeneracy 27 2.4 Representation of a Scalar and a Vector 28 2.4.1 Invariant Meaning of a Position 29 2.5 Representation of a Tensor 29 2.5.1 Parity 34 2.5.2 Weyl's Theorem 35 2.6 Analysis of Optical Flow for a Planar Surface 35 2.6.1 Optical Flow 45 2.6.2 Translation Velocity, Rotation Velocity, and Egomotion. 46 2.6.3 Equations in Terms of Invariants 48 2.7 Shape from Texture for Curved Surfaces 48 2.7.1 Curvatures of a Surface 56 Exercises 58

3. 3D Rotation and Its Irreducible Representations 61 3.1 Invariance for the Camera Rotation Transformation 61 5.1.1 SO(3) Is Three Dimensional and Not Abelian 62 3.2 Infinitesimal Generators and 63 3.3 Lie Algebra and Casimir Operator of the 3D Rotation . 69 3.3.1 Infinitesimal Commute 71 3.3.2 Reciprocal Representations 73 3.3.3 74 X Contents

3.4 Representation of a Scalar and a Vector 74 3.5 Irreducible Reduction of a Tensor 76 3.5.1 Canonical Form of Infinitesimal Generators 80 3.5.2 Alibi vs Alias 82 3.6 Restriction of SO(3) to SO(2) 83 3.6.1 Broken 85 3.7 Transformation of Optical Flow 85 3.7.1 Invariant Decomposition of Optical Flow 98 Exercises 99

4. Algebraic Invariance of Image Characteristics 103 4.1 Algebraic Invariants and Irreducibility 103 4.2 Scalars, Points, and Lines 105 4.3 Irreducible Decomposition of a Vector 109 4.4 Irreducible Decomposition of a Tensor 112 4.5 Invariants of Vectors 115 4.5.1 Interpretation of Invariants 119 4.6 Invariants of Points and Lines 121 4.6.1 Spherical Geometry 126 4.7 Invariants of Tensors 129 4.7.1 Symmetric Polynomials 134 4.7.2 Classical Theory of Invariants 136 4.8 Reconstruction of Camera Rotation 137 Exercises 142

5. Characterization of Scenes and Images 147 5.1 Parametrization of Scenes and Images 147 5.2 Scenes, Images, and the Projection Operator 148 5.3 Invariant Subspaces of the Scene Space 152 5.3.1 Tensor Calculus 154 5.4 157 5.5 Tensor Expressions of Spherical Harmonics 161 5.6 Irreducibility of Spherical Harmonics 165 5.6.1 Laplace Spherical Harmonics 170 5.7 Camera Rotation Transformation of the Image Space 172 5.7.1 Parity of Scenes 176 5.8 Invariant Measure 176 5.8.1 First Fundamental Form 178 5.8.2 Fluid Dynamics Analogy 179 5.9 Transformation of Features 181 5.10 Invariant Characterization of a Shape 186 5.10.1 Invariance on the Image 191 5.10.2 Further Applications 192 Exercises 194 Contents XI

6. Representation of 3D Rotations 197 6.1 Representation of Object Orientations 197 6.2 Rotation ,, 197 6.2.1 The nD Rotation Group SO{n) 201 6.3 Rotation Axis and Rotation 202 6.4 Euler 205 6.5 Cayley-Klein Parameters 207 6.6 Representation of SO(3) by SU(2) 210 6.6.1 Spinors 215 6.7 of SU(2) 215 6.7.1 Differential Representation 218 6.8 220 6.8.1 Field 224 6.9 Topology of SO(3) 225 6.9.1 Universal 229 6.10 Invariant Measure of 3D Rotations 230 Exercises 233

7. Shape from Motion 239 7.1 3D Recovery from Optical Flow for a Planar Surface 239 7.2 Flow Parameters and 3D Recovery Equations 240 7.2.1 Least Squares Method 243 7.3 Invariants of Optical Flow 245 7.4 Analytical Solution of the 3D Recovery Equations 249 7.5 Pseudo-orthographic Approximation 253 7.5.1 Robustness of Computation 256 7.6 Adjacency Condition of Optical Flow 256 7.7 3D Recovery of a Polyhedron 260 7.7.1 Noise Sensitivity of Computation 262 7.8 Motion Detection Without Correspondence 263 7.8.1 Stereo Without Correspondence 271 Exercises 274

8. Shape from Angle 278 8.1 Rectangularity Hypothesis 278 8.2 Spatial Orientation of a Rectangular Corner 280 8.2.1 Corners with Two Right Angles 288 8.3 Interpretation of a Rectangular Polyhedron 289 8.3.1 Huffman-Clowes Edge Labeling 299 8.4 Standard Transformation of Corner Images 301 8.5 Vanishing Points and Vanishing Lines 309 8.5.1 Spherical Geometry and Projective Geometry 322 Exercises 323 XII Contents

9. Shape from Texture 327 9.1 Shape from Texture from Homogeneity 327 9.2 Texture Density and Homogeneity ' 328 9.2.1 Distributions 333 9.3 Perspective Projection and the First Fundamental Form 334 9.4 Surface Shape Recovery from Texture 337 9.5 Recovery of Planar Surfaces 340 9.5.1 Error Due to Randomness of the Texture 341 9.6 Numerical Scheme of Planar Surface Recovery 343 9.6.1 Technical Aspects of Implementation 353 9.6.2 Newton Iterations 353 Exercises 354

10. Shape from Surface 356 10.1 What Does 3D Shape Recovery Mean? 356 10.2 Constraints on a 2|D Sketch 359 10.2.1 Singularity of the Incidence Structure 364 ' 10.2.2 Integrability Condition 367 10.3 Optimization of a 2|D Sketch 368 10.3.1 Regularization 374 10.4 Optimization for Shape from Motion 377 10.5 Optimization of Rectangularity Heuristics 379 10.6 Optimization of Parallelism Heuristics 384 10.6.1 Parallelogram Test and Computational Geometry. . . . 392 Exercises 394

Appendix. Fundamentals of Group Theory 398 A.I Sets, Mappings, and Transformations 398 A.2 Groups 402 A.3 Linear Spaces 405 A.4 Metric Spaces 408 A.5 Linear Operators 412 A.6 415 A.7 Schur's Lemma 418 A.8 Topology, , and Lie Groups 421 A.9 Lie Algebras and Lie Groups 426 A.10 Spherical Harmonics 431

Bibliography 436

Subject Index 448