3D Surface Models by Geometric Constraints Propagation M. Farenzena A. Fusiello LASMEA, UMR6602 CNRS Dipartimento di Informatica Université Blaise Pascal, Clermont, France University of Verona, Verona, Italy [email protected] [email protected] Abstract consuming. In other cases [3, 7], geometric constraints are detected automatically thanks to prior knowledge about the This paper proposes a technique for estimating piece- model to reconstruct. wise planar models of objects from their images and geo- Besides, the analysis of constraints is usually over- metric constraints. First, assuming a bounded noise in the looked. In fact, datasets with many points and geometric localization of 2D points, the position of the 3D point is constraints do not necessarily define a consistent and unique estimated as a polyhedron containing all the possible solu- 3D object. Parts of the scene may not be rigidly connected, tions of the triangulation. Then, given the topological struc- so that there exist various shapes that verify the geometric ture of the 3D points cloud, geometric relationships among constraints and project to identical image points. In addi- facets, such as coplanarity, parallelism, orthogonality, and tion, constraints may be redundant, making the optimiza- angle equality, are automatically detected. A subset of them tion uselessly harder or even unfeasible. [10] proposes an that is sufficient to stabilize the 3D model estimation is se- algebraic method to check whether a configuration of points lected with a flow-network based algorithm. Finally a fea- and constraints leads to a unique reconstruction, but it does sible instance of the 3D model, i.e. one that satisfies the not deal with redundancies. As far as we know, a principled selected geometric relationships and whose 3D points lie analysis of constraints has not been considered in literature within the associated polyhedral bounds, is computed by yet. solving a Constraint Satisfaction Problem. After detecting them, geometric constraints may be di- rectly embedded into the minimization of the reprojection error (or bundle adjustment) [3, 7], but this causes a sub- 1. Introduction stantial increase of computational costs and both conver- The problem of recovering 3D surface models from im- gence and exact constraint satisfaction are not guaranteed. ages and geometric clues has been widely studied in liter- An alternative is to make the geometric constraints implicit ature. The proposed methods can be mainly classified as in the parametrization of 3D points [1, 10, 21], so as they model-based and constraint-based. are satisfied exactly at every optimization step. A different In model-based methods [6, 13, 16, 20], the scene is de- approach [8] avoids altogether the non-linear least-squares fined as in CAD systems: objects are the assemblage of problem arising in the methods above, for it casts the prob- known primitive shapes. Reconstruction is carried out by lem as a Constraint Satisfaction (CSP), where the 3D point fitting a 3D model to image data, thus determining its di- positions are bounded by 3D boxes (obtained by triangula- mension, its position and orientation. The fact that the scene tion with Interval Analysis) and a feasible solution is one must be decomposable in primitive shapes is the main limi- that satisfies all selected geometric relationships and whose tation of such methods. 3D points lie within the associated bounds. Constraint-based methods [1, 3, 7, 8, 10, 21] are more This paper follows the same general approach to the flexible, as they do not rely on a-priori models but use sim- problem as in [8], but differently from that the bounds on ple primitives like points and lines. Geometric information, the 3D points are precisely estimated. It is known in fact such as orthogonality, parallelism, or planarity, is given in that Interval Analysis provides overestimations of the solu- the form of constraints on 3D points and reconstruction is tion, and can explode quite unpredictably. Moreover, this obtained as the solution of an optimization process. work is enriched by the automatic detection of constraints, In most of previous works, e.g. [1, 8, 10, 20], the con- provided manually in [8], and by the consequent analysis straints detection phase requires the user to provide a geo- and pruning of these constraints. That permits to verify if metrical description of the model, which can be very time- a unique solution can be provided, and at the same time to 1 prune redundancies. 3. Polyhedral triangulation Once camera matrices are known, the first and most im- 2. System Overview portant stage of model reconstruction consists in recovering The approach, summarized in Fig. 1, is made of two the coordinates of points in 3D space given their images in parts. The first part deals with triangulation, i.e., recon- two or more views. It is usually assumed that the camera structing 3D points from their corresponding image points matrices are known exactly, or at least with greater accu- (provided manually in this paper) and known camera ma- racy than point localization. In the absence of noise, i.e. trices (from a Structure-and-Motion pipeline). A statistical when correspondences are perfectly detected, the problem optimal solution, under the assumption of Gaussian noise, is trivial, involving only finding the intersection of rays in exists for two [11] and three views [19], but seems to be the space. If data are perturbed, however, the rays corre- unfeasible beyond that. We use instead as noise model, sponding to back-projections of image points do not inter- a uniform distribution inside a rectangular region centered sect, and obtaining the 3D coordinates of the reconstructed around each image point. This enables us to compute the points becomes far from trivial, as witnessed by the renewed correct solution for any number of views as a polyhedron interest aroused by this issue [15, 19]. that contains all the possible 3D point positions. This poly- As in [8], the proposed method refrains from searching hedron can be regarded as representing the probability dis- for one optimal solution and compute instead a set of possi- tribution function over the 3D point positions, which is uni- ble solutions (defined in terms of errors affecting the image form inside the polyhedron, and zero outside. points) that contains the error-free solution. This permits to Evaluating uncertainty is crucial when the results are to bound the exact solution in the 3D space for any number of be used as input for other processes. Albeit simple in con- views and to estimate, at the same time, the uncertainty of cept, this polyhedral triangulation is a principled and effi- the result. cient approach for evaluating 3D point positions and the as- Let P i, i = 1, . , n be a sequence of n known cam- sociated uncertainty, and represents the counterpoise of the eras and mi be the image of some unknown point M in 3D ML approach using the Gaussian noise model. space, both expressed in homogeneous coordinates. It is The second part of the paper focuses on obtaining a com- assumed that the localization error is bounded by a rectan- plete surface model. Given a set of reconstructed 3D points, gular region Bi centered around each image point (one can represented by the polyhedra provided by the above 3D tri- imagine a uniform noise distribution inside Bi). Each re- angulation, and the connectivity of the points into triangular gion Bi bounds the possible locus of the 3D point inside a facets (provided manually), the method consists in a three- semi-infinite pyramid Qi with its apex in the camera center steps automatic process. First the geometric relationships, (see Fig. 2). The solution set is defined as the polyhedron such as coplanarity, parallelism, etc., are detected; then a formed by the intersection of the n semi-infinite pyramids set of minimal relationships that allow a unique reconstruc- generated by the intervals B1,..., Bn. Analytically, this re- tion is selected using the structural rigidity analysis; finally, gion is defined as the following set: a feasible instance of the 3D model, i.e. one that satisfies all selected geometric relationships and whose 3D points lie D = Q1 ∩ Q2 · · · ∩ Qn = within the associated polyhedral bounds, is computed using ={M: ∃mi ∈ B , i = 1 . n s.t. ∀i: mi ' P iM}. (1) a constrained optimization technique. i &£©;< !¥© =!¦¥ $¡©¥¢¦£ #78 9 # ¡¢£¤¥¢¦§ # £ ¢!¥¦ ¢ £ ¢!¥¦ ¢ ¨©¥ ¨!¡ ¦ $ "¢!¥§¢¦£ ¢ §¢¦£ : : £ ¢!¥¦ ¢ %& ()*+,-,.+/, ¨!£¨¥$¥¢¦£ £!¤¡©¥¢¦£ ' Figure 2. The semi-infinite pyramid Qi is defined from the camera *01213456,0 centre Ci and the bound Bi. Figure 1. Overview of the proposed method. External inputs are: camera matrices, 2D point correspondences, 3D points connectiv- Instead of approximating D using Interval Analysis as ity. in [8], it can be estimated efficiently using computational geometry techniques. The semi-infinite pyramid Qi can be written as the inter- 2. Then, within each group, the clustering is refined by i i i i section of the four negative half-spaces H1, H2, H3, H4 de- taking into account also the distance to the origin of the fined by its supporting planes. Thus, the solution set D can plane containing the facet. In this way facets belonging be expressed as the intersection of 4n negative half-spaces: to parallel planes are separated. \ D = Hi (2) We adopted in both cases the uniform kernel, i.e., a mul- ` tidimensional unite sphere, with bandwidth automatically i=1...n `=1...4 selected as described in [5]. Please note that the process clusters together facets be- The vertices and the faces of D can be enumerated in longing to the same plane, regardless of their distance. O(n log n) time, being n the number of cameras [18]. As an example, Fig. 3 shows the polyhedral triangu- 4.2. Constraints extraction lation result obtained from seven calibrated images of a Lego object.