3D Modeling from Wide Baseline Range Scans Using Contour Coherence

3D Modeling from Wide Baseline Range Scans using Contour Coherence Ruizhe Wang Jongmoo Choi Gerard´ Medioni Computer Vision Lab, Institute for Robotics and Intelligent Systems University of Southern California fruizhewa, jongmooc, [email protected] Abstract 1 2 Registering 2 or more range scans is a fundamental problem, with application to 3D modeling. While this problem is well addressed by existing techniques such as ICP when the views overlap significantly at a good initialization, no satisfactory solution exists for wide baseline regis- 1 2 tration. We propose here a novel approach which leverages (a) contour coherence and allows us to align two wide baseline range scans with limited overlap from a poor initializa- 1 2 tion. Inspired by ICP, we maximize the contour coherence by building robust corresponding pairs on apparent contours and minimizing their distances in an iterative fashion. We use the contour coherence under a multi-view rigid registration framework, and this enables the reconstruction of accurate and complete 3D models from as few as 4 frames. We further extend it to handle articulations, and this allows 1 2 us to model articulated objects such as human body. Exper- (b) imental results on both synthetic and real data demonstrate Figure 1. (a) Two roughly aligned wide baseline 2.5D range scans the effectiveness and robustness of our contour coherence of the Stanford Bunny with the observed and predicted apparent based registration approach to wide baseline range scans, contours extracted. The two meshed points cloud are generated and to 3D modeling. from the two 2.5D range scans respectively (b) Registration result after maximizing the contour coherence 1. Introduction points on the surface where the tangent plane contains the Registering 2 or more range scans is a fundamental prob- line of sight from the camera. This contour has been shown lem with application to 3D modeling. It is well addressed to be a rich source of geometric information for motion es- in the presence of sufficient overlap and good initializa- timation and 3D reconstruction [7,8,9, 12]. tion [3,4,5, 14]. However, registering two wide baseline Inspired by their works, we propose the concept of con- range scans presents a challenging task where two range tour coherence for wide baseline range scan registration. scans barely overlap and the shape coherence no longer pre- Contour coherence is defined as the agreement between the vails. An example of two wide baseline range scans of the observed apparent contour and the predicted apparent con- Stanford Bunny with approximately 40% overlap is given tour. As shown in Fig. 1(a), the observed contours extracted in Fig. 1(a). The traditional shape coherence based meth- from the original 2.5D range scans, i.e., red lines in image ods may fail as most closest-distance correspondences are 1 and blue lines in image 2, do not match the corresponding incorrect. contours extracted from the projected 2.5D range scans, i.e., In computer vision dealing with intensity images, a large blue lines in image 1 and red lines in image 2. We maximize body of work has been devoted to study the apparent con- contour coherence by iteratively building robust correspon- tour, or simply contour. An apparent contour is the projec- dences among apparent contours and minimizing their dis- tion of a contour generator, which is defined as the set of tances. The registration result is shown in Fig. 1(b) with the 1 contour coherence maximized and two wide baseline range (ICP) algorithm [5] and its variants [6, 19] prove to be ef- scans well aligned. The contour coherence is robust in the fective in accurate rigid registration for high-quality 3D in- presence of wide baseline in the sense that only the shape door scene reconstruction [17]. Few efficient attempts have area close to the predicted contour generator is considered been made for solving rigid range scan registration of par- when building correspondences on the contour, thus avoid- tial overlap and poor initialization. Silva et al.[22] solve the ing the search for correspondences over the entire shape. alignment by an expensive search over the entire pose-space The recently released low-cost structured light 3D sen- which is speeded up by the genetic algorithm. Gelfand sors (e.g., Kinect, Primesense) enable handy complete scan- et al.[11] propose a shape descriptor based method and ning of objects in the home environment. While the state- perform the registration by matching descriptors under the of-the-art scanning methods [17, 23] generate excellent 3D RANSAC framework. Similar approaches have been pro- reconstruction results of rigid or articulated objects, they posed [1,2]. These metric-based registration methods re- assume small motion between consecutive views and as a quires extensive computation of similarity values across a result either the user has to move the 3D sensor carefully set of candidate matches and they are only pairwise. Fur- around the object or the subject must turn slowly in a con- thermore, they fail when two range scans overlap on fea- trolled manner. Even with best effort to reduce the drifting tureless region. error, sometimes the gap is still visible when closing the Articulated non-rigid registration. Allen et al.[3] extend loop (Section7). rigid registration to a template-based articulated registration In our work, we explicitly employ contour coherence un- algorithm for aligning several body scans. Their method der a multi-view framework and develop the Multi-view It- utilizes a set of manually selected markers. Pekelney and erative Closest Contour (M-ICC) algorithm which rigidly Gotsman [18] achieve articulated registration by perform- aligns all range scans at the same time. We further extend ing ICP on each segment and their method requires a man- M-ICC to handle small articulations and propose the Multi- ual segmentation of the first frame. Chang and Zwicker [4] view Articulated Iterative Closest Contour (MA-ICC) algo- further remove the assumption of known segmentation by rithm. Using our proposed registration methods, we suc- solving a discrete labeling problem to detect the set of opti- cessfully address the loop-closure problem from as few as mal correspondences and apply graph cuts for optimization. 4 wide baseline views with limited overlap, and reconstruct General non-rigid registration. Li et al.[14] develop a accurate and complete rigid as well as articulated objects, registration framework that simultaneously solves for point hence greatly reducing the data acquisition time and the reg- correspondences, surface deformation, and region of over- istration complexity, while providing accurate results. lap within a single global optimization. More recently, sev- To the best of our knowledge, we are first to introduce eral methods based on the embedded deformation model contour coherence for multi-view wide baseline range scan have been proposed for modeling of non-rigid objects, ei- registration. Our main contributions include: 1) The con- ther by initializing a complete deformation model with the cept of contour coherence for robust wide baseline range first frame [23] or incrementally updating it [27]. scan registration; 2) Contour coherence based multi-view Apparent contour or silhouette for pose estimation. Ap- rigid registration algorithm M-ICC which allows 3D mod- parent contour has been used for camera motion estimation eling from as few as 4 views; 3) Extension to multi-view [8] relying on the notion of epipolar tangency points. In par- articulated registration algorithm MA-ICC and its applica- ticular, the work of [26] uses only the two outermost epipo- tion to 3D body modeling. lar tangents for the camera pose estimation. More recently, Section2 presents the relevant literature. Section3 de- Hernandez et al.[12] propose the notion of silhouette co- scribes how to extract robust correspondences from the ob- herence, which measures the consistency between observed served and predicted contours. Section4 employs the con- silhouettes and predicted silhouettes, and maximize it to es- tour coherence in a multi-view rigid registration framework, timate the camera poses. Cheung et al.[7] calculate camera which is further extended to consider articulation in Sec- poses by locating colored surface points along the bounding tion5. Section6 briefly covers implementation. Section7 edges. presents the experimental evaluation results while section8 ends with a conclusion and future work. 3. Contour Coherence 2. Related Work We perform wide baseline range scan registration by maximizing contour coherence, i.e., the agreement between We roughly classify all relevant work into 4 categories: the observed and predicted apparent contours. From an im- rigid registration, articulated non-rigid registration (assum- plementation point of view, M-ICC and MA-ICC alternate ing an articulated structure), general non-rigid registration, between finding closest contour correspondences and mini- and apparent contour or silhouette for pose estimation. mizing their distances. However an intuitive closest match- Rigid registration. The classic Iterative Closest Point ing scheme on all contour points fails, mainly due to the Robust Closest Contour (RCC) frame 2 belonging to the back part of the Armadillo are not Ri Extracting Pruning Bijective closest visible in view 1 (Fig. 3(b)). Hence we prune Ci based on contour points contour points matching in 3D Mi, j i R ji the visibility of the corresponding contour generator in view j, Figure 2. General pipeline of our Robust Closest Contour (RCC) method p T Ci=j = fu 2 CijNi(u) · (oj!i − Vi(u)) > 0g; (3) presence of self-occlusion, 2D ambiguity, and outliers as where oj!i is the camera location of frame j in camera i. described later. Hence we propose the Robust Closest Con- An example of pruned contour points is shown in Fig. 3(c). Bijective closest matching in 3D.

Load more