Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17) Non-Rigid Point Set Registration with Robust Transformation Estimation under Manifold Regularization Jiayi Ma,1 Ji Zhao,1 Junjun Jiang,2 Huabing Zhou3 1Electronic Information School, Wuhan University, Wuhan 430072, China 2School of Computer Science, China University of Geosciences, Wuhan 430074, China 3Hubei Provincial Key Laboratory of Intelligent Robot, Wuhan Institute of Technology, Wuhan 430073, China {jyma2010, zhaoji84, zhouhuabing}@gmail.com, [email protected] Abstract robust transformation estimation from putative correspon- dences. First, the putative correspondences are usually es- In this paper, we propose a robust transformation estimation tablished based on only local feature descriptors, where the method based on manifold regularization for non-rigid point set registration. The method iteratively recovers the point cor- unavoidable noise, repeated structures and occlusions often respondence and estimates the spatial transformation between lead to a high number of false correspondences. Therefore, two point sets. The correspondence is established based on a robust procedure of outlier removal is required. Second, existing local feature descriptors which typically results in a to establish reliable correspondence, the putative set usually number of outliers. To achieve an accurate estimate of the removes a large part of the original point sets whose feature transformation from such putative point correspondence, we descriptors are not similar enough. However, the point sets formulate the registration problem by a mixture model with are typically extracted from the contour or surface of a spe- a set of latent variables introduced to identify outliers, and cific object, and hence can provide intrinsic structure infor- a prior involving manifold regularization is imposed on the mation of the input data which is beneficial to the transfor- transformation to capture the underlying intrinsic geometry mation estimation. Therefore, it is desirable to incorporate of the input data. The non-rigid transformation is specified in a reproducing kernel Hilbert space and a sparse approxima- the whole point sets into the objective function formulation tion is adopted to achieve a fast implementation. Extensive during transformation estimation. Third, for large scale point experiments on both 2D and 3D data demonstrate that our cloud data, the number of points can reach tens of thousands. method can yield superior results compared to other state-of- This poses a significant burden on typical point registration the-arts, especially in case of badly degraded data. methods, particularly in the non-rigid case. Therefore, it is of particular advantage to develop a more efficient technique. Introduction To address these issues, we formulate the registration problem by a mixture model with a set of latent variables Point set registration is a fundamental problem and fre- introduced to identify outliers. We also assume a prior on quently encountered in computer vision, pattern recognition, the geometry involving manifold regularization to impose a medical imaging and remote sensing (Brown 1992). Many non-parametric smoothness constraint on the spatial trans- tasks in these fields including 3D reconstruction, shape formation (Ma et al. 2014; Belkin, Niyogi, and Sindhwani recognition, panoramic stitching, feature-based image reg- 2006). The manifold regularization defined on the whole istration and content-based image retrieval can be solved by input point sets controls the complexity of the transforma- algorithms operating on the point sets (e.g., salient point fea- tion and is able to capture the underlying intrinsic geome- tures) extracted from the input data (Ma et al. 2015a; 2015c; try of the input data. This leads to a maximum a posteriori Bai et al. 2017; Zhou et al. 2016). The goal of point set (MAP) estimation problem which can be solved by using registration is then to determine the right correspondence the Expectation-Maximization (EM) algorithm (Dempster, and/or to recover the spatial transformation between the two Laird, and Rubin 1977) to estimate the variance of the prior, point sets (Jian and Vemuri 2011). In this paper, we focus on while simultaneously estimating the outliers, with the vari- non-rigid registration where the transformation is character- ance given a large initial value. Moreover, a sparse approx- ized by a nonlinear or non-parameterized model (Zhao et al. imation based on a similar idea as the subset of regressors 2011; Ma et al. 2013; Wang et al. 2015). method (Poggio and Girosi 1990) is introduced to improve The registration problem is typically solved by using the computational efficiency. an iterative framework, where a set of putative correspon- Our contribution in this paper includes the following three dences is established and used to refine the estimate of aspects. Firstly, we introduce the manifold regularization to transformation, and vice versa (Besl and McKay 1992; the point set registration problem, which can capture the in- Chui and Rangarajan 2003). In this process, the most chal- trinsic geometry of the input point sets and hence helps to lenging and critical task is to develop an efficient strategy for estimate the transformation. Secondly, we propose a new Copyright c 2017, Association for the Advancement of Artificial formulation for robust transformation estimation based on Intelligence (www.aaai.org). All rights reserved. manifold regularization, which could estimate transforma- 4218 tion from point correspondences contaminated by outliers. geometric structures (e.g., neighborhood structures) which Thirdly, we provide a fast implementation for our method could be incorporated into a feature descriptor. Therefore, by using sparse approximation, which enables our method the correspondences could be established by finding for to handle large scale datasets such as 3D point clouds. each point in one point set (e.g., the model) the point on the other point set (e.g., the target) that has the most sim- Related work ilar feature descriptor. Fortunately, there are several well- The iterated closest point (ICP) algorithm (Besl and McKay designed feature descriptors that can fulfill this task, both 1992) is one of the representative approaches using the iter- in 2D and in 3D cases (Belongie, Malik, and Puzicha 2002; ative framework to solve the registration problem. In (Chui Rusu, Blodow, and Beetz 2009). and Rangarajan 2003), Chui and Rangarajan developed a For 2D case, the shape context (Belongie, Malik, and general framework for non-rigid registration called TPS- Puzicha 2002) has been a widely used feature descriptor. x y RPM. Different from ICP, which uses the nearest point strat- Consider two points i and j, their SCs which capture egy in learning the correspondence, TPS-RPM introduces the distributions of their neighborhood points are histograms {p k }K {q k }K χ2 soft assignments and solves it in a continuous optimization i( ) k=1 and j( ) k=1, respectively. The distance C x , y framework involving deterministic annealing. Zheng and is used to measure their difference ( i j): K 2 Doermann (Zheng and Doermann 2006) proposed a method 1 [pi(k) − qj(k)] called RPM-LNS which can preserve local neighborhood C(xi, yj)= . (1) 2 pi(k)+qj(k) structures during matching, where the shape context (SC) k=1 feature (Belongie, Malik, and Puzicha 2002) is used to ini- After we have obtained the distances of all point pairs, i.e., tialize the correspondence. Ma et al. (Ma et al. 2015b) in- {C(xi, yj),i=1, ··· ,M, j =1, ··· ,N}, the Hungarian troduced a non-rigid registration strategy based on Gaussian method (Papadimitriou and Steiglitz 1982) is applied to seek M N fields, which was later improved in (Wang et al. 2016) by the correspondences between {xi}i=1 and {yj}j=1. using inner distance shape context (Ling and Jacobs 2007) For 3D case, we consider the fast point feature histograms to construct initial correspondences. In the recent past, the (FPFH) (Rusu, Blodow, and Beetz 2009) as the feature de- point registration is typically solved by probabilistic meth- scriptor. It is a histogram that collects the pairwise pan, tilt ods (Jian and Vemuri 2011; Myronenko and Song 2010; and yaw angles between every point and its k-nearest neigh- Horaud et al. 2011; Ma, Zhao, and Yuille 2016). Specifi- bors, followed by a reweighting of the resultant histogram of cally, to cope with highly articulated deformation, A global- a point with the neighboring histograms. The computation of local topology preservation (GLTP) method (Ge, Fan, and the histogram is quite efficient which has linear complexity Ding 2014; Ge and Fan 2015) is proposed based on coherent with respect to the number of surface normals. The match- point drift (CPD) (Myronenko and Song 2010). These meth- ing of FPFH descriptors is performed by a sample consensus ods formulated registration as the estimation of a mixture of initial alignment method. densities using GMMs, and the problem is solved using the After using some local feature descriptor to establish cor- L framework of maximum likelihood and the EM algorithm. respondence, we obtain a putative set S = {(xi, yi)}i=1, where L ≤ min{M,N} is the number of correspondence. L Method Without loss of generality, we assume that {xi}i=1 and L M {yi} in the putative set correspond to the first L points in Suppose we are given a model point set {xi}i=1 and a target i=1 N {x }M L {yj} xi yj D the original model point set i i=1 and the