Non-rigid Articulated Point Set Registration with Local Structure Preservation Song Ge and Guoliang Fan∗ School of Electrical and Computer Engineering Oklahoma State University, Stillwater, OK 74078 {song.ge,guoliang.fan}@okstate.edu Abstract ment between two distributions [8, 9]. Coherent Point Drift (CPD) [16, 15] is an effective GMM-based non-rigid reg- We propose a new Gaussian mixture model (GMM)- istration algorithm that imposes a coherence constraint to based probabilistic point set registration method, called Lo- preserve the topology of the point sets. cal Structure Preservation (LSP), which is aimed at com- On the other hand, articulated point set registration is be- plex non-rigid and articulated deformations. LSP integrates coming an attractive topic, e.g, human pose estimation [23] two complementary shape descriptors to preserve the lo- or body shape modeling [21]. Some initial conditions or cal structure. The first one is the Local Linear Embedding new constraints are needed to cope with the complicated (LLE)-based topology constraint to retain the local neigh- non-rigid articulated deformation. In [23], a pose initial- borhood relationship, and the other is the Laplacian Coor- ization from a large training dataset is used to reduce the dinate (LC)-based energy to encode the local neighborhood articulated deformation between two point sets to be regis- scale. The registration is formulated as a density estima- tered. In [13], Laplacian eigenfunctions are used to align tion problem where the LLE and LC terms are embedded in two voxels represented by spectral graphs for registration the GMM-based Coherent Point Drift (CPD) framework. A initialization. By incorporating the global (CPD) and lo- closed form solution is solved by an Expectation Maximiza- cal topological constraints (LLE) into a GMM-based frame- tion (EM) algorithm where the two local terms are jointly work, the Global-Local Topology Preservation (GLTP) al- optimized along with the CPD coherence constraint. The gorithm was proposed in [6] which obtains promising per- experimental results on a challenging 3D human dataset formance. Some approaches treat the articulated deforma- show the accuracy and efficiency of our proposed approach tion as a chain of constrained rigid motions, where pre- to handle non-rigid highly articulated deformations. segmentation and locally rigid assumption are usually re- quired. An articulated ICP (AICP) algorithm was proposed in [18] which adopts a divide-and-conquer strategy to iter- 1. Introduction atively estimate the articulated structure by assuming it is Point set registration is a fundamental issue in computer partially rigid. The similar strategy was applied in [7] by vision. The registration techniques usually fall into two using a GMM-based rigid registration algorithm instead of categories: rigid and non-rigid depending on the under- ICP. By using the exponential maps based parametrization, lying transformation model. Iterative Closest Point (ICP) the articulated deformation is effectively embedded in the [2, 26] is a classic rigid registration method which itera- GMM-based framework in [24]. tively assigns correspondence and then finds the least square We propose a new GMM-based point set registration al- transformation by using the estimated correspondence. For gorithm, called Local Structure Preservation (LSP), to deal non-rigid registration, shape features are commonly used with non-rigid highly articulated deformations, such as 3D for correspondence initialization [27, 11, 12] or directly in- human data. Without involving the locally rigid assump- volved in the matching process [22, 20]. Gaussian Mixture tion or requiring any initial conditions, LSP is non-rigid Model (GMM)-based registration algorithms are becoming both globally and locally which is more flexible and gen- an important category, where the point sets are represented eral to handle the non-rigid articulated deformations than by density functions and then registration is cast as either those with the locally rigid assumption (e.g. [18, 7, 24]). a density estimation problem [4, 5, 16, 15] or an align- The key is to involve two local shape descriptors, the Local ∗ Linear Embedding (LE) and Laplacian Coordinate (LC), to This work is supported by the Oklahoma Center for the Advancement of Science and Technology (OCAST) under grant HR12-30 and the Na- preserve the local structure in terms of the neighborhood tional Science Foundation (NSF) under grant NRI-1427345. relationship and the neighborhood scale, respectively. The two local regularization terms are unified with the CPD co- old GMM parameters based on the Bayes rule as herence constraint into the GMM-based density estimation exp(− 1 xn−T (ym,Θ) 2) framework. A joint optimization of three terms is achieved old 2 σold p (m|xn)= D . 2 by an Expectation and Maximization (EM) algorithm for M exp(− 1 xn−T (yi,Θ) 2)+ (2πσ ) 2 ωM i=1 2 σold (1−ω)N Maximum Likelihood (ML) optimization. (3) Then in the M-step the new GMM parameters are updated 2. Proposed Method by minimizing (2). The EM algorithm performs iteratively by alternating between E-step and M-step until it converges. We first review the GMM-based registration frame- The different type of transformations can lead to different work along some existing regularization terms for non-rigid optimization strategies. transformation estimation. Then we will introduce a new local regularization term ELSP . Afterwards, we present 2.2. Non-rigid Transformation Regularization the formulation and optimization of the proposed LSP al- Particularly, Coherent Point Drift (CPD) [15] is a pow- gorithm followed by a discussion of parameter selection. erful and noteworthy GMM-based non-rigid registration method where the underlying non-rigid transformation 2.1. Registration as Mixture Density Estimation T (Y, Θ) is defined as the initial position Y plus a dis- We consider point set registration as a probability den- placement function f(Y). The displacement function f is sity estimation problem, where one point set (Y = modeled in a Reproducing Kernel Hilbert Space (RKHS) T D [y1, ··· , yM ] , ym ∈ R ) is treated as the template with and the spatial smoothness regularization is defined as the a sparse distribution which presents the centroids from the Fourier domain norm of f. It has been proved that the opti- Gaussian mixture model (GMM), while the other point set mal function f which minimizes the objective (2) under the X =[x ··· x ]T x ∈ D ( 1, , N , n R )isservedasthetar- spatial smoothness constraint is given by a linear combina- get with a dense distribution, and M,N,D are the num- tion of Gaussian kernel functions as ber of points and the dimension of each point respectively. Then the goal is to find the optimal GMM parameters (e.g. T (Y, W)=Y + GW, (4) the centroids which are controlled by unknown transforma- tion) to maximize the GMM posterior probability. We de- where GM×M is the Gaussian kernel matrix with element note the spatial transformation as T (Y, Θ), which can be 1 yi−yj 2 gij =exp(− ) and WM×D is the weight ma- considered as a function of Y with parameters Θ. To ac- 2 β X trix of the Gaussian kernel. The corresponding constraint count for outliers in , a uniform component with weight W ω (0 ≤ ω ≤ 1) is added [15, 4]. For simplicity, we consider which regularizes the weight matrix has the form as all Gaussian components are independently distributed with T 2 EMC(W)=Tr(W GW). (5) an equal isotropic variance σ and an equal weight, then the joint GMM probability density function is written as It was shown in [15] that the selection of the Gaussian ker- N N M+1 nel makes the regularization equivalent to the one in the Mo- p(X)= p(xn)= πmp(xn|m), (1) tion Coherence Theory [25] which forces points to move to- n=1 n=1 m=1 gether as a group to keep the motion coherence. The motion coherence is helpful to keep the overall spatial connectivity 2 1 x−T (ym,Θ) of a complicated point set during the registration. However, where p(x|m)= D exp(− 2σ2 ) (m = (2πσ2) 2 it may not handle the non-rigid highly articulated deforma- 1 ··· (x| )= 1 = +1 = 1−ω , ,M), p m N for m M , πm M tion where the motion coherence assumption may be vio- =1 ··· = (m , ,M), and πM+1 ω. Then registration is con- lated. Θ 2 verted to the problem of finding and σ that minimizes Recently some specific regularization terms were pro- the negative log-likelihood of (1). posed and embedded in the GMM-based CPD framework Following the EM algorithm for GMM-based clustering for improving registration performance. In [17], a Graph- [3, 15], we can find the objective (E-step) as Laplacian (GL) based regularization term, which enforces transformation smoothness to avoid mis-matches during N M x −T(y , Θ) 2 Q(Θ,σ2)= pold(m|x ) n m registration, is added to restrict a large displacement be- n 2σ2 n=1 m=1 tween two neighboring points. This regularization term N D enhances the registration robustness under noise, outliers + p ln(σ2), 2 (2) and occlusions. It mainly penalizes large displacements be- tween the adjacent neighbors, but does not consider the spa- N M old old where Np = n=1 m=1 p (m|xn) and p (m|xn) are tial relationship between the edges (constructed by neigh- the posterior probabilities that can be computed using the boring points) which may not be effective to preserve the local shape structure. On the other hand, the LLE-based pressed as topology constraint proposed in [6] is intended to preserve the neighborhood structure by retaining the local neighbor- L(vi)= Aij (vi − vj ), (8) hood relationship. This was found helpful to cope with non- (i,j)∈E rigid articulated deformations.