Density Adaptive Point Set Registration Felix Jaremo¨ Lawin, Martin Danelljan, Fahad Shahbaz Khan, Per-Erik Forssen,´ Michael Felsberg Computer Vision Laboratory, Department of Electrical Engineering, Linkoping¨ University, Sweden {felix.jaremo-lawin, martin.danelljan, fahad.khan, per-erik.forssen, michael.felsberg}@liu.se Abstract Probabilistic methods for point set registration have demonstrated competitive results in recent years. These techniques estimate a probability distribution model of the point clouds. While such a representation has shown promise, it is highly sensitive to variations in the den- sity of 3D points. This fundamental problem is primar- ily caused by changes in the sensor location across point sets. We revisit the foundations of the probabilistic regis- tration paradigm. Contrary to previous works, we model the underlying structure of the scene as a latent probabil- ity distribution, and thereby induce invariance to point set density changes. Both the probabilistic model of the scene and the registration parameters are inferred by minimiz- ing the Kullback-Leibler divergence in an Expectation Max- imization based framework. Our density-adaptive regis- tration successfully handles severe density variations com- monly encountered in terrestrial Lidar applications. We perform extensive experiments on several challenging real- world Lidar datasets. The results demonstrate that our ap- proach outperforms state-of-the-art probabilistic methods for multi-view registration, without the need of re-sampling. Figure 1. Two example Lidar scans (top row), with signifi- cantly varying density of 3D-points. State-of-the-art probabilistic method [6] (middle left) only aligns the regions with high density. This is caused by the emphasis on dense regions, as visualized by 1. Introduction the Gaussian components in the model (black circles in bottom 3D-point set registration is a fundamental problem in left). Our method (right) successfully exploits essential informa- tion available in sparse regions, resulting in accurate registration. computer vision, with applications in 3D mapping and scene understanding. Generally, the point sets are acquired using a 3D sensor, e.g. a Lidar or an RGBD camera. The parameters, and a Gaussian mixture model (GMM) of the task is then to align point sets acquired at different po- point distribution. Our formulation instead minimizes the sitions, by estimating their relative transformations. Re- Kullback-Leibler divergence between the mixture model cently, probabilistic registration methods have shown com- and a latent scene distribution. petitive performance in different scenarios, including pair- Common acquisition sensors, including Lidar and wise [19, 14, 15] and multi-view registration [10, 6]. RGBD cameras, do not sample all surfaces in the scene In this work, we revisit the foundations of the probabilis- with a uniform density (figure 1, top row). The density of tic registration paradigm, leading to a reformulation of the 3D-point observations is highly dependent on (1) the dis- Expectation Maximization (EM) based approaches [10, 6]. tance to the sensor, (2) the direction of the surface relative In these approaches, a Maximum Likelihood (ML) formu- to the sensor, and (3) inherent surface properties, such as lation is used to simultaneously infer the transformation specularity. Despite recent advances, state-of-the art prob- 3829 abilistic methods [19, 10, 6, 15, 14] struggle under varying tration. Another line of research is to use feature descriptors sampling densities, in particular when the translational part to find point correspondences in a robust estimation frame- of the transformation is significant. The density variation is work, such as RANSAC [20]. Zhou et al.[27] also use problematic for standard ML-based approaches since each feature correspondences, but minimize a Geman-McClure 3D-point corresponds to an observation with equal weight. robust loss. A drawback of such global methods is the re- Thus, the registration focuses on regions with high point liance on accurate geometric feature extraction. densities, while neglecting sparse regions. Probabilistic registration methods model the distribution This negligence is clearly visible in figure 1 (bottom of points as a density function. These methods perform left), where registration has been done using CPPSR [6]. alignment either by employing a correlation based approach Here the vast majority of Gaussian components (black cir- or using an EM based optimization framework. In cor- cles) are located in regions with high point densities. A relation based approaches [25, 15], the point sets are first common consequence of this is inaccurate or failed regis- modeled separately as density functions. The relative trans- trations. Figure 1 (middle right) shows an example regis- formation between the points set is then obtained by mini- tration using our approach. Unlike the existing method [6], mizing a metric or divergence between the densities. These our model exploits information available in both dense and methods lead to nonlinear optimization problems with non- sparse regions of the scene, as shown by the distribution of convex constraints. Unlike correlation based methods, the Gaussian components (figure 1, bottom right). EM based approaches [19, 10] find an ML-estimate of the 1.1. Contributions density model and transformation parameters. Most methods implicitly assume a uniform density of We propose a probabilistic point set registration ap- the point clouds, which is hardly the case in most applica- proach that counters the issues induced by sampling den- tions. The standard approach [22] to alleviate the problems sity variations. Our approach directly models the underly- of varying point density is to re-sample the point clouds in ing structure of the 3D scene using a novel density-adaptive a separate preprocessing step. The aim of this strategy is to formulation. The probabilistic scene model and the trans- achieve an approximately uniform distribution of 3D points formation parameters are jointly inferred by minimizing the in the scene. A common method is to construct a voxel Kullback-Leibler (KL) divergence with respect to the latent grid and taking the mean point in each voxel. Compara- scene distribution. This is enabled by modeling the acqui- ble uniformity is achieved using the Farthest Point Strategy sition process itself, explicitly taking the density variations [8], were points are selected iteratively to maximize the dis- into account. To this end, we investigate two alternative tance to neighbors. Geometrically Stable Sampling (GSS) strategies for estimating the acquisition density: a model- [11] also incorporates surface normals in the sample selec- based and a direct empirical method. Experiments are per- tion process. However, such re-sampling methods have sev- formed on several challenging Lidar datasets, demonstrat- eral shortcomings. First, 3D scene information is discarded ing the effectiveness of our approach in difficult scenarios as observations are grouped together or removed, leading to with drastic variations in the sampling density. sparsification of the point cloud. Second, the sampling rate, e.g. voxel size, needs to be hand picked for each scenario 2. Related work as it depends on the geometry and scale of the point cloud. The problem of 3D-point set registration is extensively Third, a suitable trade-off between uniformity and sparsity pursued in computer vision. Registration methods can be must be found. Thus, such preprocessing steps are compli- coarsely categorized into local and global methods. Local cated and their efficacy is questionable. In this paper, we methods rely on an initial estimate of the relative transfor- instead explicitly model the density variations induced by mation, which is then iteratively refined. The typical ex- the sensor. ample of a local method is the Iterative Closest Point (ICP) There exist probabilistic registration methods that tackle algorithm. In ICP, registration is performed by iteratively the problem of non-uniform sampling density [2, 13]. In alternating between establishing point correspondences and [2], a one class support vector machine is trained for pre- refining the relative transformation. While the standard ICP dicting the underlying density of partly occluded point sets. [1] benefits from a low computational cost, it is limited by The point sets are then registered by minimizing the L2 dis- a narrow region of convergence. Several works [23, 21, 4] tance between the density models. In [16], an extended EM investigate how to improve the robustness of ICP. framework for modeling noisy data points is derived, based Global methods instead aim at finding the global solu- on minimizing the KL divergence. This framework was tion to the registration problem. Many global methods rely later exploited for outlier handling in point set registration on local ICP-based or probabilistic methods and use, e.g., [13]. Unlike these methods, we introduce a latent distribu- multiple restarts [17], graph optimization [24], branch-and- tion of the scene and explicitly model the point sampling bound [3] techniques to search for a globally optimal regis- density using either a sensor model or an empirical method. 3830 3. Method likelihood function, In this work, we revisit probabilistic point cloud registra- M Ni (Θ; ,..., M )= log(pV (φ(xij; ωi) θ)) . (2) tion, with the aim of alleviating the problem of non-uniform L X1 X | i j point density. To show the impact of our model,