Robust Registration of 2D and 3D Point Sets
Total Page:16
File Type:pdf, Size:1020Kb
Image and Vision Computing 21 (2003) 1145–1153 www.elsevier.com/locate/imavis Robust registration of 2D and 3D point sets Andrew W. Fitzgibbon1 Department of Engineering Science, University of Oxford, 19 Parks Road, Oxford OX1 3PJ, UK Abstract This paper introduces a new method of registering point sets. The registration error is directly minimized using general-purpose non-linear optimization (the Levenberg–Marquardt algorithm). The surprising conclusion of the paper is that this technique is comparable in speed to the special-purpose Iterated Closest Point algorithm, which is most commonly used for this task. Because the routine directly minimizes an energy function, it is easy to extend it to incorporate robust estimation via a Huber kernel, yielding a basin of convergence that is many times wider than existing techniques. Finally, we introduce a data structure for the minimization based on the chamfer distance transform, which yields an algorithm that is both faster and more robust than previously described methods. q 2003 Published by Elsevier B.V. Keywords: Iterated Closest Point; Range image registration; Levenberg–Marquardt 1. Introduction robustness [4,24,28] are often many times slower than the proposal of this paper. A common problem in computer vision is the registration The paper is in several sections. Section 2 defines the of 2D and 3D point sets [1,5,7,8,21,28]. Applications include problem and the notation used in the rest of the paper. the integration of range datasets [14,25], and alignment of InSection3,wereviewexistingworkonpoint-set MRI/CAT scans [9,22]. Typically, a cloud of point samples registration, ICP and otherwise, and also summarize the from the surface of an object is obtained from two or more key ideas in non-linear optimization that will be required. points of view, in different reference frames. The task of Section 4 redefines registration as a non-linear optimization registration is to place the data into a common reference problem. This approach is compared with traditional ICP in frame by estimating the transformations between the 2D and 3D experiments. The second half of the paper, datasets. What makes the problem difficult is that beginning in Section 5, exploits the simplicity of the new correspondences between the point sets are unknown a approach to develop a robust error function, which greatly priori. A popular approach to solving the problem is the class increases the radius of convergence on supplied examples, of algorithms based on the Iterated Closest Point (ICP) again with no significant loss in speed. technique introduced by Besl [1] and Zhang [28]. ICP is In all the above examples, closest point computations are attractive because of its simplicity and its performance. based on explicit Delaunay simplicization of the point set, Although the initial estimate does need to be reasonably allowing Oðlog nÞ closest point computations. In Section 6, good, the algorithm converges relatively quickly. we show how both the new and traditional procedures can This paper abandons one of the basic characteristics of be modified to use a fast lookup based on the distance ICP—its closed-form inner loop—and employs instead a transform. This speeds up both algorithms significantly, but standard iterative non-linear optimizer, the Levenberg– the proposed technique benefits more, resulting in an Marquardt (LM) algorithm [19]. This approach, perhaps algorithm which is faster and more accurate than traditional surprisingly, incurs no significant loss of speed, but allows ICP, with a wider basin of convergence. the extension of ICP to use truly robust statistics, with a More generally, the message of the paper is that concomitant reduction of dependence on the initial estimate. In contrast, existing ways of introducing specialized ‘home-grown’ strategies for function minimization (of which ICP is one example) do not E-mail address: [email protected] (A.W. Fitzgibbon). necessarily outperform more general—but more sophisti- 1 http://www.robots.ox.ac.uk/~awf cated—non-linear optimization algorithms. 0262-8856/$ - see front matter q 2003 Published by Elsevier B.V. doi:10.1016/j.imavis.2003.09.004 1146 A.W. Fitzgibbon / Image and Vision Computing 21 (2003) 1145–1153 2. Problem statement and definitions The paper deals exclusively with the two-frame case, although multiple-frame approaches [8,21] should immedi- ately benefit. We are given two sets of points in Rn; which for convenience we shall denote by model and data, with their Nm Nd : elements being denoted by {mi}i¼1 and {di}i¼1 The task of registration is to determine the parameters of a transform- ation T which, when applied to the data points, best aligns model and data. The parameters of T are represented by a p-vector a: Common transformations and corresponding values of p are listed in Table 1. For 2D registration (see Fig. 1 for an example problem and Fig. 2 for an example of LM-ICP output) under Euclidean transformations, the number of parameters p ¼ 3 comprising rotation angle u and translation vector ðtx; tyÞ: Collecting the parameters into a parameter vector a ¼½u; tx; ty; we define ! ! cos u sin u t ; ; x T2Dða;xÞ¼Tðu tx ty;xÞ¼ x þ Fig. 1. 2D curves to be registered. The curves are subsets of the output of an 2sin u cos u ty ð1Þ edge detector applied to the image. Synthetic rotations and translations of the lower-right ‘C’ are registered to the upper left one. for x [ R2 Alignment is measured by an error function e2ðlxlÞ; and a and data, yielding the error function typical choice is to define XNd 2 l l 2 2 EðaÞ¼ wi mine ð mj 2 Tða;diÞ Þð3Þ ðl lÞ¼k k j e x x i¼1 In order to measure alignment, we require that and finally the estimate of the optimal registration is given correspondence between the model and data points is by minimizing over a: specified. This correspondence is denoted by the function XNd fðiÞ; which selects, for each data point, the corresponding 2 a^ ¼ argmin wi mine ðlmj 2 Tða;diÞlÞ model point. In order to cope with data points for which no j a i¼1 correspondent is found, we also introduce weights wi; which are set to zero for points with no match, and one 2.1. The ICP algorithm otherwise. Thus, the error to be minimized is In its simplest form, the ICP algorithm iterates two steps. XNd Beginning with an initial estimate of the registration Eða;fÞ¼ w e2ðlm 2 Tða;d lÞð2Þ i fðiÞ i parameters, a ; the algorithm forms a sequence of estimates i¼1 0 In general, the function f is considered part of the minimization process: in ICP-like algorithms fðiÞ is chosen as the point which minimizes the distance between model Table 1 Transformations commonly occurring in registration problems nD transformation p Parameters 2D Euclidean 3 Rotation u; 2D translation ðtx; tyÞ 2D Affine 6 2 £ 3 matrix 2D Projective 8 3 £ 3 homography matrix (defined up to scale) 3D Euclidean 6 3 rotation, 3 translation 3D Similarity 7 3 rotation, 3 translation, 1 scale 3D Affine 12 3 £ 4 matrix 2 3D Projective 15 4 £ 4 matrix (defined up to scale) Fig. 2. Convergence of LM-ICP with Lorentzian kernel eðrÞ¼logð1 þ r Þ from a 408 rotation. A.W. Fitzgibbon / Image and Vision Computing 21 (2003) 1145–1153 1147 ak, which progressively reduce the error EðaÞ: Each iteration If the techniques of that section are impossible to implement of the algorithm comprises the following two steps, labelled in a given application, there are better ways to use function C and T: evaluations than to compute finite-difference derivatives as C: Compute correspondences, f : Set suggested above. See Brent. [3] for a detailed discussion of derivative-free minimization strategies. One further note: 2 l l ; ; fðiÞ¼ argmin e ð mj 2 Tðak; diÞ Þ i ¼ 1 … Nd if derivatives are computed via finite differences, j[{1;…;N } m calculations of the form ðEða þ dÞ 2 EðaÞÞ=kdk are required. It is important that the closest point computations are so that mfðiÞ is the closest model point to the datum di repeated when making these calculations. It might be transformed by the current estimate ak: T: Update transformation, a : Set thought more efficient to compute closest points once, when calculating EðaÞ; and to use the same values of f to compute XNd Eða þ dÞ: However, this will compute the derivatives with ¼ 2ðl ð ÞlÞ akþ1 argmin e mfðiÞ 2 T a; di fixed correspondences, and the gradient descent direction a i¼1 will then be towards the same optimum as would be In many common cases, step T can be performed in closed attained by the basic ICP algorithm. It is better, particularly form, or via well understood, numerically stable, procedures with a robust kernel, to allow the correspondences to change such as singular value decomposition. for each d: It is easy to see that both steps must reduce the error, and that the error is bounded below. Thus convergence to a local 2.2.2. The LM algorithm in detail minimum is guaranteed. Furthermore, it is straightforward There now follows a derivation of the LM algorithm. to discern a termination criterion: when the set of Readers who are familiar with its operation may wish to correspondences does not change in step C, the value of skim this description in order to synchronize notation, akþ1 will be set equal to ak in the T step, so no further and proceed to Section 3. The error function EðaÞ can be change is possible. written as the sum of Nd residuals as follows XNd 2.2.