Edge and Corner Detection for Unorganized 3D Point Clouds with Application to Robotic Welding Syeda Mariam Ahmed1, Yan Zhi Tan2, Chee Meng Chew1, Abdullah Al Mamun2, Fook Seng Wong 3 Abstract—In this paper, we propose novel edge and corner detection algorithms for unorganized point clouds. Our edge detection method evaluates symmetry in a local neighborhood Detected Detected and uses an adaptive density based threshold to differentiate 3D Corners Edges edge points. We extend this algorithm to propose a novel corner detector that clusters curvature vectors and uses their geometrical statistics to classify a point as corner. We perform rigorous evaluation of the algorithms on RGB-D semantic segmentation and 3D washer models from the ShapeNet dataset and report higher precision and recall scores. Finally, we also demonstrate how our edge and corner detectors can be used as a novel approach towards automatic weld seam detection for Fig. 1. Demonstration of our edge and corner detectors on 3D pointcloud of robotic welding. We propose to generate weld seams directly panel workpiece. from a point cloud as opposed to using 3D models for offline planning of welding paths. For this application, we show a for real-time applications. However, their fast speed is a result comparison between Harris 3D and our proposed approach on of the nearest neighbor search in a regular 3D grid and thus a panel workpiece. their performance is restricted to organized point clouds. I. INTRODUCTION Similarly corner detection methods in 3D are often an extension of 2D techniques [9] and require an organized Visual features like edge, corner and texture are widely structure or a depth image for detection [12]. Additionally, used in computer vision and robotic perception. They can be various keypoint detection algorithms exist in 3D, detected through low-level cues and then subsequently used however,they aim to find stable and repeatable points that are for higher level perception tasks. A rapidly growing not directly on the edge [10, 11]. Thus it is not possible to use application is robotic welding which depends on extraction of such methods for accurate corner detection. corners and edges to generate welding paths for the robot. An This paper has two main contributions: the first is a new example of a welding workpiece, known as panel workpiece, is shown in Fig. 1. Existing approaches for such welding tasks edge detection algorithm; second is a novel corner detector. are based on offline simulation and planning using 3D CAD The edge detection method evaluates the level of symmetry models. With the growing demand for automation, alternative in a local region of a 3D point. A main advantage of this solutions are required that are more open to adaptation to approach is that it is independent of surface normals due to previously unseen and complicated welding tasks. which the detected edges show a higher accuracy. We extend Though a 3D point cloud is more informative as compared the edge detection algorithm to differentiate between line- to a 2D image, extracting edge and corner features in 3D is type or sharp features and propose a novel corner detector. much more challenging. In case of edge detection for Our approach is most similar to the Gauss map clustering unorganized 3D point clouds, conventional methods perform method, proposed by Weber [3]. They observe that sharp surface reconstruction to form a mesh [1, 8, 16, 17] or build a features separate two or more surfaces and in presence of a graph [3] to analyze the neighborhood of each point through discontinuity, the points of a sphere will build more than one principal component analysis (PCA). However, distinct cluster. Using a similar intuition, our corner detection reconstruction tends to wash out sharp edges and intricate approach is also based on clustering of curvature vectors. features while graph based methods are computationally However the key differences are: 1) we only evaluate edge expensive [7]. The most recent approach, applicable to points in the point cloud, thus reducing the number of points unorganized point clouds, is based on discriminative learning to search for corners. 2) We compare features extracted from [2]. The authors train a random forest based binary classifier the size and average curvature vector associated with each on a set of features that learn to classify points into edge cluster to detect a corner, as opposed to evaluating the number versus non-edge points. One drawback of their method is poor of clusters formed. As a result, our proposed method can performance on unseen data. detect accurate corners. Real-time state-of-the-art methods for 3D edge detection II. 3D EDGE DETECTION are much more efficient for organized point clouds [5, 6]. This is due to the fast nearest neighbor search that can be The proposed edge detection method is based on the mean performed in an organized structure and makes them suitable shift algorithm [18] which is a non-parametric iterative Syeda Mariam Ahmed1 and Chee Meng Chew1 are with the Department of Department of Electrical Engineering, National University of Singapore. Mechanical Engineering, National University of Singapore (email: Fook Seng Wong3 is with Keppel Technology and Innovation, Singapore (e- [email protected] ). Yan Zhi Tan2 and Abdullah Al Mamun2 are with the mail: [email protected]). 0.035 technique to locate the maximum density/modes of a function 0.184 0.182 0.03 in feature space. Our algorithm for edge detection in 3D point 0.18 0.178 0.025 cloud uses the same intuition, but instead of finding the local 0.176 0.174 0.02 maxima, we evaluate the degree of shift in centroid from its 0.172 0.17 initial position to classify points as edge or non-edge. 0.168 0.015 0.005 0.01 0.015 0.02 0.025 0.03 More formally, we define the approach as follows. Given a 0.01 0.115 0.11 0.105 0.1 0.095 0.09 query point 푝푖, we determine its k-nearest neighbors. For an unorganized point cloud, this is achieved through a k- dimensional (K-d) tree [13]. For an organized point cloud, the nearest neighbors are all points in a square of size 푠, while 푝푖 is the centroid. These neighboring points of 푝푖 are given as Ѵ푖 = {푛1, 푛2, … . 푛푘 }. Initially we assume that the centroid of Ѵ푖 is the query point itself, while a new centroid 퐶푖 is Fig. 2. Demonstration of shift in centroid for edge and non-edge points for a computed by taking the mean of the neighboring points as partial view of the Stanford bunny model. The query point 푝푖 is shown in red, follows: the neighboring points Ѵ푖 are shown in green and the new centroid 퐶푖 is shown in blue. 1 푘 퐶푖 = ∑푗=1 푛푗 (1) |Ѵ푖| points are classified as edge 퐸, and non edge points 풩 such To cater variation in density, we compute the resolution that 풫 = (퐸, 풩). For all points in 퐸, we compute their 푍푖(Ѵ푖), as defined by (2), of the neighboring points. This is curvature defined by the Eigenvector 푒 of the smallest achieved by determining the distance of the nearest neighbor Eigenvalue, 휆1, determined from the covariance tensor 훴퐸 of of 푝푖 among all the k neighbors. the neighboring points in 퐸. The covariance tensor used is a weighted linear combination of neighboring points proposed 푍푖(Ѵ푖) = min‖푝푖 − 푛푖‖ (2) by [19]. The covariance tensor is defined as follows: 푛∈Ѵ푖 1 푛 푇 ‖퐶푖 − 푝푖‖ > 휆 ∙ 푍푖(Ѵ푖) (3) 훴퐸 = ∑푖=1(푅 − 푑푖)(푝푖 − 푝̅)(푝푖 − 푝̅) ∀ 푝 ϵ 퐸 (4) ∑(푅−푑푖) Evaluation of 푍 ensures scale invariance as the local density 푖 where 푅 is a predefined radius, 푑 = ‖푝 − 푝‖ and 푝̅ is the of points is considered for each point individually. Finally, 푍 푖 푖 2 푖 centroid of the neighboring points 푝 ϵ 퐸. The advantage of is multiplied by a fixed parameter 휆 which serves as the this tensor is that it assigns smaller weights to distant points classification threshold. If the distance between the new so as to increase repeatability in presence of clutter. centroid 퐶 and the query point 푝 is greater than this threshold 푖 푖 Next, for each point 푝 휖 퐸, we find its k nearest neighbors times 푍 (Ѵ ) as defined by (3), the point is classified as an 푖 푖 푖 from 퐸 defined as follows: edge. Fig. 2 illustrates the concept behind the proposed approach 풦 = {푢1, 푢2, … . 푢푘 } 푢 ϵ 퐸, |풦| = 푘 (5) on a partial view of the Stanford bunny model. It can be seen that the neighboring points Ѵ (shown in green) of the query Since corners in 3D may exist at the intersection of numerous 푖 edges, we assume to be dealing only with objects where the point 푝푖 (shown in red), do not have a symmetrical circular shape, but rather ends abruptly at the edge. This results in a corners lie at intersection of either two or three edges. Thus it is important to distinguish between the type of corner, so as to larger shift in the position of the centroid 퐶푖 (shown in blue). As opposed to this, a point on a smoother surface will have a define the required number of clusters. This check is shift in centroid position that is comparatively small. This performed by evaluating the maximum variation in Euclidean fundamental idea of symmetry among neighboring points can distance within each of the x, y and z directions as follows: . enable quick classification of 3D data.