https://doi.org/10.20965/ijat.2021.p0274 Umehara, Y. et al. Paper: Research on Identification of Road Features from Point Cloud Data Using Deep Learning Yoshimasa Umehara ∗1,†, Yoshinori Tsukada∗2, Kenji Nakamura∗3, Shigenori Tanaka∗4, and Koki Nakahata∗5 ∗1Organization for Research and Development of Innovative Science and Technology, Kansai University 3-3-35 Yamate-cho, Suita-shi, Osaka 564-0073, Japan †Corresponding author, E-mail: [email protected] ∗2Faculty of Business Administration, Setsunan University, Neyagawa, Japan ∗3Faculty of Information Technology and Social Sciences, Osaka University of Economics, Osaka, Japan ∗4Faculty of Informatics, Kansai University, Takatsuki, Japan ∗5Graduate School of Informatics, Kansai University, Takatsuki, Japan [Received October 30, 2020; accepted February 10, 2021] Laser measurement technology has progressed signifi- raised concerns [6], where the use of point cloud data can cantly in recent years, and diverse methods have been be expected to achieve automation and higher efficiency. developed to measure three-dimensional (3D) objects Previous studies in this area include those proposing the within environmental spaces in the form of point cloud generation of dynamic maps from point cloud data [7], data. Although such point cloud data are expected efficient 3D data construction of deteriorated bridges [8], to be used in a variety of applications, such data do and the inspection of incidental road structures [9]. Fur- not possess information on the specific features repre- thermore, the Japanese Ministry of Land, Infrastructure, sented by the points, making it necessary to manually Transport and Tourism (MLIT) has set up a committee to select the target features. Therefore, the identification promote the introduction of new technologies and systems of road features is essential for the efficient manage- for infrastructure maintenance [10], under which efforts ment of point cloud data. As a technology for identify- are being made to research and develop technologies to ing features from the point cloud data of road spaces, efficiently administer facilities using point cloud data. In in this research, we propose a method for automati- addition to such efforts to develop various technologies, cally dividing point cloud data into units of features there have been undertakings to make point cloud data and identifying features from projected images with available to the general public, including the Shizuoka added depth information. We experimentally verified Point Cloud DB (PCDB) [11] by the Shizuoka prefectural that the proposed method accurately identifies and ex- government, “My City Construction” by the Association tracts such features. for Promotion of Infrastructure Geospatial Information Distribution [12], and the MLIT Data Platform 1.0 [13]. As can be seen from the active undertakings to make Keywords: i-Construction, road feature, point cloud use of point cloud data, as well as MLIT’s agenda of pro- data, deep learning, feature identification moting i-Construction [14], which aims to improve con- struction productivity by making use of 3D information, opportunities for employing point cloud data in road space 1. Introduction management are steadily rising. However, point cloud data are merely the sets of a vast number of points to Laser measurement technology has progressed drasti- which XYZ coordinates are attached that represent spatial cally in recent years, and diverse methods have been de- positions or laser reflection intensities, and do not carry veloped to measure three-dimensional (3D) objects that information about the specific features indicated by the exist within road spaces in the form of point cloud data. In points or their relations with other points, which makes it addition, 3D measurement devices that apply point cloud difficult to use them wisely in a manner suitable for the data acquired in many locations in Japan include mobile intended application. For example, in a previous research mapping systems [1], airborne lasers [2], terrestrial laser on the use of point cloud data for inspection of road inci- scanners [3], unmanned aerial vehicle lasers [4], and mo- dental structures [9], it was necessary to manually select bile lasers [5]. Because such point cloud data are an ef- the point cloud data of road incidental structures such as fective means to capture the current detailed configuration signposts from among the vast point cloud data. As this of urban spaces, their use is expected in a variety of ap- case shows, the identification of road features is an essen- plications. In particular, the declining work force in the tial item in the efficient treatment of point cloud data. The field of maintaining and administrating road spaces has identification of road features makes it possible to manage 274 Int. J. of Automation Technology Vol.15 No.3, 2021 © Fuji Technology Press Ltd. Creative Commons CC BY-ND: This is an Open Access article distributed under the terms of the Creative Commons Attribution-NoDerivatives 4.0 International License (http://creativecommons.org/licenses/by-nd/4.0/). Research on Identification of Road Features from Point Cloud Data Using Deep Learning point cloud data according to road feature units in a time based on deep learning, which will enable feature iden- series, which allows various advanced usages such as the tification in any kind of road space, in addition to those detection of differences or changes. road sections for which drawings are available. This can The authors have been involved in research to identify be expected to promote the wider use of point cloud data road features from point cloud data, where they have de- and encourage advanced applications. veloped technologies to identify physical features present in road spaces [15–20] as well as conceptual features without concrete physical presence such as the road cen- 2. Proposal to Resolve Issues terline [21, 22]. The present research, which deals with real objects such as traffic signs, falls under the former 2.1. Issues category. As stated above, our objective is to resolve the issues As a method to identify physically present road fea- remaining in [19], namely, the “need to manually divide tures, we proposed a method based on the completed plan the features” and “the loss of depth information owing to drawings produced at the time of road construction [15]. an image projection.” Hatchings that represent road features on completed plans With respect to the former issue, we remove those fea- were used to extract road features with high accuracy. tures that come into contact with the target features, such Some unresolved issues are the failure to identify grade as the ground, utility lines, and buildings, based on their separations of road surface features such as driveways and geometric characteristics, and then apply 3D labeling to sidewalks or steep gradients, as well as a misalignment the point cloud data based on Euclidean distances for di- between the completed plan and point cloud data. To ad- vision into units of features (i.e., objects). dress these issues, we propose a method to improve the The latter issue arose because we applied a method that extraction accuracy of road surface features by estimating is intended for use with 2D images [24] to 3D point cloud their elevations from the completed plans [16], and an- data. Although a method [25] was devised to treat 3D other method to carry out high-precision registration be- point cloud data using deep learning for feature identifi- tween the completed plan and point cloud data [17]. As cation, it does not allow learning to take place within a a result, we were able to establish a method to identify, realistic time when the point cloud data of road spaces using completed plans, road features in diverse sites that include a variety of features. Therefore, in this research, also include grade separations and inclines. when the point cloud data are projected onto an image, in- Although it is possible to extract road features with formation on its 3D configuration is reflected in the RGB high accuracy using the methods employing completed values of the generated image, which allows learning and plans [15–17], they can only be used for sections of identification without a loss of information. national roads constructed since 2006, when a govern- ment manual [23] was put into effect, making it manda- tory to prepare drawings for road works. Therefore, we 2.2.ContentofProposal studied technologies to identify road features from point The processing flow of the proposed method is shown cloud data without resorting to completed plans [18, 19]. in Fig. 1. With the proposed method, the input of the point In [18], we were able to robustly identify features using cloud data is subjected to three functions: removing the a random forest, but found it difficult to define the char- connected features and then dividing and identifying them acteristic features when the targeted road space included before being output as the point cloud data of individual features with diverse shapes. Meanwhile, in [19], we were features. able to achieve, by employing deep learning, a highly The connected-feature removal function removes the versatile identification without the need to pre-define the features that connect and merge individual features tar- feature characteristics. Focusing on bridges, particularly geted for extraction from the point cloud data to in- among road features, we then studied a method of iden- crease the accuracy of the feature-division function. The tifying bridge parts based on deep learning [20] to assess feature-division function divides the point cloud data into the possibility of the advanced use of a deep learning ap- data representing individual features by labeling based on proach. The results of these experiments confirmed that Euclidean distances. The feature identification function it is possible to identify road features using this method. applies deep learning to the point cloud data, which is di- However, in [19], the authors brought to the surface such vided according to individual features, allowing the fea- issues as the need to manually divide the point cloud data ture types to be identified.