Self-Organized Model Fitting Method for Railway Structures Monitoring Using Lidar Point Cloud

remote sensing

Article Self-Organized Model Fitting Method for Railway Structures Monitoring Using LiDAR Point Cloud

Amila Karunathilake * , Ryohei Honma and Yasuhito Niina Advanced Technologies Research Laboratory, Asia Air Survey Co. Ltd, 1-2-2 Manpukuji, Asao-ku, Kawasaki-shi 215-0004, Kanagawa Prefecture, Japan; [email protected] (R.H.); [email protected] (Y.N.) * Correspondence: [email protected]

Received: 12 October 2020; Accepted: 9 November 2020; Published: 11 November 2020

Abstract: Mobile laser scanning (MLS) has been successfully used for infrastructure monitoring apt to its fine accuracy and higher point density, which is favorable for object reconstruction. The massive data size, computational time, wider spatial distribution and feature extraction become a challenging task for 3D point data processing with MLS point cloud receives from terrestrial structures such as buildings, roads and railway tracks. In this paper, we propose a new approach to detect the structures in-line with railway track geometry such as railway crossings, turnouts and quantitatively estimate their dimensions and spatial location by iteratively applying a vertical slice to point cloud data for long distance laser measurement. The rectangular vertical slices were defined and their boundary coordinates were estimated based on a geometrical method. Estimated vertical slice boundaries were iteratively used to evaluate the point density of each vertical slice along with a cross-track direction of the railway line. Those point densities were further analyzed to detect the railway line track objects by their shape and spatial location along with the rail bed. Herein, the survey dataset is used as a dictionary to preidentify the spatial location of the object and then as an accurate estimation for the rail-track, by estimating the gauge corner (GC) from dense point cloud. The proposed method has shown a significant improvement in the rail-track extraction process, which becomes a challenge for existing remote sensing technologies. This adaptive object detection method can be used to identify the railway track structures prior to the railway track extraction, which allows in finding the GC position precisely. Further, it is based on the parallelism of the railway track, which is distinct from conventional railway track extraction methods. Therefore it does not require any inertial measurements along with the MLS survey and can be applied with less background information of the observed MLS point cloud. The proposed algorithm was tested for the MLS data set acquired during the pilot project collaborated with West Japan Railway Company. The results indicate 100% accuracy for railway structure detection and enhance the GC extraction for railway structure monitoring.

Keywords: mobile laser scanning (MLS); railway infrastructure monitoring; model fitting; 3D point cloud

1. Introduction The mobile laser scanning (MLS) point cloud becomes a more accurate and sophisticated technique to acquire terrestrial information from the surrounding environment. Currently, it has become a more popular method to monitor the quality and the safety of the social infrastructure systems such as railway roads [1], highways and public roads [2,3], bridges and other peripherals [4–8]. During the past few years, maintenance of public transportation, especially, railway track monitoring by MLS has been widely used in many institutions due to its advantage of providing precise estimation in railway structures up to the millimeter level accuracy [9,10]. Figure1 depicts the 3D models generated by MLS survey to monitor the railway infrastructure and surrounding environment, which is very important and coequal to the physical inspections for railway maintenance. The structures in (a), (b) and (c)

Remote Sens. 2020, 12, 3702; doi:10.3390/rs12223702 www.mdpi.com/journal/remotesensing Remote Sens. 2019, 11, x FOR PEER REVIEW 2 of 16 Remote Sens. 2020, 12, 3702 2 of 16 environment, which is very important and coequal to the physical inspections for railway maintenance. The structures in (a), (b) and (c) depict the ridged metal structures, non-ridged metal depictstructures the ridgedsuch as metal electric structures, wires and non-ridged non-metal metal equipment structures, which such are as located electric above wires the and human non-metal eye equipment,contacting range. which Some are located of those above locations the human can be eye accessible contacting for range. inspection, Some ofbut those some locations of them can need be accessiblespecially equipped for inspection, machinery but some such of as them cranes need to speciallytake the measurements. equipped machinery Figure such 1d depicts as cranes a railway to take thetrack measurements. passing through Figure the1 dconcrete depicts atunnel railway, which track seems passing monotonous through the for concrete low resolution. tunnel, which However, seems monotonoushigher point fordensity low resolution.of the 3D point However, cloud higher inside point the tunnel density surface of the 3Dpermitted point cloud to capture inside thestructural tunnel surfacedefects of, permitted not only to the capture railway structural infrastructure, defects of,but notalso only the thesurrounding railway infrastructure, environment. butVisualizing also the surroundingsuch information environment. in the GIS Visualizingplatform [11 such,12] increases information the inefficiency the GIS of platform infrastructure [11,12] increasesmanagement the eandﬃciency decision of infrastructure making. Minimum management workforce, and decisionlong observation making. Minimum within a short workforce, period, long day observation and night withintime monitoring a short period, process day and and the night possibility time monitoring of keeping process a track and history the possibility for further of analysis keeping become a track historyfavorable for factors further ofanalysis MLS become based survey favorable for factors railway of maintenanceMLS based survey companies for railway and theirmaintenance regular companiesmaintenance and work. their Therefor regulare maintenancerailway structure work. monitoring Therefore by railway MLS becomes structure the monitoring most reliable by MLS and becomescost effective the most method reliable accepted and costby many eﬀective countries method for accepted both private by many and countriesgovernment for operated both private railway and governmentsystems [13– operated15]. railway systems [13–15].

(a) (b) (c)

(d) (e) (f)

(g) (h) (i)

Figure 1. Point cloud acquired near to the railway:railway: ( a) station platform (b) bridge (c) catenary system ((d)) concreteconcrete tunnel tunnel (e ()e track) track bed bed in ﬂatin flat terrain terrain (f) track (f) track bed in bed slope in terrainslope terrain (g) level (g crossing) level crossing (h) turnout (h) (tiurnout) multipel (i) multipel trackes during trackes the during mobile the laser mobile scanning laser scanning (MLS) survey. (MLS) survey.

However, due due to experimental to experimental defects defects during during the MLS the survey MLS such survey as poor such sampling, as poor misalignment sampling, ofmisalignment the data acquisition of the data system, acquisition perturbations system, in GPS perturbations and limitations in GPS in the and data limitations processing in algorithm, the data theprocessing accurate algorithm, estimation the of 3D accurate terrestrial estimation structures, of 3D their terrestrial dimensions structures, and special their locations dimensions become and morespecial challenging locations become [16,17]. more The inside challenging corners [1 of6,1 the7]. The rail headinside is corners deﬁned of as the the rail railway head gaugeis defined corner as (hereinafter,the railway gauge this is corner abbreviated (hereinafter, to “GC”), this is which abbreviated becomes to an“GC”) important, which position becomes for an pointimportant data alignmentposition for [1 point]. The data precise alignment estimation [1]. The of railway precise infrastructureestimation of railway such as infrastructure the rail track depends such as the on rail the accuratetrack depends estimation on the of accurate the railway estimation GC. The of point the cloudrailway captured GC. The from point the cloud ideal captured railway track from bed the in ideal ﬂat terrainrailway is track depicted bed in in flat Figure terrain1e. In is thisdepicted scenario, in Figure both left1e. andIn this right scenario, rail track both positions left and are right in therailsame track elevation.positions are Therefore in the same the distributionelevation. Therefore of point cloudthe distribution near GC in of both point the cloud left andnear right GC in of both the railway the left trackand right has similaritiesof the railway in horizontal track has andsimilarities zero altitude in horizontal change inand the zero vertical altitude plane. change However, in the Figure vertical1f

Remote Sens. 2020, 12, 3702 3 of 16 depicts the railway track running on a slope surface. In this scenario, the distribution of point cloud near GC in left or right or both railway tracks are varying in horizontal and vertical planes. Moreover, railway crossings, turnouts and combined locations depicted in Figure1g–i make the GC extraction process more complicated than the ideal location in (e). This can be identified as one of the major limitations for accurate estimation of GC. This affects specially for long distance MLS survey in the urban environment, which contains a considerable number of crossings and turnouts due to other transportation infrastructures. Several methods have been proposed to overcome these issues and Arastounua M. [18] presented a diverse approach to detect the railway infrastructure in track bed and above. However, in the case of point extraction within the same height of the neighborhood, GC extraction criteria need to be improved for more accurate and efficient processing. Zhou Y. et al. [19] discusses the course and fine extraction of the railhead. In the course extraction step, the center point coordinate of the railway track was calculated to estimate the rough aliment direction of the two railway tracks in the preliminary stage. In the next step, the fine extraction was done by considering the elevation value of the railhead point extracted from the previous step. However, in the practical field experiments, rough estimation becomes a challenging task if the GPS and LiDAR sensors set independently. Elberink S. O. et al. [20] discusses a generalized approach to determine the railway track and surrounding with a model-based method. It discusses the applicability of the rail track and inside structure detection based on thresholding the points, which are located in a specific distance from the centre line. However, this method has considerable discrepancies while applying to the real experimental datasets of the railway track. In the real LiDAR dataset of the railway track bed, the points incident not only by track objects such as railway turnouts or crossings, but also due to signaling components, bridge structures or vegetation. In those cases, the above methods fail to give accurate results to identify each of the above objects from the railway track. Niina Y. et al. [21] presents a method to accurately extract the railway track by considering the point cloud around the known distance from the GPS trajectory line by applying the iterative closest point (ICP) algorithm for railway track extraction. This can be used with higher efficiency and accuracy while processing large datasets. However, this method needs to be improved especially in the point cloud extraction near composite environments such as railway crossings and turnouts, which needs to have further attention for more precise analysis. The main aim of this study is to enhance the GC extraction for precise estimation of railway tracks. The precisely estimated results of GC become the key component to evaluate the other secondary analysis such as train clearance check, temporal change of the distance between closely located railway tracks and temporal change of the levels between two rails. Thus, the above mentioned inspections have a higher influence on today’s railway industry, and leads to fulfilling the marketing requirements. In this paper, we proposed new criteria to detect the rail crossings and turnouts based on parallelism of the railway track and the spatial distribution of the neighboring MLS point cloud.

2. Materials and Method

2.1. Overview The rail structures in a railway line can be identified by point cloud recorded during the MLS survey, after generating a 3D model as depicted in Figure1. Then the feature extraction algorithm can be applied to extract particular structures such as GC. In order to monitor the GC, we differentiated the railway track into 3 categories according to the formation of the railway GC position compared to its neighborhood. Figure2 depicts the physical formation of 3 di fferent scenarios in GC positions incorporated with the rail track, by 3 different colors. Blue color represents the ideal case of the train movement, while orange and green colors represent near the turn-out and near the crossing respectively. The ideal railway track that runs through the flat surface is depicted in Figure2a where the track separation distance was fixed to a particular type of compartment. In this campaign, we designed the MLS experimental setup for the local railway line with the constant track separation distance. Remote Sens. 2020, 12, 3702 4 of 16

Remote Sens. 2019, 11, x FOR PEER REVIEW 4 of 16 However, it can be adjusted according to the MLS survey setup. Figure2b,c depicts the second scenario,depicts the before second and scenario, after the before right-handed and after turnout. the right Here,-handed the conventional turnout. Here, GC the detection conventional algorithms GC aredetection compromised algorithms [18 are–20 compromised] due to multiple [18– occurrence20] due to multiple of GC in occurrence variable distances of GC in variable in the horizontal distances plane.in the Ithorizontal adversely plane. affects It the adversely rail track affect extractions the railresults track by extractionmaking a sudden results fluctuation by making duea sudden to the wrongfluctuation estimation due toof the the wrong GC position. estimation On of the the other GC hand,position. the MLSOn the survey other has hand, a large the MLS amount survey of point has clouda large data amount in long of distance point cloud observation data in during long distance the field observation experiment. during Therefore, the the field rail experiment. extraction processTherefore, needs the rail to be extraction very effi cientprocess and needs sophisticated to be very notefficient only forand an sophisticated ideal track, not but only also for for an a vividideal environmenttrack, but also [22 for,23 a]. vivid Figure environment2d depicts the [22 third,23]. Figure scenario, 2d whichdepicts explains the third the scenario GC extraction, which nearexplains the railthe crossing.GC extraction There, near the theshape rail of crossing. the rail head There, is visible the shape as half of the of its rail physical head is shape visible and as remaining half of its sidesphysical of theshape rail and head remaining were curved sides by of smooththe rail head passing were of curved road vehicles. by smooth Therefore, passing theof road shape vehicles. of the railTherefore, head near the shape the crossing of the rail is slightlyhead near diff theerent crossing than the is slightly shape ofdifferent the rail than head the in shape ideal trackof the andrail turnout.head in ideal The track point and cloud turnout. received The from point each cloud of thesereceived scenarios from each has identicalof these scenarios responses, has which identical can beresponses distinguished, which fromcan be the distinguished recorded dataset. from the On recorded the other dataset. hand, GC On positionthe other extraction hand, GC using position the 3Dextraction model byusing visual the inspection3D model by can visual be considered inspection as can a more be considered precise attempt as a more for the precise inhomogeneous attempt for datasetthe inhomogeneous of the rail track. dataset However, of the separationrail track. However, of the ideal separation track from of the the irregular ideal track neighborhood from the irregular with a largeneighborhood dataset is withvery time a large consuming dataset and is very resource time intensive.consuming Therefore, and resource a general intensive. method Therefore, needs to be a implementedgeneral method for needs the enhancement to be implemented of the GC for extractionthe enhancement process. of the GC extraction process.

(a) (b) (c) (d)

Figure 2. RailRail track track fastning fastning:: ( (aa)) i idealdeal case, case, t turnouturnout ( (bb)) before before chang changinging the track and ( c) after chang changinging the track and ((d)) nearnear rail crossing.

2.2.2.2. MLS Sensor and Survey Setup The MLS system has been optimized as a combination of several sensors [2[244,2255]. The key components of a MLS include 3D laser scanners, GNSS, inertial measurement unit (IMU)(IMU) and optical cameras. Figure 33 depictsdepicts thethe MLSMLS systemsystem usedused inin thisthis experiment.experiment. ThisThis systemsystem waswas fullyfully equippedequipped with the above specifiedspecified sensor units,units, which facilitate to acquire point cloud by a density of 1600 per square meter at a 10 m distance from the LiDAR source while having a moving at a speed of 10 m per second. It increase increasedd the data acquisition rate up to 1.016 million pixelspixels perper second.second. This becomes a huge advantageadvantage especiallyespecially for 3D3D modelmodel generationgeneration andand featurefeature extraction.extraction. The systems can be mounted on vehicles, trolleys or boats, with the former two often often being being used used in in urban urban areas areas [2 [266,,2277].]. The system specificationsspecifications andand limitationslimitations areare summarizedsummarized inin TableTable1 .1. In this experiment, point cloud around the rail track was acquired by MLS on a trolley pulled by a motor car. Figure4 depicts the experiment setup of the monitoring system. The Leica Pegasus MDA software was used for data acquisition [28]. It allows to store point cloud, images, IMU and GNSS data in the system. The continuously acquired point cloud along the rail track was registered with time and the GNSS coordinate. The preliminary data processing was done by the system client software Leica Pegasus Manager [29] and then the self-script algorithm was coded by C++ to analyze the point cloud data. For the rail track extraction, the point cloud extraction range was defined as depicted by the rectangular area covered by dashline with the height offset value from the GC position in the vertical plane. The experimental setup parameters are summarized in Table2.

Remote Sens. 2020, 12, 3702 5 of 16 Remote Sens. 2019, 11, x FOR PEER REVIEW 5 of 16

FigureFigure 3. 3. MLSMLS sensor sensor..

Table 1. Sensor parameters. Table 1. Sensor parameters.

ProductProduct Name Name Pegasus: Pegasus: Two Two ManufactureManufacture Leica Leica Geosystems Geosystems LaserLaser scanner scanner Z+FZ+ FPROFILE PROFILE®® 90129012 DataData acquisition acquisition rate 1.016 1.016 million million pixel/s pixel/s (200 (200 Hz) Point density 1600 pts./m2 (@10 m, 10 m/s) Point density 1600 pts./m2 (@10 m, 10 m/s) Distances 0.3–119 m Distances 0.3–119 m Relative accuracy About 0.2–0.5 mm (@10 m) Relative accuracy About 0.2–0.5 mm (@10 m) Absolute accuracy 2 cm (Open sky condition) Absolute accuracy 2 cm (Open sky condition)

In thisRemote experiment, Sens. 2019, 11, pointx FOR PEER cloud REVIEW around the rail track was acquired by MLS on a trolley6 of 16 pulled by a motor car. Figure 4 depicts the experiment setup of the monitoring system. The Leica Pegasus MDA software was used for data acquisition [28]. It allows to store point cloud, images, IMU and GNSS data in the system. The continuously acquired point cloud along the rail track was registered with time and the GNSS coordinate. The preliminary data processing was done by the system client software Leica Pegasus Manager [29] and then the self-script algorithm was coded by C++ to analyze the point cloud data. For the rail track extraction, the point cloud extraction range was defined as depicted by the rectangular area covered by dashline with the height offset value from the GC position in the vertical plane. The experimental setup parameters are summarized in Table 2.

FigureFigure 4. 4.Experiment Experiment setup. setup.

Table 2. Experiment setup parameters.

Parameter Value GPS mounting height 1500 mm Extraction height 400 mm Height offset 200 mm Half width 1067/2 mm Point cloud extraction range 1000 mm Height of the trolley from rail head 500 mm

Remote Sens. 2020, 12, 3702 6 of 16

Table 2. Experiment setup parameters.

Parameter Value GPS mounting height 1500 mm Extraction height 400 mm Height oﬀset 200 mm Half width 1067/2 mm Point cloud extraction range 1000 mm Height of the trolley from rail head 500 mm

2.3. Vertical Slicing The point cloud distribution in the plane parallel to the railway track has a sensibility to the rail bed structures such as the rail bottom, rail web and surface of the rail head, which lies on the vertical plane. In the proposed method, we used vertical slices defined from GNSS trajectory to extract the corresponding point densities. The GNSS trajectory lies parallel to the railway track as depicted in Figure5a. We defined vertical slices by centering the GNSS track position. The total slice width 2r was divided into equal sectors. The same process was consecutively applied to every 1 m position along the railway track. In the proposed method, the estimation of the vertical slice edge coordinates become significant for fast and accurate data processing. Those edge coordinates, in another word, boundary coordinates were used to define the rectangular area as depicted in Figure5b to extract point densities from the relevant spatial location. In this algorithm, we used the geometrical method to calculate the edges of each vertical slice. This process was divided into left and right side slices from the GNSS track position to separate the structure position in the horizontal plane. The circle with center xc, yc and zc in Figure5a with radius r and AB was defined as a line perpendicular to the trajectory. This was calculated by considering the consecutive GNSS coordinates of xi, yi, zi and xi+1, yi+1 and zi+1, which is separated by a 1 m distance. The intersection position between the line and the circle in each r values of both left and the right sides were used to estimate the boundary coordinates of left vertical slices and the right vertical slices. The estimated boundary values were used to clip corresponding points, which are located inside the slice area and count the points of each vertical slice. The number of vertical slices within the 2r diameter becomes an important parameter for target identification and processing time estimation. In this experiment, the structure identification was limited to precise identification of crossings and turnout. Therefore, 2r diameter was divided into equal slices by considering the dimension of the rail in Figure6 rail and the distance in between railway tracks. Figure7 depicts the expected formation of the rail position within the vertical slices discussed in the proposed method. In the local railway line, the dimension of the rail varies and can be standardized by the weight of its unit length of 1 m. In the Japan Railway (JR) West operator, 7 stranded rail shapes were used for train operation. This rail shapes and their dimensions were summarized in Figure6. In this study, the width of the rail head a varied between 60 and 70 mm (<100 mm) and b, c and d varied between 100 and 150 mm, 100 and 175 mm and 10 and 20 mm respectively. The total track width had a fixed value of 1067 mm between the rails. Therefore, the vertical slice number was defined as 10 units. Figure7a depicts the ideal railway track formatted within 10 strips. The first most slices in the left handed side and last most slices in the right handed side were densely populated. However, the point count of each slice varied due to survey condition. The monitoring speed of the MLS could not be maintained as a constant value due to speed limits of the railway track. Sudden slowdown while passing railway stations and slowdown near the track turnout or stop near railway crossings affected the stability of the moving speed. This will affect the observed point density, which changes along the same track in different situations. Therefore, the mean threshold value for peak detection was estimated for the ideal track, left/right turn out and crossing detection. The threshold value for turnout detection is defined Remote Sens. 2020, 12, 3702 7 of 16

asRemote CT1 Sens.and 2019 the, threshold11, x FOR PEER value REVIEW for crossing detection is deﬁned as CT2. On the other hand, the point7 of 16 density peak of the individual vertical slice has dynamically moved from left to right as in Figure7b,c incorresponding left-turn out points or in the, which alternative are located direction inside in the right slice handed area and turnout. count Therefore, the points a of method each vertical based onslice. vertical The number slices and of vertical its spatial slices change within in the horizontal2r diameter plane become needss an to important be implemented parameter for for accurate target estimationidentification of railand structures. processing time estimation.

(a) (b)

Figure 5.5. The recorded GNSS for (a) boundary coordinate estimation and (bb) vertical slice definition. Remote Sens. 2019,The 11, x recorded FOR PEER GNSS REVIEW for ( ) boundary coordinate estimation and ( ) vertical slice deﬁnition.8 of 16

In this experiment, the structure identification was limited to precise identification of crossings and turnout. Therefore, 2r diameter was divided into equal slices by considering the dimension of the rail in Figure 6 rail and the distance in between railway tracks. Figure 7 depicts the expected formation of the rail position within the vertical slices discussed in the proposed method. In the local railway line, the dimension of the rail varies and can be standardized by the weight of its unit length of 1 m. In the Japan Railway (JR) West operator, 7 stranded rail shapes were used for train operation. This rail shapes and their dimensions were summarized in Figure 6. In this study, the width of the rail head a varied between 60 and 70 mm (<100 mm) and b, c and d varied between 100 and 150 mm, 100 and 175 mm and 10 and 20 mm respectively. The total track width had a fixed value of 1067 mm between the rails. Therefore, the vertical slice number was defined as 10 units. Figure 7a depicts the ideal railway track formatted within 10 strips. The first most slices in the left handed side and last most slices in the right handed side were densely populated. However, the point count of each slice varied due to survey condition. The monitoring speed of the MLS could not be maintained as a

constant value due to speed limits of the railway track. Sudden slowdown while passing railway stations and slowdown near theFigureFigure track 6. 6.turnout DimenDimensions ionor stop of of the the near standard standard railway rail. rail. c rossings affected the stability of 2.4.the moving Structure speed Detection. This will affect the observed point density, which changes along the same track in different situations. Therefore, the mean threshold value for peak detection was estimated for the idealThe track, point left/right density turn value out and and its crossing position detection. in the vertical The slicethreshold array becomevalue for essential turnout factors detection while is detectingdefined as the CT1 crossings and the threshold and turnouts value from for crossing the point detection cloud. is Figure defined7b–d as depictsCT2. On anthe example other hand, of the the turnoutpoint density and crossing peak of formation the individual in vertical vertical slices. slice We has used dynamically that information moved with from the followingleft to right pattern as in detectionFigure 7b methods,c in left-turn to identify out or in the the structures alternative and direction to measure in right their handed lengths. turnout. In the proposed Therefore, method a method for crossingbased on detection vertical slices in Equation and its(a (1),spatia) verticall change slices in werethe horizontal formatted plane in liner needs 2D coordinates;to be implemented where iforis theaccurate slice iterationestimation along of rail the structures. track and j is the slice iteration in the cross-track direction. Si,j denotes the point density of vertical slice. Then the point density variation was examined in each consecutive iteration captured by a 1 m interval. Near to the railway crossing, both vertical slices in the ith and jth

(b)

(c)

(d)

Figure 7. Proposed vertical slice for point density extraction near to the railway: (a) ideal track (b) beginning of the track turnout (c) passing of the track turnout (d) the level crossing.

2.4. Structure Detection The point density value and its position in the vertical slice array become essential factors while detecting the crossings and turnouts from the point cloud. Figure 7b–d depicts an example of the turnout and crossing formation in vertical slices. We used that information with the following pattern detection methods to identify the structures and to measure their lengths. In the proposed method for crossing detection in Equation (1), vertical slices were formatted in liner 2D coordinates; where i is the slice iteration along the track and j is the slice iteration in the cross-track direction. Si,j denotes the point density of vertical slice. Then the point density variation was examined in each consecutive iteration captured by a 1 m interval. Near to the railway crossing, both vertical slices in the ith and jth

Remote Sens. 2019, 11, x FOR PEER REVIEW 8 of 16

Remote Sens. 2020, 12, 3702 8 of 16

direction simultaneously get a higher value as shown in Figure7d. Therefore, the crossing Pi can be detected by evaluating Qi,j the area, which exceeds the threshold CT1,

L ( X 0, Si,j < CT1 Pi = Qi,j (1) 1, Si j CT j=1 , ≥ 1 Figure 6. Dimension of the standard rail.

(a)

(b)

(c)

(d)

Figure 7. Proposed vertical slice for point density extraction near to the railway railway:: ( (aa)) ideal ideal track track ( (bb)) beginning of the track turnout (c) passing of the track turnoutturnout ((d)) thethe levellevel crossing.crossing.

2.4. StructureL is the maximum Detection number of iterations in the cross-track direction. If j moves until 1 to L, then the crossings can be identified by exceeding the value of CT1 as depicted in Figure8a. The point density value and its position in the vertical slice array become essential factors while On the other hand, near to the turnouts, the point densities in both ith and jth directions have peak detecting the crossings and turnouts from the point cloud. Figure 7b–d depicts an example of the values consecutively. In the proposed method for turnout detection, the point density in the middle of turnout and crossing formation in vertical slices. We used that information with the following pattern the railway track was checked first and the relevant threshold value (CT2) was testified in both left detection methods to identify the structures and to measure their lengths. In the proposed method and right handed neighborhood of the vertical slice until the local maxima for turnout selection as for crossing detection in Equation (1), vertical slices were formatted in liner 2D coordinates; where i depicted in Figure8b. The number of iterations was counted for length estimation, until satisfying the is the slice iteration along the track and j is the slice iteration in the cross-track direction. Si,j denotes following logical condition for peak detection within neighboring slices, the point density of vertical slice. Then the point density variation was examined in each consecutive iteration captured by a 1 m interval. Near to the railway crossing, both vertical slices in the ith and jth Si,j Si+1,j & Si,j Si 1,j (2) ≥ ≥ − Remote Sens. 2019, 11, x FOR PEER REVIEW 9 of 16 direction simultaneously get a higher value as shown in Figure 7d. Therefore, the crossing Pi can be detected by evaluating Qi,j the area, which exceeds the threshold CT1, L 0,S  C i, j T1 Pi  Qi, j  (1) j1 1,Si, j  CT1 L is the maximum number of iterations in the cross-track direction. If j moves until 1 to L, then the crossings can be identified by exceeding the value of CT1 as depicted in Figure 8a. On the other hand, near to the turnouts, the point densities in both ith and jth directions have peak values consecutively. In the proposed method for turnout detection, the point density in the middle of the railway track was checked first and the relevant threshold value (CT2) was testified in both left and right handed neighborhood of the vertical slice until the local maxima for turnout selection as depicted in Figure 8b. The number of iterations was counted for length estimation, until satisfying the following logical condition for peak detection within neighboring slices,

Si, j  Si1, j & Si, j  Si1, j (2) This method was repeatedly applied until it reached the left and right edges of the vertical slices corresponds to point densities of the rail-track. The difference between the number of slices count in left and right stands as a total length of the turnout. Thereby the turnout initiation location and total Remotelength Sens. was2020 estimated, 12, 3702 to distinguish them from ideal model fitting [30]. 9 of 16

CT1 Along the track direction (m) theAlongtrack direction

Cross track direction (a) (b)

FigureFigure 8.8. ApplyApply thethe proposedproposed methodmethod toto thethe realreal datasetdataset nearnear thethe ((aa)) railrail crossingcrossing andand ((bb)) turnoutturnout inin thethe 2D2D plane.plane.

2.5. ModelThis method Fitting was repeatedly applied until it reached the left and right edges of the vertical slices corresponds to point densities of the rail-track. The difference between the number of slices count in The preliminary scanning of the MLS server dataset detects and classifies the ideal track, left and right stands as a total length of the turnout. Thereby the turnout initiation location and total left/right turnout and crossing locations. It identifies the locations of the rail track, which need to be length was estimated to distinguish them from ideal model fitting [30]. matched with vivid shapes for precise estimation of GC in the second stage. Therefore, the survey 2.5.dataset Model itself Fitting is used as the organizer to classify each different structural shape and locations. This is used as a dictionary to match the model according to the probable shape of the classified track The preliminary scanning of the MLS server dataset detects and classifies the ideal track, left/right location using the iterative closest pair method [31–34]. This self-organized frame of the proposed turnout and crossing locations. It identifies the locations of the rail track, which need to be matched method enhances the accuracy of the conventional GC detection method [18–20]. In this survey, we with vivid shapes for precise estimation of GC in the second stage. Therefore, the survey dataset used three identical shapes to match the structures, which was described in Section 2.3. The identical itself is used as the organizer to classify each different structural shape and locations. This is used as shapes of the rail track were synthesized by measuring the dimensions of the rail head. a dictionary to match the model according to the probable shape of the classified track location using The clipped point cloud in Figure 9a is projected into the plane, which is vertical to the trajectory the iterative closest pair method [31–34]. This self-organized frame of the proposed method enhances direction as depicted in Figure 9b. Next, the position of the GC, which is the inside corner of a rail the accuracy of the conventional GC detection method [18–20]. In this survey, we used three identical head, was extracted by matching the shape of the ideal rail head with respect to the projected point shapes to match the structures, which was described in Section 2.3. The identical shapes of the rail cloud in Figure 9c by adopting the ICP algorithm. The reason for using the shape of a rail head instead track were synthesized by measuring the dimensions of the rail head. of a whole shape is to avoid the mismatching of GCs’ height due to rail wear in each model. Then the The clipped point cloud in Figure9a is projected into the plane, which is vertical to the trajectory direction as depicted in Figure9b. Next, the position of the GC, which is the inside corner of a rail head, was extracted by matching the shape of the ideal rail head with respect to the projected point cloud in Figure9c by adopting the ICP algorithm. The reason for using the shape of a rail head instead of a whole shape is to avoid the mismatching of GCs’ height due to rail wear in each model. Then the precise position of GC was estimated and the railway track position was accurately estimated as in Figure9d. The mirror image of synthesized shape in each model in Figure9d was applied to track the extraction in both left and right railway lines. This method was repeatedly applied to each of the track points located in a 1 m distance. Finally, the rail geometry and track center line are defined as each connected GC positions and each points of track center sequentially. Remote Sens. 2019, 11, x FOR PEER REVIEW 10 of 16 precise position of GC was estimated and the railway track position was accurately estimated as in Figure 9d. The mirror image of synthesized shape in each model in Figure 9d was applied to track the extraction in both left and right railway lines. This method was repeatedly applied to each of the trackRemote points Sens. 2020 located, 12, 3702 in a 1 m distance. Finally, the rail geometry and track center line are defined10 of as 16 each connected GC positions and each points of track center sequentially.

Model1

2 Model

3 Model

(a) (b) (c) (d)

FigureFigure 9. 9. SpatialSpatial d dispertionispertion of of ( (aa)) extracted point cloudcloud inin 3D3D planeplane ( b(b)) projected projected points point ins in 2D 2D plane, plane (c, ) (cproposed) proposed model model ﬁtting fitting shapes shapes (d )(d ICP) ICP applied applied ideal ide trackal track in Modelin Model 1, level 1, level crossing crossing in Model in Model 2 and 2 andturnout turnout in Model in Model 3. 3.

3. Experiment and Evaluation 3. Experiment and Evaluation TheThe onsite onsite experiment experiment was was carried carried out out by by the the MLS MLS sensor sensor mount mount on on a atrolley trolley,, which which was was attached attached toto the the motor motor car. car. The The complete complete system; system; sensor, sensor, trolley trolley and andmotor motor car are car depicted are depicted in Figure in Figure 10a. The 10a. systemThe system acquired acquired point point cloud cloud from fromthe surrounding the surrounding environment environment while while moving moving along alongthe railway the railway line. Theline. recorded The recorded point point cloud cloud includes includes the therail rail track track and and other other railway railway structures; structures; signaling signaling polls, polls, cantilevers,cantilevers, kilo kilo post post and and distributed distributed structures; structures; station station platforms, platforms, bridges, bridges, tunne tunnels,ls, etc. etc. Therefore, Therefore, thethe identificationidentification of of the the point point cloud cloud corresponded correspond toed the to rail the track rail becoming track becom challenging.ing challenging The geometry. The geometryof the experimental of the experimental setup and setup acquired and acquired point cloud point were cloud used were to used obtain to obtain a relative a relative position position of the ofrailway the railway track. Thetrack. GNSS The alongGNSS with along MLS with was MLS defined was asdefined the initiative as the positioninitiative of position recorded of point recorded cloud. pointThereby cloud. extraction Thereby length extraction was set length as the was maximum set as the distance maximum in the distance vertical in plane the andvertical 1 m plane interval and was 1 mset interval as the regionwas set of as observation the region inof theobservation horizontal in plane. the horizontal The overall plane. processing The overall flow wasprocessing summarized flow wasin Figure summarized 10b. in Figure 10b. Two local railway lines were tested in the preliminary survey using the same experimental setup. Railway line A is located near Osaka city, Japan and is depicted in Figure 11. Railway line B is located near Kyoto city, Japan and is depicted in Figure 12. Both locations are heavily urbanized and the transportation infrastructure lies favorable for the complicacy of the surrounding. The stranded distance between two rail heads (2r) was fixed as 1067 mm. The height information within a 1 m interval was calibrated with GPS and the motion sensor. Then the fixed height adjustment factor was estimated to detect the rail position in the vertical plane. The height adjustment factor becomes an important parameter to estimate the threshold value CT. In the presurvey, point density recorded in the ideal track, turnout and crossings was examined. Then the estimated average value of point density for each railway structure ideal track, turnout and crossing was used as a threshold value CT during the regular MLS survey. The recorded point densities and their spatial location along the track (horizontal plane) and in the cross track (vertical plane) were used to identify the structures lie on the railway track as depicted in Figure 13. Remote Sens. 2019, 11, x FOR PEER REVIEW 11 of 16

Remote Sens. 2020, 12, 3702 11 of 16 Remote Sens. 2019, 11, x FOR PEER REVIEW 11 of 16

(a) (b)

Figure 10. (a) MLS survey setup with the LiDAR sensor and (b) processing flow chart.

Two local railway lines were tested in the preliminary survey using the same experimental setup. Railway line A is located near Osaka city, Japan and is depicted in Figure 11. Railway line B is located near Kyoto city, Japan and is depicted in Figure 12. Both locations are heavily urbanized and the transportation infrastructure lies favorable for the complicacy of the surrounding. The stranded distance between two rail heads (2r) was fixed as 1067 mm. The height information within a 1 m interval was calibrated with GPS and the motion sensor. Then the fixed height adjustment factor was estimated to detect the rail position in the vertical plane. The height adjustment factor becomes an important parameter to estimate the threshold value CT. In the presurvey, point density recorded in the ideal track, turnout and crossings was examined. Then the estimated average value of point density for each railway structure ideal track, turnout and crossing was used as a threshold value CT during the regular MLS survey. The(a) recorded point densities and their spatial location(b) along the track (horizontal Figureplane)Figure 10.10.and ((aa in)) MLSMLS the surveycrosssurvey track setupsetup (vertical withwith thethe plane) LiDARLiDAR were sensorsensor used andand (to(bb) )identify processingprocessing the ﬂowflow struc chart. chart.tures lie on the railway track as depicted in Figure 13.

Two local railway lines were testedOsaka aoyama in minoh the campus preliminary survey using the same experimental Kawanishi Settsu-minami IC Minoo setup. Railway line A is located near Osaka city, Japan and is depictedOsaka University Ibaraki in CC Figure 11. Railway line B is Nakayamadera Ikeda located near Kyoto city, Japan and is depicted in Figure 12. Both locations areIbaraki heavily urbanized and Suita IC Chugoku-suita IC Chugoku-suita IC the transportation infrastructure lies favorableOsaka University for the complicacy of the surrounding. The stranded Takarazuka distance between twoAkuara -minamirail heads (2r) was fixed as 1067 mm. The height information within a 1 m Settsu-kita IC Keihan GC

Takarazuka GC Road no 42 no Road interval was calibrated withItami GPS and the motion sensor. Then theSuita fixed height adjustment factor was Hanshin Toyonaka Kasuga estimated to detect the rail position in the vertical plane. The height adjustmentSettsu factor becomes an Ai River importantNishiyama parameter-machi to estimate the thresholdShido value CT. In the presurvey, pSettsuoint-minami densityIC recorded in Muko River Suita Toyonaka IC Moriguchi JC the ideal track, turnout and crossingsKamisakabe was examined. Then the estimated average value of point density for each railway structure ideal track, turnoutNishi and Mikuni crossing was used as a threshold value CT Tachibana-machi HIGASHIYODOGAWA WARD Kadoma Nishinomiya Moriguchi during the regular MLS surveyAmagasaki. The recorded point densities and theirYodo River spatial location along the track Kadoma IC Nishinomia IC

(horizontal plane) and in the cross track (vertical plane)YODOGAWA WARDwere used to identifyASAHI WARD the structuresRoad no 16 lie on the Tsukamoto Kadoma JC railway track as depicted in Figure 13.Tsukuda NISHIYODOGAWA WARD KITA WARD Daito-tsurumi IC JOTO WARD TSURUMI MIYAKOJIMA WARD WARD Osaka aoyama minoh campus Higashiosaka-kita IC Kawanishi Settsu-minami IC Osaka Amagasaki Port Minoo Otemae Clinics Osaka University Ibaraki CC Nakayamadera Ikeda Sakanatsuri Park KONOHANA CHUO WARD Chuo Ibaraki WARD Higashiosaka NISHI WARD no Road 2 Osaka Hokko Marina Suita IC Chugoku-suita IC Chugoku-suita IC Osaka University HIGASHINARI WARD Maishima Sports Island Takarazuka Akuara-minami MINATO Osaka Shoin Women’s University Yahataya Park WARD Osakako NANIWA WARD TENNOJI TAISHO WARD WARD Settsu-kita IC Keihan GC

Takarazuka GC IKUNO WARD Shrine Road no 42 no Road Itami Suita Hanshin Toyonaka Kasuga Settsu Figure 11. MLS survey site JR Tozai railway line, Osaka city, JapanAi River (Railway line A). Nishiyama-machiFigure 11. MLS survey site JR TozaiShido railway line, Osaka city, Japan (RailwaySettsu-minami lineIC A). Muko River Suita Toyonaka IC Moriguchi JC However, the rail track conditionsKamisakabe in each line have some diﬀerences to the acquired MLS Nishi Mikuni Tachibana-machi HIGASHIYODOGAWA WARD Kadoma datasets.Nishinomiya The average moving speed during the experiment has been reportedMoriguchi as 20 km/h in the Amagasaki Yodo River Kadoma IC locomotive. HoweverNishinomia IC this speed has been diﬀered in both railway tracks due to the number of bends

YODOGAWA WARD ASAHI WARD Road no 16 and slope conditions. It becomes a noticeable factorTsukamoto while estimating the thresholdKadoma parameters JC CT1 Tsukuda NISHIYODOGAWA WARD and CT2 as estimated and summarized in Tables3 and4. TheKITA WARD proposed algorithmDaito was-tsurumi appliedIC to JOTO WARD TSURUMI MIYAKOJIMA WARD WARD the large point cloud dataset received from the railway track survey. The locationsHigashiosaka of-kita IC crossings, Osaka left-turnouts and right-turnoutsAmagasaki Port were notiﬁed with each trackOtemae iteration Clinics in the railway monitoring Sakanatsuri Park KONOHANA CHUO WARD Chuo WARD Higashiosaka NISHI WARD no Road 2 system. As a quantitative estimation,Osaka Hokko Marina we summarized the number of detected structures and results of HIGASHINARI WARD Maishima Sports Island visual inspection after the manual survey of eachMINATO railway line in Tables3 and4.Osaka Shoin Women’s University Yahataya Park WARD Osakako NANIWA WARD TENNOJI TAISHO WARD WARD IKUNO WARD Shrine

Figure 11. MLS survey site JR Tozai railway line, Osaka city, Japan (Railway line A).

Remote Sens. 2020, 12, 3702 12 of 16 Remote Sens. 2019, 11, x FOR PEER REVIEW 12 of 16

Lake Biwa Lake Sainoko Aisho

Omihachiman Higashiomi

Yasu Moriyama SAKYO WARD Ryuo KITA WARD

KAMIGYO WARD Hino Otsu Ritto SHIMOGYO WARD Kyoto Kusatsu Konan

MINAMI WARD YAMASHINA WARD Koka

FUSHIMI WARD

Uji

Joyo

Ujiawara

Kyotanbe Ide Wazuka

Remote Sens. 2019Figure, 11 ,12. x FOR MLS PEER survey REVIEW site JR Kusatsu railway line, Kyoto city, Japan (Railway(Railway lineline B).B). 13 of 16

However, the rail track conditions in each line have some differences to the acquired MLS

datasets. The average moving speed during the experiment has been reported as 20 km/h in the locomotive. However this speed has been differed in both railway tracks due to the number of bends

and slope conditions. It becomes a noticeable factor while estimating the threshold parameters CT1 Model 1 Model and CT2 as estimated and summarized in Table 3 and Table 4. The proposed algorithm was applied to the large point(a) cloud dataset received(d) from the railway track(g) survey. The locations(j) of crossings, left-turnouts and right-turnouts were notified with each track iteration in the railway monitoring system. As a quantitative estimation, we summarized the number of detected structures and results

of visual inspection after the manual survey of each railway line in Table 3 and Table 4. The comparison of model fitting results for GC detection, before and after applying the proposed

results for the left rail was depicted in Figure 13 the left side. The comparison of model fitting results Model 2 Model for GC detection, before and after applying the proposed results for the right rail was depicted in Figure 13 the right(b) side. It shows that (Modele) 1, GC of the ideal(h )track remained as the(k best) fitting in both conventional and proposed methods. In Model 2, the GC detection near to the crossing fell under a fail attempt for GC detection in Figure 13b,h. On the other hand, predetection of the crossing location by self-organized query changes the shape of the GC for both left and right side rail heads. Then, the GC position was accurately estimated. Therefore, the proposed alternative process becomes a very efficient3 Model method to detect and further analyze the locations, which need more attention for the ICP based model fitting method. Model 3 in Figure 13c depicts the point cloud near to turnout (c) (f) (i) (l) location. The conventional method and the proposed method accurately estimated the left GC corner, but theFigureFigure right 13. 13. GC GCGC fell detection under results false estimation. of of the the left left rail rail This track track could by by the thebe c onventional conventionalcause due trackto track shape extraction extraction irregularities met method:hod: (ina) the neighborhood(aideal) ideal track track, of, (b the) (nearb) turnout. near crossing crossing Theand and( railc) near ( headc) nearturnout shape turnout and near the and proposed to the the proposed turnout track extraction track combined extraction method as a: method: ( compoundd) ideal structure(dtrack) ideal, ,which (e track,) near is (e ) crossingvery near challenging crossing and ( andf) near (forf) near turnout.shape turnout. simulation.GC GC detection detection This results resultsshape of of theasymmetry the right right rail rail depends track by by the theon the directionconventionalconventional of the (left/right track track extraction extraction sided) method: methodturnout.: (g( gTherefore,)) ideal ideal track, track those, ( h(h)) near near mismatching crossing crossing and and locations ( i()i) near near turnout turnoutneed to and and be the furtherthe assessed.proposedproposed The trackmanual track extraction extraction survey method: methodresults (: jproved()j) ideal ideal track, trackthat (,the k(k)) near nearproposed crossing crossing algorithm and and (l ()l near) near extract turnout. turnouted .all the turnouts, without missing any of the real structures. However, the crossing estimation shows one overdetection. This was causedTable due 3. Structure to the overlap detection of resultsthe point—Railway cloud data,line A. which later was identified as a correction position ofHeight the raw point cloud in the originalBy Manual dataset. Survey By Algorithm Threshold Structure Adjustment (Number of (Number of (points/m2) (m) Structure) Structure) Crossing 1.48 200 4 5 Left-turnout 1.48 400 2 2

Right- 1.48 400 2 2 turnout

Table 4. Structure detection results—Railway line B.

Height Threshold By Manual Survey By Algorithm Structure Adjustment (points (Number of (Number of (m) /m2) Structure) Structure) Crossing 1.5 200 16 17 Left-turnout 1.5 400 5 5 Right- 1.5 400 9 9 turnout

Remote Sens. 2020, 12, 3702 13 of 16

Table 3. Structure detection results—Railway line A.

Height Adjustment Threshold By Manual Survey By Algorithm Structure (m) (points/m2) (Number of Structure) (Number of Structure) Crossing 1.48 200 4 5 Left-turnout 1.48 400 2 2 Right-turnout 1.48 400 2 2

Table 4. Structure detection results—Railway line B.

Height Adjustment Threshold By Manual Survey By Algorithm Structure (m) (points /m2) (Number of Structure) (Number of Structure) Crossing 1.5 200 16 17 Left-turnout 1.5 400 5 5 Right-turnout 1.5 400 9 9

The comparison of model fitting results for GC detection, before and after applying the proposed results for the left rail was depicted in Figure 13 the left side. The comparison of model fitting results for GC detection, before and after applying the proposed results for the right rail was depicted in Figure 13 the right side. It shows that Model 1, GC of the ideal track remained as the best fitting in both conventional and proposed methods. In Model 2, the GC detection near to the crossing fell under a fail attempt for GC detection in Figure 13b,h. On the other hand, predetection of the crossing location by self-organized query changes the shape of the GC for both left and right side rail heads. Then, the GC position was accurately estimated. Therefore, the proposed alternative process becomes a very efficient method to detect and further analyze the locations, which need more attention for the ICP based model fitting method. Model 3 in Figure 13c depicts the point cloud near to turnout location. The conventional method and the proposed method accurately estimated the left GC corner, but the right GC fell under false estimation. This could be cause due to shape irregularities in the neighborhood of the turnout. The rail head shape near to the turnout combined as a compound structure, which is very challenging for shape simulation. This shape asymmetry depends on the direction of the (left/right sided) turnout. Therefore, those mismatching locations need to be further assessed. The manual survey results proved that the proposed algorithm extracted all the turnouts, without missing any of the real structures. However, the crossing estimation shows one overdetection. This was caused due to the overlap of the point cloud data, which later was identified as a correction position of the raw point cloud in the original dataset.

4. Accuracy Estimation As a qualitative estimation, we compared the detected length of both crossings and turnouts with their real length by manually measuring each of the structure locations as the standard for veriﬁcation. Figure 14 proved that both crossings and turnout had 100% detection accuracy. The comparison of crossings in Figure 14a depicts that the lengths of the crossing structures length had a better agreement with a manual survey. However, Figure 14b, the length of the turnout shows some shifts. This could occur due to the point density variation near the target due to the characteristics of the laser scanner and the moving speed of a bogie pulled motor car, which needs to be improved in future. Therefore, the quantitative estimation of both rail crossing and turnout detection achieved a 100% successive rate. The detected length of each turnout was smaller than its actual length measured by a manual survey, but proportionally equal to its real length. Therefore, turnout length should be calculated and updated before applying the model ﬁtting step of the algorithm. Remote Sens. 2019, 11, x FOR PEER REVIEW 14 of 16 the laser scanner and the moving speed of a bogie pulled motor car, which needs to be improved in future. Therefore, the quantitative estimation of both rail crossing and turnout detection achieved a 100% successive rate. The detected length of each turnout was smaller than its actual length measured Remoteby a manual Sens. 2020 survey,, 12, 3702 but proportionally equal to its real length. Therefore, turnout length should14 of be 16 calculated and updated before applying the model fitting step of the algorithm.

(a) (b)

Figure 1 14.4. TheThe comparison comparison of of ( (aa)) d detectedetected crossings, crossings, distance distance with manual inspection and ( b) d detectedetected turnouts, distance with manual inspection inin KusatsuKusatsu line.line.

5. Conclusions 5. Conclusions The proposed algorithm presents a novel framework for automatic rail track extraction for MLS data. It allows oneone to to detect detect the the requested requested object object very very precisely precisely from from surrounding surrounding railway railway structures structures with withless background less background information. information. This was This recorded was recorded in the track in the information track information dictionary, dictionary which contained, which containedideal track, ideal turnout track, and turnout crossing and detailscrossing with details their with spatial their locations. spatial locations. Then the Then appropriate the appropriate model modelfitting method fitting method can be applied can be toapplied increase to theincrease accuracy the accuracyof the rail of GC the position rail GC estimation. position estimation. Therefore, Therefore,the automatic the rail automa extractiontic rail can extraction be done forcan the be spatialdone for locations the spatial with locations multiple with structures, multiple which structures become, problematicwhich become to evaluateproblematic by conventional to evaluate by GC conventional detection methods. GC detection Further, methods. sensitive Further, analysis sensitive can be analysisdone in order can be to estimate done in the order railway to estimate track with the the railway optimum track precision. with the In optimum the conventional precision. methods, In the conventionalthe accuracy ofmethods, the structure the accuracy locations of derived the structure by an acquiredlocations pointderived cloud by isan based acquired on the point accuracy cloud ofis basedthe received on the GNSSaccuracy reading. of the However,received GNSS the proposed reading. rail However, structure the detection proposed method rail structure is based detection on both pointmethod cloud is based density on both and patternpoint cloud recognition density and along pattern the track recognition and cross-track along the direction. track and The cross railway-track direction.structure position The railway along thestructure track was position estimated along by the pattern track recognition was estim (Sectionated by 2.3 pattern) and the recognition structure position(Section 2.3) in cross-track and the structure direction position was detected in cross by-track the direction threshold was value detected of the by vertical the threshold slice. Thus value an ofaccurate the vertical alignment slice. Thus of the an GNSS accurate position alignment will notof the become GNSS mandatoryposition will for not the become process, mandatory which is for an theadvantage process, during which the is fieldan advantage experiment. during The proposedthe field experiment. method was The tested proposed with two method local railway was tested lines. Thewith results two local indicate railway that lines. the proposed The results method indicate could that be appliedthe proposed with 100% method structure could detection be applied accuracy. with 100%However, structure the detected detection length accuracy. and theHowever, measurement the detected length length show and some the discrepancies, measurement especially length show for somelengthy discrepancies, structures, which especially need for to be lengthy compensated structures, as a which secondary need step.to be Furthermore, compensated point as a secondary density of step.each verticalFurthermore, slice becomes point density an important of each parameter vertical slice to set becomes the threshold an important value for parame structureter extraction.to set the Thisthreshold will also value depend for structure on the LiDAR extraction. system This and will moving also speeddepend of theon locomotive.the LiDAR system Those dependenciesand moving speedcan be of eliminated the locomotive. by estimating Those dependencies the threshold can values be eliminated through the by sample estimating dataset. the Inthreshold this campaign, values wethrough mainly the focused sample ondataset. three ofIn thethis most campaign, common we railmainly structures; focused ideal on three track, of turnouts the most and common crossings. rail Thosestructures; three ideal structures track, showedturnouts the and best crossings. successive Those rate three with structures no fall detection. showed Moreover, the best successive the proposed rate withstructure no fall detection detection. method Moreover, could alsothe proposed be adjusted structure according detection to the physicalmethod dimensioncould also be of theadjusted other accordiobjectsng such to the as railphysical bottom, dimension fastening of nuts,the other signaling objects sensors, such as etc.rail bottom, The resultant fastening point nuts, density signaling and its spatial location could be used for rail track structure identification. This enhanced the rail track extraction process and performed well in any complex situation with multiple disturbances. Remote Sens. 2020, 12, 3702 15 of 16

Author Contributions: Conceptualization, Y.N. and A.K.; investigation, A.K.; methodology, A.K.; algorithm, A.K. and R.H.; formal analysis, A.K.; data curation, Y.N. and R.H.; writing—original draft preparation, A.K.; writing—review and editing, A.K.; visualization, A.K.; supervision, Y.N. and R.H. All authors have read and agreed to the published version of the manuscript. Funding: This research received no external funding. Acknowledgments: We are thankful for West Japan Railway Company for the provision of MLS data. Conﬂicts of Interest: The authors declare no conﬂict of interest.

References

1. Niina, Y.; Oketani, E.; Yokouchi, H.; Honma, R.; Tsuji, K.; Kondo, K. Monitoring of railway structures by MMS. J. Jpn. Soc. Photogramm. Remote Sens. 2016, 55, 95–99. [CrossRef] 2. Miao, Z.; Wang, B.; Shi, W.; Wu, H. A Method for Accurate Road Centerline Extraction from a Classified Image. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2014, 7, 4762–4771. [CrossRef] 3. Miao, Z.; Shi, W.; Zhang, H.; Wang, X. Road Centerline Extraction from High-Resolution Imagery Based on Shape Features and Multivariate Adaptive Regression Splines. IEEE Geosci. Remote Sens. Lett. 2012, 10, 583–587. [CrossRef] 4. Wang, Y.; Chen, Q.; Zhu, Q.; Liu, L.; Li, C.; Zheng, D. A Survey of Mobile Laser Scanning Applications and Key Techniques over Urban Areas. Remote Sens. 2019, 11, 1540. [CrossRef] 5. Petrie, G. Mobile mapping systems: An introduction to the technology. GeoInformatics 2010, 13, 32–43. 6. Puente, I.; González-Jorge, H.; Martínez-Sánchez, J.; Arias, P. Review of mobile mapping and surveying technologies. Measurement 2013, 46, 2127–2145. [CrossRef] 7. Tang, P.; Huber, D.; Akinci, B.; Lipman, R.; Lytle, A. Automatic reconstruction of as-built building information models from laser-scanned point clouds: A review of related techniques. Autom. Constr. 2010, 19, 829–843. [CrossRef] 8. Khoshelham, K.; Li, Z. A Model-Based Approach to Semi? Automated Reconstruction of Buildings from Aerial Images. Photogramm. Rec. 2004, 19, 342–359. [CrossRef] 9. Mikrut, S.; Kohut, P.; Pyka, K.; Tokarczyk, R.; Barszcz, T.; Uhl, T. Mobile Laser Scanning Systems for Measuring the Clearance Gauge of Railways: State of Play, Testing and Outlook. Sensors 2016, 16, 683. [CrossRef] 10. Soni, A.; Robson, S.; Gleeson, B. Extracting rail track geometry from static terrestrail laser scans for monitoring purposes. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2014, 45, 553–557. [CrossRef] 11. Railway ICT Solution “RaiLis” by 3D Laser Measurement, Asia Air Survey Co., LTD. Available online: https://www.ajiko.co.jp/product (accessed on 6 November 2020). 12. Soni, A.X. Non-Contact Monitoring of Railway Infrastructure with Terrestrial Laser Scanning and Photogrammetry at Network Rail. Ph.D. Thesis, University College London, London, UK, 2016. 13. Zou, R.; Fan, X.; Qian, C.; Ye, W.; Zhao, P.; Tang, J.; Liu, H. An Efficient and Accurate Method for Different Configurations Railway Extraction Based on Mobile Laser Scanning. Remote Sens. 2019, 11, 2929. [CrossRef] 14. Coelho, B.E.Z. Dynamics of Railway Transition Zones in Soft Soil. Master’s Thesis, University of Porto, Porto, Portugal, 2010. 15. Yang, B.; Fang, L. Automated Extraction of 3-D Railway Tracks from Mobile Laser Scanning Point Clouds. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2014, 7, 4750–4761. [CrossRef] 16. Besl, P.; McKay, N.D. A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 1992, 14, 239–256. [CrossRef] 17. Diaz Benito, D. Automatic 3d Modelling of Train Rails in a Lidar Point Cloud. Master’s Thesis, University of Twente, Enschede, The Netherlands, 2012. 18. Arastounia, M. Automated Recognition of Railroad Infrastructure in Rural Areas from LIDAR Data. Remote Sens. 2015, 7, 14916–14938. [CrossRef] 19. Zhou, Y.; Wang, S.; Mei, X.; Yin, W.; Lin, C.; Hu, Q.; Mao, Q. Railway Tunnel Clearance Inspection Method Based on 3D Point Cloud from Mobile Laser Scanning. Sensors 2017, 17, 2055. [CrossRef] 20. Elberink, S.O.; Khoshelham, K. Automatic Extraction of Railroad Centerlines from Mobile Laser Scanning Data. Remote Sens. 2015, 7, 5565–5583. [CrossRef] Remote Sens. 2020, 12, 3702 16 of 16

21. Niina, Y.; Honma, R.; Honma, Y.; Kondo, K.; Tsuji, K.; Hiramatsu, T.; Oketani, E. Automatic rail extraction and celarance check with a point cloud captured by mls in a railway. ISPRS Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2018, 767–771. [CrossRef] 22. Kukko, A.; Kaartinen, H.; Hyyppä, J.; Chen, Y. Multiplatform Mobile Laser Scanning: Usability and Performance. Sensors 2012, 12, 11712–11733. [CrossRef] 23. Brédif, M.; Vallet, B.; Ferrand, B. Distributed Dimensonality-Based Rendering Of Lidar Point Clouds. ISPRS Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2015, 40, 559–564. [CrossRef] 24. Masiero, A.; Fissore, F.; Guarnieri, A.; Yang, B.; Visintini, D.; Vettore, A. Performance Evaluation of Two Indoor Mapping Systems: Low-Cost UWB-Aided Photogrammetry and Backpack Laser Scanning. Appl. Sci. 2018, 8, 416. [CrossRef] 25. Furitsu, Y.; Deguchi, D.; Kawanishi, Y.; Ide, I.; Murase, H.; Mukojima, H.; Nagamine, N. Semantic Segmentation of Railway Images Considering Temporal Continuity. In Pattern Recognition; Springer: Cham, Switzerland, 2019. 26. Leica Pegasus: Two Ultimate Mobile Reality Capture, Specification, Leica Geosystems AG 2018. Available online: www.leica-geosystems.com (accessed on 6 November 2020). 27. Leica Pegasus: Two Ultimate Mobile Reality Capture, Smart City Applications, Leica Geosystems AG 2018. Available online: www.leica-geosystems.com (accessed on 6 November 2020). 28. Leica Pegasus: Map Factory Mobile Reality Capture, Leica Geosystems AG 2018. Available online: www. leica-geosystems.com (accessed on 6 November 2020). 29. Leica Pegasus: Manager Complete Work Flows for Reality Capture Projects, Specification, Leica Geosystems AG 2018. Available online: www.leica-geosystems.com (accessed on 6 November 2020). 30. Karunathilake, A.; Honma, R.; Niina, Y. Detection of level crossing and turnouts by vertical slicing of railway MLS point cloud. In Proceedings of the Annual Conference of Japan Society of Photogrammetry and Remote Sensing, Tokyo, Japan, 23–24 May 2019. 31. Gelfand, N.; Ikemoto, L.; Rusinkiewicz, S.; Levoy, M. Geometrically Stable Sampling for the ICP Algorithm. In Proceedings of the Fourth International Conference on 3-D Digital Imaging and Modeling, Banff, AB, Canada, 6–10 October 2003; pp. 260–267. 32. Yoshimura, R.; Date, H.; Kanai, S.; Honma, R.; Oda, K.; Ikeda, T. Automatic registration of MLS point clouds and SfM meshes of urban area. Geo-Spat. Inf. Sci. 2016, 19, 171–181. [CrossRef] 33. Lai, K.; Fox, D. Object Recognition in 3D Point Clouds Using Web Data and Domain Adaptation. Int. J. Robot. Res. 2010, 29, 1019–1037. [CrossRef] 34. Lowe, D.G. Three-dimensional object recognition from single two-dimensional images. Artif. Intell. 1987, 31, 355–395. [CrossRef]

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional aﬃliations.

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).