2D Image Based Road Pavement Crack Detection By

2D IMAGE BASED ROAD PAVEMENT CRACK DETECTION BY CALCULATING MINIMAL PATHS AND DYNAMIC PROGRAMMING Manuel AVILA(1), Stéphane BEGOT(1), Florent DUCULTY(1), Tien Sy NGUYEN(2)

(1) PRISME Laboratory, University of Orleans, France, (2) Vectra road engineering, www.vectra.fr Email: [email protected]

ABSTRACT provide development of apparatus like AMAC© [2] in Road distress needs to be detected early to optimize road Europe or ARAN [3] in North America. These apparatus maintenance cost; automatic survey of road distress is a big collect images and various features of road pavement challenge, particularity for the detection of tiny cracks due surface. Transversal or lengthways profiles are collected. It to important variation of pavement textures. This paper is a mix of all these information that provide the diagnosis presents a new method for crack detection by finding the used to program maintenance of the road [4]. minimal path passing on each pixel of image from every We focus, in this work, on the detection of crack defect path with a length d; we propose also a dynamic on images that have these following characteristics [1], [5]: programming implementation to make it applicable in real - Intensity: crack pixels are darker than their neighbours, condition. Methods are tested on synthesis images set and a - Geometry: crack is continuous or could be composed of large set of real images. Results show that cracks as small as several continuous segments. Its length is greater than its 2mm could be detected. width and to the granulate size. These characteristics could be noised by the road texture Keywords — road distress, crack, detection, dynamic typically when they are considered separately, for example, programming pixels between granulate can be as dark as a crack pixel. Recent publications propose literature review of crack 1. INTRODUCTION detection methods [6], [7]. Most simple methods use From more than two decades, there is a growing interest in intensity thresholding to segment crack image, fixed pavement defect detection using image processing threshold in [8], [9] or fuzzy threshold in [10]. The techniques [1]. Crack detection on pavement surfaces is a segmentation results then are followed by various post- difficult problem due to the noisy rate of the pavement processing to filter-out noises. Neighbourhood is considered surfaces relative to crack size. There are different kinds of in [11], [12] or 2D Continuous Wavelet Transform in [13]. texture that can be encountered on road pavements. Cracks In [6], a region based filtering is used in segmentation can have many forms; crack size can be as small as 1 pixel step. The database containing about 100 images is in width and thinner than inter-aggregate distance. Figure 1 automatically split into two subsets used for training or for illustrates some pavement image samples. tests. Six unsupervised classifiers are used with features extracted on each region. Finally, results are given on test database with qualification of detected crack (three levels) to respond to local road administration. In [7], authors propose a method with a geodesic shadow-removal algorithm. Then a local intensity-difference measure is used for crack pixel detection. This measure used the 8-connected neighbourhood of the pixel. It mixes intensity and geometrical features of crack. To increase this Figure 1: pavement image samples: (a) longitudinal mixing, a tensor voting method is used to connect crack crack, (b) transversal crack, (c) alligator crack. pixels and to filter the others. Finally, results are presented on a set of 200 images. After this introduction, in the part 2, we propose a short review of recent works. Our minimal path method is In the next part, we propose a method that takes into detailed in the part 3. The Free Form Anisotropy (FFA) account simultaneously intensity and crack geometry based on this minimal path is defined in part 4. Different features for segmentation step. tests and results are proposed in part 5. Then we conclude. 3. MINIMAL PATH 2. RELATED WORK 3.1. Image Model: Weighted graph In the last years, many works dealing with analysis of road An image can be modelled as a graph. So, this image can be pavement surface have been proposed. The advance in considered as set of nodes (pixels) that are weighted by technologies like dynamic image capturing, laser pixel intensity. illuminating or high capacity to store huge amount of data

978-1-4799-5751-4/14/$31.00 ©2014 IEEE 783 ICIP 2014 Neighbourhood defines the set of edges that can be Input: G = oriented weighted graph, modelling connected to a node. Instead of considering a classical 8- image through one direction. �� is a matrix with as many connected neighbourhood, we chose a 4-connected oriented nodes as pixels of the inspected image. Each node need to neighbourhood (Figure 2) that provide an oriented graph. be determined nearest neighbours at a distance from 0 to d and their weights: Nl : {weight[0..d]; NearestNeighbour[0..d]}. A defines oriented neighbours. � Output: Matrix � fill with minimal path from Nl with � length 0…d for all nodes, �� ∈ � (with Nl the node corresponding to pixel l).

Algorithm 1: Searching minimal path Figure 2: Neighbourhood for oriented graph. 1. Procedure SearchMinimalPath 2. Input: G = � 3.2. Definition of minimal path 3. � According to image model previously proposed, a minimal 4. Output: �� ! ! path is defined as a path for which sum of pixel intensities is 5. Set � for each node �! ∈ � : ! the smallest in every path passing by a considered pixel with 6. �! . ��ℎ� 0 = �� a length d. 7. �!. ��ℎ�� 0 = �! ! ! A minimal path is composed of 2-oriented graphs 8. For length k from 1 to d do ! � (Figure 3). This choice prevents the algorithm against path’s 9. For each node �! ∈ � do

loopback. 10. Search neighbour ��! ∈ (��ℎ�� !) which has the minimum weight for length k-1 : 11. ��!,!"# = ��(��!. ��ℎ�[� − 1]) ! 12. Update �! : 13. �!. ��ℎ� � = � + ��!. ��ℎ�[� − 1] 14. �!. ��ℎ�� = ��!,!"# 15. End For Each 16. End for length Figure 3: Sample of minimal path merged with 17. End procedure 2-oriented graphs. 4. FREE-FORM ANISOTROPY For each pixel of the image, we found four minimal paths Free-Form anisotropy is computed by using minimal paths, with help of 4-oriented graphs (Figure 4), which can be used which have been selected by dynamic programming. For to compute Free-Form Anisotropy [14]. each minimal path extracted for the four orientations (Figure 4), features are calculated and converted into sources using possibility theory [16] (Figure 5). Source πi is composed of mean µi and standard deviation σi of pixel intensities of a minimal path.

Figure 4: Minimal paths of pixel l according to four orientations: (a) transversal, (b) longitudinal, (c) diagonal 135°, (d) diagonal 45°.

h(,ππ )sup{min(,)}= ππ (4.1) Our minimal path is different from Dijkstra path [15] that 12 12 finds the path with minimal cost between two points Figure 5: Sup-min function to evaluate degree of regardless to their length. In our case, the minimal path is coherence between two sources. defined from the centre pixel towards two unknown extremities at a fixed distance d. This path is supposed to be To avoid a learning step for backgrounds estimation, we a crack path when the centre pixel is a cracking pixel. suppose that crack affects texture only in one orientation. Then we can use, for the background source, πmax the 3.3. Algorithm: Searching minimal path minimal path that has the maximum weight. Considering crack characteristics and definition of

978-1-4799-5751-4/14/$31.00 ©2014 IEEE 784 ICIP 2014 minimal path, we observe that (for samples in Figure 6): the method to filter the background and enhance crack. In a - if pixel l is part of a crack, the crack is one of second test, we compare results of our method with one the oriented minimal paths (Figure 6a), this one developed in France with the same kind of images. Then we

that has the minimal weight πmin, give some results on a database composed of more than - if pixel l is a defect-free pixel, variation 16.000 images captured on highways of French road between weight of minimal paths is smaller network. than in previous case (Figure 6c). 5.1. Tests on synthetic cracks Different kinds of pavement images have been used. In Table 1, we see textures attributes for each kind of texture: granulate size, contrast. The two last parameters are Haralick attributes [17]. Image Granulate size Contrast Correlation Entropy (mm)min/max 1 0/4 308.1 0,001 8,7 2 0/10 537,3 0,0005 9,38 3 0/13 612,4 0,0032 9,4 4 0/18 876,72 0,00031 9,9 Table 1: Texture attributes of selected images. On these images (Figure 7), we generate synthetic cracks. Crack pixels intensities were randomly chosen in a range near crack pixel intensity. Cracks have “no segment” forms.

Figure 6: FFA computed for 2 pixels with d = 30. (a,b) crack pixel (c,d) defect-free pixel. The FFA of a pixel l can be formulated as: Figure 7: Synthetic cracks on different kinds of �� = 1 − ℎ(�!"#, �!"#) (4.2) pavement images. πmin and πmax are found comparing weight of the four minimal paths (Figure 6) with pixel l at the centre of the In Figure 8, we see FFA results for two distances d=4 or path with (2*d+1) length. We are not able to compute the d=32. In each case, all cracks are fully detected. If this distance d but this distance must be chosen larger than the distance is too small, FFA is not able to filter background. granulate size but not too high to avoid burdening the The distance must be higher than granulate size to obtain an calculations. efficient filtering. If distance d is high enough, there is no The FFA measure is close to 1 for crack pixels and close noise detection. The method is able to perform background to 0 for defect-free pixels. In Figure 6a, we see that the free- suppression for different kinds of textures. This form path follows the crack. Figure 6b illustrates the demonstrates the robustness of the method. coherence degree value produced by the crack pixel (h=0.09 so FFA(l)=0.91). This shows the ability of FFA to explore with accuracy different crack forms. Without crack, the minimal paths produce sources with high correlation level (Figure 6d).

5. RESULTS Perform objective tests to compare different methods, with different datasets, is a difficult task. Differences between administrations, in different countries of Europe or between continents, induce different goals for developments. Due to difficulties (for example costs) to have access to high Figure 8: FFA results on different kinds of pavement resolutions images of road pavement, it is not easy to images. propose a comprehensive reference images database to 5.2. Comparative results compare with others methods.. For this test, we compare our results with those presented in In this context, we propose three ways to test our [13]. This method is based on a 2D continuous wavelet method. In a first time, we generate default in images of transform (CWT) followed by a Markov random field different kind of pavement. This test shows the efficiency of segmentation step. Then a Hough Transform is used to

978-1-4799-5751-4/14/$31.00 ©2014 IEEE 785 ICIP 2014 detect segment. Results of a threshold method are also with information of expert about defect nature (many proposed to justify that intensity cannot be used to detect images are without defect). We compare our crack detection crack. method to expert tags. Table 3 gives defect detection rate on Table 2 give results for the three methods (with three this database. indicators: Recall (Rec), Precision (Pr) and degree of discontinuity (DD)). Figure 9 shows results of the three methods on three static images. We can see efficiency of FFA to detect crack. The CWT & Markov method enhance crack but segmented images are very noisy. Method Image 1 Image 2 Image 3 (Without crack) Rec Pr DD Rec Pr DD Rec Pr DD

CWT&Markov 23,8 37 122 38,5 58 272 0/3256 - - Figure 10: FFA results. (a) Longitudinal crack, (b) Threshold 34,5 75 113 39 50 311 0/6351 - - transversal crack, (c) alligator crack, (d) defect-free.

FFA 100 96 1 99,5 92 3 0/0 - - Kind of default number FFA Table 2: Comparative results on static images. of images Good detection Image without crack 1000 86,4 % Longitudinal crack 14335 94,6 % Transversal crack 533 76,7 % Alligator crack 605 83 % Total 16473 87,9 % Table 3: Defect detection rate on real defect database. The FFA obtains very good results with a rate of up to 88%. Worst results are obtained on images with transversal cracks. In this case, we can supposed that the system used to capture images (line scan camera with laser illumination) cannot provide enough contrast for cracks because laser beam and cracks are parallel so, crack profile cannot be enhanced by illumination. For this reason, the FFA method obtains the best results for longitudinal cracks (near 95%).

6. CONCLUSION AND PERSPECTIVES

Figure 9: Comparative results on static images. Row1: In this paper, we proposed a 2D image based method for original images (the last without crack), row2: CWT & Markov road pavement crack detection. The originality of the method, row3: Threshold, row4: FFA method is to use in a single step, geometry and intensity properties of crack pixels. To perform this, we developed an 4.3. Results on real defects on pavement images algorithm to find the minimal path from a considered pixel In these tests performed on real images (Figure 10) captured with a predefined length. In case of crack, this minimal path dynamically [2], [5], computation time, for these high- follows crack pixels. resolution images (2048x2048 pixels) is about 20 seconds This method has been test on a large database of real (Dell Precision PWS670, Xeon 3.6 GHz, 4 Go de RAM). images captured by a high-resolution system installed on In Figure 10, all defects (Figures 10 a, b, c) are detected AMAC© [2]. AMAC © is usually used to the diagnostic of (with details for alligator crack (c)). There is no false Highway road network in France. Comparative tests and positive because for images without defect (Figure 10d), tests on synthetics defects have been made to show the nothing is detected. Figure shows the efficiency of the efficiency of the method. method to perform a good detection with very low signal (crack) to noise ratio. ACKNOWLEGDMENTS French operators give a great number of real images We want to give thanks to Vectra Company to support this captured dynamically on highways of French road network, works.

11. REFERENCES [1] Schmidt B., Automated Pavement Cracking Assessment Equipment - State of the Art. 2003, World Road Association (PIARC). [2] Vectra' s Amac specification , http://www.vectra.fr. [3] Roadware, Roadware, Ontario, Canada ARAN Wise Crack specification, website www.roadware.com. [4] Relevé des dégradations de surface des chausses ,May 1997, media.lcpc.fr/ext/pdf/me/me_38-2.pdf [5] Nguyen T.S., et al., Detection of defects in road surface by a vision system, in Proc. 14th IEEE Mediterranean Electrotechnical Conference MELECON 2008, 2008. pp 847-851. [6] H. Oliveira, P.L. Correira, Automatic Road Crack Detection and Characterization, IEEE Transactions on Intelligent Transportation Systems, Vol 14 , No 1, March 2013, pp 155-168 [7] Qin Zoua, Yu Caoc, Qingquan Lib, Qingzhou Maob, Song Wangc, CrackTree: Automatic crack detection from pavement images, Pattern Recognition Letters, Vol 33, No 3, February 2012, pp 227– 238 [8] Nallamothu S. and K.C.P. Wang, Experimenting with Recognition Accelerator for Pavement Distress Identification, in Transportation Research Record. 1996. pp. 130-135. [9] Chua K.M. and L. Xu, Simple Procedure for Identifying Pavement Distresses from Video Images, in J. Transp. Engrg. 1994. pp. 412- 431. [10] Cheng, .D., et al., Novel Approach to Pavement Cracking Detection Based on Fuzzy Set Theory, in Journal of Computing in Civil Engineering. 1999. pp. 270-280. [11] Oliveira H., Correia P.L., Identifying and retrieving distress images from road pavement surveys, in Proc. 15th IEEE International Conference on Image Processing ICIP 2008. pp. 57-60. [12] Huang Y. and B. Xu, Automatic inspection of pavement cracking distress, in Journal of Electronic Imaging. 2006. [13] Chambon S., J. Dumoulin, and P. Subirats, Introduction of a wavelet transform based on 2D matched filter in a Markov Random Field for fine structure extraction: Application on road crack detection, in SPIE Conference on Image Processing: Machine Vision Applications II, San Jose, United-States. 2009. [14] Nguyen TS, Begot S, Duculty F, Avila M, Free-Form Anysotropie: A new method for crack detection on pavement surface images, 18th IEEE International Conference on Image Processing, Bruxelles, Belgium (2011), pp 1069-1072. [15] Thomas, H.C., et al., Introduction to Algorithms. 2001: McGraw- Hill Higher Education. [16] Dubois D. and H. Prade, On the use of aggregation operations in information fusion processes. Fuzzy Sets and Systems. 2004. 142,143-161. [17] Theodoridis S. and K. Koutroumbas, Pattern Recognition, Fourth Edition. 2008: Academic Press. pp 418.