sensors Review Vision-Based Traffic Sign Detection and Recognition Systems: Current Trends and Challenges Safat B. Wali 1 , Majid A. Abdullah 2, Mahammad A. Hannan 2,* , Aini Hussain 1 , Salina A. Samad 1, Pin J. Ker 2 and Muhamad Bin Mansor 2 1 Centre for Integrated Systems Engineering and Advanced Technologies, Universiti Kebangsaan Malaysia, Bangi 43600, Malaysia; [email protected] (S.B.W.); [email protected] (A.H.); [email protected] (S.A.S.) 2 Institute of Power Engineering, Universiti Tenaga Nasional, Kajang 43000, Malaysia; [email protected] (M.A.A.); [email protected] (P.J.K.); [email protected] (M.B.M.) * Correspondence: [email protected] Received: 2 April 2019; Accepted: 26 April 2019; Published: 6 May 2019 Abstract: The automatic traffic sign detection and recognition (TSDR) system is very important research in the development of advanced driver assistance systems (ADAS). Investigations on vision-based TSDR have received substantial interest in the research community, which is mainly motivated by three factors, which are detection, tracking and classification. During the last decade, a substantial number of techniques have been reported for TSDR. This paper provides a comprehensive survey on traffic sign detection, tracking and classification. The details of algorithms, methods and their specifications on detection, tracking and classification are investigated and summarized in the tables along with the corresponding key references. A comparative study on each section has been provided to evaluate the TSDR data, performance metrics and their availability. Current issues and challenges of the existing technologies are illustrated with brief suggestions and a discussion on the progress of driver assistance system research in the future. This review will hopefully lead to increasing efforts towards the development of future vision-based TSDR system. Keywords: Traffic sign detection and tracking (TSDR); advanced driver assistance system (ADAS); computer vision 1. Introduction In all countries of the world, the important information about the road limitation and condition is presented to drivers as visual signals, such as traffic signs and traffic lanes. Traffic signs are an important part of road infrastructure to provide information about the current state of the road, restrictions, prohibitions, warnings, and other helpful information for navigation [1,2]. This information is encoded in the traffic signs visual traits: Shape, color and pictogram [1]. Disregarding or failing to notice these traffic signs may directly or indirectly contribute to a traffic accident. However, in adverse traffic conditions, the driver may accidentally or deliberately not notice traffic signs [3]. In these circumstances, if there is an automatic detection and recognition system for traffic signs, it can compensate for a driver’s possible inattention, decreasing a driver’s tiredness by helping him follow the traffic sign, and thus, making driving safer and easier. Traffic sign detection and recognition (TSDR) is an important application in the more recent technology referred to as advanced driver assistance systems (ADAS) [4], which is designed to provide drivers with vital information that would be difficult or impossible to come by through any other means [5]. The TSDR system has received an increasing interest in recent years due to its potential use in various applications. Some of these applications have been well defined and summarized in [6] as checking the presence and condition of the signs Sensors 2019, 19, 2093; doi:10.3390/s19092093 www.mdpi.com/journal/sensors Sensors 2019, 19, 2093 2 of 28 Sensors 2019, 19, x FOR PEER REVIEW 2 of 30 Sensors 2019, 19, x FOR PEER REVIEW 2 of 30 re-localization of autonomous vehicles; as well as its use in the application relevant to this research, on highways, sign inventory in towns and cities, re-localization of autonomous vehicles; as well as as a driver support system. However, a number of challenges remain for a successful TSDR systems; itsre-localization use in the application of autonomous relevant vehicles; to this research,as well as as its a use driver in the support application system. relevant However, to this a number research, as the performance of these systems is greatly affected by the surrounding conditions that affect ofas challenges a driver support remain system. for a successful However, TSDR a number systems; of challenges as the performance remain for of a thesesuccessful systems TSDR is greatlysystems; road signs visibility [4]. Circumstances that affect road signs visibility are either temporal because of affasected the byperformance the surrounding of these conditions systems thatis greatly affect roadaffected signs by visibility the surrounding [4]. Circumstances conditions that that a ffaffectect illumination factors and bad weather conditions or permanent because of vandalism and bad roadroad signs signs visibility visibility are [4]. either Circumstan temporalces because that affect of illumination road signs visibili factorsty and are bad either weather temporal conditions because or of postage of signs [7]. Figure 1 shows an example of some non-ideal invariant traffic signs. These permanentillumination because factors of vandalismand bad weather and bad conditions postage of signsor permanent [7]. Figure because1 shows of an vandalism example of and some bad non-identicalpostage of signstraffic [7]. signs Figure cause 1 difficulties shows an forexample TSDR. of some non-ideal invariant traffic signs. These non-ideal invariant traffic signs. These non-identical traffic signs cause difficulties for TSDR. non-identical traffic signs cause difficulties for TSDR. (a) (b) (c) (d) Figure 1.(a )Non-identical traffic signs:(b) (a) Partially occluded traffic(c) sign, (b) faded traffic sign,(d) (c) damaged traffic sign, (d) multiple traffic signs appearing at a time. FigureFigure 1. Non-identical1. Non-identical traffi trafficc signs: signs: (a) Partially (a) Partially occluded occluded traffic sign, traffic (b) sign, faded ( trab) ffifadedc sign, traffic (c) damaged sign, (c) tradamagedffic sign, (trafficd) multiple sign, ( trad) ffimultiplec signs appearingtraffic signs at appearing a time. at a time. This paper provides a comprehensive survey on traffic sign detection, tracking and classification.ThisThis paper paper The provides details provides aof comprehensive algorithms, a comprehensive methods survey onansurveyd tra theirffi con signspecifications traffic detection, sign on tracking detection, and classification.trackingtracking and and Theclassificationclassification. details of algorithms,are The investigated details methods of algorithms, and and summarized their methods specifications in an thed their tables on detection,specifications along trackingwith onthe detection,and corresponding classification tracking key are and investigatedreferences.classification A and comparative are summarized investigated study in the onand tables each summarized alongsection with ha sin the been the corresponding providedtables along to key evaluate with references. the the corresponding TSDR A comparative methods, key studyperformancereferences. on each Ametrics section comparative hasand been studytheir provided onavailability. each to section evaluate Curr haents the been TSDRissues provided methods,and tochallenges evaluate performance theof TSDRthe metrics existingmethods, and theirtechnologiesperformance availability. are metrics Currentillustrated and issues withtheir and brief challengesavailability. suggestion of theCurrs and existingent a issuesdiscussion technologies and onchallenges arethe illustrated progress of the withof driverexisting brief suggestionsassistancetechnologies system and are a discussionresearch illustrated in on the with the future. progress brief The suggestion of rest driver of this assistances and paper a systemdiscussionis organized research on as the infollows: the progress future. In Section Theof driver rest 2, ofanassistance thisoverview paper system ison organized traffic research signs as follows: in and the recentfuture. In Section trends The2 ,rest anof overviewofthe this research paper on tra isin organizedffi thisc signs field and asis follows: recentpresented. trends In SectionThis of the is 2, researchfollowedan overview in by this providing fieldon traffic is presented. a signsbrief andreview This recent is followedon trendsthe available by of providing the researchtraffic a briefsign in reviewthisdatabases field on isthein presented. availableSection 3. traThis Theffic is signmethodsfollowed databases of bydetection, inproviding Section tracking,3 .a Thebrief methods andreview classificatio ofon detection, the availablen are tracking, categorized, traffic and sign classificationreviewed, databases and arein comparedSection categorized, 3. inThe reviewed,Sectionmethods 4. Section andof detection, compared 5 revises tracking, in current Section issuesand4. Sectionclassificatio and challenges5 revisesn are current facingcategorized, issuesthe researchers andreviewed, challenges in andTSDR. facingcompared Section the 5in researcherssummarizesSection 4. in Sectionthe TSDR. paper, 5 Section revises draws5 currentsummarizesthe conclusion issues the and and paper, challenges suggestions. draws thefacing conclusion the researchers and suggestions. in TSDR. Section 5 summarizes the paper, draws the conclusion and suggestions. 2. Traffic Traffic SignsSigns andand ResearchResearch TrendsTrends 2. TrafficAiming Signs atat standardizing standardizing and Research tra ffiTrendstrafficc