Traffic and Road Sign Recognition
Total Page:16
File Type:pdf, Size:1020Kb
Traffic and Road Sign Recognition Hasan Fleyeh This thesis is submitted in fulfilment of the requirements of Napier University for the degree of Doctor of Philosophy July 2008 Abstract This thesis presents a system to recognise and classify road and traffic signs for the purpose of developing an inventory of them which could assist the highway engineers’ tasks of updating and maintaining them. It uses images taken by a camera from a moving vehicle. The system is based on three major stages: colour segmentation, recognition, and classification. Four colour segmentation algorithms are developed and tested. They are a shadow and highlight invariant, a dynamic threshold, a modification of de la Escalera’s algorithm and a Fuzzy colour segmentation algorithm. All algorithms are tested using hundreds of images and the shadow-highlight invariant algorithm is eventually chosen as the best performer. This is because it is immune to shadows and highlights. It is also robust as it was tested in different lighting conditions, weather conditions, and times of the day. Approximately 97% successful segmentation rate was achieved using this algorithm. Recognition of traffic signs is carried out using a fuzzy shape recogniser. Based on four shape measures - the rectangularity, triangularity, ellipticity, and octagonality, fuzzy rules were developed to determine the shape of the sign. Among these shape measures octangonality has been introduced in this research. The final decision of the recogniser is based on the combination of both the colour and shape of the sign. The recogniser was tested in a variety of testing conditions giving an overall performance of approximately 88%. Classification was undertaken using a Support Vector Machine (SVM) classifier. The classification is carried out in two stages: rim’s shape classification followed by the classification of interior of the sign. The classifier was trained and tested using binary images in addition to five different types of moments which are Geometric moments, Zernike moments, Legendre moments, Orthogonal Fourier-Mellin Moments, and Binary Haar features. The performance of the SVM was tested using different features, kernels, SVM types, SVM parameters, and moment’s orders. The average classification rate achieved is about 97%. Binary images show the best testing results followed by Legendre moments. Linear kernel gives the best testing results followed by RBF. C-SVM shows very good performance, but ν-SVM gives better results in some case. II Acknowledgements First and foremost, I would like to thank my supervisors Professor Howard Kirby and Dr. Chuen Wan. Their suggestions, guidance, encouragement, and support helped me greatly with this thesis. I would like to gratefully acknowledge the support of Dalarna University, Sweden which funded and financially supported this work from the very beginning. I must especially mention Professor Mark Dougherty, Dean of Research, Dalarna University, my friend and supporter who suggested the cooperation between Dalarna University and Napier University, and who supported me from my first trip to Edinburgh to the present day. For his valuable time which is given to me freely and for the discussions and criticisms which make this work what it is today, I would like to say thank you. My thanks go to my friends Yella Siril and Pascal Rebreyend at Dalarna University for the useful discussions, help and support. I would like to recognise the help and support of Dr. Ernst Nordström, Computer Engineering Department, Dalarna University and Prof. Björn Sohlberg, Electrical Engineering Department, Dalarna University for reviewing my papers, and for the valuable discussions and encouragement. My acknowledgements go to Mrs. Sarah Berglind, English Department, Dalarna University and Mr. Jalal Alrazzaz who proof-read this thesis. Without their suggestions the thesis would not be what it is today. Most of all, I would like to express my great thanks to the person who is closest to my heart, my wife Jinan Hussein who has shared the past twenty years of my life both in its sweet and sad days; who helped me during this thesis by collecting the traffic sign images; and, by taking on more domestic responsibilities to provide a quiet environment for my research. Without her love, support, and encouragement, this thesis would not have been possible. My thanks go to my daughter Riam and my son Sadeem for their continual love and support. III To Jinan My whole life… and more. IV TABLE OF CONTENTS LIST OF FIGURES ......................................................................................................... X LIST OF TABLES ....................................................................................................... XIV 1. Introduction............................................................................................................... 1 1.1 Background............................................................................................................... 1 1.2 Complexity of the Recognition Task ........................................................................ 3 1.3 Aims and Objectives of the Research ....................................................................... 4 1.4 Outline of the Thesis................................................................................................. 4 2. Traffic Signs .............................................................................................................. 7 2.1 Properties of Road and Traffic Signs........................................................................ 7 2.2 Swedish Road and Traffic Signs............................................................................... 8 2.3 Summary................................................................................................................. 11 3. Problem Formulation ............................................................................................. 14 3.1 What is Road Sign Recognition? ............................................................................ 15 3.2 Road Sign Recognition Applications...................................................................... 16 3.3 Potential Difficulties ............................................................................................... 17 3.4 Summary................................................................................................................. 24 4. Technical Overview ................................................................................................ 25 4.1 Literature Review.................................................................................................... 25 4.1.1 Colour-Based Detection of Traffic Signs ........................................................ 26 4.1.2 Shape-Based Detection of Traffic Signs.......................................................... 27 4.1.3 Colour-Shape-Based Detection of Road Signs................................................ 30 4.1.4 Recognition and Classification ........................................................................ 34 4.1.5 OCR and Pictograms Recognition................................................................... 39 4.1.6 Analysis of the Literature Review ................................................................... 40 4.2 Colour ..................................................................................................................... 47 4.2.1 Variation of Colour in Outdoor Images........................................................... 49 4.2.2 Hue Invariance................................................................................................. 51 4.2.3 Colour Constancy............................................................................................. 52 V 4.3 Moments and Invariants.......................................................................................... 53 4.3.1 Zernike Moments............................................................................................. 53 4.3.2 Legendre Moments .......................................................................................... 57 4.3.3 Orthogonal Fourier-Mellin Moments .............................................................. 59 4.3.4 Binary Haar Features ....................................................................................... 60 4.4 Support Vector Machines ....................................................................................... 62 4.4.1 Linear Classification with Maximal Margin Classifier ................................... 63 4.4.2 Non-Linear Classification................................................................................ 69 4.4.3 Learning in Feature Space ............................................................................... 70 4.4.4 Implicit Mapping to Feature Space.................................................................. 71 4.4.5 Kernels ............................................................................................................. 72 4.4.6 Properties of Kernels........................................................................................ 73 4.4.7 Examples of Kernels........................................................................................ 73 4.4.8 Soft Margin Classifier...................................................................................... 74 4.4.9 Multi-class Classifier ....................................................................................... 75 4.4.10 Types of SVM...............................................................................................