ROADSIGN DETECTION AND RECOGNITION (RSDR) LIM CHIN HUEY A report submitted in partial fulfilment of the requirements for the award of the degree of Bachelor of Computer Science (Software Engineering) Faculty of Computer Systems & Software Engineering University Malaysia Pahang (UMP) MAY 2010 V ABSTRACT Road-sign Detection and Recognition via Video (RSDR) is a system that able to detect and recognize a road-sign on a motion video. This is due to the rapid changes of the technologies, more and more technologies recognize are able to provide high performance for people to carry their daily task. In RSDR, it is a system help the driver to recognize the shape of road-sign while the driving task is carried. Before this, most of the processing in digital image is based on still image. The limitation on still images is unable to perform effectively then motion video. Motion video can be processed directly in real-time by capturing the data to be examine, while for the still image the captured data is not in real-time thus the information cannot be delivered in short time. This RSDR will be developed according to methodology of waterfall models and it weakness is overcome by applied the incremental and iterative development process. Because with the well defined of the requirement on RSDR, the waterfall model is chooses to produce a high quality system. In case, there is some of the uncountable event occur required changes on development process the support of incremental and iterative development process can be help to overcome this problem arise. The method of template matching is used to recognize the road-sign. Before the recognition process, the detection of the road-sign on motion video is being done in Matlab by applied the Video and Image Processing techniques control by simulation process to detect an object from motion video. Only the successful of the object extracted from video frame will proceed to template matching on recognition process based on the template in the database. In conclusion, RSDR is potentially being a smart system in future for user to gather information from real-time process such as the driving task. vi ABSTRAK Road-sign Detection and Recognition via Video (RSDR) adalah satu sistem yang akan dapat mengesan dan mengenal simbol-simbol di jalan raya. Dengan teknologi yang sering berubah semakin banyak technology diperkenalkan untuk memberi kemudahan kepada manusia menjalankan tugas harian mereka. Sebelum in kebanyakan aplikasi dijalankan ke atas gambar digital yang statik, dengan mi ia tidak dapat beroperasi dengan lebih pantas daripada video. Video dapat menangkap gambar dalam masa nyata. Oleh itu, maklumat yang diminta dapat sampai dalam masa yang laju dalam keadaan masa nyata. Metodologi yang digunakan ialah "Waterfall" tetapi dengan kelemahanya, satu cara yang dinamakan perkembangan dan perulangan ("incremental and iterative development process") digunakan untuk mengatasi kelemahan mi. "Waterfall" dipilih adalah disebabkan dari segi maklumat, permintaan dan pencapaian bagi sistem telah jelas disenaraikan, ia tidak terlibat dalam sebarang perubahan yang mermelukan aplikasi rekaan mi balik semula ke awal aplikasi rekaan. Dalam aplikasi ini, teknik "template Matching" digunakan untuk mencari kesamaan dan keunikan bagi dua (2) gambar untuk memperkenalkan simbol-simbol di jalan raya. Sebelum melalukan aplikasi "template matching", simbol yang digunakan mestilah dapat di dalam frame yang beroperasi dengan mengunakan Matlab untuk jalankan aplikasi dalaman. Aplikasi dalaman Matlab terlibat "Video and Image Processing" dan simbol yang dapat adalah berdasarkan simulasi yang dijalankan ke setiap frame yang ditangkap. Hanya simbol yang diperkenalkan diguna dalam aplikasi "template matching". RSDR boleh katakan sebagai sam aplikasi yang dibawa kepada masa depan manusia dengan mendapatkan makiumat dalam masa nyata. vii TABLE OF CONTENTS CHAPTER TITLE PAGE DECLARATION 11 DEDICATION 111 ACKNOWLEDGEMENT iv ABSTRACT V ABSTRAK Vi TABLE OF CONTENTS vii LIST OF TABLES x LIST OF FIGURE xi LIST OF ABBREVIATION xiii LIST OF APPENDICES xiv 1 INTRODUCTION 1 1.1 Introduction 1 1.2 Problems Statement 2 1.3 Objective 3 1.4 Scope 3 2 LITERATURE REVIEW 4 2.1 Introduction 4 2.2 Road-Sign 5 2.2.1 International Road-Sign 5 2.2.2 Road-Sign Standard Used in Malaysia 6 viii CHAPTER TITLE PAGE 2.3 Motion Video and Still Image 8 2.4 Related System 9 2.4.1 Classified Background Learning 9 2.4.1.1 Robustness and Performance on Video Stream 9 2.4.1.2 Identifying Moving Object 10 2.4.1.3 Correlation Technique 11 2.4.1.4 Comment Among Techniques 11 2.4.2 Detection, Categorization and Recognition of Road- 12 Sign for Autonomous Navigation 2.4.2.1 Bayes Classification 12 2.4.2.2 Scale Invariant Feature Transform (SIFT) 13 2.4.2.3 Comment Among the Technique 13 2.5 Techniques and Method Tends to Apply in RSDR 14 2.5.1 Template Matching of Shape Detection Method 14 2.5.2 Summary for The Techniques and Method Used 15 3 METHODOLOGY 19 3 3.1 Introduction 19 3.2 Software Process 20 3.3 Software System Planning 21 3.4 Software System Requirements 22 3.5 Software System Analysis and Design 23 3.5.1 Software System Analysis 24 3.5.2 Software System Design 26 3.5:2.1 Road-Sign Template 28 3.5.2.2 Detection 29 3.5.2.3 Tracking and Recognition 29 3.5.2.4 GUI Design for RSDR 30 lx CHAPTER TITLE PAGE 3.6 Software System Implementation 30 3.7 Software System Verification 38 3.7.1 Structure and Design Test 38 3.7.2 Coding Test 38 3.7.3 Full Prototype 38 3.7.4 Plan of Software Testing on System Component 39 3.8 Software System Maintenance 40 3.9 Software and Hardware Specification 41 4 IMPLEMENTATION 42 4.1 Introduction 42 4.2 Interface and Code Review 43 4.3 Database 58 4.3.1 Database for Image Template 58 4.3.2 Database for the Video 60 S RESULT ANALYSIS AND DISCUSSION 61 5.1 Introduction 61 5.2 Result and Discussion 61 5.3 Constraints 65 5.4 Future Research 66 6 CONCLUSION 67 REFERENCES 70 Appendix A Gantt Chart 74 Appendix B Template Stored in Database 76 Appendices C Result in Video vl.AVI 78 Appendices D Result in Video v2.AVI 96 x LIST OF TABLES TABLES NO TITLE PAGE• 3.1 Sample Testing on System Component in RSDR 39 3.2 Software and Hardware Specification 41 5.1 Result of Detection and Recognition 63 xi LIST OF FIGURES FIGURE NO TITLE PAGE 2.1 Road-Sign in Malaysia 7 2.2 Basic Process in RSDR Template Matching of Shape 16 Detections 2.3 Classification of Shapes for Road-Sign Detection and 17 Recognition 2.4 Basic Flow for RSDR 18 3.1 Methodology of Waterfall Model 20 3.2 Template Matching Process With Correlation Value Defined 25 3.3 Overview of Operation Phase on Storyboard 27 3.4 Simple Design for GUI on RSDR 30 3.5 Code for Accessing A Video Source 32 3.6 Code For Loading the Image from Database 33 3.7 Code for Drawing function in Motion Video 34 3.8 Code for Drawing Circle, Line and Box 35 3.9 Code for Converting Image To Gray-Scale 36 3.10 Code for Drawing Frame Per Second on Motion Video 37 4.1 Executable Window 56 4.2 Error Handling on No-Loading Video 57 xli LIST OF FIGURES FIGURE NO TITLE PAGE 4.3 End of Playing on Motion VideoDetections 57 4.4 Image of Template to be Match in System 58 4.5 Video Stored in Database 60 5.1 100% Accuracy of Template in Database 63 5.2 High Accuracy Detected Template in Video v4.AVI 64 5.3 High Accuracy Detected Template in Video vi .AVI 64 xlii LIST OF ABBREVIATION RSDR - Road-Sign Detection and Recognition Via Video JPJ - Jabatan Pengangkutan Jalan SIFT - Scale Invariant Feature Transform RGB - Red, Green and Blue CPU - Central Processing Unit 3D - Three- dimensional GUI - Graphic User Interface RAM - Random Access Memory xlv LIST OF APPENDICES APPENDICES TITLE PAGE A Gantt Chart 76 78 B Template Stored in Database 80 C Result In Video vi .AVI 98 D Result In Video v2.AVI CHAPTER 1 INTRODUCTION 1.1 Introduction A Road Sign Recognition System is a computerized system that detects and recognizes road sign from video. The road sign on the roadside is the source of information. It gives information such as right way, direction and warning of some roads which might have risky factors so that a driver can drive in more effectively. All road signs are created according to the international standard which helps prevent barrier between the different languages and symbol. For outsiders or unfamiliar drivers who depend much on the road sign, it acts as a driving assistance. Besides, it also reduces accident cases that probably occur and confusion that arises among new drivers. Most recognition systems apply image processing technique to detect and recognize road-sign. Image processing uses any form of signal processing for which the input is an image, such as photographs or frames of video. The output of image processing can be either an image or a set of characteristics or parameters related to the image. With the image-processing techniques involve, it treating the image as a two- 2 dimensional signal and applying standard signal-processing techniques. Image processing usually refers to digital image processing, but optical and analog image processing are also possible to be implement. Video is a technology that electronically captured, recorded, processed, stored, and transmitted scenes in motion. In this system the video is used for two (2) purposes; to improve the visual appearance of image to human viewer and prepare the images for measurement of the features and structures present.