Vision-Based Vehicle Tracking Using Image Alignment With
Total Page:16
File Type:pdf, Size:1020Kb
Motion Tracking on Embedded Systems - Vision-based Vehicle Tracking using Image Alignment with Symmetrical Function CHEUNG, Lap Chi A Thesis Submitted in Partial Fulfillment of the Requirements for the Degree of Master of Philosophy in Computer Science and Engineering © The Chinese University of Hong Kong August 2007 The Chinese University of Hong Kong holds the copyright of this thesis. Any person(s) intending to use a pari or whole of the materials in the thesis in a proposed publication must seek copyright release from the Dean of the Graduate School. Iti^Hj u 謹 ITY y^/j ^XLIBRARY SYSTEM^纷 Thesis/Assessment Committee Professor WONG Kin Hong (Chair) Professor MOON Yiu Sang (Thesis Supervisor) Professor JIA Jiaya Leo (Committee Member) Professor FAIRWEATHER Graeme (External Examiner) i Abstract The large number of rear end collisions due to driver inattention has been identified as a major automotive safety issue in Intelligent Transportation Systems (ITS), part of whose goals is to apply automotive technology to improve driver safely with efficiency. In this thesis, we describe a 3-phase vehicle tracking methodology with a single moving camera mounted on the driver's automobile as input for detecting rear vehicles on highways and city streets for the purpose of preventing potential rear-end collisions. In the first phase, a rear vehicle is detected by symmetrical measurement and Haar transform. A template is also created for tracking the vehicle using image alignment techniques in the second phase. The template is continuously monitored and updated to handle abrupt changes in surrounding conditions in the third phase. Different from most previous methods which simply delect vehicles in each frame without correlating the successive frames, our methodology gives precise tracking, an essential requirement for estimating the distances between the rear vehicles and the driver's car using the newly proposed formula. To implement our methodology in real-time embedded systems, we further enhance the efficiency of our algorithms with the aid of symmetrical function and simplified image alignment techniques. Preliminary testing results show that our system acquires a successful tracking rate over 97%. ii 論文摘要 由於駕駛者的低警覺性造成的車輛尾端碰撞意外已成爲智能運輸系統(Intelligent Transportation Systems)的一大議題°智能運輸系統就是透過應用自動化技術來提 高駕駛安全及效率。在本論文中,我們會描述一個三階段車輛追蹤法,以設於駕 駛者車輛上的單鏡攝像頭作爲系統輸入來偵測後逐車輛’防止尾端碰撞的發生。 在第一階段,我們使用對稱測量(Symmetrical Measurement)及哈爾轉換(Haar Transform)兩方法來偵測後逐車輛,並創造模板供第二階段的圖像拼接法(Image Alignment)作物體追蹤。其間模板會不斷受到監察,一旦遇到環境突變而影響追 蹤效能,第三階段會即時作出模板更新以保持精確的車輛追蹤。新提出的追蹤方 法著重於影像片間之交互作用’有別於現有的簡單單一影像片偵測法。透過應用 三階段車輛追縱法及最新提出的車輛相互距離計算方程式,便能對後逐車輛的距 離作出準確的運算°在對稱測量及簡化圖像拼接法(Simplified Image Alignment) 的幫助下,這實時多車輛追蹤法更能在嵌入式系統上實現,而初步的實驗追蹤率 更維持在97個巴仙以上。 iii To my dearest family and friends iv Acknowledgement I wish to thank my supervisor, Prof. Moon Yiu Sang, for his continuous guidance and supervision, for supporting me to attend academic conferences and for all his teachings in research, learning and living. Also, gives me many opportunities being the leader of projects and competition. I wish to express my gratitude to Prof. Wong Kin Hong and Prof. JIA Jiaya Leo for being members of my thesis committee. I would also like to thank Prof. G. Fairweather for being my external marker. Special thanks are due to the Robotics Institute at the Carnegie Mellon University for making their image alignment algorithm implementation publicly available. Without their contribution, this research work would not have such a good start. I gratefully acknowledge helpful discussions with Mr, Chen Jiansheng, Mr. Chan Ka Cheong, Mr. Chan Fai and Mr. Tang Xinmin for sharing their research insight and ideas with me. My sincere thanks to Wong Kwong Wa and Tse Wing Cheong for doing the initial study of the vehicle tracking system. I must thank Tony, Kim, Oscar, Simon and other technical staff for their patience with me and excellent work in maintaining a stable computing environment. I would like to thank the Chinese University of Hong Kong for providing such a beautiful campus, a resourceful library and excellent recreational facilities and Shaw College for providing such a wonderful college life and comfortable living place. Then there is the list of friends and colleagues that deserve my deepest gratitude. Many thanks to Kuen, Sunny, Siu Ming, Felix, Yu Tsz and Fung, for bringing me the V memorable joy in Intel Cup competition including working overnight at room 1024, trip lo Shanghai and Beijing and all of the follow-up events; Ming, Brittle, Kit, Brian, Cheong, Ocean, CJ, Ma Fai, Kevin, Lo Wing and Chan Fai, for creating such enjoyable and relaxing atmosphere; all of the friends in room 1003, for letting me know what is meant by efficient and hardworking; all of the hostel tutors in Student Hostel II,Shaw College, for working together to organize joyful and meaningful hostel activities. And, my heartfelt thanks to those who have helped me one way or the other, even though both you and I may not be aware of it. I wish to express my deepest gratitude to Ming, Chan Fai, Cyrus and Peng Xiang for giving me humble advice and chatting with me during trouble limes. Thanks Ming and Chan Fai, my TA partners, for sharing the workload when I am busy in research. You both are enthusiastic in leaching! My special thanks goes to Peng Xiang, who gave me motivation to finish this thesis by telling me thai he had already finished his in April. To the researchers in SPIE and MVA conferences, thanks your comments and suggestions which help me keep improving my thesis. I would like to thank the world for being such a wonderful place to live in, to appreciate and to explore. Thanks the Department of Computer Science and Engineering, the Chinese University of Hong Kong. I am glad to study here for 5 years of undergraduate and postgraduate study. And most of all, thanks to my family members, Mum, Father, Sister, Brother and Ching for giving me love, support and concern during trouble limes. vi Table of Contents 1. INTRODUCTION 1 1.1. BACKGROUND I 1.1.1. Introduction to Intelligent Vehicle 1 1.1.2. Typical Vehicle Tracking Systems for Rear-end Collision Avoidance 2 1.1.3. Passive VS Active Vehicle Tracking 3 1.1.4. Vision-hased Vehicle Tracking Systems 4 1.1.5. Characteristics of Computing Devices on Vehicles 5 1.2. MOTIVATION AND OBJECTIVES 6 1.3. MAJOR CONTRIBUTIONS 7 1.3.1. A 3-phase Vision-based Vehicle Tracking Framework 7 7.3.2. Camera-to-vehicle Distance Measurement by Single Camera 9 1.3.3. Real Time Vehicle Detection 10 1.3.4. Real Time Vehicle Tracking using Simplified Image Alignment JO 1.4. EVALUATION PLATFORM 11 1.5. THESIS ORGANIZATION 11 2. RELATED WORK 13 2.1. STEREO-BASED VEHICLE TRACKING 13 2.2. MOTION-BASED VEHICLE TRACKING 16 2.3. KNOWLEDGE-BASED VEHICLE TRACKING ] 8 2.4. COMMERCIAL SYSTEMS 19 3. 3-PHASE VISION-BASED VEHICLE TRACKING FRAMEWORK 22 3.1. INTRODUCTION TO THE 3-PHASE FRAMEWORK 22 3.2. VEHICLE DETECTION 23 3.2.1. Overview of Vehicle Detection 23 3.2.2. Locating the Vehicle Center — Symmetrical Measurement 25 3.2.3. Locating ihe Vehicle Roof and Bottom 28 3.2.4. Locating the Vehicle Sides 一 Over-complete Haar Transform 30 3.3. VEHICLE TEMPLATE TRACKING - IMAGE ALIGNMENT 37 3.3.5. Overview of Vehicle Template Tracking 57 3.3.6. Goal of Image Alignment 41 3.3.7. Allernative Image Alignment - Compositional Image Aligmnent 42 3.3.8. Efficient Image Alignment - Inverse Compositional AIgorithm 43 vii 3.4. VEHICLE TEMPLATE UPDATE 46 3.4.1. Situation of Vehicle lost 46 3.4.2. Template Fitting by Updating the posiliom of Vehicle Features 48 3.5. EXPERIMENTS AND DISCUSSIONS 49 3.5.1. Experiment Setup 49 3.5.2. Successful Tracking Percentage 50 3.6. COMPARING WITH OTHER TRACKING METHODOLOGIES 52 3.6.1. 1-phase Vision-based Vehicle Tracking 52 3.6.2. Image Correlation 54 3.6.3. Confinuously Adapiive Mean Shift 58 4. CAMERA-TO-VEHICLE DISTANCE MEASUREMENT BY SINGLE CAMERA 61 4.1. THE PRINCIPLE OF LAW OF PERSPECTIVE 61 4.2. DISTANCE MEASUREMENT BY SINGLE CAMERA 62 5. REAL TIME VEHICLE DETECTION 66 5.1. INTRODUCTION 66 5.2. TIMING ANALYSIS OF VEHICLE DETECTION 66 5.3. SYMMETRICAL MEASUREMENT OPTIMIZATION 67 5.3.1. Diminished Gradieni Image for Symmelrical Measurement 67 5.3.2. Replacing Division hy Multiplication Operations 77 5.4. OVER-COMPLETE HAAR TRANSFORM OPTIMIZATION 73 5.4.1. Char acl eristics of Over-complete Haar Transform 73 5.4.2. Pre-compntation of Haar block 74 5.5. SUMMARY 77 6. REAL TIME VEHICLE TRACKING USING SIMPLIFIED IMAGE ALIGNMENT 78 6.1. INTRODUCTION 78 6.2. TIMING ANALYSIS OF ORIGINAL IMAGE ALIGNMENT 78 6.3. SIMPLIFIED IMAGE ALIGNMENT 80 6.3.1. Reducing the Number of Parameters in AJfine Transformation 80 6.3.2. Size Rednclion of Image A ligmneni Matrixes 85 6.4. EXPERIMENTS AND DISCUSSIONS 85 6.4.1. Successful Tracking Percentage 86 6.4.2. Timing Improvement 87 7. CONCLUSIONS 89 8. BIBLIOGRAPHY 91 viii List of Tables Table 1. Specification of the evaluation system - Intel ECX platform 11 Table 2. Setting of camera mounted on the testing vehicle 50 Table 3. Information and tracking percentage of road image sequences 51 Table 4. The successful tracking percentage of 3-phase structure versus 1 -phase structure 53 Table 5: Experiment information of the adaptive correlation vehicle tracking 56 Table 6. Timing information of the vehicle detection 67 Table 7. Timing information of the vehicle detection after optimization 77 Table 8. Information of the image sequence in the timing analysis 79 Table 9. Timing information of the original vehicle template tracking per frame of one vehicle 79 Table 10. Timing information of the original vehicle tracking methodology per frame 80 Table 11. Parameters involved in each basic affine transformation 81 Table 12.