FAST TEMPLATE MATCHING FOR VISION- BASED LOCALIZATION by JASON HARPER Submitted in partial fulfillment of the requirements For the degree of Master of Science Thesis Advisor: Wyatt Newman Department of Electrical Engineering and Computer Science CASE WESTERN RESERVE UNIVERSITY May 2009 Table of Contents Table of Contents ................................................................................................................................. 1 List of Figures ....................................................................................................................................... 4 List of Tables ........................................................................................................................................ 7 1 Introduction .................................................................................................................................. 9 1.1 Localization .......................................................................................................................... 9 1.1.1 Global Visual Localization ....................................................................................... 12 1.1.2 Local Visual Localization ......................................................................................... 13 1.2 Template Matching ............................................................................................................ 14 1.2.1 Edge Matching ........................................................................................................... 14 1.2.2 Pattern-Based Template Matching ......................................................................... 14 1.2.3 Grid Template Matching .......................................................................................... 15 1.3 Combining Vision Data with a State Observer ............................................................. 16 1.4 Contributions ..................................................................................................................... 17 2 Problem Definition .................................................................................................................... 18 3 Platform Setup ............................................................................................................................ 21 3.1 The Robot Platform .......................................................................................................... 21 3.2 The Chosen Camera .......................................................................................................... 22 3.3 Camera Calibration ............................................................................................................ 23 3.4 Timing Calibration ............................................................................................................. 28 1 4 Fast Template Matching ........................................................................................................... 32 4.1 Overview ............................................................................................................................. 32 4.2 Camera Data Acquisition ................................................................................................. 33 4.3 Finding Tiles ....................................................................................................................... 33 4.4 Plan View to Hough Space Transforms ......................................................................... 38 4.5 Transforming Hough Space into Modulo Space .......................................................... 41 4.6 Normalizing the Transform ............................................................................................. 46 4.7 Pose Estimation from Mod-Space for Localization ..................................................... 51 4.8 Making Template Matching Fast ..................................................................................... 55 5 Integration with a Localization System ................................................................................... 58 5.1 Overview ............................................................................................................................. 58 5.2 Variance from Helper Functions ..................................................................................... 58 5.3 Credibility Functions ......................................................................................................... 60 5.4 Setting Credibility Thresholds ......................................................................................... 64 6 Results .......................................................................................................................................... 74 6.1 System Results .................................................................................................................... 74 6.2 Algorithm Runtime ........................................................................................................... 79 6.3 Robustness with Respect to Noise .................................................................................. 81 6.4 Resistance to Distracter Lines ......................................................................................... 82 6.5 Integration with Localization System ............................................................................. 84 2 7 Conclusions ................................................................................................................................. 89 8 Future Work ............................................................................................................................... 91 Appendix A ......................................................................................................................................... 93 Appendix B ......................................................................................................................................... 94 B.1 Canny Edge Detector ........................................................................................................ 94 B.2 Laplace Edge Detector ..................................................................................................... 94 B.3 Weighted Local Variance .................................................................................................. 94 Bibliography ........................................................................................................................................ 96 3 List of Figures Figure 1: ALEN, the robot used for algorithm testing. ................................................................ 21 Figure 2: Firefly MV from Point Gray Research (Firefly MV, 2008). ........................................ 22 Figure 3: Camera view of aligned grid points. ................................................................................ 25 Figure 4: Standard deviation of the i-components manually chosen for Cartesian space subset. ................................................................................................................................................... 26 Figure 5: Standard deviation of the j-component of the Cartesian space subset. ..................... 26 Figure 6: Cartesian space image transformed from camera space using the calibration lookup table. ..................................................................................................................................................... 28 Figure 7: Image of circuit used to provide a timestamp for timing calibration. ....................... 29 Figure 8: Example of binary timestamp 0b00100101101011. ..................................................... 30 Figure 9: Timing delay of camera to PSO. ..................................................................................... 31 Figure 10: Pipeline architecture diagram. ........................................................................................ 33 Figure 11: Edge detection of hallway image using the Sobel operator....................................... 34 Figure 12: Edge detection of hallway image using the Canny algorithm. .................................. 34 Figure 13: Edge detection of hallway image using the Laplace convolution kernel. ................ 35 Figure 14: Edge detection of hallway image using the weighted local variance method. ........ 35 Figure 15: Hough space transformed from output of Sobel Figure 13...................................... 40 Figure 16: Example of distracting lines. .......................................................................................... 42 Figure 17: Illustration of equations 8 and 9.................................................................................... 43 Figure 18: Sample sections of mod space. ...................................................................................... 46 Figure 19: Mod space with distracter lines. .................................................................................... 47 Figure 20: Mod space after normalization. ..................................................................................... 50 4 Figure 21: Process of projecting mod space and creating a combined confidence function. 52 Figure 22: Depiction of creating the x-confidence function. ....................................................... 52 Figure 23: Depiction of creating the y-confidence function. ....................................................... 53 Figure 24: