University of Central Florida STARS Electronic Theses and Dissertations, 2004-2019 2006 Automated Visual Database Creation For A Ground Vehicle Simulator Pedro Claudio University of Central Florida Part of the Computer Engineering Commons Find similar works at: https://stars.library.ucf.edu/etd University of Central Florida Libraries http://library.ucf.edu This Doctoral Dissertation (Open Access) is brought to you for free and open access by STARS. It has been accepted for inclusion in Electronic Theses and Dissertations, 2004-2019 by an authorized administrator of STARS. For more information, please contact [email protected]. STARS Citation Claudio, Pedro, "Automated Visual Database Creation For A Ground Vehicle Simulator" (2006). Electronic Theses and Dissertations, 2004-2019. 998. https://stars.library.ucf.edu/etd/998 AUTOMATED VISUAL DATABASE CREATION FOR A GROUND VEHICLE SIMULATOR By PEDRO J. CLAUDIO B.S.Ch.E., University of Florida 1981 M.S.E., University of Central Florida 1986 A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in the School of Electrical Engineering and Computer Science in the College of Engineering and Computer Science at the University of Central Florida Orlando, Florida Summer Term 2006 Major Professor: Christian Bauer ©2006 Pedro J. Claudio ii ABSTRACT This research focuses on extracting road models from stereo video sequences taken from a moving vehicle. The proposed method combines color histogram based segmentation, active contours (snakes) and morphological processing to extract road boundary coordinates for conversion into Matlab™ or Multigen OpenFlight™ compatible polygonal representations. Color segmentation uses an initial truth frame to develop a color probability density function (PDF) of the road versus the terrain. Subsequent frames are segmented using a Maximum Apostiori Probability (MAP) criteria and the resulting templates are used to update the PDFs. Color segmentation worked well where there was minimal shadowing and occlusion by other cars. A snake algorithm was used to find the road edges which were converted to 3D coordinates using stereo disparity and vehicle position information. The resulting 3D road models were accurate to within 1 meter. iii Dedicated to Alexander, Katherine, Christina and Nathaniel Claudio iv ACKNOWLEDGEMENTS I would like to express my gratitude to my advisor, Dr. Christian Bauer, for his support, patience, and encouragement throughout the long years of my graduate studies. I also thank my son, Alexander and his friend Anthony Justiniano for assisting me in constructing the hardware and in data collection. I would also like to thank my wife Vicki for her editorial assistance in assembling this dissertation and for her love and support. Special thanks also go to my fellow students who supported me in my endeavors, Dahai Guo, Bobby Harris, Lijian Xu, and Vaibhav Joshi. I would also like to express my appreciation to the Link Foundation for awarding me an Advanced Simulation and Training Fellowship in 2000-2001. This work also received financial support from Lockheed Martin with a Synthetic Environment Learning Laboratory (SELL) equipment grant. Thanks to Dr. Alberto Broggi from the Artificial Vision and Intelligent Systems Lab (VisLab), University of Parma, Italy for permission to use images from the GOLD website. Thanks to Drs. Karl Kluge, Sridhar Lakshmanan and Christopher Kreucher for permission to reprint images from "Tracking Lane and Pavement Edges Using Deformable Templates" in Proceedings of SPIE Vol. 3364 "Enhanced and Synthetic Vision 1998" v TABLE OF CONTENTS LIST OF FIGURES .............................................................................................viii LIST OF TABLES...............................................................................................xiii LIST OF ABBREVIATIONS.............................................................................. xiv 1 INTRODUCTION .......................................................................................... 1 1.1 The UCF Driver Simulator ..................................................................... 1 1.2 Road Model Generation for the UCF Driving Simulator ....................... 2 1.3 Other Techniques for Generating Road Models ..................................... 2 1.4 Statement of the Problem........................................................................ 3 1.5 Overview................................................................................................. 3 2 LITERATURE REVIEW............................................................................... 5 2.1 Overview/History.................................................................................... 5 2.1.1 ALV .................................................................................................... 5 2.1.2 Demo I/II/III........................................................................................ 7 2.1.3 Navlab/CMU....................................................................................... 8 2.1.4 Vamors/Vamp..................................................................................... 8 2.1.5 ARGO/MOBLAB............................................................................. 10 2.2 Road Modellers..................................................................................... 11 2.2.1 Unstructured Roads........................................................................... 11 2.2.2 Structured Roads............................................................................... 15 2.2.3 Road Followers................................................................................. 19 2.3 Conclusion ............................................................................................ 21 3 APPROACH ................................................................................................. 22 3.1 System Analysis.................................................................................... 22 3.1.1 System Description ........................................................................... 22 3.1.2 Throughput Requirements ................................................................ 28 3.1.3 Error Analysis................................................................................... 29 3.2 Hardware Description.......................................................................... 61 3.2.1 Mobile Platform................................................................................ 62 3.2.2 Cameras............................................................................................. 65 3.2.3 Sensor Capture Operation................................................................. 65 3.3 Software Description............................................................................ 72 vi 3.3.1 Sensor Capture.................................................................................. 72 3.3.2 Instrumentation Capture.................................................................... 75 3.4 Proposed Method.................................................................................. 83 3.4.1 Distortion Correction & Rectification .............................................. 83 3.4.2 Initialization ...................................................................................... 92 3.4.3 Color Histogram Based Segmentation.............................................. 93 3.4.4 Snake Algorithm for finding Road Boundaries .............................. 100 3.4.5 Matching Boundary Points Between Stereo Pairs .......................... 103 3.4.6 Extraction of 3D Coordinates ......................................................... 105 3.4.7 Polygonal Model Extraction........................................................... 112 3.4.8 Summary of Proposed Method ....................................................... 117 4 RESULTS ................................................................................................... 119 4.1 Segmentation Results.......................................................................... 120 4.2 Road Boundary Extraction Results..................................................... 123 4.3 3D Model Extraction Results.............................................................. 134 5 CONCLUSION & SUGGESTIONS FOR FUTURE WORK.................... 137 5.1 Conclusion .......................................................................................... 137 5.2 Suggestions for Future Work.............................................................. 138 APPENDIX: PERMISSIONS............................................................................. 140 REFERENCES ................................................................................................... 146 vii LIST OF FIGURES Figure 1 – UCF Driving Simulator ..................................................................................... 1 Figure 2 – LOIS Results on a Road with Shadows........................................................... 17 Figure 3 – Inverse Perspective Mapping for Road Localization ...................................... 18 Figure 4 – ALVINN.......................................................................................................... 19 Figure 5 – System Block Diagram.................................................................................... 23 Figure 6 – Video Frame Format........................................................................................ 24 Figure 7 – Camera