Mobile Robot Tank with GPU Assistance By Allen Chih Lun Pan A thesis submitted to the faculty of graduate studies Lakehead University In partial fulfillment of the requirements for the degree of Master of Science in Electrical Computer Engineering Department of Eletrical Engineering Lakehead University October 25 2013 Copyright © Allen Chih-Lun Pan 2013 Acknowledgements I am very grateful to my supervisor Dr. Xiao Ping Liu, who has the patience that allows me to complete my Master study and allows me to explore different solutions to a problem. He is a great professor with great vision on robotics, but most importantly I am grateful for him to take me under his wing. His trust and support encourages me to complete my thesis and venture into field in search of a solution to any problem. I sincerely thankful for Dr. Xiao Ping Liu’s time and effort for the past 2 years for the completion of my thesis. I would like to express my gratitude to my co-supervisor Dr. Kefu Liu for his knowledge input and to ensure quality of my mobile robot tank design. This entire robot will not exist without the countless hours and assistance from Mr. Kailash Bhatia, whose expertise and knowledge in fabrication and manufacturing are most critical to the existence of this mobile tank robot. I would also like to thank Lakehead University and stuffs for all this years of education and support. Without them I would never have such wonderful life changing opportunity. I delicate my thesis to my loving wife “Savage Monkey” (Izabela), who’s kid-savageness restored my confidence and helped me discover my capability and passion of creation. Without her I would have never consider doing Master degree. To my daughter “I Help!” (Zahra) whose personality is embedded into my mobile robot tank, which explains the robot’s complexity and difficulty to follow directions and also its stubbornness. With my daughter’s “volunteer” enlisted help, I modeled this mobile robot tank’s logic after her To my son “Explorer 2.0” (Zephyr), this unstoppable little boy, inspires the more ruggedness build structure and curiosity of exploration of the unknown territories. My son’s navigation and mapping capability is featured in the V3 version of my mobile robot tank. To my parents, I am extremely grateful for all the support and encouragement, their love and care is the only reason I am where I am. Whenever I need technical support and programming advice, my father is always there for me, whenever I misses food from back home, my mother will ship those food to me, and for my sister who always provided me opportunity for fixing extra technical issues and difficulties. 2 | P a g e Abstract Robotic research has been costly and tremendously time consuming; this is due to the cost of sensors, motors, computational unit, physical construction, and fabrication. There are also a lot of man hours clocked in for algorithm design, software development, debugging and optimizing. Robotic vision input usually consists of 2 or more color cameras to construct a 3D virtual space. Additional cameras can also be added to enrich the detailed virtual 3D environment, however, the computational complexity increases when more cameras are added. This is due to not only the additional processing power that is required running the software but also the complexity of stitching multiple cameras together to form a sensor. Another method of creating the 3D virtual space is the utilization of range finder sensors. These types of sensors are usually relatively expensive and still require complex algorithms for calibration and correlation to real life distances. Sensing of a robot position can be facilitated by the addition of accelerometers and gyroscope sensors. A significant robotic design is robot interaction. One type of interaction is through verbal exchange. Such interaction requires an audio input receiver and transmitter on the robot. In order to achieve acceptable recognitions, different types of audio receivers may be implemented and many receivers are required to be deployed. Depending on the environment, noise cancellation hardware and software may be needed to enhance the verbal interaction performance. In this thesis different audio sensors are evaluated and implemented with Microsoft Speech Platform. Any robotic research requires a proper control algorithm and logic process. As a result, the majority of these control algorithms rely heavily on mathematics. High performance computational processing power is needed to process all the raw data in real-time. However, any high performance computation proportionally consumes more energy, so to conserve battery life on the robot, many robotic researchers implement remote computation by processing the raw data remotely. This implementation has one major drawback: in order to transmit raw data remotely, a robust communication infrastructure is needed. Without a robust communication the robot will suffers in failures due to the sheer amount of raw data in communication. I am proposing a solution to the computation problem by harvesting the General Purpose Graphic Processing Unit (GPU)’s computational power for complex mathematical raw data processing. This approach utilizes many-cores parallelism for multithreading real-time computation with a minimum of 10x the computational flops of traditional Central Processing Unit (CPU). By shifting the computation on the GPU the entire computation will be done locally on the robot itself to eliminate the need of any external communication system for remote data processing. While the GPU is used to perform image processing for the robot, the CPU is allowed to dedicate all of its processing power to run the other functions of the robot. Computer vision has been an interesting topic for a while; it utilizes complex mathematical techniques and algorithms in an attempt to achieve image processing in real-time. Open Source Computer Vision 3 | P a g e Library (OpenCV) is a library consisting of pre-programed image processing functions for several different languages, developed by Intel. This library greatly reduces the amount of development time. Microsoft Kinect for XBOX has all the sensors mentioned above in a single convenient package with first party SDK for software development. I perform the robotic implementation utilizing Microsoft Kinect for XBOX as the primary sensor, OpenCV for image processing functions and NVidia GPU to off-load complex mathematical raw data processing for the robot. This thesis’s objective is to develop and implement a low cost autonomous robot tank with Microsoft Kinect for XBOX as the primary sensor, a custom onboard Small Form Factor (SFF) High Performance Computer (HPC) with NVidia GPU assistance for the primary computation, and OpenCV library for image processing functions. 4 | P a g e Table of Contents Acknowledgements ....................................................................................................................................... 2 Abstract ......................................................................................................................................................... 3 Table of Contents .......................................................................................................................................... 5 List of Figures ................................................................................................................................................ 7 List of Tables ................................................................................................................................................. 9 Chapter 1 Introduction ............................................................................................................................... 10 1.1 Challenges in Stereo Computer Vision .............................................................................................. 10 1.2 Challenges in Voice Recognitions...................................................................................................... 13 1.3 Challenges in Cost and Performance ................................................................................................ 14 1.4 Contribution and Design Goal ........................................................................................................... 15 1.5 Organization of Thesis ....................................................................................................................... 16 Chapter 2 Components ............................................................................................................................... 17 2.1 Sensors Selection .............................................................................................................................. 17 2.1.1 Distance Sensors ........................................................................................................................ 17 2.1.2 Imaging Sensor ........................................................................................................................... 20 2.1.3 Acoustic Sound Sensor ............................................................................................................... 22 2.2 Processor Selection ........................................................................................................................... 24 2.2.1 Intel x86 Families ....................................................................................................................... 25 2.2.1 AMD x86 Families