Industrial Vision Robot with Raspberry Pi Using Pixy Camera
Total Page:16
File Type:pdf, Size:1020Kb
GBENGA MONDAY OMOSEKEJI INDUSTRIAL VISION ROBOT WITH RASPBERRY PI USING PIXY CAMERA Stereo Vision System Information Technology, Embedded System Engineering 2018 ACKNOWLEDGEMENTS I am most grateful to the Almighty God for the successful completion of my Bach- elor’s degree program. Also, I am grateful to the Finnish government, Vaasa Uni- versity of Applied Sciences, the Information Technology Group for giving me the privilege to learn and apply my knowledge to solving real life problem at no cost. My appreciation goes to my supervisor and Lecturer – Jukka Matila for His pro- fessional, care and inspiration in this project. Also, I will like to thank all the lec- turers who have taught me all through my study. Big thanks to my lovely fiancée and longtime friend; Olwaseun Salami, for the love, understanding and encouragement at all time. Special thanks to my Mother Mrs Omosekeji for her prayers and continue support and to my late Father Mr Omosekeji thank you for showing me what needed to be successful in life. And to my siblings; Ilemobola, Adedowo, Adegbite, Atinuke, Olabisi, Esther, Adeniyi, Mercy, and Adetola and their families, thanks for the love we share and encourage- ment all through my studies. Also to my friends in Vaasa for your support. Finally, my appreciation goes to Professor Manuel Silver and Germano Veiga and my wonderful colleague Alexandre Filipe at ISEP Portugal for their assistance in times of need in this project. ABSTRACT Author Gbenga Monday Omosekeji Title Industrial Vision Robot with Raspberry Pi using Pixy Cam- era Year 2018 Language English Pages 106 Name of Supervisor Jukka Matila Industrial robots are not human, they are machines. They are programmable ma- nipulator devices which can move tools or parts via a set sequence of motions. In addition, they can be reprogrammable, that is, the robot’s action can be modified by changing the control settings without replacing the hardware. They add some characteristics of traditional machines likewise as characteristics of machine oper- ators. For an operator, it is easy to be taught to do a new task. But, for a machine, a task can be repeated for prolonged times with great precision. This project focused at developing a Robot Vision system using a combination of low-cost camera hardware and computer algorithms to enable robots to process visual data from the world. The stereo vision algorithm which consists of two cam- eras, and the developed application are able to calculate a 3D position from s 2D detected object. In addition, the detection algorithm based on color differences was used by the cameras which enable 2D object tracking and outputted data coordi- nates of the object being detected. Then, the 3D object position is produced through the calculated 2D object data coordinates, which made ready for robot teaching. Furthermore, the developed application was based on OpenCV API in C++, which was an interest in the development of this project. The use of this was to treat the image capture by the cameras. TIY software with modification was used to do the object tracking. BOOST is a set of C++ libraries that provide image processing, and linear algebra functionalities. This library was the appropriate choice because of the reliance of TIY on it, and some other aspects of it that are important to the rest of the project. Finally, the test results showed that the project was successfully developed. In ad- dition, with the developed project, my expertise in embedded system programming has been consolidated and I have obtained further knowledge in the field of robotics and computer vision. Keywords: Stereo Vision, OpenCV API, TIY, C++, Boost, MatLab, Linux TABLE OF CONTENTS Table of Figures ................................................................................................................ 8 List of Tables .................................................................................................................. 10 ABBREVIATIONS ........................................................................................................ 11 1. INTRODUCTION .................................................................................................. 14 1.1. Contextualization .............................................................................................. 15 1.2. Objectives ......................................................................................................... 15 1.3. Thesis Report Organization .............................................................................. 15 2. DEVELOPING ROBOT 3D VISION SYSTEM ................................................... 17 2.3 Measuring object distance from the camera ..................................................... 21 2.4.1 Measuring Object Distance Using Monocular Camera (Fixed) ............... 21 2.4.2 Stereo Vision Method ............................................................................... 21 2.4.3 Chapter 2: Conclusion .............................................................................. 22 3.1 Tracking System ............................................................................................... 23 3.2 Object Tracking ................................................................................................ 24 3.3 Problem in Object Tracking .............................................................................. 24 3.4 Feature for Tracking an Object ......................................................................... 25 3.5 Algorithms or Methods ..................................................................................... 26 3.5.1 Algorithm: background subtraction .......................................................... 26 3.5.2 Tracking system bases on color ................................................................ 28 3.6 Track It Yourself (TIY) .................................................................................... 29 3.6.1 Chapter 3: Conclusion .............................................................................. 30 4. COMPUTER VISION ............................................................................................ 31 4.1 Machine Vision ................................................................................................. 32 4.1.1 Machine Vision Overview ........................................................................ 32 4.1.2 Machine Vision Operation........................................................................ 32 4.1.3 Financial Justification of Machine Vision Systems ................................. 34 4.2 Capture Image ................................................................................................... 34 4.2.1 Perspective Projection .............................................................................. 35 4.3 Camera Calibration ........................................................................................... 43 4.4 Chapter 4: Conclusion ...................................................................................... 45 5. SYSTEM ARCHITECTURE ................................................................................. 46 5.1 Decide the Position for Tracking System Cameras .......................................... 47 5.2 Raspberry Pi Overview ..................................................................................... 47 5.2.1 Processor ................................................................................................... 48 5.2.2 Peripherals ................................................................................................ 48 5.2.3 Operating Systems .................................................................................... 50 5.2.4 Raspberry Pi 2 Overview .......................................................................... 50 5.3 CMUcam5 Pixy ................................................................................................ 51 5.4 Open Source Computer Vision Library (OpenCV) .......................................... 54 5.5 Boost ................................................................................................................. 56 5.6 Chapter 5: Conclusion ...................................................................................... 56 6. IMPLEMENTATION ............................................................................................ 57 6.1 Raspberry Pi Communication and Power Connection ..................................... 57 6.1.1 Power Connection .................................................................................... 57 6.1.2 Communication ........................................................................................ 57 6.1 Pixy Communication and Power Connection ................................................... 59 6.2.1 Power Connection .................................................................................... 59 6.2.2 Communication ........................................................................................ 60 6.2.3 Setting the Interface .................................................................................. 60 6.3 Level Shifter ..................................................................................................... 63 6.4 Inter Processing Communication ...................................................................... 64 6.4.1 Mapped Memory ...................................................................................... 65 6.4.2 Shared Memory .......................................................................................