Industrial Vision Robot with Raspberry Pi Using Pixy Camera

Total Page:16

File Type:pdf, Size:1020Kb

Industrial Vision Robot with Raspberry Pi Using Pixy Camera GBENGA MONDAY OMOSEKEJI INDUSTRIAL VISION ROBOT WITH RASPBERRY PI USING PIXY CAMERA Stereo Vision System Information Technology, Embedded System Engineering 2018 ACKNOWLEDGEMENTS I am most grateful to the Almighty God for the successful completion of my Bach- elor’s degree program. Also, I am grateful to the Finnish government, Vaasa Uni- versity of Applied Sciences, the Information Technology Group for giving me the privilege to learn and apply my knowledge to solving real life problem at no cost. My appreciation goes to my supervisor and Lecturer – Jukka Matila for His pro- fessional, care and inspiration in this project. Also, I will like to thank all the lec- turers who have taught me all through my study. Big thanks to my lovely fiancée and longtime friend; Olwaseun Salami, for the love, understanding and encouragement at all time. Special thanks to my Mother Mrs Omosekeji for her prayers and continue support and to my late Father Mr Omosekeji thank you for showing me what needed to be successful in life. And to my siblings; Ilemobola, Adedowo, Adegbite, Atinuke, Olabisi, Esther, Adeniyi, Mercy, and Adetola and their families, thanks for the love we share and encourage- ment all through my studies. Also to my friends in Vaasa for your support. Finally, my appreciation goes to Professor Manuel Silver and Germano Veiga and my wonderful colleague Alexandre Filipe at ISEP Portugal for their assistance in times of need in this project. ABSTRACT Author Gbenga Monday Omosekeji Title Industrial Vision Robot with Raspberry Pi using Pixy Cam- era Year 2018 Language English Pages 106 Name of Supervisor Jukka Matila Industrial robots are not human, they are machines. They are programmable ma- nipulator devices which can move tools or parts via a set sequence of motions. In addition, they can be reprogrammable, that is, the robot’s action can be modified by changing the control settings without replacing the hardware. They add some characteristics of traditional machines likewise as characteristics of machine oper- ators. For an operator, it is easy to be taught to do a new task. But, for a machine, a task can be repeated for prolonged times with great precision. This project focused at developing a Robot Vision system using a combination of low-cost camera hardware and computer algorithms to enable robots to process visual data from the world. The stereo vision algorithm which consists of two cam- eras, and the developed application are able to calculate a 3D position from s 2D detected object. In addition, the detection algorithm based on color differences was used by the cameras which enable 2D object tracking and outputted data coordi- nates of the object being detected. Then, the 3D object position is produced through the calculated 2D object data coordinates, which made ready for robot teaching. Furthermore, the developed application was based on OpenCV API in C++, which was an interest in the development of this project. The use of this was to treat the image capture by the cameras. TIY software with modification was used to do the object tracking. BOOST is a set of C++ libraries that provide image processing, and linear algebra functionalities. This library was the appropriate choice because of the reliance of TIY on it, and some other aspects of it that are important to the rest of the project. Finally, the test results showed that the project was successfully developed. In ad- dition, with the developed project, my expertise in embedded system programming has been consolidated and I have obtained further knowledge in the field of robotics and computer vision. Keywords: Stereo Vision, OpenCV API, TIY, C++, Boost, MatLab, Linux TABLE OF CONTENTS Table of Figures ................................................................................................................ 8 List of Tables .................................................................................................................. 10 ABBREVIATIONS ........................................................................................................ 11 1. INTRODUCTION .................................................................................................. 14 1.1. Contextualization .............................................................................................. 15 1.2. Objectives ......................................................................................................... 15 1.3. Thesis Report Organization .............................................................................. 15 2. DEVELOPING ROBOT 3D VISION SYSTEM ................................................... 17 2.3 Measuring object distance from the camera ..................................................... 21 2.4.1 Measuring Object Distance Using Monocular Camera (Fixed) ............... 21 2.4.2 Stereo Vision Method ............................................................................... 21 2.4.3 Chapter 2: Conclusion .............................................................................. 22 3.1 Tracking System ............................................................................................... 23 3.2 Object Tracking ................................................................................................ 24 3.3 Problem in Object Tracking .............................................................................. 24 3.4 Feature for Tracking an Object ......................................................................... 25 3.5 Algorithms or Methods ..................................................................................... 26 3.5.1 Algorithm: background subtraction .......................................................... 26 3.5.2 Tracking system bases on color ................................................................ 28 3.6 Track It Yourself (TIY) .................................................................................... 29 3.6.1 Chapter 3: Conclusion .............................................................................. 30 4. COMPUTER VISION ............................................................................................ 31 4.1 Machine Vision ................................................................................................. 32 4.1.1 Machine Vision Overview ........................................................................ 32 4.1.2 Machine Vision Operation........................................................................ 32 4.1.3 Financial Justification of Machine Vision Systems ................................. 34 4.2 Capture Image ................................................................................................... 34 4.2.1 Perspective Projection .............................................................................. 35 4.3 Camera Calibration ........................................................................................... 43 4.4 Chapter 4: Conclusion ...................................................................................... 45 5. SYSTEM ARCHITECTURE ................................................................................. 46 5.1 Decide the Position for Tracking System Cameras .......................................... 47 5.2 Raspberry Pi Overview ..................................................................................... 47 5.2.1 Processor ................................................................................................... 48 5.2.2 Peripherals ................................................................................................ 48 5.2.3 Operating Systems .................................................................................... 50 5.2.4 Raspberry Pi 2 Overview .......................................................................... 50 5.3 CMUcam5 Pixy ................................................................................................ 51 5.4 Open Source Computer Vision Library (OpenCV) .......................................... 54 5.5 Boost ................................................................................................................. 56 5.6 Chapter 5: Conclusion ...................................................................................... 56 6. IMPLEMENTATION ............................................................................................ 57 6.1 Raspberry Pi Communication and Power Connection ..................................... 57 6.1.1 Power Connection .................................................................................... 57 6.1.2 Communication ........................................................................................ 57 6.1 Pixy Communication and Power Connection ................................................... 59 6.2.1 Power Connection .................................................................................... 59 6.2.2 Communication ........................................................................................ 60 6.2.3 Setting the Interface .................................................................................. 60 6.3 Level Shifter ..................................................................................................... 63 6.4 Inter Processing Communication ...................................................................... 64 6.4.1 Mapped Memory ...................................................................................... 65 6.4.2 Shared Memory .......................................................................................
Recommended publications
  • Upgrading a Legacy Outdoors Robotic Vehicle4
    Upgrading a Legacy Outdoors Robotic Vehicle4 1 1 2 Theodosis Ntegiannakis , Odysseas Mavromatakis , Savvas Piperidis , and Nikos C. Tsourveloudis3 1 Theodosis Ntegiannakis and Odysseas Mavromatakis are graduate students at the School of Production Engineering and Management, Technical University of Crete, Hellas. 2 corresponding author: Intelligent Systems and Robotics Laboratory, School of Production Engineering and Management, Technical University of Crete, 73100 Chania, Hellas, [email protected], www.robolab.tuc.gr 3 School of Production Engineering and Management, Technical University of Crete, Hellas, [email protected], www.robolab.tuc.gr Abstract. ATRV-mini was a popular, 2000's commercially available, outdoors robot. The successful upgrade procedure of a decommissioned ATRV-mini is presented in this paper. Its robust chassis construction, skid steering ability, and optional wifi connectivity were the major rea- sons for its commercial success, mainly for educational and research pur- poses. However the advances in electronics, microcontrollers and software during the last decades were not followed by the robot's manufacturer. As a result, the robot became obsolete and practically useless despite its good characteristics. The upgrade used up to date, off the shelf compo- nents and open source software tools. There was a major enhancement at robot's processing power, energy consumption, weight and autonomy time. Experimental testing proved the upgraded robot's operational in- tegrity and capability of undertaking educational, research and other typical robotic tasks. Keywords: all terrain autonomous robotic vehicle, educational robotics, ROS, robot upgrade, solar panels. 1 Introduction This work deals with the upgrading procedure of a decommissioned Real World Interface (RWI) ATRV-mini all-terrain robotic vehicle.
    [Show full text]
  • The Personal Rover Project: the Comprehensive Design of A
    Robotics and Autonomous Systems 42 (2003) 245–258 The Personal Rover Project: The comprehensive design of a domestic personal robot Emily Falcone∗, Rachel Gockley, Eric Porter, Illah Nourbakhsh The Robotics Institute, Carnegie Mellon University, Newell-Simon Hall 3111, 5000 Forbes Ave., Pittsburgh, PA 15213, USA Abstract In this paper, we summarize an approach for the dissemination of robotics technologies. In a manner analogous to the personal computer movement of the early 1980s, we propose that a productive niche for robotic technologies is as a long-term creative outlet for human expression and discovery. To this end, this paper describes our ongoing efforts to design, prototype and test a low-cost, highly competent personal rover for the domestic environment. © 2003 Elsevier Science B.V. All rights reserved. Keywords: Social robots; Educational robot; Human–robot interaction; Personal robot; Step climbing 1. Introduction duration of interaction and the roles played by human and robot participants. In cases where the human care- Robotics occupies a special place in the arena of giver provides short-term, nurturing interaction to a interactive technologies. It combines sophisticated robot, research has demonstrated the development of computation with rich sensory input in a physical effective social relationships [5,12,21]. Anthropomor- embodiment that can exhibit tangible and expressive phic robot design can help prime such interaction ex- behavior in the physical world. periments by providing immediately comprehensible In this regard, a central question that occupies our social cues for the human subjects [6,17]. research group pertains to the social niche of robotic In contrast, our interest lies in long-term human– artifacts in the company of the robotically uninitiated robot relationships, where a transient suspension of public-at-large: What is an appropriate first role for in- disbelief will prove less relevant than long-term social telligent human–robot interaction in the daily human engagement and growth.
    [Show full text]
  • 4 Perception 79
    4 Perception 79 4 Perception One of the most important tasks of autonomous systems of any kind is to acquire knowledge about its environment. This is done by taking measurements using various sensors and then extracting meaningful information from those measurements. In this chapter we present the most common sensors used in mobile robots and then discuss strategies for extracting information from the sensors. For more detailed information about many of the sensors used on mobile robots, refer to the comprehensive book Sensors for Mo- bile Robots written by H.R. Everett [2]. 4.1 Sensors for Mobile Robots a b c d Fig 4.1 Examples of robots with multi-sensor systems: a) HelpMate from Transition Research Corp. b) B21 from Real World Interface c) Roboart II, built by H.R. Everett [2] d) The Savannah River Site nuclear surveillance robot There is a wide variety of sensors used in mobile robots (Fig. 4.1). Some sensors are used to measure simple values like the internal temperature of a robot’s electronics or the rota- R. Siegwart, EPFL, Illah Nourbakhsh, CMU 80 Autonomous Mobile Robots tional speed of the motors. Other, more sophisticated sensors can be used to acquire infor- mation about the robot’s environment or even to directly measure a robot’s global position. In this chapter we focus primarily on sensors used to extract information about the robot’s environment. Because a mobile robot moves around, it will frequently encounter unfore- seen environmental characteristics, and therefore such sensing is particularly critical. We begin with a functional classification of sensors.
    [Show full text]
  • Pixy Hardware Interface
    SSttrriikkeerr EEL 4914 Senior Design I Group 8 Cruz, Efrain Decamp, Loubens Narvaez, Luis Thomas, Brian Competitive Autonomous Air-Hockey Gaming system Table of Contents Table of Contents .................................................................................................. i 1 Executive Summary .......................................................................................... 1 2 Project Description ............................................................................................ 3 2.1 Motivation ................................................................................................................. 3 2.2 Goals and Objectives ................................................................................................. 4 2.3 Requirements and Specifications ........................................................................... 4 2.3.1 User Interface .................................................................................................... 4 2.3.2 Audio and Visual Effects .................................................................................... 6 2.3.3 Tracking System ................................................................................................. 7 2.3.4 Software .............................................................................................................. 9 2.3.5 System Hardware .............................................................................................. 10 2.3.6 Puck Return Mechanism ..................................................................................
    [Show full text]
  • Botball Kit for Teaching Engineering Computing
    Botball Kit for Teaching Engineering Computing David P. Miller Charles Winton School of AME Department of CIS University of Oklahoma University of N. Florida Norman, OK 73019 Jacksonville, FL 32224 Abstract Many engineering classes can benefit from hands on use of robots. The KISS Institute Botball kit has found use in many classes at a number of universities. This paper outlines how the kit is used in a few of these different classes at a couple of different universities. This paper also introduces the Collegiate Botball Challenge, and how it can be used as a class project. 1 Introduction Introductory engineering courses are used to teach general principles while introducing the students to all of the engineering disciplines. Robotics, as a multi-disciplinary application can be an ideal subject for projects that stress the different engineering fields. A major consideration in establishing a robotics course emphasizing mobile robots is the type of hands-on laboratory experience that will be incorporated into the course of instruction. Most electrical engineering schools lack the machine shops and expertise needed to create the mechanical aspects of a robot system. Most mechanical engineering schools lack the electronics labs and expertise needed for the actuation, sensing and computational aspects required to support robotics work. The situation is even more dire for most computer science schools. Computer science departments typically do not have the support culture for the kind of laboratories that are more typically associated with engineering programs. On the other hand, it is recognized that computer science students need courses which provide closer to real world experiences via representative hands-on exercises.
    [Show full text]
  • A Multi-Robot Search and Retrieval System
    From: AAAI Technical Report WS-02-18. Compilation copyright © 2002, AAAI (www.aaai.org). All rights reserved. MinDART : A Multi-Robot Search & Retrieval System Paul E. Rybski, Amy Larson, Heather Metcalf, Devon Skyllingstad, Harini Veeraraghavan and Maria Gini Department of Computer Science and Engineering, University of Minnesota 4-192 EE/CS Building Minneapolis, MN 55455 {rybski,larson,hmetcalf,dsky,harini,gini}@cs.umn.edu Abstract We are interested in studying how environmental and control factors affect the performance of a homogeneous multi-robot team doing a search and retrieval task. We have constructed a group of inexpensive robots called the Minnesota Distributed Autonomous Robot Team (MinDART) which use simple sen- sors and actuators to complete their tasks. We have upgraded these robots with the CMUCam, an inexpensive camera sys- tem that runs a color segmentation algorithm. The camera allows the robots to localize themselves as well as visually recognize other robots. We analyze how the team’s perfor- mance is affected by target distribution (uniform or clumped), size of the team, and whether search with explicit localization is more beneficial than random search. Introduction Cooperating teams of robots have the potential to outper- form a single robot attempting an identical task. Increasing Figure 1: The robots with their infrared targets and colored task or environmental knowledge may also improve perfor- landmarks. mance, but increased performance comes at a price. In addi- tion to the monetary concerns of building multiple robots, AAAI 2002 Mobile Robot Competition and Exhibition. Fi- the complexity of the control strategy and the processing nally, we analyze how we would expect these robots to per- overhead can outweigh the benefits.
    [Show full text]
  • Real Time 3D Mapping for Small Wall Climbing Robots
    Real Time 3D Mapping for Small Wall Climbing Robots Blair David Sidney Howarth A thesis submitted in fulfilment of the requirements for the degree of Doctor of Philosophy School of Mechanical and Manufacturing Engineering The University of New South Wales February 2012 THE UNIVERSITY OF NEW SOUTH WALES Thesis/Dissertation Sheet Surname or Family name: HOWARTH First name: BLAIR Other name/s: DAVID SIDNEY Abbreviation for degree as given in the University calendar: MTRN (1662) School: MECHANICAL AND MANUFACTURING Faculty: ENGINEERING Title: REAL TIME 3D MAPPING FOR SMALL WALL CLIMBING ROBOTS Abstract 350 words maximum: Small wall climbing robots are useful because they can access difficult environments which preclude the use of more traditional mobile robot configurations. This could include an industrial plant or collapsed building which contains numerous obstacles and enclosed spaces. These robots are very agile and they can move fully through three dimensional (3D) space by attaching to nearby surfaces. For autonomous operation, they need the ability to map their environment to allow navigation and motion planning between footholds. This surface mapping must be performed onboard as line-of-sight and wireless communication may not always be available. As most of the methods used for robotic mapping and navigation were developed for two dimensional usage, they do not scale well or generalise for 3D operation. Wall climbing robots require a 3D map of nearby surfaces to facilitate navigation between footholds. However, no suitable mapping method currently exists. A 3D surface mapping methodology is presented in this thesis to meet this need. The presented 3D mapping method is based on the fusion of range and vision information in a novel fashion.
    [Show full text]
  • The Personal Rover Project Illah R
    The Personal Rover Project Illah R. Nourbakhsh The Robotics Institute Carnegie Mellon University Pittsburgh, PA 15213 [email protected] Abstract effort to explore the space of possible computer programs In this paper, I summarize a new approach for the and thus invent new human-computer interaction dissemination of robotics technologies. In a manner paradigms. analagous to the personal computer movement of the The goal of the Personal Rover project is analogous: to early 1980’s, we propose that a productive niche for design and deploy a capable robot that can be deployed robotic technologies is as a creative outlet for human into the domestic environment and that will help forge a expression and discovery. This paper describes our community of create robot enthusiasts. Such a personal ongoing efforts to design, prototype and test a low- rover is highly configurable by the end user, who is cost, highly competent personal rover for the creatively governing the behavior of the rover itself: a domestic environment. physical artifact with the same degree of programmability as the early personal computer combined with far richer 1. Motivation and more palpable sensory and effectory capabilities. Our goal is to produce a Personal Rover suitable for As with most leading technological fields, robotics children and adults who are not specialists in mechanical research is frequently focused upon the creation of engineering or electrical engineering. We hypothesize technology, not on creating compelling applications. that the right robot will catalyze such a community of Although the search for new technologies is a valid early adopters and will harness their inventive potential.
    [Show full text]
  • Proceedings FRE 2007
    Proceedings 2007 Proceedings of the 5th Field Robot Event 2007 Wageningen, June 14, 15 & 16 2007 ISBN : 978ĉ90ĉ8585ĉ168ĉ4 Date : December 2007 Publisher : Farm Technology Group Address : Technotron (Building 118) Bornsesteeg 59 6708 PD Wageningen The Netherlands Tel: +31 317 482980 Fax: +31 317 484819 Contact : [email protected] The information in these proceedings can be reproduced freely if reference is made to this proceedings bundle. Sponsors Field Robot Event 2007 Preface In summer 2003, when the 1st Field Robot Event was “born” at Wageningen University, it was an experiment to combine the “serious” and “playful” aspects of robotics to inspire the upcoming student generation. Specific objectives have been: • Employing students’ creativity to promote the development of field robots, • Promoting off-curriculum skills like communication, teamwork, time management and fundraising, • Attracting public interest for Agricultural Engineering, • Creating a platform for students and experts to exchange knowledge on field robots. Driven by the success of the previous events in Wageningen in 2003, 2004 and 2005 and the Field Robot Event hosted by Hohenheim University in Stuttgart, Germany in 2006, Wageningen University organized the 5th Field Robot Event on June 14-16, 2007. This event was accompanied by a workshop and a fair where the teams were able to present their robots. Teams also had to write a paper describing the hard- and software design of their robot. These papers collected in this ‘Proceedings of the 5th Field Robot Event’ are a very valuable source of information. This edition of the proceedings ensures that the achievements of the participants are now documented as a publication and thus being accessible as basis for further research.
    [Show full text]
  • Robotics Minifaq for Beginners Modified 22 October 2006: 13 Pages
    Robotics miniFAQ For Beginners Modified 22 October 2006: 13 pages Copyright Notice This miniFAQ was compiled and written by John Piccirillo. This FAQ may be referenced as: John Piccirillo (2006) “Robotics miniFAQ for Beginners”. This post, as a collection of information, is Copyright (c) 2006 by John Piccirillo. Verbatim copying and distribution of this entire document is permitted in any medium, provided this notice is preserved and as long as the miniFAQ is posted in its entirety and includes this copyright paragraph. This FAQ may not be distributed for financial gain or included in commercial collections or compilations without express written permission from the author. Please send changes, additions, suggestions, questions, and broken link notices to: John Piccirillo, Ph.D. Dept. of Electrical and Computer Engineering University of Alabama in Huntsville Huntsville, AL 35899 email: [email protected] Contents • Intro – What’s New, Why A miniFAQ, Scope • Where Can I Find Information on Robots and Robot Clubs? • What Introductory Books and Magazines Are Available? • Where Can I Find Parts For My Robot ? • What Sensors Are Available For Mobile Robots? • What’s An Easy To Use Computer For A Robot Brain ? • Are There Mobile Robot Kits Available ? What’s New ? This updated miniFAQ: • Updates some of the product descriptions and adds a few links. All links checked as of the distribution date. • Adds special subsections on Grippers and Wheel Hubs in the Parts section Why A miniFAQ ? Requests for information on robotics from beginners appears from time-to-time on various newsgroups and list servers. The response to these requests is highly variable depending on when and how the request is stated.
    [Show full text]
  • A Summer of Robots
    A Summer of Robots: Seeding Creative Robotics Across the Urban Landsdcape Carl DiSalvo, Illah Nourbakhsh, Kevin Crowley School of Literature, Communication and Culture, Georgia Institute of Technology Robotics Institute, Carnegie Mellon University Learning Research and Development Center, University of Pittsburgh Abstract. This article describes an urban experiment in community-based robotics deployed in Pittsburgh, Pennsylvania during 2007 and 2008. Twenty-one community-based organizations deployed robotics workshops, exhibits and curated pieces throughout the city, creating a space for technology fluency and exposition throughout the urban landscape, from galleries and schools to parks. The details of the creative design and arts approach employed and the results garnered elucidate our definitions of technology fluency, community-based robotics and the city-wide impact possible through synchronized outreach activities deployed at a large scale. Keywords: community-based robotics, creative design and arts, workshops, educational robotics 1 Introduction As Tom encouraged the students to begin considering their own creation, one team settled on their project At the Factory 14 workshop Neighborhoods and Play idea: an exquisite corpse display that would reflect hosted by the Mattress Factory museum, artist and the diversity of neighborhood residents in nearby coordinator Tom Sarver led twenty-five participants parks. Camera and Canary in hand, the team through a weekly evening course during October and interviewed and documented a dozen strangers in the November 2007. Open to any age and any level of park, asking them to talk about their sense of prior experience with technology and art, a collection neighborhood and community. Images of these of teams formed, with ages ranging from an eight strangers were divided into head, torso and leg shots, year old to middle-aged artists unfamiliar with and the team spent two weeks building a human-size technology, and middle-aged engineers with little robotic display that would interact with a visitor by background in art.
    [Show full text]
  • Embedded Computer Vision System Applied to a Four- Legged Line Follower Robot
    23rd ABCM International Congress of Mechanical Engineering December 6-11, 2015, Rio de Janeiro, RJ, Brazil EMBEDDED COMPUTER VISION SYSTEM APPLIED TO A FOUR- LEGGED LINE FOLLOWER ROBOT Beatriz Arruda Asfora Universidade Federal de Pernambuco, Av. Professor Morais Rego, 1235 - Cidade Universitária, Recife - PE, 50670-901. [email protected] Abstract. Robotics can be defined as the connection of perception to action. Taking this further, this project aims to drive a robot using an automated computer vision embedded system, connecting the robot's vision to its behavior. In order to implement a color recognition system on the robot, open source tools are chosen, such as Processing language, Android system, Arduino platform and Pixy camera. The constraints are clear: simplicity, replicability and financial viability. In order to integrate Robotics, Computer Vision and Image Processing, the robot is applied on a typical mobile robot's issue: line following. The problem of distinguishing the path from the background is analyzed through different approaches: the popular Otsu's Method, thresholding based on color combinations through experimentation and color tracking via hue and saturation. Decision making of where to move next is based on the line center of the path and is fully automated. Using a four-legged robot as platform and a camera as its only sensor, the robot is capable of successfully follow a line. From capturing the image to moving the robot, it's evident how integrative Robotics can be. The issue of this paper alone involves knowledge of Mechanical Engineering, Electronics, Control Systems and Programming. Everything related to this work was documented and made available on an open source online page, so it can be useful in learning and experimenting with robotics.
    [Show full text]