DOCSLIB.ORG

Computation Offloading of Augmented Reality in Warehouse Order Picking

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Computation Offloading of Augmented Reality in Warehouse Order Picking Computation Offloading Of Augmented Reality In Warehouse Order Picking Creative Technology Bachelor of Science Thesis Harald Eversmann July, 2019 UNIVERSITY OF TWENTE Faculty of Electrical Engineering, Mathematics and Computer Science (EEMCS) Supervisor Dr. Job Zwiers Critical observer Dr. Randy Klaassen Client Gerben Hillebrand CaptureTech Corporation B.V. [Page intentionally left blank] 1 ABSTRACT A novel method is proposed to implement computation offloading in augmented reality (AR) technology, such that said technology can be of more use for industrial purposes, and in this case specifically, warehouse order picking. The proposed method utilises a wireless connection between the AR device in question and a server and lets them communicate with each other via video and MJPEG live streams. Experiments show promising results for the prototype, but not yet in terms of fully offloading the AR devices workload. It is expected that rising technologies like faster Wi-Fi connection can help in the successful conclusion of fully offloading AR devices. 2 3 ACKNOWLEDGEMENTS The author would like to express his deep gratitude towards CaptureTech Corporation B.V. and in particular Gerben Hillebrand for the opportunity, knowledge, and resources to make this research project possible. The author thanks Dr. J. Zwiers for his continuous supervision and assistance throughout the entire graduation project in question. Additionally, the author would also like to ex- press his thanks to Dr. R. Klaassen for his role as critical observer during the process of this research. 4 TABLE OF CONTENTS ABSTRACT ........................................................................................................................................................ 2 ACKNOWLEDGEMENTS .................................................................................................................................... 4 LIST OF TABLES ................................................................................................................................................ 7 LIST OF FIGURES ............................................................................................................................................... 8 LIST OF ABBREVIATIONS .................................................................................................................................. 9 I. INTRODUCTION .......................................................................................................................................... 10 II. BACKGROUND ........................................................................................................................................... 12 AUGMENTED REALITY ............................................................................................................................................ 12 USER TEST ........................................................................................................................................................... 12 OFFLOADING ........................................................................................................................................................ 14 POTENTIAL SUPPORTING DEVICES.............................................................................................................................. 16 PROBLEM STATEMENT ............................................................................................................................................ 17 III. IDEATION .................................................................................................................................................. 18 CREATIVE TECHNOLOGY DESIGN PROCESS ................................................................................................................... 18 STAKEHOLDERS ..................................................................................................................................................... 18 OBSERVATIONS ..................................................................................................................................................... 18 DEVELOPMENT ..................................................................................................................................................... 19 COMMUNICATION ................................................................................................................................................. 20 SENDING AND RECEIVING IMAGES TO PROCESS .......................................................................................................... 20 OFFLOADING AS TOTAL BACKEND ............................................................................................................................ 21 IMAGE RECOGNITION ............................................................................................................................................. 21 USER EXPERIENCE ................................................................................................................................................. 22 INTERACTION DIAGRAM .......................................................................................................................................... 23 REQUIREMENTS .................................................................................................................................................... 24 IV. IMPLEMENTATION .................................................................................................................................... 26 FIRST PROTOTYPE .................................................................................................................................................. 26 SECOND PROTOTYPE .............................................................................................................................................. 27 THIRD PROTOTYPE ................................................................................................................................................. 29 SETTING UP THE SERVER ........................................................................................................................................ 32 ANDROID APPLICATION .......................................................................................................................................... 33 V. EVALUATION ............................................................................................................................................. 36 POSSIBLE EVALUATIONS .......................................................................................................................................... 36 TEST SETUP .......................................................................................................................................................... 37 COLLECTED DATA .................................................................................................................................................. 38 DISCUSSION ......................................................................................................................................................... 39 EVALUATION CONCLUSION ...................................................................................................................................... 40 VI. CONCLUSION ............................................................................................................................................ 41 VII. DISCUSSION AND FUTURE RESEARCH ...................................................................................................... 42 IV. REFERENCES ............................................................................................................................................. 43 APPENDIX A: QUESTIONNAIRE ....................................................................................................................... 45 APPENDIX B: CONSENT FORM........................................................................................................................ 46 5 APPENDIX C: SEQUENCE DIAGRAM COMMANDS ........................................................................................... 48 APPENDIX D: FIRST PROTOTYPE ..................................................................................................................... 49 APPENDIX E: CODE FOR MEASURING TIME BETWEEN FRAMES ...................................................................... 50 APPENDIX F: SECOND PROTOTYPE WITH ZBAR .............................................................................................. 51 APPENDIX G: THIRD PROTOTYPE WITH VIDEOSTREAM AND WHITE BACKGROUND ...................................... 52 APPENDIX H: IMAGE PROCESSING SERVER CODE ........................................................................................... 53 APPENDIX I: ANDROID APPLICATION ............................................................................................................. 55 6 LIST OF TABLES Page number Table number Description 25 3.1 A list of requirements for the prototype, classified in general and prototype specific necessities. 27 4.1 The measurements of time between two frames for the first prototype. 29 4.2 The time between two frames measured for the program with ZBar implemented. 31 4.3 The time between two frames measured for the program with Imutils and a transparent back- ground. 31 4.4 The time between two frames measured for the program
Load more

Recommended publications
  • A Regularization Approach to Blind Deblurring and Denoising of QR Barcodes Yves Van Gennip, Prashant Athavale, Jer´ Omeˆ Gilles, and Rustum Choksi
    IEEE TRANSACTIONS ON IMAGE PROCESSING ******** 1 A Regularization Approach to Blind Deblurring and Denoising of QR Barcodes Yves van Gennip, Prashant Athavale, Jer´ omeˆ Gilles, and Rustum Choksi Abstract—Using only regularization-based methods, analyzed by a programmed processor ([2], [4]). Key we provide an ansatz-free algorithm for blind deblur- to this detection are a set of required patterns, so- ring of QR bar codes in the presence of noise. The called finder patterns, which consist of three fixed algorithm exploits the fact that QR bar codes are prototypical images for which part of the image is a squares at the top and bottom left corners of the priori known (finder patterns). The method has four image, a smaller square near the bottom right corner steps: (i) denoising of the entire image via a suitably for alignment and two lines of pixels connecting the weighted TV flow; (ii) using a priori knowledge of one two top corners at their bottoms and the two left of the finder corners to apply a higher-order smooth corners at their right sides. Figure2 shows gives a regularization to estimate the unknown point spread function (PSF) associated with the blurring; (iii) schematic of a QR bar code, in particular showing applying an appropriately regularized deconvolution these required squares. using the PSF of step (ii); (iv) thresholding the output. We assess our methods via the open source bar code reader software ZBar [1]. Index Terms—QR bar code, blind deblurring, TV regularization, TV flow. I. INTRODUCTION Fig. 1: A QR bar code (used as “Code 1” in the tests Invented in Japan by the Toyota subsidiary Denso described in this paper).
    [Show full text]
  • CATS News, Fall 2015
    Newsletter for data prep engineers Issue 6 | Fall 2015 Dear CATS User: Mask manufacturing at technology nodes CATS news below 20nm presents new challenges for mask lithography and verification. Release An Abbreviated CPEC User Guide: K-2015.09 of CATS breaks new ground Simple Steps to Improve the Dose Correction in addressing these challenges with Alex Zepka, CATS CAE comprehensive support for multi beam mask CATS CPEC allows for the correction of e-beam proximity effects due to e-beam scattering. This writers. In addition, this correction is done via modulations on the exposed dose and can be applied to any lithographic process release delivers higher that involves the exposure of a material stack by an electron beam, e.g., mask manufacturing as well as fracture performance direct-write to wafer. and new capabilities The setup for CPEC can be a bit overwhelming as some settings are interdependent. This article to meet the throughput attempts to provide some guidance about typical cases and what types of parameters are most likely to requirements as mask require finer adjustments in order to improve the correction. It is not intended to be a complete manual data sets get larger and (that is provided in the CATS documentation) and it does not cover all CPEC settings. Instead, it is more complex. CATS meant to help the user in deciding which settings to use in most cases, and which settings are more is the most widely used likely to require fine-tuning in order to improve the quality of the correction or turnaround time. mask data prep solution and this is dependent on Minimum Settings (“Pipe-cleaning” Mode) our customers success When using CPEC for the first time, the user may merely be trying to obtain a simple correction to in using the product visually verify that CPEC is indeed assigning doses.
    [Show full text]
  • Springer.Com
    AB2009 ABCD springer.com Springer NEWS Apress AIP Press American Institute of Physics Birkhäuser Copernicus Current Medicine Friends of ED Want this by e-mail? Humana Press Subscribe now to the monthly Lars Müller Library Books E-Newsletter: Lavoisier-Intercept springer.com/librarybooks Physica Verlag Get all the content of this catalog in PDF, Excel, and HTML. Springer-Praxis 10 The Royal Society of Chemistry OCTOBER Springer Wien NewYork Steinkopff Verlag 2009 Vieweg ABCD springer.com Notice of Price Changes Springer announces that as of April 15, 2009 a new price list for print books will take effect. The list of price changes will be posted for download from springer.com/booksellers new price list ANNOUNCEMENT 014161x springer.com/librarybooks Contents I Author Index . II Title Index . VII Proceedings . 113 Apress . .36 Arts/Design . 112 Biomedicine . .16 Business/Economics . .93 Chemistry . .48 Computer Science . .44 Earth Sciences/Geography . .80 Education . 105 Engineering . .51 Environmental Sciences . .82 General Science . 104 Law . 101 Life Sciences . .85 Management/Business for Professionals . .99 Materials Science . .76 Mathematics . .24 Medicine . 1 Philosophy . 109 Physics/Astronomy . .68 Psychology . .23 Social Sciences . 107 Statistics . .35 New in the NEWS! Each eBook subject collection (12 collections for English-language/ International titles) is designated by its own icon. The icon appears next to each title, indicating its subject group. II Author Index Springer News 10/2009 springer.com/booksellers A 93 Bischi et al., Nonlinear Oligopolies 25 Chriss/Ginzburg, Representation Theory and 1 Blair (Eds), Imaging of Gynecological Disor- Complex Geometry (Modern Birkhäuser 1 Athanasiou (Eds), Key Topics in Surgical ders in Infants and Children (Med.
    [Show full text]
  • Manuscript Preparation Instruction for Publishing in Computer Modeling In
    Computers, Materials & Continua CMC, vol.64, no.3, pp.1885-1895, 2020 An Efficient Bar Code Image Recognition Algorithm for Sorting System Desheng Zheng1, *, Ziyong Ran1, Zhifeng Liu1, Liang Li2 and Lulu Tian3 Abstract: In the sorting system of the production line, the object movement, fixed angle of view, light intensity and other reasons lead to obscure blurred images. It results in bar code recognition rate being low and real time being poor. Aiming at the above problems, a progressive bar code compressed recognition algorithm is proposed. First, assuming that the source image is not tilted, use the direct recognition method to quickly identify the compressed source image. Failure indicates that the compression ratio is improper or the image is skewed. Then, the source image is enhanced to identify the source image directly. Finally, the inclination of the compressed image is detected by the barcode region recognition method and the source image is corrected to locate the barcode information in the barcode region recognition image. The results of multitype image experiments show that the proposed method is improved by 5+ times computational efficiency compared with the former methods, and can recognize fuzzy images better. Keywords: Bar code recognition, Hough transformation, binarization, image processing. 1 Introduction Bar code is a graphic recognizer [Hu (2011)] that arranges several black bars and white spaces of different widths according to certain coding rules to express a group of information. With the rapid development of science and technology, bar code technology is becoming more and more mature, and it has applications in all areas of commodity circulation.
    [Show full text]
  • Clickshare CSE-200
    ClickShare CSE-200 User guide ENABLING BRIGHT OUTCOMES Barco NV Beneluxpark 21, 8500 Kortrijk, Belgium www.barco.com/en/support www.barco.com Copyright © All rights reserved. No part of this document may be copied, reproduced or translated. It shall not otherwise be recorded, transmitted or stored in a retrieval system without the prior written consent of Barco. Trademarks USB Type-CTM and USB-CTM are trademarks of USB Implementers Forum. Trademarks Brand and product names mentioned in this manual may be trademarks, registered trademarks or copyrights of their respective holders. All brand and product names mentioned in this manual serve as comments or examples and are not to be understood as advertising for the products or their manufacturers. Product Security Incident Response As a global technology leader, Barco is committed to deliver secure solutions and services to our customers, while protecting Barco’s intellectual property. When product security concerns are received, the product security incident response process will be triggered immediately. To address specific security concerns or to report security issues with Barco products, please inform us via contact details mentioned on https://www.barco.com/psirt. To protect our customers, Barco does not publically disclose or confirm security vulnerabilities until Barco has conducted an analysis of the product and issued fixes and/or mitigations. Patent protection Please refer to www.barco.com/about-barco/legal/patents Guarantee and Compensation Barco provides a guarantee relating to perfect manufacturing as part of the legally stipulated terms of guarantee. On receipt, the purchaser must immediately inspect all delivered goods for damage incurred during transport, as well as for material and manufacturing faults Barco must be informed immediately in writing of any complaints.
    [Show full text]
  • Brno University of Technology Vysoké Učení Technické V Brně
    BRNO UNIVERSITY OF TECHNOLOGY VYSOKÉ UČENÍ TECHNICKÉ V BRNĚ FACULTY OF MECHANICAL ENGINEERING FAKULTA STROJNÍHO INŽENÝRSTVÍ INSTITUTE OF SOLID MECHANICS, MECHATRONICS AND BIOMECHANICS ÚSTAV MECHANIKY TĚLES, MECHATRONIKY A BIOMECHANIKY QR CODE DETECTION UNDER ROS IMPLEMENTED ON THE GPU DETEKCE QR KÓDŮ NA GRAFICKÉ KARTĚ PRO PLATFORMU ROS MASTER'S THESIS DIPLOMOVÁ PRÁCE AUTHOR Bc. Milan Hurban AUTOR PRÁCE SUPERVISOR doc. Ing. Jiří Krejsa, Ph.D. VEDOUCÍ PRÁCE BRNO 2017 Master'sThesis Assignment Institut: Instituteof SolidMechanics. Mechatronics and Biomechanics Student: Bc. MilanHurban Degreeprogramm: AppliedSciences in Engineering Branch: Mechatronics Supervisor: doc. Ing.Jiii Krejsa,Ph.D. Academicyear: 2016t17 As providedfor bythe Act No. 111198Coll. on highereducation institutions and the BUTStudy and ExaminationRegulations, the directorof the Institutehereby assigns the followingtopic of Master's Thesis: QR code detectionunder ROSimplemented on the GPU Brief description: The thesisgoal is to implementa methodof imageprocessing capable of QR codedetection in an imageacquired by an onboardcamera of a mobilerobot. To reducethe work load on the onboard computer,the methodis implementedon the graphicalcard (GPU). Additionally, the methodshould be integratedinto the RoboticOperating System (ROS) platform. Maste/s Thesisgoals: 1. Researchthe algorithmsof QR codedetection in an image 2. Researchthe methodsfor the implementationof computationaltasks on the GPU 3. Selectan appropriatealgorithm, implement it andintegrate under the ROSplatform 4.
    [Show full text]
  • Active Recommendation Method Based Mobile Network Using Book Management Application
    International Journal of Machine Learning and Computing, Vol. 9, No. 2, April 2019 Active Recommendation Method Based Mobile Network Using Book Management Application M. B. Chung or enemies are displayed on the camera overlay through the Abstract—This paper suggests a mobile-based book marker recognition function, have emerged [1], [2]. recommendation method utilizing a book management Navigation apps (App: Application) based on AR can lead a application. It will help smart phones to recognize Barcodes and user to a destination by displaying the targeted destination and QR codes and collect information on books through the open APIs. Such information is divided into that of books already the distance to it on the camera overlay, utilizing the smart purchased and that of the books the user would like to buy. This phone‟s GPS and Mapkit framework [3], [4]. divided information is stored in an application, while the server Technology based on smart phone cameras has been measures the similarity among users in terms of their applied not just for AR games and navigation functions, but, information stored in the database created. Then, if it is found in many cases, also for the recognition of objects or markers. that users with similar preferences do not have certain books, One application, called Pudding Camera, provides a service recommendations for the books will be made through push notification; the users receiving the recommendations get the that uses Open CV to recognize a face and match a photo of book details directly from the server. To evaluate the the face to an entertainer with a similar image.
    [Show full text]
  • Technical Notes All Changes in Fedora 13
    Fedora 13 Technical Notes All changes in Fedora 13 Edited by The Fedora Docs Team Copyright © 2010 Red Hat, Inc. and others. The text of and illustrations in this document are licensed by Red Hat under a Creative Commons Attribution–Share Alike 3.0 Unported license ("CC-BY-SA"). An explanation of CC-BY-SA is available at http://creativecommons.org/licenses/by-sa/3.0/. The original authors of this document, and Red Hat, designate the Fedora Project as the "Attribution Party" for purposes of CC-BY-SA. In accordance with CC-BY-SA, if you distribute this document or an adaptation of it, you must provide the URL for the original version. Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law. Red Hat, Red Hat Enterprise Linux, the Shadowman logo, JBoss, MetaMatrix, Fedora, the Infinity Logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries. For guidelines on the permitted uses of the Fedora trademarks, refer to https:// fedoraproject.org/wiki/Legal:Trademark_guidelines. Linux® is the registered trademark of Linus Torvalds in the United States and other countries. Java® is a registered trademark of Oracle and/or its affiliates. XFS® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries. All other trademarks are the property of their respective owners. Abstract This document lists all changed packages between Fedora 12 and Fedora 13.
    [Show full text]
  • Monkeysign Documentation Release ???
    Monkeysign Documentation Release ??? Antoine Beaupré Oct 21, 2019 Contents 1 Features 3 2 Similar projects 5 2.1 Install...................................................5 2.2 Usage...................................................7 2.3 Support.................................................. 10 2.4 Contribute................................................ 12 2.5 History.................................................. 19 2.6 API documentation............................................ 27 2.7 UI mockups and design......................................... 32 2.8 Terminology............................................... 33 2.9 Credits.................................................. 33 Python Module Index 35 Index 37 i ii Monkeysign Documentation, Release ??? Monkeysign is a tool to overhaul the OpenPGP keysigning experience and bring it closer to something that most primates can understand. The project makes use of cheap digital cameras and the type of bar code known as a QRcode to provide a human- friendly yet still-secure keysigning experience. No more reciting tedious strings of hexadecimal characters. And, you can build a little rogue’s gallery of the people that you have met and exchanged keys with! (Well, not yet, but it’s part of the plan.) Monkeysign also features a user-friendly commandline tool, similar to caff, to sign OpenPGP keys following the current best practices. Monkeysign was written by Jerome Charaoui and Antoine Beaupre and is licensed under GPLv3. Contents 1 Monkeysign Documentation, Release ??? 2 Contents
    [Show full text]
  • Design Manual 2020-12-11
    SNOW CRASH Using QR codes as a means of cyber attack Design Manual 2020-12-11 Brendan D. Burke [email protected] Snow Crash – Design Manual | Brendan D. Burke Contents Introduction ............................................................................................................................................ 2 Technology Stack .................................................................................................................................... 2 Libqrencode ........................................................................................................................................ 2 ZBar Bar Code Reader ......................................................................................................................... 2 feh ....................................................................................................................................................... 2 Dialog .................................................................................................................................................. 2 GitHub ................................................................................................................................................. 2 Virtualbox ............................................................................................................................................ 3 Kali ....................................................................................................................................................... 3 Wifipumpkin3.....................................................................................................................................
    [Show full text]
  • Web-Based Paper Exam Grading System
    Zesje Web-based paper exam grading system Nick Cleintuar Justin van der Krieken Jamy Mahabier Zesje Web-based paper exam grading system by Nick Cleintuar Justin van der Krieken Jamy Mahabier to obtain the degree of Bachelor of Science at the Delft University of Technology, to be defended publicly on Tuesday July 3, 2018 at 11:00. Project duration: April 23, 2018 – July 3, 2018 Project committee: A.R. Akhmerov TU Delft, client M.F. Aniche TU Delft, coach H. Wang TU Delft, coordinator An electronic version of this thesis is available at https://repository.tudelft.nl/. Preface This document is the final report for our bachelor end project of the Bachelor of Science in Computer Science at the Delft University of Technology. The project consisted of a research phase of two weeks, followed by an implementation phase of eight weeks. The report includes various details about the development on the already existing system called Zesje, an online exam grading tool for paper-based exams. It details the various steps and decisions taken to add the functionality for teachers to create compatible exams using Zesje. We express our gratitude to a number of people for making this project possible and supporting us throughout: the Zesje team, consisting of Anton Akhmerov (our client), Thomas Roos and Joseph Weston (listed in alphabetical order); and our coach, Maurício Aniche. Nick Cleintuar Justin van der Krieken Jamy Mahabier Delft, June 2018 1 Summary Grading can be a very time-consuming activity for teachers. For this reason, numerous tools exist to aid teachers in grading.
    [Show full text]
  • A Study on Multiple Barcode Detection from an Image in Business System
    International Journal of Computer Applications (0975 – 8887) Volume 181 – No. 37, January 2019 A Study on Multiple Barcode Detection from an Image in Business System Atiqul Islam Chowdhury Mushfika Sharmin Rahman Nazmus Sakib Department of Computer Science Department of Computer Science Department of Computer Science & Engineering & Engineering & Engineering Ahsanullah University of Science Ahsanullah University of Science Ahsanullah University of Science and Technology and Technology and Technology Dhaka, Bangladesh Dhaka, Bangladesh Dhaka, Bangladesh ABSTRACT Code, PDF417, MaxiCode are some examples of 2-D The pillar of automatic identification is Barcode technology barcodes. But among these, EAN-13 [3], Code 128 and QR which is used comprehensively in real time applications with codes [4] are mostly used. Some of 1D and 2D barcodes are various types of codes. The different types of codes and shown in Fig.1. applications sometimes faces special problems, so the improvement of the effectiveness must be done persistently. This paper’s proposal mainly deals with multiple product’s barcode to be detected simultaneously. The proposed algorithm which will have implementation for supershop billing system and inventory management. The method will recognize the barcodes using image processing. Images will be taken using mobile camera sensor. It will detect 1D barcodes such as EAN-13 barcodes, Code-128 barcodes, 2D barcodes such as QR codes. Moreover, it will be angle invariant, requires less user interaction than usual and can be executed on available computers. This model helps consumers to minimize the time for the billing system in shopping. We Fig.1: Some 1D & 2D barcodes have implemented this in Python IDE using OpenCV library.
    [Show full text]