Automatic Generation of Augmented Reality Guided Assembly Instructions Using Expert Demonstration Bhaskar Bhattacharya Iowa State University
Total Page:16
File Type:pdf, Size:1020Kb
Iowa State University Capstones, Theses and Graduate Theses and Dissertations Dissertations 2016 Automatic generation of augmented reality guided assembly instructions using expert demonstration Bhaskar Bhattacharya Iowa State University Follow this and additional works at: https://lib.dr.iastate.edu/etd Part of the Computer Sciences Commons, and the Mechanical Engineering Commons Recommended Citation Bhattacharya, Bhaskar, "Automatic generation of augmented reality guided assembly instructions using expert demonstration" (2016). Graduate Theses and Dissertations. 15877. https://lib.dr.iastate.edu/etd/15877 This Dissertation is brought to you for free and open access by the Iowa State University Capstones, Theses and Dissertations at Iowa State University Digital Repository. It has been accepted for inclusion in Graduate Theses and Dissertations by an authorized administrator of Iowa State University Digital Repository. For more information, please contact [email protected]. Automatic generation of augmented reality guided assembly instructions using expert demonstration by Bhaskar Bhattacharya A dissertation submitted to the graduate faculty in partial fulfilment of the requirements for the degree of DOCTOR OF PHILOSOPHY Major: Human Computer Interaction Program of Study Committee: Eliot Winer, Major Professor Jim Oliver Rafael Radkowski Stephen Gilbert Michael Dorneich Iowa State University Ames, Iowa 2016 Copyright © Bhaskar Bhattacharya, 2016. All rights reserved. ii TABLE OF CONTENTS LIST OF FIGURES .............................................................................................................................. vi LIST OF TABLES ................................................................................................................................ ix ABSTRACT ......................................................................................................................................... x CHAPTER 1 INTRODUCTION ............................................................................................................ 1 1.1. Augmented Reality Defined ................................................................................................. 1 1.2. AR Guided Assembly ............................................................................................................ 4 1.3. Problems with Authoring AR Guided Assembly Content .................................................... 6 CHAPTER 2 LITERATURE REVIEW ................................................................................................... 11 2.1. AR in Education .................................................................................................................. 12 2.2. AR in Medicine ................................................................................................................... 16 2.3. AR in Entertainment........................................................................................................... 18 2.4. Miscellaneous Uses of AR .................................................................................................. 21 2.5. AR in Manufacturing .......................................................................................................... 22 2.6. Authoring AR Guided Assembly ......................................................................................... 28 2.6.1. Priorities ...................................................................................................................... 28 2.6.2. Current AR guided assembly based authoring tools .................................................. 29 2.7. Research Issues .................................................................................................................. 34 CHAPTER 3 DECOMPOSING AN AR AUTHORING PROCESS ........................................................... 37 3.1. AR Authoring Tools and Roles Within the Authoring Process ........................................... 37 3.2. Hampshire Classification .................................................................................................... 38 3.3. Functional Categorization of AR Authoring Tools .............................................................. 44 3.3.1. Environment ............................................................................................................... 45 Indirect scene interaction ...................................................................................................... 45 Direct scene interaction ........................................................................................................ 46 3.3.2. Interaction metaphors ................................................................................................ 47 Markers .................................................................................................................................. 47 Interfaces ............................................................................................................................... 50 Points of View ........................................................................................................................ 52 3.4. Author ................................................................................................................................ 53 3.5. AR Authoring Via Expert Demonstration ........................................................................... 54 iii CHAPTER 4 USING A PARTICLE SWARM OPTIMIZED GAUSSIAN MIXTURE MODEL FOR SKIN DETECTION DURING AUTOMATED AR AUTHORING ..................................................................... 56 Abstract ......................................................................................................................................... 56 4.1. Introduction ....................................................................................................................... 57 4.2. Background ........................................................................................................................ 61 4.2.1. Skin detection ............................................................................................................. 61 Popular methods of skin detection ....................................................................................... 61 Gaussian mixture models ...................................................................................................... 62 Particle swarm optimization .................................................................................................. 63 4.2.2. Particle swarm optimized Gaussian mixture model ................................................... 66 4.3. Expert Demonstration ........................................................................................................ 71 4.3.1. Setup ........................................................................................................................... 71 4.3.2. Background processing ............................................................................................... 73 4.3.3. Skin detection and finding the number of steps ........................................................ 74 4.4. Results ................................................................................................................................ 75 4.4.1. Skin detection ............................................................................................................. 75 4.4.2. Expert demonstration dataset.................................................................................... 79 4.5. Summary ............................................................................................................................ 84 4.6. Future Work ....................................................................................................................... 85 4.7. Acknowledgment ............................................................................................................... 85 CHAPTER 5 AUGMENTED REALITY VIA EXPERT DEMONSTRATION AUTHORING (AREDA) ........... 86 Abstract ......................................................................................................................................... 86 5.1. Introduction ....................................................................................................................... 87 5.2. AR Guided Assembly .......................................................................................................... 91 5.3. AR Authoring Tools ............................................................................................................ 93 5.3.1. Challenges of authoring AR content ........................................................................... 95 5.3.2. List of AR authoring tools ........................................................................................... 97 Linking system ....................................................................................................................... 97 AR previewer ......................................................................................................................... 97 Virtual registration ................................................................................................................. 98 Hybrid methods ..................................................................................................................... 98 iv Context aware ......................................................................................................................