Virtual Reality in Assembly Simulation — Collision Detection, Simulation Algorithms, and Interaction Techniques Dem Fachbereich Informatik der Technischen Universität Darmstadt eingereichte Dissertation zur Erlangung des akademischen Grades eines Doktor-Ingenieurs (Dr.-Ing.) von Dipl.-Inform. Gabriel Zachmann Referenten der Arbeit: Prof. Dr.-Ing. Dr. h.c. Dr. Eh J. L. Encarnação Dr. Carolina Cruz-Neira Tag der Einreichung: 29. Mai 2000 Tag der mündlichen Prüfung: 10. Juli 2000 Darmstädter Dissertation Hochschulkennziffer D17 To my wife Biggi and my little daughter Mirjam Acknowledgements I would like to thank Professor Dr. Encarnação for his advice and support. The Fraunhofer Institute for Computer Graphics, which he is head of, has been a great work environment. I am grateful to Dr. Cruz-Neira for accepting co-advisorship and for travel- ling a great distance to attend my defense. From my work in many stimulating projects, I owe thanks in particular to Dipl.-Ing. Antonino Gomes de Sá, Dipl.-Ing. Rainer Klier, and Dipl.-Ing. Peter Zimmermann. Special thanks go to all current and former members of our extraordinary group “Visualization and Virtual Reality”, of which Dr. Stefan Müller is the department head: Dr. Peter Astheimer, Uli Bockholt, Dr. habil. Fan Dai, Dr. José Dionisio, Dr. Wolfgang Felger, Torsten Fröhlich, Dr. Thomas Frühauf, Dr. Martin Göbel (former department head, now with GMD), Dr. Helmut Haase, Elke Hergenröther, Udo Jakob, Dr. Kennet Karlsson, Christian Knöpfle, Wol- fram Kresse, Stefan Lehmann, Bernd Lutz, Mimi Lux, Wolfgang Müller, Dr. Alexander del Pino, Alexander Rettig, Marcus Roth, Frank Schöffel, Dr. Florian Schröder, Andrzej Trembilski, Dr. Matthias Unbescheiden, Gerrit Voss, Jens Weidenhausen, and Dr. Rolf Ziegler. Our secretary Renate Gröpler deserves special mention, because she has shielded me from quite a bit of paper work. A4 is simply the best. I also would like to thank all my research assistants and students for their efforts (roughly in chronological order): Reiner Schäfer, Andreas Flick, Axel Feix, Jochen Ehnes, Andreas Giersig, Andreas Zieringer, Van Do, Stefan Hoppe, Andreas Hess. Last but not least, my wholehearted thanks go: to my wife Biggi, for her pa- tience, for drawing some of the figures, and for proof-reading the whole thing (it is understood that all bugs still persisting are mine, not hers); to my little daughter Mirjam, who has witnessed the final phase and cheered it up with iii her cuteness; to my parents and brothers, for showing the way in life and for always being there. Gabriel Zachmann, Darmstadt, Mai 2000 iv Contents 1 Introduction 1 1.1 Architecture of VR systems ...................... 5 1.2 Overview ................................. 6 2 Simulation of Virtual Environments 9 2.1 Describing human-computer interaction ............... 10 2.1.1 User-interface management systems ............. 10 2.1.2 Transition networks ...................... 12 2.1.3 Context-free grammars .................... 14 2.1.4 Event languages ........................ 15 2.1.5 Interaction trees ......................... 15 2.1.6 Expressive power of the notations .............. 16 2.1.7 Scripting languages ...................... 17 2.2 Authoring virtual environments ................... 18 2.2.1 Design premises ........................ 20 2.2.2 Other VR systems ........................ 21 2.2.3 The AEIO paradigm ...................... 23 2.2.4 Semantic attributes, object lists, and object groups ..... 24 2.2.5 Grammar ............................ 27 2.2.6 Time ............................... 27 2.2.7 Inputs and events ........................ 28 2.2.8 A collection of inputs ...................... 30 2.2.9 Actions .............................. 32 2.2.10 A Collection of Actions .................... 33 2.3 Examples ................................. 38 2.4 Implementation ............................. 39 2.4.1 Distributing the system .................... 42 2.4.2 The three layers of authoring ................. 43 v CONTENTS 3 Collision Detection 47 3.1 The setting ................................ 48 3.1.1 The simulation loop ...................... 48 3.1.2 Requirements and characterization .............. 48 3.1.3 Object Representations ..................... 50 3.1.4 Definitions ............................ 51 3.2 The basic operation ........................... 52 3.3 Bounding-box pipelining ........................ 54 3.3.1 Good and bad cases ...................... 58 3.4 Convex polytopes ............................ 59 3.4.1 Static algorithms ........................ 60 3.4.2 Incremental convex algorithms ................ 62 3.4.3 Separating Planes ........................ 62 3.4.4 A simplified Lin-Canny algorithm .............. 70 3.5 Hierarchical collision detection .................... 72 3.5.1 Outline of hierarchical algorithms .............. 73 3.5.2 Optimal BV hierarchies .................... 77 3.5.3 The cost of hierarchies ..................... 79 3.5.4 The BoxTree ........................... 79 3.5.5 BoxTree traversal by clipping ................. 80 3.5.6 BoxTree traversal by re-alignment .............. 83 3.5.7 Constructing the BoxTree ................... 86 3.5.8 Oriented boxes ......................... 92 3.5.9 Discretely oriented polytopes ................. 94 3.5.10 Comparison of four hierarchical algorithms ......... 111 3.5.11 Incremental hierarchical algorithms ............. 113 3.6 Non-hierarchical algorithms ...................... 120 3.6.1 Points and Voxels ........................ 120 3.7 Flexible Objects ............................. 122 3.7.1 The “grow-shrink” algorithm ................. 122 3.7.2 Sorting .............................. 126 3.8 The object level ............................. 128 3.8.1 Other approaches ........................ 129 3.8.2 Bounding Volumes ....................... 131 3.8.3 Space-indexing data structures ................ 132 3.8.4 Octrees .............................. 134 3.8.5 Grids ............................... 138 3.8.6 Comparison of grid and octree ................ 140 3.8.7 Comparison of grid and separating planes ......... 141 vi CONTENTS 3.8.8 Combining grid and separating planes ........... 142 3.9 The collision detection pipeline .................... 142 3.10 Parallelization .............................. 145 3.10.1 Coarse-grain parallelization .................. 145 3.10.2 Fine-grain parallelization ................... 147 3.11 Implementation issues ......................... 148 3.11.1 Requirements .......................... 149 3.11.2 Time-stamping ......................... 149 3.11.3 The CPU cache ......................... 150 3.11.4 Concurrent collision detection ................ 152 4 Interacting with Virtual Environments 153 4.1 VR devices ................................ 154 4.1.1 Input device abstraction .................... 154 4.1.2 The data pipeline ........................ 159 4.1.3 Dealing with lag ........................ 159 4.2 Processing input data .......................... 160 4.2.1 Posture recognition ....................... 160 4.2.2 Voice input ........................... 164 4.3 Tracking ................................. 166 4.3.1 Filtering ............................. 166 4.3.2 Correction of magnetic tracking errors ............ 175 4.3.3 Scattered data interpolation .................. 181 4.3.4 Hardy’s Multiquadric ..................... 187 4.4 Navigation ................................ 201 4.4.1 Controlling the cart and camera ............... 204 4.4.2 Human factors ......................... 205 4.4.3 Constraints ........................... 205 4.4.4 A model of the head ...................... 206 4.4.5 Implementation ......................... 209 4.5 Interaction techniques ......................... 209 4.5.1 Virtual buttons and menus .................. 209 4.5.2 Selection ............................. 213 4.5.3 Grasping ............................. 215 4.5.4 Sliding .............................. 221 vii CONTENTS 5 Applications 235 5.1 Virtual prototyping ........................... 235 5.1.1 From rapid prototyping to virtual prototyping ....... 236 5.1.2 Definitions of virtual prototyping .............. 238 5.1.3 The right display ........................ 238 5.1.4 Other VP applications ..................... 239 5.1.5 The virtual seating buck .................... 241 5.1.6 Exchanging an alternator ................... 242 5.2 Assembly simulation .......................... 243 5.2.1 Scenarios ............................. 244 5.2.2 Interaction Functionality .................... 245 5.3 Immersive Investigation of CFD Data ................ 250 5.4 Shows ................................... 251 6 Epilogue 253 6.1 Summary ................................. 253 6.2 Future directions ............................ 259 Bibliography 263 About ... 295 Color Plates 297 viii Chapter 1 Introduction Sie sitzen schon, mit hohen Augenbrauen, gelassen da und möchten gern erstaunen. GOETHE, Faust, Vorspiel auf dem Theater lthough research in virtual reality has been done for over 10 years,1 only a Afew years ago the non-academic world started to evaluate its use to solve real-world problems. As of this writing, virtual reality is in the process of leav- ing the realm of purely academic research. Among others, the automotive in- dustry is evaluating its potential in design, development, and manufacturing processes [DR86, DFF+96]. In fact, the automotive industry has been among the first, but others, such as suppliers, have begun to evaluate VR, too. While simulators (flight simulators in particular) have been in regular use for several