Master Thesis
Total Page：16
File Type：pdf, Size：1020Kb
Master Thesis Uli Meyer Virtual environments - a sensorimotor framework for perceptual design Faculty of Design, Media and Information - Department of Media Technology Time-dependent media: Sound, Vision, Games Uli Meyer Virtual environments - a sensorimotor framework for perceptual design Master Thesis HAW Hamburg Department of Media Technology Faculty of Design, Media and Information - Time-dependent media: Sound, Vision, Games Supervisors: Prof. Dr. Boris Tolg Dr. Susanne Draheim Hamburg, Germany, April 2018 Abstract Understanding the parameters and underlying rules of perception of high quality, complex VR environments has become more important since the availability of consumer market, real-time rendered VR applications. VR is a medium that strongly affects the body, and due to its medium specificity, creates an experience of place illusion (Slater, 2009) that poses unique challenges for the environment designer. The study develops a sensorimotor framework for the underlying rules of perception, or sensorimotor contingencies, and the basic parameters, or sensorimotor affordances, of VR environments. It integrates the framework with a model of perceptual design (Ward, 2015), by describing how environmental affordances create information, simulation and spatial-temporal structuring in VR environments. The study shows that the framework can be used to plan and analyse different types of complex VR environments, both for scientific purposes and for day-to-day design decision. It can help to clarify research questions, and integrate existing and future research on VR environments. Keywords: virtual reality, virtual environments, sensorimotor contingencies, affordances, perceptual design, conceptual metaphor. Acknowledgments First of all my thanks go to my supervisors, Prof. Dr. Boris Tolg, SIMLab, Faculty of Life Sciences at the University of Applied Sciences (HAW), Hamburg, and to Dr. Susanne Draheim, CSTI, Department of Engineering and Computer Science, HAW Hamburg, for their continuing support and helpful input during the writing of this thesis. I thank Prof. Gunther Rehfeld and Prof. Ralf Hebecker of the Department of Media Technology/Time-dependent media: Sound, Vision, Games, HAW Hamburg, for creating the inspiring environment that made this thesis possible in the first place. Big thanks also goes to the team of the CSTI lab and “think tank” at the HAW Hamburg for providing the context, technology and projects that form the basis of this thesis - and for their sustained flow of ideas, encouragement, and game & pizza parties, especially to Prof. Dr. Kai von Luck, Jonathan Becker, Jessica Broscheit, Tobias Eichler, André Jeworutzki and Martin Kohler. Further thanks goes to Prof. Mel Slater, Immersive Virtual Environments Laboratory, University College, London and University of Barcelona, and to Mark Ward, University of Canberra, for their generous help and input; to Prof. Dr. Vera Schorbach, Department of Mechanical Engineering & Production, HAW Hamburg, for the fun cooperation and scientific input during the Wind Turbine project; to the team of CeNak (Center of Natural History, Hamburg), especially Prof. Dr. Matthias Glaubrecht and Dr. Lioba Thaut, for their generous supply of scan data and scientific support during the development of the Fin Whale prototype; to Dr. Heinrich Mallison for his expertise on scanning technology; and last but not least, to the games developer Crazy Bunch, especially Heiner Schmidt for their technical support and lots of cookies during the development of the Outsider prototype. Table of Contents Table of Contents ............................................................................................................. 1 1. Introduction .................................................................................................................. 7 2. Theoretical background and terminology ................................................................... 11 2.1. VR Technology ..................................................................................................... 11 2.1.1. Spatially immersive technologies.................................................................... 12 2.1.2. Presence and Place Illusion (PI) ...................................................................... 16 2.1.3. Plausibility Illusion (Psi) and levels of immersion ......................................... 17 2.2. Sensorimotor models of experience ...................................................................... 18 2.2.1. Self-movement and the sensorimotor loop...................................................... 19 2.2.2. Sensorimotor Affordances (SMA) .................................................................. 19 3. A sensorimotor framework for VR environments ...................................................... 22 3.1. Perceptual design .................................................................................................. 22 3.2. Sensorimotor information ..................................................................................... 23 3.2.1. Self-perception ................................................................................................ 24 3.2.2. Environment perception .................................................................................. 25 3.2.3. Perception of self-movement .......................................................................... 27 3.3. Simulation ............................................................................................................. 27 3.3.1. “Filling in”....................................................................................................... 28 3.3.2. Action activation and neural “mirroring”........................................................ 28 3.3.3. Cognitive mapping .......................................................................................... 29 3.4. Spatial-temporal structuring ................................................................................. 32 3.5. Sensorimotor affordances ..................................................................................... 34 3.5.1. Sensorimotor affordances: position, rotation, scale ........................................ 37 3.5.2. Sensorimotor affordances: layout.................................................................... 38 3.5.3. Sensorimotor affordances: surface and light ................................................... 43 3.5.4. Sensorimotor affordances: Self-movement, animation, interaction ................ 46 3.5.5. Additional input: sound, haptics, chemical senses .......................................... 49 3.6. Sensorimotor framework ...................................................................................... 51 4. Implementation ........................................................................................................... 52 4.1. Wind Turbine ........................................................................................................ 52 4.2. Fin Whale .............................................................................................................. 55 0 TABLE OF CONTENTS 6 4.3. The Outsider ......................................................................................................... 58 4.4. Rotation Room ...................................................................................................... 61 5. Discussion ................................................................................................................... 63 5.1. Position, scale, layout and self-movement ............................................................ 63 5.2. Balance and animated movement ......................................................................... 65 5.3. Surface, lighting and interaction ........................................................................... 66 5.4. Sound and haptics ................................................................................................. 68 6. Conclusion .................................................................................................................. 69 6.1. Summary ............................................................................................................... 69 6.2. Future research ...................................................................................................... 70 7. References .................................................................................................................. 72 8. List of figures ............................................................................................................. 75 1 INTRODUCTION 7 1. Introduction While real-time rendered Virtual Reality (VR) technology has been used in research labs for several decades, it has only been available as a mass medium since the introduction of consumer market headsets such as the Oculus Rift or the HTC Vive in 2015. In research applications, the role of VR environments is often purely functional, posing few problems for the designer. But in commercial applications, be they product simulations, computer games or even scientific visualisations for a larger audience, questions of realism, quality and style become salient. Real-time rendered VR has a strong impact on perception and physiological processes. Even early demos for the Oculus Rift Developer Kit (DK2), such as the Rollercoaster Demo affected user balance. The impact is explained by the fact that VR environments create “presence” or “place illusion” (Slater,