Effects of Control-Display Mapping on 3D Interaction in Immersive Virtual Environments
Total Page:16
File Type:pdf, Size:1020Kb
EFFECTS OF CONTROL-DISPLAY MAPPING ON 3D INTERACTION IN IMMERSIVE VIRTUAL ENVIRONMENTS by Jialei Li A dissertation submitted to the faculty of The University of North Carolina at Charlotte in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Computing and Information Systems Charlotte 2017 Approved by: ______________________________ Dr. Zachary Wartell ______________________________ Dr. William Ribarsky ______________________________ Dr. Aidong Lu ______________________________ Dr. Paula Goolkasian ______________________________ Dr. Isaac Cho ii ©2017 Jialei Li ALL RIGHTS RESERVED iii ABSTRACT JIALEI LI. Effects of control-display mapping on 3D interaction in immersive virtual environments. (Under the direction of DR. ZACHARY J. WARTELL) There is a significant amount of research on the mapping of the interactive control space to the display space in 3D user interfaces. While most work has focused on control- display gain (or ratio), one factor that has received relatively less attention is cursor offset, which is a vector between the input device in physical space and the virtual cursor in display space. Empirical results of the efficiency and usability of a translational offset have been provided by multiple studies, but only for a particular type of interaction, on a specific display system. Anecdotal evidence suggests that results may differ on different types of tasks or in different types of VR systems. Therefore, this research focuses on designing and evaluating virtual cursor offset techniques for 3D interaction in immersive virtual environments in a more comprehensive manner. Three user studies are carried out to explore the effect of various offset techniques on a 7 degree-of-freedom navigation task in a surround-screen CAVE system, for both one-handed and two-handed interactions. Results show that the Linear Offset technique outperforms other offset techniques for exocentric travel tasks and provides superior performance under a variety of conditions. To compare the same offset techniques in an HMD environment, an evaluation on 3D object selection tasks that follows Fitts’ law model under ISO 9241-9 standard using two different input devices is first presented. The result indicates that direct selection of nearby objects with No Offset remains the most efficient and the Linear Offset technique could enhance selection performance with objects at a distance. Further, two experiments iv on unimanual and bimanual object manipulation are conducted respectively, using the same input devices. For both studies, No Offset does not reveal advantages over Linear Offset or Go-Go Offset when the object is within reach, while for out of reach condition, Linear Offset outperforms Fixed-Length and Go-Go Offset. In all three studies conducted within Oculus Rift DK2, Razer Hydra proves to be more stable and effective than Leap Motion. v DEDICATION This dissertation is dedicated to Mom and Dad. vi ACKNOWLEDGMENTS First and foremost, I want to thank my parents, for their unconditional love and endless support. Thank you for always believing in me and supporting me throughout my entire education. I would like to express my deepest thanks to my academic advisor, Dr. Zachary Wartell, for providing me with the opportunity to pursue a PhD and for his advice and guidance during my research. I deeply appreciate the valuable suggestions from members of my dissertation committee, Dr. William Ribarsky, Dr. Aidong Lu, Dr. Paula Goolkasian and Dr. Isaac Cho. In particular, I thank Dr. Isaac Cho for his help and encouragement during my PhD study in the Charlotte Visualization Center. Finally, I would like to thank all the people who have participated in my user studies and contributed to this dissertation. vii TABLE OF CONTENTS LIST OF TABLES xii LIST OF FIGURES xiii LIST OF ABBREVIATIONS xvii CHAPTER 1: INTRODUCTION 1 1.1 Immersive Virtual Environments 1 1.2 3D Interaction 4 1.3 Control-Display Mapping 6 1.4 Research Problem 8 1.5 Expected Contributions 9 1.6 Dissertation Overview 9 CHAPTER 2: BACKGROUND AND LITERATURE REVIEW 11 2.1 Immersive VE Display 11 2.1.1 Head-Mounted Displays 11 2.1.2 Surround-Screen Displays 12 2.2 3D Interaction Techniques 13 2.2.1 Object Selection 14 2.2.2 Navigation in Multi-Scale Virtual Environments 20 2.2.3 Bimanual Interaction 22 2.2.4 Mid-Air Interaction with Hand Tracking 27 2.3 Control-Display Mapping 32 CHAPTER 3: VIRTUAL CURSOR OFFSET TECHNIQUES 43 3.1 Motivation 43 viii 3.2 Fixed-Length Offset Technique 45 3.3 Go-Go Offset Technique 46 3.4 Linear Offset Technique 46 CHAPTER 4: EVALUATION ON ONE-HANDED 7DOF NAVIGATION IN A MULTI-DISPLAY VIRTUAL ENVIRONMENT 49 4.1 Introduction 49 4.2 Evaluation 50 4.2.1 Environment 50 4.2.2 Experimental Design 50 4.2.3 Procedure 53 4.2.4 Experiment 1: Unimanual 7DOF Navigation in CAVE 55 4.2.4.1 Quantitative Results 56 4.2.4.2 Subjective Preferences 58 4.2.5 Experiment 2: 7DOF Travel with Different Linear Offset Lengths 59 4.2.5.1 Quantitative Results 60 4.2.5.2 Subjective Preferences 62 4.3 Discussion 63 4.4 Conclusion 65 CHAPTER 5: EVALUATION ON BIMANUAL 7DOF NAVIGATION IN A SURROUND-SCREEN VIRTUAL ENVIRONMENT 66 5.1 Introduction 66 5.2 Bimanual Offset Technique 67 5.3 Experiment 3: Bimanual 7DOF Navigation in CAVE 70 5.3.1 Environment 71 ix 5.3.2 Experimental Design 71 5.3.3 Procedure 76 5.3.4 Quantitative Results 77 5.3.5 Subjective Results 81 5.3.6 Kinematic Results 82 5.4 Discussion 84 5.5 Conclusion 86 CHAPTER 6: EVALUATION ON 3D OBJECT SELECTION IN A HEAD- MOUNTED DISPLAY SYSTEM 88 6.1 Introduction 88 6.2 Methodology 90 6.2.1 Fitts’ Law 90 6.2.2 ISO 9241-9 92 6.3 Experiment 4: Object Selection in HMD − Razer Hydra vs Leap Motion 94 6.3.1 Apparatus 95 6.3.2 Experimental Design 96 6.3.3 Procedure 101 6.3.4 Quantitative Results 102 6.3.4.1 Movement Time 102 6.3.4.2 Error Rate 106 6.3.4.3 Effective Throughput 110 6.3.5 Subjective Preferences 115 6.4 Discussion 116 6.5 Conclusion 120 x CHAPTER 7: EVALUATION ON 7DOF OBJECT MANIPULATION IN HMD ENVIRONMENTS 121 7.1 Introduction 121 7.2 Evaluation 123 7.2.1 Experimental Design 123 7.2.2 Procedure 128 7.2.3 Experiment 5: Unimanual 7DOF Object Manipulation in HMD 129 7.2.3.1 Quantitative Results 130 7.2.3.1.1 Within Reach Condition 131 7.2.3.1.2 Out of Reach Condition 137 7.2.3.2 Subjective Preferences 141 7.2.3.3 Discussion 142 7.2.4 Experiment 6: Bimanual 7DOF Object Manipulation in HMD 145 7.2.4.1 Quantitative Results 146 7.2.4.1.1 Within Reach Condition 146 7.2.4.1.2 Out of Reach Condition 149 7.2.4.2 Subjective Preferences 151 7.2.4.3 Discussion 152 7.3 Overall Discussion 154 7.4 Conclusion 156 CHAPTER 8: CONCLUSION AND FUTURE WORK 157 8.1 Primary Contributions 157 8.2 Limitations 160 8.3 Future Work 161 xi REFERENCES 163 APPENDIX A: EXPERIMENT DOCUMENTS FOR CHAPTER 4 178 APPENDIX B: EXPERIMENT DOCUMENTS FOR CHAPTER 5 189 APPENDIX C: EXPERIMENT DOCUMENTS FOR CHAPTER 6 198 APPENDIX D: EXPERIMENT DOCUMENTS FOR CHAPTER 7 209 xii LIST OF TABLES Table 4.1: Average completion time (CT) and standard deviation (SD) of each condition in Experiment 1. 57 Table 4.2: Average completion time (CT) and standard deviation (SD) of each condition in Experiment 2. 60 Table 5.1: Average completion time (CT) and standard deviation (SD) of each condition. 77 Table 6.1: Test conditions for Offset Technique × Depth combination. 99 Table 6.2: Significant effects on movement time for both within reach and out of reach conditions. 103 Table 6.3: Significant effects on error rate for both within reach and out of reach conditions. 106 Table 6.4: Significant effects on effective throughput for both within reach and out of reach conditions. 110 Table 7.1: Significant effects on task completion time and number of controls in within reach condition. 131 Table 7.2: Significant effects on task completion time and number of controls in out of reach condition. 137 Table 7.3: The marginal means of task completion time and number of clutched maneuvers for Razer Hydra and Leap Motion in within reach and out of reach conditions. 143 Table 7.4: Significant effects on task completion time and number of effective controls in within reach condition. 146 Table 7.5: Significant effects on task completion time and number of effective controls in out of reach condition. 150 Table 7.6: The marginal means of task completion time and number of clutched maneuvers for Razer Hydra and Leap Motion in within reach and out of reach conditions. 154 xiii LIST OF FIGURES Figure 1.1: Hand and finger tracking using Leap Motion controller. 3 Figure 1.2: Interaction in virtual environments [102]. 4 Figure 3.1: Range of the user’s hand (red circle) and the 3D cursor (blue circle). 44 Figure 3.2: A side view of Figure 3.1. 44 Figure 3.3: Mapping functions of the four offset techniques. 48 Figure 4.1: Our three-side CAVE system. Polhemus Fastrak tracks the position and orientation of the user’s head and 3D input. 51 Figure 4.2: A pair of the buttonball devices. Each buttonball has three buttons on its surface and a Fastrak receiver inside of the ball to track the user’s hand position. 51 Figure 4.3: A screenshot of our virtual scene.