Walk-Centric User Interfaces for Mixed Reality

Walk-Centric User Interfaces for Mixed Reality Wallace Santos Lages Dissertation submitted to the faculty of the Virginia Polytechnic Institute and State University in partial fulfillment of the requirements for the degree of Doctor of Philosophy In Computer Science and Applications Doug A. Bowman, Committee Chair Joe Gabbard Tobias Höllerer Chris L. North Nicholas F. Polys June 22, 2018 Blacksburg, Virginia Keywords: Augmented Reality, Virtual Reality, 3D User Interfaces Copyright © 2018 by Wallace Santos Lages Walk-Centric User Interfaces for Mixed Reality Wallace Santos Lages ABSTRACT Walking is a natural part of our lives and is also becoming increasingly common in mixed reality. Wireless headsets and improved tracking systems allow us to easily navigate real and virtual environments by walking. In spite of the benefits, walking brings challenges to the design of new systems. In particular, designers must be aware of cognitive and motor requirements so that walking does not negatively impact the main task. Unfortunately, those demands are not yet fully understood. In this dissertation, we present new scientific evidence, interaction designs, and analysis of the role of walking in different mixed reality applications. We evaluated the difference in performance of users walking vs. manipulating a dataset during visual analysis. This is an important task, since virtual reality is increasingly being used as a way to make sense of progressively complex datasets. Our findings indicate that neither option is absolutely better: the optimal design choice should consider both user’s experience with controllers and user’s inherent spatial ability. Participants with reasonable game experience and low spatial ability performed better using the manipulation technique. However, we found that walking can still enable higher performance for participants with low spatial ability and without significant game experience. In augmented reality, specifying points in space is an essential step to create content that is registered with the world. However, this task can be challenging when information about the depth or geometry of the target is not available. We evaluated different augmented reality techniques for point marking that do not rely on any model of the environment. We found that triangulation by physically walking between points provides higher accuracy than purely perceptual methods. However, precision may be affected by head pointing tremors. To increase the precision, we designed a new technique that uses multiple samples to obtain a better estimate of the target position. This technique can also be used to mark points while walking. The effectiveness of this approach was demonstrated with a controlled augmented reality simulation and actual outdoor tests. Moving into the future, augmented reality will eventually replace our mobile devices as the main method of accessing information. Nonetheless, to achieve its full potential, augmented reality interfaces must support the fluid way we move in the world. We investigated the potential of adaptation in achieving this goal. We conceived and implemented an adaptive workspace system, based in the study of the design space and through user contextual studies. Our final design consists in a minimum set of techniques to support mobility and integration with the real world. We also identified a set of key interaction patterns and desirable properties of adaptation-based techniques, which can be used to guide the design of the next-generation walking-centered workspaces. Walk-Centric User Interfaces for Mixed Reality Wallace Santos Lages GENERAL AUDIENCE ABSTRACT Until recently, walking with virtual and augmented reality headsets was restricted by issues such as excessive weight, cables, tracking limitations, etc. As those limits go away, walking is becoming more common, making the user experience closer to the real world. If well explored, walking can also make some tasks easier and more efficient. Unfortunately, walking reduces our mental and motor performance and its consequences in interface design are not fully understood. In this dissertation, we present studies of the role of walking in three areas: scientific visualization in virtual reality, marking points in augmented reality, and accessing information in augmented reality. We show that although walking reduces our ability to perform those tasks, careful design can reduce its impact in a meaningful way. To my wife, my parents, and my family. iv Acknowledgments This work would not be possible without the support I received from many people over these four years. They allowed me to focus on my research, navigate the complexities of a new culture, have fun, and grow along the way. Thank you! I owe a special thanks to my advisor Doug Bowman, for all the enjoyable discussions and for the incentive to keep pushing forward when things seemed difficult. Thanks for being an excellent mentor, an exemplar scientist and person. I am also very grateful to David Laidlaw. David made himself available to listen and guide me during my whole second year. Even though none of that work became part of this dissertation, it was part of the process that lead me to it. I also would like to acknowledge: My committee, Joe Gabbard, Tobias Höllerer, Chris L. North and Nicholas F. Polys for the guidance, encouragement, and hard questions. The 3DI group members: Bireswar Laha, Felipe Bacim, Mahdi Nabiyuni, Panagiotis Apostolellis, Run Yu, Lawrence Warren, Yuan Li, Javier Tibau, Leonardo Soares, and Lee Lisle; for the all the discussions and piloting of my studies. Sharon Kinder-Potter, Andrea Kavanaugh, Cal Ribbens, Phyllis Newbill, Tanner Upthegrove, Zach Duer, Melissa Wyers, Lisa Jansen, Holly Williams, and Teresa Hall, for the administrative and technical support that made my life so much easier. Funding support from the Brazilian Council for Scientific and Technological Development, the National Science Foundation, and from the Immersive Sciences program in the Office of Naval Research. v Attributions Doug A. Bowman is the Frank J. Maher Professor of Computer Science and Director for the Center for Human-Computer Interaction at Virginia Tech. As my PhD advisor, he was my collaborator and co-author on all papers forming this dissertation. Tobias Höllerer is a Professor of Computer Science at the University of California, Santa Barbara. He contributed on the writing of the paper forming the Chapter 2 of this dissertation. Yuan Li is a Master in Computer Science and Research Assistant in the 3D interaction group at Virginia Tech. He was a co-author in the research described in Chapter 2 of this dissertation. Chapter 2 originally appeared in: Lages, W. S., & Bowman, D. 2018. Move the Object or Move Myself? Walking vs. Manipulation for the Examination of 3D Scientific Data. Frontiers in ICT, 5, pp. 15. DOI=10.3389/fict.2018.00015 vi Table of Contents 1 INTRODUCTION 1 1.1 WALK-CENTRIC USER INTERFACES 2 1.2 RESEARCH GOALS AND SCOPE 6 1.3 RESEARCH QUESTIONS 7 1.4 METHODS AND SCOPE 8 1.5 MAIN CONTRIBUTIONS 9 1.6 THE STRUCTURE OF THIS DISSERTATION 10 2 UNDERSTANDING WALKING TRADEOFFS IN VIRTUAL REALITY 11 2.1 INTRODUCTION 11 2.2 BACKGROUND AND RELATED WORK 12 2.3 OVERALL EXPERIMENTAL DESIGN 17 2.4 PILOT STUDY 21 2.5 MAIN EXPERIMENT 23 2.6 CONCLUSION 37 3 MODEL-FREE MARKING IN AUGMENTED REALITY 38 3.1 INTRODUCTION 38 3.2 BACKGROUND AND RELATED WORK 39 3.3 GEOMETRY-BASED MARKING TECHNIQUES 41 3.4 EXPERIMENT I: PERCEPTUAL VS. GEOMETRIC MARKING 41 3.5 TECHNIQUES 42 3.6 APPARATUS 43 3.7 TASK AND ENVIRONMENT 44 3.8 PARTICIPANTS AND PROCEDURE 45 3.9 RESULTS 46 3.10 DISCUSSION 48 3.11 EXPERIMENT II: IMPROVING PRECISION WITH MULTI-SAMPLE MARKING 50 3.12 EXPERIMENT DESIGN 53 3.13 APPARATUS 54 3.14 PARTICIPANTS AND PROCEDURE 54 3.15 RESULTS 55 3.16 DISCUSSION 57 3.17 QUALITATIVE USER STUDY 57 3.18 CONCLUSIONS 59 4 MANAGING WORKSPACES IN AUGMENTED REALITY 60 4.1 INTRODUCTION 60 4.2 RELATED WORK 61 4.3 RESEARCH APPROACH 67 4.4 DEFINING THE DESIGN SPACE 68 4.5 AR ADAPTIVE WORKSPACE - VERSION I 75 4.6 EVALUATING THE AR WORKSPACE 80 4.7 STUDY I 85 4.8 AR ADAPTIVE WORKSPACE – VERSION II 100 vii 4.9 STUDY II 103 4.10 DISCUSSION 109 4.11 DESIGNING FOR WALKING 112 4.12 LIMITATIONS 113 4.13 CONCLUSION 115 5 CONCLUSION AND FUTURE WORK 116 5.1 SUMMARY OF THE CONTRIBUTIONS 116 5.2 FUTURE WORK 117 REFERENCES 121 APPENDIX A: TABLES 132 APPENDIX B: EXPERIMENTAL DOCUMENTS STUDY 1 133 APPENDIX C: EXPERIMENTAL DOCUMENTS STUDY 2 154 APPENDIX D: EXPERIMENTAL DOCUMENTS STUDY 3 157 viii List of Figures Figure 2-1: Examples of datasets from the Complex (left) and Simple (right) groups. 18 Figure 2-2: View of the Experimental Environment. 20 Figure 2-3: Task completion time for each participant in the two conditions.Left: Simple dataset, Right: Complex datasets. 28 Figure 2-4: Participants grouped according to high (H) and low (L) scores. 28 Figure 2-5: Time Difference Between Walking and Non-Walking Conditions. 32 Figure 2-6: Time in the Walking Condition. 32 Figure 3-1: Perceptual and Geometric Techniques. 42 Figure 3-2: Experimental apparatus. 44 Figure 3-3: Photo of the experimental site taken from the participant’s point of view. 45 Figure 3-4: Experiment I results. Bars show the standard error at each target distance. 46 Figure 3-5: Increase in variability with distance in Experiment I. 47 Figure 3-6: The VectorCloud technique. 51 Figure 3-7: Top view of the distribution of intersections generated by two viewpoints A and B. 52 Figure 3-8: The VR experimental setup emulating the first experiment. 53 Figure 3-9: Experiment II results. 55 Figure 3-10: Variability increase for original Geometric technique and the VectorCloud 56 Figure 3-11: Variability on the view plane is also reduced in the VectorCloud technique.

Load more