WITHINDOWS: A UNIFIED FRAMEWORK FOR THE DEVELOPMENT OF DESKTOP AND IMMERSIVE USER INTERFACES BY ALEX S. HILL B.S., Trinity University, 1988 M.S., University of Texas at Austin, 1992 THESIS Submitted as partial fulfillment of the requirements for the degree of Doctor of Philosophy in Computer Science In the Graduate College of the University of Illinois at Chicago, 2007 Chicago, Illinois I would like to dedicate this thesis to my grandfather, Alex S. Hill Sr., without whom I would have never had the opportunity to pursue it. iii ACKNOWLEDGEMENTS I would like to thank my thesis committee, Andrew Johnson, Daniel Sandin, Jason Leigh, Tom Moher, and Steve Jones, for their continued patience and support. I would also like to acknowledge the support of all the staff, faculty and students at the Electronic Visualization Laboratory and the School of Art and Design that have provided the support, emotional, technical and financial, that made this research possible. ASH iv TABLE OF CONTENTS CHAPTER PAGE 1. INTRODUCTION.................................................................................................................................................... 1 1.1 The Ultimate Interface .............................................................................................................................. 1 1.2 Augmenting Human Ability ....................................................................................................................... 2 1.3 The Coming Augmented Reality............................................................................................................... 3 1.4 Efficient Augmented Reality Interfaces..................................................................................................... 4 1.5 The Withindows Framework ..................................................................................................................... 5 2.0 THREE DIMENSIONAL USER INTERFACES .................................................................................................... 8 2.1 Human Computer Interaction.................................................................................................................... 8 2.1.1 Augmenting Human Ability............................................................................................................... 8 2.1.2 Virtual Reality................................................................................................................................. 10 2.1.3 The Emergence of Two Themes ................................................................................................... 11 2.2 The Role of Haptics ................................................................................................................................12 2.2.1 Interaction with the Hands ............................................................................................................. 13 2.2.2 Travel in Virtual Space................................................................................................................... 14 2.2.3 Virtual Travel Techniques .............................................................................................................. 15 2.2.4 A Separation between Virtual and Physical Reality....................................................................... 16 2.3 Proprioceptive Interaction.......................................................................................................................17 2.3.1 Selecting and Manipulating Objects .............................................................................................. 18 2.3.2 The Separation between Egocentric and Exocentric Interaction................................................... 20 2.4 Manipulation at a Distance .....................................................................................................................22 2.4.1 Distance Manipulation Techniques................................................................................................ 23 2.4.2 Corporealism.................................................................................................................................. 24 2.5 Relationship to Desktop Systems...........................................................................................................26 2.5.1 System Control .............................................................................................................................. 27 2.5.2 Usability.......................................................................................................................................... 31 3. THE COMING AUGMENTED REALITY ..............................................................................................................37 3.1 A Decontextualized Computer Revolution.............................................................................................. 37 3.2 The Emergence of Three Trends ...........................................................................................................39 3.2.1 The Geo-Referenced Internet........................................................................................................ 39 3.2.2 The Ubiquitous Internet.................................................................................................................. 40 3.2.3 The Ubiquitous Display .................................................................................................................. 41 3.3 A Truly Augmenting Reality ....................................................................................................................42 3.3.1 A Truly Universal Remote.............................................................................................................. 43 3.3.2 The Application of Computation to the Physical ............................................................................ 44 3.4 Future 3D User Interfaces ......................................................................................................................46 4. GUIDELINES FOR PRODUCTIVE 3D USER INTERFACES .............................................................................48 4.1 Domain....................................................................................................................................................48 4.1.1 Simulation ...................................................................................................................................... 49 4.1.2 Augmentation................................................................................................................................. 49 4.1.3 Augmentation Application Areas.................................................................................................... 50 4.1.4 A Superset of Two Dimensional Interfaces.................................................................................... 51 4.2 Usability ..................................................................................................................................................51 4.2.1 Novice Use..................................................................................................................................... 51 4.2.2 Expert Use ..................................................................................................................................... 52 4.2.3 Fatigue ........................................................................................................................................... 52 4.2.4 Workflow ........................................................................................................................................ 53 4.3 Efficiency.................................................................................................................................................53 4.3.1 First Person Perspective................................................................................................................ 53 4.3.2 Third Person Perspective............................................................................................................... 54 4.3.3 Input Methods ................................................................................................................................ 55 4.3.4 Symbolic Manipulations ................................................................................................................. 55 v TABLE OF CONTENTS(continued) CHAPTER PAGE 4.4 Application Development........................................................................................................................56 4.4.1 Canonical Methods ........................................................................................................................ 56 4.4.2 Single Authoring............................................................................................................................. 57 4.4.3 Universal Design............................................................................................................................ 57 4.4.4 Interface Management ................................................................................................................... 57 4.4.5 A Unifying Framework.................................................................................................................... 58 5. THE WITHINDOWS FRAMEWORK ....................................................................................................................59