Virtual Reality Based Information Systems and Their Applications in the Field of Tourism

Virtual Reality Based Information Systems and their Applications in the Field of Tourism

DISSERTATION DER ABTEILUNG INFORMATIK DER WIRTSCHAFTSWISSENSCHAFTLICHEN FAKULTÄT DER UNIVERSITÄT ZÜRICH

zur Erlangung der Würde eines Doktors der Informatik

vorgelegt von KORNEL SZABO von Langendorf (SO)

genehmigt auf Antrag von PROF. DR. PETER STUCKI PROF. DR. LEWIS E. HITCHNER

Februar 1998

Die Wirtschaftswissenschaftliche Fakultät, Abteilung Informatik, gestattet hierdurch die Drucklegung der vorliegenden Dissertation, ohne damit zu den darin ausgesprochenen Anschauungen Stellung zu nehmen.

Zürich, den 2. Juli 1997

Der Abteilungsvorsteher: Prof. Dr. L. Richter

I The best user interface is the one

that people really use.

II III Abstract

Virtual Reality (VR) environments offer a new kind of human computer as well as human human interaction. The user of a VR system can immerse himself into a computer-generated, multi-sensory, intuitively operable virtual experience space, and interact with virtual objects in real-time. In this thesis, a new model for the description and classification of VR systems is presented. The discussion of basic issues and challenges in the development, operation and maintenance of VR systems leads to a set of new approaches for VR application models and systems. The coupling of VR based systems with conventional database subsystems leads to a new architecture model for VR based information systems. This opens the potential for the design and implementation of very promising new application scenarios such as VR based geographic, medical, or facility information systems at large. Another interesting specific application field is found in tourism. A scenario that has hardly been discussed within the VR community yet. Until now, existing tourist information systems can not sufficiently meet the information-intensive, multimedia, and spatial nature of tourism related services and processes. The application of VR based tourist information systems can significantly contribute to the implementation of more appropriate solutions. By the help of a new methodology for systematic analysis and evaluation of VR systems, the system architecture, design rationales, and implemented user interface metaphors of three VR application prototypes will be presented and discussed. These are accompanied by system performance and usability experiments. The first prototype, a Virtual Realiy based Geographic Information System, implements sophisticated methods and techniques for efficient data management in real-time by applying a client- server architecture. They allow the dynamic loading of virtual environment data at runtime and optimize load strategies for increased runtime performance. User interface aspects of VR applications are discussed with the help of the other two protoypes, a cockpit based immersive driving simulator and a VR based holiday house exploration environment. Furthermore, a new software architecture is proposed to improve the integration of additional peripheral devices into an existing VR system environment and to enable the development of software components in parallel.

IV The close relationship of the three application prototypes to the field of tourism allows one to apply the findings to the design of architecture models and user interface metaphors for VR based tourist information systems.

V Zusammenfassung

Das Ziel von Virtual Reality (VR) Umgebungen ist es, dem Nutzer eine neue Art der Mensch Maschine bzw. Mensch Mensch Interaktion zu ermöglichen. Dem Nutzer eines VR Systems wird die Möglichkeit geboten, in einen computer- generierten, multi-sensorischen sowie intuitiv zu bedienenden virtuellen Experimentier- und Erfahrungsraum “einzutauchen” und dort in Echtzeit mit virtuellen Objekten zu interagieren. In der vorliegenden Arbeit wird ein neues Modell vorgestellt, das zur Beschreibung und Klassifikation von VR Systemen eingesetzt werden kann. Die Diskussion der Grund- problematiken bei der Entwicklung, dem Betrieb und der Wartung von VR Systemen führt zu einer Reihe neuer Modell- und Systemansätze für VR Anwendungsum- gebungen. Die Kopplung von Virtual Reality basierten Systemen mit Datenbankverwaltungs- komponenten führt zu einem neuen Architekturmodell für Virtual Reality basierte Informationssysteme. Dies eröffnet das Potential für die Realisierung von vielver- sprechenden Anwendungsszenarien wie zum Beispiel VR basierte Geo-, Medizin- oder Anlageninformationssysteme. Als äusserst zukunftträchtiges Anwendungsfeld für Virtual Reality basierte Informationssysteme gilt das Gebiet des Tourismus. Solche Anwendungsszenarien wurden bisher kaum in der VR Fachwelt diskutiert. Dem informationsintensiven, multimedialen sowie raumbezogenen Charakter von Dienst- leistungen und Prozessen im Tourismus konnte bislang nur ungenügend durch ent- sprechende Tourismusinformationssysteme entsprochen werden. Der Einsatz von Virtual Reality basierten Tourismusinformationssystemen kann einen wesentlichen Beitrag dazu leisten, angemessenere Lösungen zu realisieren. Mit Hilfe einer neuen Methodologie zur systematischen Untersuchung und Evaluation von VR Systemen werden die implementierte Systemarchitektur, verfolgte Entwurfs- prinzipien sowie eingesetzte Benutzerschnittstellenmetaphern von drei realisierten An- wendungsprototypen vorgestellt und diskutiert. Dazu werden ausgiebige Experimente zur Messung der Laufzeit und Benutzbarkeit durchgeführt. In der Beschreibung des ersten Prototypsystems wird aufgezeigt, wie über eine Client-Server-Architektur zur Laufzeit geographische Szenendaten dynamisch nachgeladen werden können. Die angewandten Methoden und Techniken für ein effizientes Datenmanagement unter Echtzeitbedingungen werden diskutiert, um adäquate Ladestrategien für die Laufzeit-

VI optimierung postulieren zu können. Aspekte der Benutzerschnittstelle bei VR Umgebungen werden mit Hilfe der anderen beiden Prototypen, einem Cockpit basierten immersiven Fahrsimulator und einer VR basierten Anwendungsumgebung zur Erkundung eines Ferienhauses, diskutiert. Ferner wird eine neue Softwarearchitektur vorgestellt, die es erlaubt, Peripheriegeräte robuster in ein bestehendes VR System einzubinden und die Entwicklung von Softwarekomponenten parallel durchzuführen. Der starke Bezug der drei Prototypanwendungen zum Gebiet des Tourismus ermöglicht, aus den gewonnen Ergebnissen Architekturmodelle und Benutzerschnittstellen- metaphern für VR basierte Tourismusinformationssysteme zu entwerfen.

VII Acknowledgements

This thesis has been elaborated during my stay at the MultiMedia Laboratory of the Department of Computer Science at the University of Zurich. I would like to thank to all those who contributed to the completion of this thesis. I would like to give special thanks to my thesis supervisor Prof. Dr. Peter Stucki for his scientific competence, dedicated assistance, guidance, open door policy, and exchange of many fruitful ideas. I would like to thank Prof. Dr. Lewis. E. Hitchner for acting as co-referee and Prof. Dr. Peter Widmayer for supporting the ViRGIS project and his constructive cooperation. Thanks are due to all my colleagues, namely to Dr. Philipp Ackermann, Dr. Martin Bichsel, Dr. Martin Dürst, Dr. Matthias Rauterberg, Dr. Robert Schleich, Dr. Stefanie Teufel, Dominik Eichelberg, Christoph Naef, Renato Pajarola, and Christian Stern for their constructive discussions and precious suggestions, and William Harris for proof- reading the thesis. Thanks are also due to all of the students who have contributed by their projects and master thesis work to the concepts and validations presented in this thesis, namely Stefan Amand, Patrick Aschwanden, Felix Kägi, Jonas Kurth, Fabian Honegger, Roberto E. Micieli, and Martin Roth. Furthermore, I would like to thank Enrico Solca, Beat Rageth, and Michel Hafner for providing system support and helping me out with scarce resources, and Lukas Meyer from TV-Uni for his help in video recording of the VR application prototypes. Finally, I would thank my parents for their continuous support and patience and all my friends who have shown interest for my projects. This work has been supported by the Swiss National Science Foundation and the University of Zurich.

Brüttisellen, Mai 1997 Kornél Szabó

VIII Contents

1. Introduction 1 1.1 Interactive 3D Simulation and Virtual Reality 1 1.1.1 Hardware Development and System Architecture 1 1.1.2 Software Development 2 1.1.3 Methodologies for Virtual Reality Application Development 3 1.1.4 User Interfaces Metaphors 3 1.2 Motivation and Objectives 5 1.3 Thesis Organization 6

2. Introduction to Virtual Reality Technology and Systems 9 2.1 Terminology and Paradigms 9 2.2 Basic Building Blocks 12 2.2.1 Image Processing and Generation 14

2.2.2 Sound Processing and Generation 15 2.2.3 Haptic and Kinesthetic Information Processing and Generation 18 2.2.4 Interfacing Framework 19 2.2.5 Virtual Environment Modeling and Management 20 2.2.6 Peripheral IO Devices and Subsystems 21 2.2.7 Workload Balancing and Distribution 22 2.2.8 Local and Global Networking 23 2.3 Virtual Reality Modeling Language 24 2.4 Major Issues and Challenges 25 IX 2.4.1 Real-Time Interaction and Synchronization of Perceptual Channels 25 2.4.2 Intuitive and User-Friendly Human Computer Interaction 26 2.4.3 Effective Scenebase Modeling and Management 28 2.4.4 Eased System Development, Setup and Maintenance 29 2.4.5 Economical Aspects 30 2.5 Summary 30

3. Virtual Reality Based Information Systems 33 3.1 Information Systems 33 3.1.1 Terminology 34 3.1.2 Taxonomy and Classification Concepts 34 3.2 Models for Virtual Reality Based Information Systems 36 3.2.1 Virtual Reality Based Information System Configurations 37 3.3 Application Domains and Scenarios 40 3.3.1 Virtual Reality Based Geographic Information Systems 40 3.3.2 Virtual Reality Based Medical Information Systems 41 3.3.3 Virtual Reality Based Facility Information Systems 42 3.4 Summary 44

4. Methodology for the Evaluation of Virtual Reality Applications 45 4.1 Overview 45 4.2 Task and Application Space 46 4.3 Design Space 47 4.3.1 Component View 47 4.3.2 Perceptual View 49 4.3.3 Development View 53 4.3.4 Metaphor View and Space 55 4.4 Metrics Space 55

X 4.4.1 Objective Metrics 56 4.4.2 Subjective Metrics 56 4.5 System and Usability Testing 57 4.6 Summary 58

5. Development and Evaluation of Virtual Reality Application Prototypes 61 5.1 Task and Application Spaces 62 5.2 Design Spaces 63 5.2.1 Acquisition and Management of Large Volumes of Data 63 5.2.2 Setup and Operation of a Cab Based Immersive Environment 69 5.2.3 Multi-Port Service Architecture for High Degree of Functional Availability of Peripheral Devices 72 5.2.4 VR Application Prototyping with a High-Level Authoring Environment 77 5.2.5 System Development Models 77 5.3 Metaphor Spaces 79 5.3.1 Desktop Virtual Reality Based Fly-Over Metaphor 79 5.3.2 Cab Virtual Reality Based Drive-Through Metaphor 81 5.3.3 Desktop Virtual Reality Based Walk-Through and Direct 3D Manipulation Metaphor 82 5.4 System and Usability Testing Applying Different Metrics 83 5.4.1 Load Times for Client-Server Based Virtual Environments 83

5.4.2 Cockpit Design and Degree of Immersion in a Cab Virtual Reality Based Drive-Through System 87 5.4.3 Evaluation of Different IO Device Setups 88 5.5 Discussion 91 5.5.1 Scenebase Acquisition, Modeling, Tuning and Management 91 5.5.2 System Usability and Acceptance 92 5.5.3 System Development 94 5.5.4 System Configuration, Operation and Maintenance 94

XI 5.5.5 Economical Aspects 95 5.6 Summary 96

6. Virtual Reality Based Tourist Information Systems 99 6.1 Task and Application Space 99 6.1.1 Tourism Sector 99 6.1.2 Tourism Related Information and Services 102 6.1.3 Tourist Information Systems 103 6.1.4 Major Issues and Challenges 104 6.1.5 Virtual Reality Based Tourist Information Systems 106 6.2 System Models and User Interface Metaphors 107 6.2.1 Desktop Virtual Reality Based Tourist Information System 108 6.2.2 Desksize Virtual Reality Based Tourist Information System 112 6.2.3 Roomsize Virtual Reality Based Tourist Information System 114 6.3 Summary 116

7. Conclusion 117 7.1 Summary 117 7.2 Outlook 119

Appendix A - ViRGIS Scenebase 121

Appendix B - VR Project Development Phase Models 123

Appendix C - ViRGIS System Performance 127

Bibliography 131

Curriculum Vitae

XII

Chapter 1

Introduction

This chapter gives an introduction to the field of interactive 3D Simulation and Virtual Reality. The development and status of Virtual Reality hardware and software technology is described. Methodologies for Virtual Reality Application Development and the evolution of user interfaces are also discussed. Finally, the motivation and objectives of the thesis are formulated.

1.1 Interactive 3D Simulation and Virtual Reality

Continuously increasing computing power and storage capacity, advances in sensing and tracking technologies, new algorithms for information visualization, and the availability of integrated multimedia platforms led to a new paradigm for human computer interaction, called Virtual Reality (VR). This enabling technology aims to overcome disadvantages of conventional audio-visual media spaces, such as limited presentation of information, limited exploration and limited interactive operations [Bly93] [Gave92] by immersing the user into a virtual, multi-sensory computing environment.

1.1.1 Hardware Development and System Architecture Already in the 70s, flight simulation systems were used to augment the training efficiency of pilots. They served as an add-on, and not as a substitution, to existing training methods. Simulataneously, the US army began to experiment with heads-up displays (HUDs) to project flight parameters like altitude or velocity into the pilot´s field of view. Those systems were mostly prototype-like and involved a huge amount of R&D investments. Based on the availability of powerful workstations, a new generation of system platforms for 3D visual simulation, supporting new 3D peripheral devices, 2 Introduction

were prototyped in the mid 80s. At this time, many NASA and academic spin-off companies such as Fakespace Inc., Crystal River Engineering Inc., Sense8 Inc. or Telepresence Research were founded and became the first generation commercial VR equipment manufacturers. In 1989, a first commercially available VR environment was publicly announced by VPL Research Inc.. After this, more and more VR start-up companies have been founded, and the public awarness for the VR industry has simultaneously grown. Based on the tremendous experiences gained from flight simulator technology, modern VR R&D aimed to transform this knowledge into new application domains like industrial design, architectural walk-throughs, and surgical planning. Many academic and industrial VR system prototypes have shown, that there still exist shortcomings of current VR platforms. These can be summarized as follows [Dam94]: • insufficient computing power for the rendition of large and complex environments, • generally poor resolution and uncomfortable to use peripheral devices, • significant latency of peripheral devices, • limited range and environmental interference problems of tracking devices, • poor or no support for haptic dimensions, and • generally high costs of investment and maintenance for VR systems. System solutions that overcome many of these disadvantages, are based on a distributed multi-processing system architecture. This is especially true for system environments that incorporate multi-modal interaction with a high number of degrees of freedom as well as dynamic and user interaction dependent execution paths of virtual objects [Pau94a] [Brys94a].

1.1.2 Software Development At present, the acquisition and modeling of virtual objects and their behaviour is considered as one of the most time consuming and expensive parts during a VR application development cycle [Kala93]. There are no widely accepted virtual environment database representation standards yet. Standards are needed to guarantee the ease of integration and reuse of existing objects for new VR applications. The need for high- level and coherent application development environments is obvious. Further, the extension or replacement of hardware and software components is often coupled with intensive work because of missing openness of the existing VR system architectures. Moving from one VR system platform to another causes tremendous difficulties, espe- Interactive 3D Simulation and Virtual Reality 3

cially for the porting of interaction mechanisms and techniques. Architectures such as the Virtual Environment Operating System (VEOS) [Brick90] are early approaches to offer solutions for these problems. Basic tasks such as image generation, tracking and updating of user data can be split into separate processes and distributed across different platforms to gain better overall system performance. The communication between those processes is realized over shared memory areas or message passing mechanisms. Such a client-server architecture also meets the requirements for optimal resource allocation and sharing of a cooperative system environment. Existing VR application development toolkits are mostly based on a set of utility programs and a programming library. Only a few of these VR toolkits incorporate high- level authoring tools for rapid protoyping of VR environments. Further work has to investigate high-level application development software tools and frameworks to meet the requirements for real-time performance, easily extendible and reusable software components, ease of object geometry modeling and model capture [Dam94]. Vertical integration of different software components and the flexibility in VR system configuration is a key requirement for future VR application development systems [Pau94b].

1.1.3 Methodologies for Virtual Reality Application Development Although VR applications are making their way into the market, there exist few or no good design concepts for VR application development. Traditional application design paradigms such as the waterfall model in software engineering are inadequate. They do not consider the highly iterative development process needed for system tuning. The design process is primarly driven by application dependent tasks and stringent performance requirements. The capabilities of the image generation subsystem strongly determine the effect of 3D presence and immersion within a virtual environment. To increase the degree of immersion it is also important to maintain fast system responses to user actions, low latencies for real-time interaction, high accuracy of user control, and to avoid lag-induced oscillations of perceptual feedback to user manipulations. This way, unwanted side effects such as motion sickness and nausea can be avoided. An iterative design paradigm for the development of user-adequate VR applications is needed, which combines top-down design for task specification with bottom-up design for performance tuning and optimization [Brys94a].

1.1.4 User Interfaces Metaphors The development of User Interfaces (UIs) and paradigms for human computer interaction is closely related to the evolution of computer hardware and IO device 4 Introduction

technology [Niel93]. Figure 1-1 summarizes the different user interface generations and their corresponding paradigms over time.

UI generation

future NUI modern WIMP traditional full-screen menus and forms historical command language pioneer batch pre-history none year 1940 1950 19601970 1980 1990 2000

Figure 1-1: Generation of user interface paradigms over time [Niel93]

Batch-style systems are regarded as the 1st generation UIs. Interaction with the computer system is done by setting up a batch file containing command statements and submitting it to the host. Line-oriented 2nd generation interfaces allowed the user to issue commands by entering them in a command-line shell. Full-screen 3rd generation UIs extended the interaction area to the whole computer screen area by offering modifiable input of data and commands. The addition of graphical elements such as Windows, Icons, Menus, and a Pointing device (WIMP) enabled the widespread commercial and private use of computer systems. Direct manipulation as primary interaction style came into play that give the user a visual feedback of the dialogue objects. The next generation of user interfaces is already under continuous research and development in industrial and academic laboratories. The extension of 2.5D WIMP based desktop interfaces to three and more dimensional, noncommand based Natural User Interfaces (NUIs) provide a multimodal and more user-friendly human computer interactiuon [Raut96]. Early NUI based approaches are the artificial reality metaphor [Gree90] and the information visualizer, developed at Xerox PARC [Robe93]. Such 3D user interfaces allow the user an effective use of screen space. They help them to improve the visualization of relationships between individual information units. As a consequence, they provide better understanding of the interrelation of data structures. Interactive 3D Simulation and Virtual Reality 5

Other techniques for handling large volumes of information are based on a space strategy that applies layouting and graphic design techniques, or on a time strategy that uses view transitions and information distribution to multiple views [Mack91]. A further step towards NUI based systems offers Virtual Reality technology that aims to extend the user´s perceptual field by involving several senses simultaneously for interaction. Augmented Reality as a derivative technology of Virtual Reality applies see-through HMDs or stereo-glasses to overlay the real-world physical environment with computer-generated entities [Durl95]. Augmented Reality environments supplement the interaction with the physical world by adding VR interaction paradigms to it.

1.2 Motivation and Objectives

The world-wide increasing interest for spatial devices and the dynamics of the VR industry induce the need for a deeper understanding of the underlying technologies and information processing mechanisms to produce optimal spatial effects [Marc92] [Robe93]. This need is clearly reflected by substantial academic and industrial R&D efforts worldwide. In spite of the existence of many VR research prototype applications and commercially available system environments, the fundamentals for efficient and effective VR application development and system configuration are far from being fully understood. The models, metrics and metaphors of VR environments have to be carefully designed and evaluated. The enormous dynamics of the VR field, the many interfacing possibilities between VR hardware and software components, and the immaturity of this young industry leave many questions open for discussion, such as: • Where are the concrete benefits of the application of VR technology in comparison to existing computer aided or real world solutions? • Are there any facts, figures, metrics and models to measure and compare different VR systems against each other? • What components are really needed for the development of a VR based application? • What health related issues are involved and have to be considered? • How can VR solutions be effectively and efficiently developed? • What principles concerning the design, implementation and integration of VR systems have to be considered? 6 Introduction

The evaluation and everyday work with VR systems and applications made it clear that the design space for Virtual Environments (VEs) is very manifold and difficult to understand. The wide range of possible VR system configuration setups and the absence of adequate and widely accepted classification schemata or taxonomies often make it difficult to define and implement the system environment needed for a specific application and task domain. Additionally, the current dynamics of VR research and industry force a permanent refinement and redefinition of already elaborated design models. Also, urgently needed system and usability tests that deliver the highly required benchmarking facts and figures for the design and development of VEs are still rare. The significance of this thesis lies in a new approach for the description, design and evaluation of VR based information systems. The objectives of this thesis are to give further insights into the underlying concepts for the interaction with VR based applications. Different aspects concerning the development and maintenance of VR system environments will also be drawn and discussed. Among the many possible application domains for VR based information systems, tourism was chosen in order to support the development of this new and promising application field for VR.

1.3 Thesis Organization

Chapter 2 describes the terminology and underlying paradigms of Virtual Reality. The elaboration of the basic building blocks of Virtual Reality systems give a further insight into VR technology. Finally, the still open issues and challenges are discussed and serve as motivation for the following chapters. Chapter 3 gives an introduction to information systems and describes the underlying architectural components. The combination of VR with information system technology leads to a new architecture model for VR based information systems. The discussion of the resulting system configurations as well as the potential application domains highlights the future importance of VR based information systems in new areas such as tourism. Chapter 4 proposes a new methodology for the evaluation of Virtual Reality application environments. The subdivision of the VR design space into views, namely a component, a perceptual, and a development oriented view, helps to better understand the underlying principles needed for the design and implementation of efficient and effective Virtual Reality applications. It is also shown how n-dimensional VR components can be presented in a compact and concise form. Besides a new model for the classification and design of VR based user interfaces will be proposed. Thesis Organization 7

Chapter 5 applies the methodology described in Chapter 4 for the evaluation of three VR application prototypes related to the field of tourism. The discussion of their underlying architecture model, user interface metaphor, performance behaviour, system development model, and maintenance aspects contributes to the research issues outlined in Chapter 1 and 2. Chapter 6 characterizes the field of tourism, the participating actors, and tourism related information requirements and services as well as major tourism related issues and challenges. The findings gained by the development and evaluation of the three VR application prototypes lead to the proposal of architecture models and user interface metaphors for Virtual Reality based tourist information systems. Three different application scenarios for Virtual Reality based tourist information systems will be described. Chapter 7 summarizes the major results of the thesis and gives an outline of potential and interesting topics for future work.

Chapter 2

Introduction to Virtual Reality Technology and Systems

This chapter gives an introduction into the field of Virtual Reality. In the beginning, the term itself and the constitutive design and user interface paradigms will be outlined and discussed. This provides the conceptual background for a deeper look into the technology and its underlying principles. The description of the various system components and technical as well as perceptual aspects involved in the development and operation of Virtual Reality systems help to get a better understanding of the technology. Because of increasing interest within the field of VR, the Virtual Reality Modeling Language as the first widely accepted standard for the description of Virtual Environments will also be introduced. The chapter concludes with the discussion of the major issues and challenges of VR technology today and serves as a motivation for the following chapters.

2.1 Terminology and Paradigms

Since the term Virtual Reality (VR) was introduced to the public by VR pioneer Jaron Lanier in June 1989, a lot of definitions for this term have been proposed. There are probably as many definitions for VR as there are papers discussing it. Jaron Lanier himself uses the following definition [Auk92]: "A computer-generated, interactive, three-dimensional environment in which a person is immersed." A further investigation of the terms "virtual" and "reality" themselves reveals the following definitions [Horn74]: 10 Introduction to Virtual Reality Technology and Systems

virtual: being in fact, acting as, what is described, but not accepted openly or in name as such reality: the quality of being real; real existence; that which underlies appearance Due to the dichotomous character of the two words "virtual" and "reality", other terms are used more and more frequently in the academic field. Terms such as virtual environment (VE), virtual world, interactive 3D simulation, or multi-sensory and multi- modal computer environment are often used synonymously for VR. Unfortunately, the popular press has coined the term VR in the public´s mind and does not differentiate it any further. In this thesis, Virtual Reality is regarded as the term for the technology itself. It incorporates the interaction within a VE or virtual world by the help of various peripheral devices in order to achieve a multi-sensory and multi-modal experience. The key features that determine the nature of every VR system can be summarized as follows: computer-generated A VR system consists of hardware and software components. Hardware components provide the IO interface to the user, do processing-intensive calculations, and store and forward data between different (sub-)systems. Software components are mainly responsible for data management, device support, driving of interaction, and calculation of updates to the VE. immersion and presence The degree of presence or immersion is one metric for the fidelity of a VR system [Kala93]. It determines how realistic or how close the actual implementation comes to the desired application domain. A diver immerses himself in the underwater world by diving completely into it. VR technology aims to generate a sensation of immersion or presence for the user through a completely computer-generated environment. Immersion means the feeling of being actually inside and surrounded by a virtual world [Auk92]. interactivity Interactivity refers to the degree of access to the parameters or attribute values of the VE [Kala93]. Interactivity consists of functions for navigation, manipulation, retrieval of data or information, and communication with other users within the virtual world. multi-sensory To achieve a high degree of immersion, VR aims to activate as many of the various perceptual channels of the user as possible. In addition to classical Basic Building Blocks 11

computer graphics applications, where mainly the visual perceptual channel is addressed, VR aims to extend the perceptual experience field by involving other channels such as the auditory, haptic, kinesthetic, or even olfactory channel. VR can be seen as a new way of interfacing the user with computer systems and media spaces. It means a completely new paradigm for human computer interaction. The paradigm shift that comes with the fundamental principles on which VR relies are the following, as indicated by [Brick91]:

from to

symbol processing application reality generation viewing a monitor wearing a virtual environment symbolic experimental observer participant explicit interface inclusion and immersion physical programmable only visual multi-modal and multi-sensory interface metaphor virtuality

Table 2-1: Paradigm shift enabled by VR technology [Brick91]

The exploratory, intuitive, and human-adequate interface character of VR has the potential to close the gap between problem understanding and problem solution strategies. Within certain application domains such as architectural, aircraft or vehicle design, VR has shown to lead to more straightforward problem solving strategies and therefore to a more effective problem solving process. Finally, the following more general but essential definition for VR in respect to its nature is proposed: Technology for immersing a user into a computer-generated, intuitive, interactive, and multi-sensory virtual but application-related environment. In the next section, the underlying basic building blocks or constitutive system components of modern VR environments will be described. The presented model helps in the classification and comparison of VR environments. VR system designers can use it as a generic skeleton for their future system architecture at the beginning of the development process. 12 Introduction to Virtual Reality Technology and Systems

2.2 Basic Building Blocks

The implementation of a VR system environment makes it necessary to simultaneously consider the parameters that determine the behaviour of a user in the real world and in the virtual environment. This is a fundamental requirement for the development of application-dependent and user-adequate VR systems. A schematic illustration of these dependencies between the real world and virtual environment is shown in Figure 2-1.

Virtual Environment Real World

Avatar VO interrelation- interrelation- User ships ships Avatar (User) Virtual Object User (VO) perceptual geometry behaviour physical Light polygons psychic ambient primitive point status

static Interfacing directional rendering dynamic spot wireframe Real Real Object intensity flat Object color gouraud cue solid phong visual effector ray traced audio sensor Constraints / radiosity tactile Constraints / Rules textured force Rules

Figure 2-1: Dependencies between real world and virtual environment

The user´s real world environment determines his primary field of action. It mainly consists of his perceptual, psychic and physical skills. Additionally, parameters, such as gravity, temperature, etc., have a strong influence on his capabilities and can heavily re- strict his interaction abilities. In the context of the interaction with virtual environments, peripheral IO devices incorporate specific interaction mechanisms and techniques that force the user to interact with a virtual world in a predefined and often unnatural way. In the virtual environment there are virtual objects whose attributes have to be represented and made interactive with the user by sophisticated software mechanisms. Different parameters define the perceptual shape, status and behaviour of the virtual objects which are manipulable by the user. Other attributes define the virtual environment itself and are responsible for light, atmospheric, and physical effects which can strongly differ from the user´s existing real world conditions. Finally, the user and his role in the virtual environment has to be represented in the form of a so called avatar. An avatar is the mainly geometrical representation of the user in the virtual world. It gives the user and other users a feedback for its own body position within the VE. Such a proprioceptive Basic Building Blocks 13

feedback (showing the position of body parts in relation to the VE and to other avatars) increases his degree of orientation within a VE and helps him not to get lost in virtual space. The mapping or linkage between the real world and the virtual environment is implemented by adequate interfacing concepts. Their objective is to achieve a smooth transition between these two environments in such a way that the user does not perceive the gap between the two. Manipulations of the user via interaction devices have to be mapped to corresponding actions in the virtual environment. In response, invoced actions and status changes of the virtual environment and virtual objects have to be communicated back to the user. Adequate user interface metaphors, highly dependent of the actual application domain and strongly considering the human perceptual system, are major prerequisites for optimal implementation of this interfacing layer. It is highly related to the success of every VR application environment. Expanding on the previous illustrations, which presented a conceptual view of VR environments, a next step will be made to give a deeper insight into the constituent elements of modern VR systems. Figure 2-2 illustrates the major logical elements or basic hardware and software building blocks for a VR application environment.

Interfacing Framework

User Application

Synthetic and Natural Synthetic and Natural Haptic and Kinesthetic Image Processing and Sound Processing and Information Processing Generation Generation and Generation

Scenebase Modeling Work Load Balancing Communication/Net- and Management and Distribution working Subsystem

Configurationbase Scenebase

Operating System

Peripheral Computer Storage IO Devices and Local and Global System Media Subsystems Networking

Figure 2-2: Basic elements of a modern Virtual Reality application environment 14 Introduction to Virtual Reality Technology and Systems

Each of these components will be explained in detail in the following sections. This model can be used as a classification and design schema for comparing the configuration of existing and new VR systems with each other and to help categorize them. The multi-sensory nature of immersive VR application environments requires the consideration of various perceptory processes in human computer interaction during system development. Human perception consists of several sensual information processing channels: visual sense, auditory sense, smell, taste, skin sensation, kinesthesis and equilibratory senses. In the following sections, perceptual aspects will also be discussed in order to sensitize the reader for the difficulty of the design and implementation of user-adequate VR systems. Badly designed interaction concepts and unsufficiently tuned system components can significantly reduce the usability of a VR system and lead to negative perceptual side effects such as nausea or simulation sickness.

2.2.1 Image Processing and Generation One of the core elements of todays VR systems are the components for image processing and generation. Highly sophisticated hardware subsystems and software algorithms are responsible for rendering a virtual environment in a realistic and flicker- free way. Dedicated graphic subsystems are used that implement graphic algorithms such as geometric transformation, clipping, Z-buffering, and coloring or texture mapping in hardware for maximum performance. This guarantees high frame rates for virtual scenes with low (less than 1000 polygons per frame) to middle (1000 to 5000 polygons per frame) scene complexity. The scene or polygon complexity is defined by the number of polygons to be rendered within a processing frame. It has to be remarked that the rendering of virtual scenes for interaction within virtual environments follows different principles to those of computer animation. For computer animations, the main objective consists of the generation of photorealistic images where computing time per image is not key. The opposite is true for VR applications: the key factor is real-time latency-free interaction by allowing deficits in photorealistic presentation. That is also the reason why models designed for computer animations can not be reused directly for interactive 3D simulation. A polygon count reduction and processing step has to be performed to fit the requirements for continuously high frame rates. Different procedures are involved in the visual presentation of a VE. As already mentioned, virtual objects are represented as a set of polygons that can be visually presented to the user in different forms. The simplest visual presentation form is the wireframe model. It is often used to evaluate the polygon complexity of a scene and subsequently look for potential approaches to reducing the polygon count if necessary. A further improvement towards more realistic scenes gives the shading of the wireframe Basic Building Blocks 15

model. The polygons are filled by predefined color values. The most commonly implemented algorithms are flat and gouraud shading. Adding light effects to virtual scenes further increases the realism. Light sources may be of the following type: ambient, directional, point, or spot light. Depending on the normal vectors of the surface and the shading model used, objects may have different light reflection properties. This also determines the visual appearance of object material. Finally, texture mapping can be applied to increase the feeling of visual immersion tremendously. This is a very powerful technique to achieve high frame rates by minimizing polygon count and the effort of time spent for scene modeling. The role of the rendering subsystem is to traverse the VE stored as a hierarchical data structure known as scene graph, to cull the objects that are not within viewing area of the user, and finally to output the processed data to the corresponding devices or output ports. A more detailed description of the techniques and algorithms for the visual processing of virtual scenes can be found in [Fol96]. Image processing functionality comes into play when natural scenes have to be analysed and interpreted, or existing digitized pictures such as texture maps have to be processed within the scene modeling phase. A typical example for the first case is video tracking where hand or body motion has to be captured for interaction purposes.

2.2.2 Sound Processing and Generation The application of sound cues for the interaction and visualization within VEs has many advantages. An improved perception of visual information, the mediation of emotions and background information, as well as the unburdening of the visual information processing channel are examples where audio-enhanced interaction can have a positive influence on the degree of presence. Auditory perception is a temporally extended activity. Sound waves exist only in time. The listener continuously develops more or less specific readiness for what will come next, based on information he has already picked up. The perception of auditory signal patterns in everyday life can come in very different forms: a car driving by on the street, a dripping faucet, the confusion of voices from a crowd of people, opera music, a plane flying by, the buzz of a travel alarm clock, the beeping of a wristwatch, etc. All of these acoustical signals can be divided into four categories: speech, music, sound and noise. Sometimes noises and sounds are heard and grouped together. All of these categories are described sufficiently in physical terms through the mixing and superposition of different pitches, frequencies, volumes, and sound duration. In contrast to music and noise, everyday sounds have a self standing characteristic: they are extremely context sensitive and event related [Gave86]. Sounds of everyday life are created through the physical interaction of different objects in 3D space. Their propagation through the air 16 Introduction to Virtual Reality Technology and Systems

to the human auditory system generates the corresponding sound sensation. In comparison with music and noises, the semantically relevant dimension of sound lies not with the characteristic quality of the auditory signal pattern itself, but rather with the quality of the sound producing event as it respects the concerned object [Moun90]. The average adult is, for the interpretation of everyday sounds, an expert with a large degree of knowledge from experience. This knowledge allows one to evaluate everyday sounds according to the following criteria for relevant information: 1) information about the physical occurrence (hearing a fallen glass clinking or breaking); 2) information about hidden events (hearing the hollowness of a wall when knocking to it); 3) information about dynamic changes (filling a wine glass and hearing when it is full and runs over); 4) information about abnormal conditions (hear when the car engine ceases to function properly and runs irregularly); and 5) information about occurrences outside of the visual field (sound of footsteps that tell the approach of someone). Listening to everyday sounds is based upon the perception of events and not upon the perception of sounds in and of themselves. Every sound is also a result of one or more interactions between two or more objects in a definite place and in definite surroundings as defined in [Raut94a]. At the same time the shared objects can participate in the production of sound from different aggregate conditions, materials, and even their configuration. The configurations of these materials can also have an influence on the produced sound. Echo, reverberation, phase differences, overtones, sound conduction routes, masking effects, inter-aural intensity differences, position and orientation of listener´s head are important factors in the perception of auditory information. Especially the modeling and consideration of the audio source position and head related parameters (head position and orientation, anatomy of the pinna as a linear filter), intensity differences, temporal or phase differences between arriving signals play a significant role in spatial sound perception and generation using head related transfer functions (HRTFs). Timbre is affected by the envelope of the sound signal (the rate of amplitude modulations). In relation to virtual worlds three types of auditory information may be differentiated: mono, stereo, and spatialized sound. Especially the third type has gained a lot of interest in the last few years. The application of spatialized sound cues allows the 3D positioning of one or more virtual sound sources within virtual worlds in order to generate highly realistic sound effects for the human auditory system. The user perceives these sound sources as if they originate from a specific place in the virtual environment. 3D sound may occur as such in order to generate background sound or may be assigned to virtual objects. Virtual vehicles may generate Doppler effects relative to their distance and velocity. NASA spin-off companies such as Crystal River Engineering (CRE) have Basic Building Blocks 17

experimented and commercialized sophisticated hardware and software solutions for spatialized hearing within VEs [Berg94]. Figure 2-3 shows the concept of spatialized virtual sound.

reflected sound signals virtual sound by the virtual wall source (loud speaker)

virtual directly 3D object perceived sound signals

user´s location in the virtual space

Figure 2-3: Principle of 3D sound

Besides special signal processing hardware, there are also software packages available that are able to generate almost realistic 3D sound experiences using inexpensive stereo sound boards for PCs. Intel´s Realistic Sound eXperience (RSX) technology is an example. In the future, 3D sound sensations will become an integral part of VR development systems and web browsers. Sound will gain an increased focus in the interaction with information or media spaces, and will make the term sonification as important as visualization. One of the most natural human computer interfacing methods is speech input. Even though there is a variety of systems commercially available by now with recognition rates above 95%, speech recognition is rarly implemented in today´s VR systems. The intersubject variability in the production of speech, such as the difference in voices or unclear pronounciations of words, the difficulty in handling of continuous speech input, the long system training time, interfering background noise, and the limited capacity to handle large vocabularities may have been some of the hindering factors [Durl95]. Nevertheless, reasonably high accuracies for task-specific applications such as setting binary states of virtual objects may be achieved with todays systems as described in [Hohl96]. 18 Introduction to Virtual Reality Technology and Systems

2.2.3 Haptic and Kinesthetic Information Processing and Generation Even though there are basic haptic devices available on the market, this field is not yet that highly evolved as vision and audition, and is still in its infancy. Nevertheless, it is regarded as an important factor in achieving immersion. Haptic information processing or feedback can be divided into two parts: force feedback and tactile feedback. It allows the user to feel the rigidity, volume, mass, weight, friction, dynamics, position, orientation, and surface characteristics of virtual objects [Burd96]. Tactile feedback is sensed by receptors close to the skin. They sense the initial contact with the environment, its surface geometry, roughness, surface temperature, and slipperyness. Force feedback sensors are located deeper in the body. They provide feedback information on the total contact force, grasped object compliance and weight. Further, kinestethic as well as proprioceptive information give additional feedback about dynamic and spatial features of the virtual environment and often occur together with haptic sensory inputs [Durl95]. The lack of haptic feedback may result in a major decrease of information display to the user [Cutt93]. In particular, object manipulation, movement, and exploration tasks (as might occur in surgical simulators) require proper execution of tasks enhanced by haptic feedback. The existing prototypes and commercially available haptic devices still perform poorly with respect to usability and integratability into VR systems. Although haptic information processing and generation is still immature, it will gain increased attention for future VR systems and open new interesting application scenarios. Kinesthesis is the muscle, tendon, and joint sense that informs the human sensory system of the position and movement of parts of the body. It is stimulated by bodily movements and tensions. Without kinesthesis the human would have great difficulty in maintaining posture, walking, climbing, and in controlling voluntary movements such as reaching, grasping, and manipulating (see for example [Hash93]). The equilibratory senses cooperate with kinesthesis. They deal with total body position in relation to gravity and with motion of the body as a whole. The relation of body parts to each other and to external objects is the responsibility of kinesthesis. The orientation of the body in space is the responsibility of the equilibratory senses. The equilibratory senses also signal accelerated motion in a straight line, but sometimes they produce illusions that distort the true path of motion. These illusions occur in flying, because of changes in speed and the banking and climbing of the plane. For example, when a plane is increasing its speed gradually, a blindfolded subject can surely feel that the plane is climbing. If its speed is decreasing gradually, the subject can feel equally sure that the plane is diving. How important and strong is the influence of the kinesthesis and equilibratory senses of an actively moved subject to its visual and auditory information processing? This influence was demonstrated and proved by Held and Freedman Basic Building Blocks 19

[Kahn73]. Disturbances of this correlation can be the reason for vertigo and dizziness. On the other side, it is also possible to recover the three-dimensional structure from motion of observed objects. If a shadow is cast by rotating wire shape onto a screen, a passive viewer can readily perceive the shape of the structure behind the screen from the dynamic shadow pattern [Wall53]. When static, the screen looks like a random collection of lines.

2.2.4 Interfacing Framework A powerful and flexible interfacing framework, consisting of hardware as well as software components, is a major requirement for a seamless integration and maintenance of the various VR building blocks. Such a framework has to offer a large set of methods and functions to customize a VR system environment to the domain-specific needs in a user-friendly way in order to generate a broad acceptance for VR based application solutions. VR systems incorporate various functional computing and interfacing elements. A prerequisite for the successful design and implementation of VR applications is a careful tuning of those subsystems. Not only software components, but also hardware and user interface mechanisms have to be coordinated and well adapted to meet the application and user dependent specifications. Interfacing to hardware subsystems, such as peripheral devices, digital signal processing units, or networking channels, requires a deep knowledge of the involved components. This knowledge is essential when it comes to overall system performance optimization in the final phase of a VR project. Interfacing between the individual software components requires a framework of protocols, abstract data types, and transformation methods to ease the integration process. Some of the more elaborate commercially available software toolkits offer a set of data structures and methods to interface a large set of peripheral devices. A rich set of device drivers allows a flexible and fast adaptation as well as reconfiguration of an existing system setup. Finally, the interfacing framework also includes the implementation of user interface metaphors. The objective of user interface metaphors is to ease the mental model building process while learning the functionality of a computer system. The mapping of the system model to the user´s mental model about the application or task domain or a real- world reference system known to the user also helps in minimizing the user´s memory or mental load [Niel93]. The user can recognize parallels in meaning or functionality to already known phenomena in his real-world environment. As a consequence, interpretation and evaluation steps during the interaction with a VE can be shortened. 20 Introduction to Virtual Reality Technology and Systems

Furthermore, the learning process can be enhanced. Metaphors help to break down the complexity of a computer environment, mainly influenced by the immense interrela- tionship of its information entities, to a simpler and understandable form for the user. Though the application of WIMP and direct manipulation based 2.5D desktop metaphors for the interaction with VEs is still widespread, new metaphors begin to evolve. Examples such as the direct 3D manipulation of virtual objects applied by many VR software toolkits, VR widgets called Virgets as reusable elements for constructing 3D user interfaces [Musi93], the World-In-Miniature (WIM) metaphor for navigation and locomotion [Pau95], or the shopping cart metaphor presented in [Musi95] offer promising approaches.

2.2.5 Virtual Environment Modeling and Management The core element of every VR system is the simulation or scene database (scenebase). It contains all the parameters and attributes of a VE that describe its perceptual and beha- vioural features. For example, the geometry, material color, or texture of virtual objects, the sonification of the VE by virtual sound sources, as well as event triggering functions to enrich the VE with dynamic effects are defined here. Most of the existing VR system environments store their runtime scenebase in main memory using a hierachical scene graph. The generation of the scenebase is done by using a 3D modeler. To save modeling time, already existing models can be reused and integrated into the new scenebase. Conven- tional 3D animation modeling packages offer only an inadequate set of functions for the design of VR scenebases. They do not incorporate methods for performance optimization and smooth synchronization of perceptual channels. Special modeling and test environments, sophisticated for interactive 3D simulation, have to be applied. A state-of-the-art VR modeling environment has to integrate the following features:

- mechanisms for runtime and storage optimization of virtual scenes (polygon count reduction, scene partitioning for more efficient traversal and culling of the scene graph, backface removal, instancing of objects, billboarding, etc.) - support for different Level-Of-Detail (LOD) models and switching strategies - operations for texture mapping and transformation - terrain modeling integrating functions for adequate object to terrain alignment - import and export interfaces to different data formats The management and processing of large volumes of data that can occur within VEs makes it necessary to implement sophisticated data access and management techniques. Basic Building Blocks 21

This is essential to keep processing lags at a minimum in order to quarantee real-time conditions for system execution. Data has to be routed between the individual system components because of user interaction, workload distribution, or sensory feedback to the user. This requires adequate data management mechanisms for efficient organization and direction of various data flows between processing units and for guaranteing a consistent status of the scenebase.

2.2.6 Peripheral IO Devices and Subsystems In general, peripheral IO devices for VR can be divided in two groups: devices for display and devices for control [Durl95]. They subsume all kind of devices that can be integrated into a VR system to present information and to sense the human´s action or reaction in order to control the simulation sequence. Another categorization is proposed by [Pime93] who defines peripheral IO devices for VR as effectors. These specialized devices eighter communicate a user´s movements or commands to the computer or provide sensory stimulation from the computer to the user. Effectors provide and receive sensory cues to and from the human perceptual system. Both, input and output devices for VR are considered as effectors. Depending on the perceptual channel addressed and role in human computer interfacing, effectors can be grouped as indicated in Table 2-1. channel device sensing, generation addressed processing visual (high-res) graphics screen X multi-channeled graphics screens X video projector or beamer X stereo LCD shutter glasses X passive polarized stereo glasses X head-mounted display X see-through head-mounted display X video camera X auditory stereo sound board X 3D/spatialized sound board X loud speaker X head-phone X haptic and alpha-numerical keyboard X kinesthetic 2D/3D mouse X joystick X force-reflecting hand controllers X X tactile displays (shape changers, virbrotactile, electrotactile) X spaceball X wired dataglove X position and orientation tracker X other microphone X

Table 2-1: Classification of a selected number of effector types 22 Introduction to Virtual Reality Technology and Systems

The setup and maintenance of a high-end VR testbed environment has shown that VR devices require much more effort and time compared to those of classical multimedia environments. Factors such as device ergonomics, cabling layout, cable length, pin configuration, type of connector plug, or environmental interference may heavily influence the amount of resources that have to be allocated for VR system support. Another hindering fact for a smooth integration of VR devices is the distant location of manufacturers and distributers. The repair or substitution of malfunctioning devices induce long wait times and delivery costs. The absence of extended experiments and usability tests where VR device setups are compared against each other makes it still difficult to evaluate and configure the optimal setup for a targeted application scenario. These tests need to be done by independent research institutions in close cooperation with the corresponding manufacturers whose VR devices are to be tested. Ergonomic as well as setup issues have to be evaluated and published in public media such as the Internet. An underlying, broadly accepted metrics scheme should be the base for these investigations. The results gained out of these experiments will help generating facts and figures about the optimal application of VR devices and uncover the fact that there are several ways to configure a VR system environment.

2.2.7 Workload Balancing and Distribution Real-time interaction in large-scale virtual environments requires intensive computing power and puts scarce system resources under heavy load. Strategies to balance workload and to distribute computation intensive functions to several system resources have to be evaluated in order to find the optimal configuration for a desired runtime behaviour. Often, various hardware components and software techniques are combined to achieve an optimal runtime performance.

On the hardware side, the dedication of functions to single processing subsystems has prooven to offer significant benefits. The processing of perceptual channels and computing intensive functions is performed by separate computation units. Two setups can be differentiated: local and network wide distribution of processing functions. Local distribution of processing units means the usage of computing boards pluged into a PC or workstation that implement certain algorithms or functional processing units in hardware. Typically, the processing of visual and audio information is done by powerful graphics and sound boards. The exploitation of the computing power of such subsystems requires data transmission paths with a high-bandwidth. This is a major assumption for the scalability of VR systems that apply dedicated subsystems for workload balancing and distribution. A network-wide distribution of processing units on the Basic Building Blocks 23

other hand, means the application of single computer systems interlinked by a local or wide area network. To achieve a high data transmission speed it is necessary to link the systems by high-bandwidth network. Research projects such as [Cruz95] have already been done in this field. Further, the incorporation of protocols for interactive real-time simulation such as the Distributed Interactive Simulation protocol (DIS) [Blau93] or the setup of low-latency reflective shared-memory components such as SCRAMNet by Systran Corp. can bring additional improvement in performance. On the software side, processing functions can be split into single concurrent processes. This requires a multi-tasking operating system or a network wide distribution of processes. Synchronization and communication between processes is implemented by a shared memory or message passing architecture. As discussed in [Sink94], the access of system resources such as computer systems within a networked computing environment can significantly increase overall system performance for a single application. These resources are often under a low workload and can therefore be used as additional processing units within a VR system setup. The programming of distributed processing based applications is a nontrivial task. The evaluation of efficient data transmission protocols, the synchronization of processes, and the configuration and runtime profile of the involved processing units make it difficult to find the strategy for optimal overal system setting and process distribution.

2.2.8 Local and Global Networking Driven by the increased connectivity between computer systems locally and worldwide, the prerequisites have been established for the integration of VR systems, users, and information spaces within a shared computer based VE. An early description of such a VE may be found in Gibson´s Neuromancer [Gibs84]. Gibson has also coined the term cyberspace as a world-spanning information space created by the networking of multiple VR systems in which people can operate and communicate with each other [Jacob94]. Physically or geographically separated users can meet each other virtually in a common environment and exchange their ideas and knowledge. Another term closely related to cyberspace is telepresence or televirtuality that generates a sense of being in a distant real place and experiencing a real but remote environment [Auk92] [Krue91]. As an example, telepresence technology is used for remotely control and operate underwater or space vehicles by immersing the operater into the virtual 3D model of the remote environment (e.g. virtual mars) and perform user manipulations within the VE on the remote but real vehicle. In locally distributed VR setups, as well as in telepresence or multi-user environments, networking components and sophisticated data transmission protocols are applied to achieve latency free and lossless exchange of interaction and scene data. Enabled by VR research results, telepresence and tele- 24 Introduction to Virtual Reality Technology and Systems

operation platforms, based on high-bandwith digital networks, offer the potentials to revolutionize human to human communication and collaborative work [Szabo94b]. Intensive research and development has already been done in this field. Examples such as the NPSNet system [Zyda93], the actor concept within Division´s VR toolkit dVS [Grim91], the MR toolkit [Shaw93], or the DIVE system [Carl95] show the importance of this aspect. Besides protocols specially designed for interactive real-time 3D simulation environments such as the Distributed Interactive Simulation (DIS) protocol [Blau93], common networking functions such as TCP/IP or UDP/IP based Berkeley sockets [Stev90] are applied. As underlying networking infrastructure the Internet, ATM, or dedicated simulation networks such as SIMNET [Scha91] are used mostly.

2.3 Virtual Reality Modeling Language

The broad acceptance and growth of the Internet, mainly enabled by the World-Wide- Web (WWW), made the VR community focus on this global medium for communication and information exchange, too. The personal effort and visions of Marc Pesce and Tony Parisi established a language for the description and programming of Internet based VEs, called Virtual Reality Modeling Language (VRML). Their objective was to design a language that enables the implementation of 3D websites and virtual environments for the Internet by a standard scenebase description language. VRML should help to build a "planetary management system" and offer a VR based frontend for the access to all kind of data available on the Internet [Pesce95]. The underlying concepts of VRML and a prototype VRML browser called Labyrinth were presented in May 1994 at the First International World Wide Web Conference at CERN in Geneva [Ames97]. VRML browsers were first available as stand-alone applications (so called helper applications). Today, web browsers offer a seamless support for VRML functionality via built-in modules. Up to this date, three main stages or versions in the history of VRML can be differentiated: 1.) the description of static WWW based VEs with VRML 1.0 [Ames96]; 2.) enabling of dynamic, programmable and multimedia enhanced scenes by VRML 2.0 (named "Moving Worlds") [VRML96] ; 3.) adding multi-user support by VRML 3.0 (named "Living Worlds") [Szabo97]. The ever increasing interest for this topic on VR conferences, of industrial and commercial companies world-wide, and the establishment of websites and mailing-lists on the Internet dedicated to VRML show the importance of this language. Beside research prototypes in the academic field, many industrial and commercial VRML Virtual Reality Modeling Language 25

applications have already been implemented. VRML as 3D frontend for websites, for the planning and management of telematics infrastructures, for the interaction with medical data, for the reconstruction of historical buildings or as a new apporach for interactive art are only some of the many examples available on the World Wide Web. Since the presentation of the draft specification for VRML 1.0 in October 1994 at the Second International Conference on the World Wide Web in Chicago, a lot of startup companies have been founded world-wide, offering browsers, authoring software or VRML application development services. Market analysts say that VRML invoked a new industrial revolution comparable to that of the World Wide Web.

2.4 Major Issues and Challenges

Even though many VR application prototypes and usability experiments in academic and industrial research have been conducted, there are still many open questions about VR system design and implementation issues. The multi-sensory nature of VR system solutions and the simultaneous consideration of various hardware and software aspects during the development process make it often difficult to propose the right system setup for a given application scenario. This explains also the slow diffusion of VR technology into new application fields such as tourism or business oriented education. The provision of performance and development related figures by the design, implementation, and evaluation of VR systems in new application domains helps to accelerate this process. The discussion of issues and challenges in VR application development is important to identify critical success factors and propose adequate system solutions. Some of them will be highlighted in the following.

2.4.1 Real-Time Interaction and Synchronization of Perceptual Channels

One of the key factors of immersive VEs is lag free system performance and human computer interaction. Latency is defined as the rate at which the acquisition portion of a system can acquire new data [Kala93]. The data sensing frequency as well as the accuracy of a receiver unit sending updated data to an application determine the effect of latency coming from that sensing unit. Update rate, often mixed up with latency, is the ability or rate of a data channel to output sensed data to an output port. It does not reflect the acquisition process and therefore does not reflect the overall system performance. It is mainly influenced by the internal bus system, involved system processors and implemented algorithms. The sum of latency time and update time results in the phase lag. It is the quantitative difference between the actual data sensed 26 Introduction to Virtual Reality Technology and Systems

and the computed and perceived feedback by the user. Especially time-dependent system components may have significant affect on phase lag. Factors such as system architecture, type of processing units and efficiency of algorithms may have a major contribution to phase lag and consequently to overall system performance as well as to effects of nausea. For example, the synchronization between a position sensor and its visual feedback in form of a 3D cursor reflects the effect of its phase lag. If synchronization is perfect, that means the visual feedback corresponds exactly to a change in the sensed data, phase lag may be regarded as unnoticable by the user. The sensing unit and its corresponding feedback performs in real-time. The relationship between latency, update rate and phase lag is illustrated in Figure 2-4 by choosing time instead of rate as comparing unit.

input/ rendered sensed acquisition processing rendering data data

latency time update time

phase lag

Figure 2-4: Latency time, update time and phase lag [Kala93]

Because a VR system may integrate different functional building blocks, the interfacing and synchronisation of these components is a very challenging task. In the final phase of a VR project considerable effort has to be undertaken to tune the system toward a seamless interplay between the involved processing units. The knowledge of information processing mechanisms within the human perceptory system is one of the most important factors for building usable VR systems. The human is very good at noticing even a little lag in perceptual input and may react to incompatibilities between perceptual channels with simulation sickness. To avoid these externalities, that may even bring a VR project to its fall, it is advantageous to set up an interdisciplinay project team whose members cover with their knowledge and experience the single aspects of the design and implementation of human-respected VR applications.

2.4.2 Intuitive and User-Friendly Human Computer Interaction The first contact a user makes with a VR application is done through its user interface. It has a signifficant influence on the degree of acceptance of the VR system by the potential user. A carefully designed, task and user oriented human computer interface can essentially contribute to generate a high initial degree of acceptance. Unfortunately, many VR applications only reuse existing desktop metaphor based user interface Major Issues and Challenges 27

concepts and do not take the effort to realy elaborate and implement user interfaces that meet the requirements for multi-modal and multi-sensory interaction within VEs. The fact that the design space for VR system environments is still scarcely analysed causes additional difficulties for the development of VR user interfaces. During the 80s, the 2.5D WIMP based desktop metaphor and direct manipulation paradigm dominated user interface design, primarily for office applications [Gree90]. This 2.5D metaphor is not adequate for 3D data visualization and interaction [Robe93]. Therefore, new metaphors for fully 3D computer generated user interfaces are needed. A main research topic is the evaluation of new 3D peripheral devices, incorporating sound, speech and haptic user interface elements, and determining their effectivenes concerning usability [Brys94b]. The need to develop useful metrics for evaluating virtual environments cuts across all of these issues. Many of the existing VR system development environments only transform user interface concepts from 2D desktop metaphor to 3D space and do not consider the new interaction possibilities offered by the multi-modal and multi-sensory nature of VEs. Also, it is neccessary to build up an understanding of the user and his domain by a corresponding task and application domain analysis. This is a major requirement for the elaboration of user adequate and application oriented human computer interfaces. The user has to be supported by a flexible set of interaction mechanisms and metaphors that allow an individualization and customization of the user interface in respect to the actual user profile and system setup. The still insufficient parameter characteristics of peripheral devices, such as resolution, latency, comfort, sensing range, and price performance ratio, force VR application developers to equalize these deficits by workarounds on the user interface side. The active evaluation of many existing VR systems and applications made it clear that there is a wide range of interaction possibilities with virtual worlds. This also has a strong influence on the definition of adequate user interfaces. Due to the wide range of possibilities for VR application setups, it is neccessary to have qualitative and quantitative figures that help decision makers choose the adequate VR application elements for their task specific domain. Benchmarking of 3D interactive real-time applications and 3D walk-through, manipulation and information presentation user interface metaphors is therefore a major presupposition to acquire such facts and figures. The proposal of adequate models, metrics and metaphors for the interaction with Virtual Realities requires an intensive investigation with VR system components, peripheral IO-devices and prototype application scenarios. Different prototype system configurations have to be set up, implemented and tested against each other. Until now, the following factors for user-adequate interaction within VEs have been identified [Durl95]: interactive response times of approximately 0.1s or less, visual frame rates of 28 Introduction to Virtual Reality Technology and Systems

at least 10 frames/s, display resolution of at least 1000x1000 pixels, and a wide field of view to visualize the interrelationships of phenomena of varying detail.

2.4.3 Effective Scenebase Modeling and Management The implementation of different VR application prototypes has shown that data acquisition and processing is a very time-consuming and critical stage in the VR application development process. It strongly determines the quality of data within the future VR application. Often, several conversion utilities have to be applied to transform object data from one format to another. Also, negative side effects such as maltrans- formation or loss of data during the conversion can occur and force the writing of corresponding utilitiy programs. All these facts can heavily burden the available time and money budget during the development process. A well thought out data management concept supported by a rich set of data processing functions is an important requirement for a modern VR application development environment. The modeling of VEs requires a flexible framework of hardware and software components that allows a seamless integration of existing and additional system resources. A powerful authoring environment is needed to ease the development process and to provide rich content to the final application. Not only the modeling of static and dynamic behaviours, but also the multi-sensory interfacing between real and virtual world entities has to be supported. The virtual environment to be modeled always has to consider to run under real-time performance conditions. This implies that sophisticated data modeling and representation techniques, such as Level-Of-Detail (LOD), texture maping, or efficient scenebase culling, have to be applied. Permanent performance monitoring of key functions such as rendering or device data acquisition have to be a part of such an advanced authoring environment. The increasing interest for information highway environments require more sophisticated models and techniques for the implementation of various data query, access and retrieval strategies. The application of regular file system mechanisms is no longer sufficient to meet the specifications and requirements of future application scenarios such as VR based medical or geographic information systems. These kinds of application environments need interfacing between DataBase Management Systems (DBMSs) and the internal runtime scenebase. Data must be held in a multilayered storage hierarchy because of the limited system resources such as computing power of the rendering subsystem or main memory capacity. Algorithms and data structures for the paging of scene entities can contribute positively to the maintenance of high degrees of interaction while dealing with high-bandwidth data streams. Major Issues and Challenges 29

The focus on real-time, perception-synchronized data management mechanisms has not yet been that strong in the VR research community. The expected increase in VRML enhanced websites on the Internet in the future will catalyse the availability of VR adapted DBMSs and corresponding software interfaces. Finally, dynamic large-scale VEs require a well designed data storage and management strategy. The increase in demand on a VE is often positively correlated with additional storage capacity requirements. The avoidance of unwanted redundancies and data ceme- teries is key for an efficient management of scarce storage space.

2.4.4 Eased System Development, Setup and Maintenance The evaluation of existing VR systems shows that they incorporate various technologies and computer science disciplines. Many core competencies have to be integrated simultaneously in the design and implementation process. This is a key assumption to a successful completion of a VR project. An interdisciplinary project team consisting of programers, application domain specialists, database engineers, networking and user interface specialists have to be formed to meet the challenging requirements for effective and efficient VR application development. A significant factor for efficient and effective VR application development is the coherency of the development environment. The less transformation steps have to be processed, the more straight-forward will the overall development process be. The better the interfacing issues are supported by the existing development environment, the less time is spent for data format conversion, process communication, or re-generation of already existing models. Additionally, the loss of data attributes during the conversion processes can be minimized or even prevented. The investigation and use of different VR development environments highlighted the need for powerful system setup and maintenance tools and procedures. The setup of a specific system configuration and subsequent modification at runtime is still a nontrivial task. Many configuration files have to be correctly adjusted to achieve a certain system setup. Also, the failure or temporal unavailability of system components makes it difficult to guarantee a high degree of overall system availability. Errors are often hard to identify and therefore very time-consuming to debug. Incompatible connector plugs, broken cables, or even the temporal malfunction of peripheral devices often bring a VR system out of order even though all the other components operate well. Many VR application environments do not consider these cases and hardcode system setups without allowing the operator or user to reconfigure the system at runtime. Last, the maintenance issues are playing an essential role when looking at the success of VR systems. Even though maintenance is no more part of a VR project, it has to be 30 Introduction to Virtual Reality Technology and Systems

integrated into the design process. The maintenance of an acurate and perceptually satisfying VR environment requires a periodic recalibration of diverse system resources, especially when tracking and video projectors are involved. Automatic calibration processes without the need for the administrator´s presence would contribute a lot to ease the maintenance of a VR environment.

2.4.5 Economical Aspects Regarding the economical issues involved in VR technology and application development, there are two aspects that can be differentiated: the costs for the design and implementation of a VR application solution on one hand, the benefits gained by the application of VR technology in everyday business and projects on the other hand. On the expense side, costs for the evaluation, development, installation, maintenance, and user training have to be considered. As an example, the modeling of the visual simulation database requires at least 50% of the overall VR project time. The lack of benchmark figures in specific application domains, the enormous dynamics of the VR industry itself, and the relatively high expenses for the development of a VR system prevent many decision makers in the commercial and industrial field from investing into VR technology. On the revenue side, VR offers potentials of minimizing costs in product development or training, improving quality in the decision making process, or shortening product life and time-to-market cycles. For example, flight simulators beeing used by the Swiss Air Force for the training of pilots generate a saving factor of four to five compared to flying with real jets. Beside quantitave benefits gained by the application of VR technology, there are qualitative benefits. The simulation and testing of different solution alternatives in an intuitive and easy understandable way offers the potential to improve the quality and time of a decision making process. Unfortunately, such benefits are often hard to quantify and transform into monetary values.

2.5 Summary

Virtual Reality is a technology that enables the design and implementation of new paradigms for human computer interaction. VR systems immerse a user into a computer-generated, interactive, and multi-sensory virtual environment. VR based application environments offer new ways to get a better understanding of the problem space within a specific application domain and to elaborate more straigth-forward solution approaches. Summary 31

After describing the term Virtual Reality and its paradigms, the individual building blocks of Virtual Reality systems are described in order to give an insight into their design and implementation aspects. A flexible, easily customizable, and user-friendly interfacing framework has to be provided to integrate the various hardware and software components in a seamless, overall performance optimizing way. The Virtual Reality Modeling Language, as a very promising new description language for VEs, has already achieved large acceptance within the VR community world-wide. It allows a platform independent specification of networked VEs to make virtual worlds more portable across diferent VR systems. VRML is regarded as the driving force to make VR technology become more ubiquitous in future computing environments. Finally, the major issues and challenges within the field of VR technology are discussed. These include real-time interaction and synchronization of perceptual channels, intuitive and user-friendly human computer interaction, effective database modeling and management, eased system development, setup and maintenance, and economical factors. A basic understanding of the aspects that are relevant for the design and implementation of VR system components is necessary for an effective evaluation, development, operation, and maintenance of VR based applications.

Chapter 3

Virtual Reality Based Information Systems

After the introduction into Virtual Reality technology and the description of VR system components, it will be shown in this chapter, how VR and information system technology can be combined to implement Virtual Reality based information systems. Their underlying system architecture and derived system configuration setups will be explained. Finally, three application domains will be discussed that offer high potentials for the application of VR based information system concepts.

3.1 Information Systems

During the last 30 years the world economy has slowly shifted from an industrial to an information society. This is mainly seen by the growing importance of broadcasting media in everyday life, by the ever increasing automation of standardized processes in the industrial and service sectors, and two-digit annual growth rates of enabling technologies such as telecommunication or information technology. Many European and national research projects have been launched in the past few years to elaborate new methodologies for the design, implementation and maintenance of Information and Communication (IaC) infrastructures. The tremendeous growth of digitally available information world-wide along with increased networking of information entities makes it necessary to develop new methods for the interaction with large-scale information spaces. To understand the fundamental ideas of computer based information systems the underlying constitutive elements will be laid out in the following sections. 34 Virtual Reality Based Information Systems

3.1.1 Terminology An Information System (IS) is a socio-technical environment that integrates all resources for the collection, management, use, and dissemination of information [Elma89] [Hein93]. This includes not only computing resources, but also human resources who design, implement, maintain, or use the IS. In the context of this thesis, only the technical, computer system oriented aspects of an IS will be considered. The main focus of an IS lies in the support of intra- and interorganizational tasks and processes. As a general and simplified definition, an IS can be regarded as a computer based question-and-answer-system. The availability and merger of different technologies within the IS field has offered new application potentials. The term IS has also experienced an extension of its nature and understanding. Aspects such as communication, coordination, and cooperation will have an important influence on the meaning and architecture of future ISs [Teuf95].

3.1.2 Taxonomy and Classification Concepts The variability in design and implementation makes it often difficult to categorize existing IaCs. The simultaneous and integrative application of several views of descrip- tions helps in the classification and subsequent comparison of IaCs. Figure 3-1 shows the business-oriented model for a typology that emphasizes the integrative corporate view and functionality of IaCs.

Planning & Decision Systems

Analysis IS Control Systems Planning &

Reporting & Control Systems

Value-oriented Accounting Systems

Amount-oriented Systems

Disposition Operation Systems Administration &

Research& Marketing& Procurement& Manufacturing Distribution Development Sales Stocking Figure 3-1: Business-oriented classification model for ISs [Mert91] [Sche95] Information Systems 35

The design of a corporate IaC, following a modular "LEGO" paradigm, facilitates the implementation of a flexible and state-of-the-art IaC infrastructure. A high degree of corporation adapted functionality together with an open interfacing framework allows a seamless vertical and horizontal integration of various system components. For the classification and evaluation of IaC systems it is again essential to elaborate their underlying basic system building blocks. They build the major suppositions for the setup of classification schemes in order to compare different systems against each other. The application and concrete internal configuration of the individual building blocks depends on the domain where they run. A decision support system can have a completely different internal system configuration than a reporting system. Figure 3-2 shows the main building blocks for IaC systems.

User Interface (UI)

User Application (UA)

Interfacing Layer (IL)

Database Communication/ Expert/Knowledge Management Networking Base System (E/KBS) System (DBMS) System (C/NS)

Database (DB) Modelbase (MB) Methodbase (MeB) Knowledgebase (KB)

Operating System (OS)

Local and Global Computer Storage IO Devices (IOD) Networking System (CS) Media (SM) (LGN)

Figure 3-2: Basic building blocks of information and communication systems

The elaboration and identification of the basic building blocks of IaC systems is important in order to understand the nature and system concepts of Virtual Reality based Information Systems (ViRIS). The operation of an IaC system is done via the User Interface (UI) in combination with IO devices. The UI, implemented in textual or graphical form, serves as front-end to the user by which the User Application (UA) can be manipulated. An Interfacing Layer (IL) such as the Open DataBase Connection (ODBC) for example provides basic services to access databases, modelbases, method- bases and knowledgebases via database management, knowledgebase, or networking systems from the UA. A more detailed description of the single IaC system components shown in Figure 3-2 can be found in [Mert91].

36 Virtual Reality Based Information Systems

3.2 Models for Virtual Reality Based Information Systems

The application of VR technology as a front-end for information systems extends the above described building blocks for IaC systems by new components. The integration of VR and IaC technology results in a new system architecture illustrated in Figure 3-3.

Interfacing Framework (IF)

User Application (UA)

Synthetic and Natural Synthetic and Natural Haptic and Kinesthetic Image Processing and Sound Processing and Information Processing Generation (SNIPG) Generation (SNSPG) and Generation (HKIPG)

Workload Balancing and Scenebase Modeling and Distribution (WBD) Management (SBMM)

Database Communication/ Expert/Knowledge Management Networking Base System (E/KBS) System (DBMS) System (C/NS)

Configurationbase (CB) Scenebase (SB)

Database (DB) Modelbase (MB) Methodbase (MeB) Knowledgebase (KB)

Operating System (OS)

Computer Peripheral IO Devices and Storage Local and Global System (CS) Subsystems (PIODS) Media (SM) Networking (LGN)

Figure 3-3: Basic building blocks of a Virtual Reality based Information System

The user interface component is substituted or extended by the VR basic building blocks described earlier in Chapter 2. The integration of all the individual ViRIS blocks requires an increased emphasis on the interfacing aspects. Due to the potentially large amount of data that may be processed in a ViRIS, extensive data and message flows can occur between components. Only a well designed and optimized implementation of an adequate interfacing framework can lead to a perceptually satisfying and user-adequate overall system performance. Models for Virtual Reality Based Information Systems 37

3.2.1 Virtual Reality Based Information System Configurations The evaluation and comparison of existing VR system setups have shown that there are two main configuration classes: 1.) single- or multi-user, and 2.) non-distributed or distributed environments. The classification class is determined by the allocation and setup of the individual building blocks described above. In a single-user non-distributed ViRIS all the building blocks are allocated on one machine. Figure 3-4 lays out such a configuration where the acronyms correspond to the basic building blocks shown in Figure 3-3. It can be configured as a stand-alone or networked system.

SNIPGSNSPG HKIPG

WBD SBMM IF DBMS E/KBS C/NS

CB SB

DB MB MeB KB

CS PIODSSM LGN

Local / Global Area Network

Figure 3-4: Single-user non-distributed ViRIS configuration

The integration and simultaneous execution of all software components on the same computer system can cause heavy workload for the corresponding machine. Even for small scenebases with only a few virtual objects being processed, the resources such as processing power or storage capacity can reach their limits and prevent real-time interaction within a VE. Therefore, the distribution of computation intensive components to various systems over a local or global area network can help to minimize the workload of a machine. Two sample configuration setups of such a distributed ViRIS will now be described. Figure 3-5 shows the setup of a single-user distributed ViRIS environment. Individual ViRIS components are distributed between two different machines. The VR system 38 Virtual Reality Based Information Systems

components run on the master, the IaC components on another platform. Both platforms are linked over a local or global area network through their networking modules. Such a configuration setup is applicable when a VR system has to be served by a DBMS whose responsibility is to store and manage the scenebase.

Master VR subsystem IS subsystem

SNIPG

WBD SBMM IF IF C/NS DBMS E/KBS C/NS

CB SB

DB MB MeB KB

OS OS

CS PIODSSM LGN CS SM LGN

Local / Global Area Network

Figure 3-5: Sample configuration for a single-user distributed ViRIS integrating a VR and IS subsystem

The configuration shown in Figure 3-6 goes a step further. The hardware and software components for auditory and haptic information processing and generation are distributed each to a single platform. A master machine, labeled as Master VR subsystem, coordinates the other subsystems by message passing mechanisms and handles all the other IO functions (except audio and haptics) locally. A powerful rendering platform serves as main front-end to the user. The IS subsystem acts as a server for scenebase data. The audio subsystem receives data from the master engine for the sonification of audio information. Corresponding components such as a powerful audio board and loud speakers are also part of this subsystem. Finally, haptic information processing and generation functions together with related IO devices are provided by a further separate machine. It is important to notice here that the interfacing framework is inherent to all four subsystems. It enables the seamless implementation and integration of the individual building blocks within a ViRIS. It offers a homogeneous environment for eased system maintenance and extension. The interfacing framework, together with standardized but Models for Virtual Reality Based Information Systems 39

efficient data transfer protocols for networking distributed ViRIS subsystems form the basis for protection of investments: New releases of subsystem modules can be exchanged with existing ones with minimal or no effort and costs.

Master VR subsystem IS subsystem

SNIPG

WBD SBMM IF IF C/NS DBMS E/KBS C/NS

CB SB

DB MB MeB KB

OS OS

CS PIODSSM LGN CS SM LGN

Local / Global Area Network

Audio Subsystem Haptics Subsystem

SNSPG HKIPG

WBD WBD IF IF C/NS C/NS

OS OS

CS PIODSSM LGN CS PIODSSM LGN

Figure 3-6: Sample configuration of a distributed ViRIS applying separate machines for the processing and generation of audio and haptic information

The transition from single to multi-user environments and system configurations takes place within the involved modules. The communication of changes of user or avatar status and behavior is handled by additional functions within the communication and 40 Virtual Reality Based Information Systems

networking subsystem. The interaction between users is mainly controlled by the SBMM, SNIPG, SNSPG, HKIPG, WBD, and C/NS components. There are no new building blocks needed, but rather a futher enhancement as well as refinement of the existing ones. Also, the increase in the amount of data to manage puts a stronger emphasis on the corresponding modules. DBMS functionality, adapted to distributed multi-user VEs, will be of ever increasing importance for future VR systems.

3.3 Application Domains and Scenarios

The very first VR systems and applications arose from academic and governmental research projects. The focus was mainly on the development and testing of new devices and training environments within the domain of flight and battle field simulation. Enabled by the availability of powerful low-cost computing platforms and peripheral IO devices, VR diffused slowly into new application domains. This led to an increased public awarness of VR technology and induced a further growth of the VR industry. A major driving force was the entertainment industry that discovered VR as a new market opportunity. Today, a fusion of VR and IS technology to ViRISs can be noticed. Some of the most promising application domains for ViRISs will be presented in the following sections.

3.3.1 Virtual Reality Based Geographic Information Systems A Geographic Information System (GIS) is a computer-based infomation system that processes spatially or geographically referenced data [Star90]. The huge market potential for GISs has caused companies who originally focused their activities elsewhere to migrate into this field and adapt their technology to fit the requirements and functionality needed. Adding VR technology to information systems such as GIS offer the potential of experiencing large volumes of highly interrelated data and allows faster and more intuitive problem understanding. Benefits gained from such new intuitive and natural interaction paradigms also have implications for the reduction of information retrieval and manipulation costs. In the Virtual GIS (ViGIS) project at Georgia Tech, a 3D GIS was developed that offers window-based as well as immersive capability for the interaction with terrain-based databases [Riba94] [Koll95]. Unique terrain data structures and algorithms allow direct manipulation and navigation within large, high resolution GIS datasets at high frame rates. ViGIS provides techniques for exploring VEs containing DEMs, textures from satellite images, GIS raster layers, buildings, vehicles, and other objects. The Application Domains and Scenarios 41

application domain of ViGIS was in the military field. One of the conclusions of the ViGIS projects is that VR and GIS technology can be successfully combined. The virtual GIS room, presented in [Neve95], transparently integrates VE and GIS with communication of objects between application domains using interoperable objects technology. Taking a real GIS room as user interface metaphor, digital terrain models (DTMs) can be draped on a virtual workbench. Several layers of GIS related information maps, used to overlay the DTM, hang as portraits on the wall surrounding the workbench. Several map layers can be projected simultaneously onto the DTM to generate different views of a spatial data set for better analysis and understanding of interrelated issues. The fusion of VR with GIS technology to a VR based GIS (ViRGIS) opens new ways for interaction with geographic datasets. Many research projects make it clear that there has to be a focus on efficient data management and access techniques to fulfill the requirement for real-time performance within virtual worlds. Sophisticated data access structures, clustering of neighbouring objects, Level-of-Detail, and parallel loading of scene data have to be carefully designed and implemented to maintain lag free performance and interaction within the virtual environment. The loose coupling of system components allows flexible modification and extension of the ViRGIS database with new entities, which can be used by any other multimedia or VR based application. The application of advanced DBMSs offers the possibility to manage large amounts of data and launch data queries that go far beyond the capability of traditional file system based interactive simulation systems.

3.3.2 Virtual Reality Based Medical Information Systems The underlying idea of a Virtual Reality based Medical Information System (ViRMIS) is in the possibility for data type sensitive visualization and interaction with medical data and information. A user, represented as a patient or a medical professional, should gain ubiquitous, and intuitive to use access to any kind of medical entities [Szabo95d]. Connecting a ViRMIS server to a global area network like the Internet, together with ubiquitous computing technology [Weis91], personal medical data can be accessed and updated at any place and at any time. Wide-scale acceptance and application of such a ViRMIS concept requires broadly supported data format standards, publicly available data management frameworks, access and usage of IaC infrastructures at fair costs, and easy to use applications and peripheral devices for the visualization and interaction with n-dimensional medical data, hopefully with low data transfer lags. One example of a ViRMIS is DIGIHOM, a 3D interactive anatomy atlas of the human body [And95]. DIGIHOM´s idea is to support and help in the understanding of human 42 Virtual Reality Based Information Systems

anatomy and the interrelations between body organs or systems. An underlying DBMS offers the possibility to query and subsequently visualize certain anatomical regions, systems and organs as 3D models. Using 3D input devices such as a spacemouse or tracker-enhanced 3D joystick together with stereo viewing by LCD shutter glasses, the user can interact and explore a 3D anatomical model. The prospective potential application domains seen by the developers of DIGIHOM are medical teaching, pre- operative planning, patient guidance, and diagnosis. The implementation and setup of an Internet based ViRMIS infrastructure for the management of personal medical data can also enable doctors in emergency cases to access, process and update digital anatomical data of the human body profile, the medical history or personal data of a patient for purposes like diagnosis or surgery planning. This can significantly ease, shorten and reduce the costs of the therapy process. The prerequisite for such a globally available personal medical data base is the previously mentioned existence of broadly supported data format standards as well as mechanisms that guarantee easy access to the needed resources across various computer platforms. Furthermore, corresponding access controls (passwords, magnetic cards, finger print tests, voice and face recognition, etc.) for authentication and authorization have to prevent the misuse of data. The ever increasing networking of computer systems will multiply the supply of medical services over the Internet or virtual private networks in the future. Finally, VR based interaction with large-scale medical data puts serious challenges to the design and implementation of such systems. Multi-sensory feedback and fidelity, interactivity between objects, respect for physical and physiological object properties are some of the criteria to be met for ViRMIS environments in order to become successful and realistic for their users [Sata95].

3.3.3 Virtual Reality Based Facility Information Systems The increasing interest in "information highway" environments and the management of company-wide resources require more sophisticated models and techniques for the development of data query, access, and interaction strategies. Also, the fusion of different technologies challenges the maintenance of large infrastructures. Virtual Reality based Facility Information Systems (ViRFIS) offer the potential to ease these tasks and allow transparent, efficient and effective methods for design, optimization and management of large-scale infrastructures. Two domains seem to be interesting in this context: the management of telecommunication networks and corporate resources. The expected liberalization of the telecommunication market will challenge the telecommunication industry world-wide. Telecommunication companies will be faced with an ever increasing pressure of competition and complexity of network infrastructure. The need for adequate models for network design and management is obvious and will Application Domains and Scenarios 43

play a key factor for successful competition in the future. Different joint projects of academic and industrial institutions like [SPP95] are investigating such ViRFIS application scenarios and are developing advanced Virtual Reality based frameworks that support telecommunication network system design, optimization and management. The help of 3D interaction and visualization is expected to yield a much better understanding of space-related information and a much faster problem solving process. This will finally result in shorter design, implementation, and service management life cycles. Xerox PARC´s Information Visualizer, a prototype system for efficient storage, retrieval, manipulation and visualization of large information spaces, in combination with their 3D/Rooms metaphor may be regarded as an early desktop ViRFIS environment [Robe93]. Organization charts, office layouts and allocation within a building, document indexes, schedule timelines, and operating plans were visualized and can be manipulated by different interaction techniques in 3D space. The objective was to bring down the costs for information retrieval and manipulation to a minimum by the application of an extended workspace, sophisticated software agents to support routine and simple tasks, high frame rates, and perception centered presentation of information entities. This project created a lot of interest in the academic field. British Telecom, the UK´s largest provider of telecommunications services, realized a "proof of concept demonstrator" for the application of VR technology in network management. A VR system is used as a tool for displaying the topological structure of their telecommunications network. It allows the user to navigate through the network structure, represented as nodes and links, and to observe its behaviour from a number of perspectives [Lowe94]. At Columbia University, a prototype of a Virtual Reality system for network management was designed and implemented. In this system the user has the opportunity to interact with a 3D representation of a network whose topology and behaviour can be specified [Fein93]. The ever increasing amount, types, and sources of information being produced, and available for any kind of domain, presses the need for better information management tools. The increasing demand for mobile data, multimedia applications, and the expected deregulation of IaC market environments requires powerful resource design and management environments. ViRFIS concepts as new means for efficient and effective configuration and maintenance of heterogeneous IaC infrastructures offer the potential for the future to support these tasks. A ViRFIS environment can assist decision makers in exploring and understanding more rapidly and better the effects of business, society, and technology related parameters on feasibility and cost issues. Due 44 Virtual Reality Based Information Systems

to its novelty, the design, implementation, and usability testing of ViRFIS application scenarios have to be processed to generate sufficient data to help elaborate the critical success factors for such systems.

3.4 Summary

In this chapter, the nature and underlying concepts of information systems have been laid out. A model of the basic building blocks of an IS has been elaborated that is relevant for the characterization and classification of existing and new information systems. The fusion of previously defined IS and VR building blocks leads to a new type of system architecture for Virtual Reality based Information Systems (ViRIS). With the increased availability of ViRIS type applications in the future this ViRIS model will again be important when comparing and classifying systems and their configuration setup. Based on the ViRIS architecture model, various system configurations can be designed and implemented. Four different configuration classes have been proposed: single-user non-distributed, single-user distributed, multi-user non-distributed, and multi-user distributed. The fact that multi-user aspects will be more prominent in the future raises the demand for adequate data management mechanisms to support multi-user virtual environments. Database management systems will therefore gain more attention in the future within ViRIS related research projects. Finally, three different application domains for ViRISs were presented: VR based Geographic Information Systems (ViRGIS), VR based Medical Information Systems (ViRMIS), and VR based Facility Information Systems (ViRFIS). Adding VR technology to information systems offers the possibility to experience large volumes of highly cross-referenced data and allows faster and more intuitive problem understanding. Benefits gained from such new intuitive and natural interaction paradigms also have implications for the reduction of information retrieval and manipulation costs. The global diffusion of Internet and World Wide Web technology has established a basis for widespread access and interlinking of all kinds of datasets. Chapter 4

Methodology for the Evaluation of Virtual Reality Application Systems

In this chapter, a methodology for the evaluation of Virtual Reality application systems is proposed and described. It serves as a guideline to systematically investigate the various design and implementation aspects of VR applications. This helps to make VR applications more comparable to each other and to better separate their special intrinsic functional features, interaction principles, and development issues. As highlighted in Chapter 2, the consideration of human factors and perceptual processes are important for the development of user-adequate VR systems. Therefore, a new design and classification model is also proposed to classify and describe VR usability tests. This model can also be applied for the specification of VR applications. It helps to ease the decision process in the evaluation of the optimal configuration setup for the targeted VR system.

4.1 Overview

A new methodology will be presented that serves for the evaluation of models, metrics, and metaphors of VR based systems. This methodology, illustrated in Figure 4-1, enables the discussion of various VR research issues in a systematic way. The methodology mainly consists in the integration and simultaneous consideration of different aspects relevant for the development of user-adequate VR applications: the task and application space, the design space, the metaphor space, the metric space, system and usability testing, and the resulting implications for the modification and further refinement of the VR application environment. 46 Methodology for the Evaluation of Virtual Reality Application Systems

Design Task and Space Application Space

Metrics Metaphor Space Space

System and Usability Testing

Figure 4-1: Methodology for the evaluation of Virtual Reality based systems

In the following sections, the individual elements of the methodology and their mutual depedencies, as indicated in Figure 4-1, will be described.

4.2 Task and Application Space

The design and implementation of a VR application system is strongly dependent of the target application field. Therefore, the specification of task and application specific functions, processes, and constraints have to be considered during the whole VR system development process. This specification builds the task and application space within the proposed methodology. The task and application space determines the setup of system components and final operating environment, the definition of adequate user interface metaphors, and the design and application of metrics to evaluate the quality of the system. The definition and configuration of the individual hardware and software components of the targeted VR environment relates heavily on the application domain and tasks that have to be supported. For example, a driving simulator requires the provision of steering wheels, pedals, and instruments to offer a realistic environment to the user and to maximize his degree of presence. Finally, the consideration of all the relevant tasks in the specification process requires a corresponding deep knowledge about the involved application domain. For example, Task and Application Space 47

the design and implementation of a VR based financial information system requires the consultance of a financial analyst in order to model financial data, processes and their interrelationsships correctly. The constitution of an interdisciplinary development team that integrates the neccessary disciplines and application specific know-how helps to specify the task and application space.

4.3 Design Space

The setup of every Virtual Reality testbed environment and the implementation of different VR protoype applications require a deep understanding of VR application design criteria. The high pace development of new peripheral devices and VR toolkits make it necessary to verify, refine, and continually adapt VR system development methodologies and design concepts. The design space identifies and lays out the different factors that contribute to the successful development of VR systems. It builds a common basis for all VR application domains. The VR design space is subdivided into four views: component view, perceptual view, development view, and metaphor view. This is shown in Figure 4-2.

component view

development metaphor

view view

view perceptual

Figure 4-2: Design space for Virtual Reality systems

4.3.1 Component View In the component view, the various VR building blocks explained in Chapter 2 are characterised. The description of the individual system components such as the hardware and software architecture, data structures, or algorithms gives a further insight into the underlying mechanisms of a VR system. The layout of the various component 48 Methodology for the Evaluation of Virtual Reality Application Systems

parameters provides a technical description within the design space and constitutes the technological framework of the design process. The evaluation of different VR components, such as Head-Mounted Displays (HMDs) or Image Generators (IGs), shows that the number of parameters describing a VR device or subsystem often spans a n-dimensional parameter space, where n is greater than three. Therefore, adequate representation models have to be applied ot offer a clear view of component parameters and their values. As an example, the value spectrum of the binocular Field-Of-View (FOV) among different HMDs (Figure 4-3a) and eight parameters for a sample HMD (Figure 4-3b) are illustrated in Figure 4-3 using an 8- dimensional Kiviat diagram.

single eye horizontal FOV HMD3 (75.30) single eye vertical HMD4 HMD2 binocular FOV FOV 0 (58.4 ) (900)

binocular display HMD5 HMD1 overlap source (60.60) (LCD)

HMD6 HMD8 horizontal weight resolution (2.5 lb) HMD7 (185 pixel) vertical resolution (139 pixel) binocular device FOV (deg.) VPL EyePhone Model 2 (HMD1)

VPL EyePhone Model 2 (HMD1): 90 VPL EyePhone LX (HMD2): 108 VPL EyePhone HRX (HMD3): 106 LEEP Cyberface 2 (HMD4): 140 Virtual Research Flight Helmet (HMD5): 100 Virtual Research Eyegen3 (HMD6): 40 Virtual Research VR4 (HMD7): 67 Virtuality Visette (HMD8): 120

a) b)

Figure 4-3: Presentation of parameter values for head-mounted displays (a) value spectrum of sinlge parameter; b) value spectrum of multiple parameters)

This kind of presentation method offers a visual profile for the n-dimensional parameter spectrum of VR components. It helps to identify characteristical shapes for VR system setups that can serve as visual classifiers for VR system setups. Design Space 49

4.3.2 Perceptual View The perceptual view, illustrated in Figure 4-4, is concerned with the channels of perceptual information processing. The question "what stages are involved between concept design by the designer and concept perception by the user" is investigated.

visual channel virtual environment perception and auditory channel (designed and mental

implemented haptic channel understanding

by the VR kinesthetic channel of the virtual application environment olfactory channel developer) by the user ......

Figure 4-4: Perceptual view within VR design space

The difficulty in designing user adequate immersive VR application environments is that several perceptual information processing channels have to be tuned and synchronized to each other simultaneously. Fundamental knowledge about the human perceptual system is required. This kind of know-how is often not available in the VR system development team. Conceptual models for the description of the various perceptual transformation processes can help to overcome this lack of expert knowledge. Therefore, a new design and classification concept is proposed [Szabo95c]. It helps to work out the parameters for immersion and their effectiveness on human perception and to describe and to categorize several empirical findings in a straightforward way. It serves as a design scheme for the development of user-friendly VR applications. For the description of VR usability studies it can be of extreme value, if test conditions will be classified in the context of such a design concept in the future. This kind of design and classification concept will significantly ease the comparison of usability studies. The design and classification concept discriminates four different domains: 1. the dimension of the computer internal world model, typically characterized by internal data structures or topological dimensions to describe virtual scenes and objects, 2. presentation effects based on different techniques to visualize and to make the objects visible or heard, 50 Methodology for the Evaluation of Virtual Reality Application Systems

3. human perception mechanisms of the presented objects and environments, and 4. the conceptualization of the world´s dimensions in the user´s mental model. Different transformation processes relate these dimensions to each other by [Drös74], [Metz74], and [Neis76]. The user´s perceptual feeling of being immersed in the context of new multi-dimensional user interfaces can be achieved by different presentation effects based on several techniques. On the user´s side, psychological and physiological factors affect the success of viewing and interpreting. In a first attempt, the concept is applied to the visual information processing channel by investigating depth cues, as presented in Figure 4-5. Depth information within a virtual environment can be achieved by different techniques. In this model, a look is taken at the amount of computed images or viewpoints to present a virtual scene to the user. Single image techniques with only one viewpoint, such as superposition or size perspective, are most common to produce depth cues. Superposition means that near objects overlap objects fare away from the user´s viewpoint. Size perspective means that nearer objects are bigger than those further away. With new visual display technologies, stereoscopic viewing can be generated. The two images to produce the stereo effect are either presented simultaneously (head mounted displays; passive red- green glasses) or consecutively (passive polarized glasses; active LCD shutter glasses).

designer´s conceptual view user´s conceptual view

dimensions visual depth visual depth concept of the presentation perception of the world model effects (vde) mechamisms (vdm) world model

n-D 1.0 one image 1.0 monocular n-D environment 1.1 shadowing&lighting 1.1 perspective environment 1.2 color 1.2 figure-ground physical 1.3 superposition psychological 1.3 grouping constraints 1.4 relative size constraints 1.4 patterning (e.g., gravity, 1.5 height in plane (e.g., capacity, 1.5 Gestalt laws space model) 1.6 gradient of texture motivation) 1.7 linear perspective n-D 1.8 size perspective n-D object-1 2.0 binocular object-1 2.0 two images 2.1 disparity attributes attributes 2.1 stereopsis 2.2 accommodation (e.g. color, (e.g., value, 2.3 convergence volume, structure, mass) 3.0 multiple images semantic) 3.1 holographics Collection 3.2 motion perspective 3.0 movements Collection 3.1 eyes n-D 4.0 head coupling 3.2 head n-D object-m 4.1 motion parallax 3.3 body object-m

Figure 4-5: Design and classification concept for the visual channel of nD user interfaces Design Space 51

The nD design concept, presented in Figure 4-5, is used to describe usability test results including their test setup conditions. The following example explains, how this design and classification concept can be used to describe usability experiments. In [Szabo93] an experiment is described where the hypothesis that 3D perception based on anaglyph (vde:{1.4, 1.5, 2.1}) is better than a 2.5D presentation based on monocular depth cues (vde:{1.4, 1.5}), could not be proven. Subjects were instructed to solve maximally 10 "cube" tasks during a trial of exactly five minutes. The subjects had to choose one cube out of 10 (cubes 141 to 150) and to look for the congruent cube among five cubes (cubes a to e). To do this, the test person had to transform the cube internally by mental rotation. The number of correctly identified cubes (#CIC) have been measured. The measure #CIC was standardized (#SCIC) to adjust the age. Figure 4-6 shows how the experimental settings and results can be represented by the previously described design and classification concept. A more detailed discussion of this empirical investigation can be found in [Raut93].

designer´s conceptual view user´s conceptual view

dimensions visual depth visual depth concept of the presentation perception of the world model effects (vde) mechamisms (vdm) world model

1.0 one image 1.0 monocular 1.1 shadowing&lighting 1.1 perspective a b c 1.2 color 1.3 superposition 1.2 figure-ground #SCIC2.5D 1.4 relative size 1.3 grouping 1.4 patterning = 106 d e 1.5 height in plane 1.6 gradient of texture 1.5 Gestalt laws 1.7 linear perspective 1.8 size perspective 2.0 binocular 2.0 two images 2.1 disparity #SCIC3D 2.1 stereopsis 2.2 accommodation = 104 141142143 2.3 convergence 3.0 multiple images 3.1 holographics 144145 146 3.2 motion perspective 3.0 movements 3.1 eyes 4.0 head coupling 3.2 head 147148 149 150 4.1 motion parallax 3.3 body

Figure 4-6: Design and classification concept for the visual channel applied for experiment described in [Szabo93]

Ware et al. [Ware93] showed that motion perspective is more important than stereo. The best combination is head coupling with stereo. Monocular depth cues only based on vde:{1.3, 1.4, 1.5} scored low in error performance. The first step of user improvement (33%) could be reached adding vde:{2.1}. The monocular depth cues with additional 52 Methodology for the Evaluation of Virtual Reality Application Systems

head coupling (vde:{4.1}) improved user performance by 83%. The greatest improvement (94%) was measured with vde:{1.3, 1.4, 1.5, 2.1, 4.1}. Pfeffer et al. [Pfeff91] could show the following improvements of distance estimations for near distance areas (6–8 meters): 18% for vde:{1.3, 1.5, 1.6, 1.7, 1.8} compared with vde:{1.3, 1.5, 1.7, 1.8}; 26% for vde:{1.2, 1.3, 1.5, 1.7, 1.8} compared with vde:{1.3, 1.5, 1.7, 1.8}; 35% for vde:{1.3, 1.5, 1.7, 1.8, 2.1} compared with vde:{1.3, 1.5, 1.7, 1.8}. On the other hand, Neisser [Neis76] could show that the quality of perception is strongly influenced by movements (vdm:{3.1, 3.2, 3.3}). The importance of motion for visual perception caused for example by changes in illumination on receptors are necessary to perceive objects (vde:{2.1, 3.2, 4.1} or vdm:{3.1, 3.2, 3.3}), as demonstrated by Cornsweet [Corn70]. In a second attempt, the concept is applied to the auditory information processing channel by investigating acoustical signals, as presented in Figure 4-7.

designer´s conceptual view user´s conceptual view

dimensions acoustical signal acoustical signal concept of the presentation perception of the world model effects (ae) mechanisms (am) world model

n-D 1.0 uni-directional 1.0 monaural n-D environment 1.1 frequency 1.1 pitch environment 1.2 amplitude physical 1.2 loudness/ psychological 1.3 overtone intensity constraints constraints 1.3 timbre (e.g., transmis- (e.g., capacity, sion medium, 2.0 bi-directional motivation) room acoustic) 2.1 horizontal position 2.2 horizontal motion n-D n-D perspective 2.0 binaural object-1 object-1 2.1 location attributes attributes 3.0 multi- directional 2.2 motion (e.g., value, (e.g. color, 3.1 three dimensional structure, volume, position semantic) mass) 3.2 three dimensional motion perspective Interaction Interaction 3.0 movements 3.1 ears by hands n-D n-D 4.0 head coupling 3.2 head object-m object-2 4.1 motion parallax 3.3 body

Figure 4-7: Design and classification concept for the auditory channel of nD user interfaces

The nD design concept can again be used to describe empirical results done in the context of auditory information perception. Design Space 53

Stevens and Newman [Stev36] presented data on sound localization in a free field. The explanation maintains that non transient tones are localized through two sets of cues: for low frequencies, temporal cues are dominant; for higher frequencies, intensive cues are dominant (ae:{1.1, 1.2}). In the midrange, neither cue is effective and localization errors occur. For complex sounds like noise and impulses, both cues operate simultaneously. However, the cues of Stevens and Newman [Stev36] are not effective in localizing sound in a non-free field. In this case, head movements provide cue information (ae:{1.1, 1.2} and am:{3.2}) [Wall40]; without movement, the time of arrival of the initial transients in the wave front provides a reliable cue of location when compared to the second (echo) transient [Wall49]. The research on lateralization, "locating" the sound image within one´s head when sound is presented dichotically with earphones, has indicated that time and intensity cues are both operable in a complementary manner (ae:{1.1, 1.2, 3.1} and am:{2.1}). In [Raut94b] and [Raut94c] an experiment was carried out to estimate the effect of sound feedback of hidden events. Eight computer science students operated a process simulation program of an assembly line with computer numeric controlled (CNC) robots. Relevant information of disturbances and machine breakdowns was given only in a purely visual (vde:{1.0} and vdm:{2.0}), and in a audio-visual form (vde:{1.0}, vdm:{2.0}, ae:{2.1} and am:{2.1}). The results indicate that additional sound feedback of hidden events significantly improves the operator´s performance and positively increases some mood aspects of users. The description of many more usability experiments using this model allows the generation of a database that serves as a look-up table for the usability figures of certain system configurations. Such a usability database can help VR system designers to determine the optimal setup of system components for a specific task and application space.

4.3.3 Development View The third view within the VR design space is concerned with application development aspects. In Figure 4-8 a general VR Application Development Model (VRADM) is proposed. It serves as a VRADM skeleton that can be applied for a VR application development project. 54 Methodology for the Evaluation of Virtual Reality Application Systems

System Requirements Definition Requirements

Preliminary System Definition Target System & System Evaluation Benchmarks

Implementation Offer & Acceptance Kick-Off

User Interface & System Design

Database Device Acquisition, Interfacing, Programming Interfacing Processing, Modeling Protoype 1 .. n

Scene and Object Geometry Drivers Interaction Environmental Effects Filtering Data Access Scene and Object Behaviour Calibration Presentation Texture Maps Communication Dynamics Application Related Data Communication

System Testing & Tuning Final System

System Installation & Operator Training

Figure 4-8: General VR Application Development Model

The design and implementation of VR applications is a highly iterative process. In a first stage, the overall system environment (hardware and software) has to be determined. This is mainly driven by the overall system requirements and benchmark figures. System benchmarks can be generated by measuring the runtime performance of a system configuration and scenebase that is highly comparable to the targeted system. After this, the final implementation phase can be initiated. Several design-modeling- programming-testing-tuning cycles lead to early application protoypes which can be used to refine or modify the system design and implementation. The system has to be tuned for real-time performance and perceptual smoothness. This tuning process can cause additional refinements or modifications of the scenebase, device interfacing and system programming. Finally, the end-system has to be installed and rechecked at the Design Space 55

site where it will be run. Careful training of system operators help minimize maintenance effort.

4.3.4 Metaphor View and Space Within the VR design space, a metaphor oriented view during application design and implementation helps to address user interface issues in an adequate way. Therefore, the metaphor view or space requires a special focus within the methodology as indicated in Figure 4-2 by drawing the metaphor space in a separate box. The metaphor space is primarly determined by the task and application space. The implementation of the various interaction paradigms within a VR application can significantly differ from application to application. It is strongly dependent of the other previously presented views within the system design space and the application domain. Therefore, a carful evaluation of these aspects has to be undertaken and drive the design and implementation of adequate user interface models and metaphors. User interface metaphors are derived from the concrete specification of the targeted application evironment and related objectives. For example, the implementation of a realistic flight simulator requires the installation of a real cockpit to offer the pilot a high degree of immersion. Application specific interaction concepts have to be considered and implemented by the combination of adequate hardware and software components. As indicated in Figure 4-2, not only application specific, but also component, perceptual and development aspects interact with the metaphor space. The desired metaphors determine the configuration of the underlying VR system environment and influence the individual development phases. Metaphors also have to fit the perceptual information processing system of the user. The implementation of a well designed user interface metaphor becomes obsolete, if the user feels uncomfortable when being immersed into a virtual environment.

4.4 Metrics Space

Beside the previously explained spaces, metrics have to be defined to build the metrics space. They are derived from the underlying task and application domain as well as from the implemented system components within the design space. These aspects determine the configuration of metrics for the evaluation a VR application environment during system and usability testing. In the 80s, early metric concepts have been applied to measure the complexity of program code, to improve the software development process and software quality, and 56 Methodology for the Evaluation of Virtual Reality Application Systems

to generate better cost estimations [Möll93]. Metrics are methods to measure the characteristics and behaviour of systems and processes. The application of metrics results in data sets and figures that quantitatively or qualitatively describe the object of investigation. They can be used to improve the quality or productivity for designing, implementing, re-enginering, or controling of systems and processes. The use of a small and simple but expressive metrics system allows the generation of easy to understand and readily available test data for their acquisition. The data generated by the application of a metrics system should be acquired and computed with a minimal effort. Base metrics can be derived from the specific figures of the involved system components and runtime characteristics of testsets on the targeted system environment. A perceptual metrics view of the VE shows up, how many perceptual channels are addressed by the system setup including related human scales. It also shows how many mapping alternatives from designer´s to user´s conceptual view are potentially possible by the targeted system configuration. Development-oriented metrics give feedback about the system life cycle.

4.4.1 Objective Metrics Objective metrics allow the generation of first quantitative numbers about a system or methodology. Their advantage of objective metrics lies in the fact that they are relatively easy to measure, to collect and to quantify. The application of corresponding software tools help to automate these processes. Examples for objective metrics for VR systems are display resolution, tracking latency, field-of-view angle, device costs, network bandwidth, or frame rate. Within usability testing, objective metrics are applied to measure the efficiency of system use. Typical quantifiable usability metrics are time needed to carry out a specific task or set of tasks; number of tasks completed within a given time limit; time to learn how to appropriately interact with a system; number or rate of user errors while carrying out a specific task or set of tasks; ratio between successful interactions and errors; time spent recovering from errors; or retention over time of system features and behaviour [Niel93] [Shne92]. Based on elementary metrics, additional derived metrics such as resolution-price-ratio or price-performance-ratio can be defined. As a result, a metrics framework can be set up that serves for quantitative identification of the features and behaviour of an application environment or its development phases.

4.4.2 Subjective Metrics The application of psychophysiological measures as objective metrics, such as EEGs, pupil dilation, or heart rate, to estimate user´s stress and comfort level are often Metrics Space 57

inappropriate because of intimidating and discomfortal experimental conditions for the test person. Therefore, subjective metrics are used instead that are applied within interviews, polls or questionnaires. Subjective metrics try to acquire impressions, feelings, and opinions of an individual before, during and after the use of a system. The aggregation of subjective measures by asking multiple users allows the generation of an objective measure about system´s pleasantness or satisfaction. Subjective measures may be derived from the individual statements of users who used the system. These may consist of statements about the personal feeling of presence or immersion, the user´s opinion about a VR system and the user´s general condition such as emotional or health state.

4.5 System and Usability Testing

Finally, the designed and implemented VR application environment has to be tested against performance behaviour and usability applying objective and subjective measures from the metrics space. Testing is relevant to evaluate, if the specification is met, and allows to determine critical system components, development phases, or perceptual processes for optimization. Furthermore, the availablity of many VR reference figures makes various VR solutions more comparable to each other. It also helps to quantify the benefits of a VR application environment compared to alternative approaches. The results and findings gained from system and usability testing help to refine the design, metaphor and metrics space for a specific VR application environment. The importance of user interface aspects within VR applications requires a special focus on usability testing. Multiple components constitute the usability of a system. Gene- rally, usability is associated with the following five attributes [Niel93]: 1. learnability (how easy is it to learn), 2. efficiency (to achieve a high level of productivity), 3. memo- rability (easy to remember the system´s functionality for later use), 4. errors (low error rates and tolerantce to user´s actions), and 5. satisfaction (pleasantness to use for subjective satisfaction). It applies to all aspects of a system with which a human might interact, including installation and maintenance procedures. [Niel93] identifies the following stages of a usability test: 1. preparation: setup of testbed environment 2. introduction: explanation and sample run to get familiar with the test scenario 3. testing: carrying-out of usability tests with test persons 58 Methodology for the Evaluation of Virtual Reality Application Systems

4. debriefing: fill in of questionnaires including personal comments by test persons and doing interviews Depending on the number of users to be tested, the test setup, and the resources available for usability testing, different usability methods can be applied separately or in combination. A detailed discussion of usability methods can be found in [Niel93]. Heuristic evaluation of the user interface can be done without any test persons. It helps to eliminate "obvious" problems or violations of widely accepted user interface style guides (principles or paradigms for user interface design followed during system development) such as proposed in [Niev86] by a systematic inspection. Furthermore, thinking aloud, observation of user behaviour, and interviews of test persons for exploratory analysis are also methods that require only a few subjects (typically no more than five) and provide subjective measures. Performance measuring, questionnaires, and statistical evaluation of usage logs requires a large amount of test subjects. Although, these methods are relevant to generate objective measures for system usability.

4.6 Summary

In this chapter, a new methodology for the evaluation of Virtual Reality based application systems has been proposed. It differentiates five steps to systematically describe and discuss various relevant aspects of VR applications: 1) task and application space, 2) design space, 3) metaphor space, 4) metrics space, and 5) system and usability testing. In a first step, the objectives and underlying tasks of the application domain have to be specified. They determine the evaluation, design and implementation of system components and user interface metaphors as well as the VR application development model.

Within the design space, a component oriented view helps to identify the underlying hardware and software architecture. Furthermore, a new design and classification model within the perceptual view has been proposed. It allows to characterize results from usability experiments in a uniform way and make them more comparable to each other. The model can also be used as a look-up table to find out the optimal system configuration for a specific application domain. A third view within the design space, the development view, focuses on the development aspects. A general VR application development model is proposed. In a third step, user interface issues wtihtin the metaphor space have to be evaluated. Summary 59

In a fourth step, adequate metrics have to be defined as a function of the design and application space. They are needed for system and usability testing. Finally, system and usability tests have to be performed in order to generate quantitative and qualitative measures. These help to verify, if the system specification is met, and allow a further refinement and optimization of the VR application environment.

Chapter 5

Development and Evaluation of Virtual Reality Application Prototypes

In this chapter, three VR application prototypes will be presented and discussed by applying the methodology previously described in Chapter 4. This helps to deepen the understanding of the design, metaphor and metric space of VR based information systems. First, the task and application space of the application prototypes will be outlined. Then, within their design spaces, the hardware components and software architectures will be described in order to explain the underlying concepts necessary to design and implement VR based information systems. Issues such as the management of and interaction with large-scale virtual environments, the setup and operation of non- desktop based VR systems, and the evaluation of different IO device configurations will be of main interest. Also, different user interface metaphors for navigation such as fly- over, drive-through and walk-through as well as for scene and object manipulation will be discussed. The carrying-out of system and usability tests applying different metrics give quantitative and qualitative feedback about the goodness and performance of the implemented hardware components, software mechanisms and user interface metaphors. The final discussion of these results serve as a decision base for the proposal of adequate models, metaphors and metrics for VR based information systems. 62 Development and Evaluation of Virtual Reality Application Prototypes

5.1 Task and Application Spaces

The specification of the task and application space of the investigated VR system prototypes is fundamental for the design and implementation of their underlying system architectures and user interface metaphors. Three application prototypes have been designed and implemented. As special foci of attention, the management of a large- scale scenebase, the application of different user interface metaphors, the usability of various peripheral device configurations and the maintenance of a heterogeneous VR system framework have been designated. The following gives a short description about the three investigated prototype application scenarios:

Virtual Reality Based Geographic Information System The basic idea of this project called ViRGIS was to develop a generic model and a prototype architecture for Virtual Reality based Information Systems (ViRIS) in order to investigate new models and techniques for the management of and interaction with large volumes of data [Szabo95a] [Szabo95b]. Such a system setup can be selected as a basic architecture for a variety of IS domains such as a VR based Geographic Information System (ViRGIS) or a VR based Tourist Information System (ViRTIS). As a first interactive 3D application, based on this architecture model, the ViRGIS application prototype was designed and implemented. ViRGIS allows the interactive exploration of the shaded or textured Digi- tal Elevation Model (DEM) of Switzerland, integrating two UNIX workstation platforms over a local or global area network by a client-server architecture. In addition to elevation and texture data, ViRGIS also offers the ability to request cultural data about more than 3000 Swiss locations such as cities or villages.

ErgoSim Driving Simulator for a Lightweight Mobile Vehicle

As a joint project between the MultiMedia Laboratory of the Department of Computer Science at the University of Zurich and the Institute of Hygiene and Applied Physiology at the Swiss Federal Institute of Technology a cab and projection screen based driving simulator was designed and implemented [Naef96]. The project´s objective was to propose an ergonomical cockpit design for a lightweight mobile vehicle in respect to active driving security. A special focus was on the setup of new non-desktop based IO device configurations, and on the evaluation of the usability of drive-through metaphor based VR systems. The tests should also give further insights in the applicability of a cab based VR simulator for driver training. The display and control elements of new versus conventional cockpit layouts have been investigated and compared against each other. The Task and Application Spaces 63

experiments should help in the optimization of display and control elements and in the verification of predefined anthropometric parameters such as viewing conditions, static or dynamic bearings, or freedom of movement.

Virtual Reality Based Tourist Information System The third VR system implements a Virtual Reality based Tourist Information System that allows the interactive exploration of a holiday house. The objective of this project was on one hand to evaluate the user performance under different IO device configurations while performing various tasks and on the other hand to investigate the general user acceptance of a ViRTIS by users.

The following sections describe the architecture and development concepts and highlight the major findings gained from the design and implementation of these VR system prototypes.

5.2 Design Spaces

After having identified the application domain of the three VR application prototypes, the hardware and software elements within the component view will now be described. Besides the outline of the individual development cycles within the development view, also software engineering issues involved in the development of VR based application systems will be highlighted. The perceptual view will be discussed within the description of the system and usability tests.

5.2.1 Acquisition and Management of Large Volumes of Data

During the ViRGIS project, a generic ViRIS architecture called ViRXIS has been developed (see Figure 5-1). The generic nature is based upon the feature that the data stored and provided by the database management system can easily be extended by new 3D objects or cultural data of any type. Also, ViRXIS can be used as a kernel architecture for various application domains that focus on the interaction with spatially related data. 64 Development and Evaluation of Virtual Reality Application Prototypes

ViRXIS-InterActor ViRXIS-DBMS topological texture cultural class input devices data data data definition

interaction manager

file manager loader / saver FORMS

scene concurrency dynamic class generator manager manager

local data ObjectStore

IRIS Performer bitmap hasher R-tree

2D query processor IS-server 2D delete output devices data loader processor

local or global area network

Figure 5-1: ViRXIS system architecture

The ViRXIS architecture consist of two main components: the ViRXIS-InterActor and the ViRXIS-DBMS. These two components are linked together over a local or global area network by a low-level networking interface. The objective of this client-server based approach is to distribute the work load over several system platforms and enable real-time interaction in a virtual environment. Different techniques such as Level-Of- Detail (LOD), dynamic scene management by paging scene data into main memory, and sophisticated spatial data access structures are applied to guarantee an optimal allocation of available resources such as main memory or processing power.

ViRXIS-InterActor The main task of the ViRXIS-InterActor, the component for interactive 3D visualization and navigation, consists in the processing of the user´s input, loading the required virtual scene data into main memory and visualizing it on the graphics display. The ViRXIS-InterActor runs on a SiliconGraphics Workstation using the IRIS Performer 1.2 (Performer) visual simulation toolkit [Perf94] [Rohl94]. Performer is a software Design Spaces 65

toolkit developed by SiliconGraphics Inc. that offers special mechanisms such as multi- processing of runtime scenebase, auto-synchronization of application and rendering processes, LOD management functions and shared memory support for interactive real- time 3D simulation. Performer allocates a visual non-persistent internal scenebase at runtime to store and manage virtual objects and their attributes. The interaction manager handles the interaction between the user and ViRXIS, mainly 2D mouse inputs and control panel manipulations. It forwards interaction events to the scene manager. The graphical control panels are implemented by the help of the FORMS software library [Over92]. All relevant system parameters, such as data loading strategy, scene fragmentation, navigation metaphor and viewpoint positioning, display and environmental effects, LOD setting, or multi-processing configuration can be interactively manipulated and tuned for real-time performance. The scene manager is responsible for the dynamic management of the actual scene data, invokes data load queries and deallocates no longer needed data or patches from main memory. The virtual environment is subdivided into logical 3D rectangular patches that build the virtual scene in the main memory on the simulation platform. This is shown in Figure 5-2.

viewing frustum

far clipping plane LOD3 (high)

LOD2 (middle) FOV near clipping LOD1 plane (low)

FOV = Field Of View patch height patch width

Figure 5-2: Subdivision of the scene into patches with different LODs

LODs of the patches are loaded into main memory and the visual scenebase of Performer as a function of the viewing frustum, defined by the user´s actual viewpoint, field-of-view, viewing direction, clipping planes, and navigation velocity. A patch may contain geometrical, textural, or cultural data. All three LODs related to a patch are held 66 Development and Evaluation of Virtual Reality Application Prototypes

in main memory to guarantee a fast switching between differrent LODs. The applied scene paging strategy is illustrated in Figure 5-3 when multiple LOD loading is enabled.

load new patches from ViRXIS-DBMS

Forward Move

deallocate "oldest" patches LOD3 LOD2 LOD1 (high) (middle) (low)

a) runtime scenebase during paging b) runtime scenebase after paging

Figure 5-3: Scene paging strategy

The concurrency manager implements methods for parallel loading of scene data. To overcome system resource limitations such as available main memory size or number of process table entries a job queue is maintained. This job queue, implemented as a priority queue, stores data loading queries that could not be launched after a request for new scene data. The job queue is processed by a server pool that consists of an individually configurable number of servers. A server is a process that maintains a network connection to the ViRXIS-DBMS, dispatches a job from the job queue and launches the corresponding data load query to the ViRXIS-DBMS. The data load processes operate on the shared memory segment, where the runtime scenebase is stored, to allow a smooth rendering transition between existing and newly loaded scene data. The data loader processes the data load queries coming from the concurrency manager and sends either a query message to the ViRXIS-DBMS or loads the requested data from the local file system. This file loading feature offers the flexibility to implement different data loading strategies. Especially, when network throughput reaches a critical low level for maintaining real-time interaction, or when the ViRXIS-DBMS is temporarly out of order, data (eg. static, non-changing objects over time such as digital elevation data) can be loaded from the local file system instead of the ViRXIS-DBMS over the local or global area network. Design Spaces 67

After the reception of the requested data, the internal visual database of Performer is updated with the newly loaded data, traversed, culled and finally visualized on the graphics screen.

ViRXIS-DBMS Beside the functionality of a real-time 3D interaction environment, mechanisms for object data queries have been implemented and integrated. An object-oriented DataBase Management System (ooDBMS) manages all the entities within the virtual environment. The role of the DBMS is to serve the ViRXIS-InterActor simulation environment with different types of multimedia information entities and store changes of entity attributes. The application of a DBMS offers the possibility to manage large amounts of data and launch data queries that go far beyond the capability of file system based mechanisms generally used by existing VR systems. Various query operations on object attributes can be executed before the actual data is loaded into the local runtime scenebase. This offers also the possibility to evaluate the optimal size and subdivision of the runtime scenebase into patches without hardcoding the scenebase hierarchy. The link between ViRXIS-DBMS and ViRXIS-InterActor is implemented by a Local or Global Area Network (LAN or GAN) via BSD sockets [Bach86]. The IS-server builds the link between the simulation and DBMS environment. It receives data load queries and sends them further to the query manager, where a corresponding data request is initiated to the ooDBMS. To filter the requested objects from the database as fast as possible, a sophisticated spatial data access structure in the form of an R-Tree [Gutt84] is implemented on top of the ooDBMS. To minimize the number of disk accesses and network transfers, the persistent neighbouring objects are stored physically in disk clusters. This is well supported by the chosen ooDBMS and includes corresponding data access mechanisms. The advantage of this clustering principle is that only neighbouring relationships need to be considered within data queries and spatial selectivity can be applied during information retrieval. The existing application prototype only deals with DEM related data and triangles that do not overlap in x/y- dimensions. Therefore, only 2D range queries have to be processed. The ViRXIS-DBMS runs on a Sun Workstation and uses ObjectStore from ObjectDesign Inc. as the underlying ooDBMS. The objective in chosing ObjectStore was to gain the advantages and features of a commercially available DBMS such as data distribution, concurrency control, failure recovery, support of storing multimedia type data structures, persistency, and a virtual memory mapping architecture to guarantee referential integrity. ObjectStore uses C++ as its host language and offers several software tools such as a graphical schema designer to ease the design and development process. 68 Development and Evaluation of Virtual Reality Application Prototypes

A further and more detailed description of the ViRXIS-DBMS can be found in [Asch94] [Kaeg95] [Paja97] and will be a topic of a future thesis.

Scenebase Acquisiton and Contents The most expensive and time-consuming work during the development of a VR application is the acquisition and modeling of the scenebase. The reuse of already existing data helps significantly to keep these expenses at a minimum. The consideration of different data sources made available for use by previous research projects helped building the scenebase within the ViRGIS project with a minimal effort. Data in the form of a DEM, satellite images and cultural information of more than 3000 Swiss locations have been processed, converted and inserted into the ViRXIS-DBMS. This is visualized by Figure 5-4.

SGO ViRXIS-DBMS ViRXIS-InterActor Oberon SGF Topo File Files Rimini Topo Data

Topo_LOD_1.db Topo_LOD_2.db IRIS SUN Topo_LOD_3.db Performer´s Raster Visual DB RGB Raw ESA/EURIMAGE Satellite Image Tex_LOD_1.db Tex_LOD_2.db Oberon Tex_LOD_3.db Data

File Raw Local or Global Area Network

InfObjects.db

Figure 5-4: Data acquisiton and transformation steps

The DEM data of Switzerland was available in a resolution of 250 meters between single vertices. The original data has been converted into the SiliconGraphics Object (SGO) data format in order to ease data transformation from ViRXIS-DBMS to ViRXIS-InterActor and to gain better system performance. On the ViRXIS-InterActor platform, dedicated routines allow an easy loading of SGO type objects. The DEM is stored in three LODs: 1.) 250 meters, 2.) 500 meters, and 3.) 1000 meters. To achieve better performance while transfering data from the DBMS to the simulation platform, the SGO data format was downsized to the new SGF format. The use of this custom format for geometrical information reduced the amount of data to be stored on the Design Spaces 69

ViRXIS-DBMS and transmitted over the network by 55% percent. This transformation also offers a better data parsing and read technique for the simulation component. The texture data is also held in three LODs: 1.) 30 meters, 2.) 60 meters, and 3.) 120 meters. Table A-1, A-2 and A-3 in Appendix A give an overview of resoultion and size of the scenebase stored on the ViRXIS-DBMS.

5.2.2 Setup and Operation of a Cab Based Immersive Environment In the ErgoSim project, a simulation VR system is implemented in order to offer a realistic application scenario to the user. Simulation VR systems places the user inside a physical mock-up of a cabin (oftern called cab or pod) that contains realistic physical control and display elements [Jabob94]. Such systems can often be found as flight or driving simulators. The physical layout of the single system components applied is shown in Figure 5-5.

Projection Screen 2

Projection Screen 1

Projection Screen 3 Driver Cab Display Panel Instrumentation Control Unit Steering Wheel WinPC Pedals (486, 33 MHz)

Drive Control WinPC Video Video Beamer 1 Beamer 3 (486, 33 MHz) Video Beamer 2 Rendering

SGI Onyx RE2 Ethernet (1 Pipe) Channel 1-3 (RGB)

Figure 5-5: ErgoSim system layout

The driver cab is designed and built in respect to the dimensions of the real vehicle. Two seats, various knobs and driving control elements such as a hand-break are installed, too. A display panel using a medium-resolution graphics monitor provides 70 Development and Evaluation of Virtual Reality Application Prototypes

several driving information such as the states of control knobs or the actual driving velocity. A steering wheel and two foot pedals for gas and brake allow the user to drive around in the VE in a realistic way. The values coming from the steering wheel and pedals are read by a PC and sent via Ethernet and TCP/IP protocol to the rendering computer system. A control unit is used to simulate and set the status of different driving instruments such as the flash-light- stick that is not wired with the simulation system. A second PC processes the values coming from the pedals and the control unit and displays this driving information on the previously mentioned display panel in front of the driver. The rendering system´s role is to receive driving information such as the status of the steering wheel and the two foot pedals, recalculate and render the new viewpoint of the user out of the virtual vehicle. Hereby, a simple vehicle dynamics model is implemented. The final display of the rendered frame is done by three video beamers using the SiliconGraphics Multi-Channel-Option (MCO) video splitter board. Additionally, autonomous vehicles can be triggered for driving allong a predefined path. Their position is updated according to a second vehicle dynamics model that is provided by the libpfutil library. As can be seen in Figure 5-6, the ErgoSim application and the vehicle dynamics functions make use of the libpfutil, libpf and libpr libraries from the IRIS Performer Toolkit [Rohl94] and mps_drive client software module. To minimize modeling effort, again an already existing scenebase is reused and enhanced with additional 3D objects such as street signs using the 3D modeler Designer´s Workbench from Coryphaeus Inc. This scenebase is also known as "Performer Toon" and is the downsized version of the widely known "Performer Town" visual database modeled by Paradigm Simulation Inc. The mps_drive module implements a network wide device driving service for the acquisition and distribution of steering wheel and foot pedal data. It is based on the Multi- Port Service (MPS) architecture, a software framework to offer a high-degree of functional availability of peripheral devices for VR applications by the simultaneous combination of networking, user interface and device data processing services. This architecture was developed during the ErgoSim project and will be described in the following. Design Spaces 71

Rendering

ErgoSim application

motion & vehicle dynamics mps_drive IRIS Performer (client)

libpf libpfutil

libpr

Graphics Library (GL) Video Beamer 1 Irix operating system

Reality Multi- 2 CPUs Channel Ethernet Video Engine2 Adapter Beamer Option 2

Video Beamer Instrumentation 3 ErgoSim Display Panel application

Display LabView Panel Monitor Windows 3.11 Steering Wheel DOS operating system

Brake Gas CPU RGB Out A/D Board Pedals

Drive Control mps_drive (server)

DOS operating system

Ethernet CPU A/D Board Adapter

Figure 5-6: ErgoSim system architecture 72 Development and Evaluation of Virtual Reality Application Prototypes

5.2.3 Multi-Port Service Architecture for High Degree of Functional Availability of Peripheral Devices An underlying feature of advanced VR systems is their support for different kinds of peripheral devices. This motivates VR system developers to make use of those features and incorporate them into the user interface model of their applications. Such a hardcoding of functionality offered by peripheral devices makes it often impossible to reuse peripheral function services by other applications. The fact, that devices may temporarly not be active, caused by hardware or software failures, or not be available, caused by running the application with a different system setup, makes many VR applications often become inoperational and unavailable for that period of time. Also, peripheral devices are mostly tightly-coupled to hardware platforms and allow only very limited access to their functionality. This makes it necessary to incorporate and offer new mechanisms that help to avoid these negative effects. The need for a flexible device and subsystem interfacing framework architecture is obvious. The Multi-Port Service (MPS) architecture offers the capability to overcome the above mentioned system failures. The paradigm of MPS is to offer a high degree of functional availability of peripheral devices for VR applications. Combining network, user interface and device data processing services in a unique way enables a network-wide distribution and open access to the functionality of peripheral devices by many applications. Another advantage of MPS is that it enables cooperative and parallelized development of VR applications. Synchronizing each system developer by predefined data structures and interface protocols, both device interfacing as well as application developers may design and implement their software modules independently from each other. The model schema of the MPS architecture is shown in Figure 5-7.

input output input output tui gui ui

connectionless application remote connection-oriented send input file connectionless local mode connection-oriented device ipc data connectionless server application remote connection-oriented client receive output file connectionless local connection-oriented device

mapper

Figure 5-7: Multi-Port Service (MPS) architecture model Design Spaces 73

The MPS model is constituted by four different ports: "data" to acquire or deliver data packages locally, "ui" to display or input parameter values, "mapper" to transform data packages, and "ipc" for network (remote) or bus (local) oriented data transmission. The aggregation of the four ports represent an MPS module that acts either as a server or as a client for a certain data stream. The data flow within an MPS module is the following: server mode: 1. A data stream is read via the data port from a device, file or application. 2. The single values of the data stream are displayed or data may be input by a textual (tui) or graphical user interface (gui) via the ui port. 3. The values of the data stream may be transformed to map the functionality of a device to another device via the mapper port. 4. The data stream is transmitted via the ipc port to another MPS module that acts as a client. client mode: 1. The data stream is received via the ipc port from another MPS module that acts as a server. 2. The values of the data stream may be transformed to map the functionality of a device to another device via the mapper port. 3. The single values of the data stream are displayed by a textual or graphical user interface via the ui port. 4. The data stream is written via the data port to a device, file or application.

The main purpose of the data port is to receive or deliver a data stream from or to a peripheral device. It can be regarded as an enhanced device driver. For testing or logging purposes data is readable from or writable to a file on the local disk. There is also an option offered to interface the data stream with an application. The MPS data port model is shown in Figure 5-8. 74 Development and Evaluation of Virtual Reality Application Prototypes

data glove

Device data file 1 Input File application 1 Application Data Device Output File Application

loud speakers

application 2 data file 2

Figure 5-8: MPS data port model

The ui port has two purposes: 1) to display the single values of the data stream as additional feedback and control information; 2) to input data directly and generate a data stream if the data port may not be activated. It can be achieved by using a textual (tui) or graphical user interface (gui). The ipc port delivers ("Send") or receives ("Receive") values of a data stream to or from other MPS modules (see Figure 5-9). This is done via BSD sockets [Stev90]. Different domains and protocols can be addressed: Unix domain ("UNET") for local interprocess data transmission over the local system bus, Internet domain ("INET") for remote interprocess data transmission over a local or global area network, "Stream" via a connection-oriented TCP/IP protocol, and "Datagram" via a connectionless UDP/IP protocol.

Datagram Remote INET Stream Send Datagram Local UNET Stream IPC Datagram Remote INET Stream Receive Datagram Local UNET Stream

Figure 5-9: MPS ipc port model Design Spaces 75

Finally, the mapper port serves to simulate the functionality of special devices (e.g. a dataglove or a 3D mouse) by more widely used and available devices such as a 2D mouse or an alphanumerical keyboard. The mapper port model is illustrated in Figure 5-10. Through it for example, the operation of a spaceball can be simulated via a 2D mouse by mapping the mouse values and events to the one of a spaceball and pass those mapped events or values further to the ipc port. The advantage in such a concept is that unavailable or temporarly inoperational devices may be virtually used within an application environment by simulating their functionality via another device. By applying such a mapping mechanism, unwanted side-effects caused by hardcoded device funtionality together with a lack of the corresponding device can be overcome. This helps increase the overall degree-of-service of critical devices within a VR system environment and makes it become more robust.

Mapper

Events: Events:

MOUSE1 SBTransX SBKey1 MOUSE2 SBTransY SBKey2 MOUSE3 SBTransZ SBKey3 LEFTMOUSE SBRotX SBKey4 MIDDLEMOUSE SBRotY SBKey5 RIGHTMOUSE SBRotZ SBKey6 MOUSEX SBButton SBKey7 MOUSEY SBKey8

Figure 5-10: MPS mapper port model

Several prototype modules based on the MPS architecture model such as mps_pedal for analog and digital foot pedals, mps_speech for speech recognition, or mps_dial for dial button support have been implemented. Figure 5-11 lays out two setups where the mps_pedal module is applied to navigate within a VE using a foot pedal. The involved settings within the single ports are highlighted in grey. Figure 5-11a shows the bus- oriented and Figure 5-11b the network-oriented implementation. 76 Development and Evaluation of Virtual Reality Application Prototypes

Client and Server on the Same Platform:

Server Client MPS_Pedal MPS_Pedal

Pedal 1 Pedal 2 Pedal 3 Pedal 1 Pedal 2 Pedal 3

ErgoSim

Local Bus

Client and Server on Different Platforms:

Server Client MPS_Pedal MPS_Pedal

Pedal 1 Pedal 2 Pedal 3 Pedal 1 Pedal 2 Pedal 3

ErgoSim

Local Area Network

Figure 5-11: Application example for mps_pedal in a bus- (a) and network-oriented (b) setup

The MPS approach can also be used to implement further network-wide services such as a data format conversion or vehicle dynamics service. Finally, an administration service called mps_admin has been designed and implemented to initiate and manage network-wide available MPS services in a user-friendly way [Aman95]. Design Spaces 77

5.2.4 VR Application Prototyping with a High-Level Authoring Environment Within the ViRTIS project, the exploration of a holiday house served as application scenario, where the user had to complete the following four tasks: 1. Controling the availability and the number of chairs and desks on the balcony. 2. Testing the functioning of water tubes in the bathroom and kitchen. 3. Testing the functioning of light switches in two bedrooms. 4. Controling the availability of plates for five persons in the kitchen. The tasks consisted of navigational, picking and manipulative operations. The ViRTIS application prototype bases on the dVS Toolkit 3.0 and dVISE 3.0 runtime authoring environment from Division Inc. [dVS94]. dVISE allows the rapid prototyping of immersive virtual worlds without any programming. By the help of 2D control panels and 3D widgets the geometry, visual attributes and interactive behaviour of virtual objects can be defined. A rich set of tools is also provided to import, process and optimize existing geometric and image data. Besides, many peripheral device setups are supported and easily configurable. An already existing dVISE scenebase in the form of a scene description file shipped with the dVISE software package has been re-used, modified and extended by additional objects and behaviours. 3D modeling of additional virtual objects such as plates was done using Designer´s Workbench from Coryphaeus Inc.

5.2.5 System Development Models The previously described hardware and software elements developed and used within the three VR projects have given the component view of these VR application scenarios. Now, a focus will be made on the development view. It helps to verify the VR system development model proposed in Chapter 4 and to identify development phases that are common to all three projects. This development view has to be seen as a contribution to other existing but still scarcely described VR development models in the literature.

ViRGIS Project This project consisted of two major development cycles. At the beginning of every major cycle, system requirements had to be defined. Also, the applied hardware and software components of the testbed environment as well as reused data sets had to be carefully evaluated and specified. In both major cycles, system design, implementation, testing and tuning was done by several iteration cycles as indicated in Figure B-1 in 78 Development and Evaluation of Virtual Reality Application Prototypes

Appendix B. In the first major project cycle, the scenebase consisted only of DEM data. Therefore, only the DEM of Switzerland had to be acquired, processed and converted. In the second cycle, satellite images and cultural data of swiss locations had additionally to be integrated in the DBMS that required corresponding data acquisition, processing and conversion steps. The programming consisted in the implementation of data structures and routines for data format conversion, elements for the interaction within the VE and with graphical user interface panels, scenebase management, visibility behaviour of InfObjects, and networking functions. Every design and implementation cycle was also accompanied by usability testings asking users about their feedback when trying out new system features. Finally, extensive system performance tests have been done.

ErgoSim Project Within the ErgoSim project, a large amount of time was invested in the definition, evaluation, and acquisition of basic system resources and know-how needed for building a driving simulator. Again, iterative design and implementation of the simulation software including acquisition of textures for the traffic signs, the enhancement of the reused "Performer Toon" scenebase via the modeling of traffic signs and additional objects, the programming of motion and vehicle dynamics, networking and pod interfacing including corresponding testing has been applied. The contruction of the cab, the prototyping of cockpit and display panel alternatives, and the design of the usability tests was done in parallel to the software development phases. The development model is shown in Figure B-2 in Appendix B.

ViRTIS Project The development of a simple VR based TIS testbed environment can be characterized in three major steps: 1.) evaluation and definition of the testbed environment and underlying application scenario, 2.) design and implementation of the application prototype and design of the usability tests, and 3.) the carrying-out and evaluation of usability tests. Again, an already existing scenebase has been reused and enhanced with additional objects, features and effects to save modeling time. The programming phase consisted of the modification of the scene description file by altering and adding routines for user interaction as well as object behaviour and dynamics. Finally, configuration files had to be edited for system and device setup. Also, the spaceball had to be modified for more convenient usability: an enhancement of its pick button by a piece of cardboard was needed to offer a better haptic feedback, and the functions keys on the spaceball had to Design Spaces 79

be covered in order not to confuse the user and focus his attention to the ball itself. The effective development model and phases carried out are shown in Figure B-3 in Appendix B.

5.3 Metaphor Spaces

Dependent on the specific application environment and on the system components used, different user interface metaphors have been designed and implemented. They serve as a proposal for ViRTIS related user interface metaphors described later in Chapter 6. The application of adequate metric schemes for system and usability testing will give a feedback on the usability and quality of the metaphors used. In the following, the metaphors and corresponding user interface elements implemented in the three VR application prototypes will be described.

5.3.1 Desktop Virtual Reality Based Fly-Over Metaphor The ViRGIS prototype is a desktop VR based system according to the classification of [Jacob94]. A three-button 2D mouse is used as a navigation device for the interactive exploration of the 3D virtual scene by applying a fly-over navigation metaphor. The following 2D mouse navigation functions have been implemented: acceleration by continually pressing the left mouse button, de-acceleration by continually pressing the right mouse button, stand still by pressing the middle mouse button and change of view and direction by moving the mouse pointer in the desired direction. Beside a text based configuration file to setup the runtime simulation environment, all the relevant ViRGIS system parameters can be manipulated interactively at runtime. Six Graphical User Interface (GUI) based control panels (see Figure 5-15) offer the possibility to manipulate system parameters such as viewpoint setting or rendering mode modification at runtime. In addition to the exploratory fly-over metaphor achieved by 2D mouse interaction within the virtual scene, a teleporting metaphor is also implemented. By entering the military coordinates or pointing to the desired position in a textured overview map, the user may directly jump to a location within the virtual environment. The textured overview map helps the user to easily and quickly build a mental model of the overall virtual scene and gives sufficient visual feedback to overcome the "lost-in-cyberspace" syndrome. 80 Development and Evaluation of Virtual Reality Application Prototypes

Figure 5-15: Loading and positioning panels of ViRGIS prototype

The cultural data available of more than 3000 different Swiss locations are represented by textured cubes, called InfObjects. They are located on top of the DEM at the military coordinate of the corresponding location. Clicking on an InfObject by positioning the 2D mouse pointer over it pops-up a GUI based window that lists the data available of a village or city. This is shown in Figure 5-16.

Figure 5-16: Representation of cultural data as InfObjects Metaphor Spaces 81

5.3.2 Cab Virtual Reality Based Drive-Through Metaphor The construction of a physical mockup of a driving cabin or cockpit, often referenced as cab or pod, and subsequent integration into a VE allows the implementation of a highly realistic drive-through navigation metaphor. Such a setup is also called simulation VR [Jacob94]. The interior dimensions of the cab relate to a lightweight mobile vehicle that has been built and crash tested. Figure 5-17 shows the actual installation and setup of the ErgoSim driving simulator at the actual test site.

Figure 5-17: Setup of the driving simulator cab environment

The input interface to the user is done over a real steering wheel and two foot pedals, one used as gas the other as brake pedal. A high-resolution graphics monitor is used to display the actual driving speed and stati of other control elements such as light or radio switches. All these control stati are allocated closely to each other in order to minimize searching activities during the driving. Different cockpit and control alternatives, consisting of real handles and knobs, have been provided to the user during usability testing. 82 Development and Evaluation of Virtual Reality Application Prototypes

5.3.3 Desktop Virtual Reality Based Walk-Through and Direct 3D Manipulation Metaphor The third VR application prototype implements a walk-through navigation, a 3D pointing and direct manipulation metaphor. The user´s task is to walk-through a virtual house in order to evaluate its features and inventory. Figure 5-18 illustrates the initial viewposition after application start and the system configuration.

Figure 5-18: Setup of the ViRTIS environment showing a screenshot of the house entrace (a) and IO device configuration setup (b) Metaphor Spaces 83

A 3D cursor, located in the center of the user´s viepoint, serves as reference point for navigation and object manipulation tasks. As input devices, a 2D mouse and a spaceball are used. The 2D mouse implements the following functions: forward moving by continually pressing the left mouse button, backward moving by continually pressing the right mouse button, stand still by pressing no mouse button, picking an object by pressing the middle mouse button, and change of direction by moving the mouse pointer up, down, left or right. The spaceball provides simultaneous six Degrees-Of-Freedom (6DOF) manipulation control. A push, pull or twist on the device platform generates x, y, z translation and rotation events that may be used for manipulation of a virtual object or navigation within the VE. A pick button on the back of the device allows the implementation of picking operations. The use of active stereo LCD shutter glasses allows to see the virtual scene in stereo mode. By this, a virtual scene may be perceived in a more natural and user-adequate way.

5.4 System and Usability Testing Applying Different Metrics 5.4.1 Load Times for Client-Server Based Virtual Environments In the ViRGIS project, system testing focused on the performance behaviour of the overall system and individual computation segments. All tests (except where indicated) were run with the following parameter setup : • size of the rendered runtime scenebase: 5x5 patches; patch size: 10 km x 10 km • LOD 2 switching distance: 16 km; LOD 1 switching distance: 32 km • ViRXIS-Client: SiliconGraphics Onyx RealityEngine (RE) 2 (2 MIPS R4400 CPUs, 100MHz, 128 MB RAM, 1 Pipe, 4 MB texture memory) and Indigo2 High IMPACT (1 MIPS R10000 CPU, 195MHz, 128 MB RAM, 4 MB texture memory) • ViRXIS-Server: Sun SPARCstation10 (4 SPARC CPUs, 64 MB RAM) and Sun UltraSPARC1 (4 SPARC CPUs, 96 MB RAM)

To evaluate the load times for scenebase entities requested by a ViRXIS-InterActor client over a local or global area network from a ViRXIS-DBMS server, the following four machine configurations have been setup: 84 Development and Evaluation of Virtual Reality Application Prototypes

Onyx RE2 SPARC10 IMPACT UltraSPARC1 InterActor DBMS InterActor DBMS (Client) (Server) (Client) (Server)

Local Area Network (Ethernet / ATM) Local Area Network (Ethernet)

a) b)

Onyx RE2 SPARC10 Onyx RE2 InterActor DBMS InterActor (Client) (Server) (Client)

Local Area Network Local Area Network (Ethernet / ATM) (Ethernet / ATM)

Global Area Network Global Area Network (Internet) (Internet)

Local Area Network (Ethernet) Local Area Network (Ethernet)

IMPACT IMPACT UltraSPARC1 InterActor InterActor DBMS (Client) (Client) (Server)

c) d)

Figure 5-19: Four system configurations for testing load times; a) and b): client and server within a LAN; c) and d): one client and the server inside a LAN, one client requesting data from outside the LAN over the Internet.

Figure C-1 and Figure C-2 in Appendix C show the total load times per LOD3 and the processing times for the database querying (dbms time) and networking (networking time) segments. Dbms time is the time spent within the ViRXIS-DBMS for processing the data request coming from the ViRXIS-InterActor. Networking time is the time for transmitting the retrieved data from the ViRXIS-DBMS to the ViRXIS-InterActor and generating the runtime scenebase within the ViRXIS-InterActor. As can be noticed, the networking segment contributes more to the overal load time than the dbms segment. This is caused on one hand by the poor bandwidth of the LAN, on the other hand by the scenebase generation process within the ViRXIS-InterActor. Especially the retrieval System and Usability Testing Applying Different Metrics 85

and generation of geometry data shows poorer performance than for texture data. The reduction of the scenebase complexity (number of triangles) by adaptive triangulation [Kropp97] of the DEM (see Figure C-3) and the use of a powerful server platform for the ViRXIS-DBMS (see Figure C-4) improve overall system performance significantly. Figure C-5 shows how different networking models on the server side can influence system performance. The pool_server model permanently maintains the communication connection to the client whereas the gis_server does not. This results in an increased system performance. To extend the system performance testing, configurations that represent real-world conditions have been setup and run. Figure C-6 shows four setups that measure performance behaviour when two clients are requesting data from the server at the same time. Figure C-6a and C-6c reflect situations where two clients connect to one single server process. In the setup represented by Figure C-6b and C-6d, each of the two clients is served by a separate server process executed on the same server platform. As can be noticed, the dedicated server model (Figure C-6b and C-6d) is more favorable for InfObjects of little data size than for geometries or textures concerning load performance. In Figure C-7 a single client single server process setup is compared with the performance of a setup where four application clients running on the same client machine and invoked at once are served by one server or by four servers running again in parallel on the server platform. As can be seen, again InfObjects show a better performance when a client is served by a dedicated server. The variation of patch sizes and LODs (see Figure 5-20, 5-21, and 5-22) has shown that total load time of the initial runtime scenebase increases faster when choosing a patch size higher than 8x8 km.

LOD1 160000 LOD2 LOD3 140000 AdaptiveLOD3 120000 LOD123 100000

80000

time [ms] 60000 40000

20000 0 4x4 6x6 8x8 10x10 12x12 16x16 20x20 km km km km km km km

patch size

Figure 5-20: Total load times for DEM data at varying patch size and LOD (at low system load, system configuration a), pool_server model) 86 Development and Evaluation of Virtual Reality Application Prototypes

LOD1 LOD2 120000 LOD3 AdaptiveLOD3 100000 LOD123 80000

60000

time [ms] 40000

20000

0 4x4 6x6 8x8 10x10 12x12 16x16 20x20 km km km km km km km

patch size

Figure 5-21: Total load times for satellite images at varying patch size and LOD (at low system load, system configuration a), pool_server model)

LOD1 4500 LOD2 LOD3 4000 AdaptiveLOD3 3500 LOD123 3000 2500 2000

time [ms] 1500 1000 500 0 4x4 6x6 8x8 10x10 12x12 16x16 20x20 km km km km km km km

patch size

Figure 5-22: Total load times for InfObjects at varying patch size and LOD (at low system load, system configuration a), pool_server model) System and Usability Testing Applying Different Metrics 87

The time to load and to render a scene patch depends on the following factors: (1) the time to retrieve the requested patch from the DBMS, (2) the time to transmit the data from the ViRXIS-DBMS to the ViRXIS-InterActor over the network, and (3) the time to render and visualize the patch by Performer. Network transmission time makes up for the biggest portion of the total load time. This results mainly because of the collision occuring on the underlying Ethernet network. Furthermore, it can be seen that load time for texture data increases at a slower pace than the one for geometrical data with increasing patch size. This is mainly based on the fact, that on one hand range queries for geometrical data (DEM data) by the R-tree are more time consuming than hash queries for textures, and on the other hand the generation and processing of the visual database within Performer takes longer than texture processing.

5.4.2 Cockpit Design and Degree of Immersion in a Cab Virtual Reality Based Drive-Through System The test set consists of twelve people, 6 males and 6 females, with a distribution in age from 20 to 55 years, with driving experience between half to 27 years, and having a fairly little frequency of accidents on the average. The measurements used for the drivers´ cognitive, motor and mental loads are viewing frequency, viewing duration and reaction time. The optimization of these parameters should lead to an optimal driving cockpit design and setup for high active driving security. The following six driving situations served as testbed scenario that the 12 test persons had to pass: 1. inside-town crossing at several crossings, 2. following a car with a predefined path, 3. accidental shock situation by suddenly inserting a vehicle from the right crossing the driver´s path, 4. cross-country driving with enforced stop at a road building site with two-way traffic, 5. monotonous cross-country driving with task to remember objects laying at the side of the road, and 6. inside-town driving under fog condition. The cockpit configuration, where all the different knobs such as light or radio switch are mounted on a central control unit, and driving condition two shows the minimal frequency of viewing to display elements. This cockpit setup shows also the minimal frequency of viewing to control elements. A tactile orientation, visual and additional acoustical individually tuned feedback of driving information shows a positive effect on the driver´s attention to the actual traffic situation. A central control element helps in 88 Development and Evaluation of Virtual Reality Application Prototypes

the optimization of search and operational processes. Further, driving conditions that require to keep track of certain driving parameters such as speed limit (e.g. driving situation two or three) cause an increased frequency of viewing to display or control elements. 80% of the test persons report some degree of nausea after a while of using the driving simulator. This is mainly caused by the absence of vestibular stimulation to the user. A slight vibration of the driver´s seat by hand already helped to overcome this lack of perceptual input. Also, driving situations cause more direct perceptual cues than flying scenarios, because the perceived objects are virtually closer to the observer and therefore the angular velocity of passing-by objects is much higher. The strong visual stimulation without corresponding vestibular input to the human nervous system causes some disturbances or perceptive incompatibilities. A feeling of sickness may often be the result of this. An interesting phenomenon could be observed that supports the above statements: an intermediary stop of seat vibration while driving causes the perceptual effect of sliding or flying above the road to the user.

5.4.3 Evaluation of Different IO Device Setups To find out the optimal device configuration setup and correspondent subjective feelings of users the following hypotheses have been proposed: 1. Navigation by a spaceball is more intuitive and more efficient (faster concerning task performance) than by a conventional 2D mouse. 2. Navigation using stereo viewing gives a higher degree of immersion and is better than mono viewing. 3. User performance (time to complete a task) is dependent on previous computer experience.

4. There should be no significant differences concerning performance and subjective feelings between genders. The IO device setups have been tested by 24 individuals, 12 men and 12 women with ages between 21 and 32, with a heterogeneity in professional background. The number of test persons needed resulted out of the four configuration alternatives tested. Using two input devices and two viewing conditions gives 24 (4!) different permutations to set up the testbed environment. The sequence of configurations was permutated in the way that six equal setup combinations occured for every of the four tasks. The sequence of task 1-4 were held constant. Figure 5-23 represents the findings gained from the ViRTIS usability experiments. As can be seen, stereo viewing using LCD shutter glasses performs significantly worse than mono viewing. System and Usability Testing Applying Different Metrics 89

designer´s conceptual view user´s conceptual view

dimensions visual depth visual depth concept of the presentation perception of the world model effects (vde) mechamisms (vdm) world model

house-level 1.0 one image 1.0 monocular objective: ViRTIS 1.1 shadowing&lighting 1.1 perspective 1.2 color mono screen house 1.3 superposition 1.2 figure-ground = 7.448 min 1.4 relative size 1.3 grouping rooms 1.5 height in plane 1.4 patterning 1.5 Gestalt laws 1.6 gradient of texture stereo windows 1.7 linear perspective screen (LCD) 1.8 size perspective = 10.344 min stairs 2.0 binocular 2.0 two images 2.1 disparity furniture subjective: 2.1 stereopsis 2.2 accommodation 2.3 convergence household mono articles 3.0 multiple images = 0.667 3.1 holographics active light 3.2 motion perspective 3.0 movements switches 3.1 eyes stereo 4.0 head coupling 3.2 head = -0.583 etc. 4.1 motion parallax 3.3 body

Figure 5-23: Design and classification concept for the visual channel applied for ViRTIS prototype

The task completion time was significantly higher using the spaceball versus 2D mouse. Stereo viewing condition caused also a higher task completion time versus mono viewing. It is interesting to notice, that women perform worse than men and wearer of glasses or contact lenses perform better on the average. The better performance of the mouse can be interpreted the way that users have more experience in using it. Users feel more familiar with a 2D mouse than with a spaceball. The spaceball is felt too confusing mainly because of its six degrees-of-freedom. Persons had difficulties in moving straight forward or performing an exact movement in the VE. Female users had severe problems to apply the required forces to control the spaceball. The performance profiles of spaceball experienced users show that an increase of experience with a device results in a decrease in task completion time and in a convergence of spaceball performance to the one with the 2D mouse. Therefore, longer training time would certainly help to improve the performance and familiarity with a new interaction device such as the spaceball. Similar conclusions can be found in other studies such as [Lamp94] or [Zhai96]. 90 Development and Evaluation of Virtual Reality Application Prototypes

The subjective judgements and configuration preferences concerning the usability of the tested IO alternatives shown in Figure 5-24 give the following order or rank of preference for the device configurations: 1. 2D mouse, mono viewing 2. 2D mouse, stereo viewing 3. spaceball, mono viewing 4. spaceball, stereo viewing The role of input device seems to be much more important to users than the one of output condition. In these experiments, the combination "2D mouse with mono viewing" was prefered most, where as "spaceball with stereo viewing" was prefered least. The wearing of stereo glasses was felt disturbing.

60 males 50 females all 40

10 preferences [#votes]

0 2D 2D spaceball spaceball mouse & mouse & & mono & stereo mono stereo

device configuration

Figure 5-24: Subjective preferences for IO devices configurations

As major conclusion it can be stated, that none of the above listed four hypotheses could be positively confirmed. The selection of test persons turned out to build an ideal test set, because no significant differences between age, computer experience, or human abilities could be found. A more detailed description of the test results may be found in [Hone97]. System and Usability Testing Applying Different Metrics 91

5.5 Discussion 5.5.1 Scenebase Acquisition, Modeling, Tuning and Management The most resource consuming phase during development of a VR application system lies in the generation of the scenebase. Therefore, transformation and reuse of already existing data has to be a main focus for this phase. In all three VR projects, already existing data could be reused and helped reduce the amount of time needed for scenebase modeling. In the ViRGIS project data only had to be transformed to the final data format managed by the DBMS. Still, additional processing of scenedata was necessary. A rich set of conversion and data processing utilities eased this process. The consideration and support of data format standards such as VRML not only in the DBMS, but also in the data sources can significantly help to streamline the data transformation process and to reduce unwanted side-effects such as falsification or even loss of data attributes. The design and implementation of the interactive 3D simulation environment within the ErgoSim project highlighted the need for a sophisticated VE authoring system. It helps in the modeling and testing of traffic scenarios and of behaviour of autonomous vehicles. Also, the exact knowledge of the application domain and its underlying rules (e.g. correct positioning and use of street signs according to federal highway code) is relevant to build a realistic simulation environment. The careful evaluation of the final underlying scenebase to achieve a desired frame rate of at least 40 frames per second (fps) turned out to be of major importance. The reuse of an already existing scenebase and modeling of additional application specific objects saves a lot of project time. Because of the irregular scene complexity while navigating within the VE, the targeted frame rate has to be the sum of the desired frame rate plus a certain reserve to compensate the occuring changes in polygon complexity per frame. The frame rate during usability testing was 30 fps that is regarded as critical. This gives no room for the modeling and integration of additional objects into the ErgoSim runtime scenebase. The different techniques applied to improve patch loading performance in the ViRGIS project show that a powerful multi-processor server platform, maintaining a permanent connection to clients and dedicating separate processes for clients requesting data, is most favorable to maximize overall system performance. The reduction of scenebase size via adaptive triangulation of the DEM decreases the amount of data to be processed and transmitted over the network. As a consequence, higher detail for rendering and therefore more realistic sceneries at increased performance was achieved. The application of a high-bandwith network (e.g. ATM or Fast Ethernet) between client and server 92 Development and Evaluation of Virtual Reality Application Prototypes

as well as a further reduction of the amount of data to be processed additionally contributes to overall performance. Paging and caching of scene patches helps a lot to optimize resource allocation, interaction and overall system workload. Also, the outsourcing of data storage and management to a DBMS increases the flexibility of the overall system. It allows the incorporation of various data query functions for the optimization of the runtime scenebase. System parameters such as main or texture memory size or processing power determine the setup of the runtime scenebase. Therefore, the interactive variation of patch and map size is important for tuning the target system towards optimal resource allocation. The system tests have shown that patch size should be no higher than 8x8 km at a scenebase size of 5x5 patches considering a scenebase structure similar to the one used in the ViRGIS project. This totals 40 km2 of runtime scenebase size. The initial loading of lowest LOD (LOD1) under multiple LOD condition is highly suggested in order to offer a fast navigational feedback to the user. A subsequent load of adaptively triangulated highest LOD (adaptive LOD3) is regarded as an optimal compromise between load time and detail in data. Finally, the main performance bottle neck lies in the data transmission process between the server and the client. A further downsizing of transmitted data, mainly geometries, together with an extended storage hierarchy can contribute to this and helps reduce total load times. A local storage of static information such as DEM or texture data and loading only InfObjects that are of more dynamic nature from the DBMS over the network is a further strategy for runtime optimization.

5.5.2 System Usability and Acceptance The ViRGIS project highlighted the fact that texturing of a DEM and offering navigation cues such as overview map or display of actual viewing parameters helps a lot to overcome the lost in cyberspace syndrome. People perform significantly better when texturing and navigation feedback was enabled. They have much less trouble in orientation and can find locations or regions much faster than those who navigat only in the shaded 3D model. The absence of vestibular feedback (sense of motion) in the ErgoSim configuration shows a significant negative impact on the overall system acceptance by test persons who report some degree of nausea. Therefore, the integration of vibration elements or devices (e.g. under the driver´s seat) with low frequencies (1-3 Hz) would produce the required vestibular stimulations. It is also recommended to remind people after using a driving simulator not to start driving a car and to recover for some time instead. For Discussion 93

example, Swiss army pilots are not allowed to fly an airplane after being in a flight simulator on the same day. The results gained from the ViRTIS experiments show, beside the main preference for the 2D mouse and mono viewing condition, a significant difference between men and women when comparing control variables such as willingness for exertion, degree of relaxation, or emotional state: male individuals perform better than femal ones. This might be derived from the fact that males tend to be more positive to new technologies than females. At the end of a test cycle within the ViRTIS project, every test person had to fill out a final questionnaire and was also interviewed about his information needs and system features when using a VR based Tourist Information System. Some of the answers given, that may again be used for a further refinement of the design and application space of ViRTIS environments, are listed here: • A ViRTIS should offer the functionality to get a feeling for the spatial dimensions and sub-division of the destination and its touristic objects (e.g. rooms in a house). • A ViRTIS should give enough proprioceptive feedback (e.g. body scales in relation to virtual objects). • A ViRTIS should offer a view of the overall environment, landscape, infrastructure, and transportation facilities including adequate interaction mechanisms such as overview maps with viewpoint indication, information boxes related to objects, integration of still pictures and video sequences, or extended query possibilities. • A major assumption for a broad acceptance of a ViRTIS is that on one hand the information provided has to be correct, updated and consistent, on the other hand it has to be easy to learn and operate.

• The experimental application prototype was a lot of fun and fascinating to use.

The evaluation of the three VR application prototypes and other VR solutions has shown that various interaction modes are made available to the user simultaneously. The following interaction modes can be differentiated: navigation: walking, driving, flying, hyperspacing manipulation: transformation, attribute processing selection: picking or highlighting a virtual object activation: invokation of dynamic effects and behaviour changes 94 Development and Evaluation of Virtual Reality Application Prototypes

retrieval: invokation of data queries to a VE management system presentation: visualization of status information and query results The consideration of these interaction modes can again be used for the description and classification of VR systems when evaluating their supported interaction techniques.

5.5.3 System Development The VR application development model, proposed in Chapter 4, has shown to be applicable for all three VR projects. As can be seen when comparing the development phases within the individual projects, a preliminary evaluation phase before actual implementation kick-off has always been carried out. The less experience within an application domain is available, that is the more uncertain the final architecture of the targeted application environment is, the more intensive this preliminary system evaluation process has to be. It helps to get an early idea about the critical success factors of the final system and to achieve better cost transparency for the whole development process. Important figures such as scenebase complexity or number of LODs to apply can be derived for the modeling and implementation phases. In contrast to the application development models for IaC systems, human factors play a more important role within VR projects. Because of the multi-sensory human computer interface of immersive VR systems, a main focus has to be on the careful design and tuning of IO processing channels and overall system architecture during the whole development cycle. To simultaneously address the manifold design and implementation issues such as synchronization of multiple human perceptory channels or scenebase modeling, an interdisciplinary development team and intensive usability engineering as integral part of the development process are major requirements for every VR project. The participation in associations such as VR Special Interest Groups (VRSIGs) opens additional channels for information acquisition and know-how exchange in order to increase the competence pool needed for the engineering of VR applications [Szabo96]. All three project models uncovered the fact that the development of VR application environments is highly iterative and involves several design-implementation-tuning cycles to achieve optimal overall system performance.

5.5.4 System Configuration, Operation and Maintenance The mechanical elements such as the pod construction turned out to be the weakest parts in a cab based simulation environment. The use of robust mechanical elements, Discussion 95

keeping wiring of peripheral devices at a minimum and using standard components is regarded as one of the key factors to ease the maintenance of VR application systems. One of the key phases in the ErgoSim project beside scenebase modeling was the tuning and calibration of the final system. Frame rate, motion dynamics of own vehicle as well as steering behaviour had to be adjusted to fit the human perceptual system. The occuring deviation effects of hardware units (e.g. noise in potentiometers) had to be smoothed out by software filters. Also, the work load distribution over a two machine local area network by applying personal computers for the acquisition of driving data helped relieve the rendering machine of computing activities. A further network wide distribution of computation intensive functions such as driving and steering dynamics or traffic simulation to inexpesive PCs can improve system performance in order to achieve a higher frame rate or increase the scene complexity. The MPS architecture proved to be a powerful model for the design and implementation of peripheral device functionality. The simulation of device behaviour early in the development process where the corresponding device was not yet available helped to shorten the development cycle and to increase the system´s flexibility during testing and tuning for optimal performance.

5.5.5 Economical Aspects The main costs within a VR project originate from the following activities: scenebase acquisition, modeling and processing, acquisition of hardware and software components, programming, installation, operation and maintenance. Therefore, cost saving efforts have to address these issues in the first place. In all three VR projects, most scenebase data was available for reuse. Only little time had to be spent on data acquisition, modeling and processing. Normally, at least 50% of the overall VR project costs go for scenebase modeling. The reuse of already existing scenebase data helped reduce these costs by a factor of 8 to 10 (taking current market prices for modeling hours into account). Another significant cost saving aspects lies in the application of standardized hardware and software components. Today, for example the same degree of immersion can be achieved by applying PC based image generators (versus graphics super workstations) cutting the costs down by a factor of 10 to 20. The use of a high-level VR authoring environment eases the design and implementation process. It enables the fast and cost effective development of simple VR application prototypes. 96 Development and Evaluation of Virtual Reality Application Prototypes

The operation and maintenance of a VR application environment requires human resource power that is often underestimated. Because of the still very sophisticated nature of individual system components such as peripheral devices special know-how is needed to keep a VR system operational as well as to continually extend or adapt its scenebase and functionality. The use of standardized hardware and software components help to streamline these tasks and to minimize overall maintenance costs.

5.6 Summary

In this chapter, architecture models, user interface metaphors, system development and maintenance issues of Virtual Reality based information systems have been discussed. Three VR application prototypes have been designed, implemented and compared against each other using the evaluation methodology described in Chapter 4. The first VR application prototype implements a VR based geographic information system. The use of a database management system for scenebase management allows on one hand the implementation of flexible mechanisms to maximize runtime performance, on the other hand a network-wide, client-server based retrieval of scenebase data. The use of a powerful server platform and the dedication of a separate server process maintaining a permanent connection to a client has shown best performance behaviour on the server side. Several techniques such as management of multiple levels-of-detail, dynamic scene paging, parallel loading of scenebase data, and sophisticated loading strategies have been designed and implemented to improve system performance for real-time interaction. The elaborated, final system architecture can serve as a prototype core architecture for a variety of VR based information systems. A second VR application prototype has been designed and implemented to evaluate various interfacing aspects in a cab based immersive virtual environment. The use of a multi-function peripheral device offering control buttons for setting parameter values via tactile orientation distracted the user less from his primary task. Futhermore, the Multi-Port Service (MPS) architecture has been designed, implemented and applied to ease the setup, operation and maintenance of peripheral devices within a VR system environment. MPS also allows to parallelize and to speed up the VR application development process. By the help of the third application prototype, four IO device setups have been evaluated against each other. The combination of 2D mouse with mono viewing showed maximal user performance. Single image (mono) visual information presentation Summary 97

techniques were preferenced over two image (stereo) presentation. Also, interaction using a 2D mouse was preferenced over a spaceball. The provision of proprioceptive and device status related feedback is also necessary to ease interaction. Interviews with test persons have also shown that there exists a general user acceptance for Virtual Reality based tourist information systems.

Chapter 6

Virtual Reality Based Tourist Information Systems

This chapter gives an introduction to the field of tourism. Market players, services, and related information processing requirements will be discussed. The description of existing tourist information systems reflects the status of applied IaC technologies in tourism. Trends, open issues, and challenges for the future developments of IaC system for tourism will also be highlighted. The findings gained from the evaluation of three VR application prototypes that have been discussed in Chapter 5 will be used to propose architecture models and user interface metaphors for three Virtual Reality based Tourist Information System scenarios.

6.1 Task and Application Space 6.1.1 Tourism Sector

Following [Schm94], tourism is defined as the production, distribution and consumption of services by travellers who stay in places differing to their domiciles or workplaces for at least 24 hours. The term "tourism" originates from the french word "tour" which means "journey out and home again during which several or many places are visited" and where tourism is understood as organized touring [Horn74]. Starting from these brief definitions the following underlying features of the tourism sector can be identified: tourism involves many aspects of service management; it is highly process oriented due to different transaction processes taking place; it is based on intrinsic organizational concepts and structures; and the traveller or tourist is regarded as the primary focus of attention and business. 100 Virtual Reality Based Tourist Information Systems

In the field of tourism, mainly three market players or actors can be differentiated: suppliers, mediators, and consumers. Suppliers are concerned with the production of services and the provision of systems supporting those services. Mediators or brokers establish the interface between elementary service suppliers and customers and handle transactions like ticketing or booking [Ern94]. On the consumer side, there are traders or travel agencies and tourists. The classical institutional structure is shown in Figure 6- 1.

client / tourist

travel agent / agency

local / regional association

tour operator

elementary service supplier

Figure 6-1: Institutional interaction structure of the tourism industry

Elementary suppliers such as railway or airline carriers offer their services to the market. Tour operators, assembling trans-regional packages based on elementary services, tourism associations or boards, offering non-sale regional information, and travel agencies, selling or providing information about services and packages, all contribute to satisfying the needs of the final customer or tourist.

To extend this view, additional actors or stakeholders [Free84] such as government, public institutions, associations, or employees can be introduced which may influence the industry and their processes. Depending on the importance and value-adding function of actors to their customers, different network models for a particular tourism industry or branch can be distinguished [Heij96]. They describe the interaction processes between the individual actors in a graphical form in order to get a better conceptual view of them. Some configuration alternatives are illustrated in Figure 6-2. Task and Application Space 101

T T T

TAG TAG TAG

TOP TOP TOP TOP

ESP ESP ESP ESP ESP ESP ESP ESP ESP

a) b) c)

T T T

TAG

TOP

ESP ESP ESP ESP ESP ESP ESP ESP ESP

d) e) f)

Figure 6-2: Network models for a tourism industry (TOP = tour operator; ESP = elementary service provider; TAG = travel agent; T = tourist; TA = tourism association): a) single-tour packaged; b) multi-tour packaged; c) single-tour packaged adviced; d) single-tour packaged individually enhanced; e) individually adviced; f) independently.

Network model a) in Figure 6-2 shows a configuration setup where a tourist books his holiday arrangements exclusively through a travel agency. Models b) and c) are variations of model a) where multiple tours have to be combined by the travel agency for an overall holiday package (b) or additional advice is requested by the tourist from an independent tourist organization (c). Booking over a travel agency and simultaneously doing some arrangements individually is represented with model d). Network model e) and f) show situations where the tourist arranges his tour with advice (e) or independently (f). 102 Virtual Reality Based Tourist Information Systems

.c.6.1.2 Tourism Related Information and Services The most important good or service in the field of tourism is tour related information, regarded as an assembly of processed service data originating from different sources. Information is used to build travel packages, to give travelling recommendations, or simply to serve as a basis for the decision making process by the tourist. Three classification criteria, as a function of the information´s nature or type, can be differentiated: the category, the behaviour, and the decision phase [Mai95] [Weis95]. These three dimensions span a basic classification cube for tourism related information and services that is shown in Figure 6-3.

category

reservation

entertainment

travel

destination decision pre post phase static

dynamic

behaviour

Figure 6-3: Classification cube for tourism related information

The categorical description of tourism related information gives anwers about ´what´ the travel is all about and ´where´ the information belongs within the whole travel. The decision phase view relates to the time or ´when´ the information is needed. The beha- vioural or frequential description defines the update rate of information, that is ´how often´ or ´how frequently´ a particular item of tourism related information is updated. The acquisition, processing, and provision of information are the basic functional building blocks for all kinds of services in the tourism sector. Tourism related information Task and Application Space 103 and services are also of a spatial and time sensitive nature. Unfortunately, a service and the provided or available information about that service often remain in an abstract and informal representation that does not perceptually mirror the real nature of the information itself when presented to the tourist. This can cause an extremely difficult and expensive decision making process for the user of tourism information. For example, the transformation of abstract, symbolical information on a map (presented as icons) into the corresponding real-world form often burdens the human mental processing abilities.

6.1.3 Tourist Information Systems Until recently, tourism information was mainly mediated through color brochures, catalogs, maps, books, video tapes, or TV broadcasting. Driven by the continuous automation and computerization of industrial and commercial sectors, Information Technology (IT) has also had a big influence on the acquisition, processing, and distribution of tourism related information. Different IaC technology based systems have been implemented and are running today. Generally, three different classes of Tourist Information Systems (TISs) can be identified: computer reservation, hypermedia, and web based systems. Computer Reservation Systems (CRSs) have been operational for several years now. In their beginning, elementary service suppliers such as airline carriers, large hotel chains, or car rental companies employed host-oriented systems with text-based user interfaces. CRSs provide information on different services covering aspects like availability, timetables or schedules, rates, or commisions [Clyd95]. They allow one to make reser- vations, issue and print tickets, provide booking confirmations, and help reduce transaction costs and time. The access to such mainframes is done over dial-up or dedicated leased-line connections using dumb terminals or PCs. The rise in productivity enabled by CRSs can be used to improve information and customer service quality [Ern94]. Besides CRSs there are so called Global Distribution Systems (GDSs) that additionally provide accomodation and entertainment programs. The main disadvantages of earlier text based CRSs were that they were difficult to use, required many hours of training, focused their content on the system supplier´s service portfolio, built a closed user group, and did not allow the incorporation of different kinds of media. The successively increasing computing power and storage capacity of low-cost computer systems made it possible to realize applications that integrate different types of media such as text, images, graphics, voice, music, computer animation, or video. Multimedia applications could therefore be considered as a real alternative to classical means of sales support (e.g. color brochures, catalogs, videos, etc.) in tourism. Their strengths lie mainly in their interactivity and dynamics of information presentation. 104 Virtual Reality Based Tourist Information Systems

The hyperlinking (cross linking or referencing of entities) of multimedia information creates hypermedia systems where CD-ROMs are applied as a common, broadly accepted storage media. The development of advanced hypermedia applications needs adequate environments for experimenting with different media types, representations or formats, and for working out new multimedia concepts. Important functional building blocks in such development are data acquisition, data processing, data format conversion, and authoring of the target application with an authoring system. The manifold media types and data to be processed require a functionally flexible hard- and software environment. Many development environments combine multiple hard- and software tools to accomplish these functional requirements for the development of multimedia applications. Essential for success are a systematic proceessing, keeping the target system’s requirements firmly in view during the development process, and a well constituted development infrastructure. A detailed description of the single development phases and involved issues as well as application scenarios can be found in [Szabo94a]. The introduction of the World Wide Web in 1992 together with the HyperText Markup Language (HTML) and freely available HTML browsers caused the spread of Internet technology from academia and the government to the public and the business community. The integration of hypermedia and standardized transmission and document description protocols in web technology enabled the implementation of more user-friendly interfaces and application environments that require minimal investment in hardware, software and training. This has also brought the actors in the tourism industry to apply web technology to their business activities. A growing number of travel information sources have began to appear. Internet and Web technology compared to other media is extremly cost effective and may generate unforseen leverage effects in revenues. The ability to provide networked hypermedia based services over the Internet to clients all over the globe offers the potential for opening new markets and distribution channels. Travel agencies and tour operators may provide their services directly to customers and other interested parties. CRS suppliers can begin to use the web as a front-end to their already existing infrastructures. Tourist associations act as mediators in promoting services and suppliers within a region or country.

6.1.4 Major Issues and Challenges The evolution of information and communication technologies (IaCT) and electronic markets offer new ways and opportunities for doing business in the field of tourism. IaCT hereby acts as a supporting tool to enable efficient and effective interaction and transactions between elementary service suppliers, agencies, and end-consumers. The Task and Application Space 105 optimal application of IaCT to tourism makes it necessary to consider the trends in the field itself. According to [Dant94] [Ebne94] [Ribb94] [Sche94] [Teu95], the trends in tourism can be seen as follows: • increasing level of individualization and autonomy of tourist, • tourist more selective and critical to offerings, • increased experience and education of traveller, • increased time and money to spend, • customization of services, • shorter and combined trips, • hybrid profile of tourist and more uncertainty in the market, • direct distribution channels between supplier and customer, and • globalization of markets. The outlined trends show that future success in tourism markets will depend strongly on the ability to focus on process and distribution oriented tasks involving the relevant parameters of services and stakeholders. Within distribution, marketing will be one key factor in overcoming an increased demand for quicker, more flexible and more complete response to customer requests [Sche94]. The design and implementation of real- time access to markets as well as to global distribution and communication channels requires the application of highly sophisticated IaCT infrastructures, along with market optimized business processes that maintain a high degree of quality and availability of tourism information. Further critical success factors are the integration of widely accepted standards for data exchange, the customization of products and services to individual preferences and needs through customer profile tracking, the provision of shopping-cart techniques, and a user-friendly and proactive presentation of up-to-date tourism information [Bloc96]. Many analysts see a lessening of the domination of classical media like brochures or videos in the future in favor of IaCT based media. The globalization of markets and existing IaCT infrastructures induces an increase in the differentiation of regional and service advantages [Ebne94]. The huge amount of data about products and secondary goods available, coupled with the user-unfriendliness of current IaCT systems challenge the tourist in his decision making process [Rui94]. Also, the lack of the integration of personal advices by travel agents or others into IaC systems is still in a prototype like status. The interaction with highly interdependent and time sensitive tourism information requires new approaches and mechanisms within IaCT infrastructures. Virtual Reality technology offers lots of potentials to close these gaps. 106 Virtual Reality Based Tourist Information Systems

6.1.5 Virtual Reality Based Tourist Information Systems A new and mostly unexplored field for Virtual Reality enhanced applications is tourism. The main advantage in the application of Virtual Reality based Tourist Information System (ViRTIS) environments is in the marketing and promotion of tourism destinations and services [Bau95] [Benj95]. Especially, the spatial and exploratory nature of both, VR and tourism, is an ideal factor to bring these two areas together. By the use of brochures, magazines, films, or videos people are already applying their power of immagination to experience new holiday destinations. They teleport themself mentally into a virtual environment. But, these means for planning future excursions and holidays lack the ability and functionality of an interactive, dynamic, and multi- sensory medium. Also, no further or cross-referenced information may be requested without consulting a travel agent or doing additional time-consuming investigations. In most cases, the tourist himself has to assemble all the information needed from many information sources for a tour. The use of a ViRTIS may significantly help to overcome the above listed disadvantages. In consideration of the issues and challenges in the field of tourism discussed above and the acceptance criteria described in Section 5.5.2, the following six requirements for a ViRTIS applicable to the marketing and promotion of touristic regions are defined: 1. interactive exploration of "real" appearing tourist regions, 2. automatic as well as user-initiated retrieval of tourism related information, 3. manipulation of entities to enrich the explorative experience, 4. availability of navigation and orientation cues to ease human computer interaction, 5. individual access via global and public networking infrastructure, and 6. easy maintenance and extendability of the scenebase and the system setup.

A ViRTIS exclusively for marketing and promotion purposes, identified as ViRTISm, can be characterized by using the classification cube for tourism related information presented before. A ViRTISm offers the functionality to interactively explore tourism destinations by retrieving static or dynamically changing travel, destination and entertainment information during the pre-decision phase for a better planning of future tours and holidays. This is illustrated in Figure 6-4. Adding reservation system functionality, mostly relevant in the post-decision phase, and additional not necessarly tourism related complementary information to ViRTISm leeds to a ViRTISm&s model for marketing and sales. Task and Application Space 107

category ViRTISm&s

reservation

entertainment

travel

destination decision phase

static

ViRTISm dynamic

pre post behaviour

Figure 6-4: Classification of ViRTIS for only marketing (ViRTISm) or marketing and sales (ViRTISm&s)

This thesis focuses on ViRTISm models. Three different application scenarios will be described that take the findings from the ViRGIS, ErgoSim and ViRTIS projects into account. The following sections highlight how the findings, gained from the design, implementation and evaluation of these three VR system prototypes, contribute to the proposal of adequate ViRTISm models and user interface metaphors.

6.2 System Models and User Interface Metaphors

As a conclusion from the VR projects described in Chapter 5, the following three application scenarios for ViRTISm are proposed: • Desktop Virtual Reality Based Tourist Information System (Desktop ViRTIS) • Desksize Virtual Reality Based Tourist Information System (Desksize ViRTIS) • Roomsize Virtual Reality Based Tourist Information System (Roomsize ViRTIS) 108 Virtual Reality Based Tourist Information Systems

The core architecture for all three ViRTIS scenarios follows from the integration of the ViRXIS and the general ViRIS architecture. A client-server based system model, where the single system building blocks are loosly-coupled by a interfacing framework (see Figure 3-3), offers the flexibility to ease the design and implementation of different application scenarios.

6.2.1 Desktop Virtual Reality Based Tourist Information System

In order to support a wide range of potential users of a ViRTISm and minimize the effort for system operation and maintenance, a desktop VR solution is proposed. It has also the advantage to be applicable for all the six tourism network models presented in Figure 4-2. Tourist offices, travel agents, tourist centers, or tourist associations can use a client-server architecture based ViRTISm as additional marketing tool to improve the quality of their services. Furthermore, tourists can log in to the ViRTISm-DBMS server over a global area network (e.g. the Internet) and request tourism related information or services. The integration of CD-ROM technology allows to use the application offline. Using standard hardware and software components on the client side enables the support of a wide user base. The design and implementation of sophisticated data management and retrieval mechanisms on the server side enables fast delivery of data to clients.

For the desktop ViRTISm client, the following configuration for the various ViRIS blocks presented earlier in Figure 3-3 is suggested:

Synthetic and Natural Image Processing and Generation (SNIPG), Synthetic and Natural Sound Processing and Generation (SNSPG) A portable software framework has to be choosen to support many computing platforms. It has to offer a rich set of functions for audio-visual interaction and for easy integration of manifold IO devices and subsystems.

Scenebase Modeling and Management (SBMM), Workload Balancing and Distribution (WBD) The use of an interactive 3D modeling and authoring system is highly recommended. New data attributes not yet supported by the actual scenebase can be represent as additional audio-visual entities and specific areas or objects can be highlighted to enhance the scenebase. Efficient data access structures for fast scenebase processing have to be incorporated to enable interaction with minimal phase lag. The parallelization of scenebase loading functions contribute to the systems scalability. The distribution of computationally intensive processing functions such as collision detection or motion dynamics calculation to System Models and User Interface Metaphors 109

other computing platforms with low workload helps further increase the client´s performance. Local as well as remote loading of scenebase data is key to guarantee high flexibility and openness in usability and data allocation.

Computer System (CS), Operating System (OS) & Storage Media (SM) To support a large user base, the system has to run on a widely used low-cost PC based platform. A multi-tasking capable operating system is also prefered to enable parallel loading of scene or peripheral device data. A multi-level storage hierarchy applying multiple caching options and supporting CD-ROM based storage of static scenebase data such as DEMs or textures should also be applied to increase the interactional flexibility and performance of the system. The use of CD-ROM technology to store the scenebase in a cost-effective way enables off-line interaction and makes the application environment more flexible in use.

Peripheral IO Devices and Subsystems (PIODS) As a conclusion of the three evaluated VR application prototypes, the use of standard desktop interaction devices enables the user to get quickly familiar with the system and helps generating a broad acceptance among users. There- fore, an alphanumerical keyboard for data input and modification, a 2D mouse for picking, selection and navigation operations, and optionally a robust joystick for more user-friendly navigation is proposed. As ouput devices, a high- resolution color graphics monitor for visual and two loudspeakers for auditory information presentation is also recommended. Further, a 3D graphics board with sufficient polygon rendering performance, on-board texture mapping capability and sufficient texture memory size as well as a soundboard with support for stereo and spatialized sound helps to unburden the CPU from these computing tasks and to apply for interfacing or calculation of dynamic effects instead.

Communication/Networking System (C/NS) & Local and Global Networking (LGN) Hardware and software for local and global area networking have to base on standard components and protocols in order to ease client-server interfacing between different system platforms and minimize maintenance effort. It is suggested to maintain a permanent communication link to the server, after connection has been established, and to apply special timeout mechanisms, if communication intensity between client and server falls below a certain threshold. Because only a small area of the scenebase is loaded at runtime on 110 Virtual Reality Based Tourist Information Systems

the client platform, load requests for additional patches are more likely during an interaction session.

Interfacing Framework (IF) An interfacing framework based on a portable programming language (e.g. Java™ from Sun Microsystems Inc.) and on flexible interfacing concepts (e.g. Multi-Port Service architecture) enables a continuous update of the functionality on the client side by new methods without recompiling and downloading the whole client software. Existing software components can be replaced or extended in order to optimize system performance, to support new IO devices, and to access new databases when they become available.

For the desktop ViRTIS server, the following configuration is suggested:

DataBase Management System (DBMS) The application of a standard, commercially available DBMS with extended functionality for clustering of data entities (see ViRGIS project description) enhanced by sophsticated data access structures enables the implementation of various loading and scenebase allocation strategies on the client side. Further- more, a rich set of data management functions provided by the DBMS allows to launch queries on the scenebase and additional databases that go far beyond file system mechanisms. The DBMS can also be used to evaluate system and user logs to optimize performance and adapt system functionality to user needs.

A continuous extension of the various databases stored and served by the DBMS requires a rich set of utilities and interfacing functions. Data format converters, database schema editors and gateways to other DBMSs have to be provided and permanently maintained. This enables the reuse and incorporation of already existing data into the DBMS in order to provide a server platform with a high degree of openness and to minimize scenebase modeling effort.

C/NS & LGN In addition to the C/NS & LGN aspects on the client side, the server has to maintain a permanent connection to a global area network (e.g. Internet) in System Models and User Interface Metaphors 111

order to become accessible for a large client base. Therefore, a leased line of high-bandwidth is highly recommended.

CS, OS & SM The DBMS server must run on an operating system supporting multi-tasking and multi-user access. This is neccessary to process data requests in parallel and serve multiple users simultaneously. A high-performance and multi- processor server platform helps to reduce the time spent for processing the large volume of data on the server side (dbms segment time).

The integration of several user interface metaphors within a VR application environment, as implemented by the ViRGIS prototype, has shown to be advantageous for interaction. On one hand users like to have at their disposal various ways to interact with a VE. Therefore, the following metaphors are proposed for the client platform of a desktop ViRTISm :

navigation metaphors The observation of the user´s behaviour within a VE showed in all three evaluated VR prototypes that there are two main strategies to interact with the system: 1) exploring or browsing to get familiar with the VE and information set provided or queriable; 2) direct VE area or information access. The first strategy can be supported by the implementation of a fly-over metaphor for coarse or overview scanning of the VE, of a drive-through metaphor for groundlevel exploration of VE areas such as cities or villages, and of a walk- through metaphor for more detailed investigation of virtual objects such as buildings. All three metaphors require a careful setting of parameters such as navigation speed, level-of-detail switching, and degrees-of-freedom. The second strategy can be supported by the implementation of a hyperspacing metaphor. It allows the user to teleport himself to the desired region by pointing to a location on an overview map. Further, direct input of object attributes such as coordinates or name of location via GUI panels or speech commands enable the access of scenebase areas or information entities in a more precise way. It also helps to surmount long navigation distances in order to get faster to the desired area of the scenebase. 112 Virtual Reality Based Tourist Information Systems

manipulation, selection and activation metaphors 3D peripheral devices such as a spaceball require previous training time before actual use. They allow to perform combined tasks (e.g. translate-rotate operations) faster, but cause difficulties for fine operations as shown in [Nuñez95] and by the house-level ViRTIS experiments. 3D devices should only be applied for coarse manipulation of 3D objects in virtual space. For fine precise transformations of objects, adequate mechanisms such as surface snapping or gravity have to be offered. Furthermore, the user should have the opportunity to decrease the degrees of freedom supported by a device. For pointing tasks such as picking, selection and activation operations on virtual objects (e.g. InfObjects) 2D mouse is considered to be most adequate. Using 2D and 3D devices in parallel for two-handed interaction can increase usability as pointed out in [Hinck94] and [Kaba94].

retrieval and presentation metaphors A permanent, user-configurable presentation of navigation values (e.g. viewpoint location, current navigation speed) and object attributes in GUI panels helps to optimize interaction activities. This way, less interaction steps are necessary to query and visualize attribute values. Audio-visual cues for navigation (e.g. 3D cursor to indicate moving direction) and manipulation (e.g. highlighting objects when they are touched or grabbed by the user) offer the user proprioceptive feedback for a smooth interaction with a virtual environment. The provision of data filtering options together with audio-visual differentiation of presented InfObject attribute values prevent the user from information overloading and help him to better identify objects of interest.

Because of the still unsatisfactory quality of stereo capable devices such as head- mounted displays or stereo glasses, the higher maintenance effort needed and the results of these usability experiments, the provision of stereo viewing for all three ViRTISm application scenarios is not recommended.

6.2.2 Desksize Virtual Reality Based Tourist Information System To extend the visual field-of-view and offer a more immersive VR application system for the user, a desksize ViRTISm environment is proposed. It is the extended version of the desktop ViRTIS. Many similar systems have already been designed and implemented. Some examples are ImmersaDesk by the Electronic Visualization Laboratory [Wlok96], Virtual Work Space [Ishi93], DigitalDesk [Well94], or System Models and User Interface Metaphors 113

Responsive Workbench [Krue95]. Figure 6-5 illustrates the proposed configuration layout for a desksize ViRTISm environment.

projection surface

2D mouse

retractable keyboard multi-function joystick

Figure 6-5: Configuration layout of a desksize ViRTISm

The outlined system architecture model of a desktop ViRTISm is also applicable for a desksize ViRTISm. The following components need a further refinement and extension:

Peripheral IO Devices and Subsystems (PIODS) As additional input device to the keyboard for alpha-numerical data input and the 2D mouse for picking and activation tasks, a multi-function joystick is proposed. On one hand it serves as a navigation device, on the other hand as a control unit to set parameter stati by corresponding buttons mounted on its surface. The results gained from the ErgoSim project have shown that such a central control element can help the user to better focus and concentrate on his task and application domain. A high-resolution color LCD beamer rear- projects rendered images via a mirror onto the desk surface. It is recommended to choose a light intensive beamer with fast decay of phosphor to avoid unwanted ghosting effects. The desk surface has to be shiftable by an amount of 110 degrees in order to set different viewing conditions for customer demos (110 degree position) or discussion of holiday routes on the desk surface (0 degree position). 114 Virtual Reality Based Tourist Information Systems

Interfacing Framework (IF) The Interfacing Framework has to be extended and adapted to offer support for the above listed devices. An additional MPS-device module has to be implemented and integrated in the framework to support the multi-function joystick. Rendering output has to be redirected to the video beamer.

The user interface metaphors proposed for the desktop ViRTISm are applicable for desksize ViRTISm, too. Two-handed interaction via the 2D mouse and the multi- function joystick is also suggested. Because of the more sophisticated and more expensive configuration setup of a desksize ViRTISm compared to a desktop environment, it is more applicable for use in large tourist offices, centers or trade shows. Also, the new multi-function joystick requires minor training time.

6.2.3 Roomsize Virtual Reality Based Tourist Information System A further increase of the degree of immersion can be achieved by applying (walk-in) spatially immersive displays (SIDs). These VR system environments physically surround the user with a panoramic virtual scenery without encumbering him with body-mounted devices (e.g. HMD) [Lantz96]. Some examples that are available today are the CAVE [Cruz93], Virtual Portal [Deer93], or the VET [Done94]. Figure 6-6 shows the virtual mockup of the proposed configuration setup of a roomsize ViRTISm environment.

Figure 6-6: Outline of a ViRTIS Roomsize Model installed in a room within a building System Models and User Interface Metaphors 115

The outlined core system architecture of the desktop ViRTISm is again applicable for a roomsize ViRTISm. The following components need a further modification and extension:

Peripheral IO Devices and Subsystems (PIODS) As can be seen in Figure 6-6, a touch screen based console serves as a main control unit. The advantage over peripheral devices such as a 2D mouse, a keyboard, or the above proposed multi-function joystick is that it provides a more robust system environment without mechanical or wired interaction components. This is important if a child proof application environment has to be operated. The overall effort for system maintenance can be minimized this way. Three light-intensive video beamers are used to rear-project the virtual scenery to three projection walls.

Computer System (CS), Operating System (OS) & Storage Media (SM) The system consists of two PC based computer platforms. The first PC acts as the master platform and is configured like the desktop ViRTISm platform. Additionally, it contains three powerful image rendering subsystems. Each of them generates output to one of the video beamers. The second PC acts as a device server. It updates GUI based control panels on the touch screen, handles data coming from the console and transmits them to the master platform.

Communication/Networking System (C/NS) & Local and Global Networking (LGN) Both computer platforms are configured for local and global networking. The master platform acts as a client to the DBMS server and as a client to the device server. The C/NS has to be extended by corresponding routines to support the local, dedicated communication between master platform and device server.

Interfacing Framework (IF) The Interfacing Framework has to be extended and adapted to offer support for the above listed devices. An MPS-device module has to be implemented and executed on the device server to support input from the touch screen. On the master platform, rendering output has to be redirected to the three video beamers and data transmitted from the device server have to be processed and directed to various system components. 116 Virtual Reality Based Tourist Information Systems

The user interface metaphors have to be modified the way that navigation, manipualtion, selection, and activation tasks are now handled via touch screen panels and no more via peripheral devices.

A roomsize ViRTISm can be used as a special attraction in exhibitions, large tourist centers or national museums. It requires significantly more resources for maintenance.

6.3 Summary

In this chapter, the findings gained from the design, implementation and evaluation of three tourism related Virtual Reality application prototypes have been applied to propose three application scenarios for Virtual Reality based Tourist Information Systems (ViRTISs). In the first section, the task and application space has been specified. The field of tourism as a promising application domain for Virtual Reality based Information System technology was discussed. The structure of tourism industry, market players, and related information needs and services have been outlined. A new classification model for tourism related information and services has been proposed. This model supports the specification process for computer based tourism applications. The discussion of computer reservation, hypermedia, and Web based systems points out current status and todays of Tourist Information Systems (TISs). Two main application segments for ViRTISs have been identified: 1) marketing and promotion, 2) marketing and sales. The application of a ViRTIS in the first segment is identified as ViRTISm, in the second segment as ViRTISm&s. The evaluation focuses on ViRTISm scenarios.

Three application scenarios of ViRTISsm have been proposed: 1) desktop, 2) desksize and 3) roomsize ViRTISm. The individual components of the underlying system architecture using the ViRIS architecture model (see Chapter 3) have been discussed. Furthermore, adequate user interface metaphors and potential user groups have been proposed. Chapter 7

Conclusion

7.1 Summary

Virtual Reality is a technology that enables the design and implementation of new paradigms for human computer interaction applicable in various application domains. In this thesis, new concepts for the development and interaction with Virtual Reality based Information Systems (ViRIS) have been introduced. The evaluation of various Virtual Reality application prototypes gave significant feedback for the final proposal of adequate architecture models, user interface metaphors, and metrics of Virtual Reality based Tourist Information Systems (ViRTIS). The discussion of VR design principles, user interface paradigms and involved system components helped to get a better understanding of the technology itself. Critical success factors for the development and operation of VR system environments have also been highlighted. These include real-time interaction and synchronization of perceptual channels, intuitive and user-friendly human computer interaction, effective scenebase modeling and management, eased system development, setup and maintenance, and economical aspects. The elaborated system model helps to better classify and compare VR systems against each other. The coupling of VR based systems with conventional database subsystems leads to a new architecture model for VR based information systems. This offers the following advantages: on one hand information systems can be operated under a new, more user- friendly environment, on the other hand VR systems can benefit from database functionality to optimize their data management mechanisms. With an expected increase of ViRIS type applications in the future, the proposed models in Chapter 3 can be used for the description, comparison and classification of ViRIS environments. This opens the potential for the design and implementation of very promising new application 118 Summary

scenarios such as VR based geographic, medical, or facility information systems at large. A new methodology has been designed for the evaluation of VR based applications. The methodology incorporates several aspects relevant for VR application development: the task and application domain, the system architecture, perceptual information processing channels addresed by the VR system, applied user interface metaphors, and the system development process. To obtain quantitative and qualitative measures, system and usability testing applying specific metrics as a function of the application domain and system architecture has to be performed. The methodology is used to evaluate three VR application protoypes: 1) a Virtual Reality based Geographic Information System (ViRGIS), 2) a cockpit and projection screen based immersive driving simulator, and 3) a VR based holiday house exploration environment as a very first prototype for a VR based Tourist Information System (ViRTIS). Different issues relevant for the design and implementation of Virtual Realiy based Information Systems have been discussed. Paging and caching of scene patches helps a lot to optimize resource allocation, interaction and overall system workload. Also, the application of a DataBase Management System (DBMS) for data storage and management increases the flexibility of the overall application environment. It allows the incorporation of various data query functions for runtime optimization of the scenebase. A powerful multi-processor server platform, maintaining a permanent connection to their clients and dedicating separate processes for clients requesting data, is most favorable to maximize overall system performance. The reduction of scenebase size via adaptive triangulation of the digital elevation model decreases the amount of data to be processed and transmitted over the network. Transformation and reuse of already existing data helps to minimize modeling effort and to save time during development. The consideration and support for data format standards such as VRML by the DBMS can significantly help to streamline the data transformation process and to reduce unwanted side-effects such as falsification or even loss of data attributes during conversion. Texturing, provision of various overview maps, proprioceptive feedback for user´s position in virtual space, vestibular stimulation in immersive drive-through environments, and mono viewing turned out to be optimal to increase user acceptance and usability of the evaluated VR application prototypes. A new software architecture, the Multi-Port Service (MPS) architecture, was designed and implemented to improve the integration of additional peripheral devices into an Conclusion 119

existing VR system environment and to enable the development of software components in parallel. Cost saving potentials are seen in the application of PC based system components linked by high-bandwidth networks and in the reuse of existing scenebases in order to reduce time needed for modeling. The use of standardized hardware and software components help to streamline maintenance effort and minimize overall costs for system operation. The findings gained from the design, implementation and evaluation of the three VR application prototypes have been used to propose three promising application scenarios for Virtual Reality based Tourist Information Systems (ViRTISs) applicable in tourism marketing and promotion: 1) a desktop, 2) a desksize and 3) a roomsize ViRTIS.

7.2 Outlook

A further refinement and tuning of the client-server oriented scenebase loading strategies as implemented in the first VR application prototype is needed to improve overall load time for scene patches requested from the DBMS. A dynamic modification of the loading strategy as a function of network and system load at runtime helps to optimize the allocation of scarce system resources such as processing power, memory capacity, or network bandwith among clients and the server. Methods to analyse and adapt scene complexity such as presented in [Hit93] can be applied to optimize client- server based scenebase management. A reduction of the amount of data that has to be transmitted over the network from the DBMS server to the client can be achieved by transmitting only vertices instead of polygons and performing triangulation of the client side.

The functional extension of the DBMS server for VRML support would allow to make the system environment more open because of a standardized data format for the scenebase. Not only data, but also methods implemented as VRMLScripts or Java applets may be distributed to clients in order to update them with new or more efficient interaction and data management routines without recompiling the client code. The incorporation of functions for access logging as well as for the generation and evaluation of access statistics on the server side would allow to build up user profiles. These may be used to implement data loading strategies for clients as a function of previously visited scenebase regions. The profiles may also be used for market research and extension of the scenebase as a function of mostly requested or visited data by clients. 120 Summary

Finally, the implementation and evaluation of the three application scenarios proposed in Chapter 6 by system performance and usability testing would give additional quantitative and qualitative measures for ViRTIS environments. Appendix A

ViRGIS Scenebase

cultural data size (in MB): 1 #entities: 3018

Table A-1: Resolution and size of cultural data stored in the ViRGIS-DBMS

geometries topo_LOD_1 topo_LOD_2 topo_LOD_3 (lowest) (highest) resolution (in meters): 1000 500 250

#polygons (per 1 km2): 2 8 32

#polygons/patch: 512 2048 8192 (per 16 km2) size (in MB): 16 68 272

Table A-2: Resolution and size of geometries stored on the ViRGIS-DBMS

textures tex_LOD_1 tex_LOD_2 tex_LOD_3 resolution (in meters): 120 60 30 size (in MB): 24 86 31

Table A-3: Resolution and size of textures stored on the ViRGIS-DBMS (textures for LOD3 not available for all over Switzerland)

Appendix B

VR Project Development Phase Models

Requirements Definition Task and Application Space

Preliminary System Definition, System & Copyrights Evaluation Target System Benchmarks

Acceptance Implementation Kick-Off

User Interface & System Design

System Implementation Protoype 1, 2

Data Acquisition - DEM - InfObjects Programming - Textures Scenebase Generation & - Data Format Conversion Processing - Interaction - Data Management - DE M - Behaviour - Textures - Networking - InfObjects - User-Interface

Simple Usability Testing

System Integration, Testing & Tuning Final System

Evaluation of System Behaviour, Performance and Features Proposals for Design, Metric, Metaphor and Application Space

Figure B-1: Development phases within the ViRGIS project 124 Appendix B

Task and Requirements and Test Scenario Definition Application Space

Preliminary System Definition & System Evaluation Target System Benchmarks

Acceptance & Acquisition of System Units Implementation Kick-Off

User Interface, System, Cab and Usability Test Design

System & Cab Implementation

Data Acquisition Cab Protoype 1, 2 - Textures Construction Scenebase Modeling and Processing & Cockpit - Scene Layout - Vehicle Path Prototyping - Object Geometry - Trigger Points - Texture Maps

Programming - Interaction - Networking - Dynamics - Device Simulator - Behaviour

System Integration, Testing & Tuning Final System

Final System System Installation, Testing & Tuning at Test Site

Usability Testing Usability Test Results

Questionnaire 1

System Adjustment to Individual Human Scales

Testing 6 Scenarios under different setups

Questionnaire 2

System Deinstallation Proposals for Design, Metric, Metaphor and Application Evaluation of Usability Tests Space

Figure B-2: Development phases within the ErgoSim project 125

Task and Requirements and Test Scenario Definition Application Space

Preliminary System Definition & Device Setup Evaluation Target System Benchmarks

Implementation Acceptance of Testbed Environment Kick-Off

User Interface & System Design

System Implementation Usability Test Design Protoype 1, 2 Scenebase Modeling and Processing - Questionnaire - Scene Layout - Lights - Interview - Object Geometry - Texture Maps - Training Programming - Test Set - Configurations - Interaction - Behaviour - Dynamics

Device Interfacing Acquisition of - System Setup Test Persons - Device Modification

System Testing & Tuning Final System

Usability Testing Usability Test Results Training Explanation of Experiments Qustionnaire 1: VR Pre-Experience Qustionnaire 2: Emotional Condition Test Scenario 1, 2, 3, or 4 Qustionnaire 3: Emotional Condition Qustionnaire 4: Handling Qustionnaire 5: Judgement of System Setups Qustionnaire 6: Judgement of Application Potentials Short Informal Interview

Evaluation of Usability Tests Proposals for Design, Metric, Metaphor and Application Space

Figure B-3: Development phases within the ViRTIS project

Appendix C

ViRGIS System Performance

3000 low system load

2500 high system load

2000

1500

1000

500 load time per patch [ms] 0 geometries textures infobjects

data objects

Figure C-1: Total load times per LOD3 patch under different system load conditions applying system configuration a) 128 Appendix C

2000 geometries 1800 textures infobjects 1600 1400 1200 1000 800 600

time per patch [ms] 400 200 0 dbms (low dbms (high networking networking system system (low (high load) load) system system load) load) system load condition and processing segment

Figure C-2: Processing times per LOD3 patch for dbms and networking segment under different system load conditions applying system configuration a)

1200 LOD3 1000 AdaptiveLOD3

800

600

400

200 load time per patch [ms] 0 geometries textures infobjects

data objects

Figure C-3: Total load times under different high LODs per patch at low system load condition applying system configuration b) 129

1000 Sun SPARC10 900 Sun Ultra1 800 700 600 500 400 300 200 100 dbms time per patch [ms] 0 geometries textures infobjects

data objects

Figure C-4: Total load times per LOD3 patch comparing different server platforms at low system load condition applying system configuration b)

1800 gis_server 1600 pool_server 1400 1200 1000 800 600 400

load time per patch [ms] 200 0 geometries textures infobjects

data objects

Figure C-5: Total load times per LOD3 patch comparing different server models at low system load condition applying system configuration a) 130 Appendix C

5000 5000

4000 4000

3000 3000

2000 2000

1000 patch [ms] 1000 load time per patch [ms]

load time per 0 0 geometries textures infobjects geometries textures infobjects data objects data objects SGI Onyx RE2 SGI Onyx RE2 SGI Indigo2 High IMPACT 10000 SGI Indigo2 High IMPACT 10000

a) two clients served by one server b) two clients, each served by one server

5000 5000

4000 4000

3000 3000

2000 2000

1000 patch [ms] 1000 patch [ms] load time per load time per 0 0 geometries textures infobjects geometries textures infobjects data objects data objects SGI Onyx RE2 SGI Onyx RE2 SGI Indigo2 High IMPACT 10000 SGI Indigo2 High IMPACT 10000

c) two clients served by one server d) two clients, each served by one server

Figure C-6: Total load times per LOD3 patch comparing different client-server setups at high system load condition: a) & b) under system configuration c); c) & d) under system configuration d).

6000 1 Client : 1 Server 4 Clients : 1 Server 5000 4 Clients : 4 Servers

4000

3000

2000 load time per patch [ms] 1000

0 geometries textures infobjects

data objects

Figure C-7: Total load times per LOD3 patch comparing different client-server setups at high system load condition using system configuration a). Bibliography

[Aman95] Amand, St., MPS-Admin - Konzeption und Implementierung eines netz- werkbasierten Verwaltungsdienstes für Multi-Port-Service (MPS) Module, Diplomarbeit in Informatik, Institut für Informatik der Univer- sität Zürich, Zürich, 1995. [Ames96] Ames, A.L., Nadeau, D.R., Moreland, J.L., The VRML Sourcebook, Wiley & Sons, New York, 1996. [Ames97] Ames, A.L., Nadeau, D.R., Moreland, J.L., VRML 2.0 Sourcebook, Wiley & Sons, New York, 1997. [And95] Andresen, H., Wapler, M., Grefen, K., DIGIHOM - The Digital Man, in: Virtual Reality World ´95, Computerwoche Verlag, München, 1995, pp.153-157. [Asch94] Aschwanden, P. W., Pajarola, R. B., ViRGIS: Virtual Reality based Geo- graphic Information System, Masters Thesis, Institute of Theoretical Computer Science, ETH Zurich, MultiMedia Laboratory, Computer Science Department, University of Zurich, Zurich, March 1994. [Auk92] Aukstakalnis, St., Blatner, D., Silicon Mirage - The Art and Science of Virtual Reality, Peachpit Press, Berkeley, 1992. [Bach86] Bach, M.J., The Design of the UNIX Operating System, Prentice-Hall, 1986. [Bau95] Bauer, Ch., Jacobson, R., in: ENTER´95: Information and Commu- nication Technologies in Tourism, Springer, Vienna, 1995, pp.17-21. [Benj95] Benjamin, I., Cooper, M., Virtual Tourism - A Realistic Assessment of Virtual Reality for the Tourist Industry, in: ENTER´95: Information and Communication Technologies in Tourism, Springer, Vienna, 1995, pp.135-143. [Beg94] Begault, D.R., 3-D Sound for Virtual Reality and Multimedia, AP Professional, Boston, 1994. [Blau93] Blau, B., Moshell, J.M., McDonald, B., The Distributed Interactive Simulation Protocols and their Application to Virtual Environments, in: Proceedings of the Meckler Virtual Reality 93 Conference, CD-ROM, Meckler Media, 1993. [Bloc96] Bloch, M., Pigneur, Y., Steiner, Th., The IT-Enabled Extended Enter- prise, Applications in the Tourism Industry, in: ENTER´96: Information and Communication Technologies in Tourism, Springer, Vienna, 1996, pp.113-120. 132 Bibliography

[Bly93] Bly, S., Harrison, S., Irwin, S., Media Spaces: Video, Audio and Com- puting, Communications of the ACM, Vol. 36, No.1, 1993, pp.28-47. [Brick90] Bricken, W., Virtual Environment Operating System: Preliminary Func- tional Architecture, Technical Memorandum M-90-2, Human Interface Technology Lab, University of Washington, Washington, 1990. [Brick91] Bricken, W., Training in Virtual Reality, in: Virtual Reality 91 - Impacts and Applications, Proceedings of the First Annual Conference on Virtual Reality 91, Meckler, London, 1991, pp.46-48. [Brys94a] Bryson, St., Designing a Virtual Reality Application, SIGGRAPH´94, Course Notes 2, ACM SIGGRAPH, Orlando (FL), 1994. [Brys94b] Bryson, St., et.al., Research Frontiers in Virtual Reality, Proceeding of ACM SIGGRAPH´94, ACM SIGGRAPH, Orlando (FL), 1994, pp.473- 474. [Burd96] Burdea, G.C., Force and Touch Feedback for Virtual Reality, Wiley & Sons, New York, 1996. [Carl95] Carlson, Ch., Jää-Aro, K.-M., The DIVE Laboratory, in: Presence, Vol. 4, No. 4, MIT Press, Cambridge, 1995, pp.431-440. [Clyd95] Clyde, St.W., Landfried, T., OuR-Town: A Distributed, Multi-media System for Travel and Tourism, Applications in the Tourism Industry, in: ENTER´95: Information and Communication Technologies in Tourism, Springer, Vienna, 1995, pp.278-287. [Corn70] Cornsweet, T., Visual Perception, Academic Press, New York, 1970. [Cruz93] Cruz-Neira, C., Sandin, D.J., DeFanti, T.A., Surround-Screen Projection- Based Virtual Reality: The Design and Implementation of the CAVE, Proceedings of ACM SIGGRAPH´93, in: Computer Graphics, Annual Conference Series, ACM SIGGRAPH, August 1993, pp.135-142. [Cruz95] Cruz-Neira, C., Roy, T.M., DeFanti, T.A., Cosmic Worm in the CAVE: Steering a High-Performance Computing Application from a Virtual Environment, in: Presence, Vol. 4, No.2, MIT Press, Cambridge, 1995, pp.121-129. [Cutt93] Cutt, P., Tactile Displays: Adding the Sense of Touch to Virtual Environ- ments, Proceedings of Virtual Reality Systems ´93, New York, March 1993, pp.120-122. [Dam94] van Dam, A., VR as a Forcing Function: Software Implications of a New Paradigm, SIGGRAPH´94, Course Notes 2, ACM SIGGRAPH, Orlando (FL), 1994. [Dant94] Dantine, W., Tourist & Guest 2000, in: ENTER´94: Information and Communication Technologies in Tourism, Springer, Vienna, 1994, pp.15-19. [Deer93] Deering, M.F., Explorations of Display Interfaces for Virtual Reality, in: Proceedings of Virtual Reality Annual International Symposium, IEEE, Seattle, 1993, pp. 141-147. [Drös74] Drösler, J., Das beidäugige Raumsehen. in: W. Metzger (Hrsg.) Handbuch der Psychologie, 1. Band: Allgemeine Psychologie, I. Der Aufbau des Erkennens, 1.Halbband: Wahrnehmung und Bewußtsein, Hogrefe, Göttingen, 1974, pp.590-615. [Done94] Donelson, A., Drafting Boards and Blueprints Give Way To Interactive Design, Virtual Reality World, Nov./Dec. 1994, Mecklermedia, Westport (CT), 1994, pp.7-8. 133

[Durl95] Durlach, N.I., Mavor, A.S. (eds.), Virtual Reality: Scientific and Techno- logical Challenges, National Academy Press, Washington D.C., 1995. [dVS94] dVS Technical Overview, Divison Inc., Bristol (UK), 1994. [Ebne94] Ebner, A., TIS Tourism Information System for the Tyrol, in: ENTER´94: Information and Communication Technologies in Tourism, Springer, Vienna, 1994, pp.35-42. [Elma89] Elmasri, R., Navathe, S.B., Fundamentals of Database Systems, Fort Collins, Redwood City, 1989. [Ern94] Ernst, M., Walpuski, D., Strategy and Information Technology in the Tourist Industry, in: ENTER´94: Information and Communication Tech- nologies in Tourism, Springer, Vienna, 1994, pp.228-235. [Fein93] Feiner, St., et al, A Virtual World for Network Management, in: VRAIS´93, IEEE Annual Virtual Reality International Symposium, Seattle (WA), September 1993, pp. 55-61. [Fol96] Foley, J.D., van Dam, A., Feiner, St.K., Hughes, J.F., Computer Graphics - Principles and Practice, Addison-Wesley, Reading, 1996. [Free84] Freeman, R.E., Strategic Management, Boston, 1984, p.55. [Gave86] Gaver, W., Auditory Icons: Using Sound in Computer Interfaces, Human Computer Interaction, Vol.2, 1986, pp.167-177. [Gave92] Gaver, W. W., The Affordances of Media Spaces for Collaboration, in: J. Turner & R. Kraut (eds.), CSCW ´92 - Proceedings of the Conference on Computer-Supported Cooperative Work, ACM, New York, 1992, pp. 17- 24. [Gibs84] Gibson, W., Neuromancer, Ace Books, New York, 1984. [Gree90] Green, M., Virtual Reality User Interfaces: Tools and Techniques, in: T.S. Chua & T.L. Kunii (eds.), Computer Graphics International ´90, Springer, Tokyo, 1990, pp.51-68. [Grim91] Grimsdale, Ch., A Parallel Vision for Virtual Reality, in: Proceedings of the First Annual Conference on Virtual Reality 91, Meckler Media, London, June 1991, pp.12-17. [Gutt84] Guttman, A., R-Trees: A Dynamical Index Structure for Spatial Searching, Proceedings of the ACM-SIGMOD International Conference on the Management of Data, Boston, 1984. [Hash93] Hashimoto, H., Kunii, Y., Buss, M., Harashima, F., Dynamic Force Simulator for Force Feedback Human-Machine Interaction. in: IEEE Virual Reality Annual International Symposium, Piscataway: IEEE Service Center, 1993, pp.209-215. [Heij96] van der Heijden, J.G.M., The Changing Value of Travel Agents in Tourism Networks: Towards a Network Design Perspective, in: ENTER´96: Information and Communication Technologies in Tourism, Springer, Vienna, 1996, pp.151-159. [Hein93] Heinrich, L., Wirthscaftsinformatik: Einführung und Grundlagen, Olden- burg, München, 1993. [Hinck94] Hinckley, K., et.al., Passive Real-World Interface Props for Neuro- surgical Visualization, in: Proceedings of Computer Human Interaction 94, ACM Press, Boston, 1994, pp.452-458. [Hit93] Hitchner, L.E., Methods for User-Based Reduction of Model Complexity for Virtual Planetary Exploration, in: IS&T/SPIE´s Symposium on Electronic Imaging, San Jose, February 1993. 134 Bibliography

[Hohl96] Hohl, B., Konzeption und Entwicklung eines UNIX Socket Basierten Virtual Device Actors für das Virtual Reality Toolkit dVS, Semesterarbeit in Informatik, Institut für Informatik der Universität Zürich, Zurich, 1996. [Hone97] Honegger, F., Kurth, J., Ergonomieuntersuchung einer Virtual Reality Umgebung mit verschiedenen Ein- und Ausgabemedien, Final Project Report, Institute of Hygiene and Applied Physiology, Swiss Federal Institute of Technology Zurich, Zurich, 1997. [Horn74] Hornby, A.S., Oxford Advanced Learner´s Dictionary of Current English, Oxford University Press, Oxford, 1974. [Ishi93] Ishi, M., Sato, M., A 3D Interface Device with Force Feedback: A Virtual Work Space for Pick-and-Place-Tasks, in: Proceedings of Virtual Reality Annual International Symposium, IEEE, Seattle, 1993, pp.331- 335. [Jacob94] Jacobson, L., Garage Virtual Reality, Sams Publishing, Indianapolis, 1994. [Kaba94] Kabbash, P., Buxton, W., Sellen, A., Two-handed Input in a Compound Task, in: Proceedings of the ACM CHI 94, Addison-Wesley, 1994, pp.417-423. [Kaeg95] Kaegi, F., Roth, M.,ViRGIS 2.0: Virtual Reality based Geographic Information System, Masters Thesis, Institute of Theoretical Computer Science, ETH Zurich, MultiMedia Laboratory, Computer Science Department, University of Zurich, Zurich, March 1995. [Kahn73] Kahneman, D., Attention and Effort. Englewood Cliffs, Prentice-Hall, 1973. [Kala93] Kalawsky, R. S., The Science of Virtual Reality and Virtual Environ- ments, Addison-Wesley, Workingham, 1993. [Koll95] Koller, D., et.al., Virtual GIS: A Real-Time 3D Geographic Information System, GVU Technical Report 95-14, Graphics, Visualization and Usa- bility Center, Georgia Institute of Technology, Atlanta, 1995. [Kropp97] Kropp, M., Remmele, P., Roos, T., A Simulated Annealing Algorithm for Approximating Triangulated Surfaces, in: Proceedings of 2nd MATH- MOD Conference, Vienna 1997. [Krue91] Krueger, M.W., Artificial Reality II, Addison-Wesley, Reading, 1991. [Krue95] Krueger, W., et.al., The Responsive Workbench - Interactive Multimedia Workplace Virtual Working Environment for Scientists, Physicians and Architects, Gesellschaft für Mathematik und Datenverarbeitung, Institut für Visualisierung und Design von Mediensystemen, Forschungsbereich Intelligente Multimediasysteme, GMD St.Augustin, 1995. [Lamp94] Lampton, D.R., et. al., The Virtual Environment Performance Assessment Battery (VEPAB): Development and Evaluation, in: Presence, Vol. 3, No.2, MIT Press, Cambridge, 1994, pp.145-157. [Lantz96] Lantz, E., et.al., The Future of Virtual Reality: Head Mounted Displays Versus Spatially Immersive Displays, SIGGRAPH´96, Panel Sessions, ACM SIGGRAPH, New Orleans, 1996, pp.485-486. [Lowe94] Lowe, B., Three Case Studies in Virtual Reality Systems, in: VR´94, Proceedings of the 4th Annual Conference on Virtual Reality, Meckler- media, Westport, February 1994, pp. 72-76. 135

[Mack91] Mackinlay, J. D., Robertson, G. G., Card, S. K., The Perspective Wall: Detail and Context Smoothly Integrated, in: Human Factors in Compu- ting Systems CHI '91, ACM Press, New York, 1991, pp.173-179. [Mai95] Maier, R., Röckelein, W., Electronic Networks: New Challenges for Tourist Organizations, in: ENTER´95: Information and Communication Technologies in Tourism, Springer, Vienna, 1995, pp.112-123. [Marc92] Marcus, A., Graphic Design for Electronic Documents and User Inter- faces, ACM Press, New York, 1992. [Mert91] Mertens, P., Griese, J., Integrierte Informationsverarbeitung: Planungs- und Kontrollsysteme in der Industrie, Gabler, Wiesbaden, 1991. [Metz74] Metzger, W., Das einäugige Tiefensehen. in: W. Metzger (Eds.) Handbuch der Psychologie, 1. Band: Allgemeine Psychologie I. Der Aufbau des Erkennens, 1.Halbband: Wahrnehmung und Bewußtsein, Hogrefe, Göttingen, 1974, pp. 556-589. [Möll93] Möller, K.H., Paulish, D.J., Software Metrics, Chapman & Hall, London, 1993. [Moun90] Mountford, S., Gaver, W., Talking and Listening to Computers, in: Laurel, B. & Mountford, S. (eds.), The Art of Human-Computer Interface Design, Addison-Wesley, Reading, 1990, pp. 319-334. [Musi93] Musil, S., Pigel, G., Virgets: Elements for Building 3D User Interfaces, in: Virtual Reality Vienna ´93, Abstracts Collection, Austrian Society for Virtual Reality, Telepresence and Cyberspace, Vienna, December 1993, p. 22. [Musi95] Musil, S., Pigel, G., Tscheligi, M., The Holiday Supermarket, in: ENTER´95: Information and Communication Technologies in Tourism, Springer, Vienna, 1995, pp.1-9. [Naef96] Naef, Ch., Empfehlungen zur ergonomischen Cockpitgestaltung, Final Report, Institute of Hygiene and Applied Physiology, Swiss Federal Institute of Technology Zurich, Zurich, 1996. [Neis76] Neisser, U., Cognition and Reality, Freeman, San Francisco, 1976. [Neve95] Neves, N., et.al., Virtual GIS Room: Interfacing Spatial Information in Virtual Environments, in: 1st Conference on Spatial Multimedia and Virtual Reality, Lisbon, 1995, pp.45-53. [Niel93] Nielsen, J., Usability Engineering, Academic Press, Boston, 1993. [Niev86] Nievergelt, J., Ventura, A., Hinterberger, H., Interactive Computer Programs for Education: Philosophy, Techniques, and Examples, Addison-Wesley, Reading (MA), 1986. [Nuñez95] Nuñez, M., ViRCAD: Einsatzmöglichkeiten für Virtual Reality im Pro- duktdesign in Verbindung mit Computer Aided Design, Semesterarbeit in Informatik, Institut für Informatik der Universität Zürich, Zürich, 1995. [Over92] Overmars, M.H., A Graphical User Interface Toolkit for SiliconGraphics Workstations, User Manual, 1992. [Paja97] Pajarola, R., Ohler, Th., Stucki, P, Szabó, K., Widmayer, P., Virtual Reality and Geoinformation Systems, Technical Report No. 266, Institute of Theoretical Computer Science, Department of Computer Science, ETH Zurich, Zurich, May 1997. [Pau94a] Pausch, R., Interface Methods in Virtual Reality, SIGGRAPH´94, Course Notes 2, ACM SIGGRAPH, Orlando (FL), 1994. 136 Bibliography

[Pau94b] Pausch, R., Software Development Environments and Architectures for Virtual Reality, SIGGRAPH´94, Course Notes 2, ACM SIGGRAPH, Orlando (FL), 1994. [Pau95] Pausch, R., Burnette, T., Navigation and Locomotion in Virtual Worlds via Flight into Hand-Held Miniatures, in: Proceedings of SIGGRAPH´95, ACM SIGGRAPH, Los Angeles (CA), 1995, pp.399- 400. [Perf94] SiliconGraphics Inc., IRIS Performer™ Programming Guide, SiliconGraphics, Mountain View (CA), 1994. [Pesce95] Pesce, M., VRML-Browsing and Building Cyberspace, New Riders, Indianapolis, 1995. [Pfeff91] Pfeffer, D., Christidis, A., Klärner, E., Untersuchung des Beitrages der Stereoskopie zur Tiefenwahrnehmumg, in: R. Möller (ed.) 2. Workshop Sichtsysteme, Springer, Berlin, 1991, pp. 17-32. [Pime93] Pimentel, K., Teixeira, K., Virtual Reality: Through the New Looking Glass, Windcrest Books, New York, 1993. [Raut93] Rauterberg, M., Kühni, M., Styger, E., Spiess P., 3-D versus 2 1/2-D: eine experimentelle Vergleichsstudie, Technical Report VR-2-93, Institut für Arbeitspsychologie, Federal Institute of Technology of Zurich, Zurich, 1993. [Raut94a] Rauterberg, M., Motavalli, M., Darvishi, A., Schauer, H., Automatic Sound Generation for Spherical Objects Hitting Straight Beams Based on Physical Models, in: Ottmann, T. & Tomek, I. (eds.), Educational Multi- media and Hypermedia, Proceedings ED-MEDIA´94, Association for the Advancement of Computing in Education, Charlottesville, 1994, pp.468- 473. [Raut94b] Rauterberg, M., Styger, E., Positive Effects of Sound Feedback During the Operation of a Plant Simulator, in: Blumenthal, B., Gornostaev, J. & C. Unger (eds.), Human Computer Interaction, Lecture Notes in Computer Science, Vol. 876, Springer, Berlin, 1994, pp.35-44. [Raut94c] Rauterberg, M., Styger, E., Baumgartner A., Jenny, A., de Lange, M., Additional Sound Feedback in Man-Computer Interaction: Empirical Investigations, in: Grieco, A., Molteni, G., Occhipinti, E. & Piccoli, B. (eds.), Book of Short Papers of 4th International Conference on Work with Display Units, Vol. 3, Institute of Occupational Health, University of Milan, 1994, pp.D18-D21. [Raut96] Rauterberg, M., Steiger, P., Pattern Recognition as a Key Technology for the Next Generation of User Interfaces, in: Proceedings of the International Conference on Systems, Man and Cybernetics, Vol.4, IEEE, Piscataway, 1996, pp.2805-2810. [Riba94] Ribarsky, W., et.al., Information Visualization and Analysis Using Virtual Reality, IEEE Computer Graphics and Applications, 10, January, 1994. [Ribb94] Ribbers, P.M., Strategy and Information Technology in the Tourist Industry, in: ENTER´94: Information and Communication Technologies in Tourism, Springer, Vienna, 1994, pp.9-14. [Rimi92] RIMINI, Das digitale Höhenmodell der Schweiz mit einer Maschenweite von 250 Metern, Bundesamt für Landestopographie, Wabern, 1992. [Robe93] Robertson, G. G., Card, S. K., Mackinlay, J. D., Information Visuali- zation Using 3D Interactive Animation, Communications of the ACM, Vol. 36 No.4, April, 1993, pp.57-71. 137

[Rohl94] Rohlf, J., Helman, J., IRIS Performer: A High Performance Multi- processing Toolkit for Real-Time 3D Graphics, Proceedings of ACM SIGGRAPH´94, in: Computer Graphics, Annual Conference Series, ACM SIGGRAPH, August 1994, pp.381-394. [Rui94] Ruisz, J., Change Demands on Hotel Management, in: ENTER´94: Information and Communication Technologies in Tourism, Springer, Vienna, 1994, pp.27-34. [Sata95] Satawa, R., Telesurgery, Virtual Reality and the New World Order of Medicine, in: Virtual Reality World ´95, Computerwoche Verlag, München, 1995, pp.173-177. [Scha91] Schaffer, R.L., Pope, A.R., The SIMNET Network and Protocols, Technical Report 7627, BBN Systems and Technologies, Advanced Simulation Division, 1991. [Sche94] Schertler, W., Tourism 2000 - An Information Business, in: ENTER´94: Information and Communication Technologies in Tourism, Springer, Vienna, 1994, pp.20-26. [Sche95] Scheer, A.W., Wirtschaftsinformatik - Referenzmodelle für industrielle Geschäftsprozesse, Springer, Berlin, 1995. [Schm94] Schmid, B., Electronic Markets in Tourism, in: ENTER´94: Information and Communication Technologies in Tourism, Springer, Vienna, 1994, pp.1-8. [Shaw93] Shaw, C., Green, M., The MR Toolkit Peers Package and Experiment, IEEE Proceedings Virtual Reality Annual International Symposium (VRAIS) 93, Washington, September 1993, pp.463-469. [Shne92] Shneiderman, B., Designing the User Interface, Addison-Wesley, Reading, 1992. [Sink94] Sinkwitz, R., Interactive Parallel Distributed Particle Simulation under Real-Time Conditions in a Workstation Cluster, Doctoral Thesis, Depart- ment of Computer Science, University of Zurich, Zurich, 1994. [SPP95] Stucki, P, et.al., A Virtual Reality Framework for the Design, Optimiza- tion and Management of Mobile Telematics Infrastructure, SPP IaC, Swiss National Science Foundation, Bern, 1995. [Star90] Star, J., Estes, J., Geographic Information Systems: An Introduction, Englewood Cliffs, 1990. [Stev36] Stevens, S., Newman, E., The Localization of Actual Sources of Sound, American Journal of Psychology, Vol. 48, 1936, pp.297-306. [Stev90] Stevens, W.R., Unix Network Programming, Prentice-Hall, Englewood Cliffs, 1990. [Szabo93] Szabó, K., Rauterberg, M., 2 1/2D versus 3D - A Classification Concept and an Experimental Comparison, in: Virtual Reality Vienna ´93, Abstracts Collection, Austrian Society for Virtual Reality, Telepresence and Cyberspace, Vienna, December 1993, p. 23. [Szabo94a] Szabó, K., Stucki, P., Blatter, M., Horezky, C., Pauli, K., Ruff, M., Simeon, P., Development of a CD-ROM-based Multimedia-Application in an Open System Environment, in: ENTER´94: Information and Communication Technologies in Tourism, W.Schertler/B.Schmid/A M.Tjoa/ H.Werthner (Eds.), Springer, Vienna, 1994, pp.110-115. [Szabo94b] Szabó, K., Computer Supported Cooperative Work 2000: Ein Ausblick, in: Computer Supported Cooperative Work, Technical Report No. 94.11, Department of Computer Science, University of Zurich, 1994, pp.93-108. 138 Bibliography

[Szabo95a] Szabó, K., Stucki, P., Aschwanden, P., Ohler, T., Pajarola, R., Widmayer, P., A Virtual Reality based System Environment for Intuitive Walk-Through and Exploration of Large-Scale Tourist Information, in: ENTER´95: Information and Communication Technologies in Tourism, Springer,Vienna, 1995, pp.10-15. [Szabo95b] Szabó, K., Kaegi, F., Pajarola, R., Roth, M., Virtual Reality and Tourism: A Swiss Project, in: Virtual Reality World ´95, Meckler Media, CD- ROM, San Jose (CA), May 1995. [Szabo95c] Szabó, K., Rauterberg, M., A Design Concept for n-Dimensional User Interfaces, in: Interface to Real and Virtual Worlds 95, Montpellier, June 1995, pp.467-477. [Szabo95d] Szabó, K., Virtual Reality basierte Informationssysteme für die Medizin, in: 49. Van Swieten Congress, Verlag der Österreichischen Ärzte- kammer, Vienna, November 1995, pp. 204-207. [Szabo96] Szabó, K., Kloss, J.H., Virtual Reality Special Interest Groups around the World, in: Virtual Reality World ´96, Computerwoche Verlag, Stuttgart, February 1996. [Szabo97] Szabó, K., Kloss, J.H., Rockwell, R., Duchrow, M., VRML97: Der internationale Standard für interaktive 3D-Welten im World Wide Web, Addison-Wesley, Bonn, to be published in Oktober 1997, ISBN 3-8273- 1187-X. [Teu95] Teunissen, W.J.M., ISDN in the Travel Industry: A Catalyst for Change?, in: ENTER´95: Information and Communication Technologies in Tourism, Springer, Vienna, 1995, pp.33-42. [Teuf95] Teufel, S., Sauter, C., Mühlherr, T., Bauknecht, K., Computerunter- stützung für die Gruppenarbeit, Addison-Wesley, Bonn, 1995. [VRML96] The VRML 2.0 Specification, VRML Consortium, 1996, URL: http://vag.vrml.org. [Wall40] Wallach, H., The Role of Head Movements and Vestibular and Visual Cues in Sound Localization, Journal of Experimental Psychology, Vol. 27, 1940, pp.339-368. [Wall49] Wallach, H., Newman, E., Rosenzweig, M., The Precedence Effect in Sound Localization, American Journal of Psychology, Vol. 52, 1949, pp.315-336. [Wall53] Wallach, H., O'Connell, D., The Kinetic Depth Effect, Journal of Experimental Psychology, Vol. 45, 1953, pp.205-217. [Ware93] Ware, C., Arthur, K., Booth, K., Fish Tank Virtual Reality, in: S. Ashlund, K. Mullet, A. Henderson, E. Hollnagel & T. White (eds.) Conference on Human Factors in Computing Systems INTERCHI '93, ACM, New York, 1993, pp.37-42. [Weis91] Weiser, M., The Computer for the 21st Century, Scientific American, No. 3, September 1991. [Weis95] Weiser, Ch., Multimedia Information Systems for Hotel Guides, in: ENTER´95: Information and Communication Technologies in Tourism, Springer, Vienna, 1995, pp.268-277. [Well94] Wellner, P., Interacting with Paper on the DigitalDesk, Technical Report 330, University of Cambridge, Computer Laboratory, Cambridge (UK), March 1994. 139

[Wlok96] Wloka, D.W., CAVE: Personal or Small Group Non-HMD-Based Head- Tracked Wrap-Around Virtual Environment - The System of the Future for Virtual Reality Applications, in: Virtual Reality World ´96, Computerwoche Verlag, Stuttgart, February 1996. [Zhai96] Zhai, S., Milgram, P., Buxton, W., The Influence of Muscle Groups on Performance of Multiple Degree-of-Freedom Input, in: CHI´96, Vancouver, 1996, pp.308-315. [Zyda93] Zyda, M., et.al., The Software Required for the Computer Generation of Virtual Environmnets, in: Presence, Vol. 2, No. 2, MIT Press, Cambridge, 1993, pp.130-140.

Curriculum Vitae

Personalien

Name: Szabó Vorname: Kornél Geboren: 11. Oktober 1965 in Solothurn Wohnhaft: Birkenstrasse 22, CH-8306 Brüttisellen (ZH) Bürgerort: Langendorf (SO)

Bildungsgang

1972 - 1978 Primarschule in Brüttisellen 1978 - 1984 Mittelschule in Zürich-Oerlikon 1986 - 1993 Studium Wirtschaftsinformatik an der Universität Zürich 1993 - 1997 Assistenz und Doktorandenstudium am MultiMedia Laboratorium, Institut für Informatik der Universität Zürich

Beruflicher Werdegang

1984 Instruktur am 1. Schweizer Computer-Camp, Weggis 1985 Programmierer bei Softlab AG, Dietlikon 1987 Management Assistent, Departement für Informatik, Winterthur Lebensversicherungsgesellschaft, Winterthur 1987 - 1993 Hilfsassistent bei Prof. Dr. Peter Stucki, MultiMedia Laboratorium, Institut für Informatik der Universität Zürich 1989 Industriepraktikum bei IBM, Kopenhagen (Dänemark) 1990 Industriepraktikum bei Arix Corp., San Jose, Californien (USA) 1993 - 1994 Produktmanager bei ASI Products AG, Zürich 1993 - 1997 Assistent und Doktorand bei Prof. Dr. Peter Stucki, MultiMedia Laboratorium, Institut für Informatik der Universität Zürich 1995 - Marketing und Verkauf, Relog AG, Zürich 1995 - Geschäftsführer Alpnet K.Szabó, Brüttisellen

Dozentenverzeichnis

Ackermann, D. Hässig, K. Mössenböck, H.P. Schauer, H. Adamov, R. Janssen, M. Mumprecht, E. Schelbert, H. Baer, R. Kiechl, R. Osterloh, M. Stucki, P. Bauknecht, K. König, P. Pfeiffer, R. Volkart, R. Boley, P. Krulis-Randa, J. Pomberger, G. Wehrli, H.P. Dittrich, K.R. Linder, W. Richter, L. Weilenmann, P. Frey, B.S. Marty, R. Rossi, A. Zweifel, P. Fortstmoser, P. Meister, B. Rühli, E.