George G. Robertson, Stuart K. Card, andJock D. Mackinlay INFORMATION VISUALIZATION USING 3D INTERACTIVE ANIMATION innovations are often driven by a combination of technology advances and application demands. On the technology side, advances in interactive computer graphics hardware, coupled with low-cost mass storage, have created new possibilities for information retrieval systems in which UIs could play a more central role. On the application side, increasing masses of information confronting a business or an individual have created a demand for information management applications. In the 1980s, text-editing forced the shaping of the desktop metaphor and the now standard GUI paradigm. In the 1990s, it is likely that information access will be a primary force in shaping the successor to the desktop metaphor. This article presents an experimental system, the Information Visual/mr (see Figure 1), which explores a UI paradigm that goes beyond the desktop metaphor to exploit the emerging generation of graphical personal computersand to support the emerging application demand to retrieve, store, manipulate, and understand large amounts of information. The basic problem is how to utilize advancing graphics cecnnology to lower the cost of finding Information and accessing it once found (the information's "cost structure"). I t b U S T ~ A T I O N : R I C O k a N $ S T U D I O ¢~NIIINUCAT, OI4$OpT~/April 1993/%1.36, No.4 sml models. At the same time, software on Software Architectures for Non- Figure 1. support for real-time operating sys- WIMP User Interfaces [9]. It is this Information tems and emerging industry stan- kind of technology change that is Visualizer Overview dard open graphics libraries (e.g., driving our research in the Informa- tion Visualizer. We take four broad strategies: mak- OpenGL and PEX) are simplifying ing the usel:'S immediate workspace the 3D programming task. The trend larger, enabfing user interaction with will bring these technologies to the Information Access vs. Document Retrieval multiple agents, increasing the real- mass market in the near future. These technology advances have Computer-aided access to informa- time interaction rate between user and created many possibilities for user tion is often thought of in the context system, and using visual abstraction to interface innovation. Yet the basic of methods for library automation. shift information to the perceptual Windows-Icons-Menus-Pointing In particular, document retrieval system to speed information assimila- (WIMP) desktop metaphor has not [19] is usually defined more or less as tion and retrieval. changed much since its emergence in follows: There exists a set of docu- Technology Advances the Alto/Smalltalk work. Nonethe- ments and a person who has an inter- Since the early development of the less, there is a great desire to explore est in the information in some of standard GUI, hardware technology new UI paradigms. Experiments them. Those documents that contain has continued to advance rapidly. with pen-based notebook metaphors, information of interest are relevant, Processor and memory technology virtual reality, and ubiquitous com- others not. The problem is to find all have far greater performance at far puting are proceeding and may and only the relevant documents. lower cost. Specialized 3D graphics eventually influence the mass mar- There are two standard figures of hardware has made it progressively ket. Brown University's Andy van merit for comparing and evaluating faster and cheaper to do 3D transfor- Dam, in several recent conferences retrieval systems: Recall is the per- mations, hidden-surface removal, has exhorted us to break out of the centage of all the relevant documents double-buffered animation, an- desktop metaphor and escape flat- found; and precision is the percentage tialiasing, and lighting and surface land and a recent workshop focused of the documents found that are rel- 58 April 1993/Vol.36, No.4 /|ONNUNICJI~'I'ION|OPTH|A|N evant. While this formulation has als is tuned to the requirements of ondary or tertiary storage with infor- been useful for comparing different the work process using them. mation access from Immediate Stor- approaches, we propose extending Computer screens provide a work- age for information in use, and (2) the document retrieval formulation space for tasks done with the com- considers information access part of to take the larger context into ac- puter. However, typical computer a larger work process. That is, in- count. From a user's point of view, displays provide limited working stead of concentrating narrowly on document retrieval and other forms space. For real work, one often wants the control of a search engine, the of information retrieval are almost to use a much larger space, such as a goal is to improve the cost structure always part of some larger process of dining room table. The Rooms sys- of information access for user work. information use [2]. Examples are tem [10] was developed to extend the With this system, we use four (building an interpreta- WIMP desktop to multiple work- sensemaking methods for improving the cost tion of understanding of informa- spaces that users could switch structure of information access: tion), design (building an artifact), de- among, allowing more information cision making (building a decision and to reside in the immediate work area. 1. Large Workspace. Make the Imme- its rationale), and response tasks (find- The added cost of switching and diate Workspace virtually larger, so ing information to respond to a finding the right workspace was re- that the information can be held in query). duced by adding the ability to share low-cost storage In each of these cases: the same information objects in dif- 2. Agents. Delegate part of the work- ferent workspaces. Rooms also had load to semiautonomous agents 1. Information is used to produce an overview and other navigational 3. Real-Time Interaction. Maximize more information, or to act di- aids as well as the ability to store and interaction rates with the human rectly retrieve workspaces, all to remove user by tuning the displays and re- 2. The new information is usually at the major disadvantages of multiple sponses to real-time human action a higher level of organization rel- desktops. constants ative to some purpose The essence of our proposal is to 4. Visual Abstractions. Use visual ab- If we represent the usual view of evolve the Rooms multiple desktop stractions of the information to information retrieval as Figure 2(a), metaphor into a workspace for infor- speed assimilation and pattern detec- we can represent this extended view mation access--an Information Work- tion as Figure 2(b). Framing the problem space [2]. Unlike the conventional in- in this way is suggestive: what the formation retrieval notion of simple user needs is not so much informa- access of information from some dis- Figure 2. (a) Tradi- tion retrieval itself, but rather the tal storage, an information work- tional Information retrieval formula- amplification of information-based space (1) treats the complete cost tion and (D) refor- work processes. That is, in addition structure of information, integrating mulatlon with to concern with recall and precision, information access from distant, sec- context of use we also need to be concerned with (b) Amplification of (a) Information Retrieval reducing the time cost of informa- Information-Intensive Work tion access and increasing the scale of information that a user can handle at one time. Information Workspaces From our observations about the problem of information access [2], we were led to develop UI paradigms oriented toward managing the cost structure of information-based work. This, in turn, led us to be concerned not just with the retrieval of informa- tion from a distant source, but also with the accessing of that informa- tion once it is retrieved and in use. The need for a low-cost, immediate storage for accessing objects in use is common to most kinds of work. The common solution is a workspace, whether it be a woodworking shop, a laboratory, or an office. A workspace is a special environment in which the cost structure of the needed materi- COMMUNICATIONS 011TNU ACU/April1993/Vol.36, No.4 ~ Table 1. Techniques used in the Information visualizer to increase These define the goals for our UI information access per unit cost paradigm. Each of these is intended to decrease the costs for performing information-intensive tasks, or, alter- natively, to increase the scope of in- formation that can be utilized for the same cost. Figure 3 shows how these goals are applied to the reformulated information access problem shown in Figure 2(b). The Information Visualizer sys- tem is our experimental embodiment of the Information Workspace con- cept with mechanisms for addressing each of these system goals (see Table 1): 1) [Large Workspace]. We use two methods to make the workspace larger: We add more (virtual) screen Table 2. In|k)rmation Visualizer solutions to basic Ul problems. space to the Immediate Workspace by using a version of the Rooms system. We increase the density of information that can be held in the same screen space by using animation and 3D perspective. 2) [Agents]. To delegate part of the workload, we use agents to conduct searches, to organize in- formation into clusters, or design presentations of information. We manage this by means of a schedul- ing architecture, called the Cognitive Informatk Coprocessor [17], that allows multi- workspa~ ple display and application processes to run together. A kind of user inter- face agent, called Interactive Objects, is used to control and communicate with the system. 3) [Real-time Inter- action]. To maximize human interac- tion rates, we use the properties of Make Larger the scheduler to provide highly in- teractive animation and communica- Offload Work tion with the Interactive Objects.