Making Space Content Specific Interactive Architectures for Information Presentation

Gregory More RMIT University

Abstract

This paper examines the connections between digital architectures and interaction design with an emphasis on how the latter informs the former. Digital spatial interfaces have been in development for well over a decade. However there is still a distinct and problematic separation between the function of these spaces architecturally and the functional use of architectural concepts in the design of these spaces. The research presented here outlines an approach to interface design that promotes an architecture that is temporal, interactive and sonic, and is defined explicitly by a functional relationship to its informational content. In particular this research reports on the design of a software prototype that incorporates spatial concepts of interactivity, visualization and sound to assist in the navigation of presentation information, promoting space as a primary interface to an information collection.

Introduction extant architectural knowledge with the additional dimensions of hypermedia, The discipline of architecture is interactivity, and streaming information. founded on the interpretation and The Virtual New York Stock Exchange by understanding of space, and in turn, Asymptote is a well known precedent of a spatial information. Architects have virtual environment that mimics a physical explored the connections between real world space (Couture and Rashid digital information spaces and physical 2002). The project synthesizes the physical space. These relationships in general trading floor with a digital trading floor follow advancements in representational where users can view the information of technologies from the broader computing the NYSE environment in realtime via a domain, for example the use of VRML digital interface. in the mid nineties as an architectural The development of realtime modeling language. Realtime three- visualization software and graphic engines dimensional visualization techniques that represent spatial environments offer an innovative space for combining continue to furnace architectural

292 Figure 1. Asymptote, Virtual Stock Exchange. Operations Center (1999) (left), and Virtual Trading Floor (1998) (right). explorations. A collection of applications information spatially, and specifically within is available to the architect to explore this spaces that mimic real world conditions. domain, some of which are: Virtools™, For example, Win3D (Clockwise) MaxMS/Jitter, Director™, uses the analogy of rooms for various Adobe Atmosphere™, Blender, and computer applications, and Task Gallery various game engine technologies, Unreal, (Microsoft) software applications are ORGE, Torque, etc. For example Dutch located and accessed via the floor, walls architect Kas Oosterhuis is a designer and the ceiling of a generic office room. who creates digital spaces (utilizing Architecture in these examples is a Virtools) that interact and control physical backdrop, or stage, for the interaction. architectures (Oosterhuis 2003). The This is a symbolic representation of space, argument of this paper is not surveying, rather than space designed as a result validating or judging the novel use of these of function; the architecture is static and technologies by architects. It is interested, has the appearance and arrangement however, in how architects can be assisted of a space designed for the real world. by the field of interaction design when What this paper is arguing is that in designing for the digital domain, and this the digital domain we should consider is examined through the development architecture not as a frame, but as the of an experimental software prototype spatial experience that reconfigures to described later in this paper. the actions of the user. Thus avoiding Shneiderman in his essay ‘Why the mimicking of a real-world space, and Not Make Interfaces Better than 3D focusing on the advantages of a space that Reality?” discusses the development is defined specifically by the nature and of 3D interfaces (Shneiderman 2003). quantity of its information. The software Shneiderman, an important figure prototype presented in this paper in interface design and Human and embodies an attempt to move towards Computer Interaction (HCI), is interested a digital architecture that is temporal, in the advantages and disadvantages of 3D reconfigurable, and inherently functional. displays over traditional two dimensional computer interfaces. Shneiderman presents a survey of attempts to render

Gregory More Making Space Content Specific: Interactive Architectures for Information Presentation 293 Figure 2. Eureka image space (left), in Eureka in library Mode (right). Eureka Prototype PowerPoint), where elements are placed into linear lists for presenting, Eureka allows the user to arrange information into Background sets (compositions) within space, which can be traversed in a linear or nonlinear The Eureka prototype is a spatially sequence. Given the computational and adaptive hypermedia. This is a combination multimedia ability of a consumer grade of adaptive hypermedia (Brusilovsky laptop, it is difficult to understand why 2001), where content is reconfigured by we traditionally resort to a linear style of user input, and spatial hypermedia, where presenting via 2D representations of our spatial relationships make new connections information assets. As illustrated in Figure between previously unrelated content 2, the main difference between Eureka (Marshall and Shipman 1995). Eureka, as a and other presentation software is that digital environment, locates informational Eureka locates all presentation information assets within a digital space that can be spatially. This spatial arrangement creates interacted with and navigated by a user a 3D interface within which one can via an OpenGL graphics display, keyboard interact with presentation content. This and mouse. Eureka allows for the user allows for easier recognition of the lecture to compile presentation content from content by the user and the ability to use a database, alter elements, present, and perspective as a means to arrange and retain information about the presentation. collate information. The space of Eureka as an information In Eureka navigation is achieved by rich environment incorporates various the user ‘clicking on’ items, upon which modes of user interaction and content the user’s point of view is moved through arrangements, and the software prototype space to address the newly selected item. has been produced with the authoring The user is able to move freely around software Macromedia DirectorTM. the content with an interaction model In contrast to other presentation of clicking on objects to navigate. When software (for example Microsoft an item is ‘double clicked’, then the

ACADIA 2006: Synthetic Landscapes Digital Exchange 294 Digital Research: Theoretical Research image is shown in full screen mode, like the information. These concepts embrace a traditional slide show. A presentation three key characteristics that advanced is a mixture of navigating the space, and 3D interface designs have: rapid situation showing content as full screen images, awareness through effective overviews; allowing for both linear transversal and the reduced numbers of actions to accomplish nonlinear jumping between content. tasks; and prompt, meaningful feedback for user actions (Shneiderman 2003). The Architectures Defined by information modes allow for different Information types of interaction to occur with the collection set, the use of perspective and Eureka relies on content to generate camera controls allow for rapid situational space. For example a presentation with awareness, and the animated transitions a limited set of content (e.g. 30 items), and sound-scapes give feedback to the will have a less expansive sense of space users’ interactions. than one with a larger collection of items To facilitate the control of the (e.g. 200 items). This is a key difference informational assets Eureka offers the between Eureka and interfaces that try user a selection of information modes. to mimic real world space; here space These modes allow the user to switch is adaptive to the content, so there is a between different configurations of the direct relationship between the content information space, entitled presentation, and the spatial experience it creates. plans, library, agent response, and history. For To manage the interaction within space example the history mode reconfigures the design of Eureka engages two main every slide that has been shown in a principles: first, arranging elements in space presentation into a linear sequence, relative to varying modes of interaction; spatially moving them out of a relationship and second, that space is reconfigurable, with the images not shown. The library so change to the information set mode exposes all library assets for viewing transforms the spatial configuration of and selection. Each information mode can have a unique spatial configuration, user interactions, environmental conditions, and the arrangements can be designed to dynamically reflect the quantity and type of information being presented. Figure 3 represents an abstraction of how the collections are spatially related. Eureka presents an animated realtime information environment. All user movements, interactions, and spatial Figure 3. Diagram of the spatial arrangement of transformations of the information are information sets in Eureka. Depth in the space connects animated, so the environment is constantly past and present action, moving from left to right denotes sequence and sorting of information, and height adapting as the user interacts with the from bottom to top, represents a transition from static content. Decisions have been made to to temporal collections of information. rationalize the location and placement of

Gregory More Making Space Content Specific: Interactive Architectures for Information Presentation 295 information elements as digital spaces are reordered, discarded and recorded for somewhat limitless. In Eureka, as illustrated future use. At the start of a presentation in Figure 3, the current spatial model the environment is generated from this positions content relative to temporality, recorded plan of information items. so metaphorically the higher the object To allow the user to structure the within the space, the more temporal presentation, new elements are defined its nature, and vice versa. So the library as either a ‘composition’ or an ‘item’. components (permanent collections A composition is a container that can of images) are located at the apparent include lists of items (images), and also bottom of the information space, the contain additional compositions. This current presentation in the middle, and the is a nested or tree hierarchy approach traced history of the content appears to to information organization that float above this presentation content. The intuitively allows the linear and nonlinear system distributes content in a dynamic transversal of its material. The ‘plan’ of manner, relocating elements in ways to a presentation is recorded in an XML avoid collision and spatial redundancy. schema. It is from this schema that the Here space is arranged to facilitate the imagery of the presentation is located and tasks of the user, not to mimic a sense of spatially distributed. Other aspects of the a physical space or recreate a real-world prototype design also rely on this schema, condition. specifically the intelligent agent support (see section 4), and the integration of a Editing and Tree Structures spatial sound component, summarized in the following section. Eureka is a space that is used for composing and presenting information. Extending the Interface through As an editing tool Eureka works by Spatial Sound selecting items from image libraries, which are placed into a presentation list, The following section of this paper briefly outlines the spatial sound design of the Eureka prototype. During the development of the prototype, five sound designers have been engaged to consider the implications of dimensionality within presentation environments. This research is developed from several key themes presented in the International Community for Auditory Display “Sonification Report: Status of the Field and Research Agenda” (Kramer et al. 1997), with additional reference to extant research on the role of non-speech audio for Figure 4. Eureka Max/MSP spatial sound patch (by the representation of data dimensions Pete Brundle). (Walker 2000). The research focuses on

ACADIA 2006: Synthetic Landscapes Digital Exchange 296 Digital Research: Theoretical Research the premise of creating sound fields within Prototype within a Research physical presentation environments that Project relate to the digital space presented on the screen. These sound fields are achieved This paper is concerned with advancing by using four channel audio, as opposed the concept that digital architectures to two channel audio (stereo), positioned should be adaptive, responsive, and sonic. around the audience. As part of this The Eureka prototype is part of the research sound designers have considered greater federally funded grant exploring and designed sound-scapes that explore unorthodox presentation formats for the concepts of user coordinates and situational awareness. The project team positional sound generators; nesting of spans three disciplines: architecture, compositions as a way of nesting sound sound design, and computer science. The files; and meditation and theatre as objectives of the larger research effort methods of considering presentation as a in this project are the human and social performance that has a beginning, middle issues affecting decision making processes and end. informed by spatial visualization and The most recent development of the sonification with integrated computational sound work is relating a positional sound support, especially in complex systems in real space to the virtual position of the environments such as design collaboration, presentation compositions in digital space, weather prediction, and disaster meaning that when the user is located management situations. within the information space of Eureka, As such, Eureka has evolved to they receive audio clues to whether the accommodate a series of key research surrounding compositions are in front, components beyond just the visualization to the sides, or behind. For the sound of space; one is spatial sound, and the design the Eureka prototype utilizes the other is the integration of intelligent agent software application Max/MSP (figure support for decision making processes. 4). A communication protocol has been This paper has focused on the spatial established to allow the Eureka application and interactive components of the work. to control the parameters of the Max/ The intelligent agent support utilizes the MSP environment. In the Eureka project, XML schema of the presentation plans with its unique spatial conditions, sound (as outlined in section 2.3) to make is considered as an integral extension, or editing suggestions about the presentation dimension, to the concept of an interface. content during the duration of the Here the digital architecture is also a presentation. Rather than discuss this generator of sonic environments. As with in detail here, it is argued that forms of the ability to adapt space to content, decision support offer ways of informing sound is also adapting to the parameters architecture, as external influencing of the presentation, and the manner in forces. For the Eureka prototype, space which it is arranged spatially. is adaptable. So as with the information modes, for example of history and presentation, one can consider additional modes that alter the manner of the space.

Gregory More Making Space Content Specific: Interactive Architectures for Information Presentation 297 The current prototype features an Agent potential for digital architectural spaces Response mode, which allows the user is for them to be defined specifically to retrieve a selection of images from through a relationship between content the intelligent agent system. Through a and user functionality. This is presented combination of spatially arranging assets, here by considering the temporal, sonic spatial sound and intelligent agent support, and interactive potentials of digital the Eureka Prototype suggests a digital architecture in the design of information architecture that doesn’t mimic physical environments. Form in this space can architectures. It presents, however, a digital follow the functional needs of the user. As space and an architectural experience, can be learned from interaction design, based on the concepts of space being rather than consider digital spaces as poor adaptive and interactive. replications of physical spaces, we can promote new methods of space making by Future Work relating form to informational content.

As this research is being reported the current phase of the project is developing an approach to undertake user testing of the prototype. A social scientist is part of the research team and is developing a series of questionnaires and presentation scenarios to test the software, the interface, and the success of the integration of spatial sound within presentation environments. This will focus on the actual user of the software prototype and its effectiveness as a presentation compiler. Following this testing phase the results will be used to develop the prototype further incorporating suggested improvements and ameliorating any problematic issues raised by the testing phase.

Conclusions

Within the digital domain the function of architecture is difficult to define, as the intrinsic physical world needs for structure and materiality are omitted. However there is an important relationship between the concepts of architecture and those of interfaces. Here it has been argued that a

ACADIA 2006: Synthetic Landscapes Digital Exchange 298 Digital Research: Theoretical Research Acknowledgements

This research is funded by an Australian Research Council Discovery grant, chief investigators Professor Mark Burry (SIAL) and Lin Padgham (CSIT). The research presented acknowledges the work of Research Assistants and Sound Designers: Pete Brundle, Jeremy Yuille, Robin Fox, Santha Press, Steve Law and Bruce Mowson.

References

Brusilovsky, P. (2001). Adaptive Hypermedia. User Modeling and User-Adapted Interaction. Normal, MA: Kluwer Academic Publishers. Couture, L. and H. Rashid. (2002). FLUX. New York: Phaidon Press Inc. Kramer G., B. Walker, T. Bonebright, P. Cook, J. Flowers, N. Miner, and J. Neuhoff. (1997). Sonification Report: Status of the Field and Research Agenda. ICAD. (http://www.icad.org/ websiteV2.0/References/nsf.html) Marshall, . and F. Shipman. (1995). Spatial hypertext: designing for change. Communications of the ACM 38(8). Oosterhuis, K. (2003). Hyperbodies, Toward an E-motive Architecture. Birkhäuser. Shneiderman, B. (2003). Why Not Make Interfaces Better than 3D Reality?, In Visualization Viewpoints IEEE, ed. Theresa-Marie Rhyne. (ftp://ftp. cs.umd.edu/pub/hcil/Reports- Abstracts-Bibliography/2004-29html/ 004-29.) Walker, B. (2000). Magnitude Estimation of Conceptual Data Dimensions for Use in Sonification. PhD Thesis. Rice University.

Gregory More Making Space Content Specific: Interactive Architectures for Information Presentation 299