UbiWorld: An EnvironmentIntegrating Virtual Reality,

Sup ercomputing, and Design

Terrence Disz, Michael E. Papka, and Rick Stevens

Mathematics and Computer Science Division

Argonne National Lab oratory

Argonne, IL 60439

fdisz,papka,[email protected]

Abstract ity and fault tolerance will b e critical; and program-

ming and co de maintenance will b e signi cant activi-

UbiWorld is a concept being developedbytheFu-

ties. Everything of value will b e available on mars.net

tures Laboratory group at Argonne National Labora-

from anywhere on the planet. Immersive telepresence

tory that ties together the notion of ubiquitous com-

will b e a critical capabilitytoovercome the obstacle

puting (Ubicomp) with that of using virtual reality

of distance.

for rapid prototyping. The goal is to develop an en-

These requirements push the b oundaries of comput-

vironment where one can exploreUbicomp-typecon-

ing and networking as weknowthemandaswecan

cepts without having to build real Ubicomp hardware.

imagine them in the near future. Today,we don't even

The basic notion is to extend object models in a vir-

have the to ols to exp eriment with implementations of

tual world by using distributed wide area heterogeneous

some of these ideas. Wecan,however, conduct exp er-

computing technology to provide complex networking

iments in a virtual world, creating and designing ob-

and processing capabilities to virtual reality objects.

jects out of \pure thought-stu ," to b orrow a phrase

1 Intro duction

from Frederick Bro oks. This is the concept b ehind

In the Futures Lab oratory [1] in the Mathematics

UbiWorld.

and Computer Science (MCS) Division at Argonne

2 Ubiquitous Computing

National Lab oratory (ANL), our research agenda is

driven partly by discussions of advanced computing In the b eginning, there were mainframe computers.

scenarios. We nd that by susp ending disb elief mo- Access to mainframes has historically b een character-

mentarily and by engaging in serious discussion of such

ized bymany p eople p er computer, batch op erations,

topics as o -planet infrastructure, green nomadic com- text input, and pap er output. Today,we are living in

puting, and molecular nanotechnology,we are able to the era of the p ersonal computer. Personal computer

pro ject b eyond the current set of problems and to con- use is characterized by one p erson p er computer, mul-

ceptualize innovative solutions. tithreaded interactive use, multimedia, windows, and

UbiWorld is a result of this fertile ground where mouse interface. The next wave of computing will

concepts converge and evolve. This convergence is ev- b e ubiquitous computing, characterized by many com-

ident in the o -planet infrastructure problem, or Peo- puters p er p erson and a transparentinterface, used to

ple to Mars scenario. It's a safe assumption that sup- amplify one's p owers, not replace them. Ubiquitous

p ort for p eople on Mars will b e primarily computing, computing means computers will b ecome as invisible

that the computing will b e ubiquitous and nearly in- to us to day as text is [2]. There was a time when

visible, and that \green" technology will b e used to the written word was the sole province of the exp erts,

minimize p ower consumption will b e imp ortant, as guarded and used sparingly,much as computing has

will the deployment of nanotechnology to manufac- been. Text technology has undergone a transforma-

ture devices. On Mars, computers will always outnum- tion from b eing written on clay tablets, then coarse

b er p eople. Computers will undoubtedly b e heteroge- pap er, up to to day's re ned pap er and display tech-

neous, it b eing hard to imagine a single architecture nology. Believers in ubiquitous computing see a day

deployed in devices from gloves and b o ots to landers, when the same transformation will o ccur with resp ect

ight control decks, and mining machines. Computers to computing; users will not b e any more aware of the

will need to b e transparently interconnected; reliabil- computers in their lives than weareaware to dayof

the text in which this do cument is written. We are b e found in [6].

already b eginning to see this happ en with the integra-

3 UbiWorld

tion of computers in automobiles: the driver is really

UbiWorld is an exp erimental system combining vir-

unaware of the computer and its function.

tual reality,advanced networking, and sup ercomput-

The ubiquitous computing philosophy originated at

ing to explore the implications of ubiquitous comput-

XeroxParc in 1988 [3], pioneered byMarkWeiser. He

ing. We use a virtual reality system as a design and

conceived of Ubicomp as nonintrusive, mobile, exible

evaluation environment. Instead of actually building

computing, highly integrated into the working and liv-

devices, we use VR techniques to mo del the represen-

ing environment. Ubicomp is not virtual reality (VR).

tation of devices. Weuseadvanced networking to link

VR techniques, which put p eople into arti cial worlds,

VR ob jects with computational servers to represent

primarily p ose a computing and graphics horsep ower

the b ehavior of the ob jects. Using these techniques,

problem. Ubicomp forces the computer to liveinthe

we can explore devices that are not yet p ossible to

real world with p eople. It is the integration of human

build.

factors, computer science, engineering, and so cial sci-

Today's hardware capabilities fall short of what

ence. The human factors issues go well b eyond yet an-

ubiquitous computing will need in terms of p ower con-

other human computer interface problem and will not

sumption, miniaturization, network bandwidth, and

b e solved with another windowing system. The com-

computing p ower. Until these capabilities can b e met,

puter science issues span all areas|networking, op er-

we feel that exp erimenting in virtual spaces is a pro-

ating systems, distribution of memory and pro cessing,

ductive metho d of exploring the concepts in ubiqui-

naming, resource management, etc. A general discus-

tous computing. The UbiWorld pro ject builds on ex-

sion of these issues can b e found in [4]. Engineering

isting software and pro jects in MCS. It serves to fo cus

will have to makeadvances in nanotechnology to man-

those e orts and leverages our long-standing exp ertise

ufacture the devices we can foresee. No one knows the

in software engineering and our strong development

so cial implications of everyone's having full-time com-

environment.

puting and network access.

TM

Starting with the CAVE family of displayde-

An example of an Ubicomp ob ject is intelligent cur-

vices weintegrate to ols for the construction of 3D ob-

tains that contain light and temp erature sensors and

jects into the existing library [7]. Using these ob jects

control ro om lighting and temp erature. As develop-

as mo dels, we can then imb ed new information tech-

ers of UbiWorld, we talk ab out bino culars with image

nology within them. These pro ducts might b e hand-

pro cessing software to detect and highlight ob jects in

held computers, intelligent pap er, image-pro cessing

a scene, to trackeyes and vary the fo cus, or to p er-

bino culars, desks, clothing, jewelry, cups, eyeglasses,

form other image transformations. We imagine scrap

or carp eting. The plan is to couple the virtual ob-

pap er that will b e aware of the identity of the user,

jects with remote computers via ne-grained heteroge-

will auto connect to the user's environment, and will

neous computing technology and to provide Ubicomp

automatically store and retrieve all notes made in a

b ehavior and functionality to the mo dels. The 3D ob-

p ersonal database accessed via contact with a table or

jects will b e placed in virtual ro oms, thereby creating

desk. We discuss many other examples, limited only

a shared virtual world (in a collection of CAVEs, Im-

by our imagination.

mersaDesks, or whatever) where users can exp eriment

with using the virtual devices.

Research in ubiquitous computing conducted at Xe-

roxParc followed standard exp erimental science pro- Eachobjectintheworld has b ehavior controlled

to cols [5]. Devices were conceived and prototyp es con- by a program running on the network. The b ehav-

structed and tried out on willing sub jects. There were ior could b e one that, in the real ob ject, would b e

three prototyp es of note: the XeroxParcTab, a palm- provided by a lo cal computer or by a combination

sized device; the Pad, a noteb o ok-sized device; and the of lo cal computer and network connection to remote

Liveb oard, a wall hanging device. Applications were pro cessors or databases. These \b ehavior" pro cesses

constructed to p erform e-mail, take notes, schedule are able to communicate with each other byusinga

meetings, checkweather, etc. [6]. The primary les- shared proto col (UbiWorldcomm). The ob jects also

son we take from the work, however,isthattechnol- react and are in uenced directly byinteractions with

ogy to dayisnowhere near what is required to design the virtual world and its users. If someone picks up an

and p erform exp eriments in truly ubiquitous comput- ob ject in the UbiWorld, that ob ject knows it has b een

ing. A discussion of the many compromises that Xerox picked up and then \do es the right thing," whichmay

had to make in the development of the ParcTab can mean communicating with other ob jects or computing

something or displaying some network stream. For ex- ety of bandwidths and proto cols.

ample, one's co ee cup could b e displaying live video,

UbiWorld is a classic example of using the p ower

or one could talk to an earring and make a phone call.

of virtual reality systems to prototyp e devices that

UbiWorld will let us debug Ubicomp years b efore

are imp ossible or to o exp ensive to build. To accom-

wehave the technology to build it. It will enable us to

plish our goals, wemust construct to ols that simplify

the connection of physical representations to computer change sp eci cations without changing hardware and

simulations that are prototyping the hardware. This to identify software requirements. Through UbiWorld

then gives us the ability to construct and test the use- we can explore the b oundaries of emb edded computing

and network computing. It serves as a testb ed for

fulness of hardware that is imp ossible to build with

heterogeneous computing to ols and systems suchas

to day's technology.

ne-grained networking proto cols, image comp osition

5 UbiWorld Development Environ-

mechanisms, and agentintegration.

ment

4 UbiWorld Design

The UbiWorld developmentenvironmentisacom-

A fundamental principle in the design of UbiWorld

bination of hardware and software environments. The

is the separation of an ob ject's representation from

hardware is a combination of sup ercomputers, net-

its b ehavior. As shown in Figure 1, the user inter-

works, and display devices. The software covers all

acts with the representation of an ob ject. The b e-

ma jor areas of software development from the low-

havior of the ob ject is sp eci ed indep endently of the

level network co de all the way up to high-level script-

representation. The actual computation of the b e-

ing languages that allow users to con gure the envi-

havior is p erformed on a simulation server, separate

ronment.

from the representation computing. Furthermore, as

5.1 Hardware

seen in Figure 2, the world in which the ob ject inter-

The following two subsections will address the dis-

acts is viewed as an orthogonal issue from the ob ject

playenvironments in which the UbiWorld system will

itself. The virtual world, its representation, and its

b e tested, and the various hardware limitations.

prop erties (such as light and gravity) are computed,

stored, and rendered separately from the ob jects we

cho ose to place in the world. In Figure 2, weshowa

5.1.1 DisplayEnvironments

virtual world record-and-playback engine that builds

The current displayenvironment is comprised of three

on work in progress at MCS. This engine, whichwe

virtual reality devices.

call VR Voyager, can b e thoughtofasavirtualworld

server, storing virtual worlds, serving them to clients,

 CAVE

recording VR exp eriences, and playing them backon

TM

demand.

The CAVE (CAVE Automatic Virtual Envi-

ronment) is a 10 x 10 x 9 fo ot ro om that uses In addition, weintro duce the concept of \dis-

rear-pro jected high-resolution pro jectors to pro- tributed rendering." This is an attempt to overcome

duce an immersive3Denvironment (Figure 3). the b ottlenecks intro duced into to day's VR systems by

The CAVE environment, originally develop ed by lack of graphics rendering p ower. By employing ren-

the Electronic Visualization Lab oratory (EVL) at dering engines in distributed machines, we can bring

the University of Illinois at Chicago, pro duces a much greater rendering p ower to b ear than would oth-

3D stereo e ect by displaying in alternating suc- erwise b e p ossible. Also, it gives us the ability to bring

cession the left and righteye views of the scene as the rendering closer to the source of the data gener-

rendered from the viewer's p ersp ective [7]. These ated in the simulations. Distributed rendering is a new

views are then seen by the user through a pair research area intro duced into the Futures Lab bythe

of LCD shutter glasses whose lenses op en and UbiWorld pro ject. We are studying ways to comp osite

close forty-eight times a second in synchronization the separately rendered images into a single image in

with the left- and right-eyeviews. The correct the VR theater, by tapping into the Op enGL pip eline,

viewer-centered pro jection is calculated based on for instance. Along with distributed rendering, we also

the viewer's p osition and orientation as deter- intro duce the concept of \aware networking." Aware

mined by a electromagnetic tracking system. The networking means that networking resources are not

p osition and orientation of a 3D wand are also precisely known at all times and implies an adaptive

tracked; this wand allows for navigation of and in- nature imb edded in the ob jects as they seek to join

put into the virtual world. Along with the visual networks. Ob jects can join networks by using a vari-

Figure 1: The Connections of Behavior, Representation, Simulation, and the User within the UbiWorld mo del.

feedbackoftheCAVE environment, a complete  Resolution

3D audio environmentisavailable to the user.

The current resolution of the CAVE is 1280 x 768

on eachwall. If wewere to attempt a resolu-

 ImmersaDesk

tion close to the capabilities of the human eye, we

The ImmersaDesk is based on the same rear-

would need a resolution of 4,800 x 3,800 [8]. That

pro jection technology as the CAVE (Figure 4). It

resolution is not available at this time, but webe-

is a fully interactive, 3D immersiveenvironment

lievewecanachieve that resolution on selected

that is ab out the size of a large drafting table.

areas of the screen by using a \high-resolution

The ImmersaDesk allows for one tracked viewer,

window." By using a separate pro jector and ren-

along with two to three passiveviewers.

dering engine and by driving the lo cation of the

pro jector based on gaze direction, we can provide

 In nityWall

an area of high resolution at the place where the

The In nityWall is a large rear-pro jected system

user is lo oking. This high-resolution windowis

that is created from comp ositing four standard

another of the research pro jects in the Futures

1280 x 1024 screens together to create one large

Lab that is motivated by the UbiWorld pro ject.

high-resolution screen. The In nityWall can b e

used as a large ImmersaDesk, where the images

 Tracking

are pro jected in stereo and the viewer is tracked,

The resolution of the tracking system is another

or can substitute for a large high-resolution work-

weakness of the currentenvironment. Today, our

station. The NI I/Wall was develop ed by EVL,

e ective sampling rate is approximately 100 ms,

the National Center for Sup ercomputing Appli-

with a spatial resolution of approximately 1 inch.

cations, and the University of Minnesota, with

In the future, for ne-grained manipulation of ob-

supp ort from Silicon Graphics, Inc.

jects, we exp ect a sampling rate closer to 1 ms and

a spatial resolution of 1 mm.

5.1.2 Hardware Limitations

 Haptics

Although this developmentenvironment is satisfac-

Today,ourCAVE environment has no haptic de- tory for early exp eriments, we b elieve that several

vices. The UbiWorld pro ject requires that we improvements will need to b e made in virtual tech-

bring these devices into the CAVE and learn to nologies to fully realize the b ene ts of the UbiWorld

register their actions with virtual representations. pro ject. Behavior Simulation of Objects Supercomputer

Virtual Model World World Record Rendering Reality Playback VR Control Theather Storage

Distributed

Rendering

Figure 2: Separation of UbiWorld Spaces

Force and feedback will b e required to fully pro- 5.2.2 CAVEcomm

totyp e actions of devices in our virtual world.

The CAVEcomm library is a communications library

that aids develop ers of virtual reality applications in

 Control Interfaces

the area of remote communications [9, 10]. The remote

Software in the CAVE to day is programmed as

communications can b e either virtual reality device to

an extension of currentwindows-based systems.

virtual reality device or virtual reality device to sup er-

We use menus, visual displays of pick lists, radio

computer. Using the CAVEcomm library, users regis-

buttons, dials, etc. We b elieve these interfaces

ter their virtual reality applications and/or sup ercom-

are wholly inadequate for use in UbiWorld. De-

puting simulations with a broker. The broker pro cess

vices and ob jects represented in UbiWorld maybe

handles the connections of separate entities. The bro-

activated byvoice, by absolute p osition or prox-

ker manages resources and connections but do es not

imity to other ob jects, byproximitytoorby sens-

handle data trac. Once the broker has set up the

ing features in the virtual world, or byany other

connection b etween the entities, they send the actual

means that wehaven't thoughtofyet. We need

data trac only b etween each other. CAVEcomm is

a new paradigm to allow freedom and innovation

sp eci cally designed to work with the CAVE group of

in control interfaces. This is yet another active

virtual reality devices, namely, the CAVE, the Imm-

research pro ject spawned by requirements of the

ersaDesk, the CAVE simulator, and the In nityWall.

UbiWorld pro ject.

The ideas of CAVEcomm can b e extended, however,

to any virtual reality-based system.

5.2 Software

CAVEcomm has b een used to connect virtual re-

Although more to ols are required, wehave a good

ality applications to sup ercomputing simualtions that

group of software already available to us for use in the

are running in real time. It has b een used to con-

UbiWorld pro ject.

nect twoCAVEs that are geographically separated,

allowing the users to collab orate on a joint task or

5.2.1 CAVElib

to demonstrate something in one CAVE to users in

The CAVE library [7], develop ed at EVL to work with

another.

the CAVE family of display devices, provides basic VR

functionality and viewer-centered p ersp ective trans-

5.2.3 CAVEav

forms automatically. This frees the VR programmer to

The CAVEav library brings multimedia capabilities to fo cus on the graphics of the problem at hand, not the

the CAVE. Using the library, programmers can con- viewer p ersp ective problem. The CAVE library pro-

nect to video sources on the network and texture map vides basic navigation functions, tracking of the user

the resulting video stream onto ob jects in the CAVE. and wand, and interaction with the wand buttons and

Live video streams from other CAVE, for instance, can joystick.

Figure 3: CAVE Virtual Environment (Milana Huang, EVL, 1994)

b e texture mapp ed onto avatars showing remote users. tive mo deling system to b e used in the CAVE. This

Live streams from rob ots or instruments are used to system will allow users to create ob jects in the native

provide an immersive telepresence capability. Prere- environment and will imp ort/exp ort VRML-based ob-

corded streams can b e used to provide instruction or jects that can b e used in or taken from other VR en-

backgrounds. vironments. Users will b e able to a ect the trans-

formations (rotate, scale, translate) of the ob ject as

A prototyp e system demonstrating these features

well as its material prop erties (ambience, di usivity,

has b een develop ed at ANL. A small rob ot mounted

shininess, sp ecularity, etc.). The abilitytoeditma-

with a variety of video and audio comp onents is con-

terials and lighting is signi cantgiven the fact that

nected to the CAVE; from within the CAVE, a user

many ob jects lo ok very di erentintheCAVE due to

can navigate the rob ot and interact with the environ-

the physical comp onents of the CAVE, (i.e., pro jec-

ment within which the rob ot lives [11]. This system is

tors and screens). This CAVE mo deling to ol will also

b eing used to test the requirements and to exp ose the

have the ability to edit the ob ject's shap e, not just its

dicult problems within a to olkit of this nature.

extraneous prop erties. Users will b e able to edit the

p olygons of the ob ject; adding, deleting, and mov-

5.2.4 CAVE-VRML Mo deler

ing vertices will give users the ability to rede ne and

combine existing shap es or create new ob jects from

Three-dimensional virtual computer environments

scratch. The created worlds and environments will b e

such as the CAVE should have the capacitytobea

exp ortable to VRML.

working developmentenvironment, not just an inter-

active displayenvironment. One asp ect of a develop-

mentenvironment is 3D mo deling. Currently,no3D

5.2.5 VR Voyager

mo delers work well in conjunction with the CAVE. Of-

The Voyager multimedia recording and playback sys- ten, in going from the mo deler to the CAVE, \what

tem has b een under development in the Futures Lab you see is what you get" is not always true. Frequently,

for the past twoyears [12]. It uses an IBM SP2 ob jects mo deled on a workstation lo ok quite di erent

for multistream, multimedia record and playbackof in the CAVE, particulary with resp ect to the ob ject's

network-based sources. We prop ose to use the Voy- scale, color, and lighting. One of the new pro jects

ager system as the basis for a new virtual world server, in the Futures Lab is the developmentofaninterac-

Figure 4: ImmersaDesk Virtual Environment (Jason Leigh, EVL, 1995)

6.1.1 Innovative Representational Design providing record and playback of VR exp eriences.

Current thinking, as evidenced by the ParcTab, is to o

restricted by technology limitations to create truly in-

6 UbiWorld Goals and Requirements

novative design. Wewould like to b e able to takethe

Our goal is to test UbiWorld ob jects and worlds in

b est from advanced industrial design and apply it to

a task-based manner, under di erent scenarios. We

the design of future Ubicomp devices in UbiWorld.

will provide several di erentenvironments, for exam-

The typ e of advanced design philosophywehavein

ple, a home, oce, airplane, hotel, car, eld, restau-

mind is emb o died in publications suchasArbitare

rant, lab oratory, and shop. Each of these environ-

magazine and the b o ok The Art Factory, Design in

ments must b e rendered with high-resolution textures,

Italy Towards the Third Mil lenium [13]. In this latter

complex spaces and lighting, and varying degress of

text, the author analyzes the birth of virtual industry,

scene complexity. Each of these mo dels is di erent,

links b etween the fashion system and the media sys-

but the requirements of ubiquitous computing span all

tem, the rediscovery of art and craft traditions, and

of these environments. In each di erentenvironment,

renewed ecological awareness of materials in terms of

Ubicomp ob jects must b ehave appropriately. Scenario

contemp orary and New Wave design.

spaces will b e used for exp erimenting with device func-

tionality and interaction by requiring task-based ex-

plorations. For instance, users may b e required to or-

6.1.2 Novel Information Technology Comp o-

der a meal, prepare a talk, negotiate a contract, drive

nents

to an unknown destination, or buy gro ceries.

Using sup ercomputer and external multimedia servers

and resources, designers in the UbiWorld will b e able

6.1 Virtual Device Requirements

to sp ecify and simulate b ehavior asso ciated with ad-

To accomplish this goal, we b elieve that computing vanced CPU capabilities, imaging technology, sensors,

ob jects in the UbiWorld must p ossess or makeuseof actuators, multimedia comp onents, communications

at least the following features. capabilities, etc.

Figure 5: In nityWall Virtual Environment (Jason Leigh, EVL, 1995)

6.1.3 Transparent Networking Agents are b eing constructed that passively view the

b ehaviors of users and learn ab out their interests. Us-

Transparent, or \aware," networking is assumed to b e

ing metho ds similiar to these techniques, we hop e to

a fundamental capability in the UbiWorld. The net-

construct within the UbiWorld a framework that al-

work in this case is transparent to the user. UbiWorld

lows the Ubicomp devices to b e reactive and proactive.

devices automatically connect to whatever other de-

vice is appropriate, using mutually acceptable band-

6.1.6 Integration Functions

widths and proto cols.

In the UbiWorld, we b elieve the network really is the

computer. As devices near other devices or ob jects

6.1.4 Device/Space Awareness

in the space, they should b e able to interrogate the

Devices in the UbiWorld must exhibit awareness of

devices and p erform acts of sp ontaneous integration

other devices and of the space in which they are op-

if it is of b ene t to do so. Bene t in this case could

erating. If a user carries a device to a di erent space,

b e de ned as access to greater bandwidth, computing

the device must automatically b e aware of the new

power or advanced capabilities. The intelligent scrap

space context and take appropriate action.

pap er idea fully emb o dies this idea, since the scrap

pap er integrates with desktops for greater bandwidth

or with imaging devices to capture multimedia infor-

6.1.5 Reactive/Proactive/Proxy Behavior

mation.

UbiWorld devices should, of course, react prop erly

6.2 UbiWorld Design Problems

to users requests, but b eyond that, they should b e

We see four critical design problems in the Ubi-

proactive as necessary. An example is the loading of a

World pro ject:

new context when the user enters a new space or the

 Ob ject shap e, form, and representation

proactivedownloading of news or information known

to b e of interest to the user. Based on current research

 Computing and communications internals

within the arti cial intelligence communityonagent

based systems, one can already see a trend emerging.  Functional b ehavior

 Integration with and awareness of environment this phenomenon in multiple-tracked, network-

connected VR systems and o ered data and re-

searchissuestobeinvestigated to mitigate the

In the UbiWorld pro ject, we b elieve it is imp ortant

latencies in the system [15].

that these design problems b e separated. Wewantto

feel free to exp eriment with the form and shap e of an

 Ob ject Representation

ob ject indep endent of its other attributes. Since the

ob ject is virtual, we can exp eriment with communica-

The representation of ob jects within the Ubi-

tions and computing internals without b eing encum-

World mo del must b e exible, so that they can

b ered byphysical packaging or p ower problems. Func-

be changed easily, without a ecting the underly-

tional b ehavior is simulated via the computational

ing simulation of the ob jects. It is imp ortantto

servers and can b e varied at will, indep endentofthe

allowfora variety of di erent representations to

packaging or other factors. Finally,integration with

b e attached to the simulation, to allow users to

and awareness of the environment are accomplished

exp eriment with di erentinterfaces. This comp o-

via simulated sensors and connection interfaces. For

nent is one of the true strengths of the connection

instance, when an ob ject such as a piece of intelligent

of Ubicomp devices to VR, since exp ensive pro-

pap er is placed on a table, we exp ect it to p erform

totyp es do not need to b e built to try out a new

the ab ove mentioned \sp ontaneous integration" with

device.

the table. Through its sensors, it must b ecome aware

of the table and its capabilities. By using an aware

 Behavior Sp eci cation

networking approach, the intelligent pap er negotiates

with the table to establish a connection, therebyinte-

grating its functionality with that of the table.

An op en question is how b est to sp ecify the b e-

For each of these design problems, appropriate to ols havior of an ob ject. For our purp oses, there are

are essential. Where existing to ols are inadequate, it three categories of b ehavior:

will b e necessary to build or invent more robust to ols.

{ Representation Dynamics

6.3 Technical Challenges

{ Functional Mechanics

The scop e of the UbiWorld pro ject pushes the

b ounds of current VR and networking technology and

{ Computational and Communication

gives rise to a host of technical challenges. The follow-

It is not likely that the same to ols will b e appro-

ing list is not meant to b e comprehensivebutserves

priate for each of these tasks. A suite of to ols

to enumerate those problems we feel are most imp or-

will need to b e develop ed to enable sp eci cation

tant to fo cus on now in the implementation of the

of each of these b ehaviors and their interactions.

UbiWorld.

 Ob ject Binding and Brokering

 Scalability

Resource brokering is the use of computationally

Adding ob jects into a UbiWorld environment

enhanced entities to aid in the requesting, allo-

stresses proto cols, bandwidths, computing, and

cation, and management of remote capabilities.

rendering p ower. Imp ortant research issues re-

Much in the same way that data hiding is used in

garding scalability of these comp onents and their

ob ject-oriented programming to make the inter-

interactions must b e investigated.

faces easier to use and understand, wide-area re-

source brokering can b e used to simplify the user

 Latency

interaction with a large-complex set of computa-

tional and collab orative resources. One approach

Weknow that there are limits on the user- can b e attacked by expanding on what wehave

p erceived latencies in an interactive system. The learned from traditional system managementsoft-

latency of the total system (including latency ware and currentwork in cluster management.

from the graphics system, the tracking systems, Through the use of resource brokering, the level

the networks and the computation engines) can- of complexity needed to con gure and control a

not exceed 100 ms - 1000 ms, dep ending on the large virtual world is reduced to a manageable

user's exp erience level [14]. Taylor et al. studied state for the user.

 Pro cess Mapping and Execution Control  Security

The management and control of pro cesses b ecome

Fine-grained, scalable authentication, authoriza-

imp ortant in the UbiWorld mo del. The ability

tion, and accounting mechanisms will b e required

to map a new pro cess into an existing computa-

to control access to information and computa-

tional framework is an essential comp onentofthe

tional resources by b oth users and computational

UbiWorld mo del. As a new user enters the Ubi-

entities. Complex issues include secure communi-

World or a new Ubicomp device is intro duced, the

cation of high-bandwidth multimedia data, access

pro cess controlling the simulation will havetobe

control of dynamically created entities, multi-

seamlessly integrated. At the computational level

entityinteractions, security of archived data,

that will require mapping the new pro cess onto a

ob ject-level security in virtual environments, and

computational resource and then controling the

delegation of authoritybetween users and asso ci-

execution from startup to termination.

ated computational entities.

 Evaluation and Measurement

7 Tools

As in anyscienti c endeavor, we desire to mea- The underlying infrastructure for the construction

of UbiWorld requires the combination of a wide area

sure, evaluate, and rep ort on our work in a rigor-

ous manner. To ols are required to enable instru- of computer science disciplines. Techniques and to ols

mentation of all the computational and commu- need to b e b orrowed from the elds of computer graph-

nications pro cesses and their relationships. Eval- ics, arti cial intelligence, and systems to name but a

uation and rep orting to ols are essential to reduce few.

and analyze the data generated by instrumented

Wehaveidenti ed a set of existing to ols for use

co des.

in implementing the UbiWorld concept. These to ols

will not b e sucient in the long term, but most rep-

 Distribution

resent a suitable b eginning for work on the UbiWorld

requirements.

Distribution covers a whole set of problems re-

7.1 Representation

lated to using networked resources: the map-

ping of pro cesses to pro cessors; the distribution of

The physical representation of the ob jects within

databases over networked resources; the issues of

UbiWorld, such as intelligent pap er or image-

redundancy, failure recovery, and the other prob-

pro cessing glasses, needs to mo deled in suchaway

lems usually asso ciated with distributed comput-

that the ob jects can b e easily changed and mo di ed.

ing | all are present in the UbiWorld pro ject.

Currently one can use a wide variety of desktop mo d-

eling and CAD packages to physically design the ob-

 Naming and Identi cation

jects. These packages are not sucienttodevelop

UbiWorld ob jects to nal state. While it is imp or-

The currentInternet provides mechanisms for

tant to separate the representation from the function,

naming computers and Web pages. Future envi-

the mo delers require the ability to provide ho oks or

ronments will require the abilitytonameamuch

connections to a b ehavior to olkit. We b elievethatthe

TM

wider variety of ob jects with varying degrees of

Op en Inventor and VMRL mo deling formats lend

p ersistence and scop e [16]. Mechanisms are also

themselves most to this goal.

required for lo cating ob jects based on di erent

7.2 Behavior Sp eci cation

criteria. Prop osals for Universal Resource Names

are a step in this direction but are designed for For the representational dynamics of an ob ject, the

long-lived ob jects. We b elieve that a new class of to ols such as VRML or Op enInventor come to mind.

naming mechanism will b e needed that can refer For the functional (or mechanical) b ehavior, pro cedu-

to much shorter-lived ob jects that would b e the ral systems suchasJava or C++ are available to day,

topic of communication b etween user agents and but they are to o lowlevel to use in the long term.

simulation spaces. Brokering and name trans- For the sp eci cation of the computational and com-

lations mechanisms are also needed. These will munications b ehavior, we can also use JavaorC++,

need to b e high p erformance and scalable and to or other still to o low-level systems suchasnperlor

incorp orate ho oks for security and access rights. Nexus.

7.3 Ob ject Binding/Brokering norm, not the exception. In this space, we can more

clearly see the technical challenges that await us, and

Binding of ob jects to resources can currently b e

we can pro ceed to invent solutions well ahead of tech-

accomplished by hard-wiring the connection or soft-

nological progress that can implement these solutions.

wiring the connection through the use of a brokering

The scop e of UbiWorld is very broad and invites

system such as CORBA or the LabSpace broker.

researchonmanyfronts. Wehave identi ed some of

7.4 Pro cess Mapping and Execution Con-

these issues, suchasnetwork latency, scalability, ob-

trol

ject representation and sp eci cation, pro cess and data

The design of UbiWorld calls for ob ject b ehavior to

mapping, ob ject brokering and binding, security, and

b e computed on separate computational servers. The

measurement. Each of these issues is deserving of a

mapping of many ob ject pro cesses to computational

fo cused research e ort in the futures-oriented ubiqui-

servers and control of the execution is a problem we

tous computing scenario, and weinvite the heteroge-

have b een examining for some time. For SC'95, a team

neous computing community to engage in discussions

at ANL develop ed custom software for scheduling and

and research in this rich problem space.

mapping pro cesses to network-based pro cessors. The

system deployed at SC'95 was the I-WAYPointof

Acknowledgments

Presence (I-POP) machine. The I-POP machine was

WethankIanFoster, Ivan Judson, Bill Nickless,

sp eci cally set up to manage security issues and was

Bob Olson, and Matt Szymanski for all their help and

con gured sp eci cally for the I-WAY[17]. It allowed

input. This work was supp orted by the Mathemat-

use of pro cess mapping and control software suchas

ical, Information, and Computational Sciences Divi-

Nexus and MPI. Subsequent to the I-WAY pro ject,

sion subprogram of the Oce of Computational and

a team at ANL has b een integrating the Adaptive

Technology Research, U.S. Department of Energy, un-

Communication Environment(ACE) [18] system into

der Contract W-31-109-Eng-38.

a parallel message-passing to olkit.

References

Each of these solutions has shortcomings, and we

[1] Terrence L. Disz, RemyEvard, Mark W. Hender-

fully exp ect that UbiWorld requirements will force the

son, William Nickless, Rob ert Olson, Michael E.

design or evolution of a new, more sophisticated sys-

Papka, and RickStevens, \Designing the future

tem for pro cess mapping and control.

of collab orative science: Argonne's futures lab o-

7.5 Evaluation and Measurement

ratory," IEEE Paral lel and DistributedTechnol-

It is imp ortant to b e able to measure and quantify

ogy Systems and Applications,vol. 3, no. 2, pp.

the progress that is b eing made. Since UbiWorld re-

14{21, Summer 1995.

quires the combination of such a wide varietyofdi er-

ent asp ects of the eld of computing, we are required

[2] Mark Weiser, \The computer for the 21st cen-

to connect various areas that are only now b eginning

tury," Scienti c American,vol. 265, no. 3, pp.

to b e tested. The connection of sup ercomputers to im-

94{104, September 1991.

mersive virtual reality display devices is just one area.

[3] Mark Weiser, \The world is not a desktop," In-

For the rst time, issues such as latency, for example,

teractions,vol. 1, no. 1, pp. 7{8, January 1994.

need to b e lo oked at outside their normal meanings

to sup ercomputer users and to graphics programmers.

[4] Ian Foster, Michael E. Papka, and Rick Stevens,

Therefore, as UbiWorld is constructed, measurements

\Tools for distributed collab orative environ-

and evaluation need to b e done. Currently,we are us-

ments: A research agenda," in Proceedings of

ing PABLO from the University of Illinois for instru-

the Fifth IEEE International Symposium on High

menting VR and simulation programs. We use MPI

Performance Distributed Compututing. IEEE,

logging and the Upshot display system for tracing and

1996, pp. 23{29, IEEE Computer So cietyPress.

evaluating MPI programs.

[5] Mark Weiser, \Some computer science issues in

8 Conclusion

ubiquitous computing," Communications of the

UbiWorld is a pro ject that pushes b eyond the

ACM,vol. 36, no. 7, pp. 74{83, July 1993.

b ounds of current technology and forces us to think

of heterogeneous computing in new terms. Rather [6] RoyWant, Bill N. Schilit, Norman I. Adams, Rich

than making incremental changes in existing technol- Gold, Karin Petersen, David Goldb erg, John R.

ogy, UbiWorld gives us a chance to leapfrog into a new Ellis, and Mark Weiser, \The parctab ubiquitous

problem space where heterogeneous computing is the computing exp eriment," Tech. Rep. CSL-95-1,

[16] John Kunze, \Functional recommendations for XeroxPARC, XeroxPalo Alto ResearchCenter,

Internet resource lo cators," Febuary 1995, Net- March 1995.

work Working Group, RFC1738.

[7] C. Cruz-Neira, D. J. Sandin, and T. A. De-

Fanti, \Surround-screen pro jection-based vir-

[17] Ian Foster, Jonathan Geisler, William Nickless,

tual reality: The design and implementationof

Warren Smith, and SteveTuecke, \Software in-

the CAVE," in Computer Graphics (Proceedings

frastructure for the i-way high-p erformance dis-

of SIGGRAPH '93).ACM SIGGRAPH, August

tributed computing exp eriment," in 5th IEEE

Symposium on High Performance Distributed 1993, pp. 135{142, Addison Wesley.

Computing. IEEE, 1996.

[8] M. McKenna and David Zeltzer, \Three dimen-

sional visual display systems for virtual environ-

[18] Douglas C. Schmidt, \The adaptive communica-

ments," Presence,vol. 1, no. 4, pp. 421{458, 1992.

tion environment an ob ject-oriented network pro-

gramming to olkit for developing communication

[9] T. L. Disz, M. E. Papka, M. Pellegrino, and

software," in Sun Users Group Conference,De-

R. Stevens, \Sharing visualization exp eriences

cemb er 1993.

among remote virtual environments," in Inter-

national Workshop on High PerformanceCom-

puting for Computer Graphics and Visualization.

1995, pp. 217{237, Springer-Verlag.

[10] Terrence L. Disz, Michael E. Papka, Michael Pel-

legrino, and Matthew Szymanski, CAVEcomm

Users Manual v1.0, Mathematics and Computer

Science Division, Argonne National Lab oratory,

1.0 edition, August 1996, ANL/MCS-TM-218.

[11] Ivan Judson and RickStevens, \Architecture of

an immersive virtual reality telepresence system,"

Private Communication, Argonne National Lab-

oratory, April 1997.

[12] RickStevens, Terrence Disz, Rob ert Olson,

Michael E. Papka, and RemyEvard, \Voyager:

A next generation hyp ermedia server to supp ort

the construction of virtual organizastions," 1995,

Grant Prop osal, Submitted to DOE 1995.

[13] Stefano Casciani, The Art Factory Disgn in Italy

Towards the Third Mil lennium, Springer-Verlag,

1996.

[14] A. Liu, G. Tharp, L. French, S. Lai, and L. Stark,

\Some of what one needs to know ab out using

head-monunted displays to imp orve teleop erator

p erformance," IEEE Transactions on robotics

and Automation,vol. 9, no. 5, pp. 638{648, 1993.

[15] Valerie E. Taylor, Milana Huang, Thomas Can-

eld, Rick Stevens, Daniel Reed, and Stephen

Lamm, \Performance mo deling of interactive,

immersive virtual envrionments for nite element

simulations," The International Journal of Su-

percomputing Applications and High Performance

Computing,vol. 10, no. 2/3, pp. 141{151, Novem- b er 1996.