Cutting Planes and Beyond

Michael Clifton and Alex Pang

Computer Information Sciences Board

University of California Santa Cruz CA

Abstract

We present extensions to the traditional cutting plane that b ecome practical with the availability of virtual

reality devices These extensions take advantage of the intuitive ease of use asso ciated with the cutting

metaphor Using their hands as the cutting to ol users interact directly with the data to generate arbitrarily

oriented planar surfaces linear surfaces and curved surfaces We nd that this typ e of interaction is

particularly suited for exploratory scientic visualization where features need to b e quickly identied and

precise p ositioning is not required We also do cument our investigation on other intuitive metaphors for use

with scientic visualization

Keywords curves cutting planes exploratory visualization scientic visualization direct manipulation

Intro duction

As a visualization to ol virtual reality has shown promise in giving users a less inhibited metho d of observing

and interacting with data By placing users in the same environment as the data a b etter sense of shap e

motion and spatial relationship can b e conveyed Instead of a static view users are able to determine what

they see by navigating the data environment just as they would in a real world environment However in

the real world we are not just passive observers When examining and learning ab out a new ob ject we

have the ability to manipulate b oth the ob ject itself and our surroundings In some cases the ability to

manipulate ob jects and surroundings may b e more imp ortant than the ability to simply move around in a

virtual world

This pap er describ es a direct manipulation system that allows the user to intuitively slice dice and carve

the data set under investigation It is well suited for exploratory spur of the moment visualizations where

the fo cus is on the data rather than the visualization system The ease of use of our system comes from

identifying intuitive metaphors that can b e conveniently mapp ed to virtual reality devices The devices used

in this work include an sensor Cyb erglove to identify hand p ostures two D trackers from Ascension

for tracking hand p osition and orientation and stereo glasses from StereoGraphics to aid navigation We

also considered immersive technologies such as headmounted displays to provide a natural way of lo oking

around the data set One of the concerns of using headmounted displays is the sensitivity of the human

users to design limitations such as p o or display resolution and lag time in head tracking The latter is

particularly troubling as it may cause simulator sickness Aside from reducing the display resolution further

one can also attempt to reduce the rendering eort eg by reducing scene complexity etc With current

technology we feel that the overall sacrice to image quality necessary to bring lag times in headmounted

displays to satisfactory levels is not acceptable for scientic visualization applications Therefore we take

the approach that eective visualizations can b e achieved without full immersion into the world of the data

b eing examined Instead by allowing the user to directly manipulate the data the user is no longer forced

into the role of an observer and can play a more active role

In order for the direct manipulation interaction to b e as natural and intuitive as p ossible we identify

metaphors and tasks that p eople use when interacting with and visualizing their data These metaphors

can b e identied by listening and observing user interactions during a visualization session One might

hear conversations ab out painting an isosurface a particular color highlighting some features in the data

or cutting o some p ortions of the data to exp ose internal structures It is not surprising that these map

to common exp eriences and D interactions This pap er fo cuses on the cutting metaphor and how it is

extended to supp ort linear and curve surface crosssections The cutting metaphor is mapp ed to to ols that

b ehave like knives or blades in the physical world Armed with this metaphor the user already has an idea

of how to manipulate these to ols when they rst start to use this system When cutting a real world ob ject

the user is exp osing some interior prop erties of that ob ject most notably represented by the color of the

internal structures Similarly when cutting with this visualization system the user exp oses the data values

along the surface of the cut To make these values meaningful they are mapp ed to colors on the cutting

surface

While the metaphors and techniques that we rep ort here can b e extended to other forms of data gridding

and organization we rst fo cus on simple threedimensional regular grids of scalar values This category

encompasses a huge variety of data including D medical data from computer aided tomography scans and

magnetic resonance imaging and D physical data sets such as pressure density and temp erature These

data also have the desirable characteristic of having a clear spatial layout Note that the cutting to ols simply

sp ecify a p ortion of the data that may b e of p ossible interest While it is mostly used for scalar data it

may also b e used for vector data In such case the selected vector data can b e visualized as hedgehog plots

streamlines etc on the cut surface

Because direct manipulation systems are b est suited for spur of the moment or exploratory typ e inves

tigations the user should consider a two pass approach if precise or rep eatable visualizations are necessary

after a p erio d of exploratory visualization to identify regions of interest the user may then switch to a

slower but more precise display There are two main reasons First is the lack of precision and rep eatability

in a users handeye co ordination in sp ecifying a cutting surface While the direct manipulation cutting to ols

are easier to use a numerically sp ecied cut surface will always b e more accurate Second is the necessity

to tradeo quality for sp eed in order to maintain reasonable interaction rates during direct manipulation

exercises With an interface meant to b e transparent to the user visualization delays are unacceptable and

the steps taken to eliminate them may lead to blurriness or blo ckiness in the result

The rest of the pap er is organized into ve sections The related work section gives a brief review of rele

vant humancomputer interaction studies in supp ort of direct manipulation and other related visualization

systems We then present the system architecture This is followed by details and results of the dierent

cutting and visualization to ols Successes and shortcomings of the system are presented in the evaluation

section Finally we p oint out areas for future work and summarize the results of this work

Related Work

Cognitive Approaches to HumanComputer Interaction

In order to make visualization systems easy to use the humancomputer interface must b e intuitive For

example when manipulating ob jects in the real world humans normally do not think in terms of vectors

and plane equations so the system should adapt to the user not the other way around Many researchers

have asked how the user thinks and how to adapt systems to b oth take advantage of thought pro cesses and

to improve user understanding in complex tasks

Sellen Kurtenbach and Buxton describ e an exp eriment in which they try to help users avoid mo de errors

Mo de errors are mistakes that o ccur when the user tries to give the computer a command when in

fact the computer is in a mo de that do es not accept that command An example would b e trying to typ e a

word in the UNIX text editor vi while the program is in navigation mo de instead of text entry mo de Their

exp eriments involved providing users with either a visual screen color or a kinesthetic fo ot p edal p osition

feedback to indicate the mo de of the editor Both forms of feedback resulted in users making fewer mo de

errors However the kinesthetic feedback setup resulted in signicantly fewer errors than the visual feedback

system The conclusion of the researchers was that the visual feedback since it was passively generated by

the system was to o easily overlo oked by the user The kinesthetic feedback on the other hand since it was

actively generated by fo ot pressure from the user was not so easily overlo oked

The results of this exp eriment suggest that a direct manipulation system may not only lead to fewer errors

by the user but also a greater understanding of what op erations are currently b eing p erformed Because

the users hand is physically active during to ol manipulation the user is more likely to maintain an active

correct notion of what to ol he is using where it is and what state it is in

Donald Norman describ es artifacts as to ols or devices that enhance human ability and p erformance

In the physical world these artifacts are actual to ols such as pulleys to make p eople stronger or cars to make

them faster Using the pulley as an example the p erson is emp owered to lift heavier ob jects However from

the p oint of view of that p erson the task has b een changed to pulling a rop e instead of lifting the ob ject

Another typ e of artifact Norman discusses is the cognitive artifact Cognitive artifacts are meant to

display or represent information in such a way as to improve human cognitive p erformance Scientic

visualization techniques in general are cognitive artifacts since they take data and give it a representation

that hop efully leads to b etter understanding than the raw data by itself In our system the cutting to ols

are further artifacts Instead of presenting the user with a pregenerated visualization to study the nature

of the task has b een changed to allow the user to generate the visualization interactively in areas of interest

in many cases leading to a more complete understanding of the data itself

Related Visualization Systems

There are numerous visualization systems to day Some of these are application sp ecic while others are more

general Most of them use traditional keyb oard and D mouse input while others use more exotic hardware

Our system b orrows some of the b est ideas and techniques from these systems and improve up on them In

particular we nd that cutting planes are almost universally available in visualization systems However

their ease of use and applicability have b een limited by the interface imp osed up on the users Included here

is a nonexhaustive but representative review of systems that provide cutting plane techniques extension to

curved cutting planes and the use of virtual reality devices for visualization applications

General visualization systems such as AVS and Data Explorer as well as more application domain sp ecic

systems such as VISD and OVIRT all provide cutting planes with options for pseudocoloring or

contouring the crosssections Some of them limit the cutting planes to the three orthogonal directions while

others allow arbitrary planes These planes are usually sp ecied by entering numeric co ordinate values or

by adjusting graphical sliders An alternative way of sp ecifying arbitrary planes is through a D spray can

widget Here the plane is normal to a line originating from the nozzle of the spray can widget As the

spray can is moved and rotated a new cut plane is generated

Rho des et al and Udupa and Odhner describ e visualization systems targeted for visualization

of volumetric medical data Both of these allow sp ecications of curved crosssections In the former the

user sp ecies p oints with a trackball along the desired curve from a sagittal side crosssectional view The

p oints are then tted with a p olynomial curve and a coronal front view through this curve is generated

In the latter case the user rst sp ecies a twodimensional curve in a xed D view of the data and then

sp ecies the depth of this curve resulting in a curved cut While these may b e the only way to sp ecify curve

crosssections with D input devices the availability of D input devices can make this task easier and more

natural

A pap er by Ney and Fishman details the features of another visualization system designed for use on

medical data Instead of using cutting planes this system uses cutting volumes to identify pieces of

a larger data set The user sp ecies the cut volume by drawing out shap es such as b oxes ellipsoids and

extruded p olygons with a mouse The extracted region is then volume rendered A similar approach is

rep orted by Cignoni Montani and Scopigno with their MagicSpheres Here a spherical volume of

interest is sp ecied Then one or more lters asso ciated with dierent visualization techniques are applied

to the volume of interest Another variation is to use the ashlight metaphor to shine on the volume of

interest Dierent shap es can b e asso ciated with the ashlight to pro duce conical pyramidal etc volumes

of interest

Another system of related interest to this thesis is NASAs Virtual Wind Tunnel develop ed by Bryson

and Levit This system is a ow eld visualization to ol that makes use of some sp ecialized input and

output hardware to immerse the user into the system First of all the Virtual Wind Tunnel uses an immersive

display so that the user gets the impression of b eing inside the wind tunnel By simply walking around and

turning his head the user can alter the view of the data set Of more interest is how the Virtual Wind

Tunnel uses a Data Glove to manipulate to ols for generating visualization The main to ol is a rake where

seed p oints along it emit particles that are then advected into space thereby showing the features of the ow

eld With the Data Glove the user can reach out to grab and manipulate the rake This ability is similar

to our system However there is an imp ortant distinction In the Virtual Wind Tunnel the user moves

to ols around with his hand In our system the hand of the user is the to ol itself Instead of grabbing and

rotating or translating the users hand is more intimately linked to the to ol

Our work also diers from other work that applies virtual reality interfaces to visualization notably the

use of D widgets such as a oating rectangle to represent a cutting plane Here the users sp ecify the

cut by grabbing anchor p oints eg vertices edges plane center and moving the rectangle around Our

work take this a step further and insist that the users hand b e the to ol That is dierent p ostures and

gestures of the users hand would activate dierent to ols It is imp ortant that these p ostures corresp ond to

the physical ob jects or metaphors in use

System Architecture and Comp onents

The system architecture is designed to b e extensible to other virtual reality devices other than those used

in this work It is also designed to supp ort other metaphors b eyond those rep orted here An overview of

the system architecture is depicted in gure It is similar to the application indep endent device inter

face describ ed in The imp ortant feature is that system comp onents such as the device interfaces and

visualization op erations are abstracted from the application There are four ma jor comp onents in our sys

tem They are the Glove and Tracker libraries the Posture Recognition comp onent and the Visualization

comp onent

Application

Posture Recognition

Glove Library Visualization Op erations Tracker Library

IRIS GL

Figure System Architecture

Tracker Library

The trackers and the glove devices communicate with the host computer through separate serial p orts

Byte instructions are dened by the manufacturers and are sp ecic to their own devices As such the

immediate layer of software is device dep endent but shields software layers ab ove it from device sp ecic

co des Other than this immediate software layer the tracker library provides necessary functionality such

as InitTracker CloseTracker and ReadTracker as well as convenience functions such as SetTrackerSp eed

MovePointWithTracker RotPointWithTracker and ResetTracker These functions provide a high level

interface for an application program to communicate with the D trackers

Glove Library

The glove used to measure the users hand p osture is the Cyb erglove It is a thin spandex glove with ex

sensors to measure joint angles The one we used is a lefthanded glove with sensors two for the rst

two joints of each nger four for the spread b etween ngers two on the wrist and two for palm exing

Although the end joints of the ngers do not have sensors in the glove the glove library simulates these

joints by setting their angles equal to those of the previous joint on each nger This approximation seem

reasonable for relaxed hand p ostures The glove also has a small switch attached to the wrist that can b e

tested for a simple ono state

The glove library has a function InitGlove similar to the tracker library that checks for the presence of

a Cyb erglove unit attached to a serial p ort The application may continue without a glove but then should

not dep end on any glove functionality

The main routine provided by the glove library ReadGlove reads the joint angles of the glove This

routine accomplishes the conversion of analog sensor signals to actual joint angles by multiplying the values

by a scaling factor and adding an oset To b e accurate sp ecic oset and scale values must b e assigned

to each of the sensors for each p erson using the device This b ecomes awkward and timeconsuming

esp ecially as the sensors might rep ort dierent values dep ending on conditions such as whether the unit

is warmed up and if the glove was pulled on tighter than last time resulting in the need for a lengthy

calibration pro cess Instead we designed a greatly simplied calibration pro cess into the glove library This

routine CalibrateGlove automatically generates the oset and scale values for each sensor To calibrate the

glove the user places his hand palm down at on the table with the wrist and ngers straight and not

spread as in gure This represents the conguration in which all joint angles should b e zero While in

this p osition the user presses a C on the keyb oard to invoke the CalibrateGlove routine

Figure Calibration Posture

As the Cyb erglove warms up and the glove stretches and shifts the calibration typically loses accuracy

but this simple calibration pro cedure is easily rep eated whenever necessary typically after every one or two

minutes of use

Most applications using the glove will want to display the users virtual hand on the screen so a routine

DrawGlove is provided for drawing a simple hand using IRIS GL graphics routines The hand mo del drawn

is a relatively simple b oxy representation but all of the ngers and joint motions are clearly observable

The user generally has a go o d sense of the joint angles of their ngers but the display of the hand is useful

for identifying the lo cation of the virtual hand and for reassuring the user that the virtual hand indeed

corresp onds to the his actual hand movements

Because of their common use the glove library also provides two other functions IndexTip and Palm

Center IndexTip returns the p osition of the tip of the index nger based on the wrist and index nger

sensors PalmCenter returns the p osition of the center of the palm based on the wrist sensors Any p oint of

the hand can b e calculated but these two are widely used to warrant inclusion in the glove library

Posture Recognition

The p osture recognition system interprets what the hand is doing based on the joint angles returned by the

glove library At rst thought a p osture recognition scheme would want to compare all joint angles in

the glove to a set of known p ostures and nd a match However the joint angles for a given user will never

exactly match a predened p osture The joint angles of a single user will not even b e the same for the same

p osture at dierent times As another complication calibration dierences make it imp ossible to get exact

matches Thus any p osture recognition scheme must account for the fact that the same p osture will never

app ear the same

One p ossibility for handling these dierences might b e a neural network With prop er training a neural

network might successfully learn how to consistently dierentiate b etween p ostures despite the variations

However this was not the approach taken for this pro ject Instead a simpler mechanism using thresholds

was employed with the idea of p ossibly moving to a neural network or some other sophisticated approach

later on However the simple approach proved to work very well

The feature vector to uniquely identify a particular p osture do es not require all glove sensors For

example the wrist sensors do es not play a role in most p ostures and can b e safely ignored In addition to the

wrists the thumb p oses some ambiguities when matching p ostures For example consider the two p ointing

p ostures illustrated in gure The thumb p osition is dierent If asked to p oint two dierent p eople might

use these dierent p ostures

Figure Two of Many Possible Pointing Postures

In such a case the thumb would b e dierent enough to declare one of these p oses incorrect In many

p ostures it turns out that the thumb is not playing an imp ortant role Thus when a program sp ecies a

table of meaningful p ostures it can tell the p osture recognizer to either use or ignore the thumb for each of

the p ostures In gure the thumb could b e ignored thereby resulting in the same p osture The thumb

angles are still available though so they can b e used indep endently of the p osture to control some action

For example if the user manipulated some virtual to ol such as a virtual spray can or ashlight the thumb

could b e used to toggle the to ol on and o Currently only the thumb can b e ignored in a p osture but it is

conceivable that certain applications might need to ignore other ngers as well

Each p osture is represented as angles and a ag signifying whether the thumb is to b e used With a call

to WhichPosture the recognizer compares the current hand conguration to the table of known p ostures

This comparison is the average of the dierence b etween each of the or with thumb angles The

p osture from the table that is closest to the current hand p osture is rep orted as the current p osture If the

users hand is currently to o dierent from any of the meaningful p ostures as dened by a threshold value

the system rep orts no current p osture

A service provided by the p osture recognizer is the interactive denition of new p ostures A user denes

new p ostures by simply putting his hand in the desired p osture and clicking a mouse button Several p ostures

can b e dened this way and then tested to see how well they work together In the testing phase the program

indicates what it thinks the p osture of the hand is in to the user This testing is imp ortant when creating a

set of multiple p ostures to ensure that none of them are to o similar to cause the p osture recognition system

to evaluate them incorrectly When the user is satised with the results the p osture utility program creates

the table of known p ostures In this fashion it is easy to add new p ostures or customize an existing p osture

set

Note that the hand may move and still b e in the same p osture Thus a static nger p ointing p osture and

the same nger p ointing p osture while the hand traces out a circle are recognized as the same p osture If

an application program needs to treat these two dierently then a separate gesture as opp osed to p osture

recognition system is required In our application gestures are not used But p ostures that may move over

time are used Each valid p osture represents one of the visualization to ols describ ed in section These to ols

are then able to generate a graphical representation of the data according to the to ols metho d of display

Visualization Op erations

The visualization system provides two basic metho ds of visualizing the data These are isosurfaces and

pseudocoloring of cut surfaces

An isosurface is created by tiling p oints in the data set with the same constant value Here we use

the marching cub es algorithm with a mo died lo okup table to remove ambiguities Isosurfaces are

rendered transparently so that to ols manipulated within the data are always visible to the user A family of

transparent isosurfaces can also serve as a reference and give the user a go o d overview of the main features

of the data However the user should realize that the shap e of the isosurface is only the shap e of the data

set at the sp ecied constant value Inside or outside of the isosurface the data may have a radically dierent

shap e so further investigation is generally necessary in order to understand the true nature of the data set

Some means of exploring is also available by selecting dierent constant values through the graphical user

interface or through the Threshold to ol see section Varying the threshold value helps the user identify

small features and general trends in the data

When constructing an isosurface data p oints are classied as b eing inside or outside of the isosurface

dep ending on whether their value is ab ove or b elow the constant value Since this inout classication is

very similar to clipping op erations the isosurfaces can also b e used as clip surfaces With this realization an

option was implemented to clip a cut surface to an isosurface That is only the p ortion of a cut surface that

lies within an isosurface is actually displayed This option makes the cut surface lo ok like a crosssection of

an ob ject

Pseudocoloring simply maps data values to color values However the color mapping pro cess can b e

very subtle or it can make a dramatic dierence in what the user learns from the data In general the

color mapping function should b e designed for the sp ecic data set b eing visualized and not just a generic

color map Thus we allow the users to sp ecify arbitrary mapping from data value to color by cho osing a

custom color palette le That is color maps are created by sp ecifying any numb er of data values and the

corresp onding color to b e mapp ed to a particular value During visualization if one of these data values is

found then the matching color is displayed If the data p oint lies b etween two values sp ecied in the color

map the color displayed is a linear interp olation of the two surrounding colors in RGB color space Color

maps are stored as external les and can b e chosen by the user when viewing a data set The user may even

try out two or more dierent color maps on the same data set to see if one works b etter or reveals some

feature hidden by the other one

User Interface and Navigational Cues

Although the visualization to ols are controlled directly by hand there are still various details that would b e

cumb ersome to sp ecify with hand p ostures such as sp ecifying data set to examine toggling isosurfaces on

and o sp ecifying the sensitivity of to ols and adjusting viewing parameters These details are controlled by

various sliders and buttons Fortunately these are used relatively infrequently and the user may concentrate

on his hands and how to move them through data

During the design phase we had the option of using a second tracker as a virtual camera with the

eyeballinhand metaphor By moving this tracker around the user would eectively change his viewing

p osition Holding the tracker away from his b o dy and p ointing it back at himself would result in a view from

b ehind the data Unfortunately when viewing the scene from some arbitrary angle the hand with the virtual

to ol would not move as exp ected Such a third p erson p oint of view would require the user to consciously

map his hand motions relative to the virtual camera resulting in a less natural and direct interface While

this virtual eye may likely b e useful in some applications it was left out of this system Instead the second

tracker is used to move and rotate the data volume Together with the direct manipulation to ols arbitrary

cuts can b e generated with this two hand approach

There are several visual cues to assist the user in navigating around the workspace while manipulating

the to ols and the data see color plate Since there is a p ossibility that these navigational aids may in fact

interfere with the exploratory visualization pro cess they are all optional and may b e disabled individually

The rst navigational aid is a wireframe b ox surrounding the data Since the user has the ability to

move the data volume anywhere this is an essential navigational aid to quickly orient the user to the current

p osition and orientation of the data volume The second navigational aid is the hand mo del that represents

the users hand p osition within the data volume At the coarsest level it gives the user the knowledge of

where the to ols should b e manipulated to visualize the data The third navigational aid is placing the data

volume within a virtual ro om Instead of just letting the data oat in empty space the o or and two walls

of the virtual ro om provide a sense of space to the user It also helps somewhat in depth p erception but

more imp ortantly it provides a context for casting shadows Shadows can b e cast by the data set and the

virtual hand onto the walls and o or of the virtual ro om Shadows are particularly useful for providing the

user with the relative depth information of the virtual hand and the data Because their primary purp ose

is as navigational aids they do not need to p ortray all the details Instead we simply pro ject the hand and

the data onto the o or and walls Hence the shadows pro duced in this manner are very fast to compute

and do not degrade the interactivity of the to ols

The nal navigational aid is stereo vision This is achieved with a pair of StereoGraphics glasses that

rapidly presents alternate left and right views to the user The stereo vision provides users with a b etter

sense of depth and allows them to manipulate the to ols and understand the data b etter

Sub division

The cutting to ols mentioned in section are resp onsible for describing the cut surface Points on the cut

surfaces are then used to sample the data volume which are then mapp ed to color If every p oint on the

cut surface were used to sample the data the computation would b e to o exp ensive to maintain interactivity

To combat this problem data values are only computed at triangle vertices Gouraud shading is then used

to render the triangles At the exp ense of slower interactions b etter approximations can b e achieved by

sub dividing the triangles We provide two sub division schemes regular and adaptive

Regular sub divison is accomplished by connecting the midp oints of each triangle edge The result is that

the original triangle is transformed into four smaller triangles Instead of just three six data p oints are

calculated after the sub division Figure shows a hyp othetical triangle that makes up a cutting surface and

the same triangle after sub dividing it into four smaller triangles

Since the recursive level of sub division will aect the uidity of the user interaction with the to ols the

level of sub division is a parameter adjustable by the user so that he can decide on an acceptable tradeo

b etween sp eed and quality for a given situation

An alternative sub division scheme is to adaptively sub divide only in regions where the data is changing

rapidly Here a triangle is split up only if the data values at its vertices are suciently dierent to warrant

another level of sub division The meaning of suciently dierent can b e mo died by the user but it is

basically a test of whether the dierence b etween the greatest data value and the least value of the three

vertices is ab ove a certain threshold The threshold is expressed as a p ercentage of the total range from

minimum to maximum in the entire data set Adaptive sub division allows the system to sp end more time

Figure Original and Sub divided Triangle

in areas of ner details without wasting time in relatively empty spaces Figure illustrates the results of

adaptive sub division on the hyp othetical triangle while gure shows the sub division through data with

varying levels of detail In areas where the data is changing rapidly the plane is sub divided much more

nely whereas areas with constant or near constant data values the plane is not sub divided very much at

all

Figure Adaptively Sub divided Triangle

Figure Adaptive Sub division

Regular sub division can b e used together with adaptive sub division By default the system is set up

to p erform two levels of regular sub division followed by two levels of adaptive sub division During the

exploration phase the user can change the sub division levels as desired

Direct Manipulation To ols

Several direct manipulation to ols are available to aid the user in exploratory scientic visualization Each

to ol is activated by forming the appropriate p osture for the to ol While we do not supp ort gestures the hand

p ostures can b e moved through space to dene to ol parameters The sensitivity of each to ol can b e adjusted

so that to ols may b e swept through large p ortions of the data with relatively small hand movements or they

can b e moved through smaller regions with the same range of hand movements for close up manipulations

All except one of the to ols p erform cutting op erations Following a knife metaphor the to ols turn the

hand of the user into a virtual blade that cuts through the data set That is they create crosssectional

surfaces that pass through the volume in which the data is dened and on which the data is displayed

The most straightforward case is a crosssection planar cut through the data volume We also provide two

extensions to the standard cutting planes One that pro duces linear surfaces dened by sweeping a straight

edge through space and one that pro duces curve surfaces dened by sweeping a varying curved segment

through space All of the cutting to ols make use of a data structure called a cutPlane It is essentially a

p olygon which is p ositioned within the data space and is later mapp ed with color according to data values

that it intersects A simple cross section may use only one p olygon while a to ol that creates curved surface

cuts may need to create many small cutPlanes to approximate the curved surface Each to ol is resp onsible

for generating cuts by following the motion of the hand according to the metho d of op eration of that sp ecic

to ol However once cutPlanes are generated they are treated the same way by the visualization system

As so on as a to ol p osture is recognized the to ol b ecomes active and continuously generates cut surfaces

In situations when the user wishes to view two cuts at once or mayb e a complex cut generated by multiple

to ols a to ol lo cking mechanism is used Once a to ol is lo cked the visualizations from that to ol are frozen

in place and a new to ol or another instance of the old to ol may b e activated by the user to create new cuts

The switch that comes with the Cyb erglove is used to lo ck the current to ol without having the user shift his

concentration away from the data If there is no current to ol when the switch is pressed the most recently

lo cked to ol is deleted

Flat Cut To ol

The rst of the cutting to ols in the system is the at cut to ol The concept for the user when this to ol is

invoked is a choppinglike action This is reinforced by the p osture required to use this to ol a at op en

palm as shown in gure

Figure Flat Cut Posture

The at cut to ol generates a plane that is oriented with the users palm and can b e moved around

intuitively as if it were an extension of the hand see color plates and To convert the plane into a

visualization ob ject the at cut to ol generates two triangleshap ed cutPlanes prop erly oriented and passes

them to the visualization system for sub division and color mapping A go o d use of the at cut to ol is to drag

it through the data and watch the selected planes for imp ortant features in the data Because it generates

a at wide surface it is a go o d to ol to use on a new unexplored data set to help lo cate features of interest

and learn the overall layout of the data

AxisLo cked Cut To ol

The axislo cked cut to ol is very similar to the at cut to ol except that any cut generated by this to ol is

constrained to b e p erp endicular to either the x y or z axis of the data set To invoke the axislo cked cut

to ol the user places his hand in a st p osture to denote a constraint or restriction as shown in gure

When manipulating this to ol it moves freely with the hand of the user but its orientation is snapp ed

to the closest of the three axes Since the data set can b e rotated indep endently with the second tracker

Figure AxisLo cked Cut Posture

the axislo cked cut to ol always aligns itself to one of the axes of the data set and not the world co ordinate

system see color plate The axislo cked cut to ol is useful when a thorough examination of a data set is to

b e made The constraints inherent in this to ol makes it easy to maintain parallel cuts as the to ol is dragged

through a data set

Linear Cut To ol

This cutting to ol is more exible than the previous two and is used to generate curved cuts The b ehavior

it tries to mimic is that of a knife blade cutting out a p ossibly curved sweeping path in space The hand

p osture for this to ol is a common imitation of a cutting pro cess as shown in gure

Figure Curved Cut Posture

Once this to ol has b een activated by the user making the appropriate p osture it b egins tracking the

hand p osition As the user moves his hand through space the rst two ngers trace out a D ruled surface

see color plate This surface is dened to b e the cut that the to ol is making through the data and is

reconstructed by placing many narrow rectangles side by side to approximate the curvature of the surface

The linear cut to ol is generally used when the user knows something ab out the layout of the data set If

this is the case the user can employ the linear cut to ol to sweep out a continuous and p ossibly complex

surface through the region of interest In keeping with the knife metaphor this to ol is also equipp ed with

an adjustable blade length The length is adjusted through a slider A short blade is useful for constructing

detailed cuts or even for simple cuts where the user wishes to restrict it to a lo cal region in the data The

blade may also b e used in a long conguration suitable for making a curved cut that extends through the

entire data set

Curve Cut To ol

The curve cut to ol generates curved D surfaces The hand p osture mimics a knife with a curved blade that

is swept through space The hand p osture in gure illustrates this p oint Note that the p osture is similar

to the curved cut to ol p osture except that the rst two ngers are b ent to indicate that the knife blade is

curved

Figure DoublyCurved Cut Posture Two Views

With the curve cut to ol the user is able to generate a surface that is as much a sco op as a cut The

curvature along the knife blade is created by the three connected nger segments As the user manipulates

this to ol he may change the amount of b ending in his rst two ngers see color plate Changing this

b ending results in a change of the curvature of the blade of the to ol Thus not only is the blade curved and

able to b e curved to dierent amounts but the curvature is adjustable while the to ol is in motion varying

the concavity of the blade as it cuts out a surface As with the linear cut to ol it also shares the variable

blade length feature

The curve cut to ol is b est suited to cases where the user wants to exclude some data feature from the

visualization or to view the shell area surrounding a feature

Prob e To ol

The prob e to ol is dierent in b ehavior and op eration from the other to ols in the system Although it still is

directly manipulated by the user through hand p ostures and motions it is not used to create pseudocolored

cuts through the data Instead the user invokes the prob e to ol in order to query the data value at a sp ecic

p oint in space The p osture used to create a prob e to ol is the most common technique to indicate a lo cation

p ointing with the index nger as shown in gure Note that the thumb p osture is ignored for the this

to ol

Figure Prob e Posture

When the prob e to ol is invoked it replaces the display of the virtual hand with a hand that has the index

nger replaced with a thin wire The tip of this wire is the p oint b eing evaluated in the data set As long as

the to ol is in use the user may move his hand around to prob e dierent parts of the data set Next to the

tip of the prob e the numerical value of the data at that p oint is displayed for the user

Threshold To ol

Like the cutting to ols the threshold to ol also generates surfaces However unlike the cutting to ols the

surfaces are isosurfaces in the vicinity of where the hand is swept To activate this to ol the user places his

hand in a p osture similar to a wiping action as shown in gure

Figure Threshold Posture

In addition to placing his hand in this p osition the user sp ecies a threshold value for using a slider

When the threshold to ol is moved through the data lo cal isosurfaces are generated through the swept area

see color plate The user only needs to sweep out general areas of interest and the threshold to ol will

seek out nearby regions of data at the threshold value In the current implementation nearby is dened to

b e in the range of the two nearest data p oints in each direction

This to ol is very useful for extracting features when the user knows the threshold value of interest but

may not know exactly where the feature lies The computer could easily generate complete isosurfaces but

this may lead to a complicated geometry By letting the users hand directly guide the isosurface construction

lo cal isosurfaces may b e constructed which are easier to understand b oth b ecause they do not take up the

entire display and b ecause they are p ersonally placed by the user

Multiple To ols and Instances

Plate shows a more complex example This example uses the same data set as in plate Several cuts

have b een applied by the user in dierent areas of the data set and the clipping and trimming features have

b een generously applied to keep the uninteresting parts of cutting planes from overlapping and obscuring

the view

Evaluation

Overall the system has b een found to b e easy to use and esp ecially wellsuited to a casual exploratory

style of visualization There are some limitations to the system but these are not fundamental aws

Improvements could b e made to correct many of the limitations making it a more usable and robust system

Successes

When evaluating this system one should rememb er that it is driven by direct human interaction This leads

to b oth p ositive and negative results On the p ositive side the system is very easy to use The user never

needs to worry ab out angles or co ordinates when placing cutting planes Just a few simple hand p ostures

need to b e rememb ered and these are easy to keep in mind b ecause they are very similar to gestures used

in the real world when p erforming various cutting op erations Direct manipulation has b een found to b e

very natural and this can b e attributed to the fact that in this system of interaction the computer is b eing

adapted to the user rather than the user to the computer at least more so than using standard numerical

or mousedriven techniques

Besides b eing easier to use direct manipulation seems to promote a dierent style of interaction Because

it is so easy to manipulate a visualization to ol in this system the user will likely tend to change to ols or

change the p osition of a single to ol at a moments notice This ability leads to a more casual typ e of

interaction in which the user is more comfortable exploring the data without feeling restricted in what they

can do An exploratory approach is very useful when dealing with a new data set since the ability to rapidly

alter cutting planes lets the user scan the overall data set quickly while lo oking for regions that might b e of

interest for later detailed analysis

Shortcomings

The limitations of this system can b e categorized in four ways First there are limitations due to the nature

of human interaction Second there are limitations due to the current sensing technologies used in the glove

and tracker devices Third there are limitations of compute p ower And fourth the visualization p ortion of

this system has some drawbacks that should b e addressed

Unfortunately the direct user interaction mo del also leads to some drawbacks First of all b ecause of

the nature of human interaction itself it is dicult to b e precise when manipulating visualization to ols If

the user has a particular cutting plane in mind it would b e very dicult if not imp ossible to recreate that

cutting plane using direct handcontrolled to ols

In addition the magnetic tracking system always has a bit of noise resulting in small vibrations of the

cutting to ols When a to ol is frozen by the user it no longer vibrates but it could freeze anywhere in this

vibration range For ordinary work the vibration is so small that its eect is irrelevant However it would

b e dicult to achieve exact p ositioning

Because the to ols are tied directly to the hand of the user drawing sp eed is a ma jor concern If something

on the screen do es not keep up with a mouse movement the user can usually deal with it But if a cutting

to ol cannot keep up with hand motions it b ecomes dicult to work with In order to keep up with the

hand p osition it is necessary to avoid nely sub dividing cutting planes This in turn leads to imprecise

pseudocoloring on cuts Using the system on faster computers will x this situation but it would b e useful

if the user could work on any machine with the required input hardware

Another drawback of the system is that it currently only supp orts data that is sampled on a regular grid

This is a limitation of the visualization p ortion of the system and not the user interaction mo del If the

user wants to visualize nonrectilinear or scattered data there is currently no choice but to rst resample

the data onto a regular grid

Summary

We presented direct manipulation to ols for the express purp ose of exploratory visualization These to ols

op erate satisfactorily for such use and within the constraints imp osed by the current technology However

they fail if precise p ositioning of to ols is required In those situations traditional numeric entry would b e

more appropriate During the course of our investigations we have also identied a few areas for further

enhancement and investigation

Improved p osture recognition is an area deserving of further study The current system can distinguish

among approximately six dierent p ostures An improved p osture recognition scheme might incorp orate

some form of learning algorithm to b e able to distinguish among more p ostures More p ostures would allow

for more to ols for the user An improved system might also recognize gestures or p ostures that change over

time Gestures provide a way to allow more complex forms of interaction

Another imp ortant enhancement would b e to supp ort other data organizations scattered data samples

curvilinear grids etc and data typ es vector data multiple scalar values at a single p oint etc In

addition timevarying data would b e a useful addition This could include timevarying scalar data such as

temp eratures in a volume or vector data that is visualized using timevarying techniques

To complement new data organizations data typ es and visualization requirements other visualization

metaphors should b e investigated We have identied and used other metaphors aside from the cutting to ols

for example spray cans and ashlights However we have not adapted these to direct manipulation

interfaces

Besides making interaction easy for a single user a multipleuser setup could assist in collab oration

and instruction If b oth users were equipp ed with gloves one user might b e the principle user examining

the data but the second user could o ccasionally reach out and create a cut instead of giving vague verbal

instructions telling the principal user how to work

Authors Biographical Sketches

MICHAEL CLIFTON RealSpace Inc BS Electrical Engineering UC Davis MS Computer Science

UCSC Current research interests are in imagebased rendering systems multimedia and realtime photore

alistic rendering eects

ALEX PANG Assistant Professor of Computer Sciences at University of California Santa Cruz Current

research interests are in computer graphics scientic visualization collab oration software multimedia and

virtual reality interfaces

References

MP Baker Human factors in virtual environments for the visual analysis of scientic data Tech

Rep ort draft National Center for Sup ercomputing Applications Septemb er

S Bryson and C Levit The virtual wind tunnel IEEE CGA July

B Buxton Smoke and mirrors Byte July

J Carroll editor Designing Interaction Psychology at the HumanComputer Interface Cambridge

University Press

P Cignoni C Montani and R Scopigno MagicSphere an insight to ol for D data visualization

Computer Graphics Forum C Septemb er

JH Clark Designing surfaces in D Communications of the ACM August

JD Foley Interfaces for advanced computing Scientic American Octob er

T He and A Kaufman Virtual input devices for D systems Proceedings Visualization pages

Octob er

William Hibbard and David Santek The VisD system for easy interactive visualization In Visual

ization pages

W Lorensen and H Cline Marching cub es A high resolution D surface construction algorithm

Computer Graphics ACM SIGGRAPH Proceedings July

Tom Meyer and Al Globus Direct manipulation of isosurfaces and cutting planes in virtual environ

ments Technical rep ort Dept of Computer Science Technical Rep ort

R Mo orhead C Everitt B Hamann S Jones J McAllister and J Barlow Oceanographic visualiza

tion interactive research to ol OVIRT Proceedings of the SPIE

D Ney and E Fishman Editing to ols for D medical imaging IEEE CGA Novemb er

J Pa jon and V Tran Direct mousedriven manipulation for interactive visualization of structured data

Third Eurographics Workshop on Visualization in Scientic Computing pages April

Alex Pang Spray rendering IEEE Computer Graphics and Applications

Alex Pang and Michael Clifton Metaphors for visualization In Sixth Eurographics Workshop on

Visualization in Scientic Computing page to app ear

FH Rabb Magnetic p osition and orientation tracking system Aerospace and Electronic Systems

AE Septemb er

M Rho des Y Azzawi E Tivattanasuk A Pang K Ly H Panicker and R Amador Curvedsurface

digital image reformations in computed tomography Proceedings of the SPIE Decemb er

LG Rob erts The Lincoln wand Tech Rep ort MIT Lincoln Lab oratories

E Sachs A Rob erts and D Sto ops Draw A to ol for designing D shap es IEEE CGA

Novemb er

B Schneiderman Direct manipulation A step b eyond programming languages Computer

August

A Sellen G Kurtenbach and W Buxton The prevention of mo de errors through sensory feedback

HumanComputer Interaction

D Sturman Wholehand input PhD Thesis Massachusetts Institute of Technology Decemb er

J Udupa and D Odhner Fast visualization manipulation and analysis of binary volumetric ob jects

IEEE CGA Novemb er

C Ware and DR Jessome Using the Bat A sixdimensional mouse for ob ject placement IEEE

CGA Novemb er

T Yoshimura Y Nakamura and M Sugiura Direct manipulation interface Development of the

ZashikiWarashi system Computers and Graphics MarchApril