FEATURE TRACKING AND VIEWING FOR TIME-VARYING DATA SETS

DISSERTATION

Presented in Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy in the

Graduate School of The Ohio State University

By

Guangfeng Ji, M.E.

*****

The Ohio State University

2006

Dissertation Committee: Approved by

Professor Han-Wei Shen, Adviser Professor Rephael Wenger Adviser Professor Raghu Machiraju Graduate Program in Computer Science and Engineering ABSTRACT

Feature tracking plays an important role in understanding time-varying data sets since it allows scientists to focus on regions of interest and track their evolution and interaction over time. In this work, we first present an efficient algorithm to track time- varying isosurface and interval volume features using isosurfacing in higher dimensions.

Instead of extracting these isosurface or interval volume features separately from mul- tiple time steps and computing the spatial correspondence between them, our algorithm extracts the correspondence directly from the higher dimensional geometry and allows more efficient feature tracking. We further show that the correspondence relationship for time-varying isosurfaces can only change at critical isovalues in R3 or R4. Based on the observation, we present a method to pre-compute the correspondence relationship at a preprocessing stage. At run time, isosurface tracking can be efficiently performed by simple table lookup operations with minimal overhead. For complex data sets, the previous feature tracking methods cannot guarantee the globally best matching because these methods use local matching criteria and track features independently in the local neighborhood. To amend the problem, we propose a novel global tracking technique to track features, which defines the globally best match as the one with a minimal overall matching cost. Noticing the deficiencies of previous tracking criteria, we use the Earth

Mover’s Distance as a better metric to measure the matching cost. We also propose an efficient branch-and-bound algorithm to search the global minimal cost.

ii In addition to tracking features, another important problem is how to view the time- varying features effectively. Due to the time-varying nature, animation remains the most general and common way to show how time-varying features evolve over time. A key issue of generating a good animation is to select ideal views through which the user can perceive the maximum information of the time-varying features. In this work, an algorithm is also presented to select dynamic views. We first propose an improved view selection method for static data, which measures the quality of a static view by analyz- ing the opacity, color and curvature distributions of the feature rendering images from the given view. A dynamic programming approach is used to select dynamic views. The process maximizes the information perceived from the time-varying features based on the constraint that the view should show smooth changes of direction and near-constant speed. Our feature tracking and viewing algorithms provide the user with a more effec- tive and efficient way to study the time-varying features, and allow the user to gain more insight into the time-varying data sets.

iii Dedicated to my parents . . .

iv ACKNOWLEDGMENTS

First of all, I would like to express my sincere gratitude to my adviser, Dr. Han-Wei

Shen, for his guidance, encouragement and support, during my years of Ph.D study at

The Ohio State University. His insights, encouragement and solid professional knowl- edge have guided me through difficulties and challenges. I learned a lot from him and his valuable advices will accompany me throughout my career. I have been truly lucky to have the opportunity to work with him. Without his help and support, this dissertation would not be possible.

I would also like to thank my dissertation committee members, Dr. Rephael Wenger and Dr. Raghu Machiraju for giving me valuable suggestions during my research and dissertation writing. Many fruitful hours were spent with them discussing research and the dissertation. They showed me new perspective of looking at problems, and broad- ened my thinking and knowledge scope. I also want to thank Dr. Roger Crawfis for his great help on my research.

I want to thank my colleagues and friends for their great help. I really enjoyed the time I work with my team members: Jonathan Woodring, Chaoli Wang, Liya Li, Jinzhu

Gao, Udeepta Bordoloi, and Antonio Garcia. We spent many hours discussing research and exchanging ideas. We shared code and helped on implementations. Yipeng Li and

Yisheng Chen have spent their precious time helping me teaching. Caixia Zhang also

v helped me a lot. I really enjoyed the time spending with all of my colleagues and friends and appreciate their great help.

My family has always been there for me. Their unqualified love and support have been accompanying me throughout the journey. I am truly grateful to have such a happy and supportive family.

vi VITA

June 19, 1976 ...... Born - Shanxi, China

1993-1997 ...... B.S. Computer Science Peking University, China 1997-2000 ...... M.E. Computer Science Chinese Academy of Sciences, China 2000-2001 ...... Graduate Teaching Associate The Ohio State University 2001-2005 ...... Graduate Research Associate The Ohio State University 2005-present ...... Graduate Teaching Associate The Ohio State University

PUBLICATIONS

Research Publications

G. Ji and H-W. Shen. Dynamic View Selection for Time-Varying Volumes. Technical report OSU-CISRC-5/06-TR50, The Ohio State University

G. Ji and H-W. Shen. Feature Tracking Using Earth Mover’s Distance and Global Optimization. Technical report OSU-CISRC-5/06-TR49, The Ohio State University

G. Ji and H-W. Shen. Time-Varying Isosurface Tracking by Global Optimization. Tech- nical report OSU-CISRC-2/05-TR12, The Ohio State University, 2005

G. Ji and H-W. Shen. Efficient Isosurface Tracking Using Precomputed Correspondence Table. In Joint Eurographics - IEEE TCVG Symposium on Visualization 2004, pages 283–292, 2004.

vii G. Ji, H-W. Shen, and R. Wenger. Volume Tracking Using Higher Dimensional Isosur- facing. In Proceedings of IEEE Visualization 2003, pages 209–216, 2003.

G. Ji, H-W. Shen and J. Gao. Interactive Exploration of Remote Isosurfaces with Point- Based Non-Photorealistic Rendering. Technical report OSU-CISRC-7/03-TR37, The Ohio State University

FIELDS OF STUDY

Major Field: Computer Science and Engineering

viii TABLE OF CONTENTS

Page

Abstract ...... ii

Dedication ...... iv

Acknowledgments ...... v

Vita ...... vii

List of Tables ...... xii

List of Figures ...... xiii

Chapters:

1. Introduction ...... 1

1.1 Problem Statement ...... 1 1.2 Challenges ...... 3 1.3 Strategies and Contributions ...... 5 1.4 Organization ...... 10

2. Related Work ...... 12

2.1 Feature Tracking ...... 12 2.1.1 Aggregate Attribute Based Methods ...... 12 2.1.2 Volume Overlapping Based Methods ...... 14 2.2 View Selection ...... 17 2.2.1 Static View Selection ...... 17 2.2.2 Dynamic View Selection ...... 19

ix 3. Volume Tracking Using Higher Dimensional Isosurfacing ...... 21

3.1 Overview ...... 21 3.2 Isosurface Tracking ...... 21 3.2.1 Extracting Overlapping Time-varying Isosurfaces ...... 22 3.2.2 Verification ...... 27 3.3 Interval Volume Tracking ...... 29 3.3.1 Extracting Overlapping Time-Varying Interval Volumes . . . 29 3.3.2 Verification ...... 34 3.4 Results ...... 34 3.5 Summary ...... 37

4. Efficient Isosurface Tracking Using Precomputed Correspondence Table . . 43

4.1 Overview ...... 43 4.2 Isosurface Tracking ...... 44 4.2.1 Tracking by Using Higher Dimensional Isosurfacing . . . . . 44 4.2.2 Generation of the Correspondence Lookup Table ...... 45 4.2.3 Optimization in Generation of the Correspondence Table . . . 54 4.2.4 Isosurface Tracking Using Correspondence Lookup Table . . 55 4.3 Results ...... 56 4.4 Summary ...... 62

5. Feature Tracking Using Earth Mover’s Distance and Global Optimization . 65

5.1 Overview ...... 65 5.2 Using Earth Mover’s Distance as Matching Cost ...... 68 5.2.1 Earth Mover’s Distance ...... 71 5.2.2 Efficient EMD Computation ...... 73 5.3 Global Optimization Based on Branch-and-Bound ...... 75 5.3.1 The Search Tree ...... 75 5.3.2 Branch and Bound ...... 78 5.3.3 Further Speedup by a Good Estimation of the Minimal Cost . 80 5.3.4 Using EMD to Identify Compound Component Candidates . 81 5.4 Results ...... 83 5.5 Summary ...... 88

6. Dynamic View Selection for Time-Varying Volumes ...... 89

6.1 Overview ...... 89 6.2 Static View Selection ...... 90

x 6.2.1 Measurement of Opacity Distribution and Projection Size . . 91 6.2.2 Measurement of Color Distribution ...... 93 6.2.3 Measurement of Curvature Information ...... 96 6.2.4 The Final Utility Function ...... 97 6.3 Dynamic View Selection ...... 98 6.3.1 Time-Varying View Selection ...... 99 6.3.2 Viewing Path Between any Two Views in a Given Timestep . 103 6.4 Results ...... 105 6.5 Summary ...... 109

7. Conclusions and Future Work ...... 117

Bibliography ...... 120

xi LIST OF TABLES

Table Page

3.1 The time to track the isosurface component illustrated in Figure 3.5 (in seconds)...... 36

3.2 The time to track interval volume components illustrated in Figure 3.6 (in seconds)...... 37

4.1 The time to track the whole isosurface illustrated in figure 4.5 (in sec- onds). Timing from 5 time steps are shown due to space limitation. . . . 58

4.2 The time to track two isosurface components illustrated in figure 4.6 (in seconds). Timing from time step 4, 9, 15 and 21 are shown due to space limitation...... 59

4.3 The time to track two isosurface components illustrated in figure 4.7 (in seconds). Timing from time step 1, 2, 6, 7 and 14 are shown due to space limitation...... 60

4.4 The comparison between two methods to generate the correspondence lookup table...... 61

5.1 The timing (in seconds) for the case in figure 5.8...... 86

3 6 11 5.2 The EMD values for F0 , F0 and F0 . The first four smallest values are shown. DIS means a feature dissipates...... 86

5.3 The timing (in seconds) for the case in figure 5.9...... 88

xii LIST OF FIGURES

Figure Page

2.1 The evolutionary events that a feature may experience...... 16

3.1 The four different types of 3-cells when two isocontours in R2 intersect. 23

3.2 The four different types of 3-cells when one isosurface is contained within the other...... 25

3.3 Only 3 sign combinations are possible when extracting interval volumes in R2...... 33

3.4 6 out 100 time steps are shown. All isosurface components are tracked and the evolutionary information determines the coloring of the compo- nents...... 39

3.5 An isosurface component is tracked. Bifurcation happens at t=10 and dissipation at t=18 and 28...... 40

3.6 Two interval volume component are tracked. These two components merge at t=29 and then split at t=31...... 41

3.7 Volume vs time for the example in Figure 3.5. Note the component bifurcates into two and each of them dissipates...... 42

4.1 Four snapshots to illustrate how the correspondence relationship be- tween isosurfaces from two consecutive time steps changes when the isovalue increases from v0 to v3 (v0 < v1 < v2 < v3)...... 47

4.2 Two previously corresponding components do not correspond to each other any more and the number of components in R3 remains the same when the isovalue changes...... 49

xiii 4.3 A plausible case where two pairs of components switch their correspon- dence without the change of number of isosurface components in R3 and R4 when the isovalue changes...... 50

4.4 Another seemingly reasonable case: one component at each time step changes its correspondence when the isovalue changes but without the change of number of isosurface components in R3 and R4...... 52

4.5 All isosurface components are tracked and the tracking information de- termines the coloring of the components. 8 out of 99 time steps are shown...... 57

4.6 Two isosurface components are tracked. These two components merge into a component at t=4 and it then bifurcates into two at t=15. The larger component bifurcates again at t=21. At t=22, the smallest com- ponent dissipates...... 63

4.7 Two isosurface components are tracked. Amalgamation occurs at t=2; bifurcation occurs at t=7 and dissipation at t=15...... 64

5.1 Two cases to illustrate that the local tracking techniques do not guaran- tee the globally best matching result...... 66

5.2 A thin feature is moving fast. The volume-overlapping criterion will not give a meaningful matching cost in this case...... 69

5.3 A figure to illustrate why EMD is a better metric than the aggregate- attribute approach...... 70

5.4 A feature (in a 16 × 16 grid)) and its simplification (in a 4 × 4 grid)). . . 74

5.5 An example of two feature sets to be matched...... 76

5.6 Part of the search tree for the case in figure 5.5...... 77

5.7 An example to illustrate how the branch-and-bound operation is applied to enable efficient tree search...... 79

xiv 5.8 Time varying features from a force field data set are tracked and the tracking result determines the coloring of the features. Four snapshots are shown...... 84

5.9 Time varying features from the vorticity data set are tracked and the tracking result determines the coloring of the features. Four snapshots are shown...... 87

6.1 The figure to illustrate that the result from [11] should be improved. . . 91

6.2 An example of a partition of a view point’s local tangent plane and one of the possible allowed turns encoded in matrix...... 101

6.3 The solid curve is the SLERP path. Our algorithm will consider all the neighbors of the SLERP path that lie within the dotted area. All the neighbors are parameterized by u and v...... 104

6.4 The figure shows the static view selection results based on opacity en- tropy for the shockwave dataset. (a) shows the worst view, (b) is the best view, and (c) and (d) are the opacity images for (a) and (b) respectively. (e) plots the change of opacity entropy with respect to different viewing angles where the shockwave is rotated around the vertical axis in a full circle...... 111

6.5 The figure shows the static view selection results based on opacity en- tropy for the tooth dataset. (a) shows the worst view, (b) is the best view, and (c) and (d) are the opacity images for (a) and (b) respectively. (e) plots the change of opacity entropy with respect to the viewing angle when the tooth is rotated around the X axis in a full circle...... 112

6.6 The figure shows the static view selection results based on color entropy for the vortex dataset. (a) shows the worst view, (b) is the best view, and (c) plots the change of color entropy with respect to different viewing angles when the vortex is rotated around the Y axis in a full circle. . . . 113

6.7 The figure shows the dynamic view selection results for the TSI dataset. (a) shows the worst view, (b) is the best view, and (c) plots the change of the final information with respect to the viewing angle when the TSI dataset is rotated around the vertical axis in a full circle...... 114

xv 6.8 The figure shows two paths which move from one view point to the other. The right path is generated by SLERP interpolation with an aver- age information of 0.51. The left path is generated by our method. The path is smooth and gives an average information of 0.56...... 115

6.9 The figure shows the dynamic view selection result for the TSI dataset. (a) shows the path of the time-varying view, which exhibits constant an- gular velocity. (b)-(e) show four snapshots captured by our time-varying view. (h)-(i) show the image from the original static view at the timestep corresponding to (b)-(e)...... 116

xvi CHAPTER 1

INTRODUCTION

1.1 Problem Statement

Scientists are now able to perform large scale time-varying simulations to model phenomena that are complex and highly unsteady. For example, meteorologists often perform simulations to study how storms form and evolve. In [37], scientists studied the autoignition phenomena by tracking time-dependent features defined as high inter- mediate concentrations. In [23], Goodseel studied the hemoglobin oxy-deoxy process by tracking the topological changes of the four heme sites which are identified as iso- surface components. To analyze data generated from those simulations, visualization has become an essential tool. Besides the basic goal of presenting an intuitive view of the data, an important aim of visualization is to highlight salient data features and offer unique insight. For time-varying data, an effective visualization tool should also compute and track features over time in an accurate and efficient manner.

Another challenging problem for visualization time-varying data is to have an effec- tive way to view the evolution of the time-varying features. Previous researchers have proposed techniques [33, 75] to allow for a better understanding of the time-dependent

1 features and their evolutions through high dimensional projection and illustration. How- ever, due to the time varying nature of the underlying datasets, the most general and commonly used method for visualizing time-varying feature is still animation, which is created by rendering each static volume data in the time sequence. One problem for pro- ducing animations for time-varying data is that the features of interest often evolve over time, with their shapes, positions, and orientations changing continuously. To provide the user with the best visualization of those features in an animation, it is very important to select dynamic views that can follow those features so that a maximum amount of information throughout the time sequence can be perceived. As a time-varying dataset is usually large in size, and thus time-consuming to render, selecting views manually can be a daunting task if it is simply done by trial-and-error. To ensure that the large scale time-varying dataset can be explored in an efficient and effective way, the process of view selection should be done automatically as much as possible. The selected views will be offered to the user as starting points for the exploration.

In short, this thesis tries to address the following problems in visualizing time- varying data sets:

• Tracking Time-Varying Features

In addition to the basic goal of presenting an intuitive view of the data set at

each individual timestep, time-varying visualization should allow the user to track

salient features over time so that the user can gain more insight into the underly-

ing phenomena. Furthermore, the feature tracking process should be done in an

efficient and accurate manner.

• Viewing Time-Varying Features

2 The time-varying features evolve over time with their shapes, positions, and orien-

tations changing continuously. To have an effective rendering of the time-varying

features, a key issue is to find the dynamic view that can follow the evolution of

the features. The dynamic view should maximize the information the user can

perceive from the rendering of the time-varying features.

1.2 Challenges

Visualization of time-varying data has been a challenging problem due to the large size and the time varying nature of the underlying datasets. To facilitate a better under- standing of the time-varying features, an effective visualization should allow the user to track salient features over time, and it should also find the best dynamic view to render the time-varying features so that the user can get the maximal information from the time- varying data set. The following challenges have to be addressed to track time-varying features:

• Efficiency

The time-varying features should be tracked in an efficient manner. This is es-

pecially important for the time-varying data sets generated by today’s large-scale

simulations, which can contain hundreds, or even thousands of time steps, and

each time step can have 5123 or 10243 grid points. The run-time computation

involved in the tracking process should be minimized. Specifically, if the tracking

of time-varying features can be assisted with a pre-processing stage, the informa-

tion should be pre-computed as much as possible and used at run time for better

efficiency.

3 • Accuracy

The time-varying features should be tracked and matched accurately, especially

when the underlying phenomena are complex and/or the sampling rate is not high

enough for small/fast-moving features.

To generate the best dynamic view to render time-varying features, the following challenges have to be addressed:

• Evaluation of Static Views

A method to evaluate static views is needed in order to generate the best dy-

namic view for time-varying features. Previous researchers have considered sev-

eral ways to evaluate static views and automate the process of view selection

[64, 11, 67, 68]. However, many important factors such as the perceived col-

ors, curvatures, and opacities in the final image were not considered in their al-

gorithms. These factors convey important information to the rendering of the

features, and an algorithm taking into account all these factors is needed.

• Efficiency

If there are T total timesteps and N viewpoints at each individual timestep, the

number of possible viewing paths is O(NT ). It is impossible to try every path

and find the best one. An efficient algorithm is needed to find the best dynamic

viewing path.

• Dynamic Views Constraints

The goal of dynamic view selection is to allow the user to find a viewing path

which shows the maximum amount of information from the time-varying dataset.

4 However, in order to generate a smooth animation, the viewing path should not

change its speed or moving direction abruptly. It is important to take the constraint

into consideration when designing the dynamic viewing path.

1.3 Strategies and Contributions

To facilitate a better understanding of the time-varying dataset, it is essential to have the capability to track time-varying features. There are many ways to characterize fea- tures in the scientific visualization literature [76, 1, 16, 35, 43, 26, 20]. For scalar fields, isosurfaces [43] and interval volumes [26, 21] are common ways to characterize fea- tures. By displaying points of a constant threshold value specified by the user, isosur- faces reveal the geometric structure of the objects represented by the data set. Examples of using isosurface to characterize features include the extraction of the heme sites of the hemoglobin molecule in [23] and the high intermediate concentrations in [37]. In practice, features are often manifested as points within a range of data values, which can be represented as an interval volume [26, 21]. An interval volume is a generalization of isosurface and represents a three-dimensional subvolume for which the associated scalar values lie within a user-specified closed interval. When there are errors caused by measurements or numerical simulations, or the target object contains structural ambi- guity, displaying interval volumes can tolerate the errors and provides more meaningful visualization results [21].

For time-varying datasets, visualization of time-varying isosurfaces and interval vol- umes offers insights into how the data evolves over time. However, one major challenge to compute dynamic isosurfaces and interval volumes is the enormous size of the data set. It can be very expensive to extract and track isosurfaces and interval volumes from

5 huge data sets. To study the temporal characteristics of time-varying data, efficient al- gorithms should be provided to allow the user to isolate local features of interest and track their evolution in time in an efficient manner.

In this thesis, we first present an algorithm to track the evolution of isosurfaces and interval volumes from time-varying data using higher dimensional isosurfacing. Instead of extracting these isosurface or interval volume features separately from multiple time steps and computing the spatial correspondence between them, our algorithm can effi- ciently track the user-selected local features, defined as connected isosurface or interval volume components, by performing local propagation and interactive slicing of isosur- faces or interval volumes in R4. By using higher dimensional isosurfacing, we can easily establish the feature correspondence in adjacent time steps.

We further speeds up the tracking of time-varying isosurfaces. The primary obser- vation is that the correspondence relationship between isosurface components of two consecutive time steps can change only at critical isovalues in R3 or R4 where the num- ber of isosurface components in that space undergoes a change. The correspondence between the isosurface components remains the same when the isovalue is within two adjacent critical isovalues. Based on the property, the correspondence between the iso- surface components for any possible isovalue can be pre-computed and stored into a ta- ble. When an isosurface component of an arbitrary isovalue and time step is selected by the user, its corresponding component(s) in the next time step can be rapidly identified by looking them up in the correspondence table. Compared with the existing algorithms

[60, 62] which compute corresponding isosurfaces in consecutive time steps at run time, this isosurface tracking method offers much higher efficiency with minimal overhead.

6 By using higher dimensional isosurfacing and pre-computing the correspondence relationship, we are able to efficiently track time-varying isosurface and interval volume features. However, all the methods, including those proposed by previous researchers

[59, 60, 61, 62, 53, 48], are generally local tracking techniques, where the tracking of each feature is performed independently with various local search schemes. The local tracking techniques do not guarantee to produce the globally best matching results in cases when the scene is crowded with many small/fast-moving features and there exist several alternatives to match those features.

To address the problem, we propose a novel global tracking algorithm. Here features are not just limited to isosurfaces or interval volumes and can be any volumetric objects as long as they can be represented by a set of occupancy cells. All the features from two adjacent time steps are considered together in the global tracking algorithm. We

first devise a method to measure the matching cost between a source and a destination feature component. Then, based on the cost metric, the tracking algorithm finds the globally best match between the two feature sets that has the minimal overall cost. In order to obtain meaningful tracking results, it is essential to have a proper measure of the cost to match two features. The cost measure should take proper consideration of the important characteristics of the feature and gives accurate matching cost. It should be robust and always returns a meaningful value as the cost to match features. To amend the problems of the previous feature tracking algorithms, we propose to use the Earth

Mover’s Distance (EMD) to measure the cost to match two features. EMD represents a feature with the cells it occupies in space, and computes the minimal cost to transport one cell distribution into the other. EMD evaluates the matching cost by using the spatial

7 occupancy information, which includes all the shape, orientation, position and scale attributes of the feature. The EMD measure also offers a meaningful cost to match non- overlapping features. Compared to the previous criteria, EMD provides a more accurate and robust cost measure. In addition to the EMD cost measure, we also propose a branch-and-bound method which enables efficient search of the global minimal cost. A tight estimation of the minimal cost further speeds up the search process. With the Earth

Mover’s Distance criterion and the branch-and-bound algorithm, features are tracked in a more accurate and efficient manner.

Besides tracking features, we also propose an algorithm to generate optimal dynamic views to render the time-varying features. Previous researchers have considered ways to automate the process of static view selection [64, 11, 67, 68]. However, their focuses had not been on time-varying data, which requires special treatments in order to max- imize the amount of information embedded in the whole time sequence. In addition, certain important factors when selecting a good view for static data such as the per- ceived colors, curvatures, and opacities in the final image were not considered in their algorithms. In this thesis, we first present an improved static view selection technique to address some issues that were not previously considered, and then use the new static view selection method and a dynamic-programming optimization approach to find the best time-varying view. The goal of identifying the optimal time-varying view is to maximize the amount of information the user can perceive from the rendering sequence, with constraints on the movement of views to ensure a smooth viewing path. Our static view method measures the quality of a view based on the opacity, color and curvature

8 images generated by the volume rendering technique [18, 42, 44, 74, 19, 41, 49]. The methods used in our algorithm are summarized as follows:

• We propose an optimization approach that finds the best time-varying view in a

polynomial time within a search space of exponential size. The approach also

takes into account the constraints of the movement of the views.

• We properly design the probability function for the opacity distribution and use

it for the opacity entropy evaluation. Our opacity entropy prefers an image with

a large projection area with an even opacity distribution. This technique avoids

some problems that are encountered in [11].

• The color transfer function conveys important information for volume rendering.

We explicitly take into account the color information by properly designing a

probability function and incorporating it into the color entropy evaluation.

• The curvature of the dataset contains essential geometric information about the

dataset. We explicitly take the curvature into account during the static view selec-

tion.

In summary, our feature tracking and viewing work makes the following contribu- tions to the visualization literature:

• We propose an efficient algorithm to track isosurface and interval volume compo-

nents based on higher-dimensional isosurfacing.

• We further report that the correspondence relationship of isosurface components

can be generated at a pre-processing stage. Therefore, the run-time effort to track

time-varying isosurfaces is minimized.

9 • To track complex data sets which involve many small/fast-moving features with

multiple possible matches for those features, we propose a global tracking tech-

nique to find the globally best match, which is defined by the match between two

adjacent feature sets with the minimal overall cost. A branch-and-bound algo-

rithm is proposed to find the globally best match efficiently.

• To amend the problems of previous feature tracking criteria, we propose to use

the Earth Mover’s Distance (EMD) as a better metric to match features. We use

EMD in the global tracking technique to calculate the matching cost.

• We propose a static view selection algorithm that takes into account the color,

curvature and opacity information of the corresponding rendering images.

• We propose an optimization algorithm to maximize the overall information per-

ceived by a dynamic view. The dynamic view also follows the smooth-movement

constraint.

1.4 Organization

The dissertation consists of seven chapters.

Chapter 2 reviews previous work on features tracking and view selection. Both the overlap-based and the attribute-based tracking methods are discussed.

Chapter 3 presents the algorithm for tracking isosurface and interval volume com- ponents by using higher dimensional isosurfacing.

Chapter 4 explains why and how the correspondence relationship of time-varying isosurfaces can be pre-computed. A tracking method using the pre-computed corre- spondence is also presented.

10 Chapter 5 presents the global tracking algorithm to track volumetric features. We describe how to use EMD as a better metric to match features, and a branch-and-bound algorithm to search the globally best match is also presented.

Chapter 6 presents our algorithm to select the best dynamic views. First a static view selection method based on color, curvature and opacity information is described.

Then we use an optimization approach to find the best dynamic view that offers the maximal amount of information about the time-varying features and also satisfies the smooth-movement constraint.

Chapter 7 summarizes the contribution of this thesis and proposes future directions.

11 CHAPTER 2

RELATED WORK

2.1 Feature Tracking

Researchers have proposed various techniques to track time-varying features. Banks and Singer [8] used a predictor-corrector method to reconstruct and track vortex tubes from turbulent time-dependent flows. Arnaud et al.[5] tracked 2D cloud patterns and used area overlap to determine correspondence. The tracking methods basically fall into two categories: aggregate attribute based methods [54, 55, 53, 48] and volume overlapping based methods [59, 60, 61, 62, 32, 31]. Next we will discuss these two categories of methods.

2.1.1 Aggregate Attribute Based Methods

The aggregate attribute based tracking methods represent features by their aggregate attributes such as centroids, masses, volumes, etc. These aggregate attributes, serving as a simplified model to characterize the features, are used to track how the features evolve over time. Many methods proposed in the computer vision and visualization literature use attributes to track features. Next we discuss three of these methods in more details.

12 In [54, 55], Sethi et al.tracked features by utilizing path coherence. The basic prop- erty of each feature is its position, but other properties such as volume and mass can also be included to describe the feature. The features are tracked by establishing path coherence in property space, i.e., attribute space. The tracking method starts with an initial trajectory configuration, where each feature is connected to its closest feature in property space. The path coherence is measured and an optimization process is applied.

Paths are refined iteratively by exchanging features in different path and calculating the gain of the exchange until the path with maximum gain is chosen. Two algorithms, namely, a greedy exchange algorithm and a simulated annealing algorithm are described to perform the optimization process.

Samtaney et al.[53] tracked features using their centroids, masses, volumes and cir- culations (in 2D). Their tracking algorithm involves the following steps:

• Extract features from data sets and maintain a list of attributes for each feature.

• For each feature in the first time step, calculate which feature from the second

time step is closest to it (using centroid) and test whether the volume, mass and

other attributes are within the prescribed tolerance. If a match is found, tag those

objects as continuation and remove them from the list.

• After all the features that continue have been removed, test combinations of two or

more features for bifurcation and amalgamation. Bifurcation and amalgamation

are determined by the difference between the average weighted centroid, total

volume, total mass, and other attributes of the combination with the original.

13 • All the left features are tagged as creation or dissipation, respectively, depending

on which time step the feature lies in.

Inspired by these two previous works, Reinders et al.[48] also calculated a set of at- tributes, such as centroid, volume, mass, best fitting ellipsoid, and skeleton for every fea- ture in every frame and used these data to track features through a prediction/verification scheme. Their tracking algorithm is based on an assumption that features evolve pre- dictably. The attributes of the features in the next time step are predicated using linear

(or higher-order) behavior rules, and the prediction is used to find corresponding feature in the next time step. All candidates in the next time step are tested for correspondence with the prediction. At the initialization stage, a correspondence is assumed between two features in two successive time steps. This assumption leads to a prediction that is compared with candidates in subsequent time steps. Interaction events such as bi- furcation and amalgamation are detected by testing a combination of features with the prediction. If a match is found and the prediction proves positive over a period of time, a path is created and the path is continued into subsequent frames. Otherwise, the current prediction is invalidated. A new prediction will be established and verified.

2.1.2 Volume Overlapping Based Methods

Silver and Wang [59, 60, 61, 62] observed that corresponding features in adjacent time steps usually overlap when the temporal sampling rate of the underlying model is high. Based on the observation, features manifested as interval volumes are first identified and stored in appropriate data structures. Then, correspondences between features in consecutive time steps are identified using a two-stage process including an overlap and a best matching test. In the overlap test, spatially overlapped features

14 from consecutive time steps are identified and the number of intersecting nodes is also computed. Octree and linked list data structures are used to accelerate this process.

The best matching test involves inspecting the ratio of the number of intersecting nodes versus the average volume among all combinations of overlapped features, with the combination of the maximum ratio as the corresponding feature(s).

Chen et al.[15] extended the work by Silver and Wang [59, 60, 61, 62] to track features in distributed AMR (Adaptive Mesh Refinement) datasets within a distributed computing environment. The resulting feature tree allows a viewer to watch how a multi-level isosurface changes over time, space and across different resolutions.

In [63], Sohn and Bajaj built contour tree based on the topology of time-varying isosurfaces. The resulting time-varying contour tree can be used to track how isosurfaces evolve.

Based on how the topological structure of a local feature evolves over time, one of the following events can occur: (see figure 2.1)

• Continuation: an object continues with possible shape deformation and change of

position, orientation, etc.

• Creation: a new object appears.

• Dissipation: an object disappears.

• Bifurcation: an object splits into several objects.

• Amalgamation: several objects merge into a single one.

It is also worth mentioning that there is a rich literature in computer vision on mo- tion tracking [7, 2, 58, 12, 29, 30, 45]. The main difference between tracking 2D objects

15 Figure 2.1: The evolutionary events that a feature may experience.

from videos and tracking features from simulation data is that features or regions of in- terest in scientific visualization applications are often manifested as 3D objects which tend to evolve and interact, while those 2D objects in computer vision interact less fre- quently. In [29, 30], Horn and Schunck introduced their pioneer work of optical flow to track 2D image features over video sequences. The optical flow assumed that the brightness of any part of the images remains constant over time. It also assumed that the image features undergo rigid motion or deformation, i.e., neighboring points have simi- lar velocities and the velocity field of the brightness pattern in the image varies smoothly almost everywhere. Based on these two constraints, they minimized an error term and generated the optical flow, which defines a velocity field in the image that transform one image into the next image in the sequence. Since its introduction, the work had inspired a substantial research effort and produced an abundance of techniques for calculating optical flows and tracking 2D image features.

16 2.2 View Selection

The study of viewpoint evaluation can be dated back to 1976, when Koenderink and van Doorn [38, 39] started a pioneer work and introduced the idea of aspect graph to partition the viewing regions surrounding an object. The node of the aspect graph is a stable view, around which the topology of the object projection does not change much.

The edge of the aspect graph represents a transition from a stable view to an adjacent one. The aspect graph defines the minimum number of views required to represent all the topologically different projections of the object. After its introduction, aspect graph has been studied intensively in computer vision, where many researchers used aspect graph for object recognition [17, 24, 4]. Recently there have been many works in computer graphics and scientific visualization about both static and dynamic view selections. We review these works in the next two sections.

2.2.1 Static View Selection Static View Selection for Polygonal Models

In computer graphics, several methods have been proposed to locate the optimal views for polygonal meshes. Kamada and Kawai [34] defined a view to be optimal if it minimizes the number of degenerated faces under orthogonal projection. However, this method fails to compare two views with the same number of degenerated faces, and it does not guarantee that the user can see a large amount of details [9]. Barral et al. [9] extended Kamada and Kawai’s idea and modified their coefficients to cope with perspective projection. They also added new parameters that take into account the projection area and the number of faces seen from each view.

17 Vazquez et al. [67, 68] formulated the quality of a viewpoint as the amount of infor- mation it gives about a scene. They novelly utilized the concept of entropy from Infor- mation Theory [56] to evaluate the amount of information the user can perceive from a scene at a given viewpoint. In order to evaluate the entropy of a polygonal model, the probability of each face is needed. Vazquez et al.used the relative visibility of each face as its probability. The information entropy function favors a view with uniform proba- bility distribution. Therefore, their method prefers a scene with more visible triangles and larger scene projection size.

Static View Selection for Volumetric Models

Recently researchers [64, 11] proposed methods to find the optimal view for vol- umetric data sets. Takahashi et al. [64] decomposed the volume into a set of feature interval volume components, and used the surface-based viewpoint selection method suggested in [67, 68] to find the optimal view for each of the components. Then they calculated the globally optimal view by a compromise among the locally optimal views of all the feature components.

In [11], Bordoloi and Shen took a volume rendering approach and used the concept of entropy (as in [67, 68]) to measure the quality of a viewpoint. Instead of using the area visibility as in [67, 68], they defined the probability of each voxel as its voxel visibility divided by its noteworthiness factor. The noteworthiness value, or the weight of the voxel, can be determined by factors such as the opacity and color of the voxel.

Based on the probability definition, the entropy function prefers a viewpoint where the visibility of each voxel is proportional to the noteworthiness value of the voxel. In their work, they also discussed view similarity and how to partition the view space.

18 2.2.2 Dynamic View Selection

There is a rich literature in computer graphics and animation about dynamic view selection [57, 3, 73, 28, 66]. Applicable techniques range from direct orientation in- terpolation [57] to complex view planning for complicated 3D scenes. Shoemake [57] suggested that in keyframe-based computer animations, quaternion is a better way to describe rotations of solid bodies and cameras than the common solution of using three

Euler’s angles. Shoemake then created a spline curve for quaternions defined on a unit sphere to smoothly interpolate sequence of arbitrary rotations. Andujar et al. [3] dealt with the problem of generating a camera path for walkthrough of complex scene mod- els. Their method is based on identifying the free-space structure of the scene and an entropy-based measurement of the relevance of a viewpoint. The entropy-based rele- vance metric decides which cells have to be visited and is also used to compute critical way-points inside each cell. Wernert and Hanson [73] discussed the camera path plan- ning based on a personal ”guide” that keeps the user oriented in the navigation space which also points to interesting subject area. Hong et al. [28] studied how to select cam- era path to navigate in the human colon. In [67], Vazquez et al.tried to find a camera path that can automatically explore a scene composed of polygonal meshes. They first put the camera at an arbitrary starting point and specify a direction of navigation. The subsequent moves of the camera are chosen among three possible new directions, with the direction that gives the maximum viewpoint entropy and the largest number of faces not yet visited as the next view.

In [66], van. Wijk and Nuij studied the exploration of large 2D information spaces such as maps, images, or abstract visualizations. These 2D information spaces often

19 require views at various levels to inspect details and obtain overviews. They introduced an elegant method to generate a smooth animation from one close-up view to another close-up view by zooming and panning. Two factors, smoothness and efficiency, are used to defined the optimal path. They formulated the factors into a computational model and provided an analytic solution for the optimal animation.

There are two major differences between our work and the previous work. First we deal with the problem of view selection for time-varying data where the underlying phenomena are changing over time. Second our problem uses a different scene setting from the previous work, where our views move on a viewing sphere and look at the center of the sphere. Our goal is to maximize the information perceived from the time- varying data while following the view movement constraints.

In [11], Bordoloi and Shen considered the problem of finding a good viewpoint for time-varying dataset. However, their method is to find a static view point throughout the animation so that the user can perceive the maximum summation of conditional entropy from the time series. The conditional entropy is the relative entropy of a datastep based on its previous step. Compared with the method, our method tries to find a dynamic viewing path.

20 CHAPTER 3

VOLUME TRACKING USING HIGHER DIMENSIONAL ISOSURFACING

3.1 Overview

In this chapter we present a volume tracking algorithm based on higher dimensional isosurfacing. Our goal is to allow interactive tracking of local features. A local feature is defined in this chapter as a connected component that belongs to an isosurface or interval volume. When a scientist is presented with a rendering of an isosurface or an interval volume at a particular time step, he or she can select a connected component of interest, and then follows its evolution over time. Instead of computing the whole isosurfaces or interval volumes in the subsequent time steps, our algorithm generates on the fly only the overlapping components to reduce the computation cost. In the following, we describe our algorithm to track isosurface and interval volume components in detail.

3.2 Isosurface Tracking

Our algorithm is based on higher dimensional isosurfacing. Computing isosurfaces or interval volumes in R4 allows us to easily track overlapped local features. In the following, we first describe the algorithm to track time-varying isosurfaces.

21 3.2.1 Extracting Overlapping Time-varying Isosurfaces

To track isosurfaces in a time-varying field, instead of first computing a complete set

of isosurfaces in the time sequence and then identify the component that overlaps with

the user selected local feature, better efficiency can be achieved if we only compute the

overlapped components from adjacent time steps. To do so, we will extract isosurfaces

in R4. Previously, researchers have proposed algorithms to extract isosurfaces in R4 from

time-varying data [10, 72, 71]. Smoother surface animations can be produced since the

isosurface in R4 can be sliced at higher temporal resolutions. In our algorithm, we use

isosurfacing in R4 to detect the overlapping isosurfaces. This is based on the following

statement:

• If two isosurface components from the consecutive time steps overlap with each

other in the spatial domain, they will belong to the same connected isosurface

component in R4.

3 Based on the statement, given a user selected isosurface component Oi in R at time t, we can rapidly compute the overlapping isosurface component(s) at time t +1 by first

4 extracting the connected isosurface component in R that intersects with Oi, and then

slicing it to get the isosurface component(s) at time t + 1. The statement is critical for

the correctness of our algorithm. Next we will first show the validity of the statement.

Validity of the Statement

3 Suppose an isosurface component Oi in R at time t overlaps with another isosurface

3 component O j in R at time t + 1. Each isosurface component goes through a list of 3-

cells (cells in R3). We can build 4-cells (cells in R4) out of the 3-cells at time t and t +1.

22 Oi

3 1 2 4

Oj

Figure 3.1: The four different types of 3-cells when two isocontours in R2 intersect.

Each 4-cell is made up of two 3-cells that have the same spatial location but one from t and the other from t + 1 . There are two cases on how these two isosurface components in R3 overlap.

• 1. Oi and O j have surface intersection.

• 2. Oi and O j have no surface intersection, but one is completely inside of the

other.

For the first case, each 4-cell in the time-varying field can be classified into one of the following four categories: (For 2D case, see Figure 3.1)

• 1. The cell passes through Oi at t but not O j at t + 1.

• 2. The cell passes through O j at t + 1 but not Oi at t.

23 • 3. The cell passes through Oi at t and O j at t + 1.

• 4. The cell passes through neither Oi at t nor O j at t + 1.

Note that the different categories are mutually exclusive and all four categories com- prise the entire 4-cells in the time-varying field. Type 1, 2 and 3 4-cells are isosurface cells in R4, since they contains isosurface in R3, while type 4 4-cells are not isosurface cells in R4. Any type 3 4-cell can propagate to any type 1, 2 and 3 4-cell; any type 1

4-cell can propagate to any type 1 and 3 4-cell; and any type 2 4-cell can propagate to any type 2 and 3 4-cell. Type 1 and type 2 4-cells can propagate to each other only if there exists a type 3 4-cell. Therefore, if Oi at t has surface intersection with O j at t + 1 in space, that is, there exists a type 3 4-cell, then there will be a connected component

4 in R that passes through Oi at t and O j at t +1. In other words, Oi and O j belong to the same connected component in R4.

In the second case, Oi has no surface intersection with O j, but the volumes inside the surfaces overlap. Without loss of generality, we assume Oi lies within O j. The 4-cells connecting them can be classified into the following four categories: (For 2D case, see

Figure 3.2)

• 1. The cell passes through Oi at t but not O j at t + 1.

• 2. The cell passes through O j at t + 1 but not Oi at t.

• 3. The two corresponding 3-cells lie outside of Oi but inside of O j.

• 4. The two corresponding 3-cells lie either inside Oi or outside O j.

24 Oi

+ −

4 2 1 3 + −

Oj

Figure 3.2: The four different types of 3-cells when one isosurface is contained within the other.

Similar to the above analysis, these four categories of 4-cells are mutually exclusive and all 4 categories comprise the entire 4-cells in the field. It can be seen that type 1 and

2 4-cells are isosurface cells in R4, since they contain isosurfaces in R3. For each type 3

4-cell, the signs of its corners at t are different to those at t+1 and thus the isosurface in

R4 will cut through the 4-cell, which implies type 3 4-cells are also isosurface cells in

R4. Type 4 4-cells have the same sign for every corner and thus are not isosurface cells in R4. Type 3 4-cells can propagate to type 1 and 2 4-cells while type 1 and 2 4-cells can propagate to each other by going through some type 3 4-cells. An isosurface component

4 in R that passes through Oi at t and O j at t + 1 lies exactly within all 4-cells of type 1,

2 and 3.

25 It is worth mentioning that here we assume that the scalar values inside the isosur-

faces in R3 at the consecutive time steps are consistent relative to the isovalue, i.e., all of

them are either smaller or greater than the isovalue. It may happen that the scalar fields

get reversed, that is, at one time step, the scalar values inside the isosurface are smaller

than the isovalue but at the other time step they are greater than the isovalue. This

happens when the scalar field changes too quickly, or when two concentric isosurface

components are tracked. In both cases, the isosurface in R4 will not pass through the

region between these two isosurfaces in R3, and it will have two disjoint components.

Specifically, when the scalar field changes too quickly, the isosurface component at time t will disappear and then another isosurface component at time t + 1 starts to appear. In

the case that two concentric isosurface components are tracked, the fact that these two

components in R3 do not belong to the same isosurface component in R4 guarantees that

the inner component at time t will be tracked to the inner component at time t + 1 and

the outer component at time t to the outer one at time t + 1.

Extracting Isosurfaces in R4

In our algorithm, isosurfaces in R4 are used to detect the overlapping isosurfaces in

R3. The isosurface in R4 can be computed by propagation from any 4-cell that inter-

sects with the user selected isosurface component in R3. When the underlying mesh is

a Cartesian grid, the 4-cell is a hypercube. If the mesh is a tetrahedral grid, then the cell

is a 4-simplex. Previously, researchers have used cell propagations to compute isosur-

faces in R3 [6]. We can perform cell propagation in R4 in a similar manner. Note when

propagating 4-cells, for a large time varying data set, it may be impractical to get the full

isosurface component in R4, since it is very likely that the component will span many

26 time steps and it will incur serious memory overhead to generate it. In our implementa- tion, we only propagate to generate the necessary part of isosurface components which lie between time t and time t + 1.

During the propagation, the 4-cells that contain the isosurface need to be triangu- lated. Previously, Weigle and Banks [71] proposed a recursive contour meshing algo- rithm, which extracts isosurfaces in R4 by first breaking the 4-cell, or hypercube into

4-simplicies, and then looping through the faces of the simplices to construct the isosur- faces. Bhaniramka et al.[10] devised an algorithm that can generate triangulation tables for arbitrary N dimensional cells. Both algorithms can be used to compute isosurfaces in

R4. Since the algorithm by Bhaniramka et al.produces a smaller number of tetrahedra, we choose their method in our algorithm and create the triangulation table for 4-cells at a preprocessing stage. Once the table is generated, isosurfaces in R4 can be generated at run time by a method similar to the Marching Cubes algorithm [43]. The resulting isosurface is a collection of tetrahedra embedded in R4. To extract the overlapping iso- surface components in the spatial domain, we can just slice the connected isosurface component in R4 at the desired time step.

3.2.2 Verification

Correspondences between isosurface components from adjacent time steps can be easily found by using isosurfacing in higher dimensions. The idea behind is that if components overlap with each other, they must belong to the same isosurface component in higher dimension. One issue needs to be addressed is that this higher dimensional isosurfacing approach does not discriminate the degree of overlap between components.

However, sometimes users might want to set a threshold so that only components that

27 have a significant overlap are concluded as having correspondence. In this section, we describe how we take this user-defined overlap threshold into account.

Two isosurface components in consecutive time steps correspond to each other if they can be generated by slicing the same isosurface component in R4. However, in some cases when the overlap between the two components is too small, the user may not want to consider them as corresponding objects. That is, component Oi and O j correspond to each other if and only if:

• 1. They are generated by slicing the same isosurface component in R4.

• 2. V(overlap(Oi, O j)) / Min(V(Oi), V(O j)) > Threshold where Threshold is a user-supplied parameter, V() returns the volume of the component, and V(overlap(Oi, O j)) gives the volume of the overlap between two components.

We measure the volume of an isosurface component in R3 based on the number of

3D cells lying inside the surface. For any cell inside the isosurface, its eight vertices should have an identical sign, i.e., they are either all greater, or all smaller than the isovalue. Furthermore, the signs are consistent for all the inside cells. Therefore, the volume of an isosurface component can be computed by first finding an inside cell, and then using the cell as a seed to flood the entire volume inside the isosurface. Cells on the boundary of surface also contribute to the volume and should be processed properly.

A good approximation of their contribution to the whole volume would be the ratio of corners that lie inside the surface to the total eight corners.

The volume of the overlapped region for two isosurface components in R3 at two consecutive time steps is the number of 3D cells that lie in both components. This can be easily calculated by counting the number of cells that belong to both components. This

28 volume is used to test whether the degree of overlap between the isosurface components satisfies the user’s threshold. If the threshold is satisfied, the correspondence holds; otherwise there will be no correspondence.

3.3 Interval Volume Tracking

Features are often manifested as points within a range of data values, which can be represented as an interval volume [26, 20]. An interval volume is a generalization of isosurface and represents a three-dimensional subvolume for which the associated scalar values lie within a user-specified closed interval. When there are errors caused by mea- surements or numerical simulations, or the target object contains structural ambiguity, displaying interval volumes can provide more meaningful visualization results [21].

To track interval volumes in a time-varying data set, we can generalize the idea of isosurface tracking described above. That is, given a user selected interval volume

4 component Vi at time t, we can extract an interval volume in R that intersects with Vi, and then slice the interval volume in R4 along the time dimension to get the interval volume component(s) at time t + 1 that overlap with Vi.

3.3.1 Extracting Overlapping Time-Varying Interval Volumes

To extract interval volumes that overlap with a user selected component Vi at time t,

4 we will first extract an interval volume in R that intersects with Vi, which will be then sliced along the time dimension to get the interval volume component(s) at time t + 1 that overlap with Vi. Previously, researchers have proposed algorithms to extract interval volumes in R3. Fujishiro et al.[21] extended the Marching Cubes algorithm and used a solid fitting algorithm to tetrahedralize the interval volume. The algorithm proposed by

29 Nielson et al.[46] performed the tetrahedralization by decomposing each volume cell to

five tetrahedra and then using an efficient lookup table to compute the interval volume within each tetrahedron. Bhaniramka et al.[10] extracted the interval volume by creating two scalar fields from the original data and then constructed an isosurface in R4. The projection of the isosurface in R4 to the spatial domain is the interval volume. All the above algorithms are for computing interval volumes in 3-space. In the following, we present an algorithm to compute interval volumes in 4-space.

Computing Interval Volumes in R4

Our method is based on five dimensional isosurfacing. Given a four dimensional scalar field f(x,y,z,t), the interval volume I[a,b] consists of all the points that satisfy a ≤ f (x,y,z,t) ≤ b. To compute the interval volume, we first artificially create a five dimen- sional scalar field g(x,y,z,t,w) with the w dimension equal to two. We let g(x,y,z,t,0) = f (x,y,z,t) − a and g(x,y,z,t,1) = f (x,y,z,t) − b. We also assume the five dimensional scalar field varies linearly along the w dimension, that is,

g(x,y,z,t,w) = (1 − w) × g(x,y,z,t,0) + w × g(x,y,z,t,1) where w is between 0 and 1. The interval volume a ≤ f (x,y,z,t) ≤ b can be extracted by first computing the zero isosurface from the 5D scalar field g, and then projecting the resulting isosurface along the w axis to 4-space. The correctness of this algorithm is shown as follows.

The zero isosurface of the five dimensional field g contains the 5D points (x,y,z,t,w) that satisfy g(x,y,z,t,w) = 0. Hereafter we call those points zero isosurface points. For the zero isosurface points that have w=0, if we project them orthographically along the

30 w direction, i.e., project from (x,y,z,t,0) to (x,y,z,t), we know that the projected point

(x,y,z,t) will satisfy f (x,y,z,t) = a since g(x,y,z,t,0) = 0 = f (x,y,z,t) − a. Similarly,

for the zero isosurface points that have w = 1, i.e., (x,y,z,t,1), we have g(x,y,z,t,1) = 0.

Since g(x,y,z,t,1) = f (x,y,z,t) − b, if we project those points along the w axis, we

know the projected point (x,y,z,t) will satisfy f (x,y,z,t) = b. Finally, for those zero

isosurface points that have w ∈ (0,1), since g(x,y,z,t,w) = (1−w)×g(x,y,z,t,0)+w×

g(x,y,z,t,1), if we project those points to 4-space, it can be seen that a < f (x,y,z,t) <

b will be satisfied. Therefore, if we project the zero isosurface points (x,y,z,t,w) to

four dimensions along the w axis, we will get all the points (x,y,z,t) that satisfy a ≤

f (x,y,z,t) ≤ b, i.e., the interval volume I[a,b] in R4.

Given a time-varying field, remember that the goal of computing the interval volume

in R4 is to enable the tracking of an interval volume in R3 from time step t to time step t + 1. Therefore, the construction of the five dimensional scalar field g(x,y,z,t,w) need

not be performed globally. Only the volume cells in R5 that will be encountered during

propagation need to be constructed. Hence, the space and computation overhead is

manageable. To compute the zero isosurface in R5, we can use the triangulation table

generated with the algorithm by Bhaniramka et al.[10]. Constructing isosurfaces in R5

requires a five dimensional triangulation table. In the next section, we discuss some

practical issues related to the generation of the table.

Triangulation Table for isosurfacing in R5

Similar to the Marching Cubes lookup table, the 5D triangulation table provides the

edge intersection and connectivity information to guide the triangulation process. Each

entry in the table contains the geometry configuration consisting of four-simplices to

31 represent the isosurface in the 5D hypercube. A 5D hypercube is composed of two

4D hypercubes defined in xyzt spatio-temporal dimensions, which are connected by 16 edges lying in the w direction. We used the algorithm proposed by Bhaniramka et al.[10] to generate the table. We also augment the table to include the propagation directions based on the case’s geometry configuration. Since a 5D hypercube contains 32 vertices, the size of the table can be quite large. Therefore, care should be taken when using the table at run time.

An N dimensional hypercube contains 2N vertices. Without performing case reduc- tion as done in the Marching Cubes algorithm, the number of cases for triangulation

N table will be 22 . For instance, in the 3D triangulation table there are 28 = 256 cases.

For a 5-cell, the number of vertices will be 32. Therefore, a 5D isosurface lookup table will contain 232=4G entries. The 256 cases in the 3D marching cubes can be reduced to 15 after applying complementary and rotational symmetry. However, in four dimen- sions, further efforts must be taken to ensure a consistent triangulation on the surfaces of adjacent 4-cells. This problem will become even more complex when the dimen- sion grows to five, which makes it very difficult to apply complementary and rotational symmetry.

When the 5D triangulation table is used to compute interval volumes in R4, not all the four billion cases are possible. In the process of computing the interval volume a ≤ f (x,y,z,t) ≤ b, the scalar field at w = 0 is generated by subtracting the 4D field by a, and the scalar field at w = 1 is generated by subtracting the 4D field by b. Since b > a, for each of the 16 edges in the 5D hypercube lying in the w dimension, the scalar value at w = 0 is always greater than the scalar value at w = 1. This implies that

32 + + − W=0

+ − − W=1

Figure 3.3: Only 3 sign combinations are possible when extracting interval volumes in R2.

the edges of the 5D hypercube lying in the w direction can not have a minus-plus sign combination, otherwise the scalar value at w = 0 will be smaller than the scalar value at w = 1. Hence, there are only 3 possible sign combinations, rather than 4, for each edge of the 5D hypercube in the w direction. For this reason, the 5D triangulation table used for interval volume generation can only contain 316 ≈ 43M, rather than 4G case. The case in R2 is illustrated in Figure 3.3.

Although the 5D triangulation table is reduced from 4G cases to 43M, its size is still too large to be processed in core. However, given a data set, we find out that some cases occur more frequently, and some other cases never happen. The reduction from 4G cases to 43M and the uneven possibility of the occurrence of each case make it feasible and effective to store the 5D triangulation table into a hash table which is small enough to fit into main memory. With a properly designed hash function, the hash table will cache almost all 5D triangulation cases that can happen for a data set. In this way, the extraction of 4D interval volumes can be achieved in an efficient manner.

33 3.3.2 Verification

A verification approach similar to that of isosurface component can be applied to tracking of interval volumes. The only difference would be the volume calculation. The volume enclosed by an isosurface is the whole region inside the isosurface; while the volume of an interval volume is the region lying between the top isosurface and the bottom one. An interval volume in R3 can be generated by projecting an isosurface in

R4 along the w direction. Notice that this isosurface in R4 goes through a 4-cell if and only if the interval volume in R3 goes through the projection of the 4-cell along the w direction, which is a 3-cell. So the number of 3-cells passed by an interval volume in R3 is equal to the number of 4-cells passed by the corresponding isosurface in R4. Hence, the volume of the interval volume in R3 can be easily calculated by counting the number of 4-cells on the isosurface in R4. After all the necessary volumes are calculated, we can test whether the degree of overlap between the interval volume components satisfies the user’s threshold. If it is satisfied, the correspondence holds; otherwise there will be no correspondence.

3.4 Results

We have tested our tracking algorithm using a 128 × 128 × 128 vorticity magnitude data set with 100 time steps. All results were computed on a Pentinum IV 1.4GHZ PC with 768 Mbytes memory. In this section, we demonstrate the results of our algorithm for tracking both isosurfaces and interval volumes.

Figure 3.4 shows an example of tracking an isosurface that consists of multiple com- ponents. The time-varying isosurface is extracted with an isovalue 6.0. The resulting

34 isosurfaces in R4 are used to guide the coloring of the connected components, which al- lows us to distinguish different local features when following their evolutions over time.

The color of each component is inherited from its parent in the previous time step. When a component is created, a new color is generated. When two or more components merge into a component, the current component would get the color of the previous dominant component. The dominance could be determined by properties such as volume, mass or local extremal value. In our result, we use volume as the criterion to determine the dominance, i.e., the current component would follow the color of the previous largest component when amalgamation happens.

Figure 3.5 shows the tracking of a local isosurface component. Snapshots from six time steps are shown. The isovalue of these components is 6.55. When one or a few components are selected, only the components from the subsequent time steps that overlap with the selected features are extracted using the four-dimensional isosurfacing algorithm. In Figure 3.5, the component bifurcates into two at t=10. These two com- ponents continue to t=17. The smaller component disappears at t=18 while the larger one continues to t=27 and disappears at t=28. The tracking time and the 4D isosurfac- ing time are shown in Table 3.1. The tracking time includes extracting the connected isosurface components in R4 across the adjacent time steps, calculating the volumes of the components and their overlaps, and slicing the isosurfaces in R4 back to three di- mensions. Several factors influence the performance of our tracking algorithm. Among them, the sizes of the generated geometries in both R3 and R4 from the adjacent time steps dominate. This shows that the complexity of our tracking algorithm is dependent

35 Time step(t) 9 10 17 18 24 27 Tracking time 0.07 0.08 0.071 0.06 0.04 0.01 4D isocont- ouring time 0.02 0.02 0.02 0.01 0.01 0.00 Num of tetrahedra 20873 21566 16980 15502 5195 99 Num of tri- angles at t 4340 4452 3936 3564 1256 32 Num of tri- angles at t+1 4452 4524 3564 3272 628 32

Table 3.1: The time to track the isosurface component illustrated in Figure 3.5 (in seconds).

on the feature size rather than the size of the data. The sizes of these geometries are also show in Table 3.1.

Our algorithm can also track all interval volume components as well as individual interval volume component effectively. Figure 3.6 shows the tracking of two interval volume components. Snapshots from five time steps are shown. The interval volume was generated with isovalues between 5.3 and 7.0. We rendered the interval volume by drawing the faces of every tetrahedron. These two components first merge at t=29 and then split at t=31. All the timing and the sizes of geometries are shown in Table 3.2.

Besides visualization, it is useful to know other properties such as the volume and mass of the features and how they change over time. This information can be used by scientists to gain further insight into the features. In our algorithm, the volumes of the components are computed in the verification process. Other properties can be computed similarly. As an example, the variations of the volume as a function of time for the case of isosurface component tracking in Figure 3.5 is illustrated in Figure 3.7.

36 Time step(t) 27 28 29 30 31 Tracking time 0.20 0.26 0.271 0.29 0.29 5D isocont- ouring time 0.09 0.13 0.14 0.15 0.15 Num of 4- simplices 102160 142393 160689 174276 172887 Num of tetra- hedra at t 25059 25070 29507 34132 35702 Num of tetra- hedra at t+1 25070 29507 34132 35702 34577

Table 3.2: The time to track interval volume components illustrated in Figure 3.6 (in seconds).

The overlapping threshold plays an important role in identifying the correspondence between features from adjacent time steps. An appropriate value should depend on the drifting speed of the features as well as the data sampling rate. If the threshold is set too high, many features would be identified as creation/dissipation instead of contin- uation/amalgamation/bifurcation since no correspondence could be established. If the threshold is too low, features that overlap little would also be identified as continuation.

In our test, we set the overlap threshold to 35% which leads to good event classification.

3.5 Summary

In this chapter we present a novel volume tracking technique based on higher dimen- sional isosurfacing. Local features defined as connected isosurface or interval volume components are tracked in an efficient manner. This is achieved by first extracting the connected components in higher dimension that intersects with the selected local com- ponents and slicing them to obtain the overlapping components at the next time step.

37 The computing cost of performing higher dimensional isosurfacing is minimized, since an output sensitive propagation method is used.

38 t=3 t=6

t=9 t=12

t=15 t=18

Figure 3.4: 6 out 100 time steps are shown. All isosurface components are tracked and the evolutionary information determines the coloring of the components.

39 t=9 t=10

t=17 t=18

t=24 t=27

Figure 3.5: An isosurface component is tracked. Bifurcation happens at t=10 and dissi- pation at t=18 and 28.

40 t=27 t=28

t=29 t=30

t=31

Figure 3.6: Two interval volume component are tracked. These two components merge at t=29 and then split at t=31.

41 Figure 3.7: Volume vs time for the example in Figure 3.5. Note the component bifur- cates into two and each of them dissipates.

42 CHAPTER 4

EFFICIENT ISOSURFACE TRACKING USING PRECOMPUTED CORRESPONDENCE TABLE

4.1 Overview

In this chapter, we track isosurfaces with even greater efficiency by using a pre- computed correspondence lookup table. It is based on the fact the correspondence rela- tionship between isosurfaces of two consecutive time steps can change only at critical isovalues in R3 or R4 and remains unchanged for those values between any two adja- cent critical isovalues. Here critical isovalues are the isovalues at which the isosurface will change its number of components. Therefore, the correspondence relationship be- tween isosurfaces from any two consecutive time steps will only change a finite number of times and thus the correspondence lookup table contains finite entries and can be precomputed. With this correspondence table, isosurface tracking can be achieved by simple table lookup operations. These operations virtually take no time and thus our algorithm has minimal overhead. In the following, we will first explain the reason why the correspondence relationship changes in this manner, and then introduce our method to build the correspondence lookup table and explain how to utilize it in the isosurface tracking process.

43 4.2 Isosurface Tracking

4.2.1 Tracking by Using Higher Dimensional Isosurfacing

If a data set has been sampled well temporally, according to Silver and Wang’s ob- servation, corresponding isosurface features between two adjacent time steps overlap.

A straightforward way to find out how an isosurface component evolves is first extract- ing all the isosurface components in the next time step and then performing an overlap test with each of the components. This could be very time consuming, since it is very likely that there exist a large number of components in the next time step and only a very small number of components overlap with the tracked isosurface. In the previous chap- ter, we tracked isosurfaces efficiently by using higher dimensional isosurfacing. The key observation that leads to efficient tracking of isosurfaces in R3 is that the isosurface component and the component(s) it overlap(s) with in the next time step belong to the same isosurface component in R4. This isosurface component in R4 can be extracted by propagating any 4-cell that intersects with the tracked isosurface component within these two time steps. From this property, in order to track where an isosurface compo- nent in R3 evolves, our algorithm [32] first extracts the isosurface component in R4 that contains the tracked isosurface component in R3. We then slice the resulting isosurface component in R4 at the next time step to get the isosurface component(s) in R3 which correspond(s) to the tracked isosurface component in R3.

44 4.2.2 Generation of the Correspondence Lookup Table

By using higher dimensional isosurfacing, we greatly improved the efficiency of tracking local isosurface components. However, there is still a large amount of over- head involved. The major part of the overhead is due to isosurfacing in higher dimen- sions (R4). If the isosurface contains multiple components and only few components are tracked, the overhead is small. However, in some cases where the tracked isosurface components contain a large part of the whole isosurface, the overhead of extracting the isosurface in R4 could make the method slower than the straightforward method. In this section, we will introduce a method to build a correspondence lookup table which con- tains the correspondence information between isosurfaces of two consecutive time steps across the whole value range. With this correspondence table, isosurface tracking can be achieved by simple table lookup operations which take almost no time. Therefore, our isosurface tracking algorithm has minimal overhead.

To facilitate our discussion, we define that two isosurface components correspond to each other if and only if they belong to the same isosurface component in R4. Now let’s examine the correspondence between the isosurfaces of these two time steps across the whole value range, i.e., from the minimum data value to the maximum value of both time steps. We will see that the isosurfaces at both time steps change their shapes con- tinuously, with possible topology changes at critical isovalues. We should also observe that the correspondence relationship between these two isosurfaces changes in the fol- lowing manner: the correspondence starts at some configuration at the minimum value and remains the same for some value interval. Then at some value the correspondence

45 changes and remain unchanged for another interval. This procedure repeats until we reach the maximum value, where both isosurfaces vanish.

A 2D example is illustrated in figure 4.1. At the minimum value of both time steps, there is no isosurface component at either time step. So the correspondence relationship is empty. Just above the minimum value, a red component appears at time step t0. The correspondence relationship changes to {Cred → empty}. As the isovalue increases, the red component changes its shape at t0 and no new components appear at t1 and thus the correspondence relationship remains unchanged. As the isovalue continues to in- crease, a green component appears at t1. Since the red component overlaps with the green one therefore they belong to the same isosurface component in R4, the correspon- dence relationship changes to {Cred → Cgreen}. Again, the red component and the green one change their shapes as the isovalue increases, but the correspondence relationship remains the same. But when the isovalue reaches an isovalue v0, the red component splits into a yellow component and a blue one. The yellow component has no overlap with the green one but the blue one does. The correspondence relationship changes to

{Cyellow → empty, Cblue → Cgreen}. This correspondence relationship will remain un- changed until the green component splits into a grey component and a black one, and both overlap with the blue component. Now the correspondence relationship changes to

{Cyellow → empty, Cblue → Cgrey, Cblue → Cblack}. This procedure will repeat until these two isosurfaces vanish at the maximum value.

A natural question is which values cause the correspondence relationship to change.

We will assume that every field value at the grid points of the data set is unique for each pair of adjacent time steps. This could be easily satisfied by a small perturbation

46 Isovalue = v0 Isovalue = v1

Isovalue = v2 Isovalue = v3

Figure 4.1: Four snapshots to illustrate how the correspondence relationship between isosurfaces from two consecutive time steps changes when the isovalue increases from v0 to v3 (v0 < v1 < v2 < v3).

to equivalent field values at the grid points for any real data set. We will also assume that the underlying data field is composed of simplicial meshes. We know that at some

47 isovalue, the isosurface in R3 would change its number of components, i.e., a compo-

nent may merge with another component, or it may split into several components. A

new component may appear, or a component disappears. We name these events critical

events and these isovalues critical isovalues in R3 [70, 22, 40, 65, 47, 14, 13]. This also

happens to isosurface in R4, that is, when the isosurface in R4 changes its number of

components at some isovalues, we call these isovalues critical isovalues in R4 (Please

note that within this chapter, the isosurface in R4 is only extracted within two consecu-

tive time steps). A key property that leads to our efficient isosurface tracking is that the

correspondence change can only occur at the critical isovalues in R3 or R4. That is to

say, the correspondence change can only occur when the number of components of the

isosurface in R3 or R4 changes. To show this is true, we show the following observation.

Observation: Assume an isosurface component in R3 is generated by slicing an

isosurface component in R4. When the isovalue changes and no critical event happens

in R3 or R4, the isosurface component in R4 changes into another new isosurface com-

ponent in R4. Likewise the isosurface component in R3 changes into a new isosurface

component in R3. The new isosurface component in R3 can still be generated by slicing

the new isosurface component in R4.

Let’s say an isosurface component Ci at t0 and another isosurface component Cj at t1 correspond to each other, i.e., they belong to the same isosurface component Ck in

R4. If no critical event happens in R3 or R4 when the isovalue changes, based on our

observation, Ci, Cj and Ck all change but after the change of isovalue, Ci and Cj can

still be generated by slicing Ck, i.e., they belong to the same isosurface component in

R4. Therefore, they still have correspondence. We can see that when isovalue changes

48 Isovalue = v0 Isovalue = v1

Figure 4.2: Two previously corresponding components do not correspond to each other any more and the number of components in R3 remains the same when the isovalue changes.

and if no critical event occurs in R3 or R4, two previously corresponding components will still correspond to each other. Therefore, the correspondence relationship between isosurface components of two adjacent time steps can only change at critical isovalues in R3 or R4. There appears to be several counter cases to this argument. We will disprove some of the counter cases in the following.

Case 1: In figure 4.2 (Figure 4.2, 4.3 and 4.4 are 2D examples), the isosurfaces at both time steps contain only one component. Initially they correspond to each other.

However, when the isovalue changes, all of a sudden they have no correspondence any more. The number of components in R3 remains the same and we will show that the number of components in R4 must change, i.e., this case can not happen without change of number of isosurface components in R4.

49 Isovalue = v0 Isovalue = v1

Figure 4.3: A plausible case where two pairs of components switch their correspondence without the change of number of isosurface components in R3 and R4 when the isovalue changes.

These two components in figure 4.2 originally correspond to each other, so they

belong to the same isosurface component in R4. After the isovalue is changed, the

component at t0 corresponds to an empty component at t1 and an empty component at t0

corresponds to the component at t1. It means that these two components belong to two

different isosurface components in R4. Therefore, there must be a split of the original

isosurface component in R4 which leads to the change of number of components in R4,

i.e., this case can not happen without change of number of isosurface components in R4.

Case 2: In figure 4.3, the isosurfaces at both time steps consist of two components.

At some isovalue, the circle component at t0 corresponds to another circle component

at t1 and the diamond component at t0 corresponds to another diamond component at t1. However, when the isovalue changes, suddenly the correspondence switches. This

50 seems to be a reasonable case where the correspondence changes but the number of components in R3 and R4 remains the same.

Originally the two diamond components belong to the same isosurface component in

R4 and the two circle components belong to another component in R4. When the isovalue changes, the diamond components changes into new diamond components and the circle components into new circle components. Since the two diamond components belonged to the same component in R4 before the change of isovalue and assuming no change of number of components in R3 or R4 therefore no critical event occurs, according to the observation, the two new diamond components will still belong to the same isosurface component in R4, i.e., the two diamond components still correspond to each other. So do the two circle components. Therefore, the case in figure 4.3 cannot happen without change of number of components in R3 or R4.

Case 3: In figure 4.4, the isosurface at both time steps are composed of three compo- nents. Two components at t0 merge into one component at t1 and the other component at t0 splits into two at t1. When the isovalue is changed, the correspondence changes as illustrated in figure 4.4. Intuitively, this could also happen and the correspondence changes without change of number of components in R3 or R4.

Originally the square and circle components at t0 and the square component at t1 be-

4 long to an isosurface component in R . The diamond component at t0 and the square and

4 circle components at t1 belong to another component in R . After the isovalue changes, the circle component at either time step belongs to a different isosurface component in

R4, which implies at least two critical events happen simultaneously. In contradiction, we have assumed that every field value at the grid points of the data set is unique for

51 Isovalue = v0 Isovalue = v1

Figure 4.4: Another seemingly reasonable case: one component at each time step changes its correspondence when the isovalue changes but without the change of num- ber of isosurface components in R3 and R4.

each pair of adjacent time steps and the underlying data field is composed of simplicial meshes. Since critical events can only happen at vertices of simplices, we get a conclu- sion that at any isovalue, critical events can only happen at most in one time step in R3, or at most between one time interval in R4. Therefore it is true that at most one critical events can happen among all the time steps. There is a contradiction here. So the case in figure 4.4 cannot happen.

Based on our analysis above, the change of correspondence relationship can only oc- cur at the critical isovalues in R3 or R4. Since there are finite critical isovalues in R3 and

R4, the correspondence relationship will only change a finite number of times across the whole value range. Thus the correspondence table will have a finite number of entries and can be precomputed. Between two adjacent critical isovalues, the correspondence relationship will remain the same. Therefore, a representative value within the interval

52 can be used to calculate its correspondence relationship. The correspondence table can then be used in isosurface tracking. When an isosurface component is selected by the user, we can query which component(s) it corresponds to by simply looking them up in the correspondence table using the isovalue and the ID of the selected component as the index. The tracking overhead is minimized because only a simple table lookup is involved.

Putting it into another perspective, we know that contour trees [40, 65, 47, 14, 13] describe the topology of a scalar field. A contour tree is composed of supernodes and superarcs. Supernodes are vertices of the scalar field where the number of isosurface components changes. The values of supernodes are critical isovalues. Superarcs con- nect two supernodes and within a superarc the associated isosurface component will not change its number of components. The correspondence relationship between two isosurfaces of any two consecutive time steps across the whole value range can be rep- resented by connecting the contour trees of these two scalar fields. The correspondence can only change at either the supernodes of two contour trees in R3 or the supernodes of the contour tree of the 4D field, which is the composition of these two 3D fields.

Actually a good way to store the correspondence relationship between isosurfaces of two consecutive time steps within two adjacent critical isovalues is to store the superarc correspondence, since there is a one-to-one correspondence between superarcs and iso- surface components. An advantage of this scheme is that the superarc provides a good way to enumerate the isosurface component and it can also be utilized to generate the seed from which the isosurface component can be propagated.

53 4.2.3 Optimization in Generation of the Correspondence Table

Based on the fact that the correspondence can change only at critical isovalues in R3 or R4 and remains constant between two consecutive critical isovalues, the correspon- dence relationship will only change a finite number of times and thus the correspondence table will only have finite entries and can be precomputed. A brute force method to gen- erate the table can work as follows: First find all the critical isovalues in R3 and R4 and sort them by field values. Within every two adjacent critical isovalues, the corre- spondence relationship remains constant. Therefore, a representative value such as the midvalue can be used to retrieve the correspondence relationship. Many methods can be used to find the correspondence between isosurface components of two adjacent time steps at a given isovalue [32, 60, 62]. The correspondence relationship from all adjacent critical isovalue intervals forms the correspondence table.

The correspondence table contains all of the correspondence relationships between isosurfaces of two consecutive time steps across the whole value range. Within every two adjacent isovalues, the correspondence relationship remains constant. At either end of the interval, due to the existence of a critical point in R3 or R4, the correspondence

3 may change. If a critical event in R happens, it means that a component at either t0 or t1 appears, disappears, merges with another component or splits into several components.

Other components at this time step retain their topology. Since no critical event occurs to these components, they will correspond to the same components as in the previous critical isovalue intervals in the correspondence table. Therefore, there is no need to recompute their correspondences and only components whose topology changes need to recompute their correspondence. If a critical event in R4 happens, a component in

54 R4 changes its topology. Recomputation of correspondence only needs to be performed for those components in R3 which belong to the affected component in R4. Therefore, by utilizing the value coherence, the correspondence relationship can be updated with respect to the previous critical isovalue interval in the correspondence table. This differ- ence updating method has two advantages. First, the computation time can be reduced significantly, since no computation is wasted on components whose correspondence re- mains the same. Second, since two adjacent critical isovalue intervals share most of the correspondence, only the correspondence relationship for components whose corre- spondence changes needs to be stored. This information is very small compared with the correspondence relationship of all components. By utilizing the value coherence, the computation time and storage overhead of the correspondence table can be significantly reduced.

4.2.4 Isosurface Tracking Using Correspondence Lookup Table

The correspondence lookup table records all the correspondence information be- tween isosurfaces of two adjacent time steps across the whole value range. After it is built, it can be used in the isosurface tracking process. We know that each isosurface component corresponds to exactly one superarc in the contour tree. An isosurface com- ponent can be generated by propagating a seed cell which is found by using the seed information associated with the superarc. Therefore, when an isosurface component is selected, its corresponding superarc can be found immediately. Recall that the corre- spondence table stores the isosurface component correspondence information as super- arc correspondence. So we can query the correspondence table with the current isovalue and the superarc the selected component belongs to and the result is the superarc(s) the

55 current superarc corresponds to. The corresponding isosurface component(s) can then be generated by propagating seed cell(s) found by using seed information associated with the superarc(s). Compared with the previous chatper where the corresponding iso- surface components are identified by isosurfacing in R4, our current method only needs simple table lookup operations to find the corresponding isosurface components. Note in both algorithms, isosurfacing in R3 is needed to extract the corresponding components.

Notice that the corresponding isosurface components generated in this way do not distinguish the degree of overlap. A verification stage similar to the previous chapter can be performed to guarantee that the overlap between the tracked isosurface component and the corresponding isosurface component is significant enough to be considered as correspondence.

4.3 Results

We have tested our algorithm using a 128 × 128 × 128 vorticity magnitude data set with 99 time steps. The machine we used was a Pentium IV 1.4GHz PC with 768MB memory.

Figure 4.5 shows an example of tracking a whole isosurface that consists of mul- tiple components. The time-varying isosurface was extracted with isovalue 6.3. The correspondence tables between isosurfaces of every two consecutive time steps are pre- computed. At run time, these tables are used to determine the correspondence among isosurface components. In figure 4.5, a component is colored in such a way that it has the same color as the components it would evolve into. When a component is created, a new color is assigned to it. When two or more components merge into a component, the component would get the color of the previous dominant component. The dominance

56 t=5 t=6 t=7 t=8

t=9 t=10 t=11 t=12

Figure 4.5: All isosurface components are tracked and the tracking information deter- mines the coloring of the components. 8 out of 99 time steps are shown.

could be determined by properties such as volume, mass or local extremal value. In our results, we used volume as the criterion to determine the dominance, i.e., the current component would follow the color of the previous largest component.

Figure 4.6 shows the tracking of two isosurface components. These two time varying components were extracted with isovalue 6.2. Snapshots from six time steps are shown.

Again the precomputed correspondence tables are used to determine which components the current one would evolve into. In figure 4.6, these two component merges into one component at t=4 and it then bifurcates into two at t=15. The larger component bifurcates again at t=21. At t=22, the smallest component dissipates. Figure 4.7 also

57 Time step 5 6 7 8 9 Total time 0.401 0.42 0.421 0.431 0.431 Correspondence determination time 0.00 0.00 0.00 0.00 0.00 Component extraction time 0.341 0.35 0.351 0.361 0.361 Verification time 0.06 0.07 0.07 0.07 0.07 Overhead 0.06 0.07 0.07 0.07 0.07 Size of tracked component(s) 148K 148K 152K 154K 155K Size of corresponding component(s) 148K 152K 154K 155K 155K

Table 4.1: The time to track the whole isosurface illustrated in figure 4.5 (in seconds). Timing from 5 time steps are shown due to space limitation.

shows the tracking of another two isosurface components. The isovalue of the isosurface was 6.9.

The total tracking time and how the time breaks down are also shown in table 4.1 for the case in figure 4.5, table 4.2 for the case in figure 4.6 and table 4.3 for the case in figure 4.7. The total tracking time includes the correspondence determination time, isosurface component extraction time and the verification time. The correspondence determination time is used to determine which components correspond to the current component. Since only very simple table lookup operations are performed, the time is minimized and our result shows 0 all the time, i.e., it is too small to be measured by the system clock. Isosurface component extraction time is the time to extract the corresponding isosurface component, which involves operations such as finding a seed

58 Time step 4 9 15 21 Total time 0.04 0.05 0.05 0.03 Correspondence determination time 0.00 0.00 0.00 0.00 Component extraction time 0.03 0.04 0.04 0.03 Verification time 0.01 0.01 0.01 0.00 Overhead 0.01 0.01 0.01 0.00 Size of tracked component(s) 13.6K 17.5K 18.5K 13.2K Size of corresponding component(s) 15.1K 18.0K 17.8K 10.7K

Table 4.2: The time to track two isosurface components illustrated in figure 4.6 (in seconds). Timing from time step 4, 9, 15 and 21 are shown due to space limitation.

associated with a superarc and propagating the seed to get the component. Verification time is used to verify if the current component has significant overlap with the generated component. If the overlap is significant, the correspondence holds; otherwise there will be no correspondence. Overhead shown in each table is the sum of correspondence de- termination time and verification time and it indicates the cost of tracking. The running time of our tracking algorithm is proportional to the size of the corresponding isosurface components. The size information is also shown in each table. The complexity of our tracking algorithm is dependent on the feature size rather than the size of the data. Fur- thermore, compared with our previous work [32] in which the corresponding isosurfaces are identified by isosurfacing in R4, our current work minimizes the correspondence de- termination time to almost 0. Other overhead is also highly reduced. For the cases in

59 Time step 1 2 6 7 14 Total time 0.01 0.02 0.02 0.02 0.02 Correspondence determination time 0.00 0.00 0.00 0.00 0.00 Component extraction time 0.01 0.01 0.02 0.02 0.02 Verification time 0.00 0.01 0.00 0.00 0.00 Overhead 0.00 0.01 0.00 0.00 0.00 Size of tracked component(s) 3.5K 4.3K 6.3K 7.0K 8.1K Size of corresponding component(s) 4.3K 5.0K 7.0K 7.4K 7.7K

Table 4.3: The time to track two isosurface components illustrated in figure 4.7 (in seconds). Timing from time step 1, 2, 6, 7 and 14 are shown due to space limitation.

figure 4.6 and figure 4.7 where two isosurface components are tracked, the overhead is reduced by at least 5 times compared with our previous work. The overhead reduction rate will be even larger when more components are tracked.

The precomputed correspondence lookup tables are used in all of our examples for the correspondence query. We have two methods to generate the lookup table, the brute force method and the optimized difference updating method. In table 4.4, we show the comparison of these two methods in terms of running time and output file size. In this test, we only generate the correspondence table between the maximum value and 6.0 for every two consecutive time steps. The reason is simply that it takes too long for the brute force method to finish the computation of the correspondence relationship across

60 Time interval 0-1 1-2 2-3 3-4 4-5 Running time speedup 16.9 18.8 16.3 15.3 16.0 Output file size reduction rate 5.1 5.5 5.4 5.0 4.1

Table 4.4: The comparison between two methods to generate the correspondence lookup table.

the whole value range. For the optimized difference updating method, the average out- put file size is 8K and the average running time is 17 seconds, with approximately 2 percent of the total critical isovalue intervals processed when the correspondence tables between the maximum value and 6.0 are generated. We can see from table 4.4 that the optimized method performs much better in terms of both running time and generated

file size. Actually if the correspondence tables for the whole value range was gener- ated, the performance difference would be bigger. The reason is as follows. Recall that the brute force method extracts the correspondence relationship between all iso- surface components from two adjacent time steps within each critical isovalue interval and stores it to disk. Conversely the optimized method will only extract the correspon- dence difference with the previous interval and only the difference is stored into disk.

Furthermore, for the vorticity data set, the complexity of the isosurface increases as the isovalue decreases. For example, at time step 0, the isosurface at isovalue 8.0 only con- tains 10 components and 30K triangles. But the isosurface at isovalue 4.0 contains 43 components and 600K triangles. So the brute force method will become slower and gen- erate a larger size of correspondence relationship as the isovalue decreases. In contrast, the optimized method only deals with the difference with respect to the previous value

61 interval, which contains a very small number of components (usually 1 or 2). Its run- ning time remains almost constant across different intervals and the same is true for the size of generated correspondence relationship. Therefore, with the whole value interval computed, we should see a much bigger performance difference. An estimation of the performance difference between these two would be at least 30 times in running time and 10 times in generated file size.

4.4 Summary

In this chapter, we introduce an efficient algorithm to track isosurfaces. A precom- puted correspondence lookup table enables us to find out how an isosurface component evolves by simply table lookup operations with the isovalue as the index. The rationale behind our approach is that the correspondence relationship between isosurfaces of two adjacent time steps can change only at critical isovalues in R3 or R4 and remains un- changed between adjacent critical isovalues. The computation of the correspondence lookup table is also optimized with better efficiency and less storage overhead. By us- ing this correspondence lookup table, isosurface tracking is performed in an extremely efficient manner with minimal overhead.

62 t=3 t=4

t=9 t=15

t=21 t=22

Figure 4.6: Two isosurface components are tracked. These two components merge into a component at t=4 and it then bifurcates into two at t=15. The larger component bifurcates again at t=21. At t=22, the smallest component dissipates. 63 t=1 t=2

t=6 t=7

t=14 t=15

Figure 4.7: Two isosurface components are tracked. Amalgamation occurs at t=2; bi- furcation occurs at t=7 and dissipation at t=15.

64 CHAPTER 5

FEATURE TRACKING USING EARTH MOVER’S DISTANCE AND GLOBAL OPTIMIZATION

5.1 Overview

Given two feature sets in the adjacent time steps with the source feature set S =

{S0,S1,...Sn−1} and the destination feature set D = {D0,D1,...Dm−1}, one way to es- tablish the correspondence between S and D is to associate every feature in S with the feature(s) of its locally best match in D. The locally best match can be defined by ei- ther the volume-overlapping [59, 60, 61, 62, 32, 31] or the similar-attribute criterion

[53, 48]. While these local tracking techniques are effective for various applications, there are many cases where the local matching techniques do not offer the globally best result (The term ”globally” refers to the case that all the features from two adjacent time steps are considered together during feature matching). For example, in the example shown in figure 5.1(a), the locally best matching feature for S1 is D0 because they have the maximal overlap, and their centroid positions and volumes are also the most similar.

However, globally S1 should correspond to D1 because what happens is that the entire feature set moves downwards in figure 5.1(a). Similarly, in the example shown in fig- ure 5.1(b), both features in S shrink as time evolves. If S0 is the first to be matched, it

65 (a) (b)

Figure 5.1: Two cases to illustrate that the local tracking techniques do not guarantee the globally best matching result.

will correspond to D0 and D1. This is because they have the maximal overlap with S0, and also the most similar centroid positions and volumes. However, globally only D0 should be matched to S0 . From these two examples, we can see that the local matching scheme does not always produce the globally best matching result.

To address the problem, we propose to use a global matching technique that takes into account the configuration of the features all together. We assume a cost is properly defined to match any source feature component to any destination feature component.

The goal of the global matching is to find the minimal cost to match the whole source feature set S to the destination feature set D. The matching result with the minimal cost will be the globally best match between S and D with a properly-designed cost function.

A feature component may undergo the following evolutionary events:

66 • Continuation: a single feature at t matches to another single feature at t + 1.

• Amalgamation: a group of features at t match to a single feature at t + 1.

• Bifurcation : a single feature at t matches to a group of features at t + 1.

• Creation : an empty feature at t matches to a single feature at t + 1.

• Dissipation : a single feature at t matches to an empty feature at t + 1.

When a group of feature components are involved in an evolutionary event such as amalgamation and bifurcation, we call them a compound component. Notice that although the five evolutionary events above do not include matching between two com- pound components, the matching between two compound components can be split into combinations of the five evolutionary events. If we denote Cost(A,B) as the minimal cost to match A to B, where A = {Si,Si+1,...Sn−1} and B ⊆ D, the following recursive function holds for Cost(A,B):

min

TA⊆A,TB⊆B Cost(Si,TB)+Cost(A−Si,B−TB) Cost(A,B)= { (5.1) Si∈TA,D j∈TB Cost(TA,D j)+Cost(A−TA,B−D j) |TA|6=1 where TA can be an empty or a compound component in A containing Si, and TB can be an empty, single or compound component of B. This equation states that the minimal cost to match the source feature subset A to the destination feature subset B is the mini- mum of the following five scenarios: (We list the corresponding evolutionary event that

Si undergoes in each scenario)

• TB = /0. In this case, Si disappears;

• TB is a single component. In this case, Si continues;

67 • TB is a compound component. In this case, Si splits;

• TA = /0. In this case, a new component (D j) appears.

• TA is a compound component containing Si. In this case, Si merges with some

other components and becomes D j at the next time step.

Notice that we do not allow TA to contain just only one component, since otherwise the continuation event will be considered twice in equation 5.1.

There are two key issues in the formulation and search for the globally best match.

First, we need to have a proper cost function to match any source and destination feature component. The evaluation of matching cost should accurately reflect the unlikelihood of matching a source component to a destination component. Furthermore, the definition should be robust enough to take all the cases into account. Second, equation 5.1 actually defines a recursive search tree structure. The root node of the tree is Cost(S,D). We need to have an efficient way to calculate Cost(S,D) and find the globally best match.

In the following sections, we first describe our definition of the matching cost, and then describe an efficient branch-and-bound algorithm to calculate Cost(S,D).

5.2 Using Earth Mover’s Distance as Matching Cost

It is essential to have a proper cost function when matching the source and des- tination feature components. The matching cost should take proper consideration of the important characteristics of the feature components, such as their shapes, orienta- tions, scales, and positions. The cost definition should also be robust in the sense that it should always give a meaningful cost. Previous researchers have mainly used either

68 Figure 5.2: A thin feature is moving fast. The volume-overlapping criterion will not give a meaningful matching cost in this case.

the volume-overlapping or similar-attribute criterion to match features. If the volume- overlapping criterion is used, the matching cost can be defined as a function which is inversely proportional to the overlapping degree of the source and destination feature components. However, the volume-overlapping criterion cannot handle the case where two corresponding features do not overlap, for example, when a small and/or thin fea- ture moves quickly so that there is no overlap between their volumes in adjacent time steps (see figure 5.2). Such a definition fails to give a meaningful cost in this case. The similar-attribute criterion describes a feature by its aggregate attributes, such as the cen- troid position, volume and mass. Two features correspond when they have close centroid positions and similar volumes and masses. If the similar-attribute criterion is used, the definition of the matching cost would be a function which is proportional to the centroid

69 Figure 5.3: A figure to illustrate why EMD is a better metric than the aggregate-attribute approach.

distance, and the volume and mass difference. But these aggregate attributes can over- simplify the feature description. Specifically, the aggregate attributes do not take proper consideration of the shape and orientation of the feature, and this can lead to inaccurate matching cost measures. For example, in figure 5.3, the two features have the same centroid positions, volumes (areas) and messes, but different shapes. The aggregate- attribute approach gives an inaccurate matching cost, since the shape information is not properly considered.

Aiming to address the above problems, we propose to use the Earth Mover’s Dis- tance (EMD) metric [51, 52] to measure the matching cost between any source and destination feature components. In the EMD computation, each feature is represented by the set of cells it occupies in space, and EMD calculates the minimal amount of work required to transport one set of cells into the other. EMD takes a unified approach to measure the matching cost by taking into account the feature’s spatial occupancy in- formation, which includes all the shape, orientation, position and scale attributes of a

70 feature (Specifically, even when the features do not overlap). Therefore, it is more ro- bust than the volume-overlapping criterion and considers more attributes properly than the aggregate-attribute criterion. Furthermore, the aggregate-attribute criterion does not answer the question of how to unify the difference from different attributes together during the calculation of matching cost, for example, how to appropriately integrate the centroid distance and the volume ratio together into the cost definition. In the following sections we first give a brief review of the Earth Mover’s Distance and explain how it is used in our feature tracking algorithm.

5.2.1 Earth Mover’s Distance

Distribution information is often used in many areas to represent features. For exam- ple, in computer vision, images are often represented by a one-dimensional distribution of image intensities or three-dimensional distribution of image colors. In scientific visu- alization, a volumetric feature can be represented by the set of cells it occupies in space.

To measure the difference (or distance) between two distributions, Rubner et al.[51, 52] propose the Earth Mover’s Distance (EMD). EMD reflects the minimal amount of work that must be performed to transport one distribution into the other. Intuitively, we can treat one distribution as a mass of earth properly spread in space and the other as a col- lection of holes in the same space. EMD measures the minimal amount of work needed to fill the holes with earth. We can always assume there is enough earth to fill the holes, since otherwise we can switch what we call earth and hole. Here, a unit of work cor- responds to transporting a unit of earth by a unit of distance. Formally, EMD assumes that the ground distance function, which measures the cost to transport a unit of the

71 source distribution into a unit of the destination distribution, is given. EMD then lifts the distance between the individual units to the whole distribution.

In our tracking algorithm, we represent a feature by the cells it occupies in space.

For example, isosurfaces [43] extracted from a volume data can be represented by all the cells (isocells) that contain the isosurface patches. Interval volumes [26, 20] can be represented by all the cells lying within the user-specified value range. We use EMD to measure the distance between two features.

In our algorithm, the Euclidean distance between cells in space is used as the ground distance. EMD naturally lifts the distance between two individual cells to define the dis- tance between two features. When two features are close in space with similar shapes and orientations, the EMD value between these two features will be small since they have very similar spatial occupancies. In contrast, if two features are far away and/or have different shapes and/or orientations, the EMD value between these two features will be large, since their spatial occupancies are different. In our implementation, we penalize more on matching two features that are far away. We used the following non- linear ground distance function GD to achieve this:

P GD(Ci,Cj) = EuclideanDistance(Ci,Ci) (5.2)

where Ci and Cj are two cells, and P > 1 is used to penalize matching features that are far away.

Notice that EMD allows partial match. This means if we match a feature distribution

F to another distribution which is a subset of F, the EMD value will be 0. The property will be useful during the identification of compound component candidates (this will be explained in section 5.3.4). However, when calculating EMD between any source and

72 destination feature components, partial match can cause some problem. For example, the EMD value between a feature and any of its subset which has a smaller feature size will be 0 if partial match is allowed, and this is undesired. Moreover, the matching cost definition should penalize matching two features with different scales. To remedy this problem, whenever we calculate the EMD value between any two features, we will make sure the two features have the same capacity. To achieve this, we first assign uniform capacity to each cell of a feature. When matching two features, we scale the capacity of all the cells of each feature uniformly so that the two features to be matched will have the same capacity. By the scaling operation, we avoid the partial matching problem and EMD will penalize matching features with different scales. Actually EMD is a true metric when comparing distributions with the same capacity.

5.2.2 Efficient EMD Computation

Computing EMD is based on a well-known linear programming problem, the trans- portation problem. The transportation problem defines a bipartite network with S as a set of suppliers and C as a set of consumers. Xi(0 ≤ i ≤ n) is the total supply of supplier i and Yj(0 ≤ j ≤ m) is the total capacity of the consumer j. Ti j is the cost to ship a unit of supply from supplier i to consumer j. The transportation problem is to find a set of

flows Fi j that minimize the overall cost:

∑(0≤i≤n) ∑(0≤ j≤m)(Ti j × Fi j) subject to the following constraints:

Fi j ≥ 0 (0 ≤ i ≤ n and 0 ≤ j ≤ m)

∑(0≤i≤n) Fi j = Yj (0 ≤ j ≤ m)

∑(0≤ j≤m) Fi j ≤ Xi (0 ≤ i ≤ n)

73 Figure 5.4: A feature (in a 16 × 16 grid)) and its simplification (in a 4 × 4 grid)).

The constraints force the consumers to fill up all of their capacities and each supplier can not transport more than its supply. We assume the total demand does not exceed the total supply, as we can switch what we call supplier and consumer if necessary. Efficient algorithms are available to solve the transportation problem. In our implementation, we used the transportation-simplex method which is a streamlined algorithm based on the simplex method [27]. Actually we can further improve the efficiency of the EMD computation. During the EMD computation between two distributions, it is unnecessary to use the whole distribution. Researchers have used a so-called signature [51, 52] of the distribution to further speed up the EMD calculation.

Now we describe a method to simplify the feature distribution information. This is very similar to the vertex clustering algorithm to simplify meshes [50]. To accelerate the EMD computation, we put the features into a coarser grid by simply merging n3 cells into a single large cell (A 2D example is shown in figure 5.4). This large cell may contain many feature cells at the original resolution. We use the average location of the feature cells as the location in the merged cell, and the capacity of the merged cell

74 equals to the total capacity of the feature cells in the original n3 cells. The process can be efficiently done by a linear scan of all the feature cells. After the simplification, the size of the feature distribution is reduced significantly. However, the simplified feature distribution maintains the same signature with the original one, that is, they show the same spatial distribution pattern. So the EMD values using the simplified features will be just as accurate.

EMD provides an accurate matching cost between any source and destination feature components. Another important issue is how to calculate Cost(S,D) efficiently. In the next section, we will explain how we achieve this by using a branch-and bound search algorithm.

5.3 Global Optimization Based on Branch-and-Bound

5.3.1 The Search Tree

Equation 5.1 implicitly defines a search tree structure. The root of the tree is Cost(S,D).

We assume that the feature components are considered sequentially from S0 to Sn−1.

Following equation 5.1, at the first level of the search tree, each node represents one of the following matching possibilities that involve the first feature S0:

• Match S0 to an individual component in D.

• Match S0 to a compound component in D.

• Match S0 to an empty component.

• Match any compound component containing S0 to an individual component in D.

75 Figure 5.5: An example of two feature sets to be matched.

In the example shown in figure 5.5 where we want to find the correspondence be- tween three source features and three destination features, we will consider the following possible matches at the first level of the tree: S0 → D0, S0 → D1, S0 → D2, S0 → D12,

S0 → /0, S01 → D0, S01 → D1, and S01 → D2. For simplicity, we limit the possible com- pound components to only S01 and D12 (A method to identify compound component candidates will be introduced in section 5.3.4). From the first level of the tree, following each branch we will consider matching the first feature in the remainder of the source features at the second level. Similarly to the first level, all the four combinations listed above will be considered as long as the feature has not been covered in the ancestor nodes. Using the same example, when following the branch of S0 → D0, the following matches will be considered at the second level: S1 → D1, S1 → D2, S1 → D12, S1 → /0.

Notice that all matches involving the compound component S01 are excluded following this branch since S0 has already been matched in the parent node. Any matches to D0 are excluded for the same reason. This process will repeat until all the source feature components have been matched. Note that there might be still some destination features

76 Figure 5.6: Part of the search tree for the case in figure 5.5.

remaining with no correspondence. We then match an empty component to each of the remaining destination features and add these to the search tree. We can see that the search tree considers all the possible match combinations. The match with the minimal overall cost, which defines a unique path from the root to one of the leaf nodes, provides the globally best match for each source feature component.

If we follow the previous description to build the tree, the size of the tree can be huge. Accordingly it will take a long time to find the minimal Cost(S,D). In the next two sections, we introduce methods to reduce the size and accordingly the search time

77 of the tree. We first present a branch-and-bound method that is able to significantly reduce the size of the search tree. Then a way to get a tight estimation of the minimal cost is introduced to further speed up the process. We also describe a method to identify compound component candidates.

5.3.2 Branch and Bound

The search tree contains all the possible matches between the source and destination feature sets. At each branch of the tree, we consider all the matches which correspond to all the evolutionary events that can happen to the first feature in the current source feature set. We calculate the EMD value for each match and sort the matches based on the EMD value in an ascending order. The tree is traversed using a depth-first method, with the node that has the smallest EMD traversed first at each level of the tree. This enables a branch-and-bound operation, which can significantly reduce the size of the search tree.

During the depth-first traversal of the tree, we keep the minimal cost of all the le- gitimate matches (MinLgtCost) that have been found so far. A match is legitimate if and only if every component in the source and destination feature sets occurs in the match exactly once. Whenever a legitimate match is reached, we use its cost to update

MinLgtCost if the cost is smaller. This MinLgtCost can be used to prune the tree in the following way. As the tree is being traversed, at each node we evaluate the total cost of the path from the root to the current node, and store it as CostSubsets. CostSubsets is the overall cost to match a subset of the source features and a subset of the destination features. If CostSubsets is smaller than MinLgtCost, the depth-first traversal continues.

Otherwise the entire branch following the current node can be pruned. This is because

78 Figure 5.7: An example to illustrate how the branch-and-bound operation is applied to enable efficient tree search.

if the cost to match only the subsets of the features (CostSubsets) is already larger than

MinLgtCost, all the matches following the current node will certainly have matching cost larger than MinLgtCost, and hence none of these matches can be optimal.

Figure 5.7 shows an example of this branch-and-bound operation. The search pro- cess currently reaches the node S1 → D0 and the MinLgtCost = 30. Suppose the match- ing cost of the path from the root to the current node, which is a summation of the costs

S0 → D1 and S1 → D0, is 35. If the search process keeps traversing down the node

S1 → D0, all the legitimate matches found along the branch will have cost greater than

35, and of course greater than MinLgtCost, which is currently 30. Therefore, there is no need to traverse down this branch. The pruning operation can be applied to remove this branch.

79 In fact, whenever a node with its current overall cost (CostSubsets) greater than

MinLgtCost is found out, an even larger portion of the tree can be pruned away in addi- tion to the branch following the node. Recall that for each source feature, we have sorted all of its matching candidates based on their EMD values in an ascending order. Thus, at the current level of the tree, there is no need to try any remaining matching candidates of the current source feature, since the overall cost of the path from the root to any of those nodes will be larger than CostSubsets, and hence larger than MinLgtCost. An example is also illustrated in figure 5.7. The branch at the node S1 → D0 has already been cut away. We know that the matching candidates of S1 have been sorted in the EMD-value ascending order. Therefore, for any matching candidates Di of S1 that appear behind D0, the matching cost of S1 → Di is greater than that of S1 → D0, therefore, the cost of the path from the root to any of these matching candidates is greater than 35. None of the legitimate matches along these branches can have cost smaller than MinLgtCost (30).

Therefore, all these branches can be pruned right away.

5.3.3 Further Speedup by a Good Estimation of the Minimal Cost

Branch-and-bound is a very powerful operation. As soon as a good evaluation of the current minimal cost (MinLgtCost) is reached, it can be used to cut away the search tree efficiently. When the global optimization is performed by the branch-and-bound operation, MinLgtCost of all the legitimate matches found so far keeps decreasing until a global minimum is reached, which defines the globally best match. We can initialize

MinLgtCost to an arbitrary large value and let the tree pruning process find the global minimum. However, a tighter bound of MinLgtCost can accelerate the tree pruning process significantly. If MinLgtCost starts with a very large value, at the beginning stage

80 of tree traversal, very few pruning operations can be applied. In fact, it may take many runs to reach a reasonable MinLgtCost. However, a tighter MinLgtCost can accelerate the tree pruning process from the very beginning and speeds up the whole process.

We find a tighter bound of MinLgtCost using a greedy method. First among all the matches between any source and destination feature components, we find the match with the smallest EMD. This may be a match between two single components, between single and compound components, or between single and empty components. After the match is discovered, we remove the features associated with this match from the feature space.

Then the above process repeats recursively in the remainder of the feature space. When at some point either the source or destination feature set becomes empty, we will match an empty component to those remaining unmatched components. Although the result of this greedy process is not the global minimum in most cases, it is a good estimation of

MinLgtCost. Using this estimation, the tree pruning operation can be applied from the very beginning of the tree buildup process. The whole search process therefore can be substantially accelerated using this good estimate of the minimal cost.

5.3.4 Using EMD to Identify Compound Component Candidates

To identify all the evolutionary events, at each branch of the search tree, we need to consider matches between single and empty components (for creation and dissipation), between two single components (for continuation), and between single and compound components (for amalgamation and bifurcation). For better efficiency, we need to have a good way to identify compound component candidates, rather than trying every possible combination of the feature components.

81 Our method to identify compound component candidates is based on the EMD’s property that EMD allows partial matching. Suppose feature Oi and O j form a com- pound component and they merge to feature Ok in the next time step. If we represent

Oi, O j and Ok by the cells they occupy in space and assign the weight of each individ- ual cell to be 1, the EMD value between feature Oi and Ok (EMDik) and that between feature O j and Ok (EMD jk) should be small compared to the EMD value between other components and Ok. In addition, the summation of EMDik and EMD jk should be close to the EMD value between Ok and the union of Oi and O j. Furthermore, compound component should lie within the close neighborhood of each other. Notice that we want

EMD to allow for partial matching here, therefore, during the identification of com- pound components, we do not scale the capacity of any feature distributions, which was used to make their capacities equivalent.

In our algorithm, we stay conservative during the identification of compound com- ponents. For any features that map to the same feature Ok with small EMD values and the summation of the EMD values is also close to the EMD value between the union of these features and Ok, we consider it as a compound component candidate. Addi- tional criterion such as spatial closeness can be applied to rule out some plausible cases.

We may identify some false-positive cases, but this will be taken care of by the global optimization algorithm. Notice that compound components identified by this method are only candidates to identify possible amalgamation/bifurcation. Whether amalgama- tion/bifurcation takes place still depends on the result of the global optimization.

It is also necessary to calculate the EMD value between a feature component and an empty component, which is used to detect creation/dissipation. We assume the empty

82 component be a cell in space with its weight equal to that of the feature component being matched. The centroid of the empty component, i.e, the position of the cell, can be predicated by using the trajectory along which the feature has evolved. The Euclidean distance can be used as the ground distance function. In our implementation, we want to penalize more on creating or dissipating large components. So the nonlinear function in equation 5.2 is used as the ground distance.

In order to detect amalgamation/bifurcation, we need to calculate the EMD value between single and compound components. This is simply achieved by merging the distribution of each individual component in the compound component as the distri- bution of the compound component. Then the EMD computation can be applied in a straightforward way.

5.4 Results

We have tested our algorithm using a 1283 time-varying force field data set and a vorticity magnitude data set of the same resolution. The machine we used was a Pentium

XEON 3GHz with 3G main memory.

The force field data set was generated by distributing ellipsoidal sites in space. The force of each point in space is the summation of the force it gathers from every site.

Each ellipsoidal site has its own weight, which determines its influence to any given point in space. The force from an ellipsoidal site is anisotropic, i.e, stronger in some directions and weaker in other directions. Those ellipsoidal sites are moving in space with their weights changing too. Figure 5.8 shows the tracking result from the time- varying force field. The feature we track is the time-varying isosurface with an isovalue of 0.9. In the figure, each feature is colored in such a way that it has the same color

83 t=0 t=1

t=2 t=3

Figure 5.8: Time varying features from a force field data set are tracked and the tracking result determines the coloring of the features. Four snapshots are shown.

as the feature it evolves from. When amalgamation happens where a feature evolves from multiple features, the current feature will get the color of the previous dominant feature. The dominance is defined by volume in our example, i.e., the current component would follow the color of the previous largest feature from which it evolves. In the case that a new feature is created, a new color is assigned to it. We also numbered part of

84 the features for later explanation. These numbers are generated by feature extraction software and do not reflect the correspondence relationship between features.

There are multiple thin/small or fast-moving features in this example. For instance,

6 F0 (feature 6 at t=0) is a thin feature. Although it is not moving very fast, it does not 6 0 5 overlap with F1 which is the feature it will evolve into. F0 and F0 are small components which move fast relative to their own sizes. They have no overlap with the features they will evolve into either. It is hard for the tracking algorithms based on volume overlapping to detect the correspondence. But by using the EMD metric and global optimization, our algorithm detected the correspondence for those features correctly, as

1 2 1 2 shown in the figure. In t=2, feature F2 and F2 move quickly. Feature F2 and F2 should

1 4 correspond to F3 and F3 , respectively. However, from the overlapping relationship,

1 1 2 1 2 F3 overlaps with both F2 and F2 and the overlap between F3 and F2 are larger than

1 1 that between F3 and F2 . Therefore, it is very likely that the tracking algorithms based

1 2 1 4 on volume overlapping correspond F3 to F2 and treat F2 and F3 as dissipation and creation respectively. If the features are matched by using attributes such as centroid

1 2 and volume, F3 is the closest to F2 and they have similar volumes. Therefore, tracking

1 2 algorithms based on similar-attribute are likely to correspond F3 to F2 as the volume- overlapping algorithms. By taking the global configuration into account and performing a global optimization based on the EMD values, our algorithm gave the correct result, as illustrated in the figure.

Timing results for the previous example are shown in table 5.1. The EMD values of some features are also shown in table 5.2. For each feature, the first four smallest

EMDs are shown. We can see that the EMD value between a feature and the feature(s)

85 Time step 0-1 1-2 2-3 EMD computation time 0.042 0.037 0.029 Global optimization time 0.039 0.031 0.041

Table 5.1: The timing (in seconds) for the case in figure 5.8.

Feature 1st EMD 2nd EMD 3rd EMD 4th EMD 3 1 3 1 3 10 F0 F1 F1 F1 F1 F1 (2.65) (4.03) (8.46) (22.09) 6 6 13 10 F0 F1 DIS F1 F1 (7.68) (23.25) (36.30) (40.61) 11 13 7 0 9 F0 F1 F1 F1 F1 (3.29) (23.65) (33.67) (34.10)

3 6 11 Table 5.2: The EMD values for F0 , F0 and F0 . The first four smallest values are shown. DIS means a feature dissipates.

it will evolve into is very small (in many cases it is the smallest). The EMD value between the feature and other features is usually much larger. This property makes the branch-and-bound method very efficient.

We also performed tests on the vorticity magnitude data set. The features we tracked are the time-varying isosurface with an isovalue of 6.9. Features are colored in the same way as in the previous example and the color reflects how the feature evolves. Four snapshots from the tracking results are shown in figure 5.9. Our global tracking algo- rithm gives accurate results with high efficiency. The result of the volume-overlapping based algorithms depends on a threshold value which measures what degree of overlap- ping should be considered as correspondence. If the threshold is set too high, there will

86 t=1 t=3

t=5 t=7

Figure 5.9: Time varying features from the vorticity data set are tracked and the tracking result determines the coloring of the features. Four snapshots are shown.

be many creation/dissipation events. If the threshold is too small, features may get clas- sified as continuation when they are not. Similar problems exist for the similar-attribute based algorithms. The result of those algorithms depends on the threshold values that are used to determine if the centroid positions, volumes, and masses are within the toler- ance. However, by taking the global configuration of the features into consideration, our

87 Time step 0-1 1-2 2-3 3-4 4-5 EMD computa 0.063 0.079 0.079 0.094 0.093 -tion time Global optimi 0.042 0.039 0.010 0.012 0.017 -zation time

Table 5.3: The timing (in seconds) for the case in figure 5.9.

algorithm can determine the tracking results effectively without using threshold values.

Timing results for the example in the figure 5.9 is shown in table 5.3.

5.5 Summary

In this chapter, we propose a global optimization algorithm to track time-varying features. Our algorithm is readily used to track any features that can be represented by the spatial distribution of cells. The algorithm can also be easily extended to track any features which can be represented by some other distribution information so that

EMD can be used to measure feature dissimilarity. EMD is used as a better metric to measure the matching cost between any source and destination feature components. An efficient global optimization process is applied to find the globally best match between two feature sets. The efficiency comes from the branch-and-bound search algorithm, and a tighter estimate of the minimal cost further speeds up the search.

88 CHAPTER 6

DYNAMIC VIEW SELECTION FOR TIME-VARYING VOLUMES

6.1 Overview

The previous three chapters address the problem of how to track features efficiently and accurately. Beside tracking features, another important problem of studying time- varying features is to have an effective way to view the evolution of time-varying fea- tures. Due to the time-varying nature, animation remains the most general and common way to render time-varying features. However, time-varying features often undergo complex evolution over time, with their shapes, positions and orientations changing continuously. Therefore, it is a key issue to find the optimal dynamic view that can follow the evolution of these features so that the user can perceive the maximal infor- mation from the rendering of the time-varying features. To enable the identification of the optimal dynamic view, we will first describe a method to identify the best view for static data. Then we will describe a dynamic programming method to find the optimal dynamic view for time-varying features.

In this chapter, we assume that all the view points are located on the surface of a viewing sphere. At each view point the user looks at the center of the sphere, where

89 the volume is located. During view selection, the view moves on the sphere, which means the distance between the view and the volume center is fixed. We also assume the viewing and projection parameters are appropriately set up so that the projection of the volume from any view will not fall outside the window.

6.2 Static View Selection

The essential problem any view selection technique tries to solve is to find a good view point through which the users are able to perceive the maximum amount of infor- mation from the underlying scene. In the context of volume visualization, Takahashi et al. [64] proposed a surface-based view point optimization algorithm where the geo- metric properties of the interval volume faces are considered. Their method produces good static views for data that can be decomposed into different interval volumes. Bor- doloi and Shen [11] took a direct volume rendering approach without the need of inter- mediate geometry. Their method generates good views in general with the exception of some cases. For example, in figure 6.1, there are two voxels in the scene, one with a weight of 0.7 and the other 0.3. Since their method prefers views from which the visibil- ity of the voxel is proportional to its weight, the voxel with weight 0.7 has to occlude the other voxel to some degree in order to achieve a higher score for their entropy formula.

However, these two voxels are readily visible through some views such as V1. From this example, we can see that if the visibility of a voxel can be maximized, it does not have to be proportional to its weight. To remedy this problem and consider additional important properties of the data, we propose an image-based view selection method. Our method measures the quality of a static view not only based on its opacity and projection size

(which is the primary criterion of some of the previous algorithms), but also explicitly

90 Figure 6.1: The figure to illustrate that the result from [11] should be improved.

considers the color and curvature distribution of the rendered images. Our motivation comes from the fact that color and curvature convey very important information about the underlying phenomenon in many applications.

6.2.1 Measurement of Opacity Distribution and Projection Size

Imagine a user is visualizing a volumetric dataset using a volume rendering tech- nique. Some voxels in the volume have higher opacity values, meaning these voxels are more important. Less important voxels are assigned with smaller opacities. Initially the user may choose a view through which many opaque voxels are aligned in the viewing direction and hence a lot of occlusion occurs. In this case, some pixels in the final image will have very high opacity values, while the opacity values at other pixels are low. The user realizes that this is not a good view, so s/he changes to a view where less occlusion occurs in the volume, so that the user can see many voxels more clearly. In this case, the opacity value in the image will be more evenly distributed. Besides this, the user also

91 generally prefers a rendering image with a larger projection area. From this example, it can be seen that an important factor that contributes to the selection of good views is the distribution of opacity values and the size of the projection area in the resulting image.

An image with an even opacity distribution and a large projection area should be more favorable than one with an uneven opacity distribution and/or a small projection area. A function is desired to reflect the property. The Shannon entropy function [56] can be utilized to perform the measurement.

In Information Theory, the Shannon entropy function is used to measure the amount of information contained in a random sequence of symbols. Suppose the symbols oc- cur in the set {a0,a1,...,an−1} with the occurrence probability {p0, p1,..., pn−1}, the average information of the sequence, called entropy, is defined as

n−1 H(x) = − ∑ pi · log2(pi) (6.1) i=0

One nice property of the entropy function is that it is a concave function. It only has one local maximum value, which is also the global maximum value. It reaches this maximum value log2n when p0 = p1 = ... = pn−1 = 1/n, that is, the distribution of the probability is perfectly even among all the symbols. As the probability moves away from the perfectly even distribution along a straight line in any direction, the probability becomes less and less evenly distributed, and the value of the entropy function will also decrease.

The Shannon entropy function can be utilized to measure the information contained in an opacity image. We now explain how the probability is designed so that the entropy function gives a higher value when the opacity value is more evenly distributed and the projection area is larger, while it gives lower values otherwise. Given an opacity image

92 which contains n pixels with opacity value {α0,α1,...,αn−1}, we define the probability pi of the ith pixel as αi pi = n−1 (6.2) ∑ j=0 α j The image entropy is calculated by equation 6.1. Although the entropy is evaluated over all the image pixels, the background pixels actually do not contribute to the entropy.

The reason is that the opacity value of any background pixel is 0, so it will not affect the probability and entropy contribution of any foreground pixel. Furthermore, since

0 · log20 is defined as 0, background pixels will not contribute to the final entropy value of the whole image. Therefore, we can define the image entropy just over the foreground area. The image entropy gets the maximum value when all the foreground pixels occur in the same probability, that is, all the foreground pixels have the same opacity values.

The entropy function also takes into account the size of the projection area, which is the foreground of the image. The reason is that the maximum entropy value of an image is log2 f , where f is the size of the foreground. Therefore, the entropy of an opacity image with a large foreground area and an even opacity distribution gets a higher value than one with smaller foreground areas. In summary, our opacity entropy function prefers an image with a large projection area with an even opacity distribution.

6.2.2 Measurement of Color Distribution

Opacity is just one factor that influences the selection of good views. Another im- portant factor that determines the quality of a view is color. In volume rendering, colors are often assigned to voxels by using a color transfer function. A well-designed color transfer function should highlight salient features by using perceptually attentive col- ors, and map unimportant voxels to some less attentive colors. The measurement of a

93 view’s quality should keep the fidelity of the color transfer function. This means that in the color-mapped volume, even though some colors (the less attentive colors assigned to unimportant voxels, for example) may occur more frequently than some other colors

(attentive colors assigned to salient features, for example), the less frequently salient feature colors actually carry more information. Therefore a good volume rendering im- age should contain more of these colors and thus more information about the salient features. Furthermore, we always want to highlight as many salient features as possible in the limited screen area. If the volume contains multiple salient features, these fea- tures should be mapped to the final images equally, i.e, the projected areas for different salient colors should be as even as possible among all the features. Based on the analy- sis, it can be seen that a good view should maximize the area of the salient colors while maintaining an even distribution among these colors.

To measure the color distribution of the volume rendering image, we also utilize the

Shannon entropy function. The entropy function and the probability evaluation should be designed so that the entropy function gives a higher value for an image with more evenly distributed and larger area of salient colors, while gives lower values for images with less evenly distributed and/or smaller area of salient colors. Suppose there are n colors {C0,C1,...Cn−1}, where C1,C2,...Cn−1 occurs in the color transfer function and C0 is the background color (actually C0 can be a spectrum of colors, which includes every pixel of the image which is not perceptually similar to any of C1,C2,...Cn−1). Given any pixel in the rendered image, we can determine which feature it belongs to by measuring the perceptual color distance between the pixel color and the feature color. If it does not belong to any feature (either the feature it should belong to is highly occluded, or it

94 comes from unimportant voxels), it will be assigned to C0. Suppose the total window area is T and the color areas of C1,C2,...Cn−1 are A1,A2,...An−1 respectively. The area

n−1 for C0 is then A0 = T − ∑i=1 (Ai). The probability is defined as

A p = i (6.3) i T

n−1 It is a probability definition since T = ∑i=0 (Ai). The color entropy function is defined as in equation 6.1. We can see that the entropy reaches its maximum value when A0 = A1 = ... = An−1, that is, all the color areas are even. Due to the inclusion of A0, large background area will incur small total salient color area, and thus uneven probability distribution and small entropy value accordingly. Therefore, the entropy function and our probability definition prefer larger total salient color area and more even distribution among all the salient colors. It should be noted that the probability definition can lead to a small undesired effect. This happens when we see each of the salient colors and the background with the same area, which reaches the maximum of the entropy. The entropy will get smaller if the area of salient colors is enlarged, and this is undesired. However, this is less likely to happen in practice since the background area for any given view is usually large enough so that the volume rendering images from all the views can be projected into the window. We can also intentionally increase the window size to avoid the problem. Furthermore, even if the error occurs, it can be as large as log(n) − log(n − 1), which is a negligible number for a relatively large n.

It is also noteworthy to mention that a perception-based color space should be used during the determination of the feature a pixel belongs to. We choose the CIELUV color model since it provides a perceptually equal color space, i.e., the distance in CIELUV space reflects the perceptual color difference. We also choose a lighting model which

95 involves only ambient and diffuse lighting calculation. Specular lighting is not included since it can alter the color of pixel by the color of the light. The color entropy eval- uation works well for a well-designed color transfer function where colors are used to highlight different features. If a color transfer function just simply assigns gray-scale or rainbow colors according to different values, the color entropy may not reflect the feature information contained in the view.

6.2.3 Measurement of Curvature Information

Opacity and color are two important factors that measure the quality of a view. In addition to opacity and color, there are other properties that also contribute to the infor- mation provided in a volume rendering image. One of such properties is the curvature.

Low curvatures imply flat areas and high curvatures mean highly irregular surfaces, which often contain more information (If the volume is noisy, a smoothing operation should be performed beforehand). Therefore, it is important to take the curvature infor- mation into account during the selection of good views.

One problem of considering curvature information in view selection is how to present the curvatures in a volume rendering image. We achieve this with two steps. First we calculate the curvature at each voxel position of the volume, using the method proposed by Kindlmann et al. [36]. When the volume is rendered, the color of a voxel is de- termined by its curvature. Voxels with high curvature are assigned with high intensity colors, while voxels below a certain low-curvature threshold are assigned with the color

(0,0,0). The opacity of the voxel is determined independently, which can be based on its original data value, or some other properties such as the gradient. After the rendering is performed and the image is generated, the intensity of the image reflects the amount

96 of curvature perceived from the visible part of the volume, that is, an image with high intensity means that the user can see many high-curvature voxels from that view.

6.2.4 The Final Utility Function

Opacity, color and curvature all contribute to the information perceived from a ren- dering of the volume. We need a function to incorporate all the factors. This utility function [69] u from a view v should have the following basic form:

u(v) = (opacity(v) + color(v) + curvature(v))/3 (6.4) that is, the utility function should consider contribution from all the factors. One prob- lem with the utility function is that the opacity, color and curvature contributions are not normalized. We should normalize each of the factors into [0,1] before the summation.

The maximum value of the entropy function of an image with a projection size of n is log2n. So if we find the maximum projection size M of the images among all the views, each of the entropies can be normalized by dividing over log2M. The maximum value of the color entropy is log2n, where n is the number of colors (see section 6.2.2).

Therefore, the color entropy can be easily normalized by a division over log2n. The normalization of the curvature contribution can also be easily done by a division over the maximum projection size M, since the maximum intensity of each pixel is 1.

If we possess any prior knowledge of the volume, it is often desirable to give differ- ent weights to different factors. The utility function then has the following form:

u(v) = α · opacity(v) + β · color(v) + γ · curvature(v) (6.5) where α +β +γ = 1. One scenario is that people often design very sophisticated opacity transfer function, but use a simple gray-scale or rainbow color transfer function. In this

97 case, it is desirable to put more weight into opacity(v) than color(v), since opacity conveys more information. However, in another case where different colors are used to highlight different features in a segmented volume, it is desirable to put large weight to color(v). In practice, we can choose proper weight for every factor based on the characteristic of the data and transfer function and the nature of the application.

6.3 Dynamic View Selection

In this section, a dynamic view selection algorithm is presented. The goal of dy- namic view selection is to allow the user to find a viewing path which shows the max- imum amount of information from the time-varying dataset, and the path should show near-constant angular velocity (all the views lie on the surface of a viewing sphere).

We formulate this into the following three principles that a good dynamic viewing path should follow:

• The view should move at a near-constant speed.

• The view should not change its direction abruptly.

• The information perceived from the time-varying data should be maximized among

all the viewing paths.

In the following subsections, we first discuss the issue of how to select time-varying views that follow the three principles. Then we present a method that allows the user to find a path between any two views in a given timestep that maximizes the perceived information while obeying the other two principles.

98 6.3.1 Time-Varying View Selection

The problem of time-varying view selection is that given a view at t = 0, among

all the possible paths along which the view can move smoothly to the final timestep

at a near-constant angular velocity, find the path that gives the maximum perceived

information. If in average a view can move to one of n possible views at the next

timestep, and there are total t timesteps, the complexity of the problem can be nt. This

search space is exponentially large. It is impractical to try all these paths and find the

optimal one.

To solve the problem more efficiently, we can employ the dynamic programming

approach. Let’s first consider selecting time-varying views with the first and third prin-

ciples in mind, that is, we want to find a time-varying view that moves at a near-constant

speed, and the information perceived from that path is maximized among all possible

paths. Suppose the view is moving with speed V, with Vmin ≤ V ≤ Vmax. Vmin and Vmax

are used to bound the speed of the view so that when Vmin is close to Vmax, the view

moves at a near-constant speed. We use Pi, j to denote the position of the jth view at t = i, and MaxIn f o(Pi, j) is the maximum amount of information perceived from Pi, j to

some view at the final timestep. The following recursive function holds:

Numo fViews−1 MaxIn f o(Pi, j) = max {u(Pi, j) −Cost(Pi, j,Pi+1,k) + MaxIn f o(Pi+1,k)} (6.6) k=0

where u(Pi, j) measures the information perceived at the view Pi, j. Cost(Pi, j,Pi+1,k) mea-

sures the cost to move from Pi, j to Pi+1,k. If the jth view point and the kth view are

within [Vmin,Vmax], the cost is 0, otherwise the cost is +∞. The equation basically says

the maximum amount of information perceived from Pi, j to some view point at the final

timestep will be equal to the sum of the information perceived at Pi, j, and the maximum

99 information perceived from Pi+1,k to some view at the final timestep. Pi+1,k represents a view point at t = i+1 that can be reached within [Vmin,Vmax] distance from Pi, j. We will consider all the views Pi+1,k at timestep i + 1. The following C-style code performs the calculation of all the MaxIn f o(Pi, j):

//Initialization for (i=0; i

//Dynamic Programming for (i=NumofTimesteps-2; i>=0; i--) for (j=0;jMaxInfo[i, j]) { MaxInfo[i, j]=Info; NextViewIndex[i, j]=k; } } }

The initial condition is MaxIn f o(Pn−1,i) = u(Pn−1,i) for i ∈ [0..Numo fViews − 1].

The dynamic programming process calculates all the MaxIn f o(Pi, j) backwards in time, according to the recursive function. NextNodeIndex(Pi, j) records the view index at the next timestep that gives the maximum information from Pi, j to some view at the final timestep, and it can be used to recover the time-varying path. The dynamic programming process finishes all the computation in O(n · v2) time, where n is the number of total

100 (a) (b)

Figure 6.2: An example of a partition of a view point’s local tangent plane and one of the possible allowed turns encoded in matrix.

timesteps, and v is the number of total views. This process only takes a polynomial time complexity.

The above dynamic programming calculates an optimal path based on the restriction that the view should move with the speed within [Vmin,Vmax]. But it does not prohibit the view from making sharp turns, which is undesirable when viewing the animation. It is also impossible to use the information stored at NextViewIndex to find the optimal path that does not make sharp turns, since NextViewIndex only records the optimal paths that move at a near-constant speed. To address this problem, at each view point on the viewing sphere, we partition its local tangent plane into many different regions, and restrict the allowed turns. Figure 6.2 illustrates a partition of eight regions and a matrix that encodes the allowed turns. We use MaxIn f o(Pi, j,r) to denote the maximum amount of information perceived from Pi, j to some view point at the final timestep, and Pi, j was

101 entered from region r from its previous view. Then the following recursive function holds:

MaxIn f o(Pi, j,r) = maxr=0..Numo f Regions−1,k∈Regionr{u(Pi, j) −Cost(Pi, j,Pi+1,k) +MaxIn f o(Pi+1,k,s)}

The following C-like code calculates all the MaxIn f o(Pi, j,r):

\\Initialization for (i=0;i

\\Dynamic Programming for (i=NumofTimesteps-2; i>=0; i--) for (j=0; jMaxInfo[i, j, r]) { MaxInfo[i, j, r]=Info; NextViewIndex[i, j, r]=k; NextRegionIndex[i, j, r]=s; } } } where o is the region number leaving the jth view and s is the region number enter- ing the kth view at the next timestep. o and s can be easily determined based on the projection to local tangent plane at the jth and kth view respectively. NextViewIndex

102 and NextRegionIndex record the view and region index at the next timestep that offers the maximum information to some view at the final timestep. These two data structures can be used to recover the path. The dynamic programming process finishes all the computation in O(n · r · v2) time, where n is the number of timesteps, v is the number of views, and r is the number of regions. This process only takes a polynomial time complexity. After the dynamic programming is done, given the initial view at t = 0, the results stored at MaxIn f o, NextViewIndex and NextRegionIndex can be used to find the maximum perceived information and the optimal time-varying view associated with the initial view.

6.3.2 Viewing Path Between any Two Views in a Given Timestep

Another case of dynamic view selection is to find a viewing path between any two viewpoints in a given timestep. This viewing path should also follow the three princi- ples, i.e., moves between these two viewpoints smoothly with a near-constant angular velocity, and maximizes the perceived data information at the same time. This tech- nique can be very useful to showcase a static dataset. When generating an animation, keyframes are usually specified by the user, and intermediate frames are generated by interpolation. If different viewpoints are assigned in different keyframes, spherical lin- ear interpolation (SLERP) is a common technique to interpolate the intermediate view positions. SLERP does give a viewing path with constant angular velocity, but it does not take the perceived information into consideration. Next we will explain how we maximize the perceived information and take all three principles into consideration.

Given any two views on a viewing sphere, there are an infinite number of paths that connect these two views. One factor in our design of the dynamic path is that it should

103 Figure 6.3: The solid curve is the SLERP path. Our algorithm will consider all the neighbors of the SLERP path that lie within the dotted area. All the neighbors are parameterized by u and v.

follow the general direction of the SLERP path, since the SLERP path is the shortest path that connects the two points with constant angular velocity. Therefore, we only allow the view to move at the neighboring views of the SLERP path (as shown in figure 6.3).

We also need to put restriction on the direction of the allowed movement so that the view will not go back and forth in a circular manner. We achieve this by parameterizing all the neighbors relative to the SLERP path, as illustrated in figure 6.3. A movement is allowed only if the u parameter is increasing and the v parameter difference is within a threshold value. We call these paths monotonic paths. We can also enforce the direction change by adopting the local coordinates and the admissible turn matrix in figure 6.2.

When evaluating the quality of different paths, the summation of information should not be used, since some paths can go through more view points than others. One good criterion would be the average information. The pseudo code below illustrates how to

104 use the propagation method similar to the single-source shortest path algorithm to find the optimal path:

//Initialization ActiveSet={Source viewpoint S}; PathLength=0; PathInfo[S,PathLength]=u(S);

Initialize all other PathInfos to a small value; NextActiveSet=empty; while(ActiveSet is not empty) { PathLength++; for each view V in ActiveSet for each neighbor N of V if (the movement from V to N is monotonic) { PathInfo[N, PathLength]=max(PathInfo[N, PathLength], u(V)+PathInfo[V, PathLength-1]); Put N in NextActiveSet; } ActiveSet=NextActiveSet; }

For all the PathInfo[D, n] where D is the destination Find the one with the maximum average information and it will be the optimal path.

Notice the above process only runs on the neighborhood of the SLERP path. It

finishes in O(N2) time, where N is the number of neighbors along the SLERP path.

6.4 Results

We have implemented and tested both the static and dynamic view selection algo- rithms on a Pentium IV 1.4GHz machine with an nVidia GeForce 6800 graphics card.

Our view selection algorithms take as input the opacity, color and curvature images ren- dered from the dataset, which can be generated by any volume rendering technique.

105 In our implementation, we choose a hardware-based volume slicing technique with 3D texture mapping to generate those images. 256 sample views were used for each dataset, and these views are evenly distributed on the viewing sphere.

The test result for the 512 × 64 × 64 shockwave dataset is shown in figure 6.4. The opacity entropy value is used during the test to show its effectiveness in determining view quality. Figure 6.4 (a) shows the worst view which has the smallest opacity entropy, and figure 6.4 (b) shows the best view with the highest opacity entropy. Figure 6.4 (c) and (d) illustrate the opacity images of the worst and best views respectively. It took

6.92 seconds to compute the opacity entropy values for the 256 views and find the best and worst views, and the size of the opacity image is 256 × 256. By using the entropy function and the proposed probability function, our opacity entropy evaluation takes both the opacity distribution and the projection area into consideration, and the opacity entropy prefers an image with an even opacity distribution and a larger projection area.

To illustrate how the opacity entropy varies according to different viewing angles, the view is rotated along the vertical axis (Y axis) in a complete circle. Figure 6.4 (e) plots the change of opacity entropy with respect to different views.

We also used the 128×128×80 tooth data to test the view selection algorithm based on the opacity entropy, and the result is shown in figure 6.5. Figure 6.5 (a) shows the worst view with the smallest opacity entropy, and figure 6.5 (b) shows the best view with the largest opacity entropy. Figure 6.5 (c) and (d) are their opacity images. It took 7.18 seconds to compute the opacity entropy values for the 256 views and find the best and worst views, and the size of the opacity image is 256 × 256. The variation of opacity

106 entropy with respect to different views is also plotted in the Figure 6.5 (e), where the viewing angle is rotated incrementally around the X axis.

We used the 1283 vortex dataset to show the effectiveness of the color entropy func- tion. The data set contains many components and we use the color transfer function to highlight components which may go through topological changes in future timesteps.

Other components are assigned a gray-scale color. Figure 6.6 (a) shows the worst view with the smallest color entropy, and figure 6.6 (b) shows the best view. It can been eas- ily seen that figure 6.6 (b) conveys more information about the topologically important features than figure 6.6 (a). In figure 6.6 (a), the total projection area of the highlighted features is small, and the projection area ratio among the highlighted features is very uneven. This leads to a very small color entropy value. In contrast, in figure 6.6 (b), the highlighted features have a large projection area and an even projection area distribu- tion, and therefore a large value for the color entropy. It took 16.3 seconds to compute the color entropy values for the 256 views and find the best and worst views, and the size of the color image is 256 × 256. Figure 6.6 (c) plots the change of color entropy with respect to different views where the viewing angle is rotated incrementally around the Y axis.

Figure 6.7 gives the view-selection result for the Terascale Supernova Initiative (TSI) dataset. The dataset is to model the core collapse of supernovae and was generated by collaboration among Oak Ridge National Lab and eight universities. In the chapter, we visualize the entropy scalar component of the dataset, which is derived from pres- sure and density scalar values. When exploring the dataset, we used the rainbow color transfer function. In our view selection test, two factors, curvature and opacity, are

107 considered in the calculation of view information. We want to design a utility function which puts more weight for views that show more jagged area. Therefore, in our design, we set the coefficients for curvature and opacity to 0.8 and 0.2 respectively. Figure 6.7

(a) shows the worst view, and figure 6.7 (b) is the best view. It is obvious that figure 6.7

(b) shows more detailed information about the jagged area than Figure 6.7 (a). It took

18.7 seconds to evaluate the curvature information and opacity entropy for all the 256 views and find the best and worst views, and the size of the image is 256×256. To show how the view utility function varies, figure 6.7 (c) plots the change of utility value with respect to different views, where the view is rotated incrementally around the vertical

(Y) axis.

We also used the TSI dataset to test our dynamic view selection algorithm. The supernova is a very dynamic phenomenon where the features are morphing and rotat- ing rapidly in space. Our previous static view selection shows that from some views very little information about the phenomenon can be perceived. If the view for an an- imation is fixed, much of the phenomenon would be occluded for many timesteps (see

figure 6.9 (f)-(i)). Recall that the goal of our algorithm is to find a viewing path with the maximum amount of information which also follows the constraint that the view moves at a near-constant angular velocity. We used our static view selection to calcu- late the view information of every view point at every timestep and used our dynamic programming algorithm to find the best path. We use the same view point setting at all the timesteps. Figure 6.9 (a) shows the best path in which viewpoint P0,0 moves in time with the speed within (0.9, 1.2) (The radius of the viewing sphere is 1). Although the supernova phenomenon is morphing rapidly, we still perceive a maximum amount

108 of information following our dynamic viewing path. It took 4.31 seconds for the dy- namic programming process to find the optimal path. Figure 6.9 (a) shows part of the path, which demonstrates near-constant angular velocity (the distance in figure 6.9 (a) is distorted). Furthermore, following the path, the overall information perceived from the time-varying data is maximized. Figure 6.9 (b)-(e) show four snapshots of the time- varying dataset captured by the time-varying view path, and figure 6.9 (f)-(i) show the images seen from the original view at the timesteps corresponding to (b)-(e). The user can apparently see more turbulent side of the phenomenon all the time from the time- varying views generated by our algorithm.

We also used the TSI dataset to show a viewing path selected from any two views in a given timestep. The TSI dataset at t = 0 is used, and figure 6.8 shows both the SLERP and the optimized paths. It took 0.08 seconds to find the optimized path. The average information perceived by the SLERP path is 0.51, while the optimized path gives 0.56.

6.5 Summary

In this chapter, we present methods for both static and dynamic view selection. Our static view selection algorithm analyzes opacity, color and curvature images generated from different view points. We properly design the probability functions and use entropy to evaluate opacity and color distributions. Our algorithm also prefers a view which shows high curvature information. Depending on the characteristic of the data set and the opacity and color transfer function, and the nature of the application, we can design different utility functions to assign different weights to the three factors. Based on our static view selection and dynamic programming, our dynamic view selection method

109 maximizes the information perceived from the time-varying dataset following a near- constant angular velocity path. The optimization is achieved in a polynomial time. Our results show the effectiveness of the static and dynamic view selection.

110 (a) (b)

(c) (d)

(e)

Figure 6.4: The figure shows the static view selection results based on opacity entropy for the shockwave dataset. (a) shows the worst view, (b) is the best view, and (c) and (d) are the opacity images for (a) and (b) respectively. (e) plots the change of opacity entropy with respect to different viewing angles where the shockwave is rotated around the vertical axis in a full circle.

111 (a) (b)

(c) (d)

(e)

Figure 6.5: The figure shows the static view selection results based on opacity entropy for the tooth dataset. (a) shows the worst view, (b) is the best view, and (c) and (d) are the opacity images for (a) and (b) respectively. (e) plots the change of opacity entropy with respect to the viewing angle when the tooth is rotated around the X axis in a full circle.

112 (a) (b)

(c)

Figure 6.6: The figure shows the static view selection results based on color entropy for the vortex dataset. (a) shows the worst view, (b) is the best view, and (c) plots the change of color entropy with respect to different viewing angles when the vortex is rotated around the Y axis in a full circle.

113 (a) (b)

(c)

Figure 6.7: The figure shows the dynamic view selection results for the TSI dataset. (a) shows the worst view, (b) is the best view, and (c) plots the change of the final information with respect to the viewing angle when the TSI dataset is rotated around the vertical axis in a full circle.

114 Figure 6.8: The figure shows two paths which move from one view point to the other. The right path is generated by SLERP interpolation with an average information of 0.51. The left path is generated by our method. The path is smooth and gives an average information of 0.56.

115 (a)

(b) (c) (d) (e)

(f) (g) (h) (i)

Figure 6.9: The figure shows the dynamic view selection result for the TSI dataset. (a) shows the path of the time-varying view, which exhibits constant angular velocity. (b)- (e) show four snapshots captured by our time-varying view. (h)-(i) show the image from the original static view at the timestep corresponding to (b)-(e).

116 CHAPTER 7

CONCLUSIONS AND FUTURE WORK

The work addresses the problem of tracking and viewing time-varying features. Af- ter the completion of the work, we have the following conclusions:

• Higher dimensional isosurfacing provides an efficient way to track isosurface and

interval volume components.

• Tracking of time-varying isosurfaces with minimal overhead is achieved by pre-

computing the correspondence relationship at a pre-processing stage, which al-

lows efficient run-time correspondence table lookup operation and simple verifi-

cation.

• The Earth Mover’s Distance is a better metric than the volume-overlapping and

the aggregate-attribute criteria in tracking features.

• The global tracking algorithm effectively tracks features in complex data sets

where there are many small/fast-moving features and there exist many alterna-

tives to match those features. The local tracking techniques can not guarantee the

global best match for these data sets.

117 • The static view selection algorithm finds the best static view by properly taking

into account the perceived color, curvature, and opacity information from the cor-

responding rendering images.

• The dynamic programming procedure finds the best dynamic view efficiently. The

dynamic view shows the maximal information for the time-vary features while

following the smooth-movement constraint.

Reinders et al.[48] used a prediction scheme extensively in their research to pre- dict how the feature attributes change over time. We believe that the prediction scheme will also be beneficial to our algorithm, especially during the EMD computation be- tween a source and destination feature component. For example, if how the source feature changes its position over time can be accurately predicted, we can utilize the position change information in the EMD calculation. Instead of calculating the EMD value between the source and destination features, we can first predict the position of the source feature at the next timestep, and then perform the EMD calculation between the predicted source feature at the next timestep and the destination feature. This process should give more accurate EMD values between features and is especially beneficial to fast-moving features. In addition to the position change, in the future work, we will also study how to incorporate the orientation and scale change information into the prediction scheme.

In addition to generating the best tracking result, it is very useful that the feature tracking algorithms can provide the uncertainty of the tracking result. In general, when uncertainty is evaluated and reported in a specified way, it indicates the level of confi- dence that the result accurately reflects the measured quantity, that is, the uncertainty is

118 a quantitative indication of the quality of the result. In our feature tracking algorithm, the feature tracking result over a time series represents the physical motion pattern of these features. In our future work, we would like to study how to employ the physical laws to measure the uncertainty of the physical motion of the features, and thus provide an uncertainty value for the tracking result.

Our algorithms to track local isosurface and interval volume features are based on the assumption that corresponding features in two consecutive timesteps overlap. Our global tracking algorithm defines the best match between features of two consecutive timesteps by the match with the minimal overall matching cost. In other words, our global tracking tries to minimize the energy of the motion of the features. In our fu- ture work, we would like to employ more physical laws in the prediction, tracking and verification of the feature motion, and also in the uncertainty evaluation.

It is also interesting to track sub-features, such as the tip on a surface, and to see how the sub-features change when time evolves.

In addition to dynamic view point planning, another important parameter for ani- mation would be lighting design. Gunhold [25] discussed light source placement for static polygonal meshes. We would like to conduct the research for lighting design for time-varying polygonal and volumetric data in our future work.

119 BIBLIOGRAPHY

[1] R. Adams and L. Bischof. Seeded region growing. IEEE Transaction on Pattern Analysis and Machine Intelligence, 16(6):641–646, 1994.

[2] J.K. Aggarwal and N. Nandhakumar. On the computation of motion from se- quences of images – a review. Proceedings of the IEEE, 76(8):917–935, 1988.

[3] C. Andujar, P. Vazquez, and M. Fairen. Way-finder: Guided tours through complex walkthrough models. Computer Graphics Forum, 23(3):488–508, 2004.

[4] T. Arbel and F. Ferrie. Viewpoint selection by navigation through entropy maps. In Proceeding of International Conference on Computer Vision, pages 248–254, 1999.

[5] Y. Arnaud, M. Desbois, and J. Maizi. Automatic tracking and characterization of african convective systems on meteosat pictures. Journal of Applied Meteorology, 31(5):443–453, 1992.

[6] C. Bajaj, V. Pascucci, and D. Schikore. Fast isocontouring for improved interactiv- ity. In Proceedings of the 1996 IEEE Symposium on Volume Visualization, pages 39–46, 1996.

[7] D.H. Ballard. Computer Vision. Prentice-Hall,Inc, Englewood, New Jersey, 1982.

[8] D. Bank and B. Singer. A predictor-corrector technique for visualizing unsteady flow. IEEE Transactions on Visualization and Computer Graphics, 1(2):151–163, 1995.

[9] P. Barral, G. Dorme, and D. Plemenos. Scene understanding techniques using a virtual camera. In Proceeding of Eurographics 2000, 2000.

[10] P. Bhaniramka, R. Wenger, and R. Crawfis. Isosurfacing in higher dimensions. In Proceedings of Visualization 2000, pages 267–274, 2000.

[11] U. Bordoloi and H-W. Shen. View selection for volume rendering. In IEEE Visu- alization Conference 2005, pages 487–494, 2005.

120 [12] I. Carlbom, I. Chakravarty, and W. Hsu. Integrating computer graphics, com- puter vision, and image processing in scientific applications. Computer Graphics, 26(1):8–17, 1992.

[13] H. Carr and J. Snoeyink. Path seeds and flexible isosurfaces using topology for exploratory visualization. In Joint Eurographics - IEEE TCVG Symposium on Visualization 2003, pages 49–58, 2003.

[14] H. Carr, J. Snoeyink, and U. Axen. Computing contour trees in all dimensions. Computational Geometry, 24(2), 2003.

[15] J. Chen, D. Silver, and L. Jiang. The feature tree: Visualizing feature tracking in distributed amr datasets. In Proceedings of IEEE symposium on Parallel and Large-Data Visualization and Graphics 2003, pages 103–110, 2003.

[16] L. Cohen. On active contour models and balloons. Computer Vision, Graphics, and Image Processing: Image Understanding, 53(2):211–218, 1991.

[17] C.M. Cyr and B.B. Kimia. 3d object recognition using shape similarity-based aspect graph. In Proceeding of International Conference on Computer Vision, pages 254–261, 2001.

[18] R.A. Drebin, L. Carpenter, and P. Hanrahan. Volume rendering. In Proceedings of ACM SIGGRAPH 1988, pages 65–74, 1988.

[19] K. Engel, M. Kraus, and T. Ertl. High-quality pre-integrated volume render- ing using hardware-accelerated pixel shading. In Proceedings of ACM SIG- GRAPH/EUROGRAPHICS Workshop on Graphics Hardware, pages 9–16, 2001.

[20] I. Fujishiro, Y. Maeda, and H. Sato. Interval volume: A solid fitting technique for volumetric data display and analysis. In Proceedings of IEEE Visualization 1995, pages 151–158, 1995.

[21] I. Fujishiro, Y. Maeda, H. Sato, and Y. Takeshima. Volumetric data exploration us- ing interval volume. IEEE Transactions on Visualization and Computer Graphics, 2(2):144–155, 1996.

[22] T. Gerstner and R. Pajarola. Topology preserving and controlled topology simpli- fying multiresolution isosurface extraction. In Proceedings of Visualization 2000, pages 259–266, 2000.

[23] D. Goodseel. Hemoglobin: Cooperation makes it easier.

[24] K.D. Gremban and K.Ikeuchi. Planning multiple observation for object recogni- tion. International Journal of Computer Vision, 12(2/3):137–172, 1994.

121 [25] S. Gumhold. Maximum entropy light source placement. In IEEE Visualization Conference 2002, pages 275–282, 2002.

[26] B. Guo. Interval set: A volume rendering technique generalizing isosurface ex- traction. In Proceedings of Visualization 1995, pages 3–10, 1995.

[27] F.S. Hillier and G.J. Liberman. Introducetion to Mathematical Programming. McGraw-Hill, 1990.

[28] L. Hong, S. Muraki, A. Kaufman, D. Bartz, and T. He. Virtual voyage: Interactive navigation in the human colon. Computer Graphics, 31:27–34, 1997.

[29] B.K.P. Horn and B.G. Schunck. Determining optical flow. Artificial Intelligence, 17:185–203, 1981.

[30] B.K.P. Horn and B.G. Schunck. Determining optical flow: a retrospective. Artifi- cial Intelligence, 59:81–87, 1993.

[31] G. Ji and H-W. Shen. Efficient isosurface tracking using precomputed correspon- dence table. In Joint Eurographics - IEEE TCVG Symposium on Visualization 2004, pages 283–292, 2004.

[32] G. Ji, H-W. Shen, and R. Wenger. Volume tracking using higher dimensional isosurfacing. In Proceedings of Visualization 2003, pages 209–216, 2003.

[33] A. Joshi and P. Rheingans. Illustration-inspired techniques for visualizing time- varying data. In IEEE Visualization Conference 2005, pages 86–93, 2005.

[34] T. Kamada and S. Kawai. A simple method for computing general position in displaying three-dimensional objects. Proceeding of International Conference on Computer Vision, 41(1):248–254, 1988.

[35] D.N. Kenwright. Automatic flow feature detection techniques for tera-scale data analysis. In IEEE Visualization 1999, Tutorial 6, Feature Extraction and Visual- ization of Time-Dependent Flow Fields, 1997.

[36] G. Kindlmann, R. Whitaker, T. Tasdizen, and T. Moller. Curvature-based transfer functions for direct volume rendering: Methods and applications. In IEEE Visual- ization Conference 2003, pages 513–520, 2003.

[37] W. Koegler. Case study: Applications of feature tracking to analysis of autoignition simulation data. In Proceedings of IEEE Visulazation 2001, pages 461–464, 2001.

[38] J.J. Koenderink and A.J. van Doorn. The sigularities of the visual mapping. Bio- logical Cybernetics, 24:51–59, 1976.

122 [39] J.J. Koenderink and A.J. van Doorn. The internal representation of solid shape with respect to vision. Biological Cybernetics, 32:211–216, 1979.

[40] M.V. Kreveld, R.V. Oostrum, C. Bajaj, V. Pascucci, and D. Schikore. Contour trees and small seed sets for isosurface traversal. In Proceedings of 13th Annual ACM Symposium on Computational Geometry, pages 212–220, 1997.

[41] P. Lacroute and M. Levoy. Fast volume rendering using a shear-warp factorization of the viewing transformation. In Proceedings of ACM SIGGRAPH 1994, pages 451–458, 1994.

[42] M. Levoy. Display of surfaces from volume data. IEEE Computer Graphics and Applications, 8(3):29–37, May 1998.

[43] W.E. Lorensen and H.E. Cline. Marching cubes: A high resolution 3d surface construction algorithm. In Proceedings of ACM SIGGRAPH 1987, pages 163– 169, 1987.

[44] N. Max. Optical models for direct volume rendering. IEEE Transactions on Visu- alizationand Computer Graphics, 1(2):99–108, 1995.

[45] H.H. Nagel. On the estimation of optic flow: Relations between different ap- proaches and some new results. Artificial Intelligence, 33:299–324, 1987.

[46] G.M. Nielson and J. Sung. Interval volume tetrahedralization. In Proceedings of Visualization 1997, pages 221–228, 1997.

[47] V. Pascucci and K. Cole-McLaughlin. Efficient computation of the topology of level sets. In Proceedings of Visualization 2002, pages 187–194, 2002.

[48] F. Reinders, F.H. Post, and H.J.W. Spoelder. Visualization of time-dependent data using feature tracking and event detection. The Visual Computer, 17(1):55–71, 2001.

[49] C. Rezk-Salama, K. Engel, M. Bauer, G. Greiner, , and T. Ertl. Interactive volume rendering on standard pc graphics hardware using multi-textures and multi-stage rasterization. In Proceedings of ACM SIGGRAPH/EUROGRAPHICS Workshop on Graphics Hardware, pages 109–118, 2000.

[50] J. Rossignac and P. Borrel. Multi-resolution 3d approximations for rendering com- plex scenes. In Geometric Modeling in Computer Graphics 1993, pages 455–465, 1993.

123 [51] Y. Rubner, L.J. Guibas, and C. Tomasi. The earth mover’s distance, multi- dimensional scaling, and color-based image retrieval. In Proceedings of DARPA Image Understanding Workshop, pages 661–668, 1997.

[52] Y. Rubner, C. Tomasi, and L.J. Guibas. A metric for distributions with applications to image databases. In IEEE International Conference on Computer Vision 1998, pages 59–66, 1998.

[53] R. Samtaney, D. Silver, N. Zabusky, and J. Cao. Visualizing features and tracking their evolution. IEEE Computer, 27(7):20–27, 1994.

[54] I.K. Sethi and R. Jain. Finding trajectories of feature points in a monocular im- age sequence. IEEE Transactions on Pattern Analysis and Machine Intelligence, 9(1):56–73, 1987.

[55] I.K. Sethi, N.V Patel, and J.H. Yoo. A general approach for token correspondence. Pattern Recognition, 27(12):1775–1786, 1994.

[56] C.E. Shannon. A mathematical theory of communication. In Bell System Technical Journal, pages 379–423 & 623–656, 1948.

[57] K. Sheomake. Animation with quaternion curves. Computer Graphics, 19:245– 254, 1985.

[58] J. Shi and C. Tomasi. Good features to track. In IEEE Conference on Computer Vision and Pattern Recognition 1994, pages 593–600, 1994.

[59] D. Silver. Object-oriented visualization. IEEE Computer Graphics and Applica- tions, 15(3), 1995.

[60] D. Silver and X. Wang. Volume tracking. In Proceedings of Visualization 1996, pages 157–164, 1996.

[61] D. Silver and X. Wang. Tracking and visualizing turbulent 3d features. IEEE Transactions on Visualization and Computer Graphics, 3(2):129–141, 1997.

[62] D. Silver and X. Wang. Tracking scalar features in unstructured datasets. In Pro- ceedings of Visualization 1998, pages 79–86, 1998.

[63] B.S. Sohn and C. Bajaj. Time-varying contour topology. IEEE Transactions on Visualization and Computer Graphics, 12(1):14–25, 2006.

[64] S. Takahashi, I. Fujishiro, Y. Takeshima, and T. Nishita. A feature-driven approach to locating optimal viewpoints for volume visualization. In IEEE Visualization Conference 2005, pages 495–502, 2005.

124 [65] S.P. Tarasov and M.N. Vyalyi. Construction of contour trees in 3d in o(nlog n) steps. In Proceedings 14th Annual ACM Symposium on Computational Geometry, pages 68–75, 1998.

[66] J.J. van Wijk and W.A.A. Nuij. Smooth and efficient zooming and panning. In IEEE Symposium on Information Visualization 2003, pages 15–23, 2003.

[67] P-P. Vazquez, M. Feixas, M. Sbert, and W. Heidrich. Viewpoint selection using viewpoint entropy. In Vision Modeling and Visualization Conference 2001, pages 273–280, 2001.

[68] P-P. Vazquez, M. Feixas, M. Sbert, and W. Heidrich. Automatic view selection using viewpoint entropy and its application to image-based modeling. Computer Graphics Forum, 22(4):689–700, 2003.

[69] J. von. Neumann and O. Morgenstern. Theory of Games and Economic Behavior. Princeton University Press, 1944.

[70] G.H. Weber, G. Scheuermann, H. Hagen, and B. Hamann. Exploring scalar fields using critical isovalues. In Proceedings of Visualization 2002, pages 171–178, 2002.

[71] C. Weigle and D.C. Banks. Complex-valued contour meshing. In Proceedings of Visualization 1996, pages 173–180, 1996.

[72] C. Weigle and D.C. Banks. Extracting iso-valued features in 4-dimensional scalar fields. In Proceedings of the 1998 IEEE Symposium on Volume Visualization, pages 103–110, 1998.

[73] E.A. Wernert and A.J. Hanson. A framework for assisted exploration with collab- oration. In IEEE Visualization Conference 1999, pages 241–248, 1999.

[74] R. Westermann and T. Ertl. Efficiently using graphics hardware in volume ren- dering applications. In Proceedings of ACM SIGGRAPH 1998, pages 169–178, 1998.

[75] J. Woodring, C. Wang, and H-W. Shen. High dimensional direct rendering of time-varying voulmes. In IEEE Visualization Conference 2003, pages 417–424, 2003.

[76] X. Xu. Image feature analysis for computer-aided diagnosis: Detection of right and left hemidiaphragmedges and delineation of lung field in chest radiograms. Medial Physiscs, 23(9):1613–1624, 1996.

125