2010 3rd International Conference on Biomedical Engineering and Informatics (BMEI 2010)

Multi-feature Based Transfer Function Design for 3D Medical Image

Yi Peng, Li Chen Institute of CG & CAD School of Software, Tsinghua University Beijing, China

Abstract—Visualization of three dimensional volumetric medical in multi-dimensional transfer function design. Each of them is data has been widely used. Even though, it still faces a lot of related to a particular local feature. Chen, et al. introduce a challenges, such as helping users find out structure of their volume illustration technique which is shape-aware, as it interest, illustrating features which are significant for diagnosis depends not only on the rendering styles, but also the shape of disease. In our paper, a multi-feature based transfer function styles [6]. Chen’s work improves the expressiveness of volume is provided to improve the quality of visualization. We compute a illustration by focusing more on shape, but it needs the help of multi-feature descriptor for both two-phase clustering and other given illustrations. transfer function design. Moreover, we test the transfer function on several medical datasets to show the efficiency and In addition, [7] presents two visualization techniques for practicability of our method. curve-centric volume reformation to create compelling comparative visualizations, and [8] introduces a texture-based Multi-feature, statistical analysis, pre-segmentation, transfer volume rendering approach which achieves the image quality function, volume rendering, visualization of the best post-shading approaches with far less slices. Their approaches use the latest techniques to accelerate volume I. INTRODUCTION rendering while keep using the general visualization methods. In direct volume rendering of medical visualization, the Based on the above methods, we propose an efficient research of specifying good transfer function of CT and MRI method for direct volume rendering which takes multi features datasets has been conducted for several years. For these years, into consideration and provides user a flexible and efficient researchers have found out several quite efficient methods in way to highlight structures of their interest. transfer function design. In our approach, we extract a large number of features from Generally, the existing methods can be divided into three the three dimensional volumetric data by computing a multi- categories. One is for interactivity between doctors and feature descriptor. The descriptor contains three kinds of local graphical user interface. One is for better illustration both in features. Each kind of features is composed by several special transfer function design and feature recognition. The other attributes with similar semantics. Then we do segmentation by focuses on run-time volume rendering. They prefer to pay more performing a two-phase clustering algorithm on the descriptor. attention to the latest techniques in hardware or new released The first-phase clustering algorithm is Watershed [9]. It programming interface. reduces the number of voxels by 3 orders of magnitude and Some researchers focus on interactive volume rendering. keeps the property of local feature from distortion. Then we Kniss, et al. compose texture-based transfer functions and perform K-Means algorithm to cluster local features into a present a set of direct manipulation widgets that make transfer specified number of groups [10]. Finally, with the multi-feature function design intuitive and convenient [1]. Bruckner, et al. descriptor we integrate our multi-feature based transfer introduce style transfer function which enables flexible data- function design method into a simple and flexible volume driven illumination [2]. Their methods are quite interactive, rendering framework developed by Stegmier, et al. [11]. however, for most users they require deep understanding of the dataset. II. RELATED WORK

Moreover, Caban, et al. introduce texture-based transfer A. Volume Rendering Framework functions. They use local textural properties rather than individual intensity values to specify the color and opacity of Spvolren is a flexible OpenGL-based framework for each voxel [3]. Correa et al. compose size-based transfer volume raycasting which allows for the easy integration of function which map the local scale of features to color [4]. highly diverse shaders. However, Spvolren only supports one They also present a mechanism for obtaining the scale fields at dimensional transfer function. So, we have made some interactive rates. [5] advances the use of curvature information improvements on it to render pre-segmented data by specifying

978-1-4244-6498-2/10/$26.00 ©2010 IEEE 410 the multi-feature descriptor through temporary volumetric data C. Clustering Algorithm and one dimensional transfer function. In the first-phase clustering, we use a simplified watershed algorithm which has a linear time complexity. And the k-means B. Local Features algorithm uses a KC-tree data structure to enhance its For a better visualization, we choose local features carefully performance. The k-means toolkit is provided by David Mount and divide them into three groups as listed in Table I. [15]. Group one is constituted by basic texture, such as standard deviation, uniformity and entropy. III. APPROACH In our approach, we use a multi-feature descriptor to Group two is a descriptor of co-occurrence matrix [12]. perform pre-segmentation and design our transfer function. By modifying the weight of each feature, we can easily emphasize TABLE I. LOCAL FEATURES the structure we are interested in. Moreover, by using texture and 3D moment feature, the volumetric data can be rendered in Descriptor Group some special styles. Feature Name Computing Formula v Intensity i A. Pre-segmentation  g 222gg Before pre-segmentation, we compute the multi-feature Gradient .()()()   x yz descriptor (Table I) which is a 15 dimensional vector Di for

L1 each voxel as shown in formula (1). Standard deviation  B 2 ()zzmpz (ii ) ( ) i0 1 Ddd FVii d i  Basic Texture Smoothness sz() 1 i HX02 14 1() 2 z  L 1 3 Third moment B For all the local features, we introduce a scale value 3 ()zzmpz (ii ) ( ) i0  which defines the size of neighborhood for sampling. For  L 1 2   Uniformity  B example, 1 means the number of neighbor voxels (including Uz() p ( zi ) i0 the voxel itself) for each voxel is (2  1)3 27 . Then, we project L1 B the intensity of samples onto an intensity histogram. Let  be a Entropy ez() pz (ii )log2 pz ( ) i0 set of neighbor voxels with intensity zi . Thus, pz()i denotes the LL11 BB 2 probability of neighbors with intensity z (Formula 2), which is Correlation Czrij() ( i j ) p i ij00 used for the computation of most features. (For an 8-bit  LL112  BB datasets, zi ranges from 0 to 255.) Other features need to be Contrast Cztij() p ij00 computed separately. For example, we compute gradient LL11 Co-occurrence  BB 2 feature by using a gradient operator offered by Spvolren. Energy Ez() pij Matrix ij00 (3 direction) LL11 pij  Homogeneity  BB   Hz() pz()ixxi , { z | z z }    3 ij001 ij (2 1)

LL11 Entropy ez()BB p log p After feature extraction, we perform a two-phase clustering cijij 2 ij00 for volume segmentation using 15 dimensional local features.  3D Moment J1 200 020 002 In the watershed-phase, the volumetric data are over-  200 020 200 002 022 002 segmented. However the number is reduced by three orders of 3D Moment 3D Moment J 2  222 magnitude after segmentation while most features are kept invariant 110 101 011 from distortion. Table II shows the number of categories during 2 3D Moment J 200 020 002 110 101 011 pre-segmentation. 3  222 002 110 020 101 200 011

TABLE II. NUMBER OF CATEGORIES DURING PRE-SEGMENTATION Group three is composed by three dimensional moment invariant [13]. For a large dataset, [14] introduces a method for Number of Categories Dataset fast computation of three-dimensional geometric moments. Initial Number After Watershed

However, in our approach we take the neighborhood size less 7 4 Head 256 256 256 1.68  10 3.30 10 than 10 voxels, and only need to deal with a quite small number of voxels for the computation at each voxel. Engine 256 256 128 8.39  106 9.77 103 Foot 256 256 256 1.68  107 1.75 104

411 The sharply reduction of category-number allows k-means IV. RESULTS algorithm to be done at run-time. The fast k-means algorithm also gives us the distance between each category and its cluster A. Structure Rendering center, which is important for transfer function design. After choosing the final cluster number k , our approach For each of the clustering algorithms, we introduce a 15 provides user an illustration of various structures with different dimensional weight vector to enhance its performance. In colors. watershed algorithm, the main feature F as shown in formula i As shown in Fig. 1, the dataset is divided into two parts.

(3) for voxel i is defined by dot product of Wws and Di , which When the two parts mixed with each other, the features are kept respectively. is used for region growing. And in k-means algorithm, Wkm is multiplied by Di to form a new 15 dimensional vector Gi in Fig. 2 shows that using different cluster number and weight the following formula (4) which is actually used in the vector provides user various structure with different color. clustering computation. Compared with Fig. 1, the number of feature in one cluster is decreased while the total number is increased.

14 4 jj 4  FiiwsiwsDWB d w j0

FVFVii i 00 4 11 4 1414 4  GikmikmikmiHXHX g02 g g 14 wd wd wd

B. Transfer Function Design After segmentation, each voxel is tagged with a label indicating which cluster it belongs to. For each voxel in the same cluster, we multiplied its value by a normalized distance between itself and cluster center to filter the noises in each cluster.

Spvolren only supports one dimensional transfer function. Figure 1. The picture shows a result of k  2 and the colors in color table are In order to render each cluster with different color, we need to c  (255,0,0,16) (top right) and c  (0,0,255,32) (bottom right).It uses divide the entries into k groups. ( k is the final cluster number, 0 1 entropy and intensity feature to perform watershed and k-means algorithm. and for 8-bit datasets, the number of entries is 256.) In each group, the color of entries is related to the specified color table. The color table contains k colors defined as (r, g, b,  ) . Each color is composed by r , g , b ,  four components and  denotes the color opacity. The r , g , b components for each entry is equal to the specified color, and the  component is defined by the product of specified color opacity and entry number that means for color opacity we use linear mapping.

Moreover, we define Wtf to support style rendering. It is used to update the intensity of each voxel. By this way, we can choose the proper style to render each cluster. For example, I 1(i  5) th let wi  J (Note that the 6 component of D is tf K0(ii 5,0 15) i uniformity, as shown in Table I), by updating the intensity of each voxel with a new value Hi (Formula 5) which is defined by the dot product of Di and Wtf , the voxels with larger uniformity will be highlighted. Also, we can use mixed weight vector to emphasize complicate features we are interested in.

14  4 ii 4   Figure 2. The picture shows a result of k  4 and the colors in color table are HiitfjtfDWB d w j0   c0 (255,255,0,64) (top right) , c1 (255,192,0,48) ( right middle),   c2 (255,0,0,32) (bottom left) and c3 (0,0,255,64) ( bottom middle). It uses smoothness and intensity feature to perform watershed and k-means algorithm. Bottom Right is the rsult of normal direct volume rendering.

412 B. Style Rendering ACKNOWLEDGMENT

By modifying the value of vector Wtf , we can get different The research was supported by Chinese 863 Program rendering style. (2010AA186002), the National Science Foundation of China (60773143, 90715043), Chinese 973 Program 2010CB328001 The top left picture is the result of normal volume rendering and NSFC-JST Key Joint Funding Project 51021140004. using the feature of intensity in transfer function. The top right picture uses standard deviation feature in transfer function. I also would like to thank my fellows Xueshen Tao, Fude From the result we can see that the intensity of each toe is more Wang, Haiyang Chu, Wei Song, Zhiwei Lin, Tao Yang for moderate than the top left picture. The bottom left picture using their helpful discussions. uniformity feature shows that the smoothness parts of bones is highlighted. The bottom right picture using entropy feature REFERENCES emphasizes the boundary of each bone. [1] Joe Kniss, Gordon Kindlmann, Charles Hansen. Multidimensional Transfer Functions for Interactive Volume Rendering. IEEE  Transactions on Visualization and , vol. 8, no. 3, pp. 270-285, July-September, 2002. [2] Stefan Bruckner, Meister Eduard Gröller. Style Transfer Functions for Illustrative Volume Rendering. Computer Graphics Forum, 26(3):715- 724, September 2007. [3] Jesus J. Caban, Penny Rheingans. Texture-based Transfer Functions for Direct Volume Rendering. IEEE Transactions on Visualization and Computer Graphics, vol. 14, no. 6, pp. 1364-1371, November/December, 2008. [4] Carlos Correa, Kwan-Liu Ma. Size-based Transfer Functions: A New Volume Exploration Technique. IEEE Transactions on Visualization and Computer Graphics, vol. 14, no. 6, pp. 1380-1387, November/December, 2008. [5] Gordon Kindlmann, Ross Whitaker, Tolga Tasdizen, Torsten Möller. Curvature-Based Transfer Functions for Direct Volume Rendering: Methods and Applications. Visualization Conference, IEEE, pp. 67, 14th IEEE Visualization 2003 (VIS 2003), 2003. [6] Wei Chen, Aidong Lu, David S.Ebert. Shape-aware Volume Illustration. Journal of Computer Graphics Forum (Proceedings of Eurographics 2007), 26(3), September 2007, 705-714. [7] Ove Daae Lampe, Carlos Correa, Kwan-Liu Ma, Helwig Hauser. Curve- Centric Volume Reformation for Comparative Visualization. IEEE Transactions on Visualization and Computer Graphics, vol. 15, no. 6, pp. 1235-1242, November/December, 2009.

[8] Engel K, Kraus M, Ertl T. High-quality pre-integrated volume rendering Figure 3. The picture shows a result of k  2 and the features used in using hardware-accelerated pixel shading. In Proceedings of Eurographics/SIGGRAPH Workshop on Graphics Hardware (2001), pp. transfer are intenseity (top left), standard deviation (top right), uniformity 9–16. (bottom left) and entropy(bottom right). [9] J.B.T.M.Roerdink,A.Meijster. The watershed transform:definitions algorithms and parallelization strategies. Fundamenta Infor-maticae 41,(2000)187-228, J. V. CONCLUSION [10] Elkan, C, 2003. Using the Triangle Inequality to Accelerate k-Means. In this paper, we propose an efficient method for direct Proceedings of the Twentieth International Conference on Machine volume rendering which uses a multi-feature descriptor to Learning (ICML-2003), pp. 609-616. perform two-phase clustering and design transfer function. The [11] S. Stegmaier, M. Strengert, T. Klein, and T. Ertl. A Simple and Flexible results show that our approach provides user an illustration of Volume Rendering Framework for Graphics-Hardware–based different structure with various styles. Since we have extracted Raycasting. In Proceedings of Volume Graphics, pages 187–195, 2005. a group of 3D moment features which is good at comparing [12] R. C. Gonzalez, R. E. Woods. Digital Image Processing, 3rd Edition, structure with similar shape feature, for medical datasets, our Publishing House of Electronics Industry, pp852-858, Janauary, 2001. [13] F. A. Sadjadi and E. L. Hall. Three-Dimensional Moment Invariants. method is quite efficient in highlighting different structures and IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 2, no. 2, pp. features in human body or other organs. 127-136, Mar. 1980. Moreover, by choosing proper weight vectors [14] Yang. L, Albregsten. F, and Taxt. T. Fast computation of three- dimensional moments using a discrete divergence theorem and a corresponding to significant structures, our method provides generalization to higher dimensions. Graphical Models and image user a style volume rendering. With structure style users can Processing, vol. 59, no. 2, pp. 97-108, March, 1997. easily find out the structures and features such as region [15] T. Kanungo, D. M. Mount, N. Netanyahu, C. Piatko, R. Silverman, and boundary, perspective of organs and structures with similar A. Y. Wu. A Local Search Approximation Algorithm for k-Means texture or shape they are interested in. Especially for doctors Clustering. Computational Geometry: Theory and Applications, 28 who are interested in tumours, using texture and moment (2004), 89-112. feature is quite efficient in searching for these special tissues.

413