Johnson/Hansen: The Visualization Handbook Page Proof 20.5.2004 12:32pm page 181 9 Multi-Dimensional Transfer Functions for Volume Rendering JOE KNISS, GORDON KINDLMANN, and CHARLES HANSEN Scientific Computing and Imaging Institute School of Computing, University of Utah single boundary, such as thickness. When 9.1 Introduction working with multivariate data, a similar diffi- Direct volume-rendering has proven to be an culty arises with features that can be identified effective and flexible visualization method for only by their unique combination of multiple 3D scalar fields. Transfer functions are funda- data values. A 1D transfer function is simply mental to direct volume-rendering because their not capable of capturing this relationship. role is essentially to make the data visible: by Unfortunately, using multidimensional trans- assigning optical properties like color and fer functions in volume rendering is complicated. opacity to the voxel data, the volume can be Even when the transfer function is only 1D, rendered with traditional computer graphics finding an appropriate transfer function is gen- methods. Good transfer functions reveal the erally accomplished by trial and error. This is one important structures in the data without obscur- of the main challenges in making direct volume- ing them with unimportant regions. To date, rendering an effective visualization tool. Adding transfer functions have generally been limited dimensions to the transfer-function domain only Q1 to 1D domains, meaning that the 1D space of compounds the problem. While this is an on- scalar data value has been the only variable to going research area, many of the proposed which opacity and color are assigned. One methods for transfer-function generation and aspect of direct volume-rendering that has re- manipulation are not easily extended to higher- ceived little attention is the use of multidimen- dimensional transfer functions. In addition, fast sional transfer functions. volume-rendering algorithms that assume the Often, there are features of interest in volume transfer function can be implemented as a data that are difficult to extract and visualize linear lookup table (LUT) can be difficult to with 1D transfer functions. Many medical data- adapt to multidimensional transfer functions sets created from CT or MRI scans contain a due to the linear interpolation imposed on such complex combination of boundaries between LUTs. multiple materials. This situation is problematic This chapter provides a detailed exposition of for 1D transfer functions because of the poten- the multidimensional transfer function concept, tial for overlap between the data-value intervals a generalization of multidimensional transfer spanned by the different boundaries. When one functions for both scalar and multivariate data value is associated with multiple boundar- data, as well as a novel technique for the inter- ies, a 1D transfer function is unable to render active generation of volumetric shadows. To them in isolation. Another benefit of higher resolve the potential complexities in a user inter- dimensional transfer functions is their ability face for multidimensional transfer functions, we to portray subtle variations in properties of a introduce a set of direct manipulation widgets 181 Johnson/Hansen: The Visualization Handbook Page Proof 20.5.2004 12:32pm page 182 182 The Visualization Handbook that make finding and experimenting with trans- for the display of iso-value contours in more fer functions an intuitive, efficient, and informa- smoothly varying data. The previous work tive process. In order to make this process most directly related to our approach for visual- genuinely interactive, we exploit the fast render- izing scalar data facilitates the semiautomatic ing capabilities of modern graphics hardware, generation of both 1D and 2D transfer functions especially 3D texture memory and pixel-textur- [17,29]. Using principles of computer-vision edge ing operations. Together, the widgets and the detection, the semiautomatic method strives to hardware form the basis for new interaction isolate those portions of the transfer function modes that can guide users towards transfer- domain that most reliably correlate with the function settings appropriate for their visualiza- middle of material-interface boundaries. Other tion and data-exploration interests. work closely related to our approach for visual- izing multivariate data uses a 2D transfer func- tion to visualize data derived from multiple MRI 9.2 Previous Work pulse sequences [20]. Scalar volume-rendering research that uses 9.2.1 Transfer Functions multidimensional transfer functions is relatively Even though volume-rendering as a visualiza- scarce. One paper discusses the use of transfer tion tool is more than ten years old, only re- functions similar to Levoy’s as part of visualiza- cently has research focused on making the space tion in the context of wavelet volume represen- of transfer functions easier to explore. He et al. tation [27]. More recently, the VolumePro [12] generated transfer functions with genetic graphics board uses a 12-bit 1D lookup table algorithms driven either by user selection of for the transfer function, but also allows opacity thumbnail renderings or by some objective modulation by gradient magnitude, effectively image-fitness function. The Design Gallery [23] implementing a separable 2D transfer function creates an intuitive interface to the entire space [28]. Other work involving multidimensional of all possible transfer functions based on auto- transfer functions uses various types of second mated analysis and layout of rendered images. derivatives in order to distinguish features in the A more data-centric approach is the Contour volume according to their shape and curvature Spectrum [1], which visually summarizes the characteristics [15,34]. space of isosurfaces in terms of metrics like Designing color maps for displaying non surface area and mean gradient magnitude, volumetric data is a task similar to finding thereby guiding the choice of iso-value for iso- transfer functions. Previous work has developed surfacing, and also providing information strategies and guidelines for color map creation, useful for transfer-function generation. Another based on visualization goals, types of data, per- recent paper [18] presents a novel transfer- ceptual considerations, and user studies function interface in which small thumbnail ren- [3,32,36]. derings are arranged according to their relation- ship with the spaces of data values, color, and opacity. 9.2.2 Direct Manipulation Widgets The application of these methods is limited to Direct manipulation widgets are geometric the generation of 1D transfer functions, even objects rendered with a visualization and are though 2D transfer functions were introduced designed to provide the user with a 3D interface by Levoy in 1988 [22]. Levoy introduced two [5,14,31,35,38]. For example, a frame widget styles of transfer functions, both 2D and both can be used to select a 2D plane within a using gradient magnitude for the second dimen- volume. Widgets are typically rendered from sion. One transfer function was intended for the basic geometric primitives such as spheres, cy- display of interfaces between materials, the other linders, and cones. Widget construction is often Johnson/Hansen: The Visualization Handbook Page Proof 20.5.2004 12:32pm page 183 Multi-Dimensional Transfer Functions for Volume Rendering 183 guided by a constraint system that binds elem- subsequent Volume-Pro PCI graphics board ents of a widget to one another. Each sub-part [28]. The VolumePro graphics board imple- of a widget represents some functionality of the ments ray casting combined with the shear widget or a parameter to which the user has warp factorization for volume-rendering [19]. access. It features trilinear interpolation with super- sampling, gradient estimation, and shaded volumes, and provides interactive frame rates 9.2.3 Hardware Volume Rendering for scalar volumes with sizes up to 5123. Many volume-rendering techniques based on graphics hardware utilize texture memory to store a 3D dataset. The dataset is then sampled, 9.3 Multidimensional Transfer Functions classified, rendered to proxy geometry, and composited. Classification typically occurs in Transfer-function specification is arguably the hardware as a 1D table lookup. most important task in volume visualization. 2D texture-based techniques slice along the While the transfer function’s role is simply to major axes of the data and take advantage of assign optical properties such as opacity and hardware bilinear interpolation within the slice color to the data being visualized, the value [4]. These methods require three copies of the of the resulting visualization will be largely volume to reside in texture memory, one per dependent on how well these optical properties axis, and they often suffer from artifacts caused capture features of interest. Specifying a good by under-sampling along the slice axis. Trilinear transfer function can be a difficult and tedious interpolation can be attained using 2D textures task for several reasons. First, it is difficult to with specialized hardware extensions available uniquely identify features of interest in the on some commodity graphics cards [6]. This transfer-function domain. Even though a fea- technique allows intermediate slices along the ture of interest may be easily identifiable in the slice axis to be computed in hardware. These spatial domain, the range