Hybrid Volume and Polygon Rendering with Cube Hardware Kevin Kreeger and Arie Kaufman* Center for Visual Computing (CVC) and Department of Computer Science State University of New York at Stony Brook Stony Brook, NY 11794-4400 Abstract 1 Introduction We present two methods which connect today’s polygon graphics Direct volume rendering acceleratorswill become commercially hardware accelerators to Cube-5 volume rendering hardware, the available this year, from Mitsubishi Electric in mid 1999 as a low- successor to Cube4 The proposed methods allow mixing of both cost plug-in PC1card called VolumePro [ 101,and from JapanRadio opaque and translucent polygons with volumes on PC class ma- Co. in late 1999 as the special purposeU-Cube ultrasound visual- chines, while ensuring the correct compositing order of all objects. ization system. Since these systemsare based on Cube-4 [l l] we Both implementations connect the two hardware acceleration sub- refer to them as the Cube family of volume rendering accelerators. systems at the frame buffer. One shares a common DRAM buffer Unfortunately, theseaccelerators work independently from current and one run-length encodes images of thin slabs of polygonal data geometry rendering hardware. Therefore, it is impossible to ren- and then combines them in the Cube composite buffer In both re- der mixtures of volumetric and polygonal data, unless none of the alizations, we take advantage of the predictable ordered access to objects intersect. Unfortunately, many applications require objects frame buffer storage that is utilized by Cube-5 and the rest of the to intersect. For example, in a flight simulation scene,a polygonal family of volume rendering accelerators based on the Cube design. plane may fly through a volumetric cloud over a textured polygo- CR Categories: 1.3.1 [Computer Graphics]: Hardware nal terrain viewed from within a translucent cockpit, as shown in Architecture-Graphics Processors; 1.3.3 [Computer Graphics]: Figure 1. In this paper,we begin expanding the Cube design to ac- Picture/Image Generation-Display algorithms; 1.3.7 [Computer celerate rendering of more than just volumetric data, creating the Graphics]: Three-Dimensional Graphics and Realism; fifth generation of Cube architectures,the Cube-5 We proposetwo methodsto connect current geometry pipelines Keywords: Mixing polygons and volumes, Volume rendering, to the Cube family of volume rendering acceleratorson PC class Ray casting, Run-length-encoding,Cube architecture machines. Both allow mixing of opaque and/or translucent poly- gons with volumetric data. Translucent polygons complicate the situation becauseall fragments(both translucentpolygon fragments and volume samples)must be drawn in topologically depth sorted order. This is required becausecompositing with the over opera- tor [ 121is not commutative. Interesting visual effects can be created with translucent polygons as shown in the translucenttank spinning in the desertkicking up a volumetric cloud of dust in Figure 2. Vir- tual environments, such as surgical simulation, require real-time or Figure 1: A jlight simulation scene miring a texture-mapped Polygonal terrain, an opaque plane (with 4420 polygons), a tmnslucent cockpit, and a volumetric cloud (also in c&rp&e). * (kkreegewi) @cs.sunysb.edu Permission to make digital or hard copies of all or part ofthis work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. TO COP)' otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Figure 2: Volumetric dust cloud kicked up by a spinning translu- 1999 Eurographics LosAngeles CA USA cent tank (wirh 5082 Polygons) in a desert (also in colorplate). Copyright ACM 1999 I-58113-170-4/99/08...%5.00 15 Figure 3: A prosthesis (containing 3758polygons) beingfit to a 256’ CT scan of a hip: (a) translucentpolygons reveal the bony structure behind the prosthesis, (b) opaque polygons obscure incorrect alignment (also in colorplate). The OpenGL extensions proposed in the HP Vbxelator [8] at- tempted to create a standard software interface to mix volumetric data with polygonal objects. However, no mention of hardware ac- celeration was given and only opaque objects were discussed,ne- glecting translucent polygons and volumes. Three dimensional texture map based volume rendering [2] al- lows the mixing of volumes and polygons. Unfortunately, 3D tex- ture map based volume rendering is only available on high end graphics workstations and is neither scalable nor capable of real shading. Additionally, these systemsrequire very expensive equip- ment to achieve even interactive frame rates. Our solution, on the other hand, provides better frame rates, real Phong shading, and an order of magnitude lower cost in addition to being designed for a PC class machine. All of the Cube family (Cube-4/Cube-YVolumeProKJ-Cube)of volume rendering acceleratorsutilize slice-order ray casting, an ob- ject order technique (e.g., [6, 161). This means that input data is processedin a regular predefined order. This exploitation of mem- ory coherenceis critical to realizing the real-time 30Hz frame rates Figure 4: A translucent ghost ship (containing 4715 polygons) of the Cube family. We also take advantageof the repeatableaccess sailing out of a volumetric fog bank in front of an opaque texture- pattern in both of our designs, specifically to the frame buffer. mapped island (also in colorplate). In Section 2, we describe our method to mix translucent poly- gons with volumes in volume object order. In Section 3, we dis- cuss the differences between a DRAM frame buffer on current PC graphics cards and the SRAM composite buffer in previous Cube at least interactive feedback.Figure 3 showsa fitting of a polygonal designs. Sections4 and 5 present our two methods to create a hy- model of a prosthesisto a CT scan of a patient. The use of translu- brid volume and polygon rendering systemusing the Cube-5 archi- cent polygons enables the surgeon to see the details of the volu- tecture and relatively low cost (comparedto high end workstations) metric bone behind the polygonal object. Finally, Figure 4 shows a PC graphics pipelines. Section 6 presentsresults and a performance translucent ghost ship sailing out of a volumetric fog bank. comparison. Software systems,developed for production applications, allow mixing of volumetric and polygonally modeledobjects. Ray tracing is commonly utilized by these systemsto render mixtures of vol- 2 Mixing Polygons into Volume Render- umes and polygons [7, 151. The flexibility of these systemscomes ing Systems at the price of an extremely slow frame generation rate. Therefore, thesesystems would not perform well for applications requiring in- Volume rendering is the more difficult of the two rendering modal- teractivity. ities. Also, the only methods to achieve interactive or real-time Volume Graphics GmbH produces a product which mixes frame rates for volume rendering utilize object order. Therefore, opaque polygons with volumes by ray casting. They achieve in- in our method presentedhere, we adapt polygon rendering to slice teractivity by utilizing reducedresolution rendering while parame- order volume ray casting (used in the Cube architectures),and orga- ters are being changed followed by slower high quality rendering nize the overall rendering processon a volume slice-by-slice basis whenever the viewing parametersbecome constant [13]. No hard- rather than a polygon-by-polygon or pixel-by-pixel basis. ware accelerationis utilized, so the performanceis limited, and no In the Cube family of accelerators,the slices of the volume are translucent polygons can be used. processedin depth order. Thus, to correctly order translucent poly-. 16 Polygon Shearedbre-warped) Actual Gecamtry (30cmletry Slab 0 I Qe-0try - Volume Sample I Plane Major, f Viewing Axis Figure 6: Top view of creating sheared viewing geometry so that Figure 5: Side view of dove-tailing translucent polygons and vol- polygon footprints are “pre-warped” and Z-depths represent dis- ume data. (The gaps between polygon slabs are shown for clarity. tance along the volume processing direction. In reality, there is no gap or overlap as shown by the boundary intervals.) that the warping operation at the end of Cube rendering creates gon fragmentswith volume samples,thin slabsof polygons are ren- correct polygons in the final image. Also, the Z-depths computed deredand compositedin between slices of the volume, as shown in are proportional to the distances along the processing axis. It is Figure 5. The polygonal slabs representall of the translucent ob- sometimespossible (if all opaque geometry fits within the volume jects which lay between two consecutive slices of the volume data. extents) to set the hither and yon clipping planes to the edges of The boundariesare createdsuch that the union of all the slabs nei- the volume and, if the precision of the depth buffer is the same, ther miss nor duplicate anything, such as less-thanthe current slice the depths computed are exactly the volume slice indices for depth and greater-than-or-equal-tothe next slice (see slab boundaries in checking. Otherwise, a simple scaling must be applied when they Figure 5). The data from the volume slices and translucent polyg- are utilized by the volume rendering system. Light positions must onal slabs are dove-tailed together in an alternating fashion. In this be considered when using this method, as the shearing may not way the correct depth ordering of all contributing entities is pre- move the lights correctly. served,and use of the over operator to compositethem createscor- The thin slices of translucent polygons should align geometri- rect colors in the final image pixels. cally with their 3D position in space. We begin by aligning the The Cube architectures take advantageof the storageorder of eye-point as before. Then, to keep the objects from projecting all voxels by processing slices orthogonal to one of the three major the way to the final image plane, we translate the geometry so that axis.