1142_CH12 pp591_ 4/26/02 9:37 AM Page 591 12 3D Image Visualization Sources of 3D data rue three-dimensional (3D) imaging is becoming more accessible with the continued devel- opment of instrumentation. Just as the pixel is the unit of brightness measurement for a two- Tdimensional (2D) image, the voxel (volume element, the 3D analog of the pixel or picture el- ement) is the unit for 3D imaging; and just as processing and analysis is much simpler if the pixels are square, so the use of cubic voxels is preferred for three dimensions, although it is not as often achieved. Several basic approaches are used for volume imaging. In Chapter 11, 3D imaging by tomographic reconstruction was described. This is perhaps the premier method for measuring the density and in some cases the composition of solid specimens. It can produce a set of cubic voxels, although that is not the only or even the most common way that tomography is presently used. Most med- ical and industrial applications produce one or a series of 2D section planes, which are spaced far- ther apart than the lateral resolution within the plane (Baba et al., 1984, 1988; Briarty and Jenkins, 1984; Johnson and Capowski, 1985; Kriete, 1992). Tomography can be performed using a variety of different signals, including seismic waves, ultra- sound, magnetic resonance, conventional x-rays, gamma rays, neutron beams, and electron mi- croscopy, as well as other even less familiar methods. The resolution may vary from kilometers (seismic tomography), to centimeters (most conventional medical scans), millimeters (typical in- dustrial applications), micrometers (microfocus x-ray or synchrotron sources), and even nanome- ters (electron microscope reconstructions of viruses and atomic lattices). The same basic presen- tation tools are available regardless of the imaging modality or the dimensional scale. The most important variable in tomographic imaging, as for all of the other 3D methods discussed here, is whether the data set is planes of pixels, or an array of true voxels. As discussed in Chap- ter 11, it is possible to set up an array of cubic voxels, collect projection data from a series of views in three dimensions, and solve (either algebraically or by backprojection) for the density of each voxel. The most common way to perform tomography however, is to define one plane at a time as an array of square pixels, collect a series of linear views, solve for the 2D array of densi- ties in that plane, and then proceed to the next plane. When used in this way, tomography shares © 2002 by CRC Press LLC 1142_CH12 pp591_ 4/26/02 9:37 AM Page 592 many similarities (and problems) with other essentially 2D imaging methods that we will collec- tively define as serial imaging or serial section techniques. A radiologist viewing an array of such images is expected to combine them in his or her mind to “see” the 3D structures present. (This process is aided enormously by the fact that the radiologist already knows what the structure is, and is generally looking for things that differ from the famil- iar, particularly in a few characteristic ways that identify disease or injury.) Only a few current-gen- eration systems use the techniques discussed in this chapter to present 3D views directly. In in- dustrial tomography, the greater diversity of structure (and correspondingly lesser ability to predict what is expected), and the greater amount of time available for study and interpretation, has en- couraged the use of computer graphics. But such displays are still the exception rather than the rule, and an array of 2D planar images is more commonly used for volume imaging. This chapter emphasizes methods that use a series of parallel, uniformly spaced 2D images, but present them in combination to show 3D structure. These images are obtained by dissecting the sample into a series of planar sections, which are then piled up as a stack of voxels. Sometimes the sectioning is physical. Blocks of embedded bi- ological materials, textiles, and even some metals, can be sliced with a microtome, and each slice imaged (just as individual slices are normally viewed). Collecting and aligning the images pro- duces a 3D data set in which the voxels are typically very elongated in the “Z” direction because the slices are much thicker or more widely spaced than the lateral resolution within each slice. At the other extreme, the secondary ion mass spectrometer uses an incident ion beam to remove one layer of atoms at a time from the sample surface. These pass through a mass spectrometer to select atoms from a single element, which is then imaged on a fluorescent screen. Collecting a se- ries of images from many elements can produce a complete 3D map of the sample. One difference from the imaging of slices is that there is no alignment problem, because the sample block is held in place as the surface layers are removed. On the other hand, the erosion rate through different structures can vary so that the surface does not remain planar, and this roughening or differential erosion is very difficult to account for. In this type of instrument, the voxel height can be very small (essentially atomic dimensions) while the lateral dimension is many times larger. Serial sections Most physical sectioning approaches are similar to one or the other of these examples. They are known collectively as serial section methods. The name serial section comes from the use of light microscopy imaging of biological tissue, in which blocks of tissue embedded in resin are cut us- ing a microtome into a series of individual slices. Collecting these slices (or at least some of them) for viewing in the microscope enables researchers to assemble a set of photographs which can then be used to reconstruct the 3D structure. This technique illustrates most of the problems that may be encountered with any 3D imaging method based on a series of individual slices. First, the individual images must be aligned. The mi- crotomed slices are collected on slides and viewed in arbitrary orientations. So, even if the same structures can be located in the different sections (not always an easy task, given that some varia- tion in structure with depth must be present or there would be no incentive to do this kind of work), the pictures do not line up. Using the details of structure visible in each section provides only a coarse guide to alignment. The automatic methods generally seek to minimize the mismatch between sections either by aligning the centroids of features in the planes so that the sum of squares of distances is minimized, or by overlaying binary images from the two sections and shifting or rotating to minimize the area re- sulting from combining them with an Ex-OR (exclusive OR) operation, discussed in Chapter 7. 592 The Image Processing Handbook © 2002 by CRC Press LLC 1142_CH12 pp591_ 4/26/02 9:37 AM Page 593 This procedure is illustrated in Figure 1. When grey-scale values are present in the image, cross- correlation can be used as discussed in Chapter 5. Unfortunately, neither of these methods is easy to implement in the general case when sections may be shifted in X and Y and also rotated. Solv- ing for the “best alignment” is difficult and must usually proceed iteratively and slowly. Furthermore, there is no reason to expect the minimum point reached by these algorithms to re- ally represent the true alignment. As shown in Figures 2 and 3, shifting or rotating each image to visually align the structures in one section with the next can completely alter the reconstructed 3D structure. It is generally assumed that given enough detail present in the images, some kind of av- erage alignment will avoid these major errors; however, it is far from certain that a best visual alignment is the correct one nor that automated methods, which overlap sequential images, pro- duce the proper alignment. One approach that improves on the use of internal image detail for alignment is to incorporate fiducial marks in the block before sectioning. These could take the form of holes drilled by a laser, threads or fibers placed in the resin before it hardens, or grooves machined down the edges of the block, for example. With several such marks that can reasonably be expected to maintain their shape from section to section and continue in some known direction through the stack of images, much better alignment is possible. Placing and finding fiducial marks in the close vicinity of the structures of interest is often difficult. In practice, if the sections are not contiguous there may still be difficulties, and alignment errors may propagate through the stack of images. Most fiducial marks are large enough to cover several pixels in each image. As discussed in Chap- ter 8, this size allows locating the centroid to a fraction of one pixel accuracy, although not all sys- tems take advantage of this capability. Once the alignment points are identified (either from fidu- cial marks or internal image detail), the rotation and translation of one image to line up with the next is performed as discussed in Chapter 3. Resampling of the pixel array and interpolation to pre- vent aliasing produces a new image. This process takes some computational time, but this is a mi- nor problem in comparison to the difficulty of obtaining the images in the first place. Unfortunately, for classic serial sectioning the result of this rotation and translation is not a true rep- resentation of the original 3D structure. The act of sectioning using a microtome generally produces Figure 1.