Motivations Visualization Methods ◦ Two-Dimensional Methods ◦ Volume Visualization (Surface-Based, Direct Volume) Volume Processing and Classification Visualization Examples
ITCS 6010:Biomedical Imaging and Visualization 1 Image Processing and Analysis Image Visualization
Visualizations have always been used to ◦ Study relationship between anatomical structure and function ◦ Detect and treat trauma and disease Traditional visualizations were direct, via surgery, biopsy, or indirect, via extensive mental reconstruction The revolution in 3D/4D biomedical imaging (CT, MRI, PET) and computer reconstruction and rendering provides new powerful op- portunities. Image Processing and Visualization can help unlock critical informa- tion of objects and their properties in medical images.
ITCS 6010:Biomedical Imaging and Visualization 2 Image Processing and Analysis Image Visualization:Why?
Multi-modal 3D and 4D imaging from CT, MRI, PET, MEG, Ultra- sound, Microscopy have fueled developments in data differentiation, fusion, visualization Datasets are generally very large (mega-giga-tera bytes). Management, Processing, Analysis and Visualization requires high performance computing. Advances in visualization technology will provide new tools/procedures that physicians “must have” to treat patients Quote from a physician, “If I can see it, I can fix it” Ability to see anatomical structures permits surgery planning, re- hearsal, to minimize trauma, avoid critical areas.
ITCS 6010:Biomedical Imaging and Visualization 3 Image Processing and Analysis Image Visualization:Why?
Realtime interactive visualization, advanced display technologies in- cluding VR, open new realms into medical practice. Images displayed and manipulated with immediacy, sufficient detail and speed to evoke a sufficiently ”real” experience. Interactive 3D environments permit physicians to view anatomy and function from any viewpoint, make accurate online measurements. Visualization Scale: ◦ Structure: From individual molecules, tissues and interstitial in- terfaces to complete organs and organ systems; ◦ Functional attributes: Biophysical, Biomechanical and physio- logical properties. Holy Grail: Fully immersive, real-time multisensory fusion of real and virtual information streams (Example: GE Surgical MRI scan- ner) ITCS 6010:Biomedical Imaging and Visualization 4 Image Processing and Analysis Visualization Methods: 2D Techniques
2D images from 3D/4D biomedical imaging datasets can show ◦ Original data ◦ No occlusion problems Difficulty: Optimal image plane might not be possible: ◦ Limitations of the imaging system ◦ Need sophisticated reconstruction techniques
ITCS 6010:Biomedical Imaging and Visualization 5 Image Processing and Analysis 2D Visualiztion: Multiplanar Reformatting
Ability to display axis-aligned images of 3D/4D volume If the entire volume is in memory, displaying the images along each image plane maps to voxel ordering problem Example: 3D CT, MRI Volume: Axial, Sagittal, Coronal Images
ITCS 6010:Biomedical Imaging and Visualization 6 Image Processing and Analysis 2D Visualiztion: Multiplanar Reformatting
ITCS 6010:Biomedical Imaging and Visualization 7 Image Processing and Analysis 2D Visualization: Oblique Sectioning
Optimal image plane more likely will lie along arbitrarily oriented plane Need to specify orientation, and accurate computer reconstruction
ITCS 6010:Biomedical Imaging and Visualization 8 Image Processing and Analysis 2D Visualiztion: Curved Planar Reformation
Structures may be curvilinear: spinal canal, blood vessels. Difficult to capture such structures completely in an planar image. Possible Implementation: Trace pixels along a path in an orthogonal section; display rows of voxels corresponding to the pixels as a 2D image.
ITCS 6010:Biomedical Imaging and Visualization 9 Image Processing and Analysis Volume Visualization:Voxels and Cells
Volume Data: An array of volume elements, termed “voxels”, or an array of “cells”. Voxel Model: ◦ Area around the grid point has a uniform value(s)(hexahedral, spherical, etc) ◦ Advantage: No assumptions on data behavior between grid- points Cell Model: ◦ Models volumes as a collection of hexahedra (or other geome- try) with gridpoints at its corners ◦ Need an estimation scheme for data variation within the cell
ITCS 6010:Biomedical Imaging and Visualization 10 Image Processing and Analysis Volume Visualization:Grids and Lattices
Regular: voxels, cells are axis-aligned rectangular prisms Rectilinear: voxels, cels are axis-aligned, but inter-voxel(cell) spac- ing varies Structured: Non axis-aligned hexahedra (warped bricks); Exam- ples: Spherical, Curvilinear grids Block-Structured: Collection of structured grids Hybrid, Unstructured: Collection of any set of grid types
ITCS 6010:Biomedical Imaging and Visualization 11 Image Processing and Analysis Volume Visualization: Processing Steps
Data acquisition: Includes processing to increase data value range, contrast enhancement, free of noise Aspect Ratio Adjustment: Volume dimensions correspond to physical dimensions of the acquired data - may require interpola- tion, additional slice generation, estimate missing values Data Classification Mapping operations: Transformation to geometric or display primi- tives
ITCS 6010:Biomedical Imaging and Visualization 12 Image Processing and Analysis Volume Visualization: Anatomic Modeling Steps
ITCS 6010:Biomedical Imaging and Visualization 13 Image Processing and Analysis Volume Visualization Methods Surface vs. Direct Volume Rendering Summarize
Simulate Playback
Measured Data Filter Images
Data Data Render
Geometric Mapping Primitives
Surface-Based Volume Rendering: Data is converted to geometric primitives and a standard graphics pipeline is used to render the volume; Examples: Isosurfacing(Marching Cubes, Dividing Cubes), Contour connecting Direct Volume Rendering: Volume is directly sampled and projected to the final raster image (rendering). Examples: Ray Casting, Pro- jection Methods (VBuffer), Volume Splatting. Examples from the Visible Human Project (VHP) at http://www.nlm.nih.gov/research/visible/animations.html ITCS 6010:Biomedical Imaging and Visualization 14 Image Processing and Analysis Volume Visualization Methods Surface vs. Direct Volume Rendering
ITCS 6010:Biomedical Imaging and Visualization 15 Image Processing and Analysis Volume Visualization Methods Surface vs. Direct Volume Rendering
Surface Based Direct Volume Render ing Object Order Image Order Contour Connecting V-Buffer Ray Casting Opaque Cubes Splatting Cell Integration Marching Tetrahedra Pixar Method Frequency Domain Methods Marching Cubes Dividing Cubes
ITCS 6010:Biomedical Imaging and Visualization 16 Image Processing and Analysis Volume Visualization: Data Classification
Process of associating voxels with useful, significant information on objects/properties represented by the medical volume. Two methods typically employed in volume visualization: ◦ Transfer Functions: Volume data (scalar/vector values, gradient, other features) mapped to opacity and color and input to render- ing pipeline ◦ Segmentation: Sophisticated algorithms are employed in asso- ciated voxels with clinically significant objects (tissues, organs, tumors)
ITCS 6010:Biomedical Imaging and Visualization 17 Image Processing and Analysis Volume Visualization:Illumination Models
Depth Shading, Depth Gradient Shading Maximum Intensity Projection Voxel Gradient Shading Volumetric Compositing
ITCS 6010:Biomedical Imaging and Visualization 18 Image Processing and Analysis Volume Visualization:Illumination Models
ITCS 6010:Biomedical Imaging and Visualization 19 Image Processing and Analysis Volume Visualization:Illumination Models
ITCS 6010:Biomedical Imaging and Visualization 20 Image Processing and Analysis