(12) United States Patent (10) Patent No.: US 7,133,041 B2 Kaufman Et Al
Total Page:16
File Type:pdf, Size:1020Kb
US007133041B2 (12) United States Patent (10) Patent No.: US 7,133,041 B2 Kaufman et al. (45) Date of Patent: Nov. 7, 2006 (54) APPARATUS AND METHOD FOR VOLUME (56) References Cited PROCESSING AND RENDERING U.S. PATENT DOCUMENTS (75) Inventors: Arie E. Kaufman, Plainview, NY (US); 4,314,351 A 2f1982 Postel et al. Ingmar Bitter, Lake Grove, NY (US); 4,371,933 A 2, 1983 Bresenham et al. Frank Dachille, Amityville, NY (US); Kevin Kreeger, East Setauket, NY (Continued) (US); Baoquan Chen, Maple Grove, FOREIGN PATENT DOCUMENTS MN (US) EP O 216 156 A2 4f1987 (73) Assignee: The Research Foundation of State University of New York, Albany, NY (Continued) (US) OTHER PUBLICATIONS (*) Notice: Subject to any disclaimer, the term of this Knittel, G., TriangleCaster: extensions to 3D-texturing units for patent is extended or adjusted under 35 accelerated volume rendering, 1999, Proceedings of the ACM U.S.C. 154(b) by 742 days. SIGGRAPH/EUROGRAPHICS workshop on Graphics hardware, pp. 25-34.* (21) Appl. No.: 10/204,685 (Continued) PCT Fed: Feb. 26, 2001 (22) Primary Examiner Ulka Chauhan (86) PCT No.: PCT/USO1 (O6345 Assistant Examiner Said Broome (74) Attorney, Agent, or Firm Hoffmann & Baron, LLP S 371 (c)(1), (2), (4) Date: Jan. 9, 2003 (57) ABSTRACT (87) PCT Pub. No.: WOO1A63561 An apparatus and method for real-time Volume processing and universal three-dimensional rendering. The apparatus PCT Pub. Date: Aug. 30, 2001 includes a plurality of three-dimensional (3D) memory units; at least one pixel bus for providing global horizontal (65) Prior Publication Data communication; a plurality of rendering pipelines; at least one geometry bus; and a control unit. The apparatus includes US 2004/O125103 A1 Jul. 1, 2004 a block processor having a circular ray integration pipeline for processing Voxel data and ray data. Rays are generally Related U.S. Application Data processed in image order thus permitting great flexibility Provisional application No. 60/185.076, filed on Feb. (e.g., perspective projection, global illumination). The block (60) processor includes a splatting unit and a scattering unit. A 25, 2000. method for casting shadows and performing global illumi (51) Int. C. nation in relation to light sources includes Sweeping a two G06T IS/00 (2006.01) dimensional array of rays through the Volume can also be (52) U.S. Cl. ...................... 345/419; 345/423: 345/424; implemented with the apparatus. A method for approximat 345/427 ing a perspective projection includes using parallel projec (58) Field of Classification Search ................ 345/427, tion. 345/423, 424 See application file for complete search history. 13 Claims, 70 Drawing Sheets NTERSECTION 82 ARGE MAGE SOURCE MAGE (Screen) (3D surfaces) m d 98 94 SCANDIRECTION WEWPON SAMPE PON US 7,133,041 B2 Page 2 U.S. PATENT DOCUMENTS “Memory and Processing Architecture for 3-D Voxel-Based Imag ery,” by A. Kaufman and and R. Bakalash, in IEEE Computer 4,593,372 A 6, 1986 Bandai et al. Graphics and Applications, 1988. 4,674,046 A 6, 1987 Ozeki et al. 4,685,070 A 8/1987 Flinchbaugh “CUBEThree-Dimensional Workstation.” by A. Kaufman, in Proc. 4,710,876 A 12/1987 Cline et al. NCGA '88: Ninth Annual Conference and Exposition, Anaheim, 4,719,585 A 1, 1988 Cline et al. Calif., Mar. 1988. 4,729,098 A 3, 1988 Cline et al. "3-D Scan-Conversion Algorithms for Voxel-Based Graphics,” by 4,737,921 A 4, 1988 Goldwasser et al. Arie Kaufman and Eyal Shimony, in Proc. 1986 ACM Workshop on 4,791,567 A 12/1988 Cline et al. Interactive 3-D Graphics held in Chapel Hill, N.C. on Oct. 1986, pp. 4,791,582 A 12/1988 Ueda et al. 45-76. 4,827.413 A 5, 1989 Baldwin et al. 4,835,712 A 5, 1989 Drebin et al. “Back-to-Front Display on Voxel-Based Objects.” IEEE CG&A. 4,879,668 A 11, 1989 Cline et al. Jan. 1985, pp. 52-60 by Frieder et al. 4,882,683 A 11/1989 Ruppet al. “Real-Time Display and Manipulation of 3-D Medical Objects: The 4,984, 157 A 1/1991 Cline et al. Voxel Processor Architecture.” Computer Vision, Graphics and 4,985,856 A 1/1991 Kaufman et al. Image Processing, 1987, pp. 1-27, by Goldwasser et al. 4,987,554 A 1/1991 Kaufman "Generating Octree Models of 3-D Objects from Their Silhouettes 5,038,302 A 8, 1991 Kaufman in a Sequence of Images.” Computer Vision, Graphic and Image 5, 101.475 A 3, 1992 Kaufman et al. Processing, 1987, pp. 1-29, by Potmesil. 5,166,876 A 11/1992 Cline et al. 5,204,625 A 4, 1993 Cline et al. "Surface Reconstruction of 3-D Object in Computerized 5,226,113 A 7, 1993 Cline et al. Tomography,” Computer Vision, Graphics, and Image Processing, 5,253,065. A 10, 1993 Richards et al. 1988, pp. 270-278, by Xu et al. 5,313,567 A 5, 1994 Civanlar et al. "3-D Transformation of Images. In Scanline Order.” by Ed. Catmull 5,317,689 A 5, 1994 Nack et al. et al., in Computer Graphics, vol. 14, No. 3, at pp. 279-285 (1980). 5,361,385 A 11/1994 Bakalash “The Image Prism: A Device for Rotating and Mirroring Bitmap 5,381.518 A 1/1995 Drebin et al. Images.” by Cary Kornfeld in I.E.E.E. Computer Graphics and 5,412,764 A 5, 1995 Tanaka Applications, at pp. 21-30 (May 1987). 5,442,733. A 8, 1995 Kaufman et al. 5,544,283 A 8, 1996 Kaufman et al. “Data Parallel Volume Rendering as a Line Drawing.” by Peter 5,557,711 A 9, 1996 Malzbender Schroder et al., Volume Visualization Workshop, at pp. 25-31 (Oct. 5,594,842 A 1/1997 Kaufman et al. 1992). 5,594,844 A 1, 1997 Sakai et al. Kaufman et al. "A Survey for Architectures of Volume Rendering.” 5,724,493 A 3/1998 Hosoya et al. IEEE Engineering in Medicine and Biology Magazine, pp. 18-23, 5,760,781 A 6, 1998 Kaufman et al. Dec. 1990. 5,847,711 A 12, 1998 Kaufman et al. “CUBE An Architecture Based on a 3-D Voxel Map.” by Arie 5,850,226 A 12/1998 Nagasawa et al. Kaufman and R. Bakalash, in Theoretical Foundations of Computer 5,877,779 A 3/1999 Goldberg et al. Graphics and CAD, R.A. Earnshaw (Ed.), Springer Verlag, pp. 5,971,767 A 10, 1999 Kaufman et al. 689-701, 1988. Foley, Computer Graphics. Principles and Practice, 2" Edition, FOREIGN PATENT DOCUMENTS pp. 738, 835-842,914, 1034-1039, 1990. J. Foley, et al., “Recursive Ray Tracing.” Computer Graphics, EP O 612 O25 A2 8, 1994 Principles and Practice, 1990. EP O 642 104 A1 3, 1995 Renate Gemballa and Rolf Lindner, “The Multiple-Write Bus FR 2597 227 A1 10, 1987 Technique.” IEEE Computer Graphics and Applications, Sep. 1982, at pp. 33-41. OTHER PUBLICATIONS "Survey of Texture Mapping,” by P.S. Heckbert, IEEE Computer “A Generalized Object Display Processor Architecture.” by S.M. Graphics and Applications, 6(11):56-67, Nov. 1986. Goldwasser in I.E.E.E. Computer Graphics and Applications, vol. "Texram: SmartMemory for Texturing,” by A. Schilling et al., IEEE 4. No. 10, at pp. 43-55, 1984. Computer Graphics and Applications, 16(3):32-41. May 1996. “The Graphics PARCUM (Processing Architecture Based on Cubic “Creating Raster Omnimax Images from Multiple Perspective Memory) System: A 3-D Memory Based Computer Architecture for Views Using the Elliptical Weighted Average Filter.” by N. Greene Processing and Display of Solid Models,” by D. Jackel, published and P.S. Heckbert, IEEE Computer Graphics and Applications, in Computer Graphics Forum, North Holland, vol. 4. No. 4, at pp. 6(6):21-27, Jun. 1986. 21-32, 1985. “Volume Rendering.” by R.A. Debin et al. Computer Graphics “A Three-Dimensional Shaded Display Method for Voxel-Based (SIGGRAPH '88 Proceedings), vol. 22, pp. 65-74, Aug. 1988. Representations.” by T. Ohashi, et al., Proc. Eurographics 85, pp. “Three-Pass Affine Transformations for Volume Rendering.” by P. 221-232, Sep. 85. Hanrahan, Computer Graphics (San Diego Workshop on Volume “A 3-D Cellular Frame Buffer.” by Arie Kaufman and R. Bakalash, Visualization), vol. 24, pp. 71-78, Nov. 1990. in Proc. EUROGRAPHICS 85, Nice, France, Sep. 1985, pp. 215-220. “Fast Rotation of Volume Data on Parallel Architectures,” by P. “Memory Organization for a Cubic Frame Buffer.” by Arie Kauf Schroder and J.B. Salem, Visualization '91, pp. 50-57, 1991. man in Proc. EUROGRAPHICS 86, Lisbon, Portugal, Aug. 1986, “Compositing Digital Images.” by T. Porter and T. Duff, Computer pp. 93-100. Graphics (SIGGRAPH 84), vol. 18, No. 3, pp. 253-259, Jul. 1984. “Towards a 3-D Graphics Workstation.” by Arie Kaufman, in “The A-Buffer, an Antialiased Hidden Surface Method.” by L. Advances in Graphics Hardware I, W. Strasser (Ed.), Springer Carpenter, Computer Graphics (SIGGRAPH 84), vol. 18, No. 3, pp. Verlag, 1987, pp. 17-26. 103-108, Jul. 1984. “Voxel-Based Architectures for Three-Dimensional Graphics,” by “Filtering Edges for Grayscale Displays.” by S. Gupta and R.F. Arie Kaufman, in Proc. IFIP 86 10th World Computer Congress, Sproull, Computer Graphics (SIGGRAPH 81), vol. 15, No. 3, pp. Dublin, Ireland, Sep. 1986, pp. 361-366. 1-5, Aug. 1981. US 7,133,041 B2 Page 3 "Object Voxelization by Filtering.” by M. Srámek and A. Kaufman, "Accelerated Volume Rendering and Tomographic Reconstruction 1998 Volume Visualization Symposium, pp.