Visualization Expanding Scientific and Engineering Research Opportunities

Thomas A. DeFanti and Maxine D. Brown, University of Illinois at Chicago Bruce H. McCormick, Texas A&M University

omputational science and engi- a tiny fraction of a given solution . That is, neering (CS&E) describes a it is impossible to investigate the qualita- researcher's use of to tive global nature of numerical solutions. simulate physical processes. CS&E paral- Visualization holds With the advent of raster graphics, re- C can convert entire fields of vari- lels the development of the two other searchers modes of science: theoretical and experi- great promise for ables (representing density, pressure, ve- mental/observational. computational science locity, entropy, and so on) to color images. In addition to new methodologies, new The information conveyed to the re- technologies or mathematical tools have and engineering, searcher undergoes a qualitative change spurred the scientific revolutions. For because it brings the eye-brain system, example, calculus allowed Newton to provided we can meet with its great pattem-recognition capabili- codify the laws of nature mathematically the immediate and ties, into play in a way that is impossible and develop analytic methods for solving with purely numeric data. simple cases. Similarly, the development long-term needs of For example, an observer instantly sees of the van Neumann architecture the vortices, shock systems, and flow pat- gave scientists the ability to solve the dis- both toolmakers and tems in a visualization of a hydrodynamic cretized laws of nature for general and tool users. calculation, while these same patterns are complex cases. invisible in mere listings of several CS&E now relies heavily on scientific hundred thousand numbers, each repre- visualization to represent these solutions, senting field quantities at one moment in enabling scientists to turn mountains of time. When computing a space-time solu- numbers into movies and graphically dis- tion to the laws of physics, the particular play measurements of physical variables tion of the National Science Foundation numeric quantities at each event in time- in space and time. This article explores the report Visualization in Scientific Comput- space are not important; rather, what is convergence of science and visualization, ing,' represents much more than that. Visu- important is understanding the global in support of its successful growth and alization is a forth of communication that structure ofthe field variables that consti- development. transcends application and technological tute the solution and the causal intercon- boundaries. nections ofthe various components of that solution. What is scientific A tool for discovery and understand- visualization? ing. The deluge ofdata generated by super- A too] for communication and teach- computers and other high-volume data ing. Much ofmodern science can nolonger Computer graphics and image process- sources (such as medical imaging systems be communicated in print. DNA se- ing are technologies. Visualization, a term and satellites) makes it impossible for quences, molecularmodels, medical imag- used inthe industry since the 1987 publica- users to quantitatively examine more than ing scans, brain maps, simulated flights

12 0018-9162/89/OBOO.OO12f01 .00 019891EEE COMPUTER

through aterrain, simulationsof fluid flow, The visualization report recommends Tool users' short-term and so on, all need to be expressed and the development of a federally funded ini- taught visually over time. To understand, tiative providing immediate and long-term needs discover, or communicate phenomena, funding of both research and technology scientists want to compute the phenomena developments (see Table 1).' Research Every researcher requires a personal her desk over time, create a series of images that developments are the responsibility oftool computer orworkstation on his or . illustrate the interrelationships of various users - experts from engineering and the connected with a remote parameters at specific time periods, down- discipline sciences who depend on compu- However, not all scientists require the power. Hence, a load these images to local for tations for their research. Technology same level of computing environment is begin- analysis, and record and play back one or developments are handled by toolmakers three-tiered model categorizes visualiza- more seconds of the animation. - the visualization researchers who can ning to emerge that According to the visualization report, develop the necessary hardware, software, tion systems by such factors as power, "We speak (and hear) - and for 5000 and systems. cost, and software support. years have preserved our words. But, we cannot share vision. To this oversight of evolution we owe the retardation of visual compared to language. communication Table 1. Recommendations for a national initiative on visualization in scientific shared communication Visualization by computing. would be much easier if each of us had a CRT in the forehead."' Short-term Needs Long-term Needs Our CRTs, although not implanted in our foreheads, are connected to computers Tool users: Funding to incorporate Funding to use model of that are nothing more than extensions Computational visualization in visualization environments our brains. These computers, however, scientists and current research might not be in the same room with us. engineers They could be down the hall, across town, or across the country. Hence, the ability to Toolmakers : No funding necessary Funding to develop model communicate visually - and remotely - Visualization visualization environments on with computers and each other depends scientists and per- the accessibility, affordability, and engineers formance of computers and computer net- works.

Table 2. Visualization facility three-tiered hierarchy.

Model A Model B Model C

Hardware Supercomputer or Minisupercomputer or Advanced workstations (mini-/ super image computer image computer micro- image computer

Bandwidth (potential >109 10'-108 interactive rates, bits/second)

Location (where users Machine room Laboratory on a high-speed Laboratory on a national/ interact with the (at the center) local area network regional network display screen)

Software (in addition Commercial packages for Commercial packages Commercial packages and todiscipline-specific output only (no steering). are mostly output only. Some tools are widely available data generation and Research required to interaction is becoming for both computation and processing) develop interactive steering available. Research required interaction. Research required capabilities to improve discipline- , in languages, operating specific interaction systems, and networking

Administration Strength: Support staff Discipline-specific Decentralization visualization goals Weakness : Centralization Small support staff No support staff

August 1989 13 Workstations. Researchers need work- dows on the screen, and scientists must be als are for personal analysis, information stations with access to for able to "cut and paste" between the super- sharing among peers, or presentations in computerand applications running on their formal surroundings, equipment for pro- " immediate access to local graphics local machines . ducing photographs, slides, videotapes, or capabilities, laser disks needs to be in place and as easy " networked access to supercomputers, Access to supercomputers. Scientists to use as sending text files to a laser printer. and need to transfer data to and from a main Scientists need the ability to create ad " hard-copy recording. computation device, but today's networks hoc graphics to verify the integrity of their Local graphics . Workstations, mini- are too slow for use in visualization. Some simulations, gain insights from their analy- computers, and image computers are sig- temporary techniques reduce the demand ses, and communicate their findings to nificantly more affordable than supercom- for high bandwidth, such as off-peak im- others . Low-cost animation facilities puters, and they are more powerful and age transmission, image compression, should be connected to every user worksta- effective visualization tools. There are image reconstruction from abstract repre- tion so researchers can make scientific already some 20 million personal comput- sentations, and local image generation . "home movies" with little effort . High-end ers and workstations in the United States, Networking is therefore as critical as visualization capabilities and facilities compared with about 200 supercomputers . computer power in helping scientists . also should be available at all research users are increasingly treat- centers; high-end graphics become impor- ing supercomputers as one of many win- Hard-copy recording. Whether the visu- tant for presentation and publication of

Low-cost, visualization-compatible workstations and networks

Our ability to communicate visually and remotely with ficiently used to run complex simulation codes, the output of supercomputers and each other depends on which is numbers. With access to graphics, researchers can convert numbers to pictures to qualitatively examine the global (1) the ease with which we can use our office/home com- nature of their simulation output. Graphics can be made avail- puters to connect with the outside world, receive and transmit able on the host machine or, more efficiently, on the local work- visual information, and record this information on videotapes station. or slides, and (2) the cost/performance of today's networks. (2) Televisualization : graphical networking. As images re- quire more colors, higher resolution, or larger volumes of data, The Electronic Visualization Laboratory (EVL) at the Uni- they need more memory and become more impractical to trans- versity of Illinois at Chicago is doing research in both areas. mit over networks or phone lines, to store on disks, or to convert We are designing as our scientific animation workstation a and display on different frame buffers. EVL's Imcomp compres- low-cost computer system with a well-integrated visualization sion and conversion software converts images consisting of 24, programming environment. 16, or 8 bits per pixel to 16 or 8 bits per pixel, then compresses Users at the National Center for Supercomputing Applica- them further to 2 or 3 bits per pixel while maintaining a reason- tions (NCSA) - or any of the National Science Foundation- able full-color representation .'-2 The program takes only 0.4 sec- funded supercomputer centers, for that matter - cannot do onds to run on the X-MP at NCSA, and it converts and graphics remotely due to slow network speeds, centralized transmits a 512 x 512 x 24-bit image from NCSA to EVL over a and expensive graphics equipment, lack of graphics software 56-kilobyte line within a few seconds. tools, and the need for specialists in film/video production . Moreover, visuals must be transmitted from memory to mem- Our research is motivated by the recent availability of low- ory (that is, from supercomputer memory to frame buffer mem- cost graphics hardware and a good PC-based visualization ory in the local computer), not just from file to file as in elec- toolkit, coupled with a growing awareness that scientists tronic mail-type networks . NCSA's Telnet communications soft- need visualization more for personal/peer analysis than for ware has been modified to do this and expanded to include presentations. Imcomp routines that automatically compress images. EVL is integrating affordable commercial equipment with In addition to compression, value-added nodes speed up gra- specially designed graphics software to make visualization a phical transmission by balancing transmission costs with local reality for computational scientists - whether they use their computing costs. Model data is sent over networks and then computers on a stand-alone basis or connected to supercom- rendered or reconstructed at the scientist's end. EVL is cut- puters over networks . (Regarding affordability, academicians rently investigating the use of its AT&T Pixel Machine as a can generally receive $10,000 in equipment monies from their graphics that would render model data transmitted over departments or colleges without applying for external grants the network from the supercomputer and then transmit the to- -our yardstick is that equipment should cost no more than a sulting images over a local area network to individuals' desktop three-year-old Buick.) computers. EVL's scientific animation workstation, shown in the ac- companying figure, has hard-copy recording capability and (3) Truevislon Vista graphics board. Scientists need to be an easy-to-use visualization environment to facilitate scien- able to preview, record, and play back animations at any speed tists' needs. The following list corresponds to items 1-6 in and in cyclical fashion to examine the dynamics of their data the figure. changing over time, to spot anomalies, or to uncover computa- tion errors . The Vista board's large configurable memory allows (1) t3upercomputer access . Supercomputers are most ef- us to get anywhere from 32 screens at 512 x 512 pixels to 128

14 COMPUTER results once researchers conclude their rently do not produce graphics: they do fill equivalent power, and that advanced work- work . arrays with information that somehow gets stations and mini-/micro- image comput- piped to display devices. (Table 2 assumes ers have equivalent power.) Three-tiered model computational that supercomputers and super image Scientists should be able to select either environment. Observations of the way computers have equivalent power. Super more-expensive workstations with power- scientists use visualization suggest that a image computers, although not commer- ful visualization potential (model B) or three-tiered model, environment is evolv- cially available today except in the form of less expensive ones (model C) while main- ing, as defined in Table 2. Each model is a special-purpose flight simulator, will taining network connections to larger distinguished by hardware costs, comput- provide the specialized processing neces- machines (model A) to do computations ing power, bandwidth, location, software sary for real-time volume visualization.) when necessary. This interdependency can support, and administrative considera- Workstations give scientists more con- work quite well . For example, a scientist tions.' trol over their visual output (models B and can calculate 20-60 frames of a simulation This model environment assumes that C). A workstation typically addresses its sequence on a supercomputer, download scientists want as direct a visual connec- display memory the same way it addresses the images to a workstation to create a tion to their computations as possible . regular memory, incurring essentially no minimovie, and then play back the se- While supercomputers (model A) provide hardware overhead to display computed quence at any speed under local control?' scientists with powerful number-crunch- results. (Table 2 also assumes that minisu- (See sidebar, "Low-cost, visualization- ing tools for generating data, they cur- percomputers and image computers have compatible workstations and networks .")

Televisualization: graphical networking

Color0monitor

RT/1 local visualization programming environment

The Electronic Visualization Laboratory's RT/1 graphics language, an 80386-based , the Truevision Vista board, and consumer video gear comprise a scientific animation production facility that Is economical enough to be made available to research scientists and engineers on a broad scale.

screens at 128 x 128 pixels, all at 8 bits per pixel. The board is and record frames of animation? This equipment also comes also video compatible, so irDages can be recorded directly to with a built-in microphone so scientists can add narration or videotape. other sounds to visual recordings.

(4) Real Time/One (RT/1) local visualization programming (6) Color monitor. Today's consumer video systems not environment. Scientists need a set of tools for picture composi- only record but also can be attached to any television for im- tion, picture saving/restoring, fonts and text, resizing, rotation, mediate viewing of recorded material. Scientists can take a moving, copying, hand retouching (painting), color manipulation, small video unit to a conference and plug it into a television etc. They also need a local graphics programming environment there to share findings with colleagues . Should peers in other in which to develop new tools or extend the capabilities of exist- towns have similar equipment, colleagues could mail tapes to ing ones. each other for viewing. RT/1, an easy-to-use graphics programming language devel- oped by EVL faculty and students, meets the criteria required of References a visualization system environment. The language, written in C all and running under Unix and MS-DOS, runs on of EVL's 1 . M.D . Brown and M. Krogh, "Imcomp -An Image Compression workstations and personal computers. EVL is porting RT/1 to and Conversion Algorithm for the Efficient Transmission, Storage, new workstations as they are acquired, extending the capabili- and Display of Color Images," NCSA Data Link, Vol. 2, No . 3. Na- ties of the language, and developing application programs tional Center for Supercomputing Applications, June 1966, pp . 11- 24 . to the creeds of scientists. Computer 2. G. Campbell et al ., "Two-Bit/Pixel Full Color Encoding," (S) Consumer video recOfder/player. If it's not recordable, Graphics (SIGGraph Proc .) . Vol. 20, No . 4, Aug. 1986. pp . 215- it's not science. Moreover, the equipment for producing video- 223. sending files to laser tapes reeds to be as easy to use as text a 3. T.A. DsFanu and D.J. Sandn, "The Usable Intersection of PC printer. We are integrating low-cost consumer video equipment Graphics and NTSC Video Recording," IEEE Computer Graphics into tie workstation so scientists can quickly and easily preview and Appticadons, Oct. 1987, pp. 50-56.

August 1989 15 Table 3. Total corporate computing needs. (Source: Larry Smarr, NCSA, Sept. 1988.)

Computational Science and Engineering Data Processing

Corporate officer Vice president of research Vice president of management responsible or long-range planning information systems (MIS)

Tiered architectures Personal computers and graphics Personal computers; ; workstations ; midrange machines mainframes (mainframes/minisupercomputers ; Software portability only partially exists between supercomputers these levels, and then only within one vendor's Need existsfor multivendor, networked, product line. hierarchical computing .

Open systems Closed systems No vendor has emerged who offers integrated IBM and Digital Equipment Corporation systems and end-to-end solutions. As a result, manufacture all levels ofcomputers and the end users are faced with a confusing set of connections between them. products from various vendors and nowhere to turn for advice on how to integrate them .

Vendors Fragmented market populated by start-ups Mature, slow-growth marketplace dominated and extremely high-growth companies: by a few giant vendors, such as IBM and DEC. Workstations (Sun, DEC, Apollo, IBM, Hewlett-Packard, Apple, Silicon Graphics, Ardent, Stellar, AT&T Pixel, etc.); Midrange (DEC, IBM, Alliant, Amdahl, Convex, Scientific Computing Systems, Multiflow, ); Supercomputers (Cray, IBM).

Operating Systems Unix MVS, DOS, VMS (proprietary)

Networking protocols; Open network standards; SNA, DECnet (proprietary); telecommunications ; Long-haul telecommunications ; High speed = 50 Mbits/second speeds High speed = 1,000 Mbits/second Within a corporation, most networks hook many Because of the scarcity of$20 million "dumb terminals" up to a central mainframe where supercomputers. most universities and all the computing power resides . PCs are corporate CSC£ users are remote and generally used stand-alone ; those networked to a must gain access to supercomputers over mainframe generally use the network to download long-haul telecommunication lines. or upload files, and computing is decoupled.

Common unit Image (megabyte) Number (byte) of information Supercomputer simulations produce such enormous amounts ofdata that visualization is essential.

Common unit Mflops MIPS for computation speed

Additional models D, E, and F, corre- Tool users' long-term vendor environments, leading-edge tech- sponding to personal computers, alphanu- nology, and customer service and support meric CRT terminals, and batch output, needs The list of research opportunities for respectively, also exist. They do not repre- visualization in scientific computing is sent advanced visualization technology, so CS&E is emerging as a new marketplace long and spans all of contemporary scien- theyare not included in-our model environ- with needs distinct from those of data tific endeavor. The sidebar "Scientific and ment. Note, however, that model F has processing, as shown in Table 3. Success engineering research opportunities" pre- been used toproduce agreat deal ofanima- in the CS&E marketplace ofthe 1990s will sents specific examples ofadvanced scien- tion for both the scientific and commercial depend ona commitmentto standards,ease tific and engineering applications to show entertainment industries for the past 20 of use, connectivity, open systems, inte- years. grated systems, softwareportability, multi- (Continued on p. 22)

16 COMPUTER Scientific and engineering research opportunities

supercomputers to model complex sys- tems. Two types of images can currently be generated : realistic pictures of mole- cules and 3D line drawings. Raster equipment is used to create realistic rep- resentations and animations, while vector hardware, used for real-time display and interaction, creates line drawings. The image at left is a 30 line drawing of the rhinovirus, the common cold virus, showing its geometric structure and com- plexity. The image at right is an artistic rendering of the human papilloma virus (HPV). It was done by a group of Chi- cago-area artists who appreciate the underlying mathematics of nature and the complexity of the inner workings between atoms.,

Left-hand ® 1988 T .J . O'Donnell . Data courtesy of Molecular modeling. The use of inter- Dr. Rossman, Crystallography Group . Purdue Univ . active computer graphics to gain insight Image courtesy of the EVL, Univ . of Illinois at Chi- into chemical complexity began in 1964 . cago. Right-hand ® 1989 (Artr Laboratory . Illinois Interactive graphics is now an integral Institute of Technology . (Art)" artists: Donna Cox, NCSA, Univ . of Illinois at Urbana-Champaign ; part of academic and industrial research Stephan Meyers. Dan Sandin, and Tom DeFanti, EVL, on molecular structures, and the method- Univ . of Illinois at Chicago ; Ellen Sandor . (Art)" ology is being successfully combined with Laboratory . Illinois Institute of Technology .

Medical Imaging . Scientific computa- show the skin surface or to reveal inter- data. Physicians can use this technique tion applied to medical imaging has cre- nal structures. The bones of the rib cage, to relate the position of internal structures ated opportunities in diagnostic medicine, shoulder blades, and spine can be seen such as tumor sites to external land- surgical planning for orthopedic prosthe- in the image on the right, as well as the marks. These images were generated us- ses, and radiation treatment planning . In trachea, lungs, heart and diaphragm . ing the Voxvu volume rendering tool on a each case, these opportunities have been The above-right image is a shaded Sur- Sun workstation with the TAAC-1 Appli- brought about by 2D and 3D visualiza- face volume rendering of a 256 x 256 x cation Accelerator. tions of portions of the body previously 61 magnetic-resonance imagery (MRI) inaccessible to view. . scan of a human head. The rendering The above-left image is a shaded sur- shows a mixture of surface and slice. face volume rendering of a 128 x 128 x based techniques, where external struc- with O 1989 Chuck Mosher and Ruth Johnson, Sun Mi- 197 computerized tomography scan of a tures such as the skin are rendered crosystems. Data for above-left image courtesy of Eric tree sloth. The opacity of various struc- surface shading, while slice planes are Hoffman, UPA. Data for above-right image courtesy of tures can be interactively modified to voxel-mapped to reveal the original MRI Jeff Shaw, Vanderbilt Univ .

August 1989 17 Brain structure and function. Rutgers system calculates the orientation of the the suspected lesions so that the system University is using computer vision and brain and uses the segmentations pro- can reject false positives and determine visualization methods to automatically vided by the low-level methods to fit a de- the affected organs . The system has detect white-matter lesions in MRI scans formable model to each patient's brain to been tested on more than 1,200 images of the human brain. determine the position and shape of diffi- from 19 patients, producing good results. In the above-left image, low-level vision cult-to-identify organs or regions of inter- methods locate the outline of the brain, est. This customized model, shown in the landmarks such as the interhemispherical above-right image, is used to obtain infor- O 1989 loannis Kapouleas. Computer Science Dept ., fissure plane, and suspected lesions . The mation about the anatomical position of Rutgers Univ .

Mathematics . These images illustrate The above-left image is a quaternion a type of fractal known as the Julia set . A filled-in Julia set minus its front-upper-left filled-in Julia set is a set of points that do octant ; the inner components of the four- not converge (or diverge) to infinity after cycle are revealed, defining its basin of repeated applications of a function, such attraction . The above-right image is a as f (z) = z 2 + c. These functions are of- visualization of a dendritic quaternion Ju- ten investigated in the complex plane, but lia set in the complex plane; the unusual they also exist in the quaternions, a coor- lighting uses a 3D gradient in the com- dinate system that spans one real and plex plane. three imaginary axes. Visualization helps -mathematicians understand these equa- tions, which are too complex to conceptu- alize otherwise .2 O 1989 John Hart . EVL, Univ. of Illinois at Chicago.

If3 COMPUTER Geosciences : meteorology . The are used to view multiple surfaces ; shad- dominant negative vorticity . study of severe storms through observa- ing is used to display individual solid sur- The above-right image is from an ani- tion and modeling helps research meteor- faces. mated simulation of a storm over Kansas, ologists understand the atmospheric con- The above-left image uses voxel (grid in which the rainwater surface was poly- ditions that breed large and violent torna- cell) data to display rainwater and vertical gonized (tiled) and then rendered. The does and the mechanisms by which tor- vorticity information about a storm,' pro- simulation clearly reveals substantial vari- nadoes form and persist .' Theoreticians viding scientists with more information ations in the structure of the rainwater and field workers obtain information on than if they had observed the storm with field not apparent earlier . behavior that cannot be safely observed ; their eyes . The fuzzy region indicates low study the interactions of various environ- rainwater amounts while the bright white ments, characterized by differing vertical regions indicate large amounts of rain- Above-left m 1988 Robert Wilhelmson and Craig . Illinois at Urbana-Champaign . wind, temperature, pressure, and mois- water within the cloud. The vertical vor- Upson. NCSA . Univ of rain- Above-right ® 1988 Robert Wilhelmson . Crystal ture structures ; and obtain useful guides ticity is texture mapped onto the Shaw . Lou Wicker. Stefen Fangmeier, and the NCSA for future research. water with color; purple indicates domi- Visualization Production Team . Univ. of Illinois at Transparency and volumetric rendering nant positive vorticity and blue indicates Urbana-Champaign .

Space exploration. The field of plane- physics, astronomy, and astrophysics . the Voyager 2 Neptune encounter to oc- tary study involves the accumulation of The above-left image is from an ani- cur irr late summer of 1989 . This image il- huge volumes of data on the planets in mated simulation of the dynamics of Ura- lustrates the path of the Voyager 2 as the solar system. Enough data is now nus' magnetosphere. The simulation viewed from Earth. available that scientists are beginning to shows that the angle of the dipole axis Above-left ® 1989 Computer Graphics Group of the integrate observed phenomena and the- (purple arrow) is offset from the planet's Jet Propulsion Laboratoryand G . Hannes Voigt of Rice ory from other fields involved in planetary angle of rotation (aqua arrow) . The Univ . Above-right m 1989 Computer Graphics Group study: meteorology, geography, planetary above-right image is from a simulation of of the Jet Propulsion Laboratory .

August 1989 19 Astrophysics . Computational astro- due to the presence of the black hole, metric embedding diagram, the curvature physicists at the NCSA work with artists while the color scale represents the of the space surrounding the black hole is in an attempt to see the unseen and cre- speed at which idealized clocks measure represented by the surface on which the ate visual paradigms for phenomena that time (with red representing the slowest waves propagate . The white ring locates have no known visual representation. clocks and blue representing the fastest) . the surface of the black hole, and the re- An embedding diagram of a Schwarz- A black hole emits gravitational radia- gions above and below represent the ex- schild black hole and the behavior of its tion after it has been struck by an incom- terior and interior of the black hole, re- gravitational field, illustrated in the above- ing gravity wave. The above-right image spectively . left image, was obtained from a numeri- is from an animated sequence that cal solution of Einstein's numerical rela- shows, for the first time, the influence of ® 1989 David Hobill, Larry Smarr, David Bernstein, tivity equations . The surface of the dia- the curved space on the propagation of Donna Co :, and Ray Idaszak, NCSA, Univ . of Illinois gram measures the curvature of space the radiation . Through the use of an iso- at Urban&-Champaign.

Computational fluid dynamics . Com- plume whose morphology, or shape, is putational astronomers rely on supercom- strikingly similar to that of a radio lobe of puting and visualization techniques to a wide-angle tailed galaxy. The morphol- understand why jets from some galaxies ogy of the jet after impact is emphasized flare dramatically . Magnetohydrodynam- through the use of pseudocolor. This re- ics code is used to solve equations that search has given astronomers important describe the flow of a fluid or gas with clues about why jets from some radio gal- magnetic fields using finite differences . axies flare into broad plumes while jets - The above image is a visualization of a from others remain remarkably straight cosmic jet traveling at Mach 2.5 passing and narrow.s.a through a shock wave (located at the left m 1989 Michael Norman and Donna Cox of the NCSA, of the image) . The jet abruptly slows and Univ . of Illinois at Urban&-Champaign, and Jack Bums breaks up into a broadened subsonic and Martin Sulkanen of the Univ . of New Mexico .

20 COMPUTER Finite element analysis. Finite ele- References ment analysis is used in this example to distribution in a beam at 1 . R.P. Feynman, Surely You're Joking, Mr. show the stress Feynman! Adventures of a Curious Charac- its maximum tip displacement in the third ter, Bantam Books, 1986, pp. 236-253. eigenmode. The results were computed using linear elastic elements and a 2. J.C. Hart . D.J . Sandin, and L.H. Kauffman, Deterministic 3D Fractals' to lumped-mass approximation. "Ray Tracing be published in SIGGraph 89 Conf. Proc., The top image uses a conventional ap- Computer Graphics, Vol. 23. No . 4, Aug. proach of displaying the stress values on 1989. the outer surface of the deformed shape. "Numerical Simulations The middle image uses a cutting plane to 3. R.B. Wilhelmson, of Severe Storms," Proc. Fourth Int'1 look at the stress values on a cross sec- Symp .: Science and Engineering on Cray tion of the root of the beam. The bottom Supercomputers, Cray Research, Oct . image shows a different view of the beam 1988 . iso-contour stress surface to and uses an 4. C. Upson and M. Keeler, 'Vauffer : Visible convey the three-dimensional nature of Volume Rendering,' SIGGraph 88 Cant. the stress concentration at the root of the Proc., Computer Graphics, Vol . 22, No. 4, beam. These images are still frames from Aug . 1988, pp. 59-64. fully animated and interactive models . 5. K.-H.A. Winkler and M.L . Norman, "Muna- They were computed and rendered on a color: Understanding High-Resolution Gas Silicon Graphics 4D/120 GTX workstation Dynamical Simulations Through Color using the SolidView program to perform Graphics,' Astrophysical Radiation Hydro- real-time cutting and iso-contour surface dynamics . D. Reidel Publishing, 1986, pp. 223-243 . generation . 6. N.J. Zabusky, "Computational Synergetics," Physics Today, July 1984, O 1989 James M. Winget, Silicon Graphics. reprint.

21 August 1989 (Continuedfrom p. 16) Patches. The next level of sophistica- copying, hand retouching (painting), color tion represents surfaces as curved surface how visualization tools are helping re- manipulation, etc. Together, these func- pieces called patches. This is still largely searchers understand and steer computa- a tions comprise a visualization environ- software domain, although we expect hard- ment system, which is to tions. Our examples fall into the following visualization ware to appear soon. The most categories: advanced what an operating system is to general software packages handle a variety of computing. " Molecular modeling, patch types. They also provide very so- " Medical imaging, phisticated lighting models with multiple- Window systems. Windowing systems " Brain structure and function, colored lights and distributed or point are commonplace in black-and-white bit " Mathematics, sources located either at infinity or in the graphics and are being extended to color " Geosciences (meteorology), scene. graphics. Visualization software must in- " Space exploration, Antialiasing is assumed, and the pack- corporate and remain consistent with win- " Astrophysics, ages handle optical effects such as trans- dowing concepts. " Computational fluid dynamics, and parency, translucency, refraction, and re- " Finite element analysis. flection . Research software provides even Volume visuali:anon. Volume visuali- more features that produce greater realism, zation software is still rudimentary. Algo- such as articulated motion blur, depth -of rithms for rendering lines, curves, sur- field, follow focus, constructive solid faces, and volumes into volume memories Toolmakers' geometry, and radiosity. are only now becoming available. .6 Hid- short-term needs The software contains no practical limit den volume removal is unknown, the coin- on scene complexity (such as the number positing of volumes is yet to be fully ad- Commercial industry currently supports of allowable polygons), but computation dressed, 3D paint programs (sculpting visualization hardware and software, as of highly complex scenes on a supercom- programs) have yet to be written, and listed below. There is a pressing need to puter can take anywhere from 0.5 to 1 .5 general utilities for arbitrary rotation and educate the scientific and engineering re- hours per frame. size change of volumes do not exist. In search communities about the available other words, there is much research to be equipment. Image processing. Image processing done in this field. software has followed a separate path over Software. Commercial visualization the last 15 years. The elaborate software Hardware. The following are available software exists in the following categories : packages now available provide functions commercial visualization hardware tools: such as convolution, Fourier transform, Lines. The earliest software forgraphics histogram, histogram equalization, edge Input devices. Current digital input drew lines in three dimensions and pro- detection, edge enhancement, noise reduc- devices include supercomputers, satel- jected them onto a two-dimensional plane, tion, thresholding, segmentation, bicubic lites, medical scanners, seismic recorders, offered viewing transformations for look- and biquadratic warping, and resampling. and digitizing cameras. The rapidly in- ing at the result, and offered transforma- Many of these functions have been hard- creasing bandwidth of these devices em- tions (scale, rotate, and translate) for de- wired into special boards . General-pur- phasizes the need for work in volume visu- scribing the line objects. The theory and pose processors have only recently be- alization. practice of drawing lines, expressed in come powerful enough to make software We expect continued improvement in homogeneous coordinates, and the control competitive with hardware while main- the resolution and bandwidth of input and display of lines using 4 x 4 matrices, taining generality. Image computers can devices. Supercomputers will get faster represented a major development in com- run both computer graphics and image and the resolution ofimages from satellites puter graphics. processing software packages . will increase. Real-time video digitizers A variety of current standards incorpo- already exist. Monochrome digital digitiz- rate these basic principles, and the CAD/ Animation. In its broadest sense, anima- ers with 2,048 x 2,048-pixel resolution are CAM industry has embraced this level of tion means movement. It frequently con- becoming quite fast, although they do not the art. It is cheap enough to put on every notes the complex motion ofmany objects, yet operate at real-time speeds. Print-qual- engineer's desk and fast enough for real- possibly articulated, moving simultane- ity input scanners are still quite expensive, time interaction. ously, and interacting with one another. but we expect the prices to fall as digital Animation is desirable for the visualiza- technology cheapens and competing scan- Polygonal surfaces. The next level of tion ofdynamic, complex processes. Basic ning technologies mature. CCD (charge- software - surfaces represented by poly- animation control routines should be part coupled device) array input scanners will gons - has only recently been built into of any standard visualization tool kit. improve in resolution and become serious hardware. Polygon filling, or tiling, is candidates for input devices in high-reso- commonly available in hardware and soft- Glue. A class of software appreciated by lutlon work. ware. Hidden surface removal is included, visualization professionals but not neces- Interactive input devices are continually and antialiasing ofpolygon edges is some- sarily by scientists is the "glue" used to improving. Common 2D devices include times provided to remove distracting stair- combine images generated or analyzed by knobs, switches, pedals, mice, and tablets. steps, or jaggies. Light sources can be the packages describedabove. Forconven- Tablets are the most general and also reed incorporated into the rendered image, but ience, a user must have tools for picture the most improvement; they need higher they are usually point sources at infinity composition, picture saving/restoring, resolution, higher speed, andmore degrees emitting white light. fonts and text, resizing, rotation, moving, of freedom.

22 COMPUTER Six-dimensional interactive devices are ware in special machines. Chips are now " Workstation-driven use of supercom- also available, providing the usual 3D being built to speed up certain aspects of puters positional information plus three degrees surface rendering, particularly the tiling of " Graphics-oriented programming envi- oforientation information (yaw, pitch, and polygons . By 1990, full hardware support ronments roll). Higher-dimensional devices, such as ofsurface graphics will beavailable, offer- " Higher-dimensional visualization of thedataglove, havebegun toappear. These ing rendering features such as texture scalar, vector, and tensor fields will improve to offer higher resolution, mapping, bump mapping, antialiasing, " Dynamic visualization of fields and higher speed, and lower cost. reflections, transparency, and shadows. flow Vector machines will initially serve as " High-bandwidth picture networks and Output devices. Raster displays of 2D powerful, real-time front ends to surface protocols frame buffers have improved steadily to machines. Eventually, surface machines " Massive data-set handling, notably for offer more colors, higher resolutions, and will be cheap and fast enough to permit signal and image processing applica- less flicker. A typical color raster display scientists to do real-time design using sur- tions today offers a 1,280 x 1,024-pixel display faces rather than lines. " Vectorized and parallelized algo- at 60 frames per second and 24 bits ofcolor Among image processors, fast planar rithms for graphics and image process- per pixel (16 megacolors). machines have existed for some time. ing High-definition television (HDTV) -a These machines contain special boards for " Specialized architectures for graphics proposed standard that will offer larger, certain aspects of image processing, such and image processing brighter, sharper pictures than currently as fast Fourier transforms . Faster versions " A framework forinternational visuali- available in video-will affect visualiza- are becoming available that have wider zation hardware and software stan- tion. Also, video is moving toward an all- processing capabilities and higher resolu- dards digital format, designated 4:2:2, to stan- tions. In fact, the notion ofa general-pur- dardize digital interconnections of diverse pose image processor that can implement Networking. The application of net- video products. any image processing algorithm as a pro- works to visualization, called televisuali- Color raster displays will evolve toward gram is becoming common. zation, encompasses much more than text 2,048 x 2,048 pixels in the next several transfer(such as electronic mail) and gate- years. The displays themselves already way protocol decoding . It also involves exist in limited quantities, but the compu- Toolmakers' image transfer, whichentails compression, tational bandwidth required to feed them is decompression, rendering, recognizing, still lacking. Black-and-white 2D raster long-term needs and interpreting . Televisualization re- displays already have resolutions greater quires a major enhancement over existing Raw computing power would be more than 2,048 x 2,048 pixels with enough network capabilities in the following ar- effectively harnessed than it is today if bandwidth to feed them. These displays eas: calculations could be understood pictori- will certainly reach even higher resolu- ally and their progress guided dynami- tions. in the next five years. Increased data rates. The sheer scale of cally. Modem modes ofcomputing involve Stereo displays are also beginning to graphics and imaging data sets challenges interactive, extemporaneous generation of appear commercially, and we confidently the current bandwidth and interactivity of views from masses ofdata and exploration predict that these will improve in screen networks. Networks handle screenfuls of of model spaces by interactive steering of size, resolution, brightness, and availabil- textual information well; network nodes computations . ity. These displays will be quite helpful in are simply gateways that neither add nor A scientist's ability to comprehend the volume visualization. detract from the quality of the message. results of his orher computations depends Other output devices are similarly im- But a 512x 512-pixel image with 8 bitsper on the effectiveness of available tools. To proving. HDTV will spur the development pixel has approximately 100 times more increase that effectiveness, we need to of compatible recorders. Film recorders information than a screen of text with 25 will become cheaper as the technology " encourage the production of docu- rows and 80 characters per row. A 1,024 x becomes cheaper and the competition mented, maintained, upward-compat- 1,024 x 1,024-voxel volume with 48 bits matures. Should stereo become a widely ible software and hardware ; per voxel contains 16,000 times more in- accepted mode of presentation for volume " motivate manufacturers to solve net- formation than a 512 x 512-pixel image. visualizations, then stereo film and video work bottleneck problems; Gigabit speeds are sufficient to pass vo- standards will have to be developed. " encourage universities to incorporate lumes of the current size of 256 x 256 x CS&E and visualization in computer 256 voxels with 4 bytes per voxel, but this Workstations. Fast vector machines are science, engineering, and discipline- rate will have to be extended within several now common and have extensive use in science curricula; and years to 1-gigabyte/second channels . such areas as CAD/CAM and real-time 3D " guarantee the publication and dissemi- design . Recently, they have improved to nation of research and results on a Campressionldecompression algo- offer color vectors and perfect end-match- variety of media. rithms . Compression improves the speed ing. Frame buffers have been added so that with which visual data is transmitted. Hardware, software, and systems. surface raster graphics can be combined Current schemes for full-color image General visualization issues that need to be with vector displays. compression work well. but other forms supported include: Also, fast surface machines are about to of compression must be researched, and arrive . They exist in simplified forms al- " Interactive steering ofsimulations and comprehensive protocols must be devel- ready and in more advanced states as firm- calculations oped for managing all these capabilities :

August 1999 23

Table 4. The evolution of communica- tists communicate with data by manipulat- text, and visual media do not count. Fund- tion tools. ing its visual representation during proc- ing itself is based on the careful prepara- essing. The more sophisticated process of tion and evaluation ofproposals, whichare Communications Number of navigation allows scientists to dynami- documents full of words and numbers. media years old cally modify, or steer, computations while As scientists depend more and more on they are occurring. This lets researchers the electronic network than on the printed Sight 5 x 108 change parameters, resolution, or repre- page, they will need new technologies to Speech 5 x 101 sentation, and then see the effects. teach, document, and publish their work. Writing 5 x 10' Until scientists can build on each other's Print broadcasting 5 x IOz Teaching CS&E and visualization. work, productivity will lag. Publishing Visual broadcasting 5 x 10' The principal barrier to growth in the (specifically textual materials) has always Visualization 5 x 10° CS&E market is the fact that corporate been acritical part of this building process, researchers and managers lack education and it is one of the primary bottlenecks in and training in CS&E technologies and CS&E's progress. methodologies. Few industrial researchers Reading and writing were only democ- " Transmit the procedures to create the know how to use distributed CS&E to do ratized in the past 100 years. Today, they images rather than the images them- their work and, more importantly, they-do are the accepted communication tools for selves. not know how to think computationally scientists and engineers. Table 4 shows " Transmit endpoints of vector images. and visually . Otherroadblocks include the that, in time, visualization will also be " Transmit polygonal, constructive following: democratized and embraced by research- solid geometry, or bicubic patches of ers. surface models. " The Association for Computing Electronic media, such as videotapes, " Transmit semantic descriptions of the Machinery's approved computer science optical disks, and floppy disks, are now objects. curriculum lists computer graphics as necessary for the publication and dissemi- merely one ofmany optional topics; image nation of mathematical models, process- Value-added processing at nodes. processing is not mentioned at all. ing algorithms, computer programs, ex- Value-added nodes also speed up graphi- " Engineering accreditorsdo not require perimental data, and scientific simula- cal transmission. Computers process text computer graphics or image processing . tions. The reviewer and the reader need to and numbers in main memory, occasion- " Many engineering school deans are test models, evaluate algorithms, and exe- ally transmitting some ofthem to peripher- unaware ofthe importance ofvisualization cute programs themselves, interactively, als. Images, however, often must be trans- or cannot justify the hardware and soft- without an author's assistance. Similarly, ferred to special memories for rendering, ware expense involved in teaching the scientific publication must extend to use 3D imaging, or viewing. Each instance of subject. visualization-compatible media. transferring and processing an image aims " The number of tenured faculty teach- to increase its visualization value to the ing computer graphics in American uni- he use of visualization in scien- scientist. The ability to process images at versities is about the same today as 15 tific computing - in academia, various nodes along a network embraces years ago, and they are roughly the same government research laborato- the central concept of distributed process- people. ries, and industry - will help guarantee ing. " Scientists, while T educated to read and In distributed computing, transmission write, are not taughtto produce or commu- " US preeminence in science and tech- costs are balanced with local computing nicate with visuals. nology, costs. It sometimes makes more sense to " a well-educated pool of scientists and send model data over networks and then Publication and dissemination. Con- engineers with the quality and breadth render or reconstruct the data at the temporary scientific communications of experience required to meet the scientist's end. This presumes that there is media are predominantly language-ori- changing needs ofscience and society, appropriate equipment at both ends, that ented. Printed media are coupled weakly, and the various software modules are compat- if at all, to the visual world ofspace-time. " American industries that can success- ible with one another, and that the software By contrast, half the human neocortex is fully compete in the international eco- can run on a variety of equipment types. devoted to processing visual information. nomic arena. A televisualization network for image In other words, current scientific commu- The information age has yet to deal passing between machines is analogous to nication leaves out half-the right half- with information transfer. Visualization the software paradigm of message passing of the brain. An integral part of our visuali- technologies can help lead the way to bet- between process layers. This type of net- zation task is to facilitate visual communi- ter global understanding and communi- working, combined with interaction, can- cation from scientist to scientist, engineer cation. not be achieved using conventional For- to engineer, through visualization-com- tran subroutine calls. Significant software patible media. development and ptotocol standardization Publication and grants, and therefore are necessary to bring televisualization to tenure, rarely come to researchers whose References the discipline sciences. productivity depends on or produces visu- 1. B.H. alization results. Superiors evaluate schol- McCormick, T.A. DeFanti, and M.D. Brown, eds., "Visualization in Scientific Interaction capabilities. Interactive vis- arly work by counting the number ofjour- Computing." Computer Graphics, Vol. 21, ual computing is a process whereby scien- nal articles published; publications are No. 6, Nov. 1987.

24 COMPUTER

2. T.A . DeFanti and M.D. Brown, "Scientific Animation Workstations : Creating an Envi- ronment for Remote Research, Education, and Communication," Academic Comput- ing, Feb. 1989, pp. 10-12, 55-57.

3 . T.A . DeFanti and M.D. Brown, "Scientific Animation Workstations," SuperCom- puting, Fall 1988, pp. 10-13.

4. T.A. DeFanti and M.D. Brown, "Insight Maxine D. Brown is associate director of the BruceH. McCormick is professor of computer Through Images," Unix Review, Mar. 1989, pp. 42-50 Electronic Visualization Laboratory at the science and director of the Visualization Labo- . University of Illinois at Chicago, where she is ratory at Texas A&M University. McCormick responsible for funding, documentation, and is an international lecturer and author in the 5. R.A . Drebin, L. Carpenter, and P. Hanra- promotion of its research activities. Brown is an areas of computer vision, computer architec- han, "Volume Rendering," Computer international lecturer and author in the com- ture, and scientific visualization . He has held Graphics (SIGGraph Conf. Proc .), Vol. 22, 4, . puter graphics field, and she has consulted for a positions at the University of Illinois at Urbana- No. Aug . 1988, pp. 65-74 number of companies in the computer graphics Champaign, where he was principal investiga- industry in the area of professional and techni- tor of the Illiac III image-processing computer, 6. C. Upson and M. Keeler, "VBuffer: Visible Volume Rendering," Computer cal communications. and at the University of Illinois at Chicago. Graphics Brown is past secretary and vice chair for McCormick has served on committees ofthe (SIGGraph Conf. Proc .), Vol. 22, No. 4, Aug. 1988, 59-64. operations of ACM SIGGraph, is a member of National InstitutesofHealth, participated in the pp. the executive committee of the IEEE Technical 1981 Fifth-Generation Computer Conference 7. G. Campbell et al ., "Two-Bit/Pixel Full Committee on Computer Graphics, and was in Japan, and was chair of the NSF-sponsored coeditor of Visualization in Scientific Comput- Panel on Graphics, Image Processing, and Color Encoding," Computer Graphics ing. (SIGGraph Conf. Proc .), Vol. 20, 4, She received her BA in mathematics from Workstations and coeditor of Visualization in No. Temple University in 1972 and her MSE in Scientific Computing. He received his BS in Aug. 1986, pp. 215-223 . computer science from the University of Penn- physics from the Massachusetts Institute of sylvania in 1976 . She is a member of the IEEE Technology in 1950 and his PhD in physics Computer Society . from Harvard University in 1955 . He is a member of the IEEE .

Acknowledgments We acknowledge the following individuals for their insightful contributions and comments during the preparation of the NSF report Visualization in Scientific Computing: David Arnett, Gordon Bell, James F. Blinn, Frederick P. Brooks Jr., Met Ciment, James Clark, John Connolly, Jerome Cox, Martin Fischler, Donald Greenberg, Andrew Hanson, Albert Harvey, Laurin Herr, Thomas A. DeFanti is a professor of electrical David Hoffman, Robert Langridge, Thomas Lasinski, Carl Machover, Partrick Mantey, Mike engineering and computer science and codirec- McGrath, Nicholas Negroponte, ArthurJ . Olson, Jeff Posdamer, Azriel Rosenfeld, David Salzman, tor of the Electronic Visualization Laboratory Larry Smarr, Alvy Ray Smith, Baroa Szabo, Richard Weinberg, Turner Whitted, Karl-Heinz at the University of Illinois at Chicago, and an Winkler. adjunct professor at the National Center for Supercomputing Applications . He is an inter- national lecturer and author in the computer graphics field, and he has had many interactive computer graphics installations in museums and conferences worldwide. He also serves on the editorial boards of several publications, is past chair of ACM SIGGraph, is editor-in-chief ofthe SIGGraph VideoReview, wasrecipient of the 1988 ACM Outstanding Contribution Award, and was recently appointed to the Illi- nois Governor's Science Advisory Committee . Moving? DeFanti was vice chair of the National Sci- Name (Please Print) ence Foundation-sponsored Panel on Graphics, Image Processing, and Workstations and coedi- PLEASE NOTIFY torofits 1987 report Visualization inScientific US 4 WEEKS New Address Computing. He received his BA in mathematics IN ADVANCE from Queen's College in New York in 1969 and his PhD in computer science from Ohio State City State/Country Zip University in 1973.

MAIL TO: " This notice of address change will apply to all IEEE Service Center ATTACH IEEE publications to which you subscribe . Readers may contact DeFanti and Brown at 445 Hoes Lane LABEL " List new address above. the Electronic Visualization Laboratory, De- Piscataway, NJ 08854 HERE " If you have a question about your subscription, partment of Electrical Engineering and Com- place label here and clip this form to your letter. puterScience, University ofIllinoisat Chicago, PO Box 4348, Chicago, IL 60680.

August 1989 25