ENNPACI & SDSC ISION+UARTERLY SCIENCE MAGAZINE V OL. 18 NO. 2 V APRIL – JUNE 2002

THE SEARCH FOR LIFE: ARE WE ALONE? A SUPERCOMPUTING SUCCESS STORY

GRID MIDDLEWARE • MODELING ECOSYSTEMS NPACI Building the Computational Infrastructure for Tomorrow’s Scientific Discovery The National Partnership for Advanced Computational Infrastructure (NPACI) joins Contents 47 partner institutions in 21 states, Australia, Italy, Spain, and Sweden, in creating the GUEST EDITORIAL NEWS foundation for a ubiquitous, continuous, and pervasive computational environment to 1 The TeraGrid, NPACI, and You 16 New Computational Method Could support research by the world’s scientists. Shorten Drug-Development Time NPACI is led by UC San Diego, funded SCIENCE SUCCESSES primarily through the NSF’s Partnerships for 16 Filling a Digital Library with Sea Bottom Advanced Computational Infrastructure 2 Data program, and has its focus of activities at the AStar Is Born: Animating the Solar 16 San Diego Supercomputer Center (SDSC). System’s Origin SDSC Leads Collaboration to Integrate Access to Government Information NPACI INFORMATION 6 The Evolution of 16 Bart McDermott Joins SDSC as Director Fran Berman, Director Binary Black Holes for Development and Communications David Hart, Communications Director 8 Projecting How 17 NPACI’s Blue Horizon Sets Usage Mark [email protected] Climate Change May 858-534-8314 17 p. 6 Rearrange Mexican John Marburger SDSC Ecosystems Advocates Balanced R&D Portfolio SDSC is a campus research unit of UC San 10 Lithium Batteries Getting a Power Boost Diego and the focus of activities for NPACI. from Supercomputer Simulations 17 Visualization Since 1985, SDSC has served the U.S. Software Improves scientific community as a national laboratory p. 8 for computational science and engineering. TERAGRID Collaborations 17 Calendar GENERAL INFORMATION 12 Retooling Middleware Phone: 858-534-5000 for Grid Computing BACK COVER [email protected] Visualizing the Internet Universe www.npaci.edu EDUCATION www.sdsc.edu 14 3-D Views of Science Subscribe to ENVISION and Online at: for the Public p.14 www.npaci.edu/Press/subscriptions.html ONLINE www.npaci.edu/Online Online is a biweekly, Web newsletter of the latest research and developments from NPACI and SDSC.

ENVISION www.npaci.edu/enVision enVision, ISSN 1521-5334, is published quarterly by NPACI and SDSC Communications. For a free subscription or to make address changes, visit the website or contact the editor. EDITOR: Rex Graham, [email protected], FRONT COVER: A STARLET IS BORN 858-822-5408 SDSC’s David Nadeau played a key role in creating a realistic DESIGNER: Gail Bamber, animation of the birth of the solar system, part of the planetarium [email protected], 858-534-5150 show “The Search for Life: Are We Alone?” dazzling thousands of WRITERS: Cassie Ferguson (CF), Mike visitors a day at the American Museum of Natural History in New York. Gannis (MG), Rex Graham (RG), David The animation sequence starts outside the Milky Way Galaxy and Hart (DH), Merry Maisel (MM), Paul zooms inward, toward a gaseous nebula in one of the galaxy’s spiral Tooby (PT) arms. Protostars condense from the cloud’s gas and dust. Then the Any opinions, conclusions, or recommenda- largest protostar ignites in hydrogen fusion; its brilliant blue-white tions in this publication are those of the light energizes nitrogen and oxygen gas in the nebula, producing eerie author(s) and do not necessarily reflect the red and green glows. The view shifts to a smaller newborn star with a views of NSF, other funding organizations, thick dusty disk around its equator and jets of material streaming from SDSC, UC San Diego, or the NPACI its north and south poles. The star brightens, the disk flattens, and partner institutions. All brand names and dust and gas accrete into lumps—infant planets. One of them product names are trademarks or registered eventually becomes Earth. Nadeau’s effort is part of NPACI’s Scalable trademarks of their respective holders. Visualization alpha project. AMNH/SDSC/NCSA The TeraGrid, NPACI, and You GUEST EDITORIAL

he TeraGrid is upon us. Many of you have probably heard about the next generation of computing T infrastructure that NPACI and our partners in the National Computational Science Alliance are deploying. You may have also seen a graphic showing that the TeraGrid consists of more than 13 teraflops of computing power distributed at SDSC, the National Center for Supercomputer Applications (NCSA) at the University of Illinois at Urbana-Champaign, Caltech in Pasadena, and Argonne National Laboratory in Argonne, IL. In addition, hundreds of terabytes of storage are being deployed at SDSC and Caltech. Each computing node by itself represents an increase in the total computational power available to the NPACI user community.

Taken individually, each TeraGrid node is capable cal roles in creating today’s grid technology, much of of enabling advances in science. However, the which is being used to build the TeraGrid. TeraGrid is more than the sum of its parts. In addition At the core of the TeraGrid software story is the BY CARL KESSELMAN to the normal high-performance networking infra- NSF Middleware Initiative, or NMI, whose goal is to Chief Software Architect, structure that links the TeraGrid sites to the rest of the produce an integrated and tested set of standard grid NPACI world, the sites are connected to one another by a software. To date, many large projects and testbeds— Director, Center for Grid dedicated 40-gigabit-per-second network, the fastest including TeraGrid—have committed to using NMI Technologies, cross-country academic network. This rich networking software as the basis for grid infrastructure, thus Information Sciences infrastructure makes it possible to use TeraGrid ensuring that TeraGrid will become an integrated part Institute, University of Southern California resources in concert, combin- of the national grid infrastruc- ing them in different ways to ture. NPACI partners at the facilitate entirely new types of University of Southern applications and new types of “When the network is as California’s Information science. Imagine being able to fast as the computer’s Sciences Institute, SDSC, UC perform large-scale data analy- Santa Barbara, and the sis on multi-terabyte data sets internal links, the machine University of Tennessee are by coupled TeraGrid storage participating in NMI. The ini- resources with compute plat- disintegrates across the net tial NMI software release con- forms. By coupling a TeraGrid tains a number of NPACI soft- compute resource in real-time into a set of special purpose ware packages including the to a scientific instrument such Globus Grid Toolkit and the as an electron microscope, we appliances.” Network Weather Service. can dynamically steer an exper- — George Gilder, While NMI provides the iment by performing on-the-fly technology writer and futurist basis of the production data analysis. Very long run- TeraGrid middleware infra- ning jobs can be supported by structure, it is not the end of moving computation from one the NPACI software story. TeraGrid platform to another, or we can perform There are many other software packages developed by extremely large computations by utilizing more than NPACI partners, which are of great value to the scien- one compute platform at a time. These are but a few tific community but not currently part of NMI. To of the types of powerful new applications that the help increase the use of NPACI software, especially in TeraGrid has the potential of enabling. production settings, NPACI is planning to create its However, for us to realize the promise of TeraGrid, own specialized software release. Called the we will need more than just chips, disks, and fiberop- NPACkage, this important infrastructure will comple- tic cable. We need the additional software layers that ment the NMI software releases, and provide one-stop make it possible to securely discover, allocate, manage, shopping for important NPACI software packages. and utilize TeraGrid resources from anyplace. This Together, NPACI contributions to NMI and the software layer, often called middleware, puts the “grid” NPACkage will provide a significant boost not only to in TeraGrid (see story, page 12). TeraGrid, but also to the global grid infrastructure. Grids are often defined as an environment that pro- With a focus on packaging, support, and production vides for resource sharing and coordinated problem infrastructure, this resource is not limited to high-end solving in dynamic, multi-institutional collaborations NPACI resources, but can extend into your university, or so-called virtual organizations. They have been part desktop, laptop, and even wireless handheld device. In of the NPACI agenda from the beginning of the part- doing so, NPACI will play an important part in nership. Building on early work in metacomputing at changing the way we use computing in our research, SDSC and Caltech, NPACI partners have played criti- and beyond. ▼

npaci & sdsc APRIL-JUNE 2002 1 SCIENCE SUCCESSES A Star Is Born: Animating the Solar System’s Origin

EXECUTIVE PRODUCER avid Nadeau says he’s lucky. “I get to turn dreams into reality, or at least the appearance of reality,” said ANTHONY BRAUN the computer visualization expert at the San Diego Supercomputer Center (SDSC). “Scientific visualizations American Museum of D Natural History can be a tremendous asset to researchers in understanding the way the world works, especially things that PARTICIPANTS are too small, too big, or too slow to observe.” Nadeau played a key role in turning raw data of a collapsing cloud DAVID R. NADEAU, of interplanetary dust and gas into a realistic animation of the birth of the solar system—part of a space theater ERIK ENGQUIST, SDSC show dazzling thousands of visitors a day at the American Museum of Natural History in New York. BENJY BERNHARDT, Narrated by Harrison Ford, “The Search for Life: ets from a nebula, a chaotic interstellar cloud of dust ANTHONY BRAUN, CLAY BUDIN, Are We Alone?” is a 23-minute visual extravaganza and gas. About 30 million years of nebula and solar CARTER EMMART, that examines the possibility of life on other worlds. It system evolution (Figures 1–3) were compressed into ARAM FRIEDMAN, premiered March 2 at the Hayden Planetarium, which the animation. MYLES GORDON, is in the museum’s Rose Center for Earth and Space. “We are delighted to continue our collaboration MORDECAI-MARK MAC LOW, GRETCHEN SCHWARZ, Reviewers raved; after all, wrote a New York Times with the Hayden Planetarium on this exciting project,” CHRISTOPHER SCOLLARD, reporter, “What would a planetarium show be without said Fran Berman, director of SDSC and the National RYAN WYATT cosmic splendor?” Partnership for Advanced Computational American Museum of The show’s eight-minute animation of the birth of Infrastructure (NPACI). “Part of the center’s mission Natural History the solar system was created through the collaboration is to assist science educators and to further the public’s DONNA COX, STUART LEVY, of Nadeau, other visualization experts at SDSC, scien- understanding of science and technology. I’m pleased ROBERT PATTERSON tists and artists at the Hayden Planetarium, and com- that SDSC was able to make a unique contribution.” NCSA puter graphics specialists at the National Center for 42,000 HIGH-RESOLUTION IMAGES JOHN HAWLEY Supercomputing Applications (NCSA). Animation University of Virginia segments show the formation of the Sun and its plan- The animation sequence starts outside the Milky Way Galaxy and zooms inward, toward a

AMNH/SDSC/NCSA nebula. Protostars fragment from the cloud and collapse, but not sufficiently for nuclear reactions to begin. Then the largest protostar ignites in hydrogen fusion; its brilliant blue-white light ener- gizes nitrogen and oxygen gas in the neb- ula, producing eerie red and green glows. The view shifts to a smaller newborn star with a thick disk of dust and gas orbiting its equator and jets of material streaming from its poles. The star brightens, the disk flattens, and dust and gas accrete into lumps—infant planets. One of them even- tually becomes Earth. “We took great care to keep the visual- ization both true to the science and enter- taining for the audience,” said Nadeau. “The result is what scientists believe you’d really see if you could somehow spend 30 million years flying around and into these amazing astronomical phenomena.” SDSC’s contribution focused on visu- alizing the evolution of the nebula, star condensation, early protoplanetary disk formation, and the protostar’s polar jets. Other team members created segments that showed detailed views of the disk around the star and the formation of FIGURE 1. IN THE BEGINNING ... planets. The segments blend seamlessly David Nadeau, a computer visualization expert at the San Diego Supercomputer Center, used computer into a continuous visual narrative. simulation data from Mordechai-Mark Mac Low at the American Museum of Natural History to visualize the The main animated segment rendered early stage of the evolution of an interstellar cloud of dust and gas into a star. by SDSC was assembled as 42,000 high-

2 APRIL–JUNE 2002 enVision AMNH/SDSC/NCSA

FIGURE 2. EMISSION NEBULA Dust and gas condense to form a star whose bright light ionizes the nearby gas to create an eerie red and green glow.

resolution video frames, selected from more than processors of NPACI’s Blue Horizon, one of the 150,000 images created during the effort. world’s largest supercomputers, for approximately a The simulation and rendering data and the finished day at near-peak processing efficiency of 1.7 trillion image files were stored and managed at SDSC. With operations per second—the equivalent of more than nearly 7 terabytes—7,000,000 megabytes—of data to three years of calculations on a single processor. share and supervise among the sites, the SDSC “The Hayden animation team used state-of-the-art Storage Resource Broker (SRB), which enables users graphics techniques that can only be employed on to access data resources anywhere on the Net without supercomputers if you want a quick turnaround—and regard to their physical location, proved essential. we were on a very tight schedule,” Nadeau said. George Kremenek of SDSC’s Data and Knowledge The entire solar system formation sequence consists Systems program led data management for the effort. of 70,000 high-resolution frames, including additional “The SRB’s file system also was essential as a col- segments rendered by NCSA and Hayden laboration tool for the distributed team,” Nadeau Planetarium programmers, scien- www.amnh.org/rose explained. “The latest test images were dropped there tists, and artists. The planetarium vis.sdsc.edu/research/hayden2.html for everyone to see, and the latest technical develop- employs seven projectors that ments and bits of source code were saved there. The blend multiple images into a www.npaci.edu/Alpha/visualization.html SRB became our whiteboard for data.” seamless, wrap-around scene on The final rendering of images utilized all 1,152 the overhead dome.

npaci & sdsc APRIL–JUNE 2002 3 out and created them.” To render the luminous fog of the nebula, Nadeau used technology developed for the previous space show at the Hayden Planetarium, a star- ship’s flight through the fiery Orion Nebula. Each volume element, or voxel, of the simu- lated region of space has three independent attributes: emissiv- ity (the amount of light emitted by any matter in the element), color of the emitted light, and opacity (the amount absorbed when a ray of light from the far side of the voxel passes through the volume). Earlier visualiza- tion techniques that don’t han- dle all three effects aren’t able to generate realistic images of dif- fuse, glowing objects. Nadeau turned to “raycast- ing,” a computer graphics tech- nique similar to the more com- mon “raytracing” to render the images (Figure 4). “Raytracing analyzes all of the lines of sight from the observer’s viewpoint, calculates the color and bright- AMNH/SDSC/NCSA FIGURE 3. GAS INTO STARS ness where they intersect the surface of an opaque object, and More stars condense from the nebula, including a star with a disk of dust and gas from which the Earth forms. builds up an image,” Nadeau explained. “Raycasting is similar, but takes the light emission and opacity of the volume elements into “Dave Nadeau used a bunch of new techniques that account and tallies up a running total along the line of made this visualization possible,” said Mike Bailey, an sight—it’s very compute-intensive.” SDSC senior principal scientist. “There’s a reason that Nadeau and SDSC visualization programmer Erik nobody else is doing this sort of work. You need spe- Engquist developed custom software for rendering cialized tools. Some of them didn’t exist, so Dave went 3-D data to meet the planetarium’s unique require- ments. One nonstandard feature of the rendering process was incorporation of multiple data sets in a scene, with voxels of various sizes in the rendering, enabling differ- ent levels of detail to be

GAIL BAMBER combined into a single sim- ulation. Some volumes of space contain more detail than others, and the amount of physical detail near a star—including the disk and jets—is much higher than in the much more homogeneous glow- ing fog of the nebula. Another rendering novelty was FIGURE 4. CONSTRUCTING ONE MOVIE FRAME WITH RAYCASTING representing the star’s disk as wedge-shaped Raycasting builds up an image by concatenating the opacity and luminosity contributions of the voxels based on cylindrical coordinates rather than volume elements along each line of sight from the viewpoint into the simulation volume. There is cubes based on rectangular coordinates. one line of sight for each pixel in each image. The method can accommodate multiple data sets with volume elements of various sizes and shapes.

4 APRIL–JUNE 2002 enVision Newborn stars often eject enormous jets of hot gas The same teams also worked on the museum’s SCIENCE SUCCESSES from their north and south poles; the jets can extend highly acclaimed “Passport to the Universe,” narrated for several light-years and collide with the gas of the by Tom Hanks, which debuted in 2000. It was the surrounding nebula, causing it to glow. The star-birth first presentation shown in the Rose Center for Earth simulation included these dramatic features, but there and Space, which includes the recently rebuilt Hayden was a problem—the simulated jets spurted out in a Planetarium. too-perfectly straight line. The turbulent jets that “ ‘The Search for Life: Are We Alone?’ is a perfect astronomers commonly observe (called Herbig-Haro example of what makes public education at the muse- objects) are lumpy. The team added randomness, but um unique,” said Myles Gordon, vice president for the jets still didn’t look realistic. They ultimately aban- education at the museum. “Through our exhibitions doned the simulation and used the real thing: a and public programs, we are not only able Hubble Space Telescope image of a 12-light-year-long to educate people about breakthrough sci- Herbig-Haro object called HH111 in the constellation entific concepts, but also to do so with

Orion. They used a bit of artistic license to define a 3- the benefit of insights and first-hand AMNH/SDSC/NCSA D model from the image, and created the rendering information provided by the museum’s from that (Figure 5). scientists and researchers.” The tools are incorporated into the NPACI Scalable Visualization Toolkits as separate compo- REMOTE COLLABORATION nents. The volume renderer (based on algorithms by The work of the visualization artists at Jon Genetti, now at the University of Alaska), the vol- NCSA was integrated into the new show ume data manipulation and visualization tools, and through a process of remote virtual col- other modules are being made available on the NPACI laboration, using NCSA’s Virtual Director website as part of the development effort. “The modu- software. To define the flight path for the larity saved us,” Nadeau said. “If we’d been forced to sequence, Patterson, Cox, and Levy ran add the new capabilities to a monolithic package, we’d Virtual Director in a vir- still be trying to make it work.” tual reality display system at NCSA, and Emmart’s TEAM EFFORT team in New York ran Simulating the evolution of the 5-light-year wide Virtual Director on the nebula over 30 million years was the work of Hayden Planetarium’s Mordecai-Mark Mac Low, assistant curator in the Digital Dome system. museum’s Department of Astrophysics, who used an This enabled the two SGI Origin 2000 computer at NCSA. Ryan Wyatt, a groups to interactively science visualizer at the museum, programmed the refine camera viewpoints simulation of the motions of stars within the nebula. and resolve other produc- John Hawley, a University of Virginia astronomer, tion issues. simulated the evolution of the protoplanetary disk 28 “We have been billion miles across over 100,000 years. involved in remote col- Key personnel also included Donna Cox, a profes- laborative efforts before, sor in the School of Art and Design at the University but certainly nothing at this level,” said FIGURE 5. HERBIG-HARO OBJECT of Illinois at Urbana-Champaign who holds a joint Patterson, who spent many late nights Newborn stars can expel jets of hot gas appointment with NCSA, Robert Patterson, an NCSA working with the museum’s production many light-years in length from their poles visualization programmer, and Stuart Levy, a senior team over an audio-visual link via into interstellar space. These jets are called research programmer at NCSA. “The team blended Internet2’s high-speed Abilene network. Herbig-Haro objects after they collide with several separate simulations together to tell one coher- “We never once needed to travel to the interstellar gas, become turbulent, and glow. ent story about the birth of our planet,” Nadeau said. museum.” Nine days were needed to The star-birth simulation features a jet The full show was developed by the American transfer all the data over the Abilene net- based on a Hubble Space Telescope image Museum of Natural History in collaboration with work, but that was still less than half the of HH111 in the constellation Orion, rendered with NPACI Scalable Visualization NASA, and was supported by Swiss Re, an insurance time that would have been required to Toolkits software. company based in Zurich. “The Search for Life” was transfer the data to tapes, fly them to New written by Ann Druyan, Carl Sagan’s widow and long- York, and read them at the destination. “It time literary collaborator, and Steven Soter, an astro- was state-of-the art—we did the entire physicist and writer in the museum’s Division of effort completely without film or tapes,” Nadeau said. Physical Sciences. “Supercomputer data analyses and simulations play The show’s executive producer was Anthony an ever-expanding role in all areas of science,” NPACI Braun, whose group at the museum was headed by and SDSC Director Berman said, “especially in explor- producer Christopher Scollard, creative director ing the fundamental mysteries of life, decoding the Gretchen Schwarz, visualization director Carter human genome, and finding cures for diseases. I’m Emmart, and sound designer Benjy Bernhardt. Aram excited that supercomputer visualizations are helping Friedman is director of Rose Center engineering. convey the thrill of exploring the universe.” —MG ▼

npaci & sdsc APRIL–JUNE 2002 5 SCIENCE SUCCESSES The Evolution of Binary Black Holes

PROJECT LEADER utgers University astronomer David Merritt and his colleagues are using supercomputers of the National DAVID MERRITT Partnership for Advanced Computational Infrastructure (NPACI) to model one of the most dramatic events Rutgers University R in astronomy—the behavior of black holes when their host galaxies collide. From revealing strong and PARTICIPANT unexpected effects on surrounding stars to the potential to produce rogue black holes wandering the universe, MILOS MILOSAVLJEVIC Rutgers University these simulations—among the largest of their kind ever run—are shedding new light on the life cycle of gravitationally bound black holes in the nucleus of a pair of merging galaxies. Observations have yet to definitively detect black hole pairs, but simulation results of Merritt’s team, published in the December 13, 2001, issue of The Astrophysical Journal, are helping astronomers as they work to find new evidence for the elusive objects.

The accepted view among astronomers is that near- gravitational fields (known as N-body simulations). ly all big galaxies harbor supermassive black holes, “What our simulations showed is that, contrary to objects roughly a few kilometers across containing expectation, the two black holes do not immediately mass equivalent to millions or billions of Suns. coalesce into a single black hole,” said Merritt. “In fact, Astronomers also believe that galaxies and their central once they’re near enough—about three light-years— black holes grow by merging. Indeed, our Milky Way they begin to orbit each other as a bound, remarkably Galaxy is currently swallowing a dwarf galaxy. In turn, stable pair.” The simulations predict that such black the larger hole pairs could continue orbiting each other for one

DAVID MERRITT, MILOS MILOSAVLJEVIC Andromeda Galaxy, billion years or more—long enough to collide with a our nearest spiral third galaxy. galaxy, will eventual- The mechanism by which the black hole binary ly merge with the achieves such longevity is itself dramatic. Initially, as Milky Way. the galaxies begin to merge, stars surround each black Previous research hole, with a peak in star density at each black hole that by Merritt and others drops off rapidly in a cusp, or sharply curved profile. has shown that dur- As the approaching black holes become a bound pair, ing the initial stages their separate cusps of surrounding stars merge, form- of the collision ing a single new peak around the black hole pair. between two galax- ies, their black holes LONELY BLACK HOLE BINARIES lose energy through “Then something surprising happens,” said Merritt. interaction with the “The surrounding stars are no longer orbiting a single surrounding stars black hole in an orderly way, but instead now orbit and “fall” relatively two black holes—resulting in far more complex orbits. rapidly toward the If you follow a single star as it orbits the black hole center of the merged pair, it will follow a chaotic path—orbiting tens or galaxy. There, the hundreds of times.” Since black holes are so small, stars black holes orbit each rarely collide with them. Eventually, however, the other in a gravita- chaotic orbit of an individual star will bring it very tionally bound pair. close to one of the black holes. Instead of being swal- EVOLVING BLACK HOLE BINARY DURING GALAXY MERGER Merritt and Rutgers lowed, the star gains so much speed that it is hurled These are star-density contours as two galaxies merge with central graduate student outward in a “gravitational slingshot” effect that gives black holes (filled circles) orbiting clockwise. Milos Milosavljevic it sufficient velocity to escape the black hole pair. Top row: Two black holes spiral closer, dissipating energy. suspected that the NASA used the same slingshot effect to fly its Second row: Star cusps merge, while black holes remain separate. dynamics of this Cassini spacecraft within 750 miles of Earth in a 1999 Third row: The binary transfers energy outward, ejecting stars. black hole pair may maneuver that threw the spacecraft outward toward its play a central role in destination of Saturn. In its close encounter with Bottom row: Stable black holes in scoured-out galactic nucleus. the evolution of the Earth, Cassini accelerated, while the decline in the new galaxy. orbital velocity of the far more massive Earth was Akey question is whether the two black holes imperceptible. Over time, as the black hole binary quickly coalesce into a single, larger black hole, or con- accelerates large numbers of stars—transferring energy tinue to orbit each other in the new galaxy. And what outward—the black holes lose sufficient energy to spi- effect does this dance of giants have on nearby stars? ral closer. “Eventually, they’re only about 0.1 light-year REFERENCE To answer these and other questions, Merritt and apart,” said Merritt. M. Milosavljevic, D. Merritt Milosavljevic used NPACI supercomputers to carry The researchers’ simulations showed for the first (2001): Formation of out precise simulations of the motions of the objects time that the binary black hole efficiently scours away Galactic Nuclei, ApJ 563: in the merging galaxies under the influence of their stars around it, hurling the stars outward and resulting 34–62.

6 APRIL–JUNE 2002 enVision in a new, emptier galaxy core with a uniform star den- said Merritt. The , JPL sity. Once the core is relatively devoid of stars, the simulations NASA black holes lose little additional energy and their orbits involved a major become stable. computational In a significant new achievement—thanks to the effort, requiring large NPACI computational resources—the thousands of hours researchers were able to start the simulations before on an NPACI Cray the galaxy merger, continue past the merged high-den- T3E supercomput- sity cusp of stars, proceed to an thinning galactic core, er at the University and conclude as the orbits of the black holes shrink to of Texas, Austin. those of a stable binary system. “This is more than The tools used four orders of magnitude, something that’s never been in the simulation done before,” said Merritt. “It has allowed us to see all are N-body codes, the stages in the merger evolution down to a separa- which model the tion of only a fraction of a light-year.” The simulation gravitational inter- work has earned Milosavljevic a Sherman Fairchild actions between Fellowship at Caltech, a prestigious national award for the bodies in the postdoctoral research in astronomy. merging galaxies. Merritt notes that the simulation is an idealized Because the scope, case because it ignores the effects of gas and other phe- or dynamic range, nomena that can make the mergers of real galaxies of the merger is so more complex. However, observations with the large, Merritt and DETECTING GRAVITATIONAL WAVES Hubble Space Telescope have shown that the density Milosavljevic had The planned Laser Interferometer Space Antenna (LISA) of stars in galactic nuclei agrees with the simulations. to use two different observatory will use three spacecraft in a triangular configuration to Eventually, the black hole binary will undergo a N-body codes. detect gravitational waves emitted by coalescing black holes in the brief, final coalescence into a single black hole, during Beyond yielding center of a galaxy. which energy radiates in the form of gravitational new insights into waves. Such gravitational waves, if detected, would black holes in www.physics.rutgers.edu/~merritt/ not only provide a “signature” revealing the black merging galaxies, the approach devel- holes, but also yield information about their orbits, oped by Milosavljevic, Merritt, and masses, and spins—and furnish the first-ever test of their colleagues can also be extended to studies of dark Einstein’s Theory of General Relativity under such matter and gravitational waves from coalescing super- extreme conditions. massive black holes. Said Merritt, “What began as a “This is a very fruitful time for interaction among rather specialized computer simulation to investigate theory, observations, and simulations,” said Donald the dynamics of binary black hole evolution during the Backer, a UC Berkeley astronomer whose research merger of two galaxies has now opened lines of involves experiments to detect gravitational waves inquiry that may be able to illuminate a broad range of emitted by coalescing black holes through their effects important unsolved questions from astronomy to fun- on pulsar timing. “We rely on these simulations to tell damental physics.” —PT ▼ us that once galaxies have merged, their binary black holes do indeed coalesce in a ‘timely’ manner.” The simulations are also useful to the designers of gravita- SCIENCE TEAM , ACS SA tional wave detectors, including NA the planned Laser Interferometer Space Antenna (LISA). The simulations suggest an exotic scenario if a third galaxy collides with a galaxy containing a binary black hole. “The com- plex orbital interactions could result in, for example, two of the three black holes forming a tight binary, with the third ejected by the gravitational slingshot— which could result in rogue black holes flying around the universe,” said Merritt. MERGING GALAXIES “Having access to large com- A good candidate for containing binary black holes, the colliding galaxies in NGC 4676 are known as “The Mice” puting resources has been vital to because of their long tails. Located 300 million light-years away in the constellation Coma Berenices, they will eventually obtaining these new insights,” merge into a single giant galaxy. npaci & sdsc APRIL–JUNE 2002 7 SCIENCE SUCCESSES Projecting How Climate Change May Rearrange Mexican Ecosystems

PROJECT LEADER set of large-scale ecological models predicts that over the next 50 years the changing climate in Mexico— A. TOWNSEND PETERSON which is expected to become warmer and frequently drier—is likely to bring about great instability for the University of Kansas A Natural History Museum nation’s animal species. The models predict that the changing climate will shuffle Mexican ecosystems— PARTICIPANTS throwing new combinations of predators and prey together, introducing invasive diseases and parasites, and MIGUEL ORTEGA-HUERTA shrinking the geographical ranges of a majority of species. The researchers were led by A. Townsend Peterson of University of Kansas the University of Kansas Natural History Museum, a National Partnership for Advanced Computational Infrastructure Natural History Museum JEREMY BARTLEY (NPACI) partner, and included David Stockwell, an environmental informatics researcher at the San Diego University of Kansas Supercomputer Center (SDSC). Their report in the April 11 issue of Nature describes the largest ever analysis of the VICTOR SÁNCHEZ-CORDERO, potential impacts of climate change on individual species in natural ecosystems, including 1,870 species in Mexico. JORGE SOBERÓN, ADOLFO NAVARRO SIGÜENZA Universidad Nacional The study, carried out in collaboration with data, and climate model predictions in a powerful soft- Autónoma de México Mexican researchers Victor Sánchez-Cordero, Jorge ware program, the Genetic Algorithm for Rule-Set Soberón, and Adolfo Navarro Sigüenza, of the Prediction (GARP). The software was developed by ROBERT BUDDEMEIER Kansas Geological Survey Universidad Nacional Autónoma de México Stockwell and was run on an NPACI Sun (UNAM), included all endemic species of mammals Microsystems supercomputer at SDSC. DAVID STOCKWELL and birds in Mexico as well as many species of butter- SDSC flies. To project the EFFECTS OF CHANGING CLIMATE

UKNHM, CORNELL UNIVERSITY LABORATORY OF ORNITHOLOGY LABORATORY UKNHM, CORNELL UNIVERSITY composition of “In some local communities, more than 40 percent future ecosystems, of species is expected to turn over, which would lead the researchers to a cascade of further effects,” said Peterson. “If you combined species remove enough species from an ecosystem, it’s like the location data, old child’s game of Pick Up Sticks—there are only so present envi- many changes you can make before the ecosystem just ronmental rearranges, and maybe strikingly.” While other factors besides climate—including human changes in land use such as building roads and cities that block animals’ escape routes—are also expected to impact species’ ranges, this innovative modeling research is providing new scientific insight into climate impacts on species’ ranges and biodiversi- Predicted habitat reduction ty. The results can eventually provide crucial scientific Predicted habitat gain guidance to policy makers in establishing effective Birds observed species conservation and management plans. Predicted suitable habitat before and after climate change “What’s unique about what we’ve done is that for the first time we were able to look at a whole commu- nity in detail,” said Peterson.

©LYNX EDICIONS/VIREO [email protected] Previous research has looked in a broad-brush way at how cli- mate change would affect an ecosystem. For example, if a warmer climate will cause a given habitat to move north- ward or to higher elevations, researchers have made the assumption that all of its ani- mal species would simply move as a whole. Other detailed studies have been limited to SHRINKING HABITAT FOR CHACHALACA looking at only a few species. Researchers project that the habitat for the West In contrast, Peterson, Mexican Chachalaca (Ortalis poliocephala) will expand in Stockwell, and their collabora- some areas and shrink in others, for a net loss of about 30 percent by 2055, under tors not only looked at many weaker (top) and stronger (bottom) climate-change scenarios. more species across a whole WEST MEXICAN CHACHALACA

8 APRIL–JUNE 2002 enVision community, but also examined them one-by-one, mate, including higher average temperatures and usu- SCIENCE SUCCESSES using realistic assumptions about their ability to move ally lower rainfall. in response to climate change. “That species-by-species The researchers analyzed how the species-location look has allowed us to appreciate just how big the dif- data is related to changing climate using the GARP ferences are in the way each species can respond to cli- software at SDSC, which is accessed through the Bio- mate change—it’s quite individual,” said Peterson. diversity Species Workshop Web interface. The soft- The researchers also found that the impacts aren’t ware allows the integration of many variables—from uniform. The strongest effects turned out to be in flat- vegetation and temperature to rainfall and topogra- lands such as the central Chihuahuan Desert, a high phy—into a robust statistical model of the response of plateau region south of New Mexico and western a species to its environment. Using several dozen loca- Texas. While the researchers project that few species in tions where a species has been observed, the GARP that zone will actually go extinct, a large percentage of software predicts the areas where a similar climate them is expected to experience adjustments—expan- might be found in 2055, in which the species could live. sions and retractions in ranges that will reshuffle ecosystems. More species may be affected in such flat areas, scientists believe, because they must move long distances to reach a preferred climate. In contrast, animals living in mountain- ous areas could simply move a short distance uphill to reach a cooler climate—provided the mountain is tall enough. INTERDISCIPLINARY COLLABORATION The international, interdisciplinary research ARL S. LIEB, CENTENNIAL MUSEUM AND INDIO MOUNTAINS RESEARCH STATION RESEARCH ARL S. LIEB, CENTENNIAL MUSEUM AND INDIO MOUNTAINS was supported by the National Science C Foundation and the government of Mexico, and involved both U.S. researchers and Mexican sci- entists. “There are really two stories here—the results of the research itself, and the interdisci- plinary approach we used,” said Sánchez- Cordero. Biologists, collections specialists, com- puter scientists, and other researchers created a CHIHUAHUAN DESERT powerful tool that can be extended to explore the combined effects of climate change and The strongest predicted effects on Mexican ecosystems of a warmer and frequently drier climate over the next 50 years will be felt in flatlands such as the central Chihuahuan Desert, a high plateau region south other scenarios. The approach can be extended of New Mexico and western Texas. to other fields to produce, for example, risk- assessment maps for emerging diseases, agricul- tural pests, and invasive species. Fundamental to this research are the extensive col- Large-scale computations contributed to these lections of observational records of species in Mexico results in several ways. “The GARP software uses com- assembled by Mexico’s biodiversity commission, putationally intensive machine-learning methods from CONABIO. “The collections are truly comprehensive, artificial intelligence research, called genetic algo- with more than 110,000 records of observed species rithms, to iteratively generate, test, and adopt or dis- locations,” said Peterson. The collections were card ecological niche descriptions, thereby evolving painstakingly brought together in database software robust sets of models,” said Stockwell. “We that unifies information on Mexican species held in then analyzed the effects of climate change nhm.ku.edu more than 40 natural history museums around the on each species, resulting in thousands of world. The researchers ran this large-scale model for separate models and predictions, which were biodi.sdsc.edu/bsw_home.html Mexico because of the completeness of the collections. used to project the changes in community www.conabio.gob.mx Nothing comparable yet exists for collections in the composition.” The researchers also made the United States. spatial scale as fine as the computational The researchers also needed climate predictions for resources permitted to achieve accuracy and definition Mexico. For this they used general circulation climate for the species dispersal modeling. The large analysis model predictions made available by the Intergovern- required about 30 days of computing on an NPACI REFERENCE mental Panel on Climate Change, an international Sun multiprocessor parallel machine at SDSC. A.T. Peterson, M.A. Ortega- organization that uses published, peer-reviewed scien- In the future, the researchers plan to make field Huerta, J. Bartley, V. tific literature and data to assess scientific, technical, measurements to test the model’s predictions. They Sánchez-Cordero, J. Soberón, R.H. Buddemeier, D.R.B. and socioeconomic aspects of human-induced climate also plan to extend this work to produce synthetic Stockwell (2002): Future change. The researchers considered two scenarios of models that explore the combined effects of climate projections for Mexican increasing greenhouse gases over the next 50 years, change and other scenarios such as the presence of faunas under global climate both of which predict major changes in Mexico’s cli- invasive species. —PT ▼ change scenarios, Nature 416: 626–629. npaci & sdsc APRIL–JUNE 2002 9 SCIENCE SUCCESSES Lithium Batteries Getting a Power Boost from Supercomputer Simulations

PROJECT LEADER aptops, cell phones, and personal digital assistants allow us to work or play while digging our toes into GERBRAND CEDER sandy beaches, but the lithium batteries that power these devices will often run down long before the sun Massachusetts Institute of L Technology does. More energy-consuming features are draining lithium batteries faster. “The wireless world has made PARTICIPANT the electronics industry’s battery problem painfully obvious,” said Gerbrand Ceder, a materials scientist at MIT who ANTON VAN DER VEN is using supercomputers provided by the National Partnership for Advanced Computational Infrastructure (NPACI) to Massachusetts Institute of design more powerful batteries. “Only a few years ago, most people kept their laptops continually connected in Technology order to access the Internet. Now, many of them have wireless Ethernet connections, but they can’t stray too far from a charger. This holds true for lots of electronic devices.”

The appetite for rechargeable lithium batteries is AN EVOLVING TECHNOLOGY soaring, with U.S. sales reaching an estimated $2 bil- Alessandro Volta (for whom the volt was named) lion last year. (Worldwide sales of cell phones, made the first batteries 200 years ago, and recharge- which use lithium batteries, totaled 500 mil- able lead-acid batteries have been manufactured for a lion units in 2001.) Moore’s Law, which holds century. Nickel-cadmium batteries can also be that chip computing power increases about 60 recharged, but exhibit a “memory effect” in which a percent annually, doesn’t apply to lithium bat- battery recharged at the same level several times con- teries, whose performance has improved only secutively acts as if it needs a charge whenever it reach- 30 percent during the past decade. Still, the es this point. Nickel-metal hydride batteries exhibit a importance of compact, lightweight, and envi- smaller memory effect. The 4-volt lithium battery, ronmentally safe batteries is growing, which has up to 33 percent higher energy densi- not just as a power source for elec- ty and 60 less weight than a nickel-metal tronics tools and toys, but also hydride battery of the same size, has made pos- as an energy-storage device for sible the miniaturization of the current genera- INCREDIBLE SHRINKING ELECTRONICS vehicles of the future. Industry-stan- tion of electronic devices. Lithium batteries permit the dard lithium batteries are expensive But handheld tools and toys tax even lithi- production of lighter, smaller because they contain cobalt, making them um batteries. Cell phones, for example, electronic devices. practical only for small, high-value appli- require bursts of power during voice trans- ances. A lithium battery big enough to missions, and consume power on standby in power an electric car would cost about order to remain in contact with transmis- $10,000, and a car with one would still have sion facilities. “The farther away from a a range of less than 150 miles. receiving station, the harder a cell phone works,” PERCENT SATURATION OF CATHODE PARTICLE Ceder said that merely doubling the said Ceder. “The spotty cell phone coverage in the WITH LITHIUM IONS power output of a lithium battery is United States means that a cell phone battery lasts 100% insufficient. For example, only a few days, even on standby.” increasing the time between 95% laptop recharges from two SIMULATING BATTERY MATERIALS 90% to four hours wouldn’t Rechargeable lithium batteries have no moving noticeably alter the way 85% parts: each battery has a negative terminal, or anode, a people use them. “But positive terminal, or cathode, and an electrolyte that 80% when you go to eight or separates the two. Electrochemical reactions lead to the 75% 10 hours of laptop oper- movement of charged lithium atoms, or ions, from the ation without a recharge, 70% anode to the cathode, producing an electric current. people’s behavior will Recharging reverses the process. Repeated discharging fundamentally change and recharging forms the lifeblood of a rechargeable because that’s a workday,” lithium battery, but until recently chemists had neither he said. “That’s where we the computational resources nor the mathematical need to go, and I think we can tools to investigate the fundamental properties of bat- get there.” He estimates that tery materials. Walter Kohn, a chemist at UC Santa SIMULATING CATHODE STRESSES supercomputer simulations combined Barbara, and John Pople, a mathematician at North- During charging, steep gradients of lithium ion with improved mathematical tech- western University simplified the mathematics of a concentration develop near the surface of niques will lead to a battery-material branch of physics called quantum mechanics, which cathode particles. The gradients lead to uneven breakthrough by 2004, and a proto- describes the bonding of atoms. This theoretical break- shrinkage of the cathode material and cracking. type ready for testing a few years through, which earned Kohn and Pople the 1998 after that. Nobel Prize in chemistry, provided a computationally

10 APRIL–JUNE 2002 enVision simpler approach to describe large molecules, such as cathode surface. “When you get such large mechanical SCIENCE SUCCESSES the lithium cobalt oxide lattice in batteries. stresses, as we predict there will be, the cathode mate- Since battery performance is limited primarily by rial will definitely start cracking,” said Ceder. “The the speed of lithium diffusion, Ceder and his collabo- cathode material would cleave, become more and rators have focused on lithium ion movement through more exposed to the electrolyte, which in turn the layers of lithium cobalt oxide in the cathode. They would lead to more and more degradation.” The are trying to describe this diffusion process in as much degradation partly explains why new laptops may mathematical detail as possible as an integral part of provide, in some models, up to eight hours of bat- their efforts to design improved cathode materials. tery-powered operation, but that time span falls to “Battery materials have quite complicated electronic two hours or less after many cycles of discharging structures and studying diffusion of lithium atoms in and charging. them can only be done on parallel computers,” said Ceder. “With these methods, we study the lat- THE ONE-ELECTRON BARRIER tice at various positions along the diffusion There are 30 transition metal elements, path at the same time, like the bright trail left including cobalt, and Ceder is searching for by a meteor streaking across the sky. There those that would allow the cathode to retain can be 16 calculations and they are done lit- its shape regardless of the degree to which erally on 16 nodes of a supercomputer at lithium ion concentration changes. Cobalt has the same time while communicating with shortcomings that a different transition metal each other.” Individual “cells” of a cathode could rectify: not only is cobalt expensive and simulation include 64 atoms, a huge relatively heavy, but each cobalt ion is capable of computational task to simulate. contributing, at most, one electron per discharge Ceder’s group has performed cycle. “We would like to use cheaper, lighter-weight its simulations on NPACI’s Cray transition metals from which we could extract multiple C90, T90, and T3E supercom- electrons per atom,” said Ceder. “We call this ‘breaking puters at the San Diego the one-electron barrier.’ ” Supercomputer Center (SDSC) Ceder’s group also is looking for combinations of and the University of Texas, and metal atoms that could provide the needed electro- on the IBM Blue Horizon super- chemical punch while also keeping the cathode struc- computer at SDSC, which is capa- turally stable over wide ranges of lithium ion concen- ble of 1.7 trillion calculations per tration. “If we can introduce transition metals that second. provide multiple electrons,” said Ceder, “we would then have the flexibility to add other ingredients to the CHEMICAL CONGA LINE cathode that provide structural integrity without loos- In a paper published last year in ing much of its electrochemical capacity.” the Journal of Power Sources, Anton Finding the right ingredients won’t be simple. Van der Ven, a postdoctoral researcher Some transition metals generate one electron at, say, at MIT, and Ceder reported the discov- 1 volt (a useless voltage) and a second electron at ery of a surprising mechanism of lithium 5 volts. “You need the electrons to come off at the REFERENCE ion diffusion in cathodes made of lithium same voltage,” Ceder said. “And there is no rule that A. Van der Ven, G. Ceder (2001): Lithium diffusion cobalt oxide—ions easily “hop” along a says a cathode should have only one metal. When you mechanisms in layered diffusion path when they have two degrees of mix transition metals in a cathode, there are limitless intercalation compounds, J. freedom for their next hop, but the energetic barrier to numbers of combinations. This is why supercomputer Power Sources 97-98: 529- such hops is three times greater when the ions have modeling is so useful.” —RG ▼ 531. only one adjacent position to hop to. The difference in energy barriers helps to par- POLYMER BINDER tially explain why lithium ion diffusion, OXYGEN INTERCALATION OXIDE while fast in a partially discharged battery, becomes balky when a battery is fully GERBRAND CEDER charged or discharged. “During charging, GE you sort of pull lithium ions out of the surface of the cathode particle, but not from the interior,” said Ceder. “It’s like a DISCH conga line. You want the people to move fast, but if you pull some people off, you ANODE CATHODE break the conga line and the rest never ELECTROLYTE COBALT come.” LITHIUM ION MOVEMENT LITHIUM CARBON BLACK In addition, because lithium ions dif- RECHARGEABLE LITHIUM BATTERY DESIGN fuse from the surface of cathodes faster than from the interior during charging, Electrochemical reactions lead to the movement of lithium ions from the anode to the cathode, producing an electric current. A theoretical breakthrough that simplified the mathematics of molecular bonding has enabled concentration gradients develop. These chemists to use a computationally less complex approach to describe large molecules, such as the lithium gradients lead to uneven shrinkage of the cobalt oxide lattice in lithium batteries. npaci & sdsc APRIL–JUNE 2002 11 TERAGRID Retooling Middleware for Grid Computing

PARTICIPANTS ust connect to the Internet, and your browser immediately demonstrates the importance of using networked FRAN BERMAN resources as a tool to access new information. The current focus on the computational “grid” aims to use NPACI and SDSC J these networked resources even more comprehensively—as a worldwide virtual computer. Development of HENRI CASANOVA UC San Diego and SDSC the software infrastructure required to make this vision a reality is being addressed in a variety of ways by the JACK DONGARRA National Science Foundation (NSF), including a new program that—with the help of the National Partnership for University of Tennessee Advanced Computational Infrastructure (NPACI) (see Guest Editorial, page 1) and other organizations—provides IAN FOSTER Argonne National downloadable middleware so that scientists and engineers can use networked resources together as a large-scale Laboratory computational and data management platform. CARL KESSELMAN University of Southern In May, the NSF Middleware Initiative (NMI) “Software is required to support computation, data California released version 1.0 of its platform containing Globus, management, access, and use at each of the sites, as Network Weather Service, and CondorG (along with well as to support coordinated computation and data JOEL SALTZ Ohio State University security and best-practices tools). “The Network activities at sites linked through the TeraGrid network. Weather Service and Globus have been key projects in Developing a robust, production-ready grid software RICH WOLSKI UC Santa Barbara NPACI and we are delighted that they are part of the infrastructure is a challenge but the TeraGrid team is first NMI release,” said Fran Berman, director of arguably the most capable group anywhere to take on NPACI and the San Diego Supercomputer Center and succeed with this challenge.” (SDSC) at UC San Diego. “NPACI plans to harden and deploy additional software for later NMI releases BUILDING ON GLOBUS as part of our commitment to building the national In addition to NMI, more than a dozen compa- information cyberinfrastructure nies—including HP, IBM, Microsoft, and Sun required to address the compu- Microsystems—are adopting Globus, which contains tational and data-management components that can be used independently or togeth- “Developing a robust, challenges of the next decade.” er to develop useful grid applications and program- The NMI program, a $12 ming tools. Development of the Globus middleware is production-ready grid software million, three-year effort, is one led by Carl Kesselman and Ian Foster. Kesselman is infrastructure is a challenge but of many signs of the rising NPACI’s chief software architect and director of the importance of grid computing, Center for Grid Technologies at the University of the TeraGrid team is arguably an infrastructure of tightly inte- Southern California’s Information Sciences Institute, grated high-end computers, and Foster is associate director of the Mathematics and the most capable networks, databases, and scien- Computer Science Division at Argonne and professor tific instruments managed by of computer science at the University of Chicago. group anywhere to multiple groups. NMI software Condor is a specialized workload management sys- will be used in the NSF’s $53 tem for computationally intensive jobs. The software take on and succeed million TeraGrid project, to be was developed by a team at the University of deployed in the next two years Wisconsin and Argonne that was led by Wisconsin’s with this challenge.” as the world’s largest, fastest, Miron Livny. CondorG leverages software from —Fran Berman most comprehensive distrib- Condor and Globus to allow users to harness multi- uted infrastructure for open sci- domain resources as if they all belong to a single per- entific research. TeraGrid sonal domain. includes four partners: SDSC Development of the Network Weather Service is at UC San Diego; the National Center for led by Rich Wolski, leader of NPACI’s Grid Supercomputing Applications (NCSA) at the Computing thrust and a professor of computer science University of Illinois, Urbana-Champaign; the Center at UC Santa Barbara. The middleware monitors net- for Advanced Computing Research at Caltech in works, CPUs, and performance sensors and uses that Pasadena; and Argonne National Laboratory in information to dynamically forecast—analogous to Argonne, IL. When the TeraGrid is completed, it will weather forecasting—the availability and performance include more than 13 teraflops of computing power of various network and computational resources that and online disk facilities capable of storing and manag- can be delivered over a given time interval. ing more than 450 terabytes of data distributed over The second release of NMI is expected to include the four sites. two additional middleware packages developed with To achieve the TeraGrid’s goals, powerful clusters, partial NPACI support: NetSolve and APST, an data-storage hardware, and high-speed networks must acronym for AppLeS (application-level scheduling) be integrated with compatible software to make the parameter sweep template. NetSolve enables users to system work. “Software is critical to achieving the solve complex scientific problems remotely by provid- potential of TeraGrid resources,” said Berman. ing access to hardware and software distributed across

12 APRIL–JUNE 2002 enVision a network. It was developed by a team at the library of linear algebra algorithms for computation TERAGRID University of Tennessee and Oak Ridge National with mathematical matrices. NPACI has supported the Laboratory led by Jack Dongarra, a professor at the hardening of LAPACK and its parallel version, University of Tennessee and director of its Center for ScaLAPACK. Many problems in mathematics and sci- Information Technology Research. ence take advantage of matrices, rectangular arrays of APST deploys and schedules parameter sweep elements. “All the old linear algebra software ran very applications, which are used to investigate many scien- slowly, and everybody realized it was time to redesign tific and engineering problems by running a program it all to be compatible with modern computers and many times while varying its parameters. APST was their memory hierarchies, memory caches, and other developed by a team led by Henri Casanova, director improvements,” said James Demmel, a professor of of SDSC’s Grid Research and Innovation Lab and co- mathematics and computer science at UC Berkeley leader of the NPACI alpha project Monte Carlo who led the LAPACK development effort with Cellular Microphysiology on the Grid. University of Tennessee’s Dongarra and others. “LAPACK may not be a computer game, but it has THE NPACKAGE enormously wide usage,” said Demmel. Other popular NPACI-supported data-manage- Indeed, LAPACK is used in several commercial ment middleware packages—including Advanced Data software toolkits such as MATLAB, which integrates Repository, DataCutter, and the Storage Resource mathematical computing, visualization, and a flexible Broker—also are being readied for NMI compliance. environment for technical computing. Roughly “NMI provides a common infrastructure that can be 500,000 engineers, scientists, mathematicians, and deployed at grid nodes all around the world,” said educators in 100 countries and at more than 2,000 Berman. “We plan to deploy NMI and an ‘NPACkage’ universities use MATLAB. “LAPACK is at the core of of our ready-for-prime-time software at all NPACI it,” said Cleve Moler, chairman and chief scientist at resource sites. The experience of deploying and using Mathworks Inc., owner and distributor of MATLAB. this software on the NPACI grid will provide a usabili- In addition, the top-500 supercomputer rankings are ty path for software not in the first NMI release.” based on the measured speed with which the machines Kesselman, Wolski, Berman, and others envision an run ScaLAPACK. expanding collection of integrated software toolkits NPACI software developed by NPACI that would be included in the will form a basis for “NPACkage” sitting atop a base layer of NMI-compli- NSF’s evolving cyber- “We would also like to provide this ant middleware. The NPACkage will contain NPACI infrastructure. same environment and portals to users value-added technology and incorporate evolving NPACI also is NPACI-enabled technologies as they mature. “Since emphasizing outreach of high-end resources like Blue NPACI and NCSA will be NMI-based, anything that to wider communities can interact with NMI will be able to interact with our of users of its infra- Horizon. Grid software will software,” said Wolski. “The idea is to follow the NMI structure in order to lead to deliver value-added components comprising bring cyberinfrastruc- unify access for numerically the NPACkage to the partnership and to other sites.” ture to a wider group Much of NPACI’s infrastructure has been devel- of scientists, engi- intensive, data intensive, and oped to support cutting-edge integrative technology neers, students, and coordinated applications.” and computational science projects—called alpha proj- educators. Over the ects—at partner sites. NPACI Chief Software Architect next decade, more —Rich Wolski Kesselman said that the partnership is placing great scientists and engi- emphasis on usability, deployment, and usefulness of neers will be develop- the software, as well as coordinating NPACI software ing real-time and on-demand applications. The appli- with software being developed for the TeraGrid. cations’ requirements will drive development and inte- “Our goal is to assemble a comprehensive, NMI- gration activities at NPACI and add to the productivi- compliant methodology to manage computation and ty of scientific and technological communities. data in high-performance grid environments like the In this scenario, software developed independently TeraGrid,” said Wolski. “In addition, we would also for distinct projects would interact. “To fulfill the like to provide this same environment and portals to potential of grid computing to aggregate the immense users of high-end resources like Blue Horizon (the 1.7 capacity and capability of the underlying resources, teraflops IBM supercomputer at SDSC). Grid software there must be a common software infrastructure which will unify access for numerically intensive, data inten- allows users and application developers to use distinct sive, and coordinated applications.” resources as an ensemble for their applications.” said NPACI does not conduct software research, per se, Berman. “NMI, NPACkage, TeraGrid software, but develops and “hardens” it—makes it user friendly Alliance software, OGSA (Open Grid Services and compatible with other software and middleware. Architecture)-based software, and other efforts can However, research is still the first step for much of the provide this common infrastructure and make it possi- software used in production today. For example, NSF ble to build out our national and global cyberinfra- and the Department of Energy provided the initial structure to address the critical computational and data research funding for the development of LAPACK, a management needs of the future.” —RG ▼ npaci & sdsc APRIL–JUNE 2002 13 EDUCATION 3-D Views of Science for the Public

PROJECT LEADER he newest exhibit at the Reuben H. Fleet Science Center in San Diego is a display of colorful images created MICHAEL J. BAILEY by Michael J. Bailey, a senior principal scientist at the San Diego Supercomputer Center (SDSC). The 16 SDSC T visualizations hanging in the Fleet’s rotunda gallery range from the geology of the Earth and other planets PARTICIPANTS to molecular science and mechanical design. “These aren’t just ordinary images,” Bailey said. “Sure, you can glance JEFF KIRSCH, PAUL M. SIBOROSKI Reuben H. Fleet Science at them and see lots of interesting things. But the exhibit also provides special glasses, and when you look through Center them—wham!—the images leap out at you in 3-D.”

A terrain map of Mars shows the planet’s rugged “Rosie the Riveter” during World War II. canyons and deep craters, and a map of Earth shows Bailey, recipient this year of UC San Diego’s continental highlands, ocean basins, and mid-ocean Barbara J. and Paul D. Saltman Distinguished ridges in startling relief. The Tibetan Plateau seems to Teaching Award for Non-Senate Members, has been jut out of the frame, and images of molecules reveal experimenting for two decades with presenting scien- their 3-D structures in a way that ordinary graphics on tific information in 3-D. He is pleased with the a computer monitor or a printed page won’t permit. ChromaDepth process used in the exhibit. “For the first time, we have a visualization technique that allows both printed and on-screen images to appear in 3-D,” said Bailey. “It also allows the same images to be viewed clearly in 2-D without the glasses.” In the ChromaDepth process, invented by Chromatek, Inc., depth is encoded into an image with color cues, then simple, inexpensive optics decode the color to produce a true, MICHAEL J. BAILEY stereoscopic, 3-D image. The color-cod- ing is the same as the one used in topo- graphic maps, with red representing the highest elevations, green the lowlands, and deep blue the ocean depths. Even regular topographical maps printed with this same color scheme look three-dimensional with ChromaDepth glasses. Working in the OpenGL programming toolset, an environment for developing portable, interactive 2-D and 3-D graphics applications, Bailey has been creating 3-D images that illustrate the concept and is teaching the process to students in computer graphics classes at UC San Diego. A paper by Bailey (dvl.sdsc.edu/chromadepth/chromapaper.pdf) provides the MOLECULE STRUCTURE details of the process. The paper contains OpenGL The 3-D structure of this protein kinase molecule was first solved at SDSC. source code, which enables any programmer to write computer graphics programs that produce the 3-D effect—even in real-time animations—on Unix work- stations, Macs, or PCs. “This is a great addition to our exhibit space,” said In a recent related activity at the Fleet Science Paul M. Siboroski, the science center’s exhibits direc- Center, Eric Frost, an associate professor of geology at tor. “The kids love it when these apparently normal San Diego State University, gave a presentation titled pictures suddenly spring out in three dimensions. “A 3-D ChromaDepth Tour of the Earth.” He distrib- What we like is that while the kids—and their par- uted 3-D glasses and used images developed by Bailey ents—are being entertained, they’re also learning to explain Earth’s geology and geophysical processes. about the universe around them.” “Eric gave a similar presentation at the IEEE The Fleet Science Center in San Diego’s Balboa Visualization 2001 conference,” Bailey said. “He can Park is known for science education and entertain- reach a wide range of audiences with his explanations, www.rhfleet.org ment. About 510,000 visitors a year are drawn to from computer science professionals to kids. It’s not the center’s planetarium and IMAX theater, lec- just the ChromaDepth—Eric uses paper cups, rubber dvl.sdsc.edu/chromadepth ture halls, science exhibits, and children’s activity balls, balloons, marshmallows, tortillas, and other and learning centers. The science center’s name- household items to explain why the continents look sake is a noted American aviator and aircraft and behave the way they do.” builder whose San Diego operations coined the term Siboroski pointed out that ChromaDepth graphics

14 APRIL–JUNE 2002 enVision MOSTLY SUBMERGED MOUNTAINS The volcanic mountains that form the Hawaiian Islands are far higher than they seem at sea level. From base to summit, the Big Island of Hawaii is the largest mountain on Earth. ChromaDepth colors reveal high elevations (red), middle ranges (yellow and green), and ocean depths (aqua and blue). MICHAEL J. BAILEY

also could be used in school textbooks to illustrate public. Our educators and theirs are concepts in astronomy, geology, and other scientific very enthusiastic about the prospects disciplines. of working together in the future.” “The Fleet Science Center does a great job of dis- “We hope that this is just the first seminating knowledge of science and technology to of many collaborations with SDSC,” the public, but they don’t have SDSC’s depth of said Jeff Kirsch, executive director of resources to draw on to produce content,” said Bailey. the center. “Our strengths are comple- “On the other hand, SDSC has supercomputers and mentary, and our cooperative efforts software experts to create technical content, but it will extend the public’s understanding doesn’t really have the venue for showing it to the of science and technology.” —MG ▼ MIKE GANNIS GAIL BAMBER ILLUSION OF DEPTH In the ChromaDepth process, depth is encoded into an image with color cues. Inexpensive lenses (www.3dglassesonline.com) with A GALLERY’S New DIMENSION diffraction gratings act like prisms, bending Visitors at the Fleet Science Center get a 3-D view of Hawaii with the museum’s ChromaDepth glasses. light of different colors by different angles.

npaci & sdsc APRIL–JUNE 2002 15 NEWS

Online NEWS ABOUT NPACI AND SDSC www.npaci.edu/Online

New Computational Method Could acquisition to management, analysis, and eXtensible Markup Language (XML), help finally publication in a digital library. This users rapidly find, access, and assemble all Shorten Drug-Development Time effort is part of a larger NSF-sponsored this information into a seamless, informative computational technique developed by National Science Digital Library (NSDL) answer to their question.” A researchers at UC San Diego, and project that is developing modern methods An important feature of the approach is reported in the May 22 Journal of the for the publication of arbitrary digital that the powerful capabilities will be provid- American Chemical Society provides a more objects, comprised of scientific data, as ed as easily available Web services. “As com- realistic picture of how potential drugs and library-grade objects. puters are linked together into grids, Web their molecular targets behave under normal The other expedition participants includ- services and their emerging standardization conditions. The method models drug targets ed the Geological Data Center at the Scripps into an Open Grid Services Architecture as flexible structures that stretch and bend at Institution of Oceanography (SIO), (OGSA) play an increasingly important role room temperatures rather than as rigid University Libraries, and the SIO Library. in enabling the components to work togeth- objects. The new approach is also much Sonar on the Melville mapped a swath of the er,” said Fran Berman, director of SDSC and faster than laboratory methods and should seafloor approximately 20 kilometer wide in NPACI. (v6.7) complement them in shortening the time it deep water, creating real-time contour maps takes to develop potent new drugs. of bathymetry as well as acoustic backscatter. Bart McDermott Joins SDSC as Director “The race is on between sophisticated lab The result of the cruise was approximately 5 for Development and Communications GB of new digital data in persistent archive procedures and their computational analogs,” art McDermott, an experienced business structures, ready for search along with the said team leader J. Andrew McCammon, a development expert, has joined SDSC evolving SIOExplorer collection of data, B Howard Hughes Medical Institute investiga- as program director for Development and documentation, reports, images, sampling tor and UCSD professor of Pharmacology Communications. SDSC has active collabora- information, and publications from 795 SIO who holds the Joseph Mayer Chair of tions with more than 50 private sector part- cruises since 1950. Much of this material Theoretical Chemistry. “Our method both ners in the San Diego area and across the will be available online as a part of the verifies and competes with costly lab meth- country, and Digital Library. (v6.9) ods that rely on thousands of trials of nearly McDermott is work- similar molecules. And it does not require SDSC Leads Collaboration to Integrate ing with those and the synthesis and purification of proteins, prospective partners nor is it limited by the sizes of the mole- Access to Government Information to leverage SDSC’s cules.” For their calculations, McCammon’s esearchers at SDSC and industry part- expertise in bio- group used computers at the San Diego ners Enosys Markets and Polexis sciences, data and Supercomputer Center(SDSC) and an R demonstrated an advanced Web services knowledge systems, SDSC “satellite site” at UCSD. approach to providing seamless access to and high-end com- Existing methods treat the receptor pro- multi-agency government information. The puting and commu- tein as a rigid object, when the structure is March 21 event in Washington, D.C., was nications. actually only a frozen snapshot of a molecule attended by 40 government agencies that “Bart brings valu- that is always in motion at room tempera- want to provide access to their statistics. able experience in partnership development ture.” The new method samples the motions The demonstration was part of the at a major government laboratory and at of the larger receptor, then sees how the lig- National Science Foundation-funded Digital firms in Silicon Valley,” said Fran Berman, and binds to an “ensemble” of receptor con- Government Information Integration SDSC director. “SDSC research, technolo- formations. (v6.11) Testbed (I2T) which has the goal of integrat- gies, and expertise are already benefiting Filling a Digital Library ing information from separate government dozens of firms, and we are looking forward to Bart expanding the center’s impact on the with Sea Bottom Data sources. The project is initially focusing on linking or “mediating” information of inter- local, state, and national economy.” he first Floating Digital Library Work- est in government applications, in particular McDermott comes to SDSC with experi- Tshop began March 5 in a New Zealand geographical and demographic-survey data. ence in partnership development and tech- harbor as the research vessel Melville set off Interest is building for rapid, universal nology licensing at Lawrence Livermore on a 14-day expedition to Samoa. John access to huge amounts of government National Laboratory (LLNL). McDermott Helly, an environmental scientist at SDSC, information. “For all but the simplest managed technology transfer activities at sailed on leg 20 of the cruise—called the queries, however, there are challenges in LLNL for the Biology and Biotechnology, Cook Expedition for Captain James Cook finding and assembling the right data, Energy, Engineering, and Environmental who sailed and mapped the same waters whether for research or decision-making,” Directorates. More recently, he worked with more than two centuries ago. said Chaitan Baru, codirector of SDSC’s Oracle Corporation as the director of busi- Helly is one of many scientists who Data and Knowledge Systems program and ness development for their Oracle Small helped develop an end-to-end scientific data leader of the I2T project. “Our technologies, Business Suite. He also has experience in management system, from shipboard data using the common data exchange language business development with two Silicon

16 APRIL–JUNE 2002 enVision CALENDAR

To view the full Online article, append the issue number to the URL: www.npaci.edu/online/v6.x

Valley Internet startups and in strategic part- “We do not need the war against terror- JULY 2002 nerships involving U.S. small businesses. ism to justify investment in scientific 14–18 2002 Summer Computer (v6.10) research. Many studies have concluded that Simulation Conference (SCSC the long term health and security of our 2002), San Diego, CA NPACI’s Blue Horizon Sets Usage Mark nation depends on continued investment in 14–18 Sixth Multi-Conference on R&D. The future economy is one of global- lue Horizon, the 1.7 teraflops IBM Systemics, Cybernetics and ized high technology, and America needs to supercomputer at SDSC with 576 giga- Informatics (SCI 2002), Orlando, B maintain its technological prowess if we are bytes of main memory, was in use 87 percent FL of the time in April. “That’s an amazing to continue to provide world leadership. 21–24 Global Grid Forum (GGF5), record for a massively parallel machine,” said “President Bush’s budget for research and Edinburgh, Scotland Nancy Wilkins-Diehr, director of the Scienti- development in fiscal year 2003 embodies fic Computing Services Division of SDSC. priorities in medical research, nanotechnolo- 24–26 Education Technology 2002, She said the excellent performance was gy, and computing. It also identifies and pro- Arlington, VA vides funding for areas such as bio-terrorism, due to several factors: the machine ran large AUGUST 2002 jobs, which tends to minimize downtime, climate change, and energy technology that the new Catalina scheduler continued to per- are important national issues. The amounts 3–7 Tenth International Conference form well, and new automated recovery sys- allocated to these priorities reflect constraints on Intelligent Systems for tem for down nodes was in use. imposed by the cost of the war on terrorism Molecular Biology (ISMB 2002), “The largest scale computing resources and a weak global economy.” (v6.11) Edmonton, Canada benefit from sustained programs of tuning Visualization Software Improves 14–16 Computer Society Bioinformatics and scheduling to maximize their perform- (CSB) Conference, Stanford ance,” Wilkins-Diehr said, “and our close Collaborations University, Palo Alto, CA attention to the experiences of users enables reg Johnson, a visualization scientist at 17–21 16th Symposium of the Protein us to respond with equitable solutions. In Gthe Texas Advanced Computing Center, Society, San Diego, CA this case, we see a great success of the and Steve Mock, leader of the Grid Portal methodology. (v6.10) 19–23 NPACI Summer Parallel Architectures group at SDSC, and SDSC Computing Institute, San Diego intern Jeff Mock have developed a new appli- John Marburger Advocates Supercomputer Center, La Jolla, cation called Collaborative Application CA Balanced R&D Portfolio Visualization System (cAVS), which enables t a recent meeting of the Council of two or more researchers to interactively and SEPTEMBER 2002 A Scientific Society Presidents, John collaboratively explore scientific visualiza- 10–12 9th Annual Parallel Tools Marburger, director of the Office of Science tion-related data. For example, a researcher Consortium (Ptools’02), analyzing the output of a fluid-flow simula- and Technology Policy, said there is no ques- University of Tennessee, tion might do so in real-time collaboration tion that the ability of terrorists to wreak Knoxville, TN havoc in our society has been enhanced by with a colleague in another state. Together, 16–18 International Conference on the technology integrated into our way of they might share each other’s views of the life. But Marburger said technology will be visualization, as well as control parameters— Computer Design, Freiburg, essential to protecting us, too. such as color and lighting—affecting the Germany “Most people understand intuitively that graphical representation of the data. 23–26 Fourth International Conference the war against terrorism will have a strong Aresearcher working at a workstation can on Cluster Computing, Chicago, technical flavor,” said Marburger, who visited also use cAVS to work with an expert in a IL UCSD in March and April . “What is per- visualization laboratory to enable any aspect 24–26 iGrid 2002, Amsterdam, The of a visualization to be selectively shared. haps not as obvious is that much of the tech- Netherlands nology needed for this war is already avail- Moreover, the collaboration can be tuned to 24–26 Second Virtual Conference on able, or within reach of the current state of match the computational and network the art. New basic research is needed in resources locally available to each participant. Genomics and Bioinformatics, some areas, especially for countering bio-ter- cAVS was built on top of the Application “Sharing Knowledge with the rorism. But much of the technical aspect of Visualization System (AVS 5), an intuitive World” the war against terrorism will be in the and widely used tool. 30–Oct. 4 Linux Clusters Institute development of existing technology—the D More information about the visualization Workshop, Albuquerque, NM in R&D. That is why I have stressed repeat- system and components of the public- edly that the war against terrorism will not, domain software can be downloaded at For more information and events, see the and should not, alter the course of science. www.tacc.utexas.edu/cavs/. (v6.11) SDSC calendar on the Web: Subscribe to Online or ENVISION: www.npaci.edu/Press/subscriptions.html www.sdsc.edu/Calendar npaci & sdsc APRIL–JUNE 2002 17 BACK COVER NPACI PARTNERS University of California, San Diego/ San Diego Supercomputer Center OUNG HYUN, BRADLEY HUFFAKER California Institute of Technology Y University of Texas University of Michigan University of California, Berkeley Baylor College of Medicine California State University/San Diego State University University of California, Davis University of California, Los Angeles University of California, Santa Barbara University of California, Santa Cruz University of Houston The Johns Hopkins University University of Kansas Long-Term Ecological Research Network/ University of New Mexico University of Maryland Mississippi State University Montana State University New York University Ohio State University Oregon State University Rice University Rutgers, The State University of New Jersey The Salk Institute for Biological Studies The Scripps Research Institute University of Southern California Stanford University University of Tennessee University of Virginia Washington University University of Wisconsin Center for Advanced Research in Biotechnology University of Colorado Jet Propulsion Laboratory Kitt Peak National Observatory Lawrence Berkeley National Laboratory/National Energy Research Scientific Computing Center Lawrence Livermore National Laboratory Los Alamos National Laboratory University of Massachusetts Michigan State University Pacific Northwest National Laboratory/Environmental Molecular Sciences Laboratory VISUALIZING THE INTERNET UNIVERSE University of Pennsylvania BioComputing Unit, Centro Nacional de Biotecnología, alrus, a new visualization tool being developed by the Cooperative Association for Madrid, Spain Center for Research on Parallel Computation and Internet Data Analysis (CAIDA) at SDSC, enables investigators to understand the Supercomputers, Naples, Italy W complex structure of the global Internet. Walrus is a tool for interactively visualizing Computer Engineering Department, University of Lecce, Lecce, Italy large “directed graph” data sets using 3-D “hyperbolic geometry”—a form of distortion Parallel Computing Center, Royal Institute of resembling a view through a fisheye lens. The user can examine the details of a small region while Technology, Stockholm, Sweden University of Queensland, Brisbane, Queensland, keeping a view of the rest of the data available as a frame of reference. This image represents Australia round-trip times of data packets sent from a site in Herndon, Virginia, to hundreds of thousands Research Institute for the Management of Archives and Libraries, University of Urbino, Italy of nodes on the Internet and back again. See www.caida.org/tools/visualization/walrus. ▼

SUBSCRIPTIONS NATIONAL PARTNERSHIP FOR ADVANCED PRESORTED STANDARD For a free subscription to enVision, send the COMPUTATIONAL INFRASTRUCTURE U.S. POSTAGE information requested below to: Gretchen Rauen, NPACI/SDSC University of California, San Diego PA I D University of California, San Diego 9500 Gilman Drive, MC 0505 WESTERN GRAPHICS 9500 Gilman Drive, MC 0505 La Jolla, CA 92093-0505 La Jolla, CA 92093-0505 [email protected], 858-534-5111 ADDRESS SERVICE REQUESTED ______NAME ______TITLE ______INSTITUTION ______ADDRESS ______CITY, STATE, AND ZIP ______COUNTRY ______E-MAIL