The Doe Csgf Annual 2 0

2005 - 2006 2005 - 2006 DEIXIS 2005 - 2006 THE DOE CSGF ANNUAL DEIXIS – THE DOE CSGF ANNUAL DEIXIS – THE DOE CSGF ANNUAL

DEPARTMENT OF ENERGY COMPUTATIONAL SCIENCE GRADUATE FELLOWSHIP

Funded by: The Krell Institute Office of Science and 1609 Golden Aspen Drive, Suite 101 National Nuclear Security Ames, IA 50010 Administration’s Office of (515) 956-3696 Defense Programs www.krellinst.org/csgf

THE DOE CSGF ANNUAL

Department of Energy DEIXIS Computational Science Graduate Fellowship TABLE OF CONTENTS

2005 - 2006 PAGE 13 DEIXIS Editor (∆ΕΙΞΙΣ) transliterated Shelly Olsan from classical Greek into the Roman alphabet, (pronounced da¯ksis) means Copy Editor a display, mode or process of proof; Ron Winther PAGE 23 the process of showing, proving or Design demonstrating. DEIXIS can also Juls Design, Inc. PAGE 15 refer to the workings of an individual’s 46 keen intellect, or to the means by which Contributors 4 Alumni Profiles such individuals, e.g. DOE CSGF Jacob Berkowitz Practicum fellows, are identified. Alan S. Brown Experiences 46 Chris Oehmen PAGE 19 Victor D. Chase 14 Pacific Northwest DEIXIS is an annual publication Karen Hede 4 The Reasearcher’s Roadtrip Department of Energy National Laboratory of the Department of Energy Michael Szpir Lab Research Computational Science Graduate 4 Bree Aldridge 30 Fusion and Ice 47 Joel Parriott Fellowship (DOE CSGF) program. DEIXIS, The DOE CSGF Annual Measuring the Stroke of 14 Inside a Flame Brookhaven National Laboratory Office of Management and Budget DEIXIS illustrates work done at is published by the Krell Institute. The Krell Institute administers the Department a Butterfly’s Wing Lawrence Berkeley eight multi-program DOE laboratories of Energy Computational Science Graduate National Laboratory 34 Of Tsunamis, Thermonuclear 48 Mayya Tokman and highlights the DOE CSGF fellows Fellowship program for the DOE under contract DE-FG02-97ER25308. The Krell Institute is a 8 Sam Stechmann Explosions, & Asteroids University of California and alumni. The DOE CSGF is non-profit company that works in partnership Modeling the Ocean’s 18 Radioactive Retention Los Alamos National Laboratory funded by the Office of Science and with national science and technology Invisible Waves Sandia National Laboratories the National Nuclear Security research and education communities to help them achieve national priorities. 38 Heavy Metal Blues 49 Administration’s Office of 10 Teresa Bailey 22 Facilitating Warp Pacific Northwest Howes Scholars Defense Programs. Faster Math Ignites Speed Computing National Laboratory For additional information about the Department of Energy Computational Fusion Experiments Oak Ridge National Laboratory Science Graduate Fellowship program, 42 PETSc Engines 50 the Krell Institute, or topics covered in 12 Mary Biddy 26 Simulating an Urban Disaster Alumni Directory this publication, please contact: Argonne National Laboratory Salad Days for Lawrence Livermore Editor, DEIXIS The Krell Institute Plant-based Lubricants National Laboratory 1609 Golden Aspen Drive, Suite 101 56 Ames, IA 50010 Fellows Directory (515) 956-3696 www.krellinst.org/csgf PAGE 29

PAGE 31 FEATURE PRACTICUM EXPERIENCES 5 The Researcher’s Roadtrip

THE DEPARTMENT OF ENERGY Computational Science Graduate Fellowship (DOE CSGF) program supports the brightest science and engineering students as they prepare for a future in their respective fields. Nearly 15 years old, the DOE CSGF has grown to over sixty active fellows and over 200 alumni.

Aldridge and her graduate advisors, Measuring the Douglas Lauffenburger and Peter Sorger of Massachusetts Institute of Stroke of a Technology, Cambridge, Massachusetts, have teamed with Steve Wiley of Butterfly’s Pacific Northwest National Laboratory (PNNL), Richland, Washington, to Wing craft a large-scale mathematical representation of interacting cell signaling pathways. BREE ALDRIDGE “The general goal of studying these Massachusetts Institute of Technology signaling networks is so that we can | Pacific Northwest National understand how cells process so many Laboratory | Story by Karyn Hede signals and always make the correct decision,” says Aldridge. “Or another More than 30 years ago, way of looking at this is for cancer meteorologist Edward therapy. Today, drug companies see Lorenz poetically protein A is allowing the cancer cell described how small to survive when it should die, so they Left to right: events can have large try to inhibit protein A. But maybe in New imaging technologies can reveal the Teresa Bailey, consequences when he some cells protein A doesn’t matter location of oncogenic proteins in living Mary Biddy, posed the question: or the cell can alter its pathway a bit cells. Shown are human epithelial cells Sam Stechmann can a butterfly flapping its wings in to go around protein A. It may be expressing the oncogene HER2 that emit and Bree Aldridge Brazil cause a tornado in Texas two that protein A or protein C or protein green fluorescence. Lysosomes are stained weeks later? Lorenz’ musing was no F is important. So inhibiting each red and the cell nucleus is stained blue mere philosophical question. He of these pathways a little bit might for reference. proceeded to show mathematically how be more effective than completely small changes in starting conditions shutting down just one of them. can cause large effects in complex, That’s the sort of information we interdependent systems. hope to get someday out of these quantitative models.” Today biologists are teaming with TO A COLLEGE STUDENT, summer typically means warm mathematicians to apply quantitative beaches, lazy weekends, and no classes, but the DOE CSGF fellows live out a approaches to complex biological different kind of dream for the warmest three months of the year. During systems that have much in common the summer, fellows take a practicum in one of the Department of Energy’s with Lorenz’ meteorological laboratories — an experience that may impact the future of their research. conundrum. Department of Energy Computational Science Graduate The idea is to eavesdrop on the cell’s communication channels The opportunity for hands-on experience with world-class researchers and Fellowship (DOE CSGF) Fellow the latest technology is more than just a chance to hone analytical skills; it Bree Aldridge has set her sights on and discern, from seemingly indecipherable “chatter,” patterns is the quintessential road trip of one’s career (and maybe even self) measuring the molecular equivalent discovery. This trip is an opportunity to see what you have of Lorenz’ butterfly effect in the of signaling that cells use to make crucial decisions, become, and, perhaps, decide where you are going. complex milieu of the living cell. such as when to divide or when it’s time to die.

4 We simulate models of the signaling pathways to gain insight into the complex 7 signaling resulting from extracellular cues (graphic: Suzanne Gaudet, adapted by Bree Aldridge)

90 80 80 70 60 60 PROGRAM 50 40 REQUIREMENTS 40 leaved20 casp-3 30 Students selected for fellowships 0 agree to undertake study and

%C 20 research in computational science. 2000 The program of study must provide 5 10 Tim 1000 2 ) background in a scientific or e 1 (x10 engineering discipline, computer 0 0 science, and applied mathematics. IAP molec./cell In order to be considered for the DOE CSGF, students must be U.S. A schematic of the signaling networks induced by cytokine activation. citizens or permanent resident Cells use this complicated and intertwined protein pathway to process aliens and working toward a Ph.D. extracellular cues into a life-or-death decision. (Drawn by Peter Sorger and “There is no consistent language in “We are using gene expression and at a United States university. Douglas Lauffenburger, MIT). systems biology,” says Aldridge. “This proteomic data to filter the diagram and makes sharing models difficult. One cull out what are the real connections Students applying for fellowships must of my goals was to merge our two and interactions,” says Wiley. “One of be undergraduate seniors or in their models together so we can continue the things Bree was involved in was first or second year of graduate study. and expand our collaboration.” taking a look at the wiring diagram and Maintaining larger ODE-based as a list of Prior to the third year of the fellowship, The idea is to eavesdrop on the Wiley’s laboratory studies the epidermal equations is increasingly difficult as the looking at our list of genes that we had fellows must complete a practicum cell’s communication channels growth factor (EGF) receptor pathway, network models are expanded. In order to Another problem was that the identified as actually being involved assignment at a Department of Energy and discern, from seemingly which is implicated in many types begin merging model together, we needed models were built based on data and then, experimentally, looking to laboratory. Currently, approximately indecipherable “chatter,” patterns of cancer. Lauffenburger’s group to represent each model in the same format from two different kinds of cells. see if we knock out a particular piece, 15% of fellows who graduate from the of signaling that cells use to make studies tumor necrosis factor alpha (SBML) and annotate it using a graphical The Lauffenberger group studies a does it have the predicted effect on DOE CSGF program work or have worked in a Department of Energy laboratory. crucial decisions, such as when to (TNFalpha), another pathway that modeling tool (Teranode VLX Design Suite). colon cancer cell line, while Wiley’s the wiring diagram?” divide or when it’s time to die. often gets activated in cancers. EGF This screen shot shows how complicated group studies mammary cancer cells. and TNFalpha are signaling molecules just a part of the model is! (graphic: Bree Aldridge) The models the two labs developed When the practicum was over, Aldridge “Every cell in your body sees information called cytokines that communicate were different, but complementary. had not completed merging the two and makes a decision to do something, “danger” signals to the cell, which And Aldridge wanted to make them models, but she had developed a better DISCIPLINES PURSUED and its decision depends on its context, typically causes it to cease growing work together. understanding of what it will take to do its state and its condition,” says Wiley. and often to die. so and how the two labs can continue The fellows involved in the DOE CSGF “The cells make decisions based on To do that, she used two approaches. to refine the existing models by use of study widely varying subjects. However, information they derive from their “When we hit the cells with TNF, incorporated both the EGF and TNF First, she transferred the two models experimental results. For example, she they all are using high performance extracellular environment. One of some of the cells die and some of pathways, with all their various signaling to a single software package, a program had discovered that signals can get computing towards their research the things we’ve tried to figure out is them don’t,” says Aldridge. “We want molecules, networks and complex called Teranode, first developed at transduced through the system in goals. Fellows’ disciplines include how do cells know where they are. to know, why is that? But these systems interactions. “We are using ODEs the University of Washington, Seattle, different ways. biophysics, chemistry, biochemistry, For example, cancer cells don’t know are so complex that you have to use (ordinary differential equations), and now available commercially. civil engineering, computer science, where they are. Or they do know modeling to try and understand which describe mass action kinetics, “I learned that we need to pay more aerospace engineering, applied math, physics, bioengineering, aeronautical where they are and they just respond what’s happening.” to understand how cells make “The size of the model that she is attention to what’s happening outside engineering, chemical engineering, inappropriately. Everything a cancer decisions,” says Aldridge. building is much greater than anything the cell,” she said. “Instead of just bioinformatics, computational chemistry, cell does is perfectly normal to a cell Each research group had built its own else we’ve ever seen,” says Wiley. saying, ‘How do these pathways talk and computational mechanics. in a different environment, but it’s model of how signals are transduced The project involved both computational to one another inside the cell?’ it inappropriate to a cancer cell in its through the cell. Haluk Resat built and biological issues. For one thing, the But Aldridge soon discovered that her may be as important to understand environment. If you understand why the EGF model at PNNL, and Birgit models were built by use of different system was too complex to run on the extracellular methods of cross talk the cell is doing the wrong thing at the Schoeberl and Suzanne Gaudet built software, even different computer existing software package, and she between the pathways.” wrong time you have an approach the TNF model in the Lauffenburger languages. The PNNL group shifted her attention to reducing the to try to convince it to do the right and Sorger groups. Aldridge’s task programmed in Fortran, while number of interacting molecules The butterfly’s wing flutters thing, which for a cancer cell is was nothing less than to combine the Lauffenberger’s lab used Matlab. required to run the model. once more. ‘stop growing; drop dead.’” two models into a single model that

6 Stechmann’s interest in combining The Navy asked Holm to find a Modeling physics and mathematics led him to way for submarines to predict when 9 the Courant Institute at New York internal waves would strike. “There’s This is a rear-end collision of two puckons. University (NYU) to study fluid no way we could do a numerical the Ocean’s A puckon's velocity is proportional to dynamics and computational science. simulation of every tide,” he explains. its height. The taller puckon catches the “To do it, we needed some sort of Invisible shorter one, they collide, and then they He also wondered how he would fare approximate equations.” move apart – in the same way two so far from home. Until he graduated Waves billiard balls or Cadillacs would collide. from college, he had never been further than the Iowa border. His first real Solitons SAM STECHMANN road trip, however, did not take him to New York. Instead, he received a To develop appropriate equations, New York University - Courant Institute summer internship at Los Alamos Holm turned to a phenomenon first This picture shows how an initial bump | Los Alamos National Laboratory | National Laboratory (LANL) in New described by famed nautical engineer breaks up into puckons. No matter what This generated waves that looked as if momentum fell to zero at the pole, but Story by Alan S. Brown Mexico. There he met Darryl Holm John Scott Russell in 1834. When a type of initial condition you pick, it will someone had dropped a hula-hoop in not fast enough to work. Instead of the and learned about waves. boat drawn through a canal suddenly break up into puckons. The picture shows a shallow basin. circles dying off, funny things happened Stand on a beach and stopped, Russell saw the water it had what shape the bump has as it evolves at the poles and then propagated SCOPE OF watch the waves. Driven put in motion accumulate around the in time. The bottom is the smooth bump “We had difficulties when the radius of everywhere. They soon screwed PROGRAM by tides and sculpted Submarines vessel’s prow and then shoot forward at the initial time. After awhile it sheds the waves generated inside the hula-hoop up everything.” by the islands and as a solitary wave. a puckon, which has a peak. approached zero,” says Stechmann. Since its inception, the atolls that break and The US Navy had asked Holm, a “The model didn’t know what to do. It It took the summer for Stechmann to DOE CSGF program has reform them, no two Laboratory Fellow and member of Classical theory held that such waves became very unstable and the code blew resolve the issue. He continued to use a supported nearly 225 are alike. No wonder LANL’s Computer and Computational should quickly dissipate. This one up.” Stechmann believes he could have grid approach, but he laid his grid out students in more than 50 mathematicians find it difficult to Science Division, to investigate the did not. Russell galloped after it on eventually puzzled out a solution. To on a sphere, and created boundaries universities all over the represent the changing sea. ocean’s internal waves. horseback for more than a mile. It Planes get something done by the end of around both poles. Once a soliton U.S. Currently it supports 64 students in 22 states. took 60 years before mathematicians summer, he tried a different approach. passed beyond the boundary, it Such challenges — and the sea itself — On the surface, internal waves are explained how Russell’s “waves of Holm asked Stechmann to find a way to ceased to matter in the simulation. were the furthest thing from Sam all but imperceptible. Below, in the translation” could propagate over visualize solitons on a two-dimensional Stechmann turned to computational Stechmann’s mind while he was thermocline — the layer of water such long distances in shallow water. plane. Past researchers had found fluid dynamics (CFD), a technique The resulting model worked about as growing up in Red Wing, Minnesota, where temperatures drop rapidly — that for any single point on a soliton, that breaks a plane into grid points. well as one from the previous summer. For nearly 15 years, the a small town about an hour’s drive they are massive. Powered by tides, ranks Interest subsided until the 1960s, when velocity is proportional to height. To describe a soliton, he subtracted DOE CSGF program has southeast of Minneapolis/St. Paul. of waves 90 meters high, 1 kilometer two researchers discovered another That made it possible to describe the the momentum of one point from Visualizing soliton waves is only part of encouraged the training thick, and stretching hundreds of property of the waves: They could pass velocity and height of a single point that of another. Since a soliton’s the larger attempt to predict the ocean’s of computational scientists by providing As a boy, Stechmann, a 2004 kilometers long race through the ocean through one another, transferring on a soliton as a function of one velocity and height are related to internal waves, but vital nonetheless. financial support to Department of Energy Computational for thousands of kilometers. From momentum but still retaining their space variable, say “x.” its momentum, the resulting cascade Stechmann’s contribution has given some of the most Science Graduate Fellowship the Space Shuttle, they look like shape. Because they acted more of numbers approximated a wave. “It researchers a way to visualize talented graduate (DOE CSGF) Fellow, was entirely great lines on the sea. like solitary particles than waves, the Representing the arc of soliton waves took longer but it worked,” he laughs. their equations. students in the nation. devoted to sports, especially hockey. researchers renamed them “solitons.” moving across a plane is far more “Until fourth grade, I only read sports When those waves reach a submerged complex, since it requires two space Meanwhile, Stechmann has returned books,” he recalls. “Every school submarine, the greater density of the Since then, mathematicians and variables, say “x” and “y.” Yet Holm Spheres to NYU, where he continues to study project was about sports. My school cold wave changes the buoyancy of the physicists have probed soliton and Vakhtang Putkaradzehad of the applied mathematics. He also continues invention was a hockey stick that sub. “There’s a big jerk as the sub properties. When Stechmann arrived University of New Mexico found a way After finishing his first year of graduate to play hockey, and last year helped curved both ways so you could accelerates,” says Holm. “You’d better after graduating from St. Thomas, to simplify the problem. They imagined school, Stechmann returned to Los his team place second in the national shoot better backhand.” be wearing a helmet if it’s happening.” Holm, an authority on solitons, had the waves as concentric circles. While Alamos for his DOE CSGF practicum club hockey championships. Internal waves also reflect active begun to apply these techniques to movement of these circles across a to work with Holm and his collaborator, Finally, when he was in middle school, sonar, the submarine’s “eyes”. internal waves. plane could be defined by their “x” Jonathan Munn of Imperial College, his mother made him enter a math and “y” position on a grid, the entire London, on solitons on a spherical contest to broaden his horizons. He circle could also be described as a surface. To most people, a sphere looks A numerical solution of surprised himself by winning a regional function of its distance from the center, like a three-dimensional object. To the ODE for the puckon's competition and continued to place i.e., its radius. By looking at only this Stechmann, it looked like a curved latitude as a function of throughout high school. one variable, radius, they vastly reduced plane. He figured he could use the time. Solving the ODEs the computer power needed to describe same approach he had tried the numerically is much faster Although he entered the University a solution. summer before. and easier than solving of St. Thomas in St. Paul as a pre-med the PDEs numerically. major, Stechmann had no idea what Numerical solution of the Stechmann’s goal was to make the “It didn’t work,” he explains. “When You can see that the he really wanted to do. That changed ODE for the puckon's model work. He started by playing with we created circles on the sphere, they two results agree. in freshman chemistry, where he made height (velocity) as a solutions that described solitons as a move toward the pole and got smaller friends with several physics majors function of time. Again, series of waves of different frequencies. and smaller. Eventually, their height and who took him to see their laboratory you can compare it with projects. By the end of his sophomore the PDE solution to see year, Stechmann had switched his that the two agree. major to a combination of physics, Every school project was about sports. My school invention was a chemistry, and math. 8 hockey stick that curved both ways so you could shoot better backhand. The program was small, and that Communication Calculations Tackling the Code Faster Math proved to be a good thing for Bailey. 11 "The faculty cared about the students," The disadvantage of the existing Adams, says Bailey, found a way to make Ignites Fusion she recalls. "There was a lot of interaction. diffusion model is that it is equation- the problem symmetric, so that one They challenged us and they trained intensive. In order to create enough equation communicates between two Experiments us well. I liked it right from the start." points for an accurate simulation, the points in both directions. This halved model divides space into hexahedrals, the amount of memory needed to run a In her senior year, she became the corners of which form the points. simulation. "In really large systems, this TERESA BAILEY fascinated with radiation transport, This creates a matrix with millions or is a big deal," says Bailey. "We can either how radiation interacts with a medium even tens of millions of points. run simulations faster or increase Texas A&M University | Lawrence as it moves through it. "These problems their accuracy in the same amount A piecewise linear Livermore National Laboratory | are really a challenge to solve, but you Each point communicates only with of time." DOE CSGF Story by Alan S. Brown (PWL) basis function can use the results in everything from its nearest neighbors to describe the in 2D. HIGHLIGHTS astrophysics and nuclear reactor design interplay of energy and matter. Typically, Bailey's job was to program the new Teresa Bailey likes a to shooting a beam of photons at a each point is connected to 27 other approach into the radiation diffusion > Payment of tuition challenge. A graduate tumor," she explains. points by means of one or more code. "Teresa had never coded in C++ or and required fees student in nuclear equations that define how each pair of used massively parallel supercomputers," engineering at Texas She went on to study radiation transport points communicate with one another. says Zika. "This went well beyond her > Yearly stipend A&M University, in graduate school at Texas A&M. graduate school experience, where she of $31,200 College Station, Texas, There, she worked closely with Marvin The existing diffusion model is and two or three others might work on and a DOE CSGF When the laser beams strike the inner The multiphysics part refers to the Adams, who developed a faster method asymmetric: The equation that describes a few thousand lines of code. Here, > A $1,000 yearly Fellow, she got one when she spent the surface of the hohlraum, they instantly model's ability to simulate a vast range to calculate radiation diffusion during how Point A talks to Point B is different she was working with 20 of us and academic allowance summer of 2004 at Lawrence Livermore vaporize the gold into boiling plasma. of physical properties simultaneously. fusion. In the summer of 2004, Bailey from the equation that describes how hundreds of thousands of lines of code." National Laboratory (LLNL), home of The plasma surrounds the pellet, The model is run on a massively joined Zika, another Adams protégé, Point B talks back to Point A. This > Matching funds of up to the National Ignition Facility (NIF), emitting high-energy X-rays. This heats parallel supercomputer, an IBM SP2 to see if this new method would work. means that each point on the grid Bailey not only completed the code, but $2,500 for a computer 50 miles east of San Francisco. the pellet so fast that it explodes inward, capable of training 8,000 powerful needs separate equations for each began testing it as well. "Both methods workstation purchase squeezing the deuterium and tritium processors on a single problem. of the 27 other points with which it work really well," she says. "It's extremely The multibillion dollar facility together until they fuse. The Task communicates. Multiply that by millions hard to find areas where they get > Opportunity to complete was built to create a fusion reaction Even with all that power, says Mike of points on the grid and it is no bad answers." a practicum working that generates more energy than it Zika, who heads a multiphysics model To understand Bailey's task, consider wonder the calculations tax even with scientists and consumes. And it has challenges Models development effort, some simulations radiation diffusion in the hohlraum the largest supercomputer. For Bailey, it was a chance to tackle researchers at a aplenty for anyone in the field. have lasted as long as four months. after ignition. The plasma inside the a challenging problem and learn DOE Laboratory The model is as complex as the process. Clearly, the lab would welcome any hohlraum gives off X- rays. These new skills. "I think it broadened her It must account for variables ranging approach that saves calculation time. energize the electrons of surrounding horizons and gave her a real flavor > Yearly fellows conference with Example of a polyhedral mesh. Ignition and Lasers from laser pulse, power, and energy atoms, scattering some and causing of what's out there for her when she fluctuations to the deuterium:tritium others to emit more X-rays, which graduates," Zika concludes. opportunities to NIF plans to fuse hydrogen isotopes ratio of the pellet. It must also calculate Finding Physics then interact with more electrons. meet other fellows and industry and in BB-sized pellets, using 192 high- the interactions of the laser beam government powered lasers. To achieve successful moving through the plasma, the Enter Bailey. As a high school student One set of equations describes how the professionals ignition, though, researchers first plasma generating the X-rays, the in Anchorage, Alaska, she thought she radiation diffuses in time and space. need to optimize their equipment, shockwave caused by the X-rays, and might want to study English or political A second set reveals how it changes > Renewable up to materials, and fusion conditions. the thermonuclear reactions among science in college. She even interned the energy of the surrounding matter. four years particles. Everything happens within for Alaska Senator Ted Stevens the Unfortunately, both sets of equations Because NIF's hardware is far too nanoseconds at extremely high summer after she graduated. are partial differential equations (PDEs). For more information: expensive to build and rebuild for temperatures and pressures. "Computers don't know how to solve www.krellinst.org/csgf each test, researchers are using When it came time to pick a college, PDEs," says Zika. "They only know how supercomputers to model what To simulate what happens inside the though, Bailey wanted more educational to deal with discrete operations like A 3 cell polygonal mesh. will happen when they turn on hohlraum, LLNL is building massively options than Alaska offered. She opted addition, subtraction, multiplication, all 192 lasers. parallel multiphysics computer codes for nuclear engineering at Oregon State division, and exponents. To solve PDEs, as part of the Department of Energy's University. "I really liked what I had we have to translate them into discrete These are not simple calculations. Advanced Simulation and Computing learned about physics in high school, operations. For a good simulation, you NIF's approach to fusion starts with (ASC) Program. and my sisters were already there," need lots of points, and calculating all fuel pellets of two hydrogen isotopes, she explains. those points eats up computer time. tritium and deuterium. The pellets There are many ways to make these are surrounded by a crust of frozen approximations, and they all have isotopes encased in a plastic skin. Each advantages and disadvantages." pellet is then hung in a hohlraum, a These problems are really a challenge chamber that resembles a very small soda can plated with gold on the to solve, but you can use the results inside and open at both ends. in everything from astrophysics and nuclear reactor design to shooting a Top Hat Plots 10 beam of photons at a tumor. prove themselves the equal of mineral That’s why Biddy’s summer practicum Salad Days for oils over a wide range of temperatures at Sandia National Laboratory (SNL) 13 and must withstand the friction of in Albuquerque, New Mexico, was Plant-based moving engine parts. so valuable to her research. Using a computer code called LAMMPS – Lubricants It turns out that vegetable oils acquit Large-scale Atomic/Molecular Massively themselves quite nicely in most of Parallel Simulator — developed by the head-to-head comparisons with Steven J. Plimpton at Sandia, Biddy UNDERSTANDING THE POUR POINT MARY BIDDY conventional motor oils, except for one was able to simulate her system on a TEMPERATURE OF VEGETABLE OILS key feature. Vegetable oils, which are much larger scale, both in terms of (a) The molecular structure of an University of Wisconsin | Sandia composed of triglycerides, comprised number of molecules and time frame individual triglyceride. The snapshots National Laboratory | Story by of glycerol linked to fatty acids, turn of the simulation. of the simulation box of a canola oil Karyn Hede to sludge when they get too cold. below its pour point temperature “It allowed us to see that on a larger display (b) all the atom sites and Should you go with This property, called the oil’s pour scale these systems do tend to have some (c) only the carbonyl groups of the canola or corn, point, is crucial to making vegetable localized ordering and that’s something triglyceride molecules. It is believed safflower or soybean? oils practical for use in cars. that we are currently investigating,” she that the clustering of the carbonyl It sounds like a says. “We want to understand what effect a method developed at SNL to model project was part of an international groups leads to the pour point in question about salad “It’s a very important property because the local ordering has on the pour lipid bilayers, highly-ordered collections contest called the Industrial Fluid vegetable oils. dressing or perhaps someone in Wisconsin, for instance, point transition.” of fatty molecules. Properties Simulation Challenge. frying oil, but in Mary doesn’t want the lubricant in the car Biddy’s research the ratio of vegetable to have a pour point temperature Working with SNL scientist John Curro, “We wanted to see if triglycerides can “I was lucky to be able to work on oils is key to whether they will perform that is warmer than the temperature Biddy developed a new technique for form bilayers,” says Biddy. “There is no another project that fits with my thesis in car engines. Yes, that was car engines. outside,” says Biddy. “One of the main approximating the molecular forces experimental evidence that they do, but work in that it looks at the industrial molecules while Mary did the other. disadvantages of these vegetable the triglyceride molecules exert on it is a difficult parameter to measure — relevance for modeling simulations,” We managed to take second prize. Biddy and her graduate advisor, Juan oil-based lubricants is this pour point one another. “We developed a coarse- you can’t see it in experiments. The only says Biddy. Not bad for six weeks of work.” de Pablo, Professor of Chemical and property. Petroleum and synthetic oil grained model, which means we way to understand the structuring is by Biological Engineering, University of have pour points of –40ºC (-40ºF) while developed a method for lumping modeling. We modified Laura Frink’s The contest involved groups of Biddy and Martin received their award Wisconsin, Madison, Wisconsin, are vegetable oil’s pour point is –18ºC (0ºF). the atoms into “super-atoms” for the model to see if we could understand scientists in research laboratories and a cash prize at a special ceremony trying to perfect the formula for a This is one property that has to be purposes of modeling. When we can how electrostatic interactions, partial around the world who were asked during the American Institute of vegetable oil that will perform as well changed if vegetable oil is ever to be lump many atoms into one “bead” charges on the molecule, affect ordering. to develop simulation methods to Chemical Engineers annual as, or perhaps even better than, motor used in cars. Our idea is that this pour we can model things on longer time If we include partial charges or leave predict physical properties of defined meeting in 2004. oil in internal combustion engines and point temperature occurs because scales. Using an atomistic model, we it off in the model, does it change chemical agents. Working separately, other industrial applications, without there is ordering that occurs on a could only model 40 molecules, but the ordering?” scientists at the National Institute of Martin and Biddy published their polluting the environment. Their molecular level in these triglycerides. with the coarse-grained model we can Standards and Technology (NIST) and results in a paper titled “Monte Carlo research could help the United States The idea is to understand what is begin to model larger systems with That project is still ongoing, and results Dow Chemical, the contest’s sponsors, Molecular Simulation Predictions for to wean itself from dependency on happening on a molecular level, so shorter computation times to predict are not in yet. Part of the reason, says made experimental measurements of the Heat of Vaporization of Acetone petroleum products. But to compete we can engineer a lubricant that has viscosity, which is an important Biddy, is that she took on four projects the properties, which were then used and Butyramide," in the journal with petroleum-based lubricants, often a lower pour point temperature.” property of lubricants.” during her practicum. “In retrospect, to judge the contestants’ predictions. Fluid Phase Equilibria. called mineral oils, vegetable oils must it probably wasn’t realistic to expect Biddy and de Pablo use molecular For Curro and his SNL colleagues, to finish them all,” she admits. “Mary instigated the project and quickly “It was challenging to work on a truly dynamics simulation techniques to the collaboration with Biddy helped came up to speed using a new (to her) difficult real-world problem and to be understand the potential for localized validate their approach to simulating But one problem that did have an simulation code called Towhee, a exposed to the whole broader world ordering of liquid triglycerides. They complex polymers. immediate payoff was her participation publicly available software program of molecular modeling,” says Biddy. had run simulations that suggested with SNL scientist Marcus Martin that uses Monte Carlo simulation to “It helped me to realize that there are local ordering of triglyceride molecules “We are interested in simulating the (a former DOE CSGF fellow) to predict complex chemical systems,” scientists who think about problems in the liquid phase, but it was hard to behavior of high molecular weight predict the vapor pressures and heats of says Martin. “I added a few new in a really different way than I have make a case for ordering with the small polymer mixtures and blends for basic vaporization for acetone and butyramide features to enable us to pursue this learned in my graduate training. It number of molecules they had been able research and defense applications,” says under specific conditions that mimic research. We then divided up the gave me a lot to think about when to simulate using the computational Curro. “Simulating triglycerides, which those of industrial applications. The simulation work as I did one of the I start applying for jobs.” resources at Wisconsin. are low molecular weight, helped us compare PRISM theory (a statistical mechanics theory) with Mary’s calculations. It helped validate the theory. It’s clear both of us benefited.”

In another project, Biddy worked To compete with petroleum-based lubricants, with SNL’s Laura Frink and Amalie often called mineral oils, vegetable oils must MODEL REPRESENTATIONS OF METHYL ACETATE Frischknecht to calculate the behavior Methyl acetate at different levels of molecular DENSITY FUNCTIONAL THEORY SIMULATIONS of large numbers of triglycerides by use prove themselves the equal of mineral oils detail: (a) all-atom where all atom sites are modeled, (a) A schematic of the coarse grained triglyceride molecule that has been studied of classical density functional theory, (b) united-atom where the methyl groups are represented (b) Density profiles of a triglyceride bilayer near the order-disorder transition over a wide range of temperatures and must as a single group, and (c) coarse-grained where all atoms are represented as a single super-atom. 12 withstand the friction of moving engine parts. DOE LAB RESEARCH Lawrence Berkeley | Sandia | Oak Ridge | Lawrence Livermore | Brookhaven | Los Alamos | Pacific Northwest | Argonne 155

By Michael Szpir Inside a Flame

SOME ANTHROPOLOGISTS BELIEVE that our ancestors were tinkering with fire in a controlled way nearly 1.5 million years ago. It’s been called our oldest technology, and we rely on it dearly. About 85 percent of the energy in the United States comes from burning fossil fuels. What’s amazing is that we still don’t fully understand the process.

“One of the things that’s not well Knowing the details is crucial if you that to a supercomputer and let it grind “The simplest low Mach number model appreciated by most people is the want to design a combustion-based away, but with the existing computer is the incompressible Navier-Stokes complexity of a flame,” says John system — say, a turbine or an internal- hardware it would take years to get equations, which describe how a Bell, group leader of the Center combustion engine — that burns cleanly a solution.” fluid moves when it can’t expand for Computational Sciences and and efficiently. “You can’t accurately or contract,” explains Bell. “The low Engineering at Lawrence Berkeley describe either the detailed motion Bell and his colleagues recognized a way Mach number model that we use for >> Combustion National Laboratory (LBNL) in Berkeley, of the flame or the emissions from an to sidestep the equations associated with combustion is a generalization of those California. Over the past 10 years, Bell engine without a detailed understanding compressible fluid mechanics, which equations. But what all low Mach and his team have put together a set of what’s happening inside the flame,” eat up a lot of computing time. Those number models have in common is >> Geochemistry of numerical algorithms that allows says Bell. equations track the sound waves in a that, because they don’t resolve the them to simulate a laboratory-scale fluid very accurately, but it turns out sound waves, they can advance the flame to see what’s going on inside. Unfortunately, experimental scientists that’s not necessary. The fluid velocity system in time with a much larger >> Computational Performance It’s not what you might imagine. who study real flames in the laboratory in a laboratory-scale flame is only about time step than the equations for a can provide only part of the picture. 10 meters per second (and the flame fully compressible flow.” When Bell looks at a flame, he sees a “Flames are hot. They’re dynamic. It’s moves even more slowly), whereas a >> Advanced Scientific Computing “reacting flow” — a fluid consisting of hard to take measurements,” Bell says. sound wave travels about 300 meters per This approach saves dramatically a number of chemical species that react “You can’t stick a meter into a flame and second. “You can show that in many cases on computing time, running about as they move through space. Consider read off all the different chemicals in the propagation of sound waves doesn’t a hundred times faster than standard a simple methane flame, the subject of the flame at that point.” Numerical affect the system,” Bell says. “There’s a codes. It’s a crucial savings that begins >> Fusion Energy Sequence of images of a Rayleigh-Taylor unstable nuclear flame Bell’s simulations. In high school you simulations can tell you all about the separation of scales, so you don’t need to make detailed simulations possible, in a Type Ia supernova. In the simulation the flow transitions from >> Thermonuclear may have learned that burning methane chemistry in the flow, but these kinds compressible fluid mechanics.” but it’s not enough to accurately model That wrinkly object is also oscillating laminar to turbulent. As the flow changes, the mode of combustion Applications with oxygen gives you heat, carbon of simulations wouldn’t be possible a real flame. Fortunately, there’s another very quickly, giving the impression changes from the “flamelet” regime to “distributed” combustion. dioxide and water. In reality, the process without a whole suite of new Instead, Bell’s team has devised way to save even more computing time. that a flame is much thicker than it Simulations of this type play a key role in validating subgrid >> Computational of burning methane involves about mathematical tools. a numerical code based on a “low This method exploits a feature of the really is. A flame that looks like it’s models required for full-star simulations. Biosciences 50 chemical species and more than Mach number formulation,” so called flame that is not obvious at first glance. about 10 millimeters across is really a 300 chemical reactions. “The traditional way to simulate a flame because it describes fluids that are very thin sheet, roughly one millimeter >> Computational would be based on the equations of moving much more slowly than the thick, which is flopping all over the Tool Kit “There are a lot of chemicals with short compressible fluid mechanics, as well as speed of sound. (The Mach number is Flaming Illusion place. The thinness of a flame has lives that are created and destroyed equations for all the chemical reactions, the ratio of the fluid speed to the speed consequences for how the scientists inside the flame,” says Bell, “and the and transport, which represents how of sound.) The low Mach number model When you look at a flame, it appears use their precious computing time. details of that chemistry make all the the chemical species diffuse inside the is more complicated mathematically to have a very smooth, tapered shape. “You want to focus your computational difference in how the flame propagates.” flame,” explains Bell. “You can send all than the equations for compressible flow. “That’s just an illusion, an average of power where the flame is,” says Bell, the flame’s real appearance,” says Bell. “not centimeters away where not If the eye and the brain could parse much is going on.” time into nanoseconds we’d see a much Over the past 10 years, Bell and his team have put more interesting object, one with kinks together a set of numerical algorithms that allows them and wrinkles and corrugations. to simulate a laboratory-scale flame to see what’s going on inside. It’s not what you might imagine. 14 175 Image of simulated V-flame showing the location of the grid patches used by the adaptive mesh refinement algorithm. Each box represents a number of computational cells aggregated into a grid to discretize a local region of Photographic image of a laboratory space. The green boxes indicate one level of refinement of the base grid V-flame, courtesy of Robert Cheng, and the pink boxes indicate an additional level of refinement. (Some boxes EETD/LBNL. The image is taken with are removed to show the flame surface.) Image of simulated V-flame a shutter speed of 1/30 sec. showing the instantaneous location of the flame.

JOHN BELL That is important when you consider The flame that Bell and his colleagues how a reacting flow is modeled in a conjure in the computer lasts for only a right experimental data to advance the John Bell received his B.S. (1975) degree from the Massachusetts computer. Computational scientists few tenths of a second, but the group’s computational study of these things?’ Institute of Technology and his M.S. (1977) and Ph.D. (1979) degrees divide the region, or “domain,” they simulation is considered to be the best It’s a very exciting time.” And he adds, There are seven people in Bell’s group, from Cornell University, all in mathematics. He is currently a Senior want to simulate into millions of small of its kind. “Ours is the only simulation “But you have to be careful what you as well as two students from the DOE Staff Scientist and Group Leader for the Center for Computational boxes, or “grid cells.” Each grid cell of a real three-dimensional turbulent, say, because they’re likely to go to the CSGF program for the summer of 2005: STAR FIRE Sciences and Engineering at Lawrence Berkeley National Laboratory. carries information about the fluid pre-mixed laboratory-scale flame,” lab and do an experiment!” Jasmine Foo from Brown University, Prior to joining LBNL, he held positions at the Lawrence Livermore velocity, its temperature and density, says Bell. “It was possible because we Providence, Rhode Island, and Amber National Laboratory, Exxon Production Research and the Naval as well as the relative amounts of each developed code technology that’s about Bell has worked in several different Sallerson from the University of North >> A chance meeting at a scientific conference in 2002 has united two Surface Weapons Center. of the chemical species — adding up three or four orders of magnitude fields since his career began in the Carolina at Chapel Hill, Chapel Hill, seemingly unrelated endeavors: terrestrial combustion research to about 25 variables in a typical faster than traditional simulation late 1970s, including aeronautics, North Carolina. Both are working on and the study of supernovae. John Bell had been standing by his Bell’s research focuses on the development and analysis of numerical presentation during a poster session in a large conference room methane-flame calculation. Each of approaches.” seismology, and petroleum reservoir problems related to fluid mechanics, methods for partial differential equations arising in science and when, during a lull, he casually asked the people at the poster next engineering. He has made contributions in the areas of finite difference these quantities must be updated in simulation, but his work on combustion although neither was involved in the to his about their work. They happened to be two astrophysicists, methods, numerical methods for low Mach number flows, adaptive every cell at every time step, adding up at LBNL has provided a unique level flame simulation. Stan Woosley and Mike Zingale, from the University of California, mesh refinement, interface tracking and parallel computing. He has to billions of calculations. The smaller A Burning Love of collaboration. “This is the first time Santa Cruz. also worked on the application of these numerical methods to problems each grid cell, the more accurate the in my career that there was such a good If you were to visit LBNL you might from a broad range of fields including combustion, shock physics, calculation. But the more grid cells Bell’s work on reacting flows has partnership between the people who find Bell walking around the halls As Bell recalls, “They talked about their research on supernovae seismology, flow in porous media and astrophysics. His group’s there are, the more work the computer already been used in a number of do experiments and the people who or standing at a whiteboard, talking and explained how hard it was because the flames inside the stars web page is at http://seesar.lbl.gov/ccse/index.html. must do. applications, including the measurement do computation,” he says. over ideas with his colleagues, but move so slowly. So I said, ‘Why don’t you do this with a low Mach of nitrogen-oxide emissions, simulations often he just sits at his desk and number model?’ They asked, ‘What’s that?’ And I said, ‘We do these Dr. Bell was the recipient of the 2005 Sidney Fernbach Award from calculations of flames and we use an algorithm that doesn’t track Using a technique called “adaptive of vortex-flame interaction and, perhaps Formerly at Lawrence Livermore works on problems. “I enjoy doing the IEEE Computer Society which was presented at Supercomputing acoustic waves. Don’t people do that in astrophysics?’” Well, it 2005 in Seattle, WA. mesh refinement,” the simulations surprisingly, the study of supernovae (see National Laboratory, Bell started the computation and getting involved turns out they hadn’t, but now they do. performed by Bell’s team use different sidebar: Star Fire). As a mathematician, working at the national laboratories in the coding,” he says. “To an outside Further Reading: sized grid cells in different regions Bell gets a tremendous amount of because they were among the observer it may look like I just scribble On the face of it, it’s hard to imagine there’s any similarity between a J. B. Bell, M. S. Day, I. G. Shepherd, M. Johnson, R. K. Cheng, of the flame. The middle of the flame pleasure from seeing his work applied few places that had the computer on a whiteboard or a yellow pad, punch modest laboratory flame and a type Ia supernova — an annihilating J. F. Grcar, V. E. Beckner, M. J. Lijewski Numerical Simulation of a contains small grid cells that allow for to these problems. “When you study hardware to tackle complex problems something into my workstation and explosion of a white-dwarf star that releases about 1044 joules of Laboratory-Scale Turbulent V-flame, LBNL Report LBNL-54198-Journal, accurate resolution of the detailed partial differential equations in in computation, but he ended up drink a lot of coffee.” He pauses, then energy and ejects matter at speeds of up to 10,000 kilometers per Proc. Natl. Acad. Sci. USA, 102(29) 10006-10011, 2005. chemistry there, but less resolution traditional mathematics, you might staying because of the people. “When adds, “But I love it.” second. As it happens, both processes involve reacting flows that travel well below the speed of sound. M. Zingale, S. E. Woosley, C. A. Rendleman, M. S. Day, and J. B. Bell is needed outside the flame. By not prove there is a solution, or come I first began working in the field, the using tiny grid cells in the whole up with some complex, infinite series application of numerical methods for All the scribbling has resulted in some Three-dimensional Numerical Simulations of Rayleigh-Taylor Unstable Just before a white dwarf obliterates itself, a thermonuclear flame Flames in Type Ia Supernovae, LBNL Report LBNL-56966, accepted, domain, the researchers can reduce solution, but it’s not tangible,” he says. differential equations was in its infancy. important contributions to applied ignites deep inside the star. The flame begins at subsonic speeds, Astrophysical Journal, Vol. 632, page 1021, October 2005. the total number of grid cells that “In computing you use all sorts of A lone investigator could program a mathematics. In July 2003, Bell was but it accelerates rapidly, ultimately propagating throughout the the computer must track. sophisticated mathematical ideas to state-of-the-art method in a few days,” he recognized by the Society for Industrial star and causing it to explode. Although the reactions in the white M. S. Day, J. B. Bell, J. F. Grcar, V. E. Beckner Numerical Control of create a new algorithm, but you end recalls. “Now we have very sophisticated and Applied Mathematics and the dwarf involve nuclear chemistry — the fusion of nuclei — the 3D Turbulent Premixed Flame Simulations, LBNL Report LBNL-56882 “The adaptive mesh algorithm up with something that is more nonlinear algorithms like the low Mach Association of Computing Machinery reacting flows are similar mathematically to the laboratory flame. Ext. Abs., Proceedings of 20th International Colloquium on the Dynamics automatically adds and removes substantial. That’s neat.” number model, and we do calculations with the first SIAM/ACM Prize in of Explosions and Reactive Systems, July 31-August 5, 2005. grid cells based on what it thinks it on hundreds of processors on a fairly Computational Science and Engineering In collaboration with Woosley and Zingale, Bell and his group have needs to be accurate,” says Bell. “At The success of the reacting flow complex parallel architecture. It’s more for his “outstanding contributions to the been able to extend their low Mach number code to describe events Contact: inside a white dwarf. The joint work has provided both a better its peak, the calculation has about 40 simulations has spawned a close than one person can do. At LBNL I can development and use of mathematical John Bell understanding of how the flame accelerates and a new tool for [email protected] to 50 million active grid cells.” That’s collaboration with experimental assemble a group of people who have and computational tools.” astrophysicists who study these types of stellar flows. a huge number, but it turns out to be scientists who study real flames in the all the different kinds of expertise one just a tenth of what the simulation laboratory. “We’ve gotten to the point needs to do cutting-edge research.” would need without adaptive where there’s actually a lot of synergy,” PRACTICUM COORDINATOR mesh refinement. says Bell. “The experimentalists are Joseph Grcar ...... [email protected] starting to say, ‘How can we provide the

By Victor D. Chase Radioactive Retention

NUCLEAR POWER, be it for weaponry, electricity generation, or medical applications, is a messy business. First comes the mining of uranium ore, a process that produces radioactive tailings that can be dangerous if not dealt with properly. Then there is the processing of the uranium into nuclear fuel or weapons grade material which creates radioactive residue. And finally there is the spent product, which after serving its intended purpose must be disposed of in a safe and secure manner.

Overseeing the life of this nuclear Energy. The processing plants were An Exceptional Sponge material from mine to burial is the shut down, and the tailings piles from responsibility of the U.S. Nuclear mill operations were abandoned. These And that is where Randall Cygan and Regulatory Commission (NRC), sites presented a potential long-term Jeffery Greathouse, of Sandia National which is charged with preventing health hazard because they contained Laboratories’ Geochemistry Department, radioactive material from entering low-level radioactive and other hazardous enter the picture. Cygan, a distinguished >> Combustion the environment. substances that migrated to surrounding member of the technical staff, heads a soil, ground water, and surface water. computational sciences project aimed One of the NRC’s major concerns Furthermore, the piles often emitted at determining how well minerals in >> Geochemistry is the potential for radioactive wastes radon gas. The tailings and other the environment, especially in clay, to leach into ground water, thereby contaminated material were also used can adsorb radioactive material. His affecting plants, animals and humans. as fill dirt or incorporated into various group has focused their molecular >> Computational Performance Such leaching has already taken place construction materials at thousands simulations on clay minerals because of at some abandoned uranium mine of offsite locations.” the exceptional ability of these materials sites, primarily in the Western to “sponge up” radioactive substances >> Advanced Scientific Computing United States. Responding to public concern that have dissolved in water, and about these sites, Congress passed because clay minerals are commonly Snapshot from a uranyl-clay adsorption simulation. SiO4 and AlO6 groups in the two clay The process by which this radioactive the Uranium Mill Tailings Radiation found in soils around abandoned mines. “A lot of waste was just dumped into sheets are shown as tetrahedral and octahedral, respectively. Negative charge sites within >> Fusion Energy waste has entered the environment is Control Act in 1978. It called on the Their particular charge was to help waste hills at the Naturita mill plant the clay sheets are shown as green MgO6 octahedra. The aqueous regions are composed of described in a U.S. Department of DOE to “stabilize, dispose of, and determine how to deal with radioactive that was abandoned in the ‘60s,” says water molecules (red/white), sodium ions (purple), and uranyl ions (red/blue). >> Thermonuclear Energy (DOE) document entitled, control uranium mill tailings and waste from one of the 24 sites covered Cygan. And though some subsequent Applications “Uranium Mill Tailings Remedial Action” other contaminated material at 24 by the Congressional Radiation Control cleanup efforts did take place, some (web.em.doe.gov/bemr96/umtra.html). uranium mill processing sites and Act — this one a mining site near of the uranium leached into the >> Computational It states, “During the 1950s and 60s, approximately 5,200 associated Naturita, a small town in the San Juan ground water. Cygan characterizes Biosciences private firms processed most uranium vicinity properties.” Responsibility Mountains of southwestern Colorado. the radioactive concentrations as ore mined in the United States for for regulating and general oversight “not huge, but enough to be of And while their models have focused on 10,000 Years >> Computational the Atomic Energy Commission, a of the cleanup fell to the NRC. concern,” especially locally, where Naturita, their work is also applicable Tool Kit predecessor of the Department of the concentrations are quite high to any other site where nuclear waste Current U.S. policy calls for the disposal and people have been instructed is either a real or a potential problem. of spent nuclear wastes at one of two not to use well water. As a result, the work is part of a much sites. Since 1999, defense wastes have larger project conducted by the U.S. been deposited at the Waste Isolation Cygan began the computer modeling Geological Survey and the NRC to Pilot Plant (WIPP) in southern New [Cygan’s] group has focused their molecular simulations work on the problem in the mid-1990s, determine how to limit the spread of Mexico, near Carlsbad. The second on clay minerals because of the exceptional ability of with Greathouse joining the team radioactive waste in groundwater not only site, known as the Yucca Mountain in 2004. They are approaching the at abandoned mill sites, but at nuclear site, located in southern Nevada and these materials to “sponge up” radioactive substances completion of their assigned task. waste storage facilities as well, should intended primarily to accept commercial there ever be a breach in containment. waste from nuclear power plants, is still that have dissolved in water, and because clay minerals undergoing excavation and has been are commonly found in soils around abandoned mines. the source of considerable controversy.

18 215

Hand specimen of Map of groundwater wells, elevation contours, and carnotite uranium ore, a hydrologic prediction of uranium concentrations in hydrated potassium uranyl the alluvial aquifer at Naturita, Colorado. From Curtis vandadate mineral. (2005); U.S. Nuclear Regulatory Commission.

3– Carbonate-containing uranyl species such as [Na(UO2)(CO3)3] (left) 0 and [(UO2)3(CO3)3(H2O)6] (right) prevent uranyl ions from adsorbing onto clays. These carbonate complexes are prevalent at high pH conditions. Electron microscopic image of a natural clay sample (montmorillonite) Guidelines call for these sites to be A Price To Pay Eliminating Variables are best- and worst-case scenarios that In all, there are four scientists on showing the very fine designed to safely contain radioactive enable the NRC hydrologists to consider the clay modeling team who interact grain size (less than RANDALL T. CYGAN wastes for a period of 10,000 years The specific purpose of the computer To overcome this hurdle, the Sandia the full range of what might happen in with other groups within Sandia, as a micron). without a breach. To ensure that modeling is to determine how various team precisely controls and constrains the real world. As Greathouse puts it, well as with other DOE laboratories Randall Cygan received his Ph.D. degree in geochemistry and these waste depositories measure up kinds of clays will sponge up a solution the input data for their computer “We are trying to bracket what can be and researchers at several universities. mineralogy in 1983 from Pennsylvania State University where his to that stringent requirement, NRC contaminated with uranium, in the models of clay, thereby eliminating expected in the natural world because One of the Sandia scientists who is not 2+ research emphasized chemical diffusion and dielectric behavior of scientists run detailed analyses known form of uranyl cation (UO2 ). In the little-understood variables that of the complex nature of these soils. in the group but who serves as what silicate minerals. In late 1983, he joined the Geochemistry Department as performance assessments. As part addition to the fact that clay occurs can influence laboratory testing. This Cygan calls “a great resource” is of Sandia National Laboratories in Albuquerque, New Mexico, where of these assessments, the scientists try naturally at Naturita, as well as at other enables the Sandia researchers to “Because these Kd s are not always Marcus Martin, who was a DOE presently he is a distinguished member of the technical staff. He to determine what would happen waste sites, its efficiency at soaking up provide NRC scientists with consistent known as accurately as they need to be Computational Science Graduate also spent two years as an assistant professor in the Geology should their best-laid plans fail and radioactive polluted water makes it information about how various clays known for the performance assessment Fellow from 1997 to 1999 and is Department at the University of Illinois in Urbana, Illinois (1987-1988). a radioactive leak occur. suitable for use as a secondary barrier will react to contaminated ground water models, we are providing a theory to at now a Sandia staff member. to prevent uranium that has leaked exposure. “Clay can adsorb uranium least constrain in terms of bracketing His research interests are varied, including investigations of mineral equilibria, chemical kinetics, surface chemistry of minerals, sorption “Part of the performance assessment from its primary confinement vessels by a couple of different mechanisms, what would be expected in a natural clay. Additionally, Cygan is finding that and dissolution of minerals, shock metamorphism, and atomistic process is to run computer models of from reaching ground water. and our job is to understand how the Our clays in the computer experiment his group is receiving considerable modeling of minerals and geochemical processes. Randall Cygan has ground water flow, expecting the worst chemical process of adsorption occurs are ideal, but we can also zero in on a attention from researchers at other published extensively in various geological, chemical, and materials case, a breach at a waste site where the “Clay is far and away the best natural and what types of clays can better adsorb particular type of clay that might best institutions whose interest is in materials science journals, and has received numerous honors including uranium is leached out into the ground,” material for uranium sorption,” says than other clays,” explains Cygan. represent what might be in the field science rather than environmental membership in Phi Eta Sigma and Phi Kappa Phi honor societies. says Cygan. “The NRC wants to know Cygan. One reason for this, he explains, and use that as a basis for our computer chemistry. This is because clay is He is a Centennial Fellow of the College of Earth and Mineral if we can determine for a 10,000-year is that it is a natural nanomaterial, in In computational chemistry terms, what simulations. We’ve done work on maybe increasingly being incorporated Science at Pennsylvania State University and a Fellow of the period how far that uranium will be that it is very fine-grained, with large the Sandia group provides to the NRC a half-dozen different types of clays,” with polymers to create new materials, Mineralogical Society of America. He is also a member of the transported. Will it rapidly move into total surface area and very small spaces is known as the distribution coefficient, says Greathouse. and the Sandia team’s work provides American Chemical Society, the American Geophysical Union, the Geochemical Society, the Materials Research Society, and the ground water, or will it remain between and within the grains. As a denoted Kd, which is a quantitative substantial predictive capability for the Mineralogical Society of America. localized and not present a problem?” result, very large total surface area is measurement of how much sorption those materials as well. As a result, available to be exposed to passing fluid, occurs. The NRC then uses this Modeling Clay with LAMMPS says team manager Cygan, “People Further Reading: These are complex questions, the which can be soaked up, while the information to develop accurate have been contacting us in terms R. T. Cygan and J. D. Kubicki Molecular Modeling Theory: Applications answers to which are the focus of the extremely small spaces between the performance assessments. The program the Geochemistry team of how to use our methods.” in the Geosciences. Reviews in Mineralogy and Geochemistry, modeling being conducted by Cygan’s grains allow clay to act as a very fine sieve. is using to run their Kd modeling is Mineralogical Society of America, Washington, D.C., 531, 2001. group. And although the answers vary “When we do computer modeling with known as LAMMPS, which is freely from one waste site to another, there But there is one problem with clay as a molecular simulation we know exactly available on Sandia Laboratory’s web site R. T. Cygan, J. J. Liang, and A. G. Kalinichev Molecular models of are certain similarities in results from uranium adsorbing material, in that it the type of clay we’re dealing with. We (www.cs.sandia.gov/~sjplimp/lammps. hydroxide, oxyhydroxide, and clay phases and the development of a general force field. Journal of Physical Chemistry B, 108(4), different sites, such as how certain is extremely difficult, if not impossible, know exactly what the surfaces are like. html). It is written specifically to be 1255-1266, 2004. clays will react when submerged in to predict with certainty precisely We know exactly what the composition run on massively parallel computers. uranium-contaminated water. It is how a given type of clay will adsorb of the clay is and, therefore, we can The clay simulations are run on the J. A. Greathouse and R. T. Cygan Molecular dynamics simulation of these similarities that make the Sandia radioactive material. In other words, go through a matrix of simulations Geochemistry Department’s uranyl(VI) adsorption equilibria onto an external montmorillonite models universally applicable. clay is unpredictable. As a result, when that provide all of the variables. We own 34 -processor cluster. surface. Physical Chemistry Chemical Physics, 7(20), 3580-3586, 2005. scientists run bench-scale tests to control the variables, then look at

determine how well specific clays do the resulting Kd,” explains Cygan. Contact: soak up radioactive water, the results Randall Cygan can differ greatly from those obtained To make sure they cover the gamut of [email protected] from natural materials found in the all of the variables that may occur in Atomic density profiles from molecular dynamics field. Compounding the problem is nature, the Sandia Geochemistry team simulation. Silicon and oxygen atoms (red and the fact that it is not known why the provides what Jeffery Greathouse black) in the two clay layers are shown along with PRACTICUM COORDINATOR results vary widely. describes as “end member cases,” which sodium (purple) and uranyl (blue). The two shaded Marcus Martin...... [email protected] regions under the uranyl graph represent adsorbed (dark blue) and diffuse (light blue) ions.

JEFFREY VETTER AND THE EIGHT OTHER MEMBERS of the Future Technologies Group he leads at the Department of Energy’s (DOE) Oak Ridge National Laboratory (ORNL) could be considered the Batman and Robins of computational science, in that it is their job to seek out and assist others who can benefit from their help in preparing for the next generation of high-end computing.

As doers of good computational deeds, In addition to working with DOE The DOE is interested in investigating Vetter’s group surveys applications researchers to optimize their applications biological systems for a number of software being used by those within software, the Future Technologies reasons, among them being the DOE’s Office of Science, at ORNL Group is charged with ensuring that development of a greater understanding and at other DOE laboratories, to the DOE always has the world’s most of the role microbes play in energy determine what they may need in advanced computing hardware at its production, and also to learn how >> Combustion terms of computing performance disposal. Toward that end, Vetter’s they can be used to help clean the enhancement. They then help those group is also involved in evaluating environment. Some of the microbes groups adapt their applications so advanced computing technologies being investigated, for example, have >> Geochemistry they can take advantage of the world’s that are currently on the cusp of the capacity to absorb carbon dioxide most powerful computers when they creation, such as optical processing from the atmosphere. Excess carbon come online. and reconfigurable computing, for dioxide in the atmosphere from fossil >> Computational Performance potential use by DOE laboratories. fuel burning is generally considered The Office of Science, to which Vetter’s As part of that process, the Future to be at least partially responsible for group is attached, is one of two main Technologies Group works closely global warming. >> Advanced Scientific Computing branches that make up the DOE, the with the companies developing this second being the National Nuclear new technology to get a jump-start A key part of the Computational Biology Security Administration. The primary for DOE when their products do team’s endeavors has been aimed at mission of the latter branch is to become available. studying a phenomenon called protein >> Fusion Energy Jamison Daniel studies a climate modeling technology in the EVEREST maintain and support the nation’s folding, a self-assembly process that the molecules that make up enzymes, thought, ‘Let’s try starting from the visualization laboratory at Oak Ridge National Laboratory’s Center for >> Thermonuclear nuclear stockpile. The Office of Science proteins undergo so that they can antibodies, and the structural elements hardest case to understand what we Computational Sciences. ORNL’s Visualization Task Group seeks out Applications includes scientists involved in a wide Folding Proteins Faster perform their functions. Little is of our bodies, including bones and need to do to solve some of these projects that might benefit from applying innovative visual data range of other computational science understood about protein folding, muscles, there would be no life. problems.’” Agarwal and his colleagues understanding techniques to scientific data. >> Computational activities, including areas such as Though it was formed less than two and such understanding is essential opted to use computer modeling to Biosciences climate modeling, fusion simulation, years ago, Future Technologies has to learning more about the function But studying the folding process is an try to solve the mysteries of protein materials modeling, and astrophysics already succeeded in helping the of various proteins. Gaining such extremely difficult task, and that is folding because studying the folding >> Computational simulations. This presents a wide Computational Biology team at ORNL knowledge is important because proteins where computer modeling comes in. of actual proteins is at best difficult Tool Kit range of challenges to the Future gear up so that they can make use of perform most of the functions of all As Pratul Agarwal, an associate staff and costly; it is frequently impossible, Technologies computer scientists. two new supercomputers — the Cray living cells — they are the machines that scientist in the Computational Biology since the conditions realizable in the XT3 and IBM’s BlueGene/L. drive life. Without proteins, which are team, puts it, “Protein folding is one laboratory are considerably different of the hardest problems to crack, so we from those found in nature.

As doers of good computational deeds, Vetter’s group surveys applications software being used by those within DOE’s Office of Science, at ORNL and at other DOE laboratories, to determine what they may need in terms of computing performance enhancement.

22 25 to run on the new supercomputers. This Describing the reconfigurable Expanding on his Moore’s Law analogy, for example,” says Vetter. “There are is where Vetter’s Future Technologies computing concept, Vetter explains Vetter says, “One of the motivations over no transistors involved in the actual Group entered the picture. that conventional microprocessors, the past 15 years of computing has operation. Instead, there is a cavity such as those built into garden variety been Moore’s Law. The same theory that these lasers are projected into, “When you write up a simulation on desktops or laptops as well as super- applies to field programmable gate and using analog-to-digital technology parallel processors, the two things you computers, are designed to respond arrays. Our hope is that over the next they can sample the result of those need to worry about are computation to a specific set of instructions. Any three to four years we are going to interactions and multiply a matrix and communications,” says Vetter, application intended to run on a see the same kind of phenomenon in and a vector together.” The result referring to the fact that the many specific type of computer, regardless field programmable gate arrays as we is lightning-fast computing. processors of a supercomputer run in of whether it is a word processing, did in microprocessors ten years ago.” parallel and need to communicate graphic design, or voice recognition Lenslet, an Israeli company, recently with each other. “So we had to go in program, must conform to those The advantage to reconfigurable introduced what it claims to be “the and modify their code, use a different instructions. With reconfigurable computing is that “all the hardware is world’s first commercial optical digital communication mechanism so it would computing, those limitations do working explicitly for one application,” signal processor,” which it says can run scale up.” The end result was that the not apply. Essentially, the hardware says Vetter. The bottom line, then, “1,000 times faster than any known DSP speed at which protein folding can be can be reconfigured to adapt to the is that it is more efficient than the (digital signal processor).” Vetter hopes modeled was increased by almost three program, rather than the other way conventional configuration. And by to obtain one of these processors for times, says Vetter. around, without any parts having to using the hardware more efficiently, his group to evaluate for possible be replaced. This is accomplished the speed of the computations being DOE use. They will also evaluate the As a result, “We’ve been able to make using a combination of special run increases significantly. considerable ramifications of having Particle distribution as a function of radius (CQL3D). Running AMBER significant progress in understanding hardware and software. to adapt or completely redo highly the protein function,” says Agarwal. In intricate software applications to run To do this modeling the biology group addition to being able to model protein The Speed of Light on this completely different type of has been using a software application, folding faster, the increased computing Field Programmable computer, carefully weighing the known as AMBER (Assisted Model power has allowed the biological group Gate Arrays When it comes to optical computing pros and cons. Building with Energy Refinement). to model more complex systems as well. the idea is to use the physics of light, AMBER takes a basic sequence of However, notes Agarwal, the work The types of devices that allow which moves a lot faster than electrons, Though the Future Technologies Group JEFFREY VETTER letter codes and, from these codes, done with AMBER and the new reconfigurable computing are known to perform computations. “Some clever might not have a Batmobile, one can develops a three-dimensional structure supercomputers is not yet being used as field programmable gate arrays folks at a company called Lenslet figured rest assured that if help with a new Jeffrey Vetter is a computer scientist in the Computer Science and of a protein. The computer power on an everyday basis. It was a learning (FPGAs). Though such devices have out how to actually shine lasers onto computing technology or hardware Mathematics Division (CSM) of Oak Ridge National Laboratory needed to accomplish this monumental experience, allowing his group to been in existence for over twenty years, a spatial light modulator that allows configuration is needed, they will (ORNL), where he leads the Future Technologies Group. His task is tremendous. Even using prepare for the future. “We are in their capability is just now reaching the one to multiply a matrix and a vector, be the first on the scene. research interests lay largely in the areas of experimental supercomputers with between 128 the very initial stage, being able to have point that developers can consider software systems and architectures for high-end computing. and 256 processors, Agarwal’s group this technology ready to investigate them for use in scientific computing. Jeff earned his Ph.D. in Computer Science from the Georgia required a full day of computation systems,” he explains. As Vetter puts it, the growth of their Institute of Technology; he joined CSM in 2003. Jeff is also an time to simulate approximately two capability is analogous to Moore’s Adjunct Professor in the College of Computing at Georgia Tech. billionths of a second of protein Law, which states that the number Further Reading: folding, according to Vetter. More Power to the Desktop of transistors that can be built onto T. H. Dunigan, Jr., J. S. Vetter, J. B. White III, P. H. Worley a single integrated circuit will double Performance Evaluation of the Cray X1 Distributed Shared To increase their computing capability, In addition to working with various every eighteen months. In the case of Memory Architecture. IEEE Micro 25(1):30-40, 2005. the computational biologists recently groups to prepare their software for FPGAs, the applicable elements are gained access to the National Leadership next-generation computing, Vetter’s cells rather than transistors. “If you M. Smith, J. S. Vetter, and X. Liang Accelerating Scientific Computing Facility, which enabled them group is also very much involved in have a specific algorithm you might Applications with the SRC-6E Reconfigurable Computer: to use next-generation supercomputers, evaluating and developing computational generate a configuration of these cells Methodologies and Analysis. Proc. 12th Reconfigurable such as the Cray XT3, which has some hardware of the future. that uses them entirely differently than Architectures Workshop (RAW). Denver, 2005. 5,212 processors, and IBM’s BlueGene/L another application that comes along,” J. S. Vetter, B. R. de Supinski, J. May, L. Kissel, and S. Vaidya computer at Argonne National One of the promising areas they says Vetter. Furthermore, these cells Evaluating High Performance Computers. Concurrency and Laboratory, with its 2,048 processors. An are investigating is reconfigurable can be reprogrammed every Computation: Practice and Experience, 2005. even larger BlueGene/L supercomputer computing, which, says Vetter, “involves ten milliseconds. at IBM’s Thomas J. Watson Research the use of devices that you can actually Contact: Center has on the order of 40,960 reprogram the hardware to accelerate Jeff Vetter processors, according to Vetter. computations.” In addition, he continues, [email protected] Being able to use supercomputers with “We are also looking at an optical thousands, rather than hundreds, of processor that actually uses light to Pratul Agarwal (left) and Stewart processors presented great potential, but compute rather than electrical signals.” Dickson (right) discuss the impact of PRACTICUM COORDINATOR along with that came problems as well. The group is also evaluating several types protein vibrations in the enzyme Cyclophilin A (background). Jeff Nichols ...... [email protected] of accelerators that can be plugged Specifically, AMBER, the protein folding into conventional desktop systems program, had to be adapted to enable it to potentially make them ten to twenty times faster. 24 DOE LAB RESEARCH Lawrence Berkeley | Sandia | Oak Ridge | Lawrence Livermore | Brookhaven | Los Alamos | Pacific Northwest | Argonne 275 Simulating an By Karyn Hede Urban Disaster

THE TOXIC PLUME MOVES SILENTLY, emanating from a point near Madison Square Garden and spreading to Penn Station before heading down 33rd Street toward the Empire State Building. Busy commuters go about their business, totally unaware that a deadly gas is swirling around them.

It’s a nightmare scenario that emergency “Urban releases pose new challenges involved in the project are conducting planners hope they never have to face. in predicting flow and spread of releases of inert gases and using those But emergency managers know that hazardous substances,” says Andy results to, among other things, validate planning for the worst is necessary to Wissink, a computer scientist in AUDIM as well as other computer ensure that first responders are ready to the Applied Math group at the models. Emergency management, law handle even this worst-case scenario. In LLNL’s Center for Applied Scientific enforcement, and intelligence personnel >> Combustion such situations, time is of the essence, Computing (CASC). “The fast plume will use the resulting simulations to plan and emergency managers need to know models used operationally at NARAC for, train for and respond to disaster where to deploy disaster teams first. do not adequately account for the scenarios, including a potential terrorist >> Geochemistry presence of buildings. Our goal is to attack or an accident involving release For them, a resource housed at develop an operational urban model of toxic industrial chemicals. Lawrence Livermore National to allow real-time simulations in an >> Computational Performance Laboratory (LLNL), Livermore, urban environment.” “The computational challenge California, offers vital information. was to be able to resolve a complex The National Atmospheric Release Wissink is leading a collaborative urban environment while accurately >> Advanced Scientific Computing Advisory Center, NARAC, is a 24/7 effort between CASC and NARAC representing the building structures,” national resource that provides real-time to develop a fast, accurate simulation says Wissink. predictions of the spread of hazardous capability of atmospheric dispersion >> Fusion Energy materials released into the atmosphere. in urban areas. The overall goal In order to describe the location of NARAC supports the new DHS-led of the project, called the Adaptive buildings, they are placed in a Cartesian An adaptive Cartesian grid around the Madison Interagency Modeling and Atmospheric Urban Dispersion Integrated Model grid, explains Wissink. “The previously- geometry quickly and accurately. The Brian Gunney, as well as atmospheric >> Thermonuclear Square Garden area in midtown Manhattan. Applications Assessment Center (IMAAC). The center (AUDIM), is nothing less than to be used approach used a conforming mesh, key technology, explains Wissink, is to scientists Branko Kosovic, Tina Chow, was developed by the Department able to predict within minutes the path meaning the mesh had to wrap around automatically enhance the fidelity of and Stevens Chan, all from LLNL, are >> Computational of Energy primarily to respond to of a toxic plume released in any city in the buildings. It was a very tedious and the grids where buildings are added. applying these new gridding techniques Biosciences radiological releases, but today’s disaster America. The project is funded by the time-consuming effort to generate the to a computational fluid dynamics code planning has taken a new direction Department of Homeland Security (DHS). grid, and because of this the simulations “The buildings cut through the grid FEM3MP developed over the past >> Computational with the potential threat of a deliberate were limited to on the order of 100 like a cookie cutter,” says Wissink. “With decade at NARAC. The code solves Each program contained elements the Tool Kit release of chemical or biological agents The team chose to model Manhattan buildings. Beyond that it just became the Manhattan simulation, we were able the Navier-Stokes (NS) equations to programmers needed to create their in an urban environment. If the release to support a larger Department of too difficult to do,” he says. to avoid all those tedious gridding resolve the atmospheric physics and has next-generation simulation capability, occurs in or near a city, many millions Homeland Security sponsored project steps, so it enables us to model on both Reynolds Averaged NS (RANS) but none was sufficient on its own. of people may be affected and called the Urban Dispersion Program To solve the gridding problem, Wissink the order of 1,000 buildings. The and Large Eddy Simulation (LES) emergency personnel need to (UDP), a four-year project to measure and his team applied a technology grid generation is done automatically turbulence models. They are also “We needed to both develop new know who is at greatest risk. the potential movement of contaminants called embedded boundary grids, which in only a couple of minutes.” combining this program with two other capabilities within the programs and in and around New York City. Scientists allowed them to resolve very complex programs: Overture, which has tools get them to work together,” says Wissink. The AUDIM programming team, for rapid geometry-to-mesh conversion, which includes programmers Kyle and SAMRAI, which supports parallel Anyone who has ever visited New York Chand, Anders Petersson, Craig computing applications that use City can attest to the gusting winds that Urban releases pose new challenges in predicting Kapfer, and former DOE CSGF fellow adaptive mesh refinement. can whip down narrow city streets. So in flow and spread of hazardous substances. 26 295

ANDREW WISSINK addition to the computing challenges, to set up and solve, to the current The AUDIM team would like to be able “In the longer term we want to try to “Currently, our simulations are limited the team had to get the program to simulations involving 1,000 buildings to include weather data such as wind couple image processing technology to a few cities where we have building Andy Wissink is a computer scientist/math programmer in accurately represent the air turbulence and taking minutes to set up and speed and direction, temperature, and to our simulations,” says Wissink. shape files generated,” says Wissink. the Applied Math group at the Center for Applied Scientific found in an urban environment. hours to solve. air viscosity as “boundary conditions” “LIDAR data is available on almost Computing (CASC). His research interests are in computational that add to the simulation accuracy. To do that, the AUDIM team is pursuing any U.S. city.” fluid dynamics, parallel computing, adaptive mesh refinement, “We are working with the atmospheric In addition, the technique allows “Eventually, we would like to add a collaboration that would add the final and atmospheric modeling. physics group here to get the adaptive users to program higher-resolution chemical reaction terms and data piece of the puzzle to allow them to If that piece is put in place, the AUDIM solver to represent air turbulence,” simulations in areas more likely to from chemical sensors that would take satellite data of an atmospheric project would be close to its goal of Andy earned his Ph.D. in Aerospace Engineering from the University says Wissink. “Actually, that part is be affected. “We can fine-tune the further improve our predictions,” release and automatically convert that delivering real-time urban release of Minnesota in 1997, and joined Lawrence Livermore National an ongoing challenge and we are simulation,” says Wissink. “The adaptivity says Wissink. data into its gridding software. Such simulations that can help provide Laboratory in 1999. He is a member of the Association for Computing still working on it.” provides the way to dial in the resolution technology is available through a better estimates of where contaminants Machinery (ACM) and the Society for Industrial and Applied Mathematics (SIAM). you want. If you need a quick solution They likely won’t know how well their technique called LIDAR, which is a may travel and to enhance emergency The group also collaborated with in 30 minutes we can do that, but we can simulations performed until all the data laser remote-sensing technique that response when time is of the essence. In the past, he has worked with overlapping grids for aerodynamics Marsha Berger, professor of computer also do a higher-resolution simulation are analyzed in late 2006. But that won’t can measure both building outlines calculations and with iterative methods for the solution of PDEs. science, New York University, who for a targeted area in hours, with the stop the group from continuing to and clouds and aerosols from satellite Currently, he is working with adaptive mesh refinement on the pioneered the development of code running on large-scale parallel refine their model. and earth-based stations. SAMRAI project and is the project lead for the Adaptive Urban embedded boundary gridding computer systems. And we can quickly Dispersion Integrated Model (AUDIM) project. methods for flow over geometrically change the resolution in the field. You complex aerospace vehicles. The might not be interested in a section Further Reading: team was able to utilize software of a city not likely to be affected, but A. Wissink Urban Environments Using Adaptive Grids, Society for Industrial and Applied Mathematics (SIAM) Annual Meeting, called Cart3D, which was written you might want a high-resolution New Orleans, LA, July 11-15, 2005. UCRL-PRES-213439. by Prof. Berger and her collaborator simulation on an area likely to be in Dr. Michael Aftosmis at NASA the path of a plume. You could refine A. Wissink, B. Kosovic, K. Chand, T. Chow, B. Gunney, C. Kapfer, Ames Research Center. the area around a plume while leaving S. Chan, M. Berger High-resolution Prediction of Airborne Dispersion the rest of the large computational Around Buildings: The Adaptive Urban Dispersion Integrated The project took 18 months to area with coarser resolution, requiring Model (AUDIM), in Proc. of R&D Partnerships in Homeland develop and has now been tested significantly fewer computational Security, Boston, MA, April 27-28, 2005. UCRL-POST-211537. in two-dimensional simulations using resources than refining the grid globally.” the adaptive mesh method and in A. Wissink, D. Hysom, and R. Hornung Enhancing Scalability of Parallel Structured AMR Calculations, in Proc. 17th ACM three-dimensional simulations without All the team’s preparation has been International Conference on Supercomputing (ICS03), San the adaptive mesh. The final test, in anticipation of a crucial test of their Francisco, CA, June 23-26, 2003, pp. 336-347. UCRL-JC-151791. combining adaptive mesh with simulation software. The AUDIM grid three-dimensional simulation, generation tools, coupled with the Contact: is planned for Fall 2005. FEM3MP flow solver, were used to Andy Wissink predict the dispersion of a tracer gas [email protected] The simulations run on LLNL’s around Madison Square Garden in Linux cluster supercomputer, named midtown Manhattan in support of the “Thunder,” a 19.94 teraflop massively March 2005 DHS UDP field experiments. parallel computer that uses 1,024 They were also used for predictions of PRACTICUM COORDINATOR California Digital 6440 servers running UDP experiments around Rockerfeller Tom Epperly ...... [email protected] on 4,096 Intel Itanium 2 processors, Plaza in August 2005. The experimental and, as of mid-2005, was the third most gas release was measured by portable, powerful computer in North America. battery-powered samplers attached to light poles and on top of surrounding By combining efficiencies in computing buildings, in addition to ground-level scalability and time savings using the gas samplers situated throughout the adaptive mesh framework, the AUDIM test area. Wind speed and direction research team has moved in a mere were measured with temporary two years from simulations involving weather stations in the study area. 100 buildings and taking days to weeks

Concentration contours of a Computational Fluid Dynamics Emergency management, law enforcement, and intelligence personnel (CFD)-based dispersion simulation will use the resulting simulations to plan for, train for and respond around the Times Square area to disaster scenarios, including a potential terrorist attack or an of Manhattan. accident involving release of toxic industrial chemicals. 28 DOE LAB RESEARCH Lawrence Berkeley | Sandia | Oak Ridge | Lawrence Livermore | Brookhaven | Los Alamos | Pacific Northwest | Argonne 315

By Jacob Berkowitz Fusion and Ice

WHAT HAPPENS IF you throw an ice cube into a fire? It’s the kind of question that could easily inspire grade school students to experimentation. Roman Samulyak and Paul Parks are working on a high-tech variation of this question. In their case, the “ice cube” is a tiny, frozen deuterium fuel pellet — colder than any natural thing on earth — that’s launched into a roiling, 100 million degree Celsius radioactive mix.

And the results of this fire-and-ice Park’s computational simulations are A star’s plasma is contained by its As with some existing tokamaks, research will help guide scientists and part of a collaborative effort involving massive gravitational pull. On Earth, ITER will be fueled with solid, frozen, engineers in a decades-old quest — to experimentation, theory and advanced the challenge is to heat the plasma to cylindrical pellets of deuterium-tritium, bring the power of the stars to Earth. computation that is pushing fusion temperatures exceeding that of the Sun’s three to five millimeters (the latter is technology from fantasy to fact. core, and safely and effectively contain about the width of a standard paper Samulyak, a scientist in Brookhaven it by using massive magnetic forces clip) in diameter and length. They’ll >> Combustion National Laboratory’s and an expert on inside donut-shaped experimental be cooled to only 10 degrees above mathematical modeling and scientific Fusion Fundamentals reactors called tokamaks, like the absolute zero. The current plan is to computing, Computational Science D-IIID tokamak at General Atomics. fire about three fuel pellets per second >> Geochemistry Center, and Parks, an internationally “Nowhere in today’s science is into the plasma. The pellets will be recognized pioneer in the field of computational simulation more The combination of heating and fired like bullets through one of two fusion fueling at San Diego-based important than in fusion physics,” containment causes deuterium and semi-circular metal guide tubes >> Computational Performance General Atomics, are bringing their says Parks. “Theory and computation tritium nuclei to collide and fuse. This connected to ITER’s inner plasma combined expertise to bear on one are driving the technology for atomic marriage releases an alpha chamber wall. of today’s greatest engineering and fusion fueling.” particle, which further heats the plasma, >> Advanced Scientific Computing physics challenges, fusion power. They’re and a high-energy neutron whose The big unknown is whether this fueling part of the broad U.S. team that is This is because actually experimenting energy heats the containment vessel system will effectively deliver fuel deep designing and building the fueling with thermonuclear fusion is phenom- and thus can be used to produce enough into the burning plasma core mechanism for the International enally difficult. The heart of fusion steam to make electricity. where the fusion reaction is taking >> Fusion Energy International Tokamak Experimental Reactor (ITER) cutaway. Tokamak Experimental Reactor (ITER). energy is the burning plasma, a very place. ITER’s volume is 40 times larger >> Thermonuclear hot mix of the hydrogen isotopes — and the proposed pellet firing speed While Parks brings world-class fusion Applications ITER, a Department of Energy deuterium and tritium, which are the A Pellet’s Fiery Fate slower — than in current tokamaks theory to the problem, Samulyak’s work (DOE) top priority, is the next step raw materials for fusion energy. Plasma, with pellet injection systems. numerically captures a wild frontier of >> Computational in the decades-long dream of making the fourth form of matter and the most The Sun has enough hydrogen to physics that’s ideal for predicting the Biosciences nuclear fusion energy a source of common form in the Universe, is a burn as fuel for the next six billion “We know how far the icy pellet goes. vapor cloud’s behavior in the plasma. The questions draw on Samulyak’s electricity for homes and industries super-heated gas that is so energized years. By comparison, ITER’s plasma What we need to know now is how far In determining how deep a fuel pellet expertise in mathematically modeling >> Computational (see sidebar: Going for Q). The fusion that an atom’s electrons and nuclei will be a fusion spark. Once a fusion the pellet’s vapor goes,” says Parks, who will penetrate ITER’s plasma, Samulyak complex free flows of material, especially Tool Kit group at DOE’s Oak Ridge National are separated to create a soup of reaction starts, the fuel will burn out in the late 1970s developed the theory says it’s necessary to understand and ones involving phase transitions, Laboratory is fabricating the ITER high-speed positively and negatively in a matter of seconds. This is what describing the pellet’s penetration model several key components: How such as the pellet’s ablation. Some fueling device. And as one part of this charged particles. makes refueling so critical. into the plasma, a theory that’s well quickly does the frozen pellet ablate, computational modeling involves international effort, Samulyak and supported by experimental evidence. changing from solid directly to a gas? solid objects, such as a car chassis, How does the growing neutral (i.e., whose shape doesn’t change. But a The computational simulations under not-yet-ionized) ablation cloud affect free flow model represents a material development by Samulyak and Parks are the bits of still-frozen pellet? And how that is constantly morphing, such as central to characterizing the initial will the ablation cloud be heated and the pellet ablation cloud. Added to this … Samulyak and Park’s computational simulations are conditions — including temperature, thus ionized by the plasma, eventually already difficult computational problem part of a collaborative effort involving experimentation, density and width — that will becoming an integral part of it? Yes, are the phase transitions. They bring ultimately determine the fate of and all of this is taking place in so-called strong discontinuities, theory and advanced computation that is pushing this deuterium-tritium vapor cloud. several milliseconds. boundary areas, such as that between fusion technology from fantasy to fact. 30 COLLABORATORS the ablation cloud and surrounding “Our computational technique is plasma, where critical properties like based on tracking the boundaries of 335 Roman V. Samulyak is a scientist at Brookhaven National Laboratory’s density, temperature and conductivity geometrically complex and evolving Computational Science Center (CSC). His research interests are in the change dramatically. areas. It allows the use of various physics area of fluid dynamics and magnetohydrodynamics, large scale scientific computing, computational accelerator physics, and mathematical models, mathematical approximations, “A fusion plasma code alone couldn’t and numerical solvers in different physics. Currently, he is leading a computational MHD project at the Direct numerical simulation model the entire pellet ablation process,” domains occupied by the solid pellet, Brookhaven National Laboratory and working on the development of of cavitation in the helium jet mathematical models, numerical algorithms, and parallel software for says Samulyak, now in his sixth year at the ablation cloud, and the ambient proposed for the mitigation of the simulation of free surface magnetohydrodynamic flows of conducting Brookhaven, after graduating from the plasma,” says Samulyak. “Running plasma disruptions. Cavitation liquids and weakly ionized plasmas in the presence of phase transitions New Jersey Institute of Technology/ these 3D simulations will require bubbles form in liquid helium in and interaction with intense particle or laser beams. The main feature Rutgers University joint program in substantial computational time.” He’s the vacuum gap behind the high of the numerical approach is the method of front tracking for material applied mathematics in 1999. “A primarily using Brookhaven’s 150-CPU pressure reservoir and nozzle. interfaces, and adaptive grid method for elliptic problems in geometrically single code usually works within some Linux-based clusters for the task. complex domains, and accurate physics models for phase transitions. Quasi steady state The developed code is being used for the study of hydro and MHD mathematical approximations. Fusion pellet ablation channel processes in liquid mercury targets for future accelerators such as plasma codes assume conductivity is in a 5 Tesla tokamak the Neutrino Factory/Muon Collider and Spallation Neutron Source, very high. But the conductivity of the Simulating the Fusion Future magnetic field. Left and in tokamak refueling devices. initial ablation cloud is actually very low.” Numerical simulation of the pellet ablation. Temperature dis- image: temperature “We hadn’t really applied supercomputing tribution and Rayleigh-Taylor instabilities of the front edge of distribution, right image: the ablation cloud are shown. Paul D. Parks is a highly versatile plasma physicist who has So Samulyak is developing the to this problem until relatively recently,” pressure distribution. made significant contributions to a number of diverse theoretical mathematical models, numerical notes Parks. “We don’t yet have a and applied areas connected with fusion and non-fusion research. algorithms and computer codes rigorous 2D numerical model of He is internationally recognized for his pioneering work on pellet necessary to create the first 3D models the ablation cloud.” ablation in tokamak plasmas. He is actively involved with researchers at the National Institute for Fusion Science in Japan on a 2D numerical of pellet ablation in a tokamak. the direction in which the ionized and simulation of pellet ablation. Dr. Parks has applied his strong analytical In 2004, he and Japanese computational ablated pellet material drifts inside the skills to the areas of rf current drive, fast wave antenna/plasma coupling, To do this, he’s building on the scientist Ryuchi Ishizaki published a plasma, this is where the pellet must rf ponderomotive stabilization of MHD instabilities, plasma-surface FronTier code — a very large code paper describing a 2D simulation of be launched from. This necessitates interactions, impurity transport and control, advanced propulsion used for modeling complex physics pellet ablation. Although this 2D model the creation of a curved guide tube concepts, and MHD generators. He helped build the pellet-alpha processes, from astrophysics problems is a step in the right direction, Parks says that carries the pellet on a slower diagnostic program, which yielded the first measurements of energetic to modeling particle accelerators — that it didn’t account for the tokamak’s roller coaster ride from the outer to alpha particles inside TFTR. Since 1996 his more recent contributions in order to model the magnetohydro- intense magnetic fields, which are the inner diameter of the tokamak. have been in the areas of disruption mitigation using liquid jets/killer dynamics of the pellet ablation cloud included in Samulyak’s models. pellets. He has authored and co-authored over 75 papers in the refereed Physical Journals and holds one U.S. patent. in ITER. The code also draws on high But Parks believes that the new associate Zhiliang Xu on modeling performance software for meshing and And this, he says, is why Samulyak’s 2D simulations might significantly clarify of liquid jets of lithium or helium Further Reading: discretization of geometrically complex and 3D simulations are important. There the effectiveness of this approach. It’s that might be used to mitigate plasma R. Samulyak, Y. Prykarpatskyy, Richtmyer-Meshkov instability in liquid domains developed by the Terascale are significant doubts as to whether the known that the magnetic field forces disruption in ITER. The two have GOING FOR Q metal flows: influence of cavitation and magnetic fields, Mathematics Simulation Tools and Technologies presently proposed ITER fueling system make the ablation cloud form a long already determined that these jets and Computers in Simulations, 65 (2004), 431-446. (TSTT) Center, and high performance has enough fire power to propel the tube parallel to the magnetic field lines. would cavitate while crossing the solvers of the Terascale Optimal PDE pellet deep enough into the plasma. This extended ablation tube acts as vacuum barrier between the plasma >> Since the earliest thermonuclear fusion research in the 1950s, two generations of physicists have tried to achieve Q. Dubbed R. Samulyak, T. Lu, Y. Prykarpatskyy, Direct and homogeneous numerical Solvers (TOPS) program as part of The central problem is that ITER’s a pellet shield, since electrons (also and the inner wall of the tokamak approaches to multiphase flows, Lecture Notes in Comp. Sci., 3039 the "magic parameter," Q is the amount of fusion power produced the DOE SciDAC program. inner diameter is too small to contain flowing along the magnetic field lines) and might not be able to effectively (2004), 653-660. Springer-Verlag, Berlin-Heidelberg, (2004). divided by the amount of power put into the system to heat the a straight guide tube. Yet, because of have to pass through the length of the penetrate the plasma. plasma and kick-start a fusion reaction. To this day, Q has always B. Fix, J, Glimm, X. Li, Y. Li, X. Liu, R. Samulyak, Z. Xu, A TSTT integrated cloud. This shielding could provide been less than one. In other words, fusion experiments have FronTier code and its applications in computational fluid physics, the pellet with a few milliseconds of In addition, although a newcomer to required more energy than they’ve produced. Journal of Physics: Conf. Series, 16 (2005), 471-475. grace, enough for it to drift into the plasma problems, like a long line of heart of the plasma and feed the researchers before him, Samulyak is This is why the International Tokamak Experimental Reactor (ITER), to P. B. Parks and L. R. Baylor Effect of Parallel Flows and Toroidicity on fusion “fire”. already looking to the big picture — be built at Cadarache, France, is the focus of such intense international Cross-Field Transport of Pellet Ablation Matter in Tokamak Plasmas, tempted by the allure of helping effort. ITER’s ultimate objective is to finally achieve a Q of 10 or more. Phys. Rev. Letters 94 (2005) 125002. “What we want to know is how good harness a source of nearly infinite "It’s hard to believe that we still haven’t gotten more fusion power a shield this tubular cloud is, and power. He’s begun collaborating R. Ishizaki and P. B. Parks et al., Two-Dimensional Simulation of Pellet out than auxiliary power put in," says General Atomics theoretical Ablation with Atomic Processes, Physics of Plasmas 11 (2004) 4064. our simulations could provide this with researchers of Princeton Plasma physicist Paul Parks. For example, Princeton’s tokamak TFTR, which information,” says Parks. Physics Laboratory to integrate the completed experiments in 1997, reached a Q of 0.3 — for about a P. B. Parks, T. E. Cowan, R.B. Stephens, and E.M. Campbell, Model pellet-level simulations with large-scale quarter of a second. of Neutron Production Rates from Femtosecond Laser-Cluster And time is now of the essence. plasma models for tokamaks. Interaction, Phys. Rev. A 63 (2001) 063203. Although the final ITER design is still ITER’s initial goal is to achieve ignition, or a Q of greater than one. in flux, it’s possible that the first cutting “For the future, the challenge will be to This is the point at which the fusion reaction products, in the form of Contact: of steel and pouring of concrete for combine the pellet ablation numerical energetic alpha particles, can transfer enough energy to the plasma to sustain the fusion reaction without the input of any additional Roman Samulyak Paul Parks this fusion machine could start in 2006. techniques with tokamak codes for [email protected] [email protected] energy. To get to ignition, scientists use microwaves or neutral high temperature plasmas,” he says. beams to pump megawatts of energy into the plasma and heat Formation of pellet ablation clouds in magnetic fields ranging from 0 While developing the 3D simulations, “Because we’re talking about adding it to the point where the fusion reactions take over. (left image) to 5 Tesla (right image). Temperature distribution in ablation Samulyak also collaborates with to the simulation of the entire device, PRACTICUM COORDINATOR clouds is shown at 2 microseconds after the start of ablation. Brookhaven postdoctoral research not just the separate components.” If ITER can achieve a Q near10, it would pave the way to the next major James Davenport...... [email protected] fusion milestone — a prototype commercial reactor that efficiently converts fusion energy into electricity. 32 DOE LAB RESEARCH Lawrence Berkeley | Sandia | Oak Ridge | Lawrence Livermore | Brookhaven | Los Alamos | Pacific Northwest | Argonne 355 Of Tsunamis, Thermonuclear By Victor D. Chase Explosions, & Asteroids

AS THE VOLCANO ERUPTS and one side of the mountain gives way, huge quantities of rock, tumbling faster than a waterfall, drop into the ocean, creating in the surface of the sea a crater that quickly closes. As the pouring rock continues through the depths to the ocean floor, it creates an underwater landslide that comes to rest several miles away.

The mighty force of the collapsing Science Applications International examining what would happen if an mountain also generates extremely Corporation (SAIC), a DOE contractor, asteroid were to hit the ocean. The powerful waves, otherwise known as a attempted to determine what would similarities in the dramatic impact tsunami, that radiate in all directions, happen to the shores of America were and devastation that could be caused with the shores of the Canary Islands, an enemy to set off an underwater by either event made SAGE the perfect Africa, and Europe in their paths. But nuclear explosion. Working in the code with which to examine the >> Combustion the coastline of the western Atlantic safety of land-locked New Mexico, asteroid question. Because Gisler is Ocean, including North and South the team developed a sophisticated a University of Cambridge trained America, is spared. computer code designed to model astrophysicist, this project was of >> Geochemistry such explosions. Dubbed SAGE, the great interest to him. Such is the scenario should the code models the movement of air, Cumbre Vieja volcano on the island water and rock, and the interactions About a year after joining the group, >> Computational Performance of La Palma in the Canary Islands between them. Doing so takes massive Gisler and his colleagues Bob Weaver, erupt with a resulting collapse of its computing power, which is provided by also of LANL, and Mike Gittings of western flank into the North Atlantic. thousands of linked Linux computers SAIC presented a paper about their >> Advanced Scientific Computing In fact, part of Cumbre Vieja did at LANL. To make the most efficient asteroid work at a tsunami symposium one that caused the devastating It was a Sunday morning when the drop away during a 1949 eruption, use of the computers, SAGE utilizes in Hawaii. The fact that they were December 26, 2004, tsunami in the underwater Lisbon earthquake hit. but came to a halt before falling continuous adaptive mesh refinement, dealing with a phenomenon as exotic Indian Ocean, although his group Because seismology did not exist at >> Fusion Energy into the sea. a process that automatically focuses as an asteroid strike, coupled with did not model that particular event. the time, the precise location of the the power of the computers on those the obvious sophistication of their As with nuclear explosions and quake remains unknown, but it is The projection of what would happen regions in the computational process computing ability, sent a wave of asteroid strikes, erupting volcanoes called the Lisbon quake because that >> Thermonuclear Applications should the side of the volcano hit the where most of the action is taking excitement through their audience and underwater earthquakes unleash is where the most damage occurred. water during its next eruption was place. Without this capability the and generated a good deal of interest tremendous energy and have the Many people attending church were >> Computational developed at the Department of Energy’s modeling would be a much slower, in the tsunami community generally. potential for great devastation, making killed as walls fell on them. The tsunami Biosciences (DOE’s) Los Alamos National Laboratory more laborious process than it is. “As a result, I started getting interested them appropriate fodder for SAGE. that followed washed over coastal (LANL) as part of a computer modeling in the general phenomenon of areas of Portugal, Spain and North >> Computational project being conducted by members tsunamis,” said Gisler. Gisler and his colleagues turned Africa. And, much like the 2004 Tool Kit of the Thermonuclear Applications Bombs to Boulders much of their attention to possible Indian Ocean tsunami, the Lisbon Group, which goes by the James In hindsight, his timing could not have occurrences in the Atlantic Ocean, tsunami was felt on the opposite side Bond-like designation X-2. When Galen Gisler joined X-2 in 2001, been better. He began considering where tsunamis do occur, although of the Atlantic as water surged onto the LANL researchers had turned eruptions of volcanoes, both underwater much less frequently than in the land in the Eastern Caribbean and During the Cold War era, LANL their attention from the study of and on land but near a shore, and Pacific and Indian Oceans. One such along the Florida coast. scientists, working in conjunction with underwater nuclear explosions to underwater earthquakes, such as the event took place on November 1, 1755.

The fact that they were dealing with a phenomenon as exotic as an asteroid strike, coupled with the obvious sophistication of their computing ability, sent a wave of excitement through their

34 audience and generated a good deal of interest in the tsunami community generally. Comparison of an inviscid-slide calculation (top frames) with a plastic-flow slide calculation at 270 seconds after the start of the slide. The plastic-flow slide is significantly retarded in velocity with respect to the inviscid case, and the ripples in the basalt slide (which would represent 375 turbidite layerings) are further apart and suppressed in amplitude. The maximum horizontal velocity of the slide material is 150 m/s in the inviscid case, 130 m/s in the plastic-flow case. The position and velocity of the water wave are very similar in both. Note the strong velocity gradient across the water-air interface at and behind the wave crest in both cases. The Kelvin-Helmholtz instability will very likely cause the wave to degenerate further into high-frequency components.

Density raster plot in a 2-dimensional run for a schematic Cumbre Vieja geometry at a time of 180 seconds after the start of the slide. The reflective region representing the “It Really Hit Me” Short and Long Waves unchanging basement of La Palma is at left in black, the basalt fluid slide material is red, the water is light blue, and One of the X-2 group’s early undertakings Since beginning the tsunami work, The reason for this has to do with dangerous. That’s what we saw in the air is dark blue. Intermediate shades represent the mixing was to study the potential hazards of an Gisler has become convinced that the the different nature of the waves they the Indian Ocean, and that’s what of fluids, in particular the turbidity currents mixing water and eruption of the underwater volcano potential for another event such as produce. Waves resulting from volcanic you can see potentially from a large basalt are readily apparent. The water wave leads the bull-nose aptly named Kick ’Em Jenny, located the Lisbon quake poses a greater eruptions tend to be high, short, and earthquake along the same lines of of the slide material by a small amount; the forward-rushing in the Eastern Caribbean near Grenada. threat to the shores of the Atlantic choppy, much like the sea whipped what happened in 1755.” slide material (with a velocity of 190 meters/second almost matching the wave velocity) continues to pump energy into It was while they were working on than does possible volcanic action. up by a strong wind. They radiate in the wave. The wave height at this time is 1,500 meters, and modeling the underwater landslide This is not to say that a volcano could all directions, and are subject to Are there fault lines under the Atlantic, the wavelength is roughly 50 km. The computational domain that might result if Jenny actually did not cause tsunami havoc, however. having their energy sapped by other in addition to the one that caused the extends to the right to a distance of 120 km. decide to “kick ’em” that the Sumatra forces, such as wind. As a result, they Lisbon quake, that could result in a tsunami occurred in the Indian Ocean. When the Krakatau volcano blew in spend themselves relatively quickly. devastating tsunami? Gisler points to August of 1883 in the Sunda Strait Earthquake-generated waves, on the the Caribbean plate, which is slowly “Up to that time the whole exercise between Java and Sumatra — not far other hand, are long — perhaps 60 sliding under the Atlantic plate in the was intellectual and academic. Then, from the region of the December 2004 miles long — and slow, flow in only Eastern Caribbean as one potential THE WORK OF suddenly over the Christmas holidays earthquake — it produced one of the one direction, and are not readily site. However, he notes, it is not as MODELING DISASTERS GALEN GISLER it really hit me how vitally important largest recorded explosions on Earth. dissipated. People on boats at sea may active as those in the Pacific and this work is to people, to societies,” not even notice such a wave, other Indian Ocean region, known as the Galen Gisler is a technical staff member in the said Gisler. “If it was just an eruption it would not than to feel the boat rise somewhat “Ring of Fire.” >> As dramatic as tsunamis are, whether caused by underwater Thermonuclear Applications group X-2 at Los Alamos have cost the 36,000 lives it did,” said and then fall, as was reported by nuclear explosions, asteroids landing in the ocean, or underwater National Laboratory (LANL) a founding member of volcanoes and earthquakes, modeling them is rather routine business. As soon as he returned from Christmas Gisler. “But the eruption produced a fishermen on the Indian Ocean as As for what would happen to the the ROTSE and RAPTOR collaboration teams. vacation, he called the International collapse of the mountain, which caused the December 2004 tsunami headed Eastern United States should such As Galen Gisler, spokesperson for the X-2 group that is doing just such He served as Principal Investigator for the Earthwatch Tsunami Information Center, near big waves that wiped out whole villages.” for Sumatra and beyond. And as the an underwater earthquake occur, computer modeling at Los Alamos National Laboratory, describes it, expeditions of 1997-1999, and as Co-PI for the Honolulu, and offered the services of He also noted that if Mount St. Helens world has seen, the power of these waves that remains for X-2 to model. In the “The day-to-day work of science is actually pretty tedious. You type expeditions of 2000-2003. These expeditions were his group. Thinking that they would had been near water it too would have is great, and does not easily dissipate. meantime, Gisler cautions against out files that are illegible to any but the trained eye, you stick them Earthwatch Student Challenge Award Program be asked to model the Indian Ocean caused a huge tsunami when it erupted excessive concern. on the server, invoke an executable, and it goes off and runs the thing. Campaigns in Transient Astrophysics at LANL. event, Gisler was surprised when he on May 18, 1980, “because the whole As Gisler explains it, “A landslide from A few days later you come back and look at the results and realize Dr. Gisler received his Ph.D. in Astrophysics from was instead asked to model a potential northern side of the mountain just a volcano is a point source. It will “As scientists, we’re interested in you have done something wrong, so you go back and do it again.” Cambridge University, Cambridge, UK, in 1976. eruption and collapse of Cumbre Vieja. went away.” disperse laterally and the waves fall hazards. It’s important for us to keep For Gisler, the exciting part of the work comes with the intellectual This volcano had become important off with distance faster because they things in proper perspective and do Further Reading: exercise of understanding the physics of what the computer is G. Gisler, R. Weaver, C. Mader, M. Gittings Two- and because the Sumatra tsunami caused But as massive as the potential are propagating out in all directions, real analyses of the risks and dangers telling him, and validating that information. Three-Dimensional Simulations of Asteroid Ocean a good deal of concern about its devastation caused by volcanic whereas a line source pushing in and not try to scare people to get Impacts, Science of Tsunami Hazards, 21, 119, 2003. threat to the east coast of the U.S. tsunamis, such as the one produced one direction doesn’t have that much publicity,” he said. “The difficulty is trying to convince yourself that the numbers that X-2’s modeling showed that the by Krakatau, the destruction is usually dispersion and becomes more are coming out of the machine represent physical reality,” says G. Gisler, R. Weaver, C. Mader, M. Gittings fears were unfounded. not as widespread as is that caused Gisler. “So a tremendous amount of work goes on in the background, Two-Dimensional Simulations of Explosive Eruptions by large underwater earthquakes. trying to make sure that things are valid; that you are not violating of Kick-’em Jenny and other Submarine Volcanos, any fundamental physical assumptions; that the energies that you Proceedings of the Caribbean Tsunami workshop, get out are reasonable and consistent with one another; and that in press, 2005. the code produces results that are consonant with observations of particular events. Contact: Galen Gisler “That is particularly difficult when you are talking about something [email protected] At right, a density raster plot in a horizontal slice through a 3D calculation for that hasn’t happened and are trying to gauge what might happen La Palma, at 500 meters altitude at a time of 209 seconds, and at left, three vertical under circumstances that are extremely uncertain.” slices through the same calculation at the positions indicated. The reflective region representing the island of La Palma is in black, and the original slide position has One thing that is certain, however, is that Gisler and his group will PRACTICUM COORDINATOR been vacated. Deep blue represents the density of air, orange the density of water, continue to examine the uncertain. Their plans for the future include and shades in between indicate mixtures of air and water. The peak of the large- fine-tuning their modeling of a Cumbre Vieja volcano eruption on the Aric Hagberg...... [email protected] amplitude wave is directed towards the SSW, and a following crest is following in island of La Palma in the Canary Islands based on more precise the same direction. A line extended through the peak strikes the northeast coast of data about the sea floor in the area, which they hope to obtain South America. The vertical slices at left (exaggerated 5 times in the vertical direction) from the Spanish government. show that the highest wave is at the southern end of the slide. Note that there is considerable wave energy at this 500 meter level directed towards the other islands They then plan to turn their attention to learning more about the in the Canaries archipelago and toward the African and Spanish mainland, indicating science of underwater earthquakes, such as the devastating the possibility of locally dangerous waves impinging on those shores. Lisbon quake of 1755 and the Sumatra tsunami of 2004. 36 DOE LAB RESEARCH Lawrence Berkeley | Sandia | Oak Ridge | Lawrence Livermore | Brookhaven | Los Alamos | Pacific Northwest | Argonne 395

By Michael Szpir Heavy Metal Blues

THERE ARE TENS OF THOUSANDS of hazardous waste sites in the United States, and the U.S. Environmental Protection Agency has designated more than 1,200 of these as Superfund sites — places so hazardous that the Federal government has dedicated billions of dollars for their remediation. Notable among these is the Hanford Site on the Columbia River in south-central Washington, which is engaged in the world’s largest environmental cleanup.

Established during World War II as notorious offenders, and they tend to Atom by Atom lipopolysaccharide molecule, and it’s the Hanford Engineering Works, be positively charged ions. “These metals the key to the simulation developed the site manufactured the plutonium are soluble, which means that they There are a number of microbial by Straatsma and his group. needed for the first atomic bomb. can make their way into groundwater candidates for the simulation, but Hanford continued to produce or surface waters,” Straatsma says. “If Straatsma and his colleagues chose Over the course of the past six years, weapons-grade plutonium and other we can reduce some of these metals Pseudomonas aeruginosa, a bacterium Straatsma’s team developed a molecular >> Combustion radionuclides well into the 1960s, but (by adding electrons to them), they that is infamous for being an model for the lipopolysaccharide and it also produced more radioactive become insoluble, which immobilizes opportunistic human pathogen (see set up a system that resembles the waste and highly toxic chemicals than them and prevents transport to the sidebar: The Bad Side of Pseudomonas). outer membrane. “It’s a very small >> Geochemistry anyone knew what to do with. Much groundwater or the rivers.” “It’s a ubiquitous microbe that’s piece of the membrane, only a few of the waste is now buried and sealed commonly found in sediments and the square nanometers of its surface, in lined pits or large tanks or even As it happens, certain bacteria are subsurface, so it is environmentally but it’s an enormous computational >> Computational Performance encased in glass…but not all of it. more than happy to contribute their relevant,” says Straatsma. “And because challenge,” he says. electrons to the cause. Straatsma leads of its medical importance, there has A few miles down the road from a team of computational scientists been a lot of research on this bug, That’s because every atom in the >> Advanced Scientific Computing Hanford, T.P. Straatsma, Laboratory who are simulating how the bacterial including the structure of the molecules membrane is described explicitly in Fellow and Associate Division Director membrane responds when it comes in in its membrane. This is the kind of the computer code. Each simulation of Computational Biology and contact with a metal ion or the surface information that we need to do has hundreds of thousands of atoms >> Fusion Energy Bioinformatics at the Pacific Northwest of a mineral. “We’re particularly the simulations.” and every atom has a force field that National Laboratory (PNNL) in interested in microbes that are able to interacts with neighboring atoms. The >> Thermonuclear Richland, Washington, considers the reduce metals by donating electrons Pseudomonas has an external “skin” forces impart a velocity to each atom, Applications problem of cleaning up this nasty to their environment,” he says. The made of two membranes — an outer and the scientists need to calculate legacy of the Cold War. “A lot of the scientific questions Straatsma’s team and an inner — that are separated the effects on each one of them to The exterior of the bacterial membrane was assembled by replication of heavy metals and radioactive isotopes are asking are so fundamental and from each other by a periplasmic understand how the membrane To make matters worse, those a single lipopolysacchride (LPS) molecule, and the interior by replication >> Computational Biosciences that were produced at Hanford and the potential payoff so enormous space. The outer membrane is a responds to ions or mineral surfaces calculations must themselves be of a single phosphatidylethanolamine molecule. Molecular dynamics other Department of Energy (DOE) that the transfer of electrons between heterogeneous bilayer consisting of that approach it from the environment. repeated millions of times. That’s (MD) simulations of the rough LPS membrane of P. aeruginosa were carried out under periodic boundary conditions, such that the membrane >> Computational lands have made their way into the microbes and the environment is lipopolysaccharides on the outside because the simulation evolves in tiny environment, into the subsurface,” the central theme of DOE’s Grand and lipids on the inside. The transfer “In principle this is an N-squared “time steps” as the forces on each atom consists of a periodic double layer of LPS/phospholipid molecules Tool Kit externally exposed to aqueous environments, as illustrated above. says Straatsma. Uranium and Challenge in Biogeochemistry. of electrons to the environment is problem, where N is the number of give it a slight nudge that changes technetium are among the most believed to be mediated by the atoms,” explains Straatsma, “...but the its position and velocity. The forces, approximate methods that we use make velocities and positions must be this nearly linear. Even then, we have a recalculated for each time step. thousand atoms in our system, we have The scientific questions Straatsma’s team are about a million interactions to calculate. “Our simulation time is on the order Of course, we have many times that of nanoseconds (or billionths of a asking are so fundamental and the potential payoff number, which should give you some second), with a half-million time so enormous that the transfer of electrons between idea of the calculations.” steps per nanosecond,” says Straatsma. microbes and the environment is the central theme of DOE’s Grand Challenge in Biogeochemistry. 38 415 T. P. STRAATSMA “That is why we do these simulations they see. On the other hand, if the on massively parallel computers, using interaction is too weak, the microbes T. P. Straatsma is a Laboratory Fellow and Associate Division Director for hundreds, up to maybe a thousand may wash out of the site immediately.” Computational Biology and Bioinformatics at Pacific Northwest National processors simultaneously. It can take Laboratory (PNNL). Dr. Straatsma is an internationally recognized scientist several months to run and analyze a “A second and related finding is that with more than 25 years of experience in the development, efficient membrane simulation, so we are the negative charge on the surface of implementation, and application of advanced modeling and simulation methods as key scientific tools in the computational study of chemical and using new methods to improve the the membrane is extremely dynamic,” biological systems. With an extensive background in quantum mechanics implementation in a massively parallel says Straatsma. “There are regions of and classical mechanics approaches he has enabled and contributed to fashion.” The implementation has been negative charge that readily adapt to technical programs focused on the understanding of the relationship between highly successful. Indeed, the massively anything that comes close. So as a structure, dynamics and function of molecular and biomolecular systems. parallel code that Straatsma and his positive ion approaches, the membrane The unique combination of expertise in electronic structure calculations, colleagues developed, called NWChem, immediately pushes negative charge molecular dynamics simulations, and the evaluation of thermodynamic was recognized by R&D Magazine with toward it to grab the ion,” he says. properties allowed Dr. Straatsma to establish an international scientific an R&D 100 award in 1999. Remarkably, the membrane can reputation for the development of new molecular simulation methodologies change its electrostatic signature to as well as the application of computational modeling and simulation methods to chemical and biological molecular systems. His research accommodate an ion within picoseconds, interests include the development, efficient implementation and application Why Bacteria Love or a few trillionths of a second. of molecular dynamics simulation as a key scientific tool in the study of Dirt and Heavy Metal To study the interaction between microbes and mineral chemical and biochemical systems, complementing analytical theories Straatsma’s team has also determined surfaces, Straatsma and his team are carrying out simulations and experimental studies. His contributions include the development of The membrane simulations have the amount of work it takes to draw the A More Perfect Membrane wonderful thing about working here at computational techniques that provide unique and detailed atomic level of the outer lipopolysaccharide membrane in contact with information that is difficult or impossible to obtain by other methods and produced some critical discoveries. ions into the membrane by calculating PNNL is the availability of resources — mineral fragments. The illustration is of a snapshot out of that enable the understanding of the properties and function of these “One of our biggest findings is that the free energy involved. “It turns out Straatsma eventually hopes to simulate that includes not only the hardware but one of the molecular dynamics simulations used to study systems. He is known for the design of efficient implementations of these there is an intrinsic electrostatic that if you do that calculation for even larger portions of the bacterial other scientists,” says Straatsma. “More the specific atomic interactions that determine the adhesion methods on modern, complex computer architectures, including vector difference, a dipole, across the positive ions, it doesn’t require work,” membrane. That will require not only so than anywhere else I’ve worked, of Pseudomonas aeruginosa with the mineral goethite. processing and massively parallel computer systems. Dr. Straatsma’s current application research focuses on the use of simulation methods to membrane so that it’s always negatively says Straatsma. “In fact, you get more computational power, but also there is the sense of accomplishing study microbial membrane mediated geochemical processes at mineral charged on the outside,” says Straatsma. energy back!” These results provide an more information about the atomic things in a team. You are working with surfaces, protein mediated transport processes across microbial membranes, “It explains why these microbes adsorb atomistic explanation for experimental structures of other membrane a large group of people, not sitting and the development and application of novel methods to study complex to positively charged mineral surfaces.” observations, which show that positive components, such as proteins that play alone in your office, and it makes you enzymatic reaction mechanisms that involve electron or proton transfer steps. Knowing how these bacteria stick to dirt ions are pulled into the membrane. fundamental roles in the microbe’s feel that what you’re doing is really Dr. Straatsma’s expertise in computational chemistry and computational is important if you want to use them “That’s the value of these computational interaction with its environment. Such worthwhile because it’s part of a biology has led him to establish and provide the technical leadership of for bioremediation. As Straatsma puts studies,” notes Straatsma. “You can proteins include cytochromes, which larger mission. It’s a great place to THE BAD SIDE OF PSEUDOMONAS the Computational Biology and Bioinformatics research group at the it, “We need to know the strength of the confirm experiments, or even help are involved in shuttling electrons; work,” he says. Laboratory. He is serving as principal investigator for multi-laboratory interaction between the microbes and to interpret them.” porins, which let water and small ions >> Although bacteria may ultimately come to our rescue, the microbe Pseudomonas and multi-disciplinary research programs, and provides a leadership mineral surfaces because we may not pass through; and transporters such Several groups at PNNL are now role in the definition of hardware and software requirements for data aeruginosa has caused more than its share of grief to the human species. intensive computing applications for complex biological systems. want them clinging to the first thing as the iron-citrate receptor, which studying various microorganisms that transports iron across the membrane. may someday join the work force that’s The bacterium rarely infects uncompromised tissues, but it will readily attack Dr. Straatsma earned a Ph.D. in Mathematics and Natural Sciences cleaning up Hanford and other DOE almost any part of the body that has been weakened by disease or injury. A from the University of Groningen. If the work isn’t done at PNNL, it sites. Biologists and chemists take on the Pseudomonas infection has a frighteningly high mortality rate for patients who are probably won’t be done anywhere experimental work, while environmental hospitalized with cystic fibrosis or burn wounds. Unfortunately, the microbe is Further Reading: naturally resistant to many antibiotics because the lipopolysaccharide molecules R. D. Lins and T. P. Straatsma, Computer simulations of the else. “It turns out that nobody else scientists and engineers are sorting out lipopolysaccharide membrane of Pseudomonas aeruginosa, is simulating membranes that involve how these microbes might be applied in its outer membrane serve as a barrier that prevents the entry of these drugs. Biophysical Journal, 81, 1037-1046 (2001). lipopolysaccharides, because these to the job. Some may find it ironic that Now that Straatsma and his team have developed a model of the Pseudomonas molecules are so large that the these simple life forms will be cleaning membrane, they are also studying ways to combat the microbe’s defense T. A. Soares, J. H. Miller, and T. P. Straatsma, Revisiting the Structural For the appropriate description Flexibility of the Complex p21ras-GTP: The Catalytic Conformation of calculations become computationally up the mess created by some of the mechanisms. “We are looking at how these bugs excrete molecules that are of the intermolecular interactions the Molecular Switch II, Proteins: Structure, Function and very expensive. But we have this big greatest scientific minds in history. harmful to them and how the membrane interacts with antibiotics,” he says. Genetics, 45, 297-312 (2001). within the lipopolysaccharide computer sitting here and so we are But the bacteria have been adapting membrane it is important to able to simulate large molecules,” to changes on (and in) the Earth for The means to achieving this goal may be a simple but important structural have an accurate electrostatic R. M. Shroll, and T. P. Straatsma, Molecular Dynamics Simulations of component of the Pseudomonas outer membrane. Even though there is an ‘signature’ of the molecular says Straatsma. nearly 3.5 billion years. They have an the Goethite-Water Interface. Molecular Simulation, 29, 1-11 (2003). electrostatic gradient across the membrane, the overall structure has a neutral components. The electrostatic ancient relationship with dirt, and it charge. That’s because counter ions, such as calcium (Ca+2), act to balance Contact: potential is shown here of the Adding large proteins into the may be this timeless wisdom we need out the negative charges that are concentrated on the outer surface of the Tjerk Straatsma lipopolysaccharide molecule of membrane models will make the to acquire if we are to maintain our membrane. The counter ions may prove to be the key. [email protected] Pseudomonas aeruginosa that simulations more realistic, but it means own relationship with the planet. was used to fit the partial atomic that there’s still an enormous amount of “It turns out that positively charged antibiotics are able to replace some of the charges used in the molecular research that must be done. Straatsma’s counter ions within the outer membrane. They have a particular molecular shape dynamics simulations of the PRACTICUM COORDINATOR team works very closely with other that disrupts the integrity of the membrane. If the membrane breaks, then the complete membrane. Stephen Elbert ...... [email protected] groups at PNNL who are determining microbe dies,” says Straatsma. The electrostatic properties of the bacterial the structures of the molecules. “The membrane that scientists may exploit for bioremediation might also prove to be the microbe’s Achilles’ heel.

By Karyn Hede PETSc Engines

IF EVERYONE WHO WANTED TO OWN A CAR had to build their own internal combustion engine, imagine how different the world would be. Luckily for us, we had Ford to build the first automobile engine and chassis, while today we just enjoy the ride.

In the world of scientific computing, libraries, each user doesn’t have to are mathematically simple linear solvers the U.S. national laboratories have understand all the details of all the that do not scale very well, but their built the engines and chassis that drive algorithms they are using in order to advantage is they use very little memory,” many of today’s high-performance use them. Trying to do so would just Adams says. “To be able to accurately computational simulations. For the be too overwhelming.” model bone requires very large mesh scientists and engineers, the availability models and people have been pushing >> Combustion of rigorously-tested dependable PETSc has become the go-to source the limits of these simple solvers. computational toolkits means spending for what PETSc user Mark Adams calls Several groups have these problems less time writing software code and “software that does all the heavy lifting.” that they literally run for weeks and >> Geochemistry more time doing science. One such weeks on end on small parallel computational “engine,” unassumingly Adams, a research scientist in applied computers like 32-processor SGIs.” called Portable, Extensible Toolkit math at Columbia University, uses >> Computational Performance for Scientific Computation or PETSc PETSc as the engine for a number of Adams worked with UC-Berkeley (pronounced “pet c”) provides a high-level programs he developed to mechanical engineering graduate library of software for solving partial solve solid-state mechanics problems student Harun Bayraktar to convert >> Advanced Scientific Computing differential equations, the most that use finite element analysis. the problem from a non-scalable common mathematical tool for linear solver to a non-linear, highly modeling relationships between One such problem, a collaboration scalable solver. >> Fusion Energy physical processes. with Tony Keaveny and Panayiotis Papadopoulos of the University of “PETSc provided a substantial amount >> Thermonuclear “If you are working on a particular California - Berkeley’s Orthopaedic of code, on the order of 40,000 lines Applications engineering problem, you might Biomechanics Laboratory, required of code, that is really very general, have to combine many different modeling the strength of the spinal very unstructured, but at the same Adams used PETSc as a foundation Adams and his colleagues presented the >> Computational numerical algorithms and that can vertebra to high resolution. time provides fully functional and to build a solver that would take the work at the SC2004 supercomputing Biosciences get complicated,” says Barry Smith, highly optimized matrix, vector and large finite element meshes and conference and received a Gordon one of PETSc’s early developers at “People working in the area of bone other parallel linear algebra objects,” perform large-scale, non-linear Bell Award, one of high-performance Argonne National Laboratory in modeling have been using what says Adams. “It lets people like me whole bone modeling on high-end computing’s most prestigious honors, >> Computational Tool Kit Illinois. “By encapsulating the are called matrix-free, element-by- concentrate on developing good parallel computers. for the work. commonly used algorithms in element preconditioners, which algorithms to build on top of it.” “The thesis of the research is that “Using this method, we were able to The researchers are hoping these bone density alone is not a good speed up these calculations by literally simulations will give them a better measure of the risk of fracture,” says two orders of magnitude in CPU time understanding of the behavior of Adams. “By understanding how the and about three or four orders of bone tissue that is thinning as a result microstructure of bone degrades, they magnitude in wall clock time,” says of bone loss. Ultimately the goal of are hoping to come up with more PETSc has become the go-to source for what Adams. “We went from seven weeks the research is to be able to take a accurate prediction of who is at risk PETSc user Mark Adams calls “software to about two minutes to solve some CT scan of bone and calculate the for fracture.” that does all the heavy lifting.” of these problems.” risk of fracture.

42 Top panel: a cross section through an experiment from Holtzman et al. (Geochem. Geophys. 455 Geosys. 2003) in which a partially molten aggregate of rock representing the earth’s mantle has been subjected to a shear strain of 370%. After a strain of about 100% the initially uniform For Richard Katz, also a DOE CSGF Armed with the basic knowledge For the novice just thinking about distribution of melt and shear strain have concentrated into bands at a low angle to the shear alum, the attraction of PETSc was its of applying PETSc, Katz set out to trying to use PETSc, Katz recommends plane. Black subvertical cracks occur on quench of experiment and are not modeled. menu of tools to solve a host of different do something no one previously had checking out the library’s extensive problems and its “inherent scalability.” been able to accomplish: modeling collection of example codes. Bottom panel: one timestep from a PETSc-based simulation by Richard Katz (Ph.D. thesis, what happens when the earth’s mantle, Columbia University, 2005) of non-Newtonian two-phase creeping flow. Colors show the “One of the great things about PETSc which is solid, interacts with molten “The typical way for someone to vorticity perturbation induced by a localization instability. Black lines are passive strain is that you can write your application magma and both substances flow. get started is to find an appropriate markers that were initially straight and vertical. and test it on a laptop. Then without (See Katz’s profile in the 2004-2005 example code and modify it to their much thinking you can scale it to issue of DEIXIS.) needs,” says Katz. In addition, he says a supercomputer. For me this was the PETSc developers are very helpful just fantastic.” “The behavior of this two-phase system and responsive to questions about is described by partial differential the library. Katz spent his practicum at Argonne in equations that are highly non-linear. 2003 learning PETSc and applying it In the past this has been very difficult It’s just like driving a car, where you to solve a set of equations that describe to solve,” he said. “PETSc allowed me have to learn how to operate it before BARRY SMITH Adams’ application of PETSc is just “As an open source project, all of the flow of the earth’s mantle over time to simulate a system that no one had starting the ignition, and it certainly what Smith had in mind when he and the developers and many users have in subduction zones, regions deep been able to model.” helps if you can check the fluid levels Barry Smith is a Computer Scientist in the Argonne co-developer William Gropp contributed valuable design ideas as within the earth where the edge of and change a tire. Likewise, Katz says it Mathematics and Computer Science (MCS) Division set out in the early 1990s to develop well as the actual source code. It has one of the earth’s tectonic plates is In addition to using PETSc, Katz’ is important to have some background at Argonne National Laboratory. His research interests include the scalable solution of algebraic equations a set of numerical tools that would evolved a great deal to match the forced under an adjacent plate. research has led to an extension that in software and be able to do a little and their use in PDE simulations. make the most out of the tremendous needs of our users,” Smith notes. will be added to the PETSc numerical bit of coding before trying to use PETSc. gains that had been made in processing “The code that comprises PETSc library. The method, called a semi- But, he adds, once you get started the Barry obtained his B.S. in mathematics from Yale speed in high-end scientific computing. Former DOE CSGF fellow Allison is highly structured and very clearly Lagrangian advection scheme, is a means library has a lot of documentation to University and received his Ph.D. from New York The project, which was sponsored by Baker, now a post-doctoral fellow written,” says Katz. “It has a very of tracking the advective movement help solve that next big problem. University. His interest in scientific computing was DOE, has had nine developers over at Lawrence Livermore National disciplined interface and so it teaches of particles in a flow. “Another great sparked in an undergraduate numerical analysis class taught by Bill Gropp, who is currently the Associate the years and currently keeps four Laboratory, spent a summer at you to write good code. I learned a feature of PETSc is its extensibility. Division Director and Senior Computer Scientist in developers busy full-time. Argonne in 1999 learning PETSc lot about writing software just by You can build your own component the MCS Division at Argonne. and implementing an additional using PETSc.” if you don’t find what you need.” “What we saw was a lack of flexible linear solver routine into PETSc. Barry’s current main interest is in the efficient software for high-performance translation of results in academic numerical analysis computing,” says Smith. “We set out “Spending that summer at Argonne to the scientific and engineering end users. to create software that was designed really helped me see how important it Further Reading: for solving partial differential equations, is to have good scientific code publicly B. Smith, P. Bjorstad, and W. Gropp Domain which can be used to represent the available,” says Baker. “It really makes Decomposition: Parallel Multilevel Methods for relationships between physical quantities life a whole lot easier.” Elliptic Partial Differential Equations Cambridge such as pressure, velocity, and force. University Press. These are continuous processes, but Her summer learning experience set T. Bittner, M. Donnelly, B. Smith Endurants and in order to solve them in a computer the stage for Baker’s thesis work at Perdurants in Directly Depicting Ontologies in you always have to represent them the University of Colorado where she AI Communications, Vol. 17, Issue 4, October 2004. in a discrete way. The process of developed a new linear solver algorithm discretization results in algebraic that reduces the movement of data A. Kumar, B. Smith, and D. Novotny Biomedical equations which you can then solve through machine memory. Baker used Informatics and Granularity: Conference Papers in using PETSc.” Comparative and Functional Genomics, Vol. 5, Issue PETSc as the backbone of her solver, 5-6, August 2004. allowing her to move more quickly into Its first users were scientists at Argonne the heart of the research question. Contact: who were modeling fluid dynamics, Barry Smith but in the ten years since its release its “My experience with learning PETSc [email protected] reputation has spread far and wide in showed me that a good implementation the computer science community. Users can be ten times or a hundred times Rendering of a human vertebral bone now include geochemists, biologists, faster than a naïve one,” says Baker. cylindrical specimen and detail PRACTICUM COORDINATOR physicists and environmental engineers, “I just really learned a lot about how showing voxel finite elements. Raymond Bair...... [email protected] among others. Since the software works important it is to think carefully fairly seamlessly with any hardware, about implementation details and except vector machines such as the what a big difference it can make. CRAY X1, it has a large fan base These are just things you don’t learn among a substantial contingent about in class or in textbooks.” of scientists.

44 ALUMNI PROFILE ALUMNI PROFILE 47 CHRIS OEHMEN JOEL PARRIOTT Chris Oehmen Joel Parriott

In Edgar Allen Poe’s FAMOUS SHORT STORY the Tell-Tale Heart, the main character is tormented by Joel Parriott’s Ph.D. is in COMPUTATIONAL ASTROPHYSICS. But today he’s working to the relentless, increasingly loud beating of his murder victim’s heart. Chris Oehmen knows as much as anybody about understand and help others grasp the forces at play not within the Milky Way but rather within the Washington D.C. seemingly dead hearts that keep beating. Beltway. And he’s a force himself.

is doctoral research provided Oehmen, a student steeped in math First, he’s applied the Global Array code, he annual revolution of the “When you’ve committed your life to an examiners with a science Ph.D. evidence for a previously and physics, two crucial things about developed by a team led by PNNL’s federal budget cycle sees Parriott academic pursuit, you have a hard time among offices of MBAs and public Hunknown mechanism that biology: it involves an enormous level of Jarek Nieplocha, to share the genome T— Program Examiner in the believing that somebody else wouldn’t policy graduates. After receiving his drives the heart’s pacemaker, a complexity; and computational science database on a high-performance Office of Management and Budget share your excitement and your doctorate in 1998, Parriot honed his remarkable bio-clock that will keep on is indispensable to understanding parallel platform. (OMB) responsible for the DOE’s conviction that this is the best way to policy smarts as a program officer on ticking even when a heart is removed this complexity. Office of Science — scrambling each spend money,” he notes. “So people say the National Research Council’s Board and suspended in a blood-like substance. Second, he’s adapted BLAST, one of autumn to provide advice as that ‘We could do my research if you could on Physics and Astronomy. It was here But these days, the heart this DOE “What we showed about the sinoatrial the codes most widely used by scientists year’s budget request makes its way build just one less tank.’ But that’s just that he developed a keen sense for the CSGF alumnus hears beating in his node could only be done with around the world for sequence through the Executive Office of the not the way decisions are made here.” push and pull of science politics through ears is his own. Just over the two-year modeling,” says Oehmen of his comparisons, for use on a parallel President and ultimately to Congress. committees, reports and lobbying. mark as a scientist at DOE’s Pacific Ph.D. research in the Joint Graduate multiprocessor architecture. It’s Parriott’s job to provide the informed Northwest National Laboratory (PNNL) Program in Biomedical Engineering To those physicists, chemists and other technical recommendations for the When the opening at OMB came three in Richland, Washington, Oehmen is program at the University of Memphis In initial trials using real data from scientists he meets on visits to FermiLab, budget decisions that are made, years ago, Parriott says he jumped at the captivated by his role in helping push and the University of Tennessee PNNL colleague Heidi Sofia, Oehmen’s the Stanford Linear Accelerator Center ensuring that they reflect the President’s chance to put his scientific and policy the boundaries of biology, in silico. Health Science Center. ScalaBLAST version showed it’s the and other DOE facilities, Parriott is policy priorities, increase management know-how to work in a career position Indy 500 version of the original. tantalizingly close to the purse strings efficiency and ultimately result in the that emphasized technical advice on “It’s almost intoxicating. I’m still Now Oehmen is parlaying this for their beloved projects. It’s Parriott’s biggest research bang for U.S. citizens’ complex and varied scientific issues. adjusting to it,” says Oehmen, a member experience into helping biologists “Overnight I was able to turn around job to make recommendations on a tax dollars. Today these issues range from the U.S. of PNNL’s 25-person Bioinformatics maximize their use of computation. 16 full genomes with about 64,000 genes, portfolio of $3.5 billion. But during the role in the international fusion energy and Computational Biology Group. whereas that would have taken her a presentations he gives about his role This means he asks the tough questions project ITER and international “In grad school you get really focused Biologists are presently faced with a data month and held down her machine and that of OMB in the Executive he didn’t dream of positing while doing competitiveness in high-performance on solving your problem. Here I’ve conundrum: they have too much of it. so she couldn’t do anything else with Office, his first PowerPoint slide his doctoral dissertation as a DOE CSGF computing, to DOE’s role in biological been exposed to so many problems The genome sequencing of the past it,” says Oehmen. shows him on the organization’s Fellow at the University of Michigan. research, and the future of Big Science that have huge possibility for impact.” 20 years on everything from worms to bottom rung — five levels from the What’s the real value of new investment projects such as particle accelerators. humans has produced gigabytes of While he revels in helping supercharge ear of President George W. Bush. in supercomputers? Are there other, Oehmen’s graduate research sequence databases. Together, these the genomics and proteomics revolution, more cost-effective ways of addressing Says Parriott, “The thing I like about demonstrated the role that cellular-level databases enable crucial pair-wise Oehmen doesn’t stray far from things “In my presentations I’m trying to a scientific problem? my job is that I have to understand the electrical resistance plays in regulating alignment studies — the comparison of the heart. He’s usually home by educate folks about how decisions are politics, but I’m explicitly forbidden, the pacemaker’s, or sinoatrial node’s, of newly discovered gene or protein 5:00 p.m. to help feed, bathe and read made and what they might do to “This job has totally changed my thankfully, from playing any of electrical wave that excites the heart sequences with known ones to help to his six- and two-year-old children. improve the case they’re trying to thinking about science,” says Parriott, the politics.” muscle to contract. It also showed identify the role of the gene or protein. make for more resources,” says who nonetheless sees himself as a But these databases are now so large “I don’t want to be a dad that only shows Parriott from his office across the card-carrying member of the scientific that they’re a bottleneck in the process. up at home periodically and works street from the White House. community. “I know the limits of 80-hour weeks,” he says, noting that computation. I’m trying to push scientific Parriott notes that while he’s now “The target databases are increasing the Battelle Corporation which manages A key part of this decision making, computing to areas where there’s an odd duck in the world of science Chris Oehmen knows as much like crazy,” observes Oehmen. “They’re PNNL is an excellent employer when says Parriott, is the reality of hard predictive capability. We need to be able graduates, he’s also an outsider of as anybody about seemingly dead growing so fast they soon won’t fit on it comes to employees balancing their trade-offs. And this, he says, is a to run simulations and get results that a single computer and this is creating work and family roles. difficult reality for many scientists to drive what we’re doing experimentally.” sorts in the OMB — one of the few hearts that keep beating. huge performance issues.” face and consequently a stumbling Says Oehmen, “If you do things the block when they come to make the Parriott notes that while he’s now an odd program examiners with a science Oehmen has created two data-intensive right way, I believe you can find a way funding pitch for their projects. duck in the world of science graduates, Ph.D. among offices of MBAs. computing techniques that hold great to work smarter not harder.” he’s also an outsider of sorts in the promise in breaking this data logjam. OMB — one of the few program 46 Dr. Judith Hill ALUMNI PROFILE 49 Dr. Ryan Elliot MAYYA TOKMAN Mayya Tokman Howes Scholars

Mayya Tokman STUDIES THE SUN, but she’s not blinded to questions here on Earth. While her research is helping discover the cause of solar storms, she’s also helping shape U.S. foreign policy through science and education THE FREDERICK A. HOWES SCHOLAR exchanges and is about to embark on a journey to build a unique interdisciplinary applied mathematics program. in Computational Science award was established in 2001 to honor the late Frederick Anthony Howes who was a champion for computational science education.

Before his death, Dr. Howes was the year as a Howes Scholar. Candidates fellow from 1999-2003, graduated “ ’ve always been interested in the Through initial collaborations with Most recently, she’s extended this work program manager for the Applied are chosen based on their academic from Carnegie Mellon University in broad impact of science because Paul Dellan, a Caltech experimental to propose a new theory to explain Mathematical Sciences (AMS) excellence, leadership and character 2004 with a Ph.D. in Computational Isometimes you publish a paper physicist interested in replicating the trigger that turns a staid coronal Program in the Department of and are nominated by their academic Science and Engineering. Dr. Elliott Judith Hill discusses her but you don’t see the big picture of astrophysical phenomena on Earth, loop into a cataclysmic solar flare or Energy’s Mathematical, Information advisors. The honor provides the was in the Fellowship program from current research projects science and society,” says Tokman, Tokman (who also has a deep larger CME. and Computational Sciences (MICS) recipients with a substantial cash 2000-2004 and received his Ph.D. with the attendees at the who for the past three years has been fascination with all things astrophysical) Division. He had held that position award, a Tiffany crystal paperweight, in Aero Engineering and Scientific annual DOE CSGF fellows a visiting assistant professor in the found her niche in the mathematical Recently returned from the American for eight years. Dr. Howes was highly and the distinction of being named a Computing from the University of conference. mathematics department at the modeling of the behavior of space Geophysical Union Conference, respected by his peers and admired Howes Scholar. Michigan in 2004. University of California, Berkeley. and laboratory plasmas. Plasma, a Tokman says observations are providing for his energy, dedication and super-hot soup of charged particles, is evidence that supports her model. personal integrity. A DOE CSGF fellow is eligible to be Dr. Hill is currently a Post-Doctoral Tokman learned early and personally the fourth state of matter, along with named the Howes Scholar if he or she Research Associate at Carnegie how small, incremental changes barely solids, liquids and gases. "Most of the solar flaring that’s being One of Howes’ responsibilities was to has completed all the requirements Mellon University, and Dr. Elliott perceptible on the surface of things observed is coming from boundary oversee the Department of Energy’s for his or her Ph.D. program while is working as an Assistant Professor can lead to enormous changes in the Her current research is focused on zones within the coronal active Computational Science Graduate being supported by the fellowship in the Department of Aerospace big picture. In 1991, the 19-year old modeling the behavior of plasma in regions. And this fits exactly with what Fellowship (DOE CSGF) program. program or having been supported Engineering and Mechanics at the Tokman arrived in the U.S. with her the solar atmosphere that results in my model predicts and has not been Without his support and dedication, by the program for the maximum University of Minnesota. parents as a Jewish refugee from her coronal mass ejections (CMEs). Also understood before," says Tokman. the fellowship program may have not number of allotted years. native Azerbaijan, just two weeks before known as solar storms, these are the survived. Currently, the program is Both award recipients were on hand the fall of the Berlin Wall and the massive eruptions of billions of tons But this year, Tokman’s solar modeling considered one of the most prestigious at the DOE CSGF annual fellows’ collapse of the Soviet Union. Her first of plasma into space. This stream of is on the back burner. She’s turning her computational science fellowships in 2005 Scholars conference where they presented academic appointment was as a financial charged particles creates the beautiful attention to creating models of good the country, and it is a tribute to his their research and received their aid worker at Santa Monica Community dancing Northern and Southern Lights scientific relations here on Earth as energy and passion. An exceptional list of nominees awards. David Brown from Lawrence College, where she “learned English very in the Earth’s upper atmosphere. But an American Association for the prompted the selection of two awards Livermore National Laboratory fast” and helped other Russian speakers. it can also wreak havoc on satellites, Advancement of Science Diplomacy To honor his memory and his this year. The winners were Dr. Judith presented the awards. communication systems and power Fellow in the State Department. Based dedication to the DOE CSGF Hill of Carnegie Mellon University Her zeal for math soon took her to grids. For this reason, the ability to in Washington, D.C., she’s helping program, one or two DOE CSGF and Dr. Ryan Elliott of the University For more information on this UCLA to complete her undergraduate predict these eruptions would be a improve U.S. relations with other fellows are chosen each calendar of Minnesota. Dr. Hill, a DOE CSGF program, contact the Krell Institute degree and then to Caltech where, major scientific coup. countries through programs in science at 515.956.3696 or email Rachel as a DOE CSGF Fellow, she earned a and math education. Huisman at [email protected]. Ph.D. in applied and computational CMEs start as coronal loops — massive mathematics. arches up to 300,000 miles high of And in September 2006, she’ll begin a magnetically confined streams of plasma tenure track appointment as a founding Ryan Elliott presents his in the sun’s atmosphere, or corona, member of the mathematics department research to the attendees that occur in groups called coronal at the new Merced campus of the at the annual DOE CSGF active regions. Tokman’s doctoral University of California. Tokman says Tokman learned early and personally fellows conference. research used computational simulations it’s her dream job — to be part of an how small, incremental changes to demonstrate that the structure of energetic, visionary faculty at a campus barely perceptible on the surface of the plasma eruptions is in part defined founded on an interdisciplinary model. by the rotation of the two footpoints things can lead to enormous changes (or sunspots) where the coronal loop She knows it will take time, but there’s David Brown of Lawrence Livermore in the big picture. joins the sun’s surface. also the knowledge that with incremental National Laboratory presents awards steps there’s room for big changes. to the two 2005 Howes Scholars, Ryan Elliott and Judith Hill.

48 ALUMNI 51 Directory

Nathan Crane E Gregory Ford University of Illinois University of Illinois Civil Engineering Ryan Elliott Chemical Engineering Alumni Directory Fellowship Years: 1999-2002 University of Michigan Fellowship Years: 1993-1995 Current Status: Sandia National Aerospace Engineering Laboratories – New Mexico Fellowship Years: 2000-2004 Oliver Fringer Current Status: Faculty, Stanford University A Martin Bazant Edward Chao Stephen Cronen-Townsend University of Minnesota Environmental Fluid Mechanics Harvard University Princeton University Cornell University Fellowship Years: 1997-2001 Marcelo Alvarez Physics Plasma Physics Computational Materials Physics Thomas Epperly Current Status: Faculty, University of Texas Fellowship Years: 1992-1996 Fellowship Years: 1992-1995 Fellowship Years: 1991-1995 University of Wisconsin – Madison Stanford University Astrophysics Current Status: Faculty, MIT Current Status: TomoTherapy Current Status: Esko-Graphics Chemical Engineering Fellowship Years: 2001-2005 Fellowship Years: 1991-1995 G Current Status: Student, Bonnie Carpenter Beyer Jarrod Chapman Robert Cruise Current Status: Lawrence Livermore University of Texas University of Illinois University of California – Berkeley Indiana University National Laboratory Kenneth Gage Mechanical Engineering Computational Biology Physics University of Pittsburgh Asohan Amarasingham Fellowship Years: 1991-1995 Fellowship Years: 1999-2003 Fellowship Years: 1997-2001 Annette Evangelisti Chemical Engineering Brown University Current Status: Rockwell Collins Current Status: DOE Joint University of New Mexico Fellowship Years: 1998-2002 Cognitive Science Genome Institute Joseph Czyzyk Computational Molecular Biology Current Status: Student, Fellowship Years: 1998-2002 Edwin Blosch Northwestern University Fellowship Years: 2001-2005 University of Pittsburgh Current Status: Staff, University University of Florida Eric Charlton Industrial Engineering Current Status: Student, of Jaffna, Sri Lanka Aerospace Engineering University of Michigan Fellowship Years: 1991-1994 University of New Mexico Nouvelle Gebhart Fellowship Years: 1991-1994 Aerospace Engineering University of New Mexico Kristopher Andersen Current Status: ESI-CFD Inc. Fellowship Years: 1992-1996 D F Chemistry University of California – Davis Current Status: Lockheed Martin Fellowship Years: 2001-2003 Physics Dean Brederson William Daughton Matthew Fago A Fellowship Years: 2001-2005 University of Utah Michael Chiu Massachusetts Institute of Technology California Institute of Technology Sommer Gentry Current Status: Naval Computer Science Massachusetts Institute of Technology Plasma Physics Aeronautical Engineering Massachusetts Institute of Technology Research Laboratory Fellowship Years: 1996-1998 Mechanical Engineering Fellowship Years: 1992-1996 Fellowship Years: 2000-2003 Optimization/Control Theory Current Status: Faculty, Current Status: LC Wright Fellowship Years: 2001-2005 B Fellowship Years: 1992-1996 C Matthew Anderson Paul Bunch Current Status: Teradyne University of Iowa Current Status: Faculty, University of Texas Purdue University Michael Falk United States Naval Academy Physics Chemical Engineering Kevin Chu Mark DiBattista University of California – Santa Barbara C Fellowship Years: 2000-2004 Fellowship Years: 1994-1997 Massachusetts Institute of Technology Columbia University Physics Charles Gerlach D Current Status: Staff, Applied Mathematics Computational Fluid Dynamics Fellowship Years: 1995-1998 Northwestern University Louisiana State University Jeffery Butera Fellowship Years: 2002-2005 Fellowship Years: 1992-1994 Current Status: Faculty, Mechanical Engineering North Carolina State University Current Status: Staff, University of Michigan Fellowship Years: 1995-1999 B Mathematics Princeton University John Dolbow Current Status: Network E Fellowship Years: 1993-1997 Northwestern University Matthew Farthing Computing Services, Inc. Allison Baker Current Status: Staff, Hampshire College Joshua Coe Theoretical and Applied Mechanics University of North Carolina University of Colorado University of Illinois Fellowship Years: 1997-1999 Environmental Science & Engineering Timothy Germann Applied Mathematics C Chemical Physics Current Status: Faculty, Duke University Fellowship Years: 1997-2001 Harvard University F Fellowship Years: 1999-2003 Fellowship Years: 2001-2002 Current Status: Staff, University Physical Chemistry Current Status: Lawrence Brandoch Calef Current Status: Student, Brian Dumont of North Carolina Fellowship Years: 1992-1995 Livermore National Laboratory University of Illinois University of Michigan Current Status: Los Alamos University of California – Berkeley G Applied Mathematics Aerospace Engineering Michael Feldmann National Laboratory Devin Balkcom Fellowship Years: 1996-2000 Ken Comer Fellowship Years: 1994 California Institute of Technology Carnegie Mellon University Current Status: Boeing North Carolina State University Current Status: Airflow Computational Chemistry Christopher Gesh Robotics Mechanical Engineering Sciences Corporation Fellowship Years: 1999-2002 Texas A&M University Fellowship Years: 2000-2004 Patrick Canupp Fellowship Years: 1991-1995 Current Status: Walleye Trading Nuclear Engineering Current Status: Faculty, Stanford University Current Status: Procter & Gamble Amanda W. Duncan Advisors LLC Fellowship Years: 1993-1997 Dartmouth College Aerospace Engineering University of Illinois Current Status: Pacific Northwest Fellowship Years: 1991-1995 Gavin Conant Electrical Engineering Stephen Fink National Laboratory Edward Barragy Current Status: Joe Gibbs Racing University of New Mexico Fellowship Years: 1991-1995 University of California – San Diego University of Texas Biology Current Status: Intel Computer Science Matthew Giamporcaro Engineering Mechanics Kent Carlson Fellowship Years: 2000-2004 Fellowship Years: 1994-1998 Boston University Fellowship Years: 1991-1993 Florida State University Current Status: Trinity College, Lewis Jonathan Dursi Current Status: IBM Cognitive and Neural Systems Current Status: Intel Mechanical Engineering Dublin Ireland University of Chicago Fellowship Years: 1998-2000 Fellowship Years: 1991-1995 Astrophysics Robert Fischer Current Status: Adaptive Optics Associates William Barry Current Status: Staff, University of Iowa John Costello Fellowship Years: 1999-2003 Harvard University Carnegie Mellon University University of Arizona Current Status: Canadian Institute for Computer Science Ahna Girshick Structural & Computational Engineering Nathan Carstens Applied Mathematics Theoretical Astrophysics Fellowship Years: 1994-1998 University of California – Berkeley Fellowship Years: 1994-1998 Massachusetts Institute of Technology Fellowship Years: 1998-2002 Vision Science Current Status: Faculty, Nuclear Engineering Fellowship Years: 2001-2005 Asian Institute of Technology Fellowship Years: 2001-2004 Current Status: Student, Current Status: Student, MIT University of California – Berkeley

50 53

Kevin Glass H Gordon Hogenson K Jack Lemmon Matthew McNenly University of Oregon University of Washington Georgia Institute of Technology University of Michigan Computer Science Aric Hagberg Physical Chemistry Richard Katz Mechanical Engineering Aerospace Engineering Fellowship Years: 1996-2000 University of Arizona Fellowship Years: 1993-1996 Columbia University Fellowship Years: 1991-1994 Fellowship Years: 2001-2005 Current Status: Faculty, Applied Mathematics Current Status: Microsoft Geodynamics Current Status: Medtronic, Inc. Current Status: Student, University of Oregon Fellowship Years: 1992-1994 Fellowship Years: 2001-2005 University of Michigan Current Status: Los Alamos Daniel Horner Fellowship Years: Staff, Mary Ann Leung Larisa Goldmints National Laboratory University of California – Berkeley Columbia University University of Washington Lisa Mesaros Carnegie Mellon University Chemistry Theoretical Physical Chemistry University of Michigan Structural Mechanics Glenn Hammond Fellowship Years: 2000-2004 Benjamin Keen Fellowship Years: 2001-2005 Aerospace Engineering & Fellowship Years: 1997-2001 University of Illinois Current Status: Lawrence Berkeley University of Michigan Current Status: Student, Scientific Computing Current Status: General Electric & Environmental Engineering & Science National Laboratory Mathematics University of Washington Fellowship Years: 1991-1995 Rensselaer Polytechnic Institute Fellowship Years: 1999-2003 Fellowship Years: 2000-2004 Current Status: FLUENT, Inc. Current Status: Sandia National William Humphrey Current Status: IDA Center for Lars Liden William Gooding Laboratories – New Mexico University of Illinois Computing Sciences Boston University Richard Mills Purdue University Physics Cognitive & Neural Systems College of William and Mary Chemical Engineering Jeffrey Haney Fellowship Years: 1992-1994 Jeremy Kepner Fellowship Years: 1994-1998 Computer Science Fellowship Years: 1991-1994 Texas A&M University Current Status: TurboLabs, Inc. Princeton University Current Status: Staff, Fellowship Years: 2001-2004 Physical Oceanography Computational Cosmology University of Washington Current Status: Oak Ridge Catherine Grasso Fellowship Years: 1993-1996 Jason Hunt Fellowship Years: 1993-1996 National Laboratory Cornell University Current Status: Dynacon, Inc. University of Michigan Current Status: Staff, Massachusetts Tasha (Palmer) Lopez Bioinformatics Aerospace Engineering & Institute of Technology University of California – Los Angeles Julian Mintseris Fellowship Years: 2000-2004 Heath Hanshaw Scientific Computing Chemical Engineering Boston University Current Status: Soar Technology, Inc. University of Michigan Fellowship Years: 1999-2003 Sven Khatri Fellowship Years: 2000-2001 Bioinformatics Nuclear Engineering Current Status: General Dynamics – California Institute of Technology Current Status: IBM Fellowship Years: 2001-2005 Kristen Grauman Fellowship Years: 2001 – 2005 Advanced Information Systems Electrical Engineering Massachusetts Institute of Technology Current Status: Sandia National Fellowship Years: 1993-1996 Christie Lundy Erik Monsen Computer Science Laboratories – New Mexico E. McKay Hyde Current Status: Honeywell, Inc. University of Missouri – Rolla Stanford University Fellowship Years: 2001-2005 California Institute of Technology Physics Aerospace and Astronautical Engineering Current Status: Student, MIT Rellen Hardtke Benjamin Kirk Fellowship Years: 1991-1994 Fellowship Years: 1991-1994 G Applied & Computational Mathematics K University of Wisconsin – Madison Fellowship Years: 1999-2002 University of Texas Current Status: State of Missouri Employee Current Status: Student, Corey Graves Physics Current Status: Faculty, Rice University Aerospace Engineering University of Colorado North Carolina State University Fellowship Years: 1998-2002 Fellowship Years: 2001-2004 M H Computer Engineering Current Status: Faculty, University I Current Status: NASA Johnson Brian Moore L Fellowship Years: 1996-1999 of Wisconsin, River Falls Space Center William Marganski North Carolina State University Current Status: Faculty, North Carolina Eugene Ingerman Boston University Nuclear Engineering Agricultural & Technical State University Eric Held University of California – Berkeley Justin Koo Biomedical Engineering Fellowship Years: 1992-1995 I University of Wisconsin – Madison Applied Mathematics University of Michigan Fellowship Status: 1998-2002 Current Status: Global Nuclear Fuel M Michael Greminger Engineering Physics Fellowship Years: 1997-2001 Aerospace Engineering Current Status: Boston Biomedical University of Minnesota Fellowship Years: 1995-1999 Current Status: Staff, University Fellowship Years: 2000-2004 Research Institute Nathaniel Morgan Mechanical Engineering Current Status: Faculty, of California – Davis Current Status: Advatech Pacific, Inc. Georgia Institute of Technology J Fellowship Years: 2002-2005 Utah State University Daniel Martin Mechanical Engineering Current Status: Seagate Technologies Ahmed Ismail Michael Kowalok University of California – Berkeley Fellowship Years: 2002-2005 Judith Hill Massachusetts Institute of Technology University of Wisconsin Mechanical Engineering Current Status: Los Alamos Noel Gres Carnegie Mellon University Chemical Engineering Medical Physics Fellowship Years: 1993-1996 National Laboratory University of Illinois Mechanics, Algorithms & Computing Fellowship Years: 2000-2004 Fellowship Years: 2000-2004 Current Status: Lawrence Berkeley Electrical Engineering Fellowship Years: 1999-2003 Current Status: Sandia National Current Status: Staff, Virginia National Laboratory James (Dan) Morrow Fellowship Years: 1999-2001 Current Status: Sandia National Laboratories – New Mexico Commonwealth University Carnegie Mellon University Laboratories – New Mexico Marcus Martin Robotics Boyce Griffith J Yury Krongauz University of Minnesota Fellowship Years: 1992-1995 New York University – Courant Institute Charles Hindman Northwestern University Physical Chemistry Current Status: Sandia National Applied Mathematics University of Colorado Nickolas Jovanovic Theoretical & Applied Mechanics Fellowship Years: 1997-1999 Laboratories – New Mexico Fellowship Years: 2000-2004 Aerospace Engineering Yale University Fellowship Years: 1993-1996 Current Status: Sandia National Current Status: Faculty, Fellowship Years: 1999-2003 Mechanical Engineering Current Status: Black Rock Laboratories – New Mexico Sarah Moussa New York University Current Status: Air Force Fellowship Years: 1992-1994 University of California – Berkeley Research Laboratory Current Status: Faculty, University L Randall McDermott Machine Learning Eric Grimme of Arkansas – Little Rock University of Utah Fellowship Years: 2003-2005 University of Illinois Jeffrey Hittinger Eric Lee Chemical Engineering Current Status: Google Electrical Engineering University of Michigan Rutgers University Fellowship Years: 2001-2005 Fellowship Years: 1994-1997 Aerospace Engineering & Mechanical Engineering Current Status: Staff, Cornell University Michael Mysinger Current Status: Intel Scientific Computing Fellowship Years: 1999-2003 Stanford University Fellowship Years: 1996-2000 Current Status: Northrup Grumman Corp. Richard McLaughlin Chemical Engineering John Guidi Current Status: Lawrence Livermore Princeton University Fellowship Years: 1996-2000 University of Maryland National Laboratory Seung Lee Applied Mathematics Current Status: Arqule, Inc. Computer Science Massachusetts Institute of Technology Fellowship Years: 1991-1994 Fellowship Years: 1994-1997 Mechanical Engineering Current Status: Faculty, University Current Status: Math High School Teacher Fellowship Years: 2001-2005 of North Carolina Current Status: Student, MIT

52 55

N P Clifton Richardson Jason Sese Mayya Tokman James Wiggs Cornell University Stanford University California Institute of Technology University of Washington Heather Netzloff Steven Parker Physics Computational Materials Science Applied Mathematics Physical Chemistry Iowa State University University of Utah Fellowship Years: 1991-1995 Fellowship Years: 2003-2005 Fellowship Years: 1996-2000 Fellowship Years: 1991-1994 Physical Chemistry Computer Science Current Status: Tutor Current Status: Faculty, University of Current Status: Novum Fellowship Years: 2000-2004 Fellowship Years: 1994-1997 Christopher Rinderspacher California – Merced Current Status: Staff, Australian Current Status: Faculty, University of Georgia Elsie Simpson Pierce Jon Wilkening National University University of Utah Chemistry University of Illinois Mario Trujillo University of California – Berkeley Fellowship Years: 2001-2005 Nuclear Engineering University of Illinois Applied Mathematics Elijah Newren Joel Parriott Fellowship Years: 1991-1993 Mechanical Engineering Fellowship Years: 1997-2001 University of Utah University of Michigan John Rittner Current Status: Lawrence Livermore Fellowship Years: 1997-2000 Current Status: Faculty, University Mathematics Astronomy & Astrophysics Northwestern University National Laboratory Current Status: Staff, Pennsylvania of California – Berkeley Fellowship Years: 2001-2005 Fellowship Years: 1992-1996 Mechanical Engineering State University Current Status: Student, Current Status: Office of Fellowship Years: 1991-1995 Melinda Sirman Glenn Williams University of Utah Management and Budget Current Status: Chicago Board University of Texas V University of North Carolina Options Exchange Engineering Mechanics Environmental Science & Engineering Pauline Ng Virginia Pasour Fellowship Years: 1994-1996 Anton Van Der Ven Fellowship Years: 1993-1996 University of Washington North Carolina State University Courtney Roby Massachusetts Institute of Technology Current Status: Faculty, Bioengineering Biomathematics University of Colorado Steven Smith Materials Science Old Dominion University Fellowship Years: 2000-2002 Fellowship Years: 1998-1999 Electrical Engineering North Carolina State University Fellowship Years: 1996-2000 Current Status: Illumina Current Status: Student, Fellowship Years: 2002-2003 Chemical Engineering Current Status: Faculty, University C. Eric Williford Cornell University Current Status: Student, Fellowship Years: 1992-1994 of Michigan Florida State University Brian Nguyen Gunney University of Colorado Current Status: Invista Meteorology University of Michigan Robert (Chris) Penland Laura Vann Dominik Fellowship Years: 1993-1996 Aerospace Engineering & Duke University David Ropp Eric Sorin Florida Atlantic University Current Status: Weather Predict Scientific Computing Biomedical Engineering University of Arizona Stanford University Electrical Engineering N Fellowship Years: 1993-1996 Fellowship Years: 1993-1997 Applied Mathematics Chemical Physics Fellowship Years: 1993-1997 Matthew Wolinsky Current Status: Lawrence Livermore Current Status: Predix Fellowship Years: 1992-1995 Fellowship Years: 2002-2004 Current Status: Pratt & Whitney Duke University National Laboratory Pharmaceuticals, Inc. Current: Student, Stanford University Earth Surface Dynamics Robin Rosenfeld Rajesh Venkataramani Fellowship Years: 2001-2005 O S Diem-Phuong Nguyen James Phillips Scripps Research Institute Scott Stanley Massachusetts Institute of Technology Current Status: Staff, University University of Utah University of Illinois Biology University of California – San Diego Chemical Engineering of Minnesota Chemical Engineering Physics Fellowship Years: 1996-1997 Mechanical Engineering Fellowship Years: 1995-1999 P Fellowship Years: 1999-2003 Fellowship Years: 1995-1999 Current Status: ActiveSight Fellowship Years: 1994-1998 Current Status: Goldman Sachs Lee Worden T Current Status: Staff, University of Utah Current Status: Staff, Current Status: Hewlett Princeton University University of Illinois S Packard Company Stephen Vinay Applied Mathematics Debra Egle Nielsen Carnegie Mellon University Fellowship Years: 1998-2002 Q Colorado State University Todd Postma Samuel Schofield James Strzelec Chemical Engineering Current Status: Staff, University V Civil Engineering University of California – Berkeley University of Arizona Stanford University Fellowship Years: 1998-2000 of California – Davis Fellowship Years: 1992-1996 Nuclear Engineering Applied Mathematics Computational Mathematics Current Status: Bettis Laboratory Fellowship Years: 1994-1998 Fellowship Years: 2001-2005 Fellowship Years: 1992-1994 Peter Wyckoff R Joyce Noah Current Status: Totality Current Status: Student, W Massachusetts Institute of Technology W Stanford University University of Arizona Rajeev Surati Chemical Engineering Theoretical Chemistry Richard Propp Massachusetts Institute of Technology Phillip Weeber Fellowship Years: 1992-1995 Fellowship Years: 2001-2003 University of California – Berkeley Robert Sedgewick Electrical Engineering & University of North Carolina Current Status: Ohio S Current Status: Student, Mechanical Engineering University of California – Santa Barbara Computer Science Environmental Science & Engineering Supercomputing Center Z Stanford University Fellowship Years: 1993-1996 Physics Fellowship Years: 1995-1997 Fellowship Years: 1994-1996 Current Status: Oracle Fellowship Years: 2000-2003 Current Status: Nexaweb Current Status: Chatham Financial Z Catherine Norman Current Status: Staff, Carnegie Northwestern University Q Mellon University Laura (Painton) Swiler Adam Weller Charles Zeeb Applied Mathematics Carnegie Mellon University Princeton University Colorado State University Fellowship Years: 2000-2004 Alejandro Quezada Susanne (Essig) Seefried Engineering & Public Policy Chemical Engineering Mechanical Engineering University of California – Berkeley Massachusetts Institute of Technology Fellowship Years: 1992-1995 Fellowship Years: 2001-2002 Fellowship Years: 1993-1997 O Geophysics Aeronautics/Astronautics Current Status: Sandia National Current Status: Los Alamos Fellowship Years: 1997-1998 Fellowship Years: 1997-2002 Laboratories – New Mexico Gregory Whiffen National Laboratory Christopher Oehmen Cornell University University of Memphis R Marc Serre T Environmental Systems Engineering Scott Zoldi Biomedical Engineering University of North Carolina Fellowship Years: 1991-1995 Duke University Fellowship Years: 1999-2003 Nathan Rau Environmental Science & Engineering Shilpa Talwar Current Status: NASA – Jet Theoretical & Computational Physics Current Status: Pacific Northwest University of Illinois Fellowship Years: 1996-1999 Stanford University Propulsion Laboratory Fellowship Years: 1996-1998 National Laboratory Civil Engineering Current Status: Faculty, University Scientific Computing Current Status: Fair Issac Corporation Fellowship Years: 2000-2001 of North Carolina Fellowship Years: 1992-1994 Collin Wick Current Status: Hanson Current Status: Intel University of Minnesota Professional Services Computational Chemistry Fellowship Years: 2000-2003 Current Status: Pacific Northwest National Laboratory

54 FELLOWS DIRECTORY FOURTH YEAR FELLOWS | DOE COMPUTATIONAL SCIENCE GRADUATE FELLOWSHIP 57

Bree Aldridge Teresa Bailey Michael Barad Jaydeep Bardhan Mary Biddy Nawaf Bou-Rabee Massachusetts Institute of Technology Texas A&M University University of California – Davis Massachusetts Institute of Technology University of Wisconsin California Institute of Technology Computational Biology Engineering Environmental Modeling Electrical Engineering Engineering Applied and Computational Mathematics Advisor: Advisor: Advisor: Advisor: Advisor: Douglas Lauffenburger Marvin Adams Geoffrey Schladow Jacob White Juan de Pablo Advisor: Practicum: Practicum: Practicum: Practicum: Practicum: Jerrold Marsden Pacific Northwest National Laboratory Lawrence Livermore National Laboratory Lawrence Berkeley National Laboratory Argonne National Laboratory Sandia National Laboratories – Practicum: Contact: & Oak Ridge National Laboratory Contact: Contact: New Mexico Los Alamos National Laboratory [email protected] Contact: [email protected] [email protected] Contact: Contact: Research Synopsis: [email protected] Research Synopsis: Research Synopsis: [email protected] [email protected] My research goals focus on understanding Research Synopsis: My field of interest is the numerical study My research focuses on the application Research Synopsis: Research Synopsis: the mechanisms which make cellular Computational science and reactor physics of environmental transport in fluid systems. of boundary element methods (BEM) to Concern over the availability of petroleum- Currently, my research covers the following signaling networks robust and precise are related fields of study. Reactor physics, My case study is the San Francisco Bay electrostatics calculations in molecular based products has generated considerable topics in geometric mechanics: geometric through modeling and mathematical the study of neutron diffusion and transport, for which I have already collected a large biochemistry. Since electrostatic forces interest in finding alternative renewable integration on manifolds and linear spaces, analysis. Cells use large, highly integrated uses computational science to model reactor validation dataset. My model is based on act over a long range, and they can have sources for these products. In the case dissipation-induced instabilities, nonlinear signaling networks to transduce and behavior. Diffusion and transport problems the variable density incompressible Navier- both favorable and unfavorable effects on of industrial lubricants, vegetable oils are stability theory, and Hamiltonian chaos. process multiple input signals. We use are too complex to solve analytically because Stokes equations in 3D, including air/water protein-protein interactions, much rational a promising alternative to comparable I am also thinking about optimal parallel computational methods to study the they are dependent on three spatial and fluid/solid interfaces and the transport drug design research has focused on petroleum-based products. Unlike petroleum computing environments for computational underlying operation of the network and dimensions, energy, angle, and time. For of passive constituents. My numerical methods to optimize the charge distribution oils, vegetable oils offer the advantages of dynamical systems applications. systematic quantitative experimental this reason, much of the focus of reactor methodology is based on a second-order in a ligand to make a ligand-receptor being environmentally friendly, renewable, measurements of several key network physics is on developing computational accurate projection method with high-order interaction as favorable as possible, in and biodegradable resources. components in order to validate the methods to find efficient solutions for the accurate Godunov finite differencing including terms of both affinity and specificity. computational model used for analysis. system of equations that governs a physical slope limiters and a stable differencing of Vegetable oils are essentially mixtures of By using mathematical models based on model. Computers are the essential tools the nonlinear convection terms. This is a The boundary element method can be triglycerides. Although triglycerides are ordinary differential equations representing to perform the large amount of calculations proven methodology for hyperbolic problems accelerated using the precorrected FFT naturally abundant, little is understood the elementary chemical reactions required during the evaluation of that yields accurate transport with low algorithm, resulting in a “matrix-free” about the effect of molecular structure (mass-action kinetics), we can make these models. phase error while minimizing the numerical calculation of the potential on the ligand on their physical properties. To formulate predictions about the response of a cell diffusion at steep gradients typically found surface; because even this fast method vegetable oils with optimal lubrication to a given set of stimuli. Further, the model in ‘classical’ high order finite difference is computationally expensive, we would characteristics, however, the influence of can be analyzed by studying its nonlinear methods. For the fast and robust solution like to minimize the number of potential the individual molecular triglycerides on the dynamics, sensitivity, and topology to of the Poisson equations in my model I am calculations required in order to find the overall oil properties must be understood. elucidate the important processing nodes using a geometric multigrid method. This optimal charge distribution. With a new Computer simulations can directly relate and computing mechanisms embedded in methodology is placed in EBChombo, a BEM formulation in hand, I have most molecular composition to physical the system. three-dimension parallel adaptive mesh recently been working to develop matrix- properties and offer a valuable tool for refinement (AMR) framework, developed free optimization methods for this problem. improving our knowledge of this important at the Applied Numerical Algorithms Group Although presently only linear inequality class of materials. (ANAG) at Lawrence Berkeley National constraints are treated, I would like to Laboratory. The application of this extend the present solution method to By employing molecular dynamics methodology together with high-order handle nonlinear constraints as well. simulations, we have obtained viscosities accurate volume-of-fluid and level set and densities that agree with experimentally interface tracking methods for modeling My long-range research goals include observed values for both vegetable oils the air/water interface to environmental the incorporation of other forces — such and triglycerides. The effect of molecular problems is new. Most of this numerical as hydrophobic and van der Waals forces structure on physical properties is and methodology was developed at DOE — into the optimization procedure, and continues to be explored for triglycerides laboratories or through DOE Office of investigating BEM formulations that permit that are difficult to isolate in vegetable oils Science funding. the variation of the charge locations as and that are chemically modified. The well as the charge values. low-temperature properties of vegetable oils, which are a major limitation in their use as lubricants, have also been characterized using molecular modeling methods. The ultimate goal of this work is to use the knowledge gained from molecular-level modeling to engineer a vegetable oil-based lubricant.

56 FOURTH YEAR FELLOWS | DOE COMPUTATIONAL SCIENCE GRADUATE FELLOWSHIP 59

Kristine Cochran Gregory Davidson Michael Driscoll Mary Dunlop Owen Hehmeyer Yan Karklin University of Illinois – University of Michigan Boston University California Institute of Technology Princeton University Carnegie Mellon University Urbana – Champaign Nuclear Engineering Bioinformatics & Systems Biology Mechanical Engineering Chemical Engineering Computational Neuroscience Structures Advisor: Advisor: Advisor: Advisor: Advisor: Advisor: Ed Larsen James Collins Richard Murray Athanassios Z. Panagiotopoulos Michael Lewicki Keith Hjelmstad Practicum: Practicum: Practicum: Practicum: Practicum: Practicum: Bettis Atomic Power Laboratory Lawrence Berkeley National Laboratory Los Alamos National Laboratory Sandia National Laboratories – Lawrence Berkeley National Laboratory Sandia National Laboratories – Contact: Contact: Contact: New Mexico Contact: New Mexico [email protected] [email protected] [email protected] Contact: [email protected] Contact: Research Synopsis: Research Synopsis: Research Synopsis: [email protected] Research Synopsis: [email protected] My field of interest is particle transport. The discovery that the human genome My research is in control and dynamical Research Synopsis: My current research focuses on the Research Synopsis: Particle transport involves the study of contains far fewer genes than had been systems with applications to biology. For the first half of my studies, my work development of computational models of The focus of my research is the development radiation flowing through and interacting widely believed underscores a point: the focused on the molecular simulation of biological visual systems. Recent work, and implementation of continuum mechanics with a background medium. The Boltzmann richness of the human organism derives The field of synthetic biology has recently polymers in confined geometries. Going aided by probabilistic modeling techniques based constitutive models to represent transport equation which mathematically not from genes per se, but from interactions surged as researchers discover how to forward, I will work towards more complex in computer science and information cyclic metal plasticity. The intended describes this process is far too complex of genes. We are fast moving away from a manipulate genetic circuits to accomplish models of tethered polymers, and study theoretical techniques in signal processing, application is the modeling of low-cycle to solve analytically, except in the most time when we can say “this is a gene for simple tasks. Toy problems such as blinking their interactions with charged colloid has yielded some insights into the basic fatigue crack growth in metals, within a elementary and idealized of problems. X”, toward recognizing that biological bacteria, cells that use digital logic to play (protein-like) particles. In my first year of principles underlying computation in the three-dimensional finite element code. Computational methods are therefore function — and its counterpart, disease — tic-tac-toe, and bacteria that perform basic research, I developed understanding of the human visual system. Using only the Two nonlinear cyclic plasticity models essential to finding solutions to any are products of systems. Boolean computations are all developments needed Monte Carlo and other algorithms, notion that the brain’s task is to extract that have occurred within the past 5 years. and also created a new implementation to useful information from a noisy scene have been added to a research oriented realistic transport problem. Such experimental advances leave biologists compute electrostatic forces in non-periodic and represent it efficiently, it has been finite element code. The nonlinear kinematic I am interested in studying the networks that The transport equation is an integro- represent these interactions of genes. The and engineers alike asking the question: geometries. Over the next six months, I possible to deduce theoretically optimal hardening Frederick-Armstrong model “Can we redesign control systems in completed work on the thermodynamics representations for low level visual provides improved cyclic response over differential equation, which typically ability of a cell to change during development, requires an iterative method to solve. division, or in response to its environment microbiological organisms?” and structure of lattice homopolymers features without making assumptions the commonly used linear hardening Often, the transport equation is spatially hinges on a balance of competing and noisy in quasi 2-D and 2-D geometries. Next, about the kinds of visual structure we model. More recently, the generalized discretized using a finite-element method. factors. Even the smallest unit of change, But as this fascinating field of engineering the structure of uncharged tethered expect to discover. plasticity model, developed by Lubliner Advances in computer power have the turning on or off of one protein’s biological systems unfolds it becomes homopolymers was studied theoretically. and improved by Auricchio and Taylor, made deterministic transport algorithms production, is governed by several other apparent that biologists need better tools During the course of my practicum at The resulting learned representations was implemented into the code. This model discretized on unstructured, arbitrary proteins, which in turn are also regulated. to help them understand and characterize Sandia and afterwards, I studied electrically have been shown to relate closely to also features nonlinear response, but solves polygonal and polyhedral meshes viable. These genetic regulatory networks form the what is actually going on within the cell. charged, tethered polymers using molecular the responses of cells in early visual some of the theoretical and numerical Unfortunately, the development of robust architecture for a cell’s genomic program, Current synthetic gene network design dynamics simulations. For the remainder processing areas of the brain. The appeal issues inherent in the Frederick-Armstrong basis functions for arbitrary zone shapes and I believe understanding their dynamics is driven by expert biologists whose of my studies, I will continue to focus on of this approach lies in the fact that it is not model as applied to low-cycle fatigue has lagged behind other progress in is a step towards understanding a cell’s understanding of very specific pathways polymers in confined geometries, but will limited by our incomplete understanding modeling. Both models are implemented the field. In 1975, Eugene Wachspress higher functions and, in the case of enables them to develop ad hoc designs, seek also to understand not only their of the physiology of the visual system, as with substepping error control and make use published a book unveiling a new class disease, malfunctions. without adding to the understanding of structure, but also their interactions with it attempts to lay out the basic theoretical of the algorithmically consistent tangent. of rational basis functions that may be the field of gene circuit design as a whole. other, larger particles. Such study could principles that apply to any sensory applied to any convex polygon or polyhedron, While traditional experimental work will be Biological systems are an enticing area reveal how polymers affect the adsorption processing system. as well as certain polycons (zones with critical to this understanding, I intend to to work in because so many genetic of proteins on surfaces. curved sides). We know of no one having focus on developing numerical models of pathways have been characterized implemented these basis functions in a finite gene regulatory networks. These models experimentally without a systematic element radiation transport algorithm. rely on existing experimental knowledge, approach towards understanding as well as principles of nonlinear systems, overarching principles. My research involves investigating the to predict the behavior of a given network properties of Wachspress rational basis of genes. Their ultimate validation is found functions, and evaluating their performance by creating an engineered cellular system, in solving radiation flow problems, where a desired genetic program — an particularly in the diffusive limit. inducible switch or a steady oscillator, for example — is achieved through the design of genes and their regulatory elements.

58 FOURTH YEAR FELLOWS | DOE COMPUTATIONAL SCIENCE GRADUATE FELLOWSHIP 61

Benjamin Lewis Alex Lindblad Gregory Novak David Schmidt Amoolya Singh Obioma Uche Massachusetts Institute of Technology University of Washington University of California – Santa Cruz University of Illinois – University of California – Berkeley Princeton University Computational Biology Structural Engineering Theoretical Astrophysics Urbana – Champaign Computational Biology Materials/Statistical Mechanics Electrical Engineering Advisor: Advisor: Advisor: Advisor: Advisor: Chris Burge George Turkiyyah Sandra Faber Advisor: Richard Karp Salvatore Torquato Practicum: Practicum: Practicum: Richard Blahut Practicum: Practicum: Lawrence Berkeley National Laboratory Sandia National Laboratories – California Lawrence Berkeley National Laboratory Practicum: Pacific Northwest National Laboratory Sandia National Laboratories – Contact: Contact: Contact: Sandia National Laboratories – Contact: New Mexico [email protected] [email protected] [email protected] New Mexico [email protected] Contact: Research Synopsis: Research Synopsis: Research Synopsis: Contact: Research Synopsis: [email protected] In recent years, the rapid accumulation My interest is in the simulation and design of My research is designed primarily to [email protected] For my doctoral work I am interested in Research Synopsis: of biological data has given rise to a new high-performance structures — systems that answer two questions: 1.) Considering Research Synopsis: developing mathematical models and The determination of the macroscopic discipline: bioinformatics. Generally, can withstand extreme loading conditions nearby galaxies as a population, can their My field of interest within electrical computational simulations of cellular and microscopic properties of materials bioinformatics involves the application with predictable performance. Earthquakes, detailed internal dynamics (as probed by engineering is signal processing, and one regulatory networks. via molecular simulation methods is of of techniques from statistics, computer fires, explosions, winds/hurricanes, etc., Integral Field Unit, or IFU, observations) of the main reasons I was drawn to signal particular interest to me. The above topic science, and mathematics, to organize, place severe demands on structural systems. be explained by state-of-the-art simulated processing more than other areas of Since this is a huge and intricate field, is one that has its foundation in theoretical interpret, and integrate large quantities Balancing economic considerations with galaxy merger remnants? If not, what are electrical engineering is the amount of I would like to initially simulate a simple chemistry and physics. Molecular simulations of data describing macromolecules of predictable performance in these systems the physical processes missing from the mathematics that is found in the field. cellular system to gain computational and methods, e.g. Monte Carlo simulation, rely biological interest. Much of the current is a challenge. Constructing accurate models that are causing the discrepancy? I really like how useful the concepts I biological insights. Under analysis in the heavily on high-performance computation focus of bioinformatics is on three major usable models of these systems requires 2.) What can we learn about the merger learned in real and complex analysis are Arkin lab are several candidate problems: for a tractable solution to the problem. In categories of information: DNA and protein not only a firm understanding of the history of an individual galaxy from its to information theory, and especially how chemotaxis and type-1 pili phase variation addition, new computer algorithms are sequences, macromolecular structures, mechanics behind the problem, but also detailed internal dynamics? topics in algebra show up once again in of E. coli, sporulation and chemotaxis of always being written to solve problems and functional genomics data (e.g. gene a considerable knowledge base in the understanding and evaluating error- B. subtilis, lysis/lysogeny pathways in that arise in the application of statistical expression profiles, two-hybrid interaction field of mathematics with an emphasis These questions are interesting, relevant, correcting codes. I plan to do research in Lambda phage, G-protein coupled signal mechanics to a variety of cases. data). Unifying elements of biology and in computation. The availability of high and ripe today because of the recent error-correcting codes because they bring transduction pathways in cardiomyocytes, computer science, bioinformatics involves performance computers has made it maturation of IFUs with wide fields of view together my favorite aspect of electrical etc. I intend to generalize the insights In general, my research involves the studying the informational content of possible to build high-fidelity numerical and spectral resolution sufficient to study engineering and my preferred area of obtained from these specific simulations to use of optimization techniques to study biological molecules and the use of simulations that can predict behavior internal dynamics of galaxies (e.g. Emsellem mathematics. I knew it was what I wanted construct high performance computational optimal geometric packing arrangements computational tools to perform comparisons, accurately, and allow us to mechanistically et al. 04). Both the quantity and quality of to study as soon as I picked up a book on tools. I will validate these tools by showing of particles in various geometries. to identify relationships and trends, to build rather than empirically make design the data are increasing dramatically. coding theory and saw Galois theory used that they can be applied to other models of interactions and networks, etc. tradeoffs and compromises. outside a math classroom. I am also greatly computational biology problems such Comparing the simulations to the interested in multi-user detection and as climate change modeling, population observations on equal footing is not trivial space-time processing after my introduction genetics, or protein folding. because of the limited number of particles to them in the class I am taking on wireless in the simulations. The brute force solution communication from Professor H. Vincent is to use more particles, but that may not be Poor. Given the explosion of cellular feasible in terms of computer time. Bendo technology and the huge gains to be and Barnes (00) and Jesseit, Naab and enjoyed by incorporating them into one’s Burkert (04) have published computationally wireless system, they seem like important cheaper methods of “smoothing” the areas of research, beyond simply being simulated observations. extremely interesting to study in their own right. Digital communications and error-correcting codes are very strongly related to computational science because of their importance to communications in general and a computer’s inherent use of digital signals. Furthermore, because many signal processing models require processor intensive simulations, high performance computing is integral in the advancement of the field.

60 FOURTH YEAR FELLOWS | DOE COMPUTATIONAL SCIENCE GRADUATE FELLOWSHIP THIRD YEAR FELLOWS SECOND YEAR FELLOWS 63

Paul Bauman Tod Pascal Brian Taylor Erik Allen University of Texas California Institute University of Illinois – Massachusetts Institute Computational and of Technology Urbana – Champaign of Technology Applied Mathematics Physical Chemistry Engineering Mechanics Chemical Engineering Advisor: Advisor: Advisor: Advisor: J. Tinsley Oden William Goddard III Scott Stewart Kenneth Beers Practicum: Practicum: Practicum: Practicum: Sandia National Laboratories Sandia National Laboratories Lawrence Livermore Sandia National Laboratories Joshua Waterfall Michael Wu – New Mexico – New Mexico National Laboratory – New Mexico Cornell University University of California – Berkeley Contact: Contact: Contact: Contact: Biophysics Computational Neuroscience [email protected] [email protected] [email protected] [email protected]

Advisor: Advisor: William Conley Christina Payne William Triffo Michael Bybee James Sethna Jack Gallant Purdue University Vanderbilt University Rice University University of Illinois Practicum: Practicum: Nanoscale Mechanics Chemical Engineering Bioengineering Chemical Engineering Lawrence Berkeley National Laboratory Los Alamos National Laboratory Advisor: Advisor: Advisor: Advisor: Contact: Contact: Arvind Raman Peter Cummings Robert Raphael Jonathan Higdon [email protected] [email protected] Practicum: Practicum: Practicum: Practicum: Research Synopsis: Research Synopsis: Sandia National Laboratories Sandia National Laboratories Lawrence Berkeley Lawrence Livermore Borrowing methods from statistical physics, My research will focus on the computational – New Mexico – New Mexico National Laboratory National Laboratory I am studying how certain bacteria combine analysis and modeling of neurophysiological Contact: Contact: Contact: Contact: cell-cell signaling with gene regulation. data collected from experiments that are [email protected] [email protected] [email protected] [email protected] Currently I am investigating the network designed to unravel the visual processing in which controls conjugal gene transfer in the primary visual cortex (V1). If one treats a Aron Cummings Emma Rainey Michael Wolf Jimena Davis Agrobacterium tumefaciens and the neuron as a computing unit that transforms Arizona State University California Institute University of Illinois – North Carolina State University pathogenic regulatory network of input (stimuli) into action potentials, then Electrical Engineering of Technology Urbana – Champaign Applied Mathematics Pseudomonas syringae. Both of these its transfer function, or kernel, can be Advisor: Planetary Sciences Computer Science Advisor: networks alter gene expression in response estimated using mathematical techniques. David Ferry Advisor: Advisor: H.T. Banks to many chemical signals. In particular, The kernel might reveal the computations Practicum: David Stevenson Michael Heath Contact: these two networks include a quorum mediating extraction of visual features to Sandia National Practicum: Practicum: [email protected] sensing module, whereby the bacterium which the cell is tuned. More sophisticated Laboratories – California Argonne National Laboratory Lawrence Berkeley changes its gene expression in response to nonlinear kernel estimation methods could Contact: Contact: National Laboratory Jeffrey Drocco the density of its neighbors. Such responses enable us to understand processing in [email protected] [email protected] Contact: Princeton University are present in many bacteria and allow extra-striate visual areas, such as V2 and V4. [email protected] Computational Condensed not only for intraspecies communication Krzysztof Fidkowski Mark Rudner Matter Physics but for cross talk between bacteria of Massachusetts Institute Massachusetts Institute Brandon Wood Advisor: different species and lead to large scale, of Technology of Technology Massachusetts Institute Shivaji Sondhi collective behavior. Computational Fluid Physics of Technology Contact: Dynamics Advisor: Computational Materials [email protected] Advisor: Leonid Levitov Science David Darmofal Practicum: Advisor: Peter Kekenes-Huskey Practicum: Brookhaven Nicola Marzari California Institute Argonne National Laboratory National Laboratory Practicum: of Technology Contact: Contact: Lawrence Berkeley Computational [email protected] [email protected] National Laboratory Chemistry/Biology Contact: Advisor: Jasmine Foo Samuel Stechmann [email protected] William Goddard Brown University New York University Practicum: Applied Mathematics Applied Mathematics Sandia National Laboratories Advisor: Advisor: – New Mexico George Karniadakis Andre Majda Contact: Practicum: Practicum: [email protected] Lawrence Berkeley Los Alamos National Laboratory National Laboratory Contact: Contact: [email protected] [email protected]

62 SECOND YEAR FELLOWS FIRST YEAR FELLOWS

Bonnie Kirkpatrick Amber Sallerson Joshua Adelman Miler Lee University of California – University of North Carolina – University of California – University of Pennsylvania Berkeley Chapel Hill Berkeley Genomics and Computational Computer Science Applied Mathematics Biophysics Biology Advisor: Advisor: Advisor: Advisor: Richard Karp Roberto Camassa George Oster Junhyong Kim Practicum: Practicum: Contact: Contact: Lawrence Livermore Lawrence Berkeley [email protected] [email protected] National Laboratory National Laboratory Contact: Contact: Zlatan Aksamija Jeremy Lewi [email protected] [email protected] University of Illinois – Georgia Institute of Urbana – Champaign Technology Matthew McGrath Michael Veilleux Electrical Engineering Neuroengineering University of Minnesota Cornell University Advisor: Advisor: Physical Chemistry Structural Fracture Mechanics Umberto Ravaioli Robert Butera Advisor: Advisor: Contact: Contact: Ilja Siepmann Anthony Ingraffea [email protected] [email protected] Practicum: Practicum: Lawrence Livermore Sandia National Laboratories Jordan Atlas David Markowitz National Laboratory – New Mexico Cornell University Princeton University Contact: Contact: Chemical Engineering Computational Neurobiology [email protected] [email protected] Advisor: Advisor: Michael Shuler David Tank Ian Parrish Allan Wollaber Contact: Contact: Princeton University University of Michigan [email protected] [email protected] Computational Plasma Physics Nuclear Engineering Advisor: Advisor: Christopher Carey Peter Norgaard James Stone Edward Larsen University of Wisconsin Princeton University Practicum: Practicum: Plasma Physics Computational Plasma Sandia National Laboratories Los Alamos National Advisor: Dynamics – New Mexico Laboratory Carl Sovinec Advisor: Contact: Contact: Contact: Edgar Choueiri [email protected] [email protected] [email protected] Contact: [email protected] David Potere Etay Ziv Ethan Coon Princeton University Columbia University Columbia University Natalie Ostroff Remote Sensing/GIS Computational Biology Applied Mathematics University of California – Advisor: Advisor: Advisor: San Diego Burt Singer Chris Wiggins Marc Spiegelman Bioengineering Practicum: Contact: Contact: Advisor: Oak Ridge [email protected] [email protected] Jeff Hasty National Laboratory Contact: Contact: John ZuHone Jeff Hammond [email protected] [email protected] University of Chicago University of Chicago Astrophysics Theoretical Chemistry Christopher Schroeder Mala Radhakrishnan Advisor: Advisor: University of California – Massachusetts Institute Donald Lamb David Mazziotti San Diego of Technology Contact: Contact: Physics Physical Chemistry [email protected] [email protected] Advisor: Advisor: Julius Kuti Bruce Tidor Asegun Henry Contact: Practicum: Massachusetts Institute [email protected] Lawrence Berkeley of Technology National Laboratory Mechanical Engineering Stefan Wild Contact: Advisor: Cornell University [email protected] Gang Chen Operations Research Contact: Advisor: [email protected] Christine A. Shoemaker Contact: Kevin Kohlstedt [email protected] Northwestern University Bio-Polymer/Soft Matter Computation Advisor: Monica Olvera de la Cruz Contact: [email protected]