THE ANNUAL HIP 2011 S ELLOW F UATE UATE D RA G E C IEN C DEIXIS NERGY COMPUTATIONAL S NERGY COMPUTATIONAL E DEPARTMENT OF DEPARTMENT FROM THE GALACTIC TO THE ATOMIC Practicum Takes Paul Sutter Deep into Inner Space PAGE 5 Ying Hu’s Road Trips Lead to Scattering Discoveries PAGE 9 THE ANNUAL TABLE OF CONTENTS DEIXIS 9 22 DEIXIS, The DOE CSGF Annual is published by the Krell Institute. Krell administers the Department of Energy Computational Science Graduate Fellowship (DOE CSGF) program for the DOE under contract DE-FG02-97ER25308. For additional information about the DOE CSGF program, the Krell Institute or topics covered in this publication, please contact: 5 24 26 Editor, DEIXIS Krell Institute 1609 Golden Aspen Drive, Suite 101 Ames, IA 50010 (515) 956-3696 www.krellinst.org/csgf A TOAST TO 20 YEARS 4 22 28 Copyright 2011 Krell Institute. Practicum Profiles Alumni Profiles Winning Essays All rights reserved. In 1991, the fates of two A Season of Discovery From Alumni to Leaders Encouraging Communication technological tools — the Internet and computational science — became 5 Paul Sutter 22 Brian Moore 28 Kenley Pelzer, Winner DEIXIS (ΔΕΙΞΙΣ — pronounced da¯ksis) transliterated linked in the High-Performance Fellow Spans Scales from A Career at the Can Peeling an Onion from classical Greek into the Roman alphabet, means a display, mode or process of proof; the process of Computing and Communications the Universal to the Atomic Nuclear Intersection Cure Cancer? showing, proving or demonstrating. DEIXIS can Act. The law backed creation of the “information superhighway,” but also refer to the workings of an individual’s keen intellect, or to the means by which such individuals, also directed the Department of Energy (DOE) to support computational 9 Ying Hu 24 Dan Martin 30 Hayes Stripling, e.g. DOE CSGF fellows, are identified. science education. Illuminating a Path “Cool Problems” Draw Honorable Mention The Internet’s phenomenal expansion is renowned, but the parallel to Cancer Cells Alumnus to Laboratory On the Quantification DEIXIS is an annual publication of the Department of Energy Computational Science Graduate Fellowship escalation in high-performance computing for science has been equally of “Maybe”: A Niche program that highlights the work of fellows and alumni. exceptional. Since 1991, the DOE Computational Science Graduate 13 Anne Warlaumont 26 Mala Radhakrishnan for Computation Fellowship has led the field’s maturation into the “third leg” of scientific Modeling a Neural Network Alumna Busts Resistance — The DOE CSGF is funded by the Office of Science and the National Nuclear Security Administration’s Office discovery. With more than 250 alumni, it’s helped create a community in the Classroom and Lab of Defense Programs. of computational science leaders. 17 Scott Clark This issue bears witness to the fellowship’s success. Our cover story Practicum Charts Career 32 features Paul Sutter, who used his practicum to step outside his scientific Course Beyond the Howes Scholars comfort zone. We also announce this year’s Frederick A. Howes Scholar Velvet Rope Living Up to a Legacy in Computational Science: Alejandro Rodriguez, who has matched his work in computational physics with his devotion to outreach. And we feature three alumni, including Brian Moore, who found a calling in Editor Publication nuclear energy. 35 Shelly Olsan Coordinator Alumni and fellows are welcome to participate in the program’s Directory Lisa Frerichs essay contest, which provides them the chance to convey their research Senior Science Writer Design to a broader, non-technical audience. The winner, fellow Kenley 35 Fellows Thomas R. O’Donnell julsdesign, inc. Pelzer, connects onions to quantum mechanics. Here’s to 20 years — and many more to come. 38 Alumni Copy Editor Ron Winther 28 P2 DEIXIS 11 DOE CSGF ANNUAL DEIXIS 11 DOE CSGF ANNUAL P3 practicum profiles A SEASON OF DISCOVERY FOR FELLOWS, THE SUMMER PRACTICUM nurtures PROFESSIONAL AND PERSONAL GROWTH The Department of FelloW SPANS ScaleS from Energy Computational THE UNIVERSAL to THE atomic Science Graduate Fellowship supports the PAUL SUTTER nation’s brightest science University of Illinois at Urbana-Champaign and engineering students, Oak Ridge National Laboratory allowing them to concentrate on learning WheN it caME tiME to choose a practicum for summer 2009, Paul Sutter and research. The work went from the enormous to the minuscule. Sutter, a Department of Energy Computational Science Graduate Fellowship recipient, of more than 250 DOE usually simulated galaxy clusters, the largest structures in the universe. At Oak Ridge National CSGF alumni has helped Laboratory (ORNL), he calculated electronic structure in atoms. “The energy scales, the spatial scales, the time scales are the complete opposite end of the the United States remain spectrum from what I usually do,” says Sutter, who earned his doctoral degree in astrophysics competitive in a from the University of Illinois at Urbana-Champaign (UIUC) this spring. “And the architecture of the computer code was completely different.” global economy. WIt’s just what he wanted. Practicums let fellows try research in areas largely outside their studies, but few go as far afield as Sutter did. “I realized that there aren’t a lot of opportunities in ~~~~~ your research career to go off and do something completely different,” he says, and he took advantage of it. After meeting several ORNL researchers Sutter opted to work with Robert Harrison, leader of the Computational Chemical Sciences Group. Sutter focused on MADNESS (Multiresolution Adaptive Numerical Environment for Scientific Simulation), a code that solves the electronic structures of atoms, molecules and Left to right: SUMMer plaNS often take more thought for recipients of the Department of nanoscale systems. Harrison led creation of the program, which is used in computational Scott Clark, Paul Sutter, Anne Energy Computational Science Graduate Fellowship than for other graduate students. chemistry, superconductor modeling and other areas. Warlaumont and Ying Hu in Summer usually is when fellows head to DOE national laboratories to complete their required Calculating electronic structure is so difficult scientists must limit their models’ accuracy Washington, D.C., at the DOE practicums. Often, they’ve met the scientist they’ll work with at a conference or through a mutual and the size of the systems they compute. Using MADNESS, researchers can compute the CSGF Annual Conference. acquaintance. But some fellows — like Ying Hu, who is profiled here — also visit several labs properties of larger systems with higher accuracy than traditional quantum chemistry before choosing one. codes allow. It divides the physical domain being modeled into parts for calculation on Practicums push fellows to explore subjects or test skills with little or no connection to parallel processing computers, while repeatedly subdividing the most interesting pieces for their doctoral research. It frequently leads them to refine a computer science skill — as fellow more precision. Paul Sutter did — or, like fellow Anne Warlaumont, to develop a greater appreciation for As the program subdivides the domain it generates an “octree” structure of branching hands-on lab work. nodes. The number of nodes depends on the level of detail researchers want, the problem size Some fellows even end the summer determined to take on entirely new research subjects. and the computer they use. Typical MADNESS problems involve octrees with 10,000 to 100 Fellow Scott Clark, for instance, headed to his practicum planning to focus his dissertation on million nodes. computational fluid dynamics. By summer’s end, he’d found a new love: metagenomics. That’s the power of the practicum: reinvigorating, redirecting and remarkable. P4 DEIXIS 11 DOE CSGF ANNUAL DEIXIS 11 DOE CSGF ANNUAL P5 practicum profiles Sutter and Hartman-Baker implemented a “melding” algorithm that splits up nodes so each processor has roughly the same amount of communication and work. ~~~~~ Sutter and Hartman-Baker implemented The second test used a real-world balancing algorithm, calculation time held Sutter’s research focused on magnetic a “melding” algorithm that splits up nodes problem: solving the Poisson equation to steady at around 60 seconds — a 60 fields within clusters of hundreds or so each processor has roughly the same calculate the electrostatic potential of a percent cut. “Overall, we found a pretty thousands of galaxies. Astrophysicists amount of communication and work. But large molecule-like structure. “This problem solid performance boost when doing this think supermassive black holes found in a “we didn’t really have a way of figuring can get very, very big very, very quickly,” improved load balancing,” Sutter says. galaxy at the clusters’ center may generate out how much time was required on each Sutter says. “There is a nasty list of things Working with MADNESS gave the fields. node” to weight them for redistribution, that have to be done.” Sutter insights into extending FLASH, The fields are weak, Ricker says — Hartman-Baker says. Again, Sutter tested the algorithm a multiphysics, multiscale code he used perhaps a fraction of Earth’s, but spread Sutter addressed that problem with a on large and small problems and averaged in his dissertation research. In fact, over a large enough area that they force “profiling” algorithm. In effect, MADNESS results over five runs. Profiling and he returned to Oak Ridge last fall accelerated particles like electrons to runs with the random load-balancing melding produced a modest gain on the to work with Hartman-Baker on move in a spiral-like pattern, generating algorithm and gathers data on how long large problem, cutting calculation time by improving FLASH. radiation in the radio spectrum. Two nodes take to compute. Then it uses that almost 10 percent when compared to the Sutter also got the chance to work clusters may look similar in the optical data to rebalance the load on the fly. standard approach.