Grid Computing

BY M. MITCHELL WALDROP ILLUSTRATION BY HOLLY LINDEM COULD PUT THE PLANET’S INFORMATION-PROCESSING POWER ON TAP. Grid Computing www.technologyreview.com TECHNOLOGY REVIEW May 2002 31 Is Internet history about to repeat itself? Maybe. Back in the 1980s, the grams running on their office computers. emergence of a new infrastructure upon National Science Foundation created the But that look will be deceptive: what which first science, and then the whole NSFnet: a communications network appear to be applications that reside on economy, will be built.” intended to give scientific researchers the local desktop machine might actually easy access to its new supercomputer be data analysis tools running on the centers. Very quickly, one smaller net- cluster at San Diego, or visualization COMPUTING AS UTILITY work after another linked in—and the software crunching bits at Argonne. The That’s a tall order. But it certainly result was the Internet as we now know it. “files” TeraGrid users are working on describes the hope at IBM, which is the The scientists whose needs the NSFnet might consist of databases scattered all prime contractor for the TeraGrid, as originally served are barely remembered over the country, containing thousands well as for similar national grids in by the online masses. of gigabytes—a.k.a. terabytes. Europe. David Turek, vice president of Fast-forward to 2002. This summer, Grid computing visionaries hope that emerging technologies for IBM’s server the National Science Foundation will this will be only the beginning—that the group, compares grid computing to the begin to install the hardware for the Tera- $53 million TeraGrid will catalyze a new familiar grid of electrical power: “To use Grid, a transcontinental supercomputer era of grid computing for the masses, a hair dryer, you just plug it into a wall that should do for computing power much as the NSFnet broke down barriers socket,”he says.“You don’t have to worry what the Internet did for documents. that led to the blossoming of the Internet. about how the turbine is designed up in First, clusters of high-end micro- Just within the past year or two, dozens of Niagara Falls, or the physics of power computers will be set up at four sites: the such projects have been announced in transmission.” That’s exactly how Turek National Center for Supercomputing Europe, Asia and the United States, with wants people to think about computing Applications at the University of Illinois more likely to come. And the developers power. “In our vision of the future, if at Urbana-Champaign; the U.S. Depart- of grid computing are now settling on a you’re a customer who occasionally needs ment of Energy’s Argonne National Labo- single standard—called the Globus 10 teraflops, for example, don’t buy a ratory outside Chicago; Caltech in Toolkit—that will help grid projects machine that’s underutilized most of the Pasadena, CA; and the San Diego Super- under development all around the world time; buy it from the grid. So grid com- computer Center at the University of coalesce into a worldwide network of puting will play into our vision of com- California, San Diego. Then, by early tappable computer power. puting as a utility.” next year, those four clusters will be net- “Completely transformational” is While companies like IBM would worked together so tightly that they will how Larry Smarr, director of the Cali- build the large-scale grids, Turek says behave as a single entity. fornia Institute for Telecommunications that many users will want to set up grids This virtual computer will rip and Information Technology, sums up of their own. “You might see 10 to 20 through problems at up to 13.6 trillion grid computing. Smarr, renowned for his departments coming together to create a floating-point operations per second, or role in developing the communications campuswide or companywide grid, each teraflops—eight times faster than the system that evolved into the Internet’s contributing some of the computer power most powerful academic supercomputer backbone, says the technology is what they control,” he says. In another sce- available today. Such speed will enable sci- the Internet has been building toward for nario, several independent companies, entists to tackle some of the most com- the past three decades. “In the first such as defense contractors, might do putationally intensive tasks on the phase,”he explains,“we got the wires up much the same thing to create “virtual research docket—from problems in pro- and hooked in all the computers. Then organizations”—ad hoc grids that would tein folding that will form the basis for with the World Wide Web, we started allow them to use one another’s proprie- new drug designs to climate modeling to hooking in all the online documents.” tary data and software to prepare, say, a deducing the content and behavior of Now, he says, with grid computing, we’ll proposal for a new military aircraft. the cosmos from astronomical data. be hooking in everything else (see “That’s why we’re not going to espouse But more than that, the TeraGrid “Planet Internet,” TR March 2002). the grid as something that can be done will be a prime example of what has This means that users will begin to only with IBM technology,” Turek come to be known as “grid comput- experience the Internet as a seamless explains. After all, he says, “if you get ing”—the massive integration of com- computational universe. Software appli- five companies wanting to come together puter systems to offer performance cations, databases, sensors, video and on a grid, the likelihood of all five having unattainable by any single machine. The audio streams—all will be reborn as ser- the same servers is pretty slim.” integration of these systems will be so vices that live in cyberspace, assembling And that, Turek adds, is the beauty of transparent that users will no more and reassembling themselves on the fly to the Globus Toolkit: a set of open-source notice they are on a network than meet the tasks at hand. Once plugged software tools that is fast emerging as motorists pay attention to which cylinder into the grid, a desktop machine will the de facto standard for grid computing, is firing at any given moment. To people draw computational horsepower from in much the same way that the hypertext logging onto the TeraGrid, the system all the other computers on the grid. transfer protocol, or HTTP, is the stan- will look like just another set of pro- “What we’re seeing,” says Smarr, “is the dard for linking documents on the Web. 32 TECHNOLOGY REVIEW May 2002 www.technologyreview.com Indeed, the growing acceptance of Globus implement security—assuring, for telescope data from the search-for- is largely responsible for today’s wave of instance, that an outside program trying extraterrestrial-intelligence project are grid computing excitement. to interact with your machine is serving distributed to PCs across the Internet. “The idea is to let the network provide a legitimate purpose and hasn’t been sent At the same time, however, the high- the basic mechanisms for moving data by some malicious hacker. performance-computer community be- around, while Globus provides mecha- Of course, none of this is entirely gan a series of less publicized but much nisms for resource sharing,”explains Carl new: “It’s worth remembering,” notes more ambitious experiments in “meta- Kesselman of the University of Southern Kesselman,“that ARPAnet [the military- computing.”The idea was to make many California’s Information Sciences Institute. built ancestor of the Internet] was built in distributed computers function like one Kesselman has been developing the the 1960s to give users on one campus giant computer. The metamachine’s Globus Toolkit over the past five years in shared access to resources on a different keyboard and display would be sitting on collaboration with Ian Foster—a Univer- campus.”Likewise, he points out, meth- someone’s desktop, as usual. But its cen- sity of Chicago computer scientist who ods for breaking computational jobs into tral processor might actually be a super- heads Argonne’s distributed-systems smaller pieces for multiple machines were computer in Illinois, say, while its laboratory. a perennial research topic throughout graphics processor might be an immer- The mechanisms that Globus pro- the 1970s and 1980s. sive-virtual-reality facility in California. vides are as essential to the computing But it was only in the 1990s, Kessel- It worked, says Kesselman—the only grid’s operation as stoplights are to city man says, that the rapidly increasing problem being that experimenters had to traffic. One set of Globus software tools, power of computers and networks reinvent the wheel every time. “There for example, automatically roots out brought this trend, known as distributed was still no standard software for dis- where on the grid a required database or computing, out of the laboratories. One tributed computing,”he says, “no infra- program can be found. Other tools allow result was a flurry of experiments in what structure to support it.” one-time login, so that the user isn’t con- is now known as “peer-to-peer” com- The technology’s watershed event stantly being asked for passwords for site puting, all devoted in one way or another came in 1995, at a supercomputing con- after site after site. Still others divide a to harnessing the computing power and ference sponsored by the Institute of computational job into multiple sub- storage capacity of idle desktop machines. Electrical and Electronics Engineers and tasks and parcel them out among the Among the best known of these efforts the Association for Computing Machin- various systems on the grid.

Load more