Toward Loosely Coupled Programming on Petascale Systems Ioan Raicu*, Zhao Zhang+, Mike Wilde#+, Ian Foster#*+, Pete Beckman#, Kamil Iskra#, Ben Clifford+ *Department of Computer Science, University of Chicago, Chicago, IL, USA +Computation Institute, University of Chicago and Argonne National Laboratory, Chicago, IL, USA #Mathematics and Computer Science Division, Argonne National Laboratory, Argonne IL, USA
[email protected],
[email protected], {wilde,foster,beckman,iskra}@mcs.anl.gov,
[email protected] Abstract— We have extended the Falkon lightweight task to-end application processes involving task coordination, execution framework to make loosely coupled programming on provenance tracking, and bookkeeping. Their approaches are petascale systems a practical and useful programming model. typically based on a model of loosely coupled computation, in This work studies and measures the performance factors which data is exchanged among tasks via files, databases or involved in applying this approach to enable the use of petascale XML documents, or a combination of these. Vast increases in systems by a broader user community, and with greater ease. data volume combined with the growing complexity of data Our work enables the execution of highly parallel computations analysis procedures and algorithms have rendered traditional composed of loosely coupled serial jobs with no modifications to manual processing and exploration unfavorable as compared the respective applications. This approach allows a new—and with modern high performance computing processes automated potentially far larger—class of applications to leverage petascale by scientific workflow systems. [3] systems, such as the IBM Blue Gene/P supercomputer. We present the challenges of I/O performance encountered in making The problem space can be partitioned into four main this model practical, and show results using both categories (Figure 1).