> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < 1 Grid Computing: Introduction and Overview Manish Parashar, Senior Member, IEEE and Craig A. Lee, Member, IEEE Its goal is to provide a service-oriented infrastructure that Abstract—This paper provides an overview of Grid computing leverages standardized protocols and services to enable and this special issue. It addresses motivations and driving forces pervasive access to, and coordinated sharing of geographically for the Grid, tracks the evolution of the Grid, discusses key issues distributed hardware, software, and information resources. in Grid computing, outlines the objective of the special issues, The Grid community and the Global Grid Forum1 are and introduces the contributed papers. Index Terms—Grid computing investing considerable effort in developing and deploying standard protocols and services that enable seamless and I. INTRODUCTION secure discovery, access to, and interactions among resources, services, and applications. This potential for seamless HE growth of the Internet, along with the availability of aggregation, integration, and interactions has also made it powerful computers and high-speed networks as low-cost T possible for scientists and engineers to conceive a new commodity components, is changing the way scientists and generation of applications that enable realistic investigation of engineers do computing, and is also changing how society in complex scientific and engineering problems. general manages information and information services. These This current vision of Grid computing certainly did not new technologies have enabled the clustering of a wide happen overnight. In what follows, we trace the evolution of variety of geographically distributed resources, such as Grid computing from its roots in parallel and distributed supercomputers, storage systems, data sources, instruments, computing to its current state and emerging trends and visions. and special devices and services, which can then be used as a unified resource. Furthermore, they have enabled seamless A. The Origins of the Grid access to and interaction among these distributed resources, While the concept of a “computing utility” providing services, applications, and data. The new paradigm that has “continuous operation analogous to power and telephone” can evolved is popularly termed as “Grid” computing. Grid be traced back to the 1960s and the Multics Project [4], the computing and the utilization of the global Grid infrastructure origins of the current Grid revolution can be traced to the late have presented significant challenges at all levels including 1980's and early 1990's and the tremendous amounts of conceptual and implementation models, application research being done on parallel programming and distributed formulation and development, programming systems, systems. Parallel computers in a variety of architectures had infrastructures and services, resource management, become commercially available, and networking hardware and networking and security, and have led to the development of a software were becoming more widely deployed. To global research community. effectively program these new parallel machines, a long list of parallel programming languages and tools were being II. GRID COMPUTING – AN EVOLVING VISION developed and evaluated [14]. This list included Linda, Concurrent Prolog, BSP, Occam, Programming Composition The Grid vision has been described as a world in which Notion, Fortran-D, Compositional C++, pC++, Mentat, computational power (resources, services, data) is as readily Nexus, lightweight threads, and the Parallel Virtual Machine, available as electrical power and other utilities, in which to name just a few. computational services make this power available to users To developers and practitioners using these new tools, it with differing levels of expertise in diverse areas, and in soon became obvious that computer networks would allow which these services can interact to perform specified tasks groups of machines to be used together by one parallel code. efficiently and securely with minimal human intervention. NOWs (Network of Workstations) were in regular use for Driven by revolutions in science and business and fueled by parallel computation. Besides just homogeneous sets of exponential advances in computing, communication, and machines, it was also possible to use heterogeneous sets of storage technologies, Grid computing is rapidly emerging as machines. Indeed, networks had already given rise to the the dominant paradigm for wide area distributed computing. notion of distributed computing. Using whatever programming means available, work was being done on M. Parashar is with the Department of Electrical and Computer fundamental concepts such as algorithms for consensus, Engineering, Rutgers: The State University of New Jersey, 94 Brett Road, synchronization, and distributed termination detection. Piscataway, NJ 08854 USA (phone: 732-445-5388; fax: 732-445-0593; e- Systems such as the Distributed Computing Environment mail: parashar@ caip.rutgers.edu). (DCE) were built to facilitate the use of groups of machines, C. Lee is with the Computer Systems Research Department, The Aerospace Corporation, 2350 E. El Segundo Blvd., El Segundo, 90245 CA USA (e-mail: [email protected]). 1 http://www.ggf.org/. > REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < 2 albeit in relatively static, well-defined, closed configurations2. machines for different applications and for security for their Similarly, the Common Object Request Broker Architecture access [7]. (CORBA) managed distributed systems by providing an The trials and tribulations of such an arduous demonstration object-oriented, client-side API that could access other objects paid-off since it crystallized for a much broader segment of through an Object Request Broker (ORB)3. the scientific community, what was possible and what needed Since different codes could be run on different machines, to be done [15]. In early 1996, the Globus Project officially yet still be considered a part of the same application, it was got under way after being proposed to ARPA in November possible to achieve a distributed end-to-end system capability, 1994. The process and communication middleware system such as data ingest, processing, and visualization/post- called Nexus [9] was originally built by Argonne National processing. This was sometimes called metacomputing [3]. Laboratory to essentially be a compiler target and provide Even then, the analogy between this style of computing and remote service requests across heterogeneous machines for the electrical power grid was clear [16]: application codes written in a higher-level language. The goal of the Globus project [1] was to build a global Nexus that “The Metacomputer is similar to an electricity grid. would provide support for resource discovery, resource When you turn on your light, you don't care where the composition, data access, authentication, authorization, etc. power comes from; you just want the light to come on. The first Globus applications were demonstrated at The same is true for computer users. They want their Supercomputing `974. job to run on the best possible machine and they really Globus was by no means alone in this arena. During this don't care how that gets done.” [S. Wallach, 1992] same time period, the Legion project [10] was generalizing the Of course, the development of programming languages and concepts developed for Mentat into the notion of a “global tools that attempted to transparently harness “arbitrary” sets of operating system”. The Condor project [6] was already machines served to highlight a host of issues and challenges. harvesting cycles from the growing number of desktop First, there was no real way to discover what machines were machines that a typical institution was now deploying. The available. In some cases, a hand-coded local resource file UNICORE project (UNiform Interface to COmputing defined the “universe” of machines that a parallel/distributed REsources) [2] was started in Germany in 1997. application knew about. Binaries had to be pre-staged by Backing up in time to 1995, Smarr and Catlett at the hand to a file system local to the remote machines on a well- National Center for Supercomputing Applications (NCSA) known path. Contacting any of these machines and starting had constructed a cluster of SGI Power Challenge parallel tasks was typically done using basic UNIX services such as computers that they called the Power Challenge Array. They rsh and managed using .rhost files. Needless to say, security envisioned a distributed metacomputer of these machines at was virtually non-existent. partner sites around the country that they intended to call the Furthermore, these new programming systems were SGI Power Grid. When the NSF supercomputing centers focusing on new and novel syntax for expressing and were re-competed in 1996, researchers at the consortium of managing the semantics of parallel and distributed NCSA, University of Illinois at Urbana-Champaign, Rice computation. Once up and running, an application code had University, Indiana University, Argonne National Laboratory, no idea of the state of its execution environment or what and University of Illinois at Chicago decided to expand on the process or network performance it was getting, unless it did Power Grid concept. In their proposal to the NSF, they stated: passive self-monitoring or deployed