sign in sign up
<<
Home , Concurrent computing

The PVM Concurrent System Evolution

Exp eriences and Trends

V S Sunderam

Department of Mathematics and Science

Emory UniversityAtlanta GA USA

G A Geist

Mathematical Sciences Section

Oak Ridge National Lab oratory

Oak Ridge TN USA

J Dongarra R Manchek

Computer Science Department

UniversityofTennessee Knoxville TN USA

Abstract

The PVM system a framework for heterogeneous

in networked environments has evolved in the past several years into a viable technol

ogy for distributed and parallel pro cessing in a variety of disciplines PVM supp orts

a straightforward but functionally complete mo del and is capable of

harnessing the combined resources of typically heterogeneous networked computing plat

forms to deliver high levels of p erformance and functionality In this pap er we describ

the architecture of PVM system and discuss its computing mo del the programming

interface it supp orts auxiliary facilities for pro cess groups and MPP supp ort and some

of the internal implementation techniques employed Performance issues dealing pri

marily with communication overheads are analyzed and recent ndings as well as

exp erimental enhancements to are presented In order to demonstrate the viabilityof

PVM for large scale scientic sup ercomputing the pap er includes representativecase

studies in materials science environmental science and climate mo deling We conclude

with a discussion of related pro jects and future directions and comment on near and

longterm p otential for network computing with the PVM system

Research supp orted by the Applied Mathematical Sciences program Oce of Basic Energy Sciences

U S Department of Energy under Grant No DEFGER and contract DEACOR

with Martin Marietta Energy Systems Inc and the National Science Foundation under Award Nos CCR

and CCR

Intro duction

The past several years has witnessed an everincreasing acceptance and adoption of paral

lel pro cessing b oth for highp erformance scientic computing as well as for more general

purp ose applications Furthermore the message passing mo del app ears to b e gaining

predominance as the paradigm of choice from the p ersp ectiveofnumb er and varietyof

multipro cessors esp ecially massively parallel pro cessors and also in terms of applications

languages and software systems for its supp ort This pap er concerns one such message

passing system PVM Parallel that is a software infrastructure that

emulates a generalized distributed memory multipro cessor in heterogeneous networked envi

ronments Such an approach whichobviates the need to p ossess a hardware multipro cessor

has proven to b e a viable and costeectivetechnology for concurrent computing in many

application domains Owing to its ubiquitous nature a virtual parallel machine maybe

constructed using PVM with any set of machines one has access to and also due to its

simple but complete programming interface the PVM system has gained widespread accep

tance in the highp erformance scientic computing community In this pap er we present

the rationale and motivations for this pro ject the mo del supp orted imp ortant design de

cisions and p erformance considerations and case studies in the use of PVM for scientic

sup ercomputing

Heterogeneous Network Computing

Heterogeneous networkbase concurrent computing refers to an evolving metho dology for

general purp ose concurrent computing where

The hardware platform consists of a collection of multifaceted computer systems of

varying architectures interconnected by one or more network typ es A sp ecial case

alb eit the most common at present is a collection of similar or iden tical workstations

on a single lo cal area network although a more sp ecic term for suchenvironments

is cluster

Applications are viewed as comprising several sub each of which is p oten

tially dierent in terms of its most appropriate programming mo del implementation

language and resource requirements

Heterogeneous network computing refers to mo dels techniques and to olkits to match

heterogeneous environments on the one hand with complete applications consisting of dif

ferent subtasks on the other While explicit attention to the heterogeneous asp ects and

functionality exploitation in such scenarios has only recently received formalized atten

tion the concept in some form has b een explored previously eg although usually with

a narrow fo cus or based on a sp ecialized architecture The PVM system was designed to re

alize a more general and encompassing interpretation of heterogeneous computing and had

a pragmatic bias aimed at providing a working system that could b e used in existing envi

ronments PVM supp orts heterogeneous machines applications and networks Research

is continuing towards the eventual goals of the pro ject namely to prop ose a heterogeneous

application development mo del and asso ciated programming frameworks to enable opti

mal mapping b etween application subtasks and the b estsuited machines and to provide

adequate infrastructure for heterogeneous debugging proling and monitor

ing In the meantime however current realizations of PVM supp orting basic heterogeneous

features and robust emulations of heterogeneous concurrentmachines have proven to b e

valuable and eective for more traditional applications esp ecially in highp erformance sci

entic computing The facilities that are currently available as well as recent results and

ongoing work are discussed in the following sections

The Evolution of PVM

The PVM pro ject started in the summer of and has evolved through three ver

sions of the software the latter twoofwhichhave b een publicly distributed The origi

nal version of the system was ambitious in that it attempted to b e heterogeneous in

terms of programming mo del as well supp ort for emulated in addition

to message passing was incorp orated The basic computing mo del which has remained

semantically unchanged views applications as consisting of componentseach representing

a sub each comp onent is an SPMD program p otentially manifested as multiple

instances co op erating internally as well as with other comp onent instances via the sup

unication and synchronization mechanisms The unit of concurrency in PVM p orted comm

is a pro cess and dep endencie s in the pro cess ow graph are implemented byemb edding ap

propriate PVM primitives for pro cess management and synchronization within control ow

constructs of the host The implementation mo del also unchanged

from the original version uses the notion of a host p o ol a collection of interconnected

computer systems that comprises the virtual machine on which daemon pro cesses execute

and co op erate to emulate a concurrent computing system Applications request and receive

services from the daemons the facilities supp orted essentially fall into the categories of

pro cess management and virtual machine conguration message passing synchronization

and miscellaneous status checking and housekeeping tasks

PVM is the mainstay of the Heterogeneous Network Computing research pro ject a

collab orativeventure b etween Emory University Oak Ridge National Lab oratory and the

UniversityofTennessee In addition to the authors Keith Mo ore and Weicheng Jiang

of UT and Adam Beguelin of CMU are coinvestigators This pro ject is a basic research

eort aimed at advancing science and is wholly funded by research appropriations from the

US Department of Energy the National Science Foundation and the State of Tennessee

However owing to its exp erimental nature the PVM pro ject pro duces software that is

of utility to researchers in the scientic community and to others This software is and

has b een distributed freely in the interest of advancement of science and is b eing used in

computational applications around the world

Related Work

Anumb er of pro jects based on the same principle namely utilizing a collection of inter

connected machines as a concurrent have b een develop ed and several

enjoy widespread adoption and following While there is some commonality with PVM

other similar systems oer sometimes radically dierent programming and implementa

tion mo dels and presentvaried functionality and p erformance The more widely adopted of

these systems are describ ed in detail elsewhere in this volume in this subsection we mention

a few representative systems and comment on dierences in functionality and p erformance

Linda is a concurrent programming mo del based on the concept of a tuplespace a

distributed shared memory abstraction via which co op erating pro cesses communicate P

and its derivativeParmacs are libraries of macros and subroutines develop ed at Argonne

National Lab oratory and GMD for programming a variety of parallel machines They

supp ort b oth the sharedmemory mo del based on monitors and the distributedmemory

mo del using messagepassing Express is a collection of to ols including a message passing

interface for programming distributed memory multipro cessors including network clusters

Various other systems with similar capabilities are also in existence a reasonably

comprehensive listing may b e found in

With the exception of Linda whose programming mo del is not based on conventional

message passing most other systems supp ort very similar facilities the core primitives b eing

systemsp ecic variants of send and receive PVM supp orts dynamic pro cess and virtual

machine management unlike other systems where the pro cess structure is statically dened

The PVM message passing primitives are oriented towards heterogeneous op eration involv

ing strongly typ ed constructs for buering and transmission some other systems provide

for untyp ed data transfer Compared to other systems the suite of in terface primitives

supp orted by PVM is small for example P and Express provide global combining rou

tines and Parmacs supp orts pro cess top ologies These omissions are delib erate to a certain

extent the PVM philosophy is to supp ort a core kernel of primitives ab ove which auxiliary

layers maybeaddedegaPICL p ort to PVM while ensuring that facilities that

can only b e provided at the system level are comprehensive and functionally complete

In terms of p erformance most message passing systems exhibit only marginal dierences

although systematic comparative studies have not b een undertaken In the context of PVM

and similar systems p erformance is usually measured by endtoend communications

sp eeds delivered by the software system and overall execution times of various complete

applications as measured by elapsed wall clo ck times Neither of these parametrizations is

very meaningful however since in typical networked environments a numb er of dynamically

varying external inuences continually aect these measures Given these qualications and

the fact that networkbased systems for the most part utilize the same underlying transp ort

mechanisms p erformance variations tend to b e b oth insignicant and inconsistent In a

later section we present exp erimental measurements of p erformance for the PVM system

and prop ose strategies under investigation to enhance them

PVM Mo del and Features

PVM Computing Mo del

Under PVM a user dened collection of serial parallel and vector emulates

a large distributedmemory computer Throughout this rep ort the term virtual machine

will b e used to designate this logical distributedmemory computer and host will b e used

to designate one of the memb er computers Multiple users can congure overlapping vir

tual machines and each user can execute several PVM applications simultaneouslyPVM

supplies the functions to automatically start up tasks on the virtual machine and allows

the tasks to communicate and synchronize with each other A task is dened as a unit

of in PVM analogous to a Unix pro cess It is often a Unix pro cess but not

necessarily so Applications which can b e written in or C can b e parallelized

by using messagepassing constructs common to most distributedmemory computers By

sending and receiving messages multiple tasks of an application can co op erate to solvea

problem in parallel Figure depicts the PVM computing mo del as wellasanarchitectural

abstraction of the system

The mo del assumes that any task can send a message to any other PVM task and that

there is no limit to the size or numberofsuch messages The PVM communication mo del

provides asynchronous blo cking send asynchronous blo cking receive and nonblo cking re

ceive functions In our terminology a blo cking send returns as so on as the send buer is

free for reuse regardless of the state of the receiver A nonblo cking receive immediately

returns with either the data or a ag that the data has not arrived while a blo cking receive

returns only when the data is in the receive buer In addition to these p ointtop oint

communication functions the mo del supp orts multicast to a set of tasks and broadcast to

a user dened group of tasks The PVM mo del guarantees that message order is preserved

y pair of communicating entities between an

Core Features

PVM supplies routines that enable a user pro cess to registerleave a collection of co op

erating pro cesses routines to add and delete hosts from the virtual machine to initiate

and terminate PVM tasks to synchronize with and send signals to other PVM tasks and

routines to obtain information ab out the virtual machine conguration and activePVM

tasks Synchronization maybeachieved in one of several ways eg by sending a Unix

signal to another task or by using barriers Another metho noties a set of tasks of an Cluster 1

Input & Partitioning Bridge/ Router

Comp 1 Comp 2 SPMD SPMD

MPP

Cluster 2

Cluster 3

Output & Display PVM: Uniform Inter-component comm & sync View of Vector SC Inter-instance comm & sync Multiprogrammed Virtual Machine

(a) PVM Computation Model (b) PVM Architectural Overview

Figure PVM System Overview

event o ccurrence by sending them a message with a usersp ecied tag that the application

can check for The notication events include the exiting of a task the deletion or failure

of a host and the addition of a host

PVM provides routines for packing and sending messages b etween tasks The core

communication routines include an asynchronous send to a single task and a multicast

to a list of tasks PVM transmits messages over the underlying network using the fastest

mechanism available eg either UDP TCP on networks based on the Internet proto cols

or other highsp eed interconnects available b etween the communicating pro cessors One

example of this third option is describ ed in section Messages can b e received by

ltering on source or message tag b oth of whichmay b e sp ecied as wildcards with either

blo cking or nonblo cking receive routines A routine can b e called to return information

ab out received messages such as the source tag and size of the data Message buers are

allo cated dynamically thereby p ermitting messages limited in size only bynative machine

parameters There are routines for creating and managing multiple send and receive buers

This feature allows the user to write PVM math libraries and graphical interfaces that can

b e called inside other PVM applications without communication conicts The user can

switchcontext from one set of buers for example used by the application to another set

of buers for example used inside a math library call

Auxiliary Features

Dynamic pro cess groups are layered ab ove the core PVM routines A pro cess can b elong

to multiple groups and groups can change dynamically at any time during a computation

Routines are provided for tasks to join and leave a named group Group memb ers are

uniquely numb ered from zero to the numb er of group memb ers minus one If gaps app ear

in this numb ering due to tasks leaving the group PVM attempts to ll these gaps with

subsequently joining tasks Tasks can also query for information ab out other group mem

b ers Functions that logically deal with groups of tasks such as broadcast and barrier use

the users explicitly dened group names as arguments

PVM version is designed so that nativemultipro cessor calls can b e compiled into the

source This allows the fast messagepassing of a particular system to b e realized bythe

PVM application Messages b etween twonodesofamultipro cessor use its native message

passing routines while messages destined for an external host are routed via the users

PVM daemon on the multipro cessor On sharedmemory systems the data movementcan

b e implemented with a shared buer p o ol and lo ck primitives The MPP subsystem of

PVM consists of a daemon that manages the allo cation and deallo cation of no des on the

multipro cessor This daemon is implemented in terms of PVM core routines The second

part of the MPP p ort is a sp ecialized libpvm library for this architecture that contains the

fast routing calls b etween no des of this host

Implementation

The PVM system is comp osed of two parts The rst is a daemon called pvmd sometimes

simply pvmd that executes on all the computers making up the virtual machine Pvmd is

designed so any user with a valid login can install this daemon on a machine A user wishing

to use PVM rst congures a virtual machine by sp ecifying a hostp o ol list the daemons

are started on each and co op erate to emulate a virtual machine The PVM application can

then b e started from a shell command line prompt on any of these computers

libpvma This The second part of the system is a library of PVM interface routines

library contains user callable routines for message passing spawning pro cesses co ordinating

tasks and mo difying the virtual machine Application programs must b e linked with this

library to use PVM

The PVM system comp onents have b een compiled and tested on the architectures shown

in table This table includes hosts ranging from laptop computers to Cray Cs

and MPP computers In addition several vendors are supplying and supp orting optimized

versions of PVM for their multipro cessor systems including Cray Research IBM Convex

Intel SGI and DEC

ARCH Machine Notes

AFX Alliant FX

ALPHA DEC Alpha DEC OSF

BAL Sequent Balance DYNIX

BFLY BBN Buttery TC

BSD Unix b ox BSDI

CM Thinking Machines CM Sun frontend

CM Thinking Machines CM

CNVX Convex Cseries

CNVXN Convex Cseries nativemode

DGAV Data General Aviion

CRAY C YMPCray UNICOS

CRAYSMP CraySMP

HP HP mo del HPUX

HPPA HP PARISC

I Intel iPSC link lrp c

IPSC Intel iPSC host SysV

KSR Kendall Square KSR OSF

NEXT NeXT

PGON Intel Paragon link lrp c

PMAX DECstation Ultrix

RSK IBMRS AIX

RT IBM RT

SGI Silicon Graphics IRIS link lsun

SUN Sun SunOS

SUN Sun SPARCstation

SYMM Sequent Symmetry

TITN Stardent Titan

UVAX DEC MicroVAX

Table Architectures tested with PVM

Performance Considerations

PVM and similar systems normally op erate in general purp ose networked environments

where neither the CPUs of the individual machines nor the interconnection network is

dedicated As a result raw p erformance or sp eedup of a given application is hard to

measure Even in a dedicated networked environment with no external use the ab oveis

true since op erating system activity window and lesystem overheads and administrative

network trac can contribute to deviated measurements If these factors are ignored

network computing systems b ehave in a manner that is reasonably predictable In such

a scenario most of the fo cus is on communications overhead CPU optimizations can b e

approached indep endently using traditional metho ds since parallelism granularityisatthe

pro cess level

Network Message Length

Typ e K K K K M

Ethernet

FDDI

Table Data transfer times milliseconds

Op eration No of Pro cs

Typ e

Barrier

Broadcast

Opt Bcast

Table Global op eration times milliseconds

Raw Communication Performance

Given the ab ove factors p erformance evaluation of PVM and similar systems normally

b egins with an analysis of data transfer costs The time required for pro cesses to exchange

messages is dep endentonseveral factors including the host machines network sp eeds and

most predominantly the message size In Table b elow weshow message passing times in

milliseconds for PVM for varying message lengths and two dierent network typ es these

exp eriments were conducted on unloaded workstations rated at MIPS

Two imp ortant observations p ertain to Table The rst concerns latency or the mini

mal time required to send a zerolength message Irresp ective of network typ e this measure

is of the order of a millisecond and dep ends largely on the sp eed of the host machines as

a signicant fraction of this overhead is incurred in withinhost proto col pro cessing The

second factor is throughput As the table shows the Ethernet network could b e driven

at near theoretical p eak capacity for large messages similar ratios are conjectured to b e

p ossible for fast networks with increases in host sp eeds and proto col optimizations

Apart from p ointtop oint data transfer group communication facilities are also an

imp ortant measure of communications p erformance Table shows times in milliseconds

for barrier synchronization and broadcasting K messages using the release version of

PVM that uses naive but robust algorithms as yet untuned for optimality Also shown

in Table is an exp erimental multicast K messages facility that exploits the broadcast

medium of networks such as Ethernet

Improving Communications Performance

As shown in the previous section communication throughput approaching the medium ca

pacity can b e achieved in PVM provided large messages are transferred This has also b een

demonstrated in situations involving multiple simultaneous message passing ie a high

p ercentage of the aggregate bandwidth of the medium is utilizable by PVM application pro

cesses The factor that is dicult to optimize however is latency implying p o or eciency

and sp eedup when message exchanges are short and intermittent

One approach that is under investigation is to enable PVM op eration directly ab ovethe

data link layer rather than the transp ort layer a feasible option when op erating in a lo cal

network environment An exp erimental version of PVM that op erates in this fashion has

b een designed and latency improvements of the order of have b een observed Work

is in progress to integrate this into the release version of PVM incorp orating intelligence

to determine on a sendbysend basis when it is appropriate to utilize this data transfer

mechanism

Scientic Sup ercomputing

PVM is b eing increasingly adopted at numerous institutions worldwide for distributed sci

entic computing Many scientic industrial and medical applications are b eing deployed

under PVM in clustered workstation networks An imp ortant motivation for the use of

PVM and other cluster computing systems is price p erformance generally clusters are

ab out times as costeective as sup ercomputers for a given p erformance capability for

Other motivations for the increasing use of PVM include several classes of applications

a high degree of p ortability and a straightforward robust interface that is well suited for

scientic application development

Three computational grand challenges b eing addressed byORNLaswell as several

other applications imp ortant to DOEs mission havebeenconverted to PVM The grand

challenges are in groundwater transp ort to assist in waste site clean up rst principles

materials calculations to assist in the design of new alloys and ceramics and global climate

mo deling to predict the eects of things such as ozone depletion and global warming In

the next sections we briey describ e these three applications and the eort involved in

converting them to PVM

Groundwater

ORNL is part of a consortium of groundwater researchers whose goal is to develop state of

the art parallel mo dels for high p erformance parallel computers These computer mo dels

will enable researchers to mo del ow with higher resolution and greater accuracy than

previously p ossible As a rst step researchers at ORNL have develop ed a parallel D

nite element co de called PFEM that mo dels water ow through saturatedunsaturated

media PFEM solves the system of equations

h

rK K rh rz q F

s r

t

where h is the pressure head t is time K is the saturated hydraulic conductivity tensor

s

K is the relativehydraulic conductivity or relative p ermeability z is the p otential head q

r

is the sourcesink and F is the water capacityF d dhwith the moisture content

after neglecting the compressibility of the water and of the media

PFEM was parallelized by partitioning the physical domain into p pieces and statically

assigning one sub domain to eachofp tasks The presentversion uses only static load

balancing and relies on the user to dene the partitioning but other consortium memb ers

are working on ways to automate these op erations Ateach timestep each task solves

the ab ove equation for its sub domain and then exchanges its b oundary region with its

neighb oring regions Originally develop ed on an Intel iPSC multipro cessor a PVM

version of PFEM was straightforward to create requiring an undergraduate student less than

ered to memb ers weeks to complete Presently the PVM version of PFEM has b een deliv

of the groundwater consortium for validation testing using networks of workstations while

they await the availability of parallel sup ercomputers No p erformance tests had b een done

at the time of this writing

Materials

ORNL material scientists are developing algorithms for studying the physical prop erties

of complex substitutionally disordered materials A few imp ortant examples of physical

systems and situations in which substitutional disorder plays a critical role in determin

ing material prop erties include highstrength alloys hightemp erature sup erconductors

magnetic phase transitions and metalinsulator transitions One of the algorithms b eing

develop ed is an implementation of the Korringa Kohn and Rostoker coherent p otential ap

proximation KKRCPA metho d for calculating the electronic prop erties energetics and

other ground state prop erties of substitutionally disordered alloys The KKRCPA metho d

extends the usual implementation of density functional theory to substitutionally disordered

materials In this sense it is a completely rst principles theory of the prop erties of substi

tutionally disordered materials requiring as input only the atomic numb ers of the sp ecies

making up the solid

Starting with the original line serial KKRCPA co de it required ab out three

months to pro duce a PVM version of the co de After proling the co de and studying

the p otential sites for coarsegrain parallelism a masterslave paradigm was chosen for

implementation The master task p erforms all the IO and co ordinates all the slaves The

slave tasks p erform the ma jority of the computational work This split reduced the amount

of memory any one task required and also allowed the master task to do dynamic load

balancing Several megabytes of data is transferred b etween master and slaves but no data

is shared b etween slaves Moreover the data transfers are organized as a few large messages

rather than many small ones to reduce message latency overhead

Using PVM the KKRCPA co de is able to achieveover Mops utilizing a network of

ten IBM RS mo del s mo del s workstations estimated to b e ab out

of the maximum achievable for this co de Given this capability the KKRCPA application

is b eing used as a researchcodetosolve imp ortant materials science problems Since

its development the KKRCPA co de has b een used to compare the electronic structure

of two high temp erature sup erconductors BaBi Pb O and Ba K BiO to explain

  

anomalous exp erimental results from a high strength alloy NiAl and to study the eect

of magnetic multilayers in CrV and CrMo alloys for their p ossible use in magnetic storage

devices

The PVM KKRCPA co de has also b een used to test concepts in

For example with help from Cray Research the KKRCPAcodewas run on a network of

C and YMP multipro cessors Using pro cessors scattered across several sites the Cray

based PVM was able to achieveanaverage aggregate p erformance of over Gops while

calculating sup erconductor prop erties In a test of the p ortabilit y of PVM the KKRCPA

co de was run across a virtual machine comp osed of twoIntel Paragons a CM an Intel

i and IBM workstations These hosts are geographically distributed at several sites In

this test the master task ran on one of the IBM workstations and slave tasks ran on no des of

the various MPP hosts The p erformance of this test was consistent with the small number

of no des used on eachhost

Climate

A collab oration of researchers from ORNL Argonne National Lab and the National Center

for Atmospheric Research NCAR was formed to address atmospheric mo deling Their rst

task has b een to develop parallel algorithms and implement the recently develop ed version

of the NCAR Community Climate Mo del CCM in a message passing version for Intel

parallel sup ercomputers This will b e one of the rst co des to run on the Intel Paragon and

ORNL will supp ort its use by the climate research community using the Paragon Work

now in progress seeks to improve on the parallel eciency of the co de and to develop more

comprehensive mo dels with improved capabilities

The climate mo del solves the nonlinear PDEs for mass momentum and energy which

govern the general circulation of the atmosphere Horizontal advection couples columns

of the atmosphere while in the vertical direction a large numb er of pro cesses are coupled

On the shortest time scales the interaction of radiation with the earths surface clouds

and absorption by the atmosphere couples a vertical column Surface moisture latent heat

exchange convectiveoverturning and precipitation pro cesses are also represented within

eachvertical column The o cean surface temp erature and the ability of the o cean surface

layer to store heat during a diurnal cycle are represented without including general o cean

circulation

Since a large p ortion of the calculations radiation absorption clouds etc in a vertical

column of atmosphere are indep endent the parallelization strategy used in the climate

co de is across columns The columns of atmosphere are divided into adjacent patches

and distributed among the pro cessors of an MPP These indep endent calculations comprise

ab out of the parallel execution time of the co de but exhibit some load imbalance

The coupling in the horizontal direction the solution of the ow equations uses a parallel

sp ectral transform algorithm for the approximation of horizontal derivatives and a semi

Lagrangian treatment of the advective term for the moisture equation

The PVM version was created from a working parallel message passing co de running

on the Intel iPSC in a couple of weeks The message passing calls were replaced with

equivalent PVM calls Some sp ecial functions that takeadvantage hyp ercub e or mesh

connectivitywere replaced with simpler PVM routines For example a global maximum is

calculated on a single pro cess Similarly all the IO is p erformed by a single pro cess The

climate researchers rep ort that PVM has oered an excellent debugging and development

to ol as well as p ortability across machines and networks

Discussion

In this pap er wehave attempted to presentoverview descriptions of some of the more

interesting and imp ortant facets of the PVM system including the design philosophycom

puting mo del p erformance issues and application exp eriences Detailed exp ositions as well

as p edagogical material on various asp ects of the PVM system may b e found for example

in in addition to the pap ers already cited At the time of writing PVM contin

ues to b e a p opular and widely used system for concurrent computing and it is exp ected

that the pro ject will mature and evolveeven further in the future In this section we briey

discuss ongoing auxiliary pro jects future plans and comment on the long term p otential

and scop e of PVM and similar technology

Ongoing and Future Work

As mentioned PVM is an ongoing exp erimental research pro ject and continually evolving

new ideas are investigated b oth by the pro ject team and at external institutions successful

exp erimental enhancements or subsystems eventually b ecome part of the software distri

bution One example of a relatively concise enhancement that is undergoing investigation

concerns system level optimizations for op erating in shared memory environments Small

scale SMMs are reemerging and a version of PVM that utilizes physical shared memory

for interaction b etween the daemon and all user pro cesses on suchmachines is b eing de

velop ed Another pro ject is aimed at providing failsafe capabilities in PVM This

enhanced version uses checkp ointing and rollback to recover from singleno de failures in

an applicationtransparent manner provided the application is not dep endent on realtime

events Several other enhancements are also in progress includin g load balancing exten

sions integrating debugging supp ort and task queue management

One somewhat dierent and more extensive subsystem under development is a gen

eralized distributed computing GDC layer for PVM While scientic applications have

provided the technical imp etus for the PVM pro ject more general and commercially ori

ented uses are nowevolving In order to supp ort such applications the GDC layer is b eing

designed and will supp ort parallel inputoutput with enhanced lesystem semantics for

shadowing interleaved access and rollback access control and authentication and ab

stract mechanisms supp ort for the clientserver mo del of distributed

computing with facilities for the transparent exp orting and invo cation of services and

distributed transaction pro cessing primitives Early results indicate that these facilities

can b e provided at high levels of eciency and that the enhanced functionality will prove

b enecial for many new classes of application domains

Auxiliary Pro jects

Apart from the exp erimental work describ ed in the previous section a numb er of other

pro jects related to PVM are in various stages of progress by nature these are less inter

twined with the internals of the core system and to a large extent are auxiliary to ols or

tioned b elow external subsystems A few are men

HeNCE is a graphical programming system for PVM this to olkit generates PVM

programs from depictions of parallelism dep endencies as directed graphs and provides

an interactive administrativeinterface for virtual machine conguration application

execution and animated visualization

Xab is a graphical to ol for the run time monitoring of PVM programs It gathers

monitoring events from applications and displays this information which can b e

useful for proling error detection and optimization

The DoPVM subsystem is aimed at supp orting the shared ob ject paradigm

in PVM By writing C programs in which ob jects derived from builtin classes

can b e declared this PVM extension p ermits a shared address space concurrentcom

puting mo del thereby alleviating the inherent complexity of explicit message passing

programming

Longterm Outlo ok

It is imp ortant to realize that PVM and other similar systems is not merely a software

framework for networkbased concurrent computing it is an integrated metho dology for

concurrent distributed and parallel pro cessing and more imp ortantlyitisaninterface

denition for p ortable application development From the p ortability p oint of view PVM

applications may b e migrated not just from one machine parallel or serial to another but

across dierent collections of machines Given that PVM can op erate within a parallel

machine across networks and combinations thereof signicant generality and exibili ty

exists and it is hard to imagine a computing environment where such a mo del would not be

appropriate From the p oint of view of p erformance PVM delivers a signicant prop ortion

of the order of of the capacityavailable from the underlying hardware op erating

system network and proto cols and we exp ect to retain this characteristic as network and

CPU sp eeds increase and as proto col and OS software b ecomes more ecient It should

b e p ointed out while on this topic that in measuring the worth of systems such as PVM

comparisons of clusters vs MPPs are inappropriate rather a meaningful metric is the

value added by PVM to a given hardware environment whether this is a multipro cessor or

a collection of workstations In terms of functionality the PVM system currently supp orts

an adequate suite of features and with the integration of extensions describ ed earlier will

b e in a p osition to cater to a much larger realm of distributed and concurrent applications

Heterogeneous networkbased concurrent computing systems like PVM are therefore likely

to remain viable technologies for concurrent and distributed computing

References

Adam Beguelin Xab A To ol for Monitoring PVM Programs Workshop on Hetero

geneous Processing IEEE Computer So ciety Press Los Alamitos California pp

April

H J Siegel et al An Overview of the PASM Parallel Pro cessing System in it

Computer Architecture D D Ga jski et al eds IEEE Computer So ciety Press

Washington DC pp

V S Sunderam PVM A Framework for Parallel Distributed Computing Journal

of Concurrency Practice and Experience pp December

A Beguelin et al HeNCE Users Guide UniversityofTennessee Technical Rep ort

May

N Carriero and D Gelernter Linda in Context Communications of the ACM 

pp April

R Butler and E Lusk Users Guide to the P Programming System Argonne Na

tional Lab oratoryTechnical Rep ort ANL

A Kolawa The Express Programming Environment Workshop on Heterogeneous

NetworkBased Concurrent ComputingTallahassee Octob er

Louis Turcotte A Survey of Software Environments for Exploiting Networked Com

puting Resources Draft Rep ort Engineering ResearchCenter for Computational Field

Simulations Mississippi State January

G A Geist M T Heath B W Peyton and PHWorleyAmachineindep endent

communication library In J Gustafson editor The Proceedings of the Fourth Confer

enceonHypercubes Concurrent Computers and Applications pp PO Box

Los Altos CA Golden Gate Enterprises

G A Geist and V S Sunderam Network Based Concurrent Computing on the PVM

System Journal of Concurrency Practice and Experience  pp June

B Schmidt and V S Sunderam Empirical Analysis of Overheads in Cluster Envi

ronments Journal of Concurrency Practice and Experience to app ear

A Beguelin J J Dongarra G A Geist R Manchek and V S Sunderam A Users

Guide to PVM Parallel Virtual Machine Technical Rep ort ORNLTM Oak

Ridge National Lab oratory July

A Beguelin J Dongarra G Geist R Manchek and V Sunderam Solving Com

putational Grand Challenges Using a Network of Sup ercomputers Pro ceedings of the

Fifth SIAM ConferenceonParal lel Processing D Sorensen ed SIAM Philadelphi a

G A Geist and V S Sunderam The Evolution of the PVM Concurrent Computing

System Proceedings th IEEE Compcon Symposium pp San Fransisco

February

J Leon et al Fail Safe PVM A Portable Package for Distributed Programming with

Transparent Recovery Scho ol of Technical Rep ort CarnegieMellon

University CMUCS February

J Dongarra et al Abstracts PVM Users Group Meeting UniversityofTennessee

Knoxville May

C Hartley and V S Sunderam Concurrent Programming with Shared Ob jects in

Networked Environments Proceedings th Intl Paral lel Processing Symposiumpp

Los Angeles April