Safer Tuple Spaces

1 2 3

Ro el van der Go ot Jonathan Schaeer and Gregory V Wilson

1

Erasmus University Rotterdam The Netherlands vandergo otfeweurnl

2

University of Alb erta Canada jonathancsualb ertaca

3

Visible Decisions Canada gregvizbizcom

Abstract The simplicity and elegance of the Linda programming mo del

is based on its single global typ eless However these virtues

come at a cost First the tuple space can b e an imp ediment to scalable

high p erformance Second the black b ox nature of the tuple space

makes it an inherently dangerous data structure prone to many typ es

of programming errors

Blossom is a C version of Linda with extensions This pap er intro

duces some of the novelties in Blossom as they p ertain to creating safe

tuple spaces These new features include multiple strongly typ ed tuple

spaces eld access patterns tuple space access patterns and assertions

Intro duction

The computing literature is replete with programming mo dels for writing par

allel programs Many of them are research prototyp es that are go o d enough

for a few academic pap ers and then quietly disapp ear Few mo dels attract sus

tained interest in the literature or establish a large user community Linda

is one of the rare parallel programming mo dels that has managed to survive for

over a decade in spite of the discriminating and critical tastes of the parallel

computing community

Lindas strength is the simplicity of its programming mo del In eect Linda

oers a blackb oard the tuple space that pro cesses can read from write to and

erase p ortions of To supp ort Linda languages such as C C and Fortran

need only have a library implementing a few calls to access the tuple space

Thus a simple implementation is easily built More sophisticated versions of

Linda exist relying on compiler techniques to understand the parallelism in the

application and optimize accordingly

Although the uniqueness of the Linda approach excited the parallel comput

ing community in the late s with time the novelty waned In part this was

b ecause Linda b ecame a commercial pro duct with the resulting licensing fees

and free versions suered in p erformance Nevertheless there continues to b e an

active Linda user community

Although there are many advantages to the Linda mo del simplicity ease of

integration in a language ease of expressing communication and synchroniza

tion there are two ma jor criticisms First and foremost the tuple space is often

seen to b e a b ottleneck and an imp ediment to high scalable p erformance For

many applications these p erformance concerns eectively rule out Linda as a

viable to ol Second an often overlo oked p oint is that the tuple space has some

dangerous design aws that can give rise to serious programming errors The

tuple space is a generic shared data structure that can b e viewed as a black b ox

one can read and write to it but an application cannot see inside it It is legal

to put anything into the tuple space including semantically invalid data Fur

thermore there is no way to p eek into this black b ox to see its internal structure

and check for errors Thus to make Linda a more attractive parallel program

ming mo del two things must b e addressed improved program p erformance and

making the mo del safer in the sense that the likeliho o d of programming errors

is reduced

This pap er intro duces Blossom an extended version of Linda for the C

programming language designed to address the ma jor concerns of Linda This

pap er describ es some of the innovations in Blossom that inuence the structure

of the tuple space By appropriately sp ecifying tuple spaces the probability

of intro ducing a programming error into a parallel Blossom program is greatly

reduced Parallel programs are notoriously dicult to design implement test

and debug Anything that can b e done to shorten the program development

cycle is welcome Although the enhancements describ ed in this pap er address

the software engineering concerns of generating correct programs most of them

also can b e used to improve program p erformance

Since CLinda and FortranLinda made their debut there has b een a ma jor

shift in programming language trends Strong typing and class libraries as in

C and have increased the expressive p ower available to the programmer

In particular these features and C templates allow us to achieve many of the

capabilities of a Linda compiler using just a standard C compiler without

intro ducing new syntax or changing the semantics of the programming language

The Linda mo del can easily b e integrated into an ob ject oriented program

ming language Linda implementations for Eiel and Java require

the user to explicitly create ob jects of class Tuple In C Linda a precom

piler is required to translate new syntax into C Our Blossom system neither

needs explicit creation of ob jects of class Tuple nor do es it use a precompiler

Section describ es the Linda mo del and intro duces some of the prop osed

mo del enhancements in the literature Section describ es four innovations in

Blossom The pap er is restricted to discussing the design of the tuple space

since these enhancements are currently implemented and working Section il

lustrates programming using Blossom Finally Section discusses Blossom work

in progress including the p erformance enhancements

Linda

Linda was intro duced in Although the Linda mo del was not in its nal

form it contained the idea of a global memory accessible by multiple pro cesses

The global memory contains a collection of data records called tuples and it is

accordingly called the tuple space Tuple space data is organized as an asso ciative

memory meaning that data is retrieved by its values not by an index

Linda provides six simple op erations to access the tuple space Op eration in

gets a tuple from the tuple space in the program out puts a tuple out of the

program into the tuple space and rd replicates a tuple The rd op eration is

semantically equivalent to an in immediately followed by an out combined in

one atomic action

The following example illustrates how a program might manipulate the tuple

space An outtodo puts a tuple with two elds into the tuple space

the rst eld is a string with value todo and the second eld is an integer with

value A variable with value as the second eld would have the same result

An intodo will get the same tuple out of the tuple space if it is

present If there is no tuple with values equivalent to all the arguments the

op eration will blo ck until another pro cess puts a matching tuple in the tuple

space Wild cards can b e used to enhance the capabilities of the op erations that

read tuples For example

int i

intodo i

matches any tuple with two elds the rst b eing a string with value todo and

the second an integer with any value Such a wild card eld i is called a

formal parameter as opp osed to an actual parameter a value eld Tuples that

contain formal elds are sometimes referred to as antituples

Linda provides an eval op eration for spawning new pro cesses For example

evaltodo sqr will spawn two pro cesses each evaluating one of the

arguments in parallel with the spawning pro cess As so on as all the arguments

of the eval are evaluated the resulting tuple is put into tuple space the tuple

todo in this case

The op erations inp and rdp are nonblo cking versions of in and rd resp ec

tively They query the tuple space for a match and return a b o olean indicating

whether they succeeded in that If no match is found the op erations will not

wait ie blo ck for another pro cess to insert a matching tuple

Because the mo del is so simple and elegant it is easy to nd fault with

it Consequently numerous extensions to the mo del have b een prop osed in

the literature Some of the more interesting ones include having multiple tu

ple spaces more p owerful tuple space op erations such as collect and

copycollect sp ecifying access patterns on tuples p ersistent tuple

spaces faulttolerant tuple spaces and op en Linda In this pap er

we will use the extension of multiple tuple spaces

This pap er discusses Blossom a Cbased implementation of Linda with

extensions Since we are using only the C compiler and not a Linda compiler

the syntax of Blossom diers slightly from that of Linda In Blossom tuple spaces

b ecome ob jects and the op erations on tuple spaces b ecome memb er functions

The original Linda op erations are relative to the program the Blossom memb er

functions are relative to the tuple space Hence new names for the op erations had

to b e chosen out b ecomes put in b ecomes get rd b ecomes copy inp b ecomes

get nb and rdp b ecomes copy nb where nb indicates the call is nonblo cking

The eval op eration is handled as a sp ecial case of put

eval sqr

b ecomes

ts is the name of a userdefined tuple space

tsput evalsqr

indicating that only one pro cess is created to evaluate the square of

Formal parameters are handled dierently as well

int i

in i

b ecomes

Argint i

tsget ivar

where Argtype is a Blossom class that enhances the type to include additional

metho ds The var metho d allows the variable to b e used as a wild card

Blossoms tuple spaces are rst class ob jects which means that it is p ossible

to have tuple spaces of tuple spaces

Enhancements

The tuple space is a typ eless black b ox The user can put data of any typ e in

it and attempt to extract data of any typ e from it But the wealth of program

ming language design exp erience suggests that this is a bad idea The trend in

computing to day is towards strong typing We have seen the evolution of BCPL

to C to C motivated in part by the b enets of strong typing These b enets

are twofold catching more errors at compiletime and preventing some classes

of runtime errors Given the inherently dicult nature of parallel programming

any language enhancements that reduce the probability of error are welcome

In this section four new enhancements to the basic Linda mo del are pre

sented All of them have to do with either preventing or detecting parallel pro

gram errors at compile or runtime This is achieved by changing the denition

of the tuple space it is given structure b ehavior and is made accessible see

Figure The user provides annotations that give the compiler and runtime

system information as to the exp ected b ehavior of the program and forces it to

meet those constraints

Blossom is a C class library Unlike CLinda and FortranLinda it do es

not add new syntax alter the semantics or extend the host language each of

which tends to result in programmer confusion C is a mo dern program

ming language with a rich set of features This enables Blossom to achieve many

a b c

Fig Dierences b etween tuple spaces a Original Linda tuple space It is a single

collection of tuples of arbitrary structure This implies that the user can try to in

sertremove semantically incorrect tuples The user cannot lo ok inside the tuple space

b Strongly typ ed tuple space Each tuple space can contain only one typ e of tu

ple Hence a program cannot inadvertently insertremove a wrongly typ ed tuple c

Strongly typ ed tuple space with assertions It provides the user the additional b enet

of b eing able to p eek inside the tuple space

of its ob jectives including preventing programming errors using only C syn

tax without the need for a separate precompiler To achieve the same b enets of

a Linda precompiler including error checking and runtime optimizations the

user has to sp ecify a small amount of additional detail in their program This is

reected in the Blossom examples given later They are slightly more verb ose

than their pureLinda counterparts

One issue not addressed in this pap er is how to handle runtime errors gen

erated by the new Blossom enhancements Since we are using C a natural

solution is to throw an exception whenever a runtime error o ccurs The user

can at their discretion catch the exception and handle it Although this sounds

go o d it do es have its problems For example do child pro cesses throw exceptions

in their parents What if a pro cess parent is no longer around but its grand

parent is Right now all runtime errors result in ab orting the application

Strongly Typ ed Tuple Spaces

Lindas single tuple space can contain tuples of any typ e Although this sounds

conceptually simple it has the disadvantage that it is easy to inadvertently

intro duce a bug in a program Accidentally omitting a tuple eld intro ducing

an additional eld changing a eld or reversing two elds is p erfectly legal

yet may result in a semantically incorrect program A sophisticated precompiler

may b e able to detect some but not all of these errors

By having multiple tuple spaces each strongly typ ed these problems cannot

o ccur Each tuple space is created with a sp ecication that gives the numb er

order and typ e of each eld of the tuples legally allowed much like the param

eters of a subroutine call With this sp ecication many commonly o ccurring

errors can b e caught at compile time without the need for a Linda precompiler

The advantages of strong typing can b e summarized as follows

Eliminate Errors A strongly typ ed tuple space can eliminate many common

errors

Performance Although this pap er emphasizes the software engineering as

p ects of Blossom it is imp ortant to realize that many of these enhancements

can also b e used to improve program p erformance In eect the typing par

titions the tuple space into multiple disjoint address spaces Hence given a

tuple of a particular typ e the system knows exactly in what address space

to lo ok for the tuple

The following example illustrates having multiple stronglytyp ed tuple spaces

in Blossom The TupleSpace declarations each create new named tuple spaces

and sp ecify the typ e of the tuples that are allowed inside them

TupleSpaceTupleint int square

TupleSpaceTupleint float sqrroot

Put values in the tuple spaces

for int i i MAX i

squareputi ii

sqrrootputi sqrti

Take values out of the tuple spaces

Argfloat answer

sqrrootget answervar Succeeds

squareget Succeeds

squareget Blocks no such tuple available

squaregetone Compiletime type error

squareget Compiletime type error

squareget Compiletime type error

Field Access Patterns

The philosophy b ehind typing a tuple space can b e carried even further A

common programming scenario in Linda is to extract tuples using one of the

elds as a discriminator For example one might have a pro ducerconsumer

computation where the tuples pro duced have to b e pro cessed in order Every

tuple contains a unique numb er which is more or less a time stamp The smaller

the numb er the older the tuple The consumer gets the next tuple by increasing

the time stamp eld In this case this eld is not only an integer but it is always

used as a value not a wild card variable

Every eld in a tuple space declaration contains a declaration as to whether

it is legal to use a wild card variable to retrieve a value or not In other words

a eld is designated as either Actual or Any Actual indicates that a tuple eld

can only b e retrieved by sp ecifying a value while Any extends this to include

wild card variables

This additional piece of information oers the user several advantages

Eliminate Errors By providing additional information ab out the intended

b ehavior of the tuple space the compiler can check whether the programmer

always satises the sp ecied requirements

Performance Sp ecic elds can b e used to determine by hashing on which

machine to store or lo ok for a tuple Another metho d to distribute the tu

ple space is by broadcasting every change of the tuple space to all Linda

pro cesses Hashing has the advantage that broadcasts are not necessary to

distribute the tuple space Instead a single communication is enough to put a

tuple in a remote tuple space and only two communications are needed to get

or to copy a tuple from a remote tuple space the rst sends the get or copy

request to the remote tuple space the second one when a matching tuple

is found and how the formals are instantiated An evenly distributed tuple

space results in less tuples to b e searched and removes p ossible b ottlenecks

of a big tuple space on one machine

The following example illustrates eld access patterns

TupleSpaceTupleActualint Anyint square

Argint sq

for int i i MAX i

squareputi ii

squareget sqvar Succeeds

squareget Legal but blocks

squaregetsqvar Compiletime error

Tuple Space Access Patterns

The ob jectoriented community has recently embraced the concept of design pat

terns generic frequently o ccurring programming paradigms Similar ideas

have played a prominent role in parallel computing with common parallel struc

tures such as masterslave and pip elines Recent studies have shown that there

1

are commonly o ccurring access patterns on ob jects As part of program de

sign the user knows the intended access patterns on their ob jects The rst two

columns of Table gives these access patterns and their description as dened

in the Munin system

The access patterns on ob jects are easily translated to access patterns on

tuples In Blossom the access patterns are applied to tuple spaces instead of

tuples There are several reasons for doing this First in a go o d design every

tuple space has a unique purp ose All memb ers of that tuple space are there

to fulll the design goals of the tuple space and should b e handled similarly

Tuples with dierent access patterns in one tuple space are an indication of

a p o or design Second access patterns can b e used to inuence the hidden

implementation of the tuple space aecting program p erformance For example

the choice of design pattern can eect the placement of the tuple space the

distribution of the tuple space and whether data should b e replicated

1

The original name is typ esp ecic coherence mechanisms

If a tuple space were annotated with an access pattern sp ecication then

that might imp ose a constraint on how the tuple space is accessed at runtime

as shown in the third column of Table Note that not all access patterns

result in constraints on the tuple space Readmostly writemany and general

readwrite all allow an arbitrary combination of readswrites to the tuple space

The only dierence is the relative frequency with which the readswrites o ccur

Hence these are to o general to allow constraints to b e imp osed

Table Access patterns for tuple spaces

Access Pattern Description Restrictions

writeonce written during initialization no puts after a get or copy

afterwards only read

private lo cal accesses only only one pro cess is allowed

access

writemany frequently mo died

result not needed until all data is

collected

synchronization semaphores no copy or copy nb allowed

migratory accessed by a single pro cess at a

time

pro ducerconsumer written by one pro cess consumed only one pro cess is allowed

by other pro cesses to put

readmostly read far more often read than

written

general readwrite general access pattern

There are a numb er of imp ortant advantages for having the user declare the

access pattern of a tuple space

Eliminate Errors If the access pattern of a tuple space do es not match

the access pattern in the users co de then warnings or errors can b e given

at compile and runtime For example if the access pattern is pro ducer

consumer and there are two or more pro cesses putting tuples in the tuple

space the program can detect this at runtime Not at compiletime b ecause

we do not use a precompiler If the program has to run eciently this run

time check can b e turned o

Performance A programmer usually knows the intended use of a tuple space

Dep ending on these intentions a distribution strategy could b e selected by

the user For example a pro ducerconsumer tuple space is b est lo cated in

the data space of the pro ducer pro cess It do es not make sense that the

pro ducer rst distributes the tuples over several pro cesses and next the

consumer gathers the tuples from all these pro cesses The former approach

needs less communications to ship the data to the place it is used consumed

and is thus faster

Awareness of access patterns results in b etter designs A go o d design never

uses an ob ject for more than one reason This general rule applies to tuple

spaces to o Access patterns can help the programmer distinguish dierent

functionalities more easily

It is imp ortant to realize that the access pattern attribute only changes the

semantics of a tuple space if a restriction is encountered if this is not the case

access pattern attributes only change the implementation and not the semantics

The following example illustrates the use of access patterns

TupleSpaceTupleAnyint ProducerConsumer buffer

PRODUCER

while

int number

bufferputnumber Succeeds

CONSUMER

while

Argint number

buffergetnumbervar Succeeds

bufferput Runtime error

Tuple Space Assertions

The simplicity of the Linda programming mo del comes in part from the capa

bilities of the tuple space it is a black b ox with all the structure and implemen

tation details hidden from the user However when lo oking for a bug you want

to b e able to lo ok inside the black b ox This cannot b e done in Linda unless

you extract each of the tuples one at a time and then put them back

Tuple space assertions allow the user to p eek into the tuple space A problem

with assertions in parallel programming is that several pro cesses have access

to the same tuple space This means that if you put a tuple in the tuple space

and then for example immediately assert that the numb er of tuples in the tuple

space is greater than zero the assertion can fail b ecause another pro cess removed

the tuple So either you have to make the assertion general ie it takes other

pro cesses into account or the assertion should form an atomic action with the

other Linda op erations We chose the latter kind of assertion b ecause it b etter

meets the needs of concurrent debugging

Assertions act on an entire tuple space which may b e very inecient if the

tuple space is distributed This is mainly b ecause the whole tuple space should

b e lo cked during the evaluation of an assertion no other pro cesses are allowed

to change its state Assertions on tuple spaces should only b e used to debug

a program However after xing the bug you want to b e able to recompile the

program without removing the assertions again Inclusion of another library a

compiletime option can achieve this

If the user wants to dene an assertion on a tuple space he has to derive a new

class from class Assertion In this class he can dene two metho ds foreach

and exit The constructor call initializes the ob ject the foreach call is

p erformed for every tuple in the tuple space and the exit call should return

a b o olean indicating whether the assertion succeeded true or failed false

If the exit call returns false the program terminates Blossom provides the

hidden co de to ensure that foreach and exit are called correctly

Assertions are illustrated in the next section

Example

The following example illustrates the usage of the concepts intro duced in this

pap er A distributed sieve of Eratosthenes calculates all primes smaller than a

certain upp er b ound LIMIT The function is prime decides whether a numb er

the rst argument is a prime or not by trying to divide the numb er by all

smaller primes passed in the tuple space in the second argument The strongly

typ ed tuple space checks at compiletime that all accesses here put and copy

have a tuple of the correct typ e put only allows actual elds and copy only

allows the sp ecied elds The SizeAssertion checks that the numb er of tuples

in the tuple space is correct after all the primes are calculated

include iostreamh

include tsh

typedef TupleSpaceActualint Anybool PrimesTS

const int LIMIT

class SizeAssertion public Assertion f

int size

int expectedsize

public

SizeAssertionint s expectedsizes size fg

SizeAssertion fg

void foreachint bool f

size

g

bool exit f

return size expectedsize g

g

bool isprimeint number PrimesTS primes f

int limit sqrtnumber

for int i i limit i f

Argbool prime

primescopy i primevar

if prime numberi

return false

g

return true

g

void main f

PrimesTS primes

Argbool prime

for int i i LIMIT i f

primescopy i primevar

if prime

cout i endl

g

g

Future Work

The four enhancements discussed in this pap er have b een implemented and are

working All four help the user eliminate andor detect common parallel pro

gramming errors By making the tuple space safer the user will sp end less time

debugging and they can invest more time in the design of their program

Regrettably it is imp ossible for us to quantify the b enets of the safe tuple

spaces of Blossom There is no easy metric such as sp eedup to compare against

A fair evaluation would require users to implement solutions using CLinda and

Blossom and then compare the time it takes to correctly implement their solu

tions Although we have exp erience p erforming these typ e of exp eriments

it is to o early in the lifecycle of Blossom to go to this eort Once we have com

pleted the full Blossom implementation we will b e very interested in quantifying

the inuence of safer tuple spaces on the program design and implementation

time

Blossom is an evolving system The following p erformance enhancements are

in various stages of completion

Futures Arguments to get and copy can b e futures Futures allow asyn

chronous versions of the aforementioned op erations

Getup dateput op eration The equivalent of a getup dateput op eration in

Linda results in three communications send get receive result up date

value and put up date By combining the op erations a reduction of one

communication can b e achieved send get up date value and put and

receive result

Reduction op erations Without a reduction op eration reducing a tuple space

requires the user to get each tuple and p erform the reduction resulting in a

p otentially large amount of communication A reduction op erator p erforms

the reduction according to the owner computes paradigm There is a strong

similarity b etween tuple space assertions and reduction op erations b oth

require access to the entire tuple space in an ecient manner The user can

sp ecify the reduction op erator and let Blossom do the rest

Conditionals Sending a conditional to the tuple space indicating what tuples

you are interested in can reduce communication and improve parallelism

Blossom can apply the conditional to all tuples in a tuple space and return

those that match The alternative in Linda is to get tuples until you nd

one that meets your conditions and then put back all unneeded tuples

Tuple space attributes Additional information can b e asso ciated with a tu

ple space Ordered vs Unordered Tuple spaces in which the tuples are or

dered sorted are b etter for database applications faster searching but

unordered tuple spaces generally have b etter p erformance Fair vs Unfair

In a fair tuple space every matching tuple has an equal opp ortunity of b eing

selected Unfair tuple spaces have biases but may have more ecient im

plementations Persistence Persistent tuple spaces are kept in les which

means that they are still present after a program terminates Databases are

in general p ersistent Size Allowing the user to sp ecify the maximum size

of a tuple space allows Blossom to more eciently organize the data

We exp ect to have complete working system by the end of the year

Acknowledgments

We would like to thank Nederlandse Organisatie vo or Wetenschapp elijk On

derzo ek NWO National Sciences Engineering Research Council of Canada

NSERC and IBM Toronto Labs for their nancial assistance Furthermore

we thank Josee La joie IBM for her patience answering numerous ANSI C

questions Bill OFarrell IBM for sharing some of his knowledge on C Steve

MacDonald University of Alb erta and Arie de Bruin Erasmus University Rot

terdam for their comments on the article

References

Bakken D E and Schlichting R D Supp orting FaultTolerant Parallel

Programming in Linda IEEE Transactions on Paral lel and Distributed Systems

Bennett J K Carter J B and Zwaenepoel W Munin Distributed

Shared Memory Based on Typ eSp ecic Memory Coherence In Second ACM

SIGPLAN Symposium on Principles Practice of Paral lel Programming PPoPP

pp

Brown T Jeong K Li B Talla S Wyckoff P and Shasha D

PLinda User Manual httpmervcsnyuedubinliplindamanualps

Butcher P Wood A and Atkins M Global Synchronisation in Linda

Concurrency Practice and Experience Sept

Callsen C J Cheng I and Hagen P L The AUC CLinda System In

LindaLike Systems and Their Implementation G Wilson Ed Tech Rep EPCC

TR Edinburgh Parallel Computing Centre June ch pp

Carreira J Silva L and Silva J G On the design of Eileen a Lindalike

library for MPI In Proceedings of the Scalable Paral lel Libraries Conference

pp

Carriero N and Gelernter D How To Write Paral lel Programs A First

Course MIT Press Cambridge MA

Carriero N and Gelernter D Tuple Analysis and Partial Evaluation Strate

gies in the Linda Precompiler Research Monographs in Parallel and Distributed

Computing Pitman London ch pp

Ciancarini P and Rossi D Jada Co ordination and Communication for Java

agents In Mobile Object Systems Towards the Programmable Internet J Vitek

and C Tschudin Eds vol of Lecture Notes in Computer Science Springer

Verlag pp

Gamma E Helm R Johnson R and Vlissides J Design Patterns

AddisonWesley

Gelernter D Multiple Tuple Spaces in Linda In Proceedings PARLE

Paral lel Architectures and Languages Europe June pp

Halstead A MultiLisp A Language for Concurrent Symb olic Computation

ACM Transactions on Programming Languages and Systems

Jellinghaus R Eiel Linda An Ob jectOriented Linda Dialect ACM SIGPLAN

Notices Dec

Kielmann T Designing a Co ordination Mo del for Op en Systems In Coordina

tion Languages and Models First International Conference COORDINATION

P Ciancarini and C Hankin Eds pp

Matsuoka S and Kawai S Using TupleSpace Communication in Distributed

Ob jectOriented Architectures Proceedings ACM Conference on ObjectOriented

Programming Systems Languages and Applications OOPSLA

Rowstron A and Wood A Solving the Linda Multiple rd Problem In Coor

dination Languages and Models First International Conference COORDINATION

P Ciancarini and C Hankin Eds pp

Szafron D and Schaeffer J An Exp eriment to Measure the Usability of

Parallel Programming Systems Concurrency Practice and Experience