LINDA Linda is different, which is why we've put it into a separate chapter. Linda is not a programming language, but a way of extending ( in principle ) any language to include parallelismIMP13, IMP14. It was originally intended for parallel programming rather than operating systems as such, but the topics are obviously closely related. It will work with any language for which one can write procedures which implement its ( rather simple ) interface, and is most usefully thought of as an extension to the application programmer interface. Its authors have aimed for machine independence, portability, automatically handled interactions between processes, automatic scheduling, and efficient implementation. Processes communicate through a shared memory called tuple space, which we discussed briefly in the MEMORY MODELS chapter. By ingenious and elegant trickery in the implementation of access to the tuple space, the Linda manager guarantees mutual exclusion and synchronisation where required. This goes quite some way towards "factoring out" the concurrency issues, as envisaged by WirthIMP10 in his intentions for Modula.

Some details are given in the excerpt which followsIMP15. Notice the appearance of "Kernel Linda", intended for lower level use than Linda proper. Linda is a parallel communication mechanism developed by David will match any tuple in tuple space with Gelernter at Yale University. Rather than three fields whose first field is the string communicating with messages or "hello," and whose second and third through shared memory locations, fields are an integer and a float, processes communicate in Linda via a respectively. In this case, the string shared data space called tuple space. "hello" is the key used to find the Tuple space acts something like an matching tuple. If a matching tuple is associative memory, since tuples are found, the variables i and f are assigned identified by matching on a key rather the corresponding values from the than using a specific address. There are matching tuple, and the tuple is removed four fundamental operations on tuple from tuple space. If no matching tuple is space: found, the current process blocks. If the template matches more than one tuple, an out place a tuple in tuple space arbitrary one is picked. The rd operation in match a tuple and remove it from tuple is like in, except that the matched tuple is space not removed from tuple space. rd match a tuple and return a copy of it eval create an active tuple (a new The eval operation resembles out, process) except that it creates an active tuple. For example, if fcn is a function, then A tuple is an ordered collection of data items. For example, the tuple eval("hello", fcn(z), true); ("hello", 42, 3.14) creates a new process to evaluate fcn, which proceeds concurrently with the contains three data items: a string, an process that made the eval call. When the integer, and a float. The operation new process completes, it leaves the resulting passive data tuple in tuple out("hello", 42, 3.14); space, just like an out operation. places this tuple into tuple space. The out Linda is a high-level tool for operation never blocks, and there can be parallel programming in that it is not any number of copies of the same tuple directly based or dependent on a specific in tuple space. hardware mechanism. As with any higher level tool, there will always be Tuples in tuple space are accessed some overhead associated with Linda, by matching against a template. For but it is not excessive (see the sections example. the operation entitled "Implementing Linda" and "Kernel Linda"). This is similar to higher int i; float f; level languages compared with assembly in("hello", ?i, ?f); languages, where any overhead is usually outweighed by the benefits of easier programming and portability. Likewise, the flexibility of Linda can Sender more than make up for any overhead. For example, dynamic load balancing, Tuple space out("x", i) where the processing load is dynamically adjusted between multiple processors, is rarely used in parallel programs because "x", 5 it is difficult to implement using message passing or semaphores, despite its potential performance improvement; but in("x", ?j) it is relatively easy to implement using Linda (an example of dynamic load balancing is given in a later section). Receiver Linda also makes it easy to experiment with the parallel structure of a program to increase its performance. Figure 2. Message passing using Linda. Using Linda like a shared-memory model. Linda can be used to model If the receiver gets ahead of the shared memory, but with the advantage sender, it will block waiting on the next that it is easy to avoid unwanted message. The sender is allowed to run nondeterminism. In the example above, faster than the receiver because the out where the shared variable x was operation is asynchronous (it never incremented by multiple processes, a blocks). As in any asynchronous semaphore was used to protect the shared message-sending situation, if the sender variable from simultaneous access. can get significantly ahead of the Using Linda, we can represent a shared receiver, some sort of flow control must variable x as a single tuple whose key is be used to avoid overflowing processor the name x (we assume that no one else memory (an example of using flow is using x as a key); to increment the control is given in a later section). variable x in tuple space, the following operations would be used: In Linda, the ordering of messages is not guaranteed. For example, if one in("x", ?i); process executes two out operations in out("x", i+1 ); the following order: Since the in operation removes the out("x", 1); matching tuple from tuple space, any out("x", 2); other process trying to access the value of x will block until the tuple is put back then a separate process doing an in by the out operation. operation on the key "x" may receive the tuples in either order. The order is often Using tuple space as a shared data unimportant but if not, an additional field space is not limited to systems with in the tuple can be used to sequence the shared memory; it will also work with messages. For example, both the sender distributed-memory computers. This and receiver can keep a local sequencing makes programs written in Linda variable, initially set to zero. The out and portable between shared-memory and in operations in Figure 2 are changed to distributed-memory systems. // in the sender process Using Linda like a distributed- out("x", send_sequence++, i); memory model. Linda can also support the message-passing style of // in the receiver process programming. In Figure 2, a message is in("x", recv_sequence++, ?j); sent using an out operation and received using an in operation. and the correct order is guaranteed. If there are multiple senders or receivers, then the sequence variables can be shared by storing them in tuple space. Linda provides decoupling between the sender and receiver of a message in both time and space. Messages are decoupled in time because the out out("x", i); operation is asynchronous and because messages travel via tuple space. Since and the second process contains a single messages can be held in tuple space, in operation: sending a message between two programs not running at the same time is in("x", ?j); even possible. This is crucial in an operating system, where operating where i and j are integer variables. Since system services must be able to the key "x" is a constant at compile time, communicate with user programs that the Linda preprocessor determines that typically were not written when the this pair of Linda operations can be operating system was written. implemented as a queue between the variables i and j. On a shared-memory Messages are decoupled in space system, this could be realized as a shared because individual messages are queue protected by a semaphore; on a identified by their contents. Neither the distributed memory system, this could be sender nor the receiver needs to know the realized as an asynchronous message other's identity or location. This allows send. Thus, no searching is done, and communication in Linda to be more the most efficient implementation for the dynamic than in systems where the target architecture is used without sender and receiver of a message must changes to the program's source code. In explicitly rendezvous. In particular, this case, there is no overhead from the sending and receiving processes can be use of Linda. located on any processor, or even dynamically moved between processors, Using a preprocessor is not always with no effect on the program. This possible if Linda is used, for example, important property of Linda facilitates with interpreted languages such as Lisp, writing parallel programs. Postscript, Smalltalk, or Prolog, since tuple space usage can change while the Unlike some message-passing program is running. Even with compiled systems, Linda is not limited to a single languages, determining tuple space usage sender and receiver for a message is not always possible at compile time if stream. With the number of clients the programs that will be communicating changing over time, this is useful for cannot be preprocessed at the same time writing server processes, which receive (in particular, for communication messages from many different clients. In between the operating system and a user this case, if the client requires a message program). in return, it should identify itself in the message to the server so the server can A common technique used by respond to the correct client. Linda implementations that do not use a preprocessor is to restrict the key to a Implementing Linda. At first glance, single field. In this case, hashing is used it might appear that tuple matching so the search can be performed (on requires searching tuple space for each average) in constant time. The operation, a process that would clearly be performance of such an implementation too slow for use on a parallel computer. is quite acceptable, since the overhead of Fortunately, a number of techniques are a hashing function is reasonably small used to speed up the matching process. compared with the cost of data transfer The goal of these techniques is to reduce on a typical system. Allowing but a the amount of searching necessary and single key in a tuple places some make the remaining searching as efficient restrictions on Linda, but Linda's main as possible. advantages—including portability and decoupling in time and space—remain. The most common approach to For instance, all examples of Linda in making Linda more efficient is to use a this article need only a single key. preprocessor to reduce or eliminate runtime searching. The preprocessor A third technique used to make analyzes a Linda program to determine Linda more efficient is to split tuple space the patterns of tuple usage. For example, into smaller pieces, for example, through consider a Linda program that contains the use of multiple tuple spaces; doing two processes. The first process contains so, less space needs to be searched even a single out operation: if searching is required. This technique is also helpful for systems that use hashing, to transmit a stream of ordered messages since smaller hash tables can be used. from one process to another. Kernel Linda Multiple tuple spaces. Kernel Linda incorporates a unique implementation of Until now, Linda has been used multiple tuple spaces that allows the tuple primarily to support application-level space name to be used somewhat like an programming. To use Linda for system- additional key in a tuple. In traditional level programming, a number of Linda, multiple keys in a tuple are often modifications were made. This section used to restrict communication to specific discusses those modifications and shows data structures; in Kernel Linda, this how they benefit system-level function can be more efficiently programming. performed using multiple tuple spaces. The provision of multiple tuple spaces Kernel Linda is a version of Linda removes much of the need for multiple designed to be used for system-level keys. programming, as the interprocess communication (IPC) mechanism for a For example, in the out operation parallel operating system. Kernel Linda was also designed for use as the runtime // which row in the matrix system for the original Linda. In this int RowNumber; case, the Linda preprocessor is used to // data for the row generate calls to Kernel Linda, rather float data[SIZE]; than to the normal Linda runtime library. out("MatrixA", RowNumber, data); This allows programs written in Linda to be integrated into an operating system the constant "MatrixA" identifies a based on Kernel Linda. specific data structure, and the identifier RowNumber identifies a specific row in Since Kernel Linda is designed to the matrix. A specific row of the matrix support communication between would be retrieved by searching on the operating system processes and user first two keys: processes, a preprocessor cannot be used to speed up tuple space operations. int RowNumber = 42; Consequently, for efficiency, tuples in rd("MatrixA", RowNumber, ?data); Kernel Linda are allowed to have only a single key. Except for the single-key In Kernel Linda, the need for restriction, Kernel Linda is a superset of multiple keys can be eliminated by Linda and includes the following placing matrix A in its own tuple space, extensions: with the key being the row number. This also reduces the amount of searching • multiple tuples spaces, required, since the name of the matrix is • the ability to store a tuple space as a not used as a key during matching. In field of a tuple, and Kernel Linda, the above out operation • a set of language-independent data would be written (in C) as types. out(aMatrix, RowNumber, data); In addition, the ordering of tuples in tuple space is guaranteed in Kernel where aMatrix is the name of a Kernel Linda. If one process executes the Linda tuple space. In C++, the special following out operations in the following role of the tuple space is syntactically order: highlighted:

out("x", 1); aMatrix.out(RowNumber, data); out("x", 2); The provision of multiple tuple then a separate process doing an in on spaces has other benefits for parallel the key "x" is guaranteed to receive the programming. In a single global tuple values 1 and 2 in that order (assuming space, name conflicts can occur between that no other Linda operations use the programs using the same key to identify key "x" in this tuple space). This avoids tuples. Multiple tuple spaces allow the need for extra sequencing keys, and communication to be more structured. In also means that Kernel Linda can be used a multiapplication environment, some tuple spaces can be local to specific applications, while other tuple spaces can provides a set of language-independent be common to some or all applications. data types. For example, this allows a This allows Kernel Linda to support both tuple space created by a C++ program to multiprogramming (separate applications be accessed by a debugger or tuple space running concurrently) and browser written in Postscript. multiprocessing (single applications running on multiple processors). In traditional Linda, all fields in a tuple are passed by value. This is Subsequent to our implementation reasonable for small fields, such as of multiple tuple spaces in Kernel Linda, numbers or even names. But, for large another design for multiple tuple spaces fields, such as structures or long was done for Linda as part of the design character strings, this can result in extra of Linda 3, although it is not the type of copying. Passing some types by system we describe here. reference avoids this extra copying. More importantly, if a tuple space is to be Tuple spaces as tuple fields. passed as a field in a tuple, it must be Kernel Linda allows a tuple space to be passed by reference, because two used as a field in a tuple, which allows processes must have references to the Kernel Linda to be used as its own name same tuple space to communicate. service. This is done by using a name as Consequently, the Kernel Linda data the key and the tuple space as the value types are divided into two groups, simple of a tuple in another tuple space. No (pass-by-value) and composite (pass-by- other naming mechanism is required. A reference). The simple types are tuple space can be used like a directory in a hierarchical file system, except that arbitrary graphs can be constructed. For example, all tuples spaces associated with an application can be given "names" by storing them in a tuple space, or one process can be given access to a tuple space owned by another application via a common tuple space. This is diagramed in Figure 3, where process P is computing with three matrices: A, B, and R, and a separate process Q has been given access to the R (result) tuple space.

P Q

Matrix A ("A", • )

("B", • ) Matrix B ("R", • ) Matrix R

Figure 3. Tuple spaces as fields in a tuple. Language-independent data types. Linda is traditionally implemented to work within a single language and borrows the data types from that language. Kernel Linda is designed to work with many different languages (both compiled and interpreted), so it int a 32-bit integer corresponds to a key in a Kernel Linda real a 32-bit floating-point dictionary having multiple values. number real64 a 64-bit floating-point To allow the user to choose number between value and reference semantics, name an atomic identifier strings and names are essentially the struct a structure containing user- same type, except that strings are passed supplied data by reference and names are passed by null the error value value. The same is true of blocks and structs. This allows copying to be The simple types are always passed avoided, if desired. For example, names by value. If a process passes one of these are typically used as keys in tuples, while objects to another process (for example, strings are used for long strings of text via tuple space), each process will have a such as shell scripts. Likewise, an separate copy of the object, so modifying executable load module is normally the object in one process will not affect stored as a block to avoid making extra the value of the object to the other copies, but simple data structures, process. The composite types are corresponding to a struct in C, are normally stored as a Kernel Linda struct. dict a Kernel Linda tuple space (dictionary) Implementation. Kernel Linda string a string of characters evolved through several block an array of bytes implementations, both on single- array a heterogeneous array of processor and distributed-memory other Kernel Linda objects computers. On a single-processor computer, and within a single processor Composite types are always passed on a distributed system, Kernel Linda is by reference; if a process passes one of implemented as a shared-memory-model these objects to another process (again, program. Internally, composite objects typically via tuple space), then both are protected by a semaphore to guard processes will have a reference to the against concurrent updates. This same object, even if the processes are on semaphore is controlled by the system different processors. Thus, modifying and is not visible to the user. the object in one process will affect the value of the object to the other process . The implementation on a This is the only Kernel Linda distributed-memory computer, where a communication mechanism: modifying a distributed-memory-model program is shared object. In particular, the Linda used, is much more interesting. In this operations (in, out, and rd) modify case, composite objects are manipulated shared dictionaries. Shared objects are using a form of distributed pointer called implemented on a distributed-memory a locator, which, in addition to other computer using a mechanism called information, contains the processor ID of locators, described in the next section. the processor on which the object is located. When an operation is to be Of course, it is possible to make a performed on a remote object, a message copy of a Kernel Linda object. Making a is sent to the processor containing that local copy of an object that was passed object, and a server process on the by reference gives the equivalent remote processor performs the operation. semantics as pass-by-value. When an operation is to be performed on a local object (including operations by the A dict (dictionary) is the Kernel server process after receiving a request Linda name for a tuple space. A tuple in a from a remote processor), the shared- dictionary consists of a single key/value memory-model program is used. The pair. The keys are almost always names implementation of Kernel Linda on a or integers but can be of any data type. A shared-memory computer is more value associated with a key in a straightforward than the one for the dictionary can also be of any type, for distributed-memory computer because it example, another dictionary. Tuples are avoids the remote server process. only allowed to have a single value, but this value can be a structured object, such Despite the fact that Kernel Linda as an array, which holds multiple values. does not use a preprocessor, the As in Linda, multiple copies of the same overhead compared to that of message tuple can be held in tuple space; this passing is reasonable. This overhead arises mainly from evaluation of the case). This is comparable to the overhead hashing function, any required searching of higher level languages such as C, and, within the hash table, and the locking of of course, when this overhead is a semaphore. For transfers of reasonable unacceptable, the user can drop down amounts of data the overhead is about 10 into machine primitives, analogous to the percent, rising to as much as 50 percent way C programmers can drop down into for shorter transfers (a synchronization assembly language. with no data transfer being the worst REFERENCES.

IMP13 : S. Ahuja, N. Carriero, D. Gelernter : "Linda and friends", IEEE Computer 19#8, 26 ( August 1986 ).

IMP14 : N. Carriero, D. Gelernter : "Linda in context", Communications of the ACM 32, 444 ( 1989 ).

IMP15 : W. Leler : “Linda meets Unix”, IEEE Computer 23#2, 43 ( February 1990 ). ––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––– –––––––––

QUESTIONS.

What problems does Linda solve?

What is it that Kernel Linda does that Linda doesn't ? What sort of support do they need from the operating system ?

–––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––