Concurrent Programming Concepts

Per Brinch Hansen Information California Institute of Technology, Pasadena, California 91109

This paper describes the evolution of language features for multiprogramming from event queues and semaphores to critical regions and monitors. It suggests that the choice of language concepts shouldbe guided by two simple principles : First, it should be possible to understand a concurrent program in time- independent terms by an effort proportional to its size; secondly, it should be possible to state assumptions about invariant relationships among program components and have these assumptions checked automatically. The central problems of multiprogramming are illustrated by annotated algorithms written in a well-structuredprogramming language. Key words and phrases: structured multiprogramming, programming languages, operating systems, programming errors, resource protection, compile-time checking, correctness proofs, sequential and concurrent processes, synchronizing events, semaphores, shared data, mutual exclusion, critical regions, monitors. CR categories: 4.22, 4.31, 4.32, 5.24

Somewhere there are decisions made that are programming systems. The techniques used not rational in any sense, that are subject to to implement these language concepts, which nothing more than the personal bias of the apply single-processor and decision maker. Logical methods, at best, re- both to multi- arrange the way in which personal bias is to processor systems, are described elsewhere. be introduced into a problem. Of course, this [1] "at best" is rather important. Present intuitive When a new programming technique, such methods unhappily introduce personal bias in as multiprogramming, is program- such a way that it makes problems impossible invented, to solve correctly. Our purpose must be to re- mers will initially use it in a completelyun- pattern the bias, so that it no longer interferes restricted manner to discover its potential. in this destructive way with the process of At a later stage, when experience with its design, and no longer inhibits clarity of form. practical limitations has been gained, de- Chistopher Alexander, Notes on the Synthesis a ofForm signers begin to recognize the benefits of more restrictive language notation which clarifies their own understanding of pro- 1. INTRODUCTION grams and enables compilers to detect seri- ous errors. This paper describes an attitude toward This survey describes the evolution of lan- multiprogramming—the programming tech- guage features for multiprogramming during niques used to control concurrent activities the past decade. I view this development by computers.* It tries to identify abstract mainly gradual shift from concepts that properties as a of programming languages that have a strong resemblance to assembly lan- facilitate the design of large,reliable multi- guage features toward a notation that en- * In this paper, the term multiprogramming is used courages hierarchical structuring of pro- consistently to denote all programming techniques grams. Several useful proposals are not used to control concurrent processes on single- processor aswell as multiprocessorsystems. mentioned here, and those that are included

Computing Surveys, Vol. 5, No. December 1973

Science,

4, 224 Per Brinch Hansen Concurrent Programming Concepts CONTENTS are only meant to illustrate a general atti- and variables) and operations on data (rep- The following assertions will hold before tude toward the subject. resented by statements). During the execu- the execution of lines 1-8,respectively. The central problems of multiprogram- tion of the program, the operations are car- comment 1 0 out of n records processed ming are illustrated by algorithms written ried out strictly one at a time, and each (where 0 n) ; in a high-level language.I have included in- operation is completed within a finite time. comment 2 i out of n records processed formal assertions in the algorithms and out- The result of an operation can be defined by (where 0 i n) ; lined arguments of their correctness to show assertions about the relationships of data comment 3 i out of n records processed 1. Introduction 223 that multiprogramming concepts can be de- before and after its execution. (where 0 i < n) ; 2. SequentialProcesses 224 fined as concisely as sequential program- In Algorithm 1, we can make the following comment 4 i out of n records processed 3. Unrestricted Concurrency 226 ming concepts. In many ways, this survey assertions about the state of the (where 4. Structured Concurrency228 variables 0 i < n), and this = input 5. Disjoint Processes 230 can be regarded as an expanded version of before and after the execution of the input record Ri + 1 ; 6. Time-DependentErrors 231 an earlier paper [2]. We will begin with a statement in line 3: comment i out n processed 7. Timing Signals233 5 of records brief characterization of sequential pro- comment 3 i n 8. Critical Regions 236 out of records processed (where 0 i < n) , and this = updated 9. Critical Regions 241 grams. (where 0 i < n) ; record Ri + 1 ; 10. Conclusion 244 get (this, reader) comment 6 out Acknowledgements244 ; i + 1 of n records proc- Bibliography 244 comment 4 i out of n records processed essed (where 0 i < n) ; 2. SEQUENTIAL PROCESSES (where 0 i < n), and this = input comment 7 i out of n records processed record Ri 1; (where 0< i n) ; Although most programmers an + have intui- Sequential programs have two prop- comment 8 n out n processed understanding vital of records tive of what a sequential erties : (where 0 n) ; is, it may be useful to illustrate this s^ process 1) The a sequential program is Once the individual- statements are under- concept by an example.Algorithm 1 defines effect of independent of its speed execution. All stood, one can define the effect of a sequence a sequential that inputs fixed of process a that matters is that operations are of by a single pair RI, R2, ,Rn carried statements of assertions. number of records . . . from out one at a time with positive speed, and As an example, the body of the while sequential updates these state- one file, records, that certain relationships hold before and ment in Algorithm 1 can outputs them on another sequential file. be defined as a and after their execution. The time-independent single operation by means of the following The language notation is borrowed from behavior of sequential programs enables the assertions: are two Pascal. [3] The files declared as user to ignore details over which he has comment 3 each consist- no i out of n records processed variables, reader and printer, control (such as the schedulingpolicy (where 0 i n) of type T of the < ; ing of a sequence of records some operating system and the precise timing of process nextrecord and increment i by (declaredelsewhere) A type can . record of T operations carried out by processors pe- one; from and be appended to or removed a sequence ripherals). comment 7 i out n put of records processed by means of two standard procedures, 2) A sequentialprogram delivers the same (where 0< i n) ; and get. The current record is kept in a vari- result each time it is executed with a given The next step of simplification is to re- of type T. The variable i counts able, this, set of input data. The reproducible behavior duce the while statement to a single opera- processed; can as- the number of records it of sequential programs under any tion: 0 to n. circum- sume integer values from stances particularly comment 2 i out n records processed Algorithm 1 Sequential file updating is important for pro- of gram validation. It enables one to isolate (where 0 i n) ; var reader, printer: sequence of T; and correct program errors by systematic process the remaining n-i records; this: T; i: 0. . n; testing. comment 8 n out of n records processed (where begin These properties of sequential programs 0 n) ; Finally, the (1) i:=0; are so well known that we tend to take them whole programcan be defined as a single operation: (2) while i

Computing Surveys, Vol. 5, No. December 1973 Computing Surveys, Vol. 5, No. December 1973

225

Conditional

4, 4, 226 Per Brinch Hansen Concurrent Programming Concepts 227

tain functional aspects of it can be described start task do SI tions. Algorithm 2 is deliberately written in in even more economical terms. This is the an unstructured manner that makes it diffi- whole purpose of structured programming! will start the execution of statement SI as cult to understand. A well-structured algo- It is also essential to use hierarchical a new process. As soon as the new process rithm will be presented later. (I recommend structuring of programs that control concur- starts to run, the old process continues to that thereader not spend too much time try- rent processes, and choose language con- execute the next statement S2. When state- ing to understand Algorithm 2 at this point.) I- structs for multiprogramming that can be ment SI has been executed, the new process Algorithm 2 Concurrent file updating (un- understood in time-independent abstract indicates its completion by means of a vari- structured version) terms; we shall find ourselves able, task, of type response. Following this, otherwise, var printer: of T; apply systematic pro- the new process ceases to exist. After the reader, sequence unable to methods of reading, printing: response; next, gram analysis and verification. execution of statement S2, the old process waits until the response variable indicates this, last: T; i: 2 . . n; begin that statement SI has terminated. The pro- if n then gramthen continues to execute statementS3 1 get (last, reader) ; 3. UNRESTRICTED CONCURRENCY as a single, sequential process. (We will not be concerned with how the process and re- if n Js 2 then update (last) ; Processes called concurrent if their sponse concepts can be implemented.) Fig- are exe- start printing do put (last, cution overlap in time. More precisely, two ure 2 shows a flowchart of this program. Algorithm is version of the printer) ; processes are concurrent if the first opera- 2 a concurrent get (this, reader) file updating program (Algorithm 1) that tion of one process starts before the last i:= 2; uses the start and complete statements. The operation of the other process ends. The while i < n do programming techniques to purpose of introducing concurrent processes used control start reading do get (next, concurrent processes are called multipro- is to permit input/output operations to pro- gramming. ceed simultaneously with updating opera- reader) ; update (this) ; No assumptions will be made about the complete printing; speed at which processes concurrent are exe- last:— this; cuted (except that be positive). We will it start printing do put (last, benefit assumption from this weak in two printer) ; ways: 1) It will encourage us to try to reading; understand complete multiprogramming systems in this:= next; time-independent terms; and 2) It will per- i:= i + l; mit the underlying implementation to use end a dynamic scheduling algorithm to achieve complete printing; sharing. J efficient resource last:= this; One of the earliest ideas of multiprogram- end ming was to comple- specify the start and update (last) ; tion of processes by two operations which Fig. put (last, printer) 1 Flowchartof Algorithm 1. start (In ; I will call and complete the litera- end ture, they are often called fork and join). end descriptions, called abstractions, is essential The meaning of these operations can best be to enable us to understand large programs. illustrated by an example: The program inputs and updates the first Figure 1 is a flowchart of Algorithm 1. It record RI. It then starts the output of the shows that the program consists of nested var task: response; first record and inputs the second record statements, each with a single starting point begin R2 at the same time. The program now re- and a single completion point. Algorithm 1 start task do SI ; peats the execution of a cycle that, in gen- can be analyzed by associating an assertion S2; eral, inputs the nextrecord Ri + 1, updates with each statement, and gradually replac- complete task; this record Ri, and outputs the last record ing these assertions by fewer and fewer as- S3; Ri-1 simultaneously. When all records have sertions. It is most important that aprogram end been input (i = n) , the program completes with a hierarchical structure can be built the output of the second last record Rn-1. and understood in detail by intellectual Initially, this will be outputs the last an program executed aa Fig. 2 Flowchart of program to illustrate start Finally, it updates and effort proportional to its size, and that cer- a single, sequential process. The statement and complete operations. record Rn.

Computing Surveys, Vol. 5, No. December 1973 Computing Surveys, Vol. 5, No. December 1973

;

4, 4, «t

Concurrent Programming Concepts 228 Per Brinch Hansen updated record In spite of this explanation, the details of If the record Ri, and last = INPUT UPDATE OUTPUT input contains more than one record Ri- 1; Algorithm 2 are still hard to follow. The (n 2), the program inputs the second comment 6i 1 out nrecord processed cause of this obscurity updates — of becomes clear when record R2 and the first record RI (where 2 i n) , next input record we examine the flowchart of the body of the simultaneously: < — Ri 1, and this updated record Ri; while statement (Fig. 3). This flowchart comment 3 0 out n records processed + = of comment 7 i 2 out n records proc- cannot be decomposed into a set of nested (where 2 n) , next input record R2, of — essed (where—2< i n) next input operations. The program is highly confusing and this updated record ; , — = RI record Ri, and this updated record because a process started outside the while Algorithm 3 Concurrent file updating — statement (output) may be completed either (structured version) Ri-1; A comparison of assertions 4 and 7 shows inside or outside the while (de- statement that assertion 4 is a relation that remains pendingon whether or not n > 2) . var reader, printer: sequence of T; It is invariant after each execution of the loop. certainly possible to write a simpler next, this, last: T; i: 2 . . n; When the loop terminates, assertion 4 still program using the start and complete begin state- holds and i is equal to n: ments (just as it is possible to structure pro- (1) if n 1 then comment 8 n 2 out n records proc- grams written in assembly language) get reader) ; — of . But (this, essed (where 2 n) , next input rec- the concepts of a high-level (2) n = language should if S3 2 then ord and this updated record be chosen such that they naturally encour- Rn, = cobegin Rn 1; age the programmerto (next, reader) - write well-structured get program outputs the second last programs. The start complete The now and state- update (this) ; record Rn 1 : ments clearly have the flavor of go to state- coend comment— 9 1 out n records proc- ments. They make it very (3) n of difficult to deter- i:= 2; essed (where—2 n) and this input mine from the program text (4) , — where concur- while i < n do record Rn; rent processes begin and end, and which last := this; this := next; The effect of the innermost statement variables they operate on. (5) cobegin if ment can be summarized as follows: get (next, reader) comment 2 0 out n records processed update (this) of ; (where 1 n) , and this input record 4. STRUCTURED CONCURRENCY put (last, printer) ; RI; — coend Fig. 3 Flowchart of the body of the while state- concur- process n — 1 records; ment in Algorithm 2. A well-structured representation of (6) i.-= i + l; rent processes in a high-level programming (7) comment 10n — 1 out of n records proc- end (where 1 n), input language was suggested by Dijkstra in 1965. (8) essed and this = put (this,printer) ; Rn; [4] The concurrent statement : record terms of time-independent assertions by an this = next; n 2 are equiv- (9) If = 1 then assertions and 10 effort proportional to its size, and assertions cobegin Si; S2; ; Sn coend end n 2 . . . (10) (this) alent, and if then assertions 9 and about simple statements can gradually be update ; put (this, printer) ; 10 This covers the two cases indicates that the statements Si, S2, ,Sn (11) are equivalent. replaced by assertions about structured . . . end in which statement is either skipped can be executed concurrently; when all of (12) the if statementsuntil the whole programhas been end or them are completed, the following statement executed. reduced to a single statement defined by two Finally, the last record Rn is updated and of the in the program (not shown here) is executed. The program now enters a main loop that assertions. This abstract description Algorithm 3 defines file updating by output: program even enables the programmer to ig- repeatedly inputs a record Ri + 1, updates comment out n records processed means of concurrent statements. We will a 11 n of nore the concurrent nature of the solution. record Ri, and outputs a record Ri — 1 (where 1 n) analyze this program, by state- ; To see this, it is sufficient to observe that statement simultaneously. The loop is initialized by as ment, in terms of assertions that hold The whole program can be understood the initial and final assertions 0 and 12 for before setting a counter i equal to 2. This leads to a single operation defined follows: the execution of lines 1-12. the as Algorithm 3 are identical to the assertions following program state: comment processed The initial assertion is the following: 1 0 out of n records made at the beginning and end of the se- comment 4 i 2 out of n records proc- (where n) ; comment 1 0 out n records processed (where— 0 quential version (Algorithm 1). of cessed 2 i n) , next = input process n records; (where 0 n) ; record Ri, In the previous discussion, the concurrent and this = updated record comment n out records processed If the input file is non-empty (n 1) , the Ri-1; 12 of n statement was defined and used informally (where 0 n) ; you program inputs the first record RI: Within the loop, the programgoes through < to give an intuitive understanding of its This concurrent has precisely the comment 2 0 out of n records processed the following sequence of states: program meaning. We will now explicitly discuss the same characteristics as a well-structured se- assumptions that led to the simplicity of (where 1 n), and this = input record comment 5 i 2 out n records proc- — of quential program. analyzed in Algorithm 3. RI; essed (where 2 < i < n) , this — input It can be

Computing Surveys, Vol. 5, No. 4, December 1973 Computing Surveys, Vol. 5. No. 4, December 1973

229

s$

;

; s»

230 Per Brinch Hansen Concurrent Programming Concepts

structured statements, and then reduce as- comment 6c i — 1 out of n records proc- ment Si must only refer to variables that (where n) accessible to Si according to the rule of r sertions about structured statements to as- essed 2 i < ; are 1 sertions about the whole algorithm. Here assertions 5 and 6 have been split into disjointness.) It may seem surprising that the use of three pairs of assertions, one for each process concurrent statements in Algorithm 3 in no statement. Each pair of assertions defines way complicated the formulation of asser- our assumptions about a variable i that re- 6. TIME-DEPENDENT ERRORS tions. The informal arguments made in fa- mains constant during the execution of the An error in a sequential program can be lo- r- i I vor of its correctness are quite similar to the concurrent statement and our assumptions —I | arguments on Algorithm 1 in Section 2. It about the variables that are private to the cated by repeating the execution of the pro- gram several times with the data that re- I I turns out that we were able to apply se- corresponding process and may be changed quential methods program by vealed the error. In each of these experi- | of analysis to a it. I concurrent program simply because the Since the processes are disjoint, the con- ments, the values of selected variables are I I processes defined by the concurrent state- junction of assertions 6a, 6b, and 6c holds recorded to determine whether or not a given I I ments are completely independent one after the execution of the concurrent state- program component works. This process of of elimination continues until the error has | L -J I another. These processes operate on disjoint ment. sets of variables: In general, we can define the properties of been located. When a given program component has process variables a concurrent statement by the following rule of disjointness: The statement been found to behave correctly in one test, l_ _l get next, reader we can ignore that component in subsequent update this cobegin SI ; S2 ; .. . Sn coend tests because it will continue to behave in put last, printer exactly the same manner each time the pro- Fig. 4 the state- A flowchart of the body of while They are disjoint or defines statements SI, S2, . . . ,Sn that can gram is executed with the given data. In ment in Algorithm 3. called non-interacting processes. The input, update, and output be executed concurrently as disjoint proc- other words, our ability to test a large se- processes are out esses. The disjointness implies that a varia- quential program a step-wise 5. DISJOINT PROCESSES carried simultaneously in manner de- only to utilize the computer more efficiently. ble vi changed by a statement Si cannot be pends fundamentally on the reproducible There is one striking difference between Al- But conceptually, these processes could just referenced by another statement Sj (where behavior of the program. gorithms 2 and 3: When concurrent state- as well be carried out strictly sequentially j 9^ \). In other words, a variable subject A careful programmerwho writes a well- ments are used, the program text directly as defined by Algorithm 1. to change by a process must be strictly pri- structured concurrent program, such as Al- shows where a given process begins and Since the processes in Algorithm 3 have vate to that process; but disjoint processes gorithm 3, and outlines an informal proof ends. This simplification is achieved by re- no variables in common, they can be ana- can refer to common variables not changed of its correctness can still make mistakes stricting the freedom of scheduling. The lyzed one at a time as unrelated sequential by any of them. when he types the final program. And he start and complete statements enable the processes. Consider, for example, the con- The general rule that we intuitively fol- may not find all these errors during a proof- programmerto initiate and terminate proc- current statement inside the loop. It can be lowed in analyzing Algorithm 3 can be reading of the program text. One possible esses in any order he pleases. A concurrent analyzed as three independent operations stated as follows: Suppose that we know mistake would be to type the concurrent statement defines processes that are started defined by the following assertions: the following about statements SI, S2, . . . , statement within the loop of Algorithm 3 as and completed at the same time. Complete comment 5a i 2 out of n records proc- Sn that operate on disjoint variables: follows : generality of programming implies complete essed (where—2 i < n) ; Statement SI will terminate with a re- cobegin absence of structure. As a compromise, one get (next, reader) ; suit RI if a precondition PI holds before get (next, reader) has to look for programming tools that are comment 6a next = input record Ri + 1 its execution. update (this) ; intellectually manageable and at the same (where 2 i < n) ; Statement S2 will terminate with a re- put (this, printer) ; time practical for most (but not necessarily suits R2 if a precondition P2 holds before coend all) applications. The concurrent statement comment 5b i 2 out of n records proc- its execution. In this case, the concurrent statement will is one such compromise. essed (where 2— i < n) , and this = in- input the nextrecord Ri + 1 correctly, but Since a concurrent statement has a single put record Ri; Statement Sn will terminate with a re- will update and output the current record starting point and a single completion point, update (this) ; suit Rn if a precondition Pn holds before Ri simultaneously. So the output record will it is well-suited to structured programming. comment 6b this = updated record Ri its execution. normally be only partially updated. In a Figure 4 shows a flowchart of the body of (where 2 i < n) Then we can conclude that a concurrent multiprocessor system with a common store, the while statement in Algorithm 3. In con- execution of SI, S2, . . . ,Sn will terminate a record occupying, say 256 machine words, trast to Fig. 3, the present flowchart can be comment 5c i — 2 out of n records proc- with the result RI & R2 ... & Rn if the may be output with x words updated and decomposed into nested operations. This ex- essed (where 2 i < n) , and last = preconditionPI &P2 & . . . &Pn holds be- 256-x words unchanged (where 0 x plains why we were able to reduce assertions updated record Ri — 1 ; fore its execution. (It should be added that 256). The processing of a single record can about simple statements to assertions about put (last printer) ; the assertions Pi and Ri made about state- therefore produce 257 different results. If

Computing Surveys, Vol. 5, No. 4, December 1973 Computing Surveys, Vol. 5, No. December 1973

231

■

;

;

4, 232 Per Brinch Hansen Concurrent Programming Concepts 233 we update a given file of 10,000 changed by records, the a statement and variables that programming, but I would expect it to be a ceived than they are sent. To satisfy program give of the 10 000 can by a not faster can order of 257 ' be referenced statement but design of small, dedi- this requirement necessary different results. convenient toolfor the it is sometimes changed by it. These two kinds of variables cated multiprogramming systems involving to delay further execution of the receiving In a multiprogramming system, the are the variable parameters execu- called and con- a predictable number of processes. I have process until the sending process produces tion of concurrent statements be inter- stant parameters a can of statement. reservations about its usefulness in another data item. Synchronization is a gen- leaved and overlapped in arbitrary order. start some When and complete statements are larger multiprogramming systems (for ex- eral term for timing constraints of this type The system uses this freedom to multiplex used, one cannot, in general, recognize con- ample, operating systems) as discussed later imposed on interactions between concurrent computer resources (processors fast and current statements from the syntax alone. paper. processes. stores) in this among several user computations to This recognition is, however, trivial when The rule of disjointness is introduced to The simplest form of interaction is an ex- achieve short response times at reasonable concurrent are a statements enclosed in brack- help the programmer ; it enables him to state change of timing signals between two proc- cost. cobegin and coend. ets, To make the check- explicitly that certain processes should be esses. A well-known example is the of The result of the version of ing of disjointness use erroneous manageable, it is neces- independent of one another and to depend interrupts to signal the completion of Algorithm 3 depends the relative rates at sary use on to restrict the of pointer variables on automatic detection of violations of this asynchronous peripheral operations to a which updating output same and of the and procedure parameters far more than assumption. To make multiprogramming central processor. Another kind of timing record take place. The present programming languages rates will be influ- do. intellectually manageable and reasonably signals, called events, was used in early enced by the presence (unrelated) Although of other the problem has not yet been disjoint processes should be multiprogramming systems to synchronize analyzed efficient, used computations, and by the absolute speeds of completely, it seems certain that wherever possible. as we shall see, all concurrent processes. process peripheral operators But, When a de- devices and interacting the necessity of compile-time checking of multiprogramming systems must occasion- cides to wait for an event, the execution of with the computations. very disjointness It is therefore will have a profound influence ally concurrentprocesses to exchange its next operation is delayed unlikely that permit until another the erroneous program will on language design. An example will make data in a well-defined manner. The com- process causes the event. An eventoccurring everdeliver the same result twice for a given this clear. In sequential programming lan- ponents a statement must, at processes waiting guages of concurrent for a time when no "are for input file. The error will be particularly a pointer variable may be bound to example, be able to indicate their termina- one has no effect. hard to find if the updated file is not in- other variables of a given type, example: for tion in a common (anonymous) variable; The following program illustrates the spected by humans, but just retained for var p: pointer integer; otherwise, it would be impossible to deter- transmission of timing signals from one later processing by other programs. to mine when a concurrent statement is termi- process to another by means of a variable c Such unpredictable program behavior This declaration indicates that variable p a cobegin nota- type event. processes are makes to nated as whole. The coend of Both assumed to it impossible locate an error by is apointer to any integervariable. The no- tion problem be cyclical: systematic hides this communication from testing. It can only be found by tation enables a compiler and its run-time the user, but it has to be solved at some studying programtext var c: event; the in detail. This can system to check that p always points to a other level of programming (in this case by be very frustrating (if cobegin not impossible) when variable of a well-defined type (in this case, the code generated by a compiler) . The fol- it consists of thousands of lines and one has an integer) . This kind of pointer variable is lowing describe language features cycle "sender" no sections cause event(c) clues about where to look. // we wish to far better than one that can point to an used to control interactions among processes. . . . succeed in designing large, reliable multipro- arbitrary store location containing data of end gramming systems, we must use program- unknown type (or even code). But the dec- cycle "receiver" ming tools that are so well-structured that laration pointer to integer still does not . . . await event(c) ; . . most time- dependent errors can be caught enable a compiler to recognize which integer 7. TIMING SIGNALS end at compile time. coend the variable p will point to during program processes that common A closer look at Concurrent access the incorrect version of execution. So, unless the programmeris will- variables are called interacting or communi- A relationship of this type exists in a real- Algorithm a ing 3 reveals clear violation of the to make all integers private to a single cating processes. When compete time system in which rule of disjointness: processes one process schedules Within the erroneous process, this pointer concept is inadequate for the use of shared resources, tasksregularly by sending timing concurrent the common concurrent statement output process re- for multiprogramming. We need to bind a variables are necessary to keep track of the signals to fers to a variable, this, other processes that carry out which is changed by pointer p to a particular set of integers, for requests for service. And when processes these tasks. the updating process. It is therefore example inevita- : cooperateon common tasks, common varia- in Section 6 we recognized that simultane- ble that theresult of the output on depends bles are necessary to processes to ous operations can the time at which var i, j, k: integer; p: pointer to i or j; enable ask on the same variable lead updating takes place. one carry out subtasks to a To make it another to and report large number of different results. In simple for a compiler to check Some of the languagerequirements needed on theirresults. the disjointness that context, the problem was caused by a it should be possible by scan- to make compile-time checking of disjoint- We study systems program program will in which one error. Now we have the sameprob- ning the text to recognize concur- ness practical are discussed in more detail in Process produces and sends a sequence of lem again: The rent statements and variables concurrent operations, await accessed by data items to another process that receives and access them. The must cause, both the same variable c. compiler be able to dis- The concurrent statement has not yet and consumes them. It is an obvious con- But we can no longer regard this viola- tinguish between variables that as a can be been implemented and used for practical straint that these data items cannot be re- tion of the rule of disjointness since our in-

Computing Surveys, Vol. 5, No. 4, December 1973 Computing Surveys, Vol. 5, No. December 1973

;

4, 234 Per Brinch Hansen Concurrent Programming Concepts

tention is that the processes should exchange is the general observation that mutual ex- operations on a semaphore exclude one varv: semaphore; data. So we must relax the rule of disjoint- clusion of all operations on a shared variable another in time. Associated with a sema- cobegin ness and permit concurrent processes to ac- enables the programmer to analyze the pos- phore v are two integers defining the num- cycle "sender" cess shared variables by means of well-de- sible effects of a process interaction in finite, ber of signals sent and received through v . . . signal (v) ; fined synchronizing operations. Our next sequential terms. For the special case of and a queue in which receiving processes end task is to determine under which conditions event variables, we have also discovered can await the sending of further timing cycle "receiver" synchronizing operationsare "well-defined". that the net effect of await and cause opera- signals by other processes. Initially, the wait (v); To analyze the effect of an interaction tions depends on the order in which these number of signals sent and received are zero end between the receiver and the sender in the operations are carried out. Or, to put it more and the queue is empty. coend previous program, we must consider all the strongly: event operations force the pro- Signals cannot be received faster than possible ways in which the execution of grammer to be aware of the relative speeds they are sent. This semaphore invariant: Since wait and signal operations on the await and cause operations can be inter- of the sending and receiving processes. semaphore v exclude each other in time, a leaved and overlapped in time. In a continu- The programmer does not control the 0 received sent signal can be sent either before or after the time are receiver decides to wait for it. In the first ous scale there infinitely many order in which concurrent statements are is satisfied by using the following synchro- possibilities case, we can make the following assertions to consider. A drastic simplifica- executed; he is therefore unable to predict nization rules: tion is the effect of a process interaction about the sequence in which an interaction clearly needed to reduce this infinity involving 1) If a wait operation on a semaphore v a events. In real-time system, the program- takes place: to finite (small) number of cases. The only a is executed at a time when received sent mer has essentially lost process < comment receiver not waiting and 0 practical solution is to assume that synchro- control of then received is increased by one and the scheduling he cannot define process received sent; nizing operationson a given variable cannot — a that receiving process continues; but if received will schedule other processes a signal (v) be executed at the same time. In other in predictable sent, the receiver is delayed in the queue Event are only = comment receiver not waiting and 0 words, await and cause operationson a given manner. variables meaning- associated with v. ful to when one can assume received sent; event variable can be arbitrarily interleaved use that a 2) A signal operation on a semaphore v < process never is asked to carry wait (v) (but not overlapped) in time. If a process out another sent by one; if one or more proc- task until it has completed its increases comment receiver not waiting and 0 tries to operate on an event variable while previous task. esses are waiting in the queue associated < Although this assumption may be satisfied received sent; another process is operating on it, the system with v, one of these processes is enabled to in some applications, it will still complicate In the second case, the wait operation must delay the former process continue its execution and received is in- until the lat- programming tremendously if one depends may or may not delaythe receiver (depend- ter process has creased by one. completed its operation on on it. The programmermust then ing on whether received sent or received be aware assume that all processes waiting to = the eventvariable. speeds We < sent). But, in any case, the subsequent of the relative of processes under all receive signals eventually will be able to If this requirement is satisfied, there are circumstances. If a multiprogramming sys- signal operation will ensure that the receiver continue their execution (provided a suffi- only two cases to consider in the previous tem large no single continues its execution: is so that person under- cient number of signals are sent by other program: either an await operation is exe- stands its dynamic comment receiver not waiting and 0 behavior in detail, the processes). The scheduling algorithm used cuted before a cause operation or it. If individual programmer cannot received sent; after makereliable for a semaphore queue must not delay any the receiver waits before the causes wait (v) sender estimates of the relative speeds of processes process indefinitely favor of more urgent the next event, an interaction in comment receiver waiting and 0 re- between the under all circumstances. In particular, it will processes. But, apart from this requirement two processes is defined by the following an ceived — sent, or receiver not waiting be intolerable burden to verify that the of fair scheduling, no assumptions are made sequence of operations: speed assumptions are uninfluenced by and 0 < received sent; about the specific order in which waiting signal (v) comment receiver not waiting; modifications or extensions of a large sys- processes are to The allowed continue. weak comment not waiting 0 await event(c) tem. (rather receiver and < assumption of finite progress than received sent; comment receiver waiting; We must therefore conclude that event absolute for any process is a recur- speed) The effect of an interaction is independ- cause event (c) ; variables of the previous type are impracti- rent theme of programming. We have made ent of the order in which the wait and signal comment receiver not waiting; cal for system design. an The effect of inter- this assumption for sequential and disjoint operations carried out. The commutative But, if the sender causes event two are the before action between processes must be inde- processes, and now we make it again for property semaphore operations enables thereceiver re- pendent of waits for it, the receiver will of the speed at which it is carried interacting processes to achieve simplicity the programmer to ignore precise mo- main delayeduntil the next the event is caused : out. of program analysis and flexibilityof imple- ment at which a timing signal is produced. comment waiting; receiver not A far more attractive synchronizing tool, mentation. This is certainly the most important contri- cause event(c) the semaphore, was invented by Dijkstra in Algorithm 4 defines a transmission of bution of semaphores to program clarity. comment receiver not waiting; 1965. [4, 5] A semaphore is a variable used timing signals from one process to another Other uses of semaphores will be described await event (c) ; to exchange timing signals amongconcurrent by means of a semaphore v. later when we have clarified the funda- comment receiver waiting; processes by means of two operations, wait Algorithm 4 Exchange of timing signals mental role of mutual exclusion in multipro- The most important result of this analysis and signal (originally called P and V). All by means of a semaphore gramming.

Computing Surveys, Vol. 5, No. 4, December 1973 Computing Surveys, Vol. 5, No. December 1973

235

;

;

;

; ;

;

4, 236 Per Brinch Hansen Concurrent Programming Concepts 237 8. CRITICAL REGIONS variable is in defined terms of actual and comment x — r = v; same variable in one of its intermediate implicit variables. An actual (such We will now consider concurrent processes variable print (v) ; [r:= r v] states, the result will normally be incorrect. as v) is declared in the + that exchange data of arbitrary type (and program and repre- comment x — r = 0; In the previous example, process sented by a store location P starts to not just timing signals). As an we during its execu- v:= 0; operate on variable v example tion. An implicit variable (such at a time when x — choose a multiprogramming system as x or r) comment x — r = v; r 0 (instead of x r v) ; to in which refers to a system = = this leads job a process property of the that is not v:= v 1; [x:= x 1] a final state — r (instead statistics are collected by P represented + + in which x = v + 1 and printed by another in the computer during program comment x r = v; of x r v). — process Q. When a execution. — — = user job has been completed in this system, But in the third case, the "invariant" no In Section 7 we found that the effect The statements executed by of process P increases an integer v by processes P longer holds after the process interaction: synchronizing operations on event variables one. At and Q can regular intervals process Q prints be arbitrarily overlapped and comment x r v; and semaphores can be predicted the value interleaved in time. We will, however, — only if of v and resets it to zero. To discover the only print (v) ; [ r:= r+— v] they exclude one another in time. The pres- analyze the effects of an problems of data sharing we begin arbitrary inter- commentx — r = 0 ; ent example shows that will with leaving of the concurrent one must also be a naive "solution": statements that v:= v + 1; [x:= x + 1] able to achieve mutual exclusion of arbi- operate on the shared variable v. The in- commentx — r = 1; trary concurrent statements referring to a var integer; crease of v by process v: P can occur either v:= 0; shared variable of an arbitrary type. begin before, after, or in the middle of the print- comment x — r = v + 1 ; Concurrent statements that refer to the v:= 0; ing and resetting of v by process Q. So we Whether or not the invariant is main- same shared variable cobegin have three toto are called critical cases consider: tamed depends on the ordering of the con- regions. The previous program contains two cycle "P" v:= v current statements in time. This is, of critical regions: V= V 1- + 1; print(v); prinl(v); __j" ' + print{v); v: course, unacceptable to the programmer who '"' =0; v:= v+ 1; v:= v 1 v:=0; ,: +ij -= o; has no control over the scheduling of con- + cTcW - current processes. and .print (v) ; v:— 0; . . . . . The correctness criterion of the possible Notice that in the first two cases, in which print (v) en^ interactions between processes the invariant P and Q is continues to be satisfied, the v:= 0 coend defined by an invariant (v — x —r) that two processes have exclusive access to the en< relates an actual variable v to two implicit variable v while they areoperating on it: Critical regions referring to the same vari- x able exclude one This program* violates the rule of disjoint- variables and r To analyze the results of another in time. They can, the three oSSlble- v however, be arbitrarily interleaved ness since processes P and Q both refer to P cases > we m st extend the P h: = + 1; Q I print (v); in time. ro ram Wlth We make three assumptions and change the same variable v. Although P S implicit statements" referring v: = 0; about critical to x r (even regions that operate on this violation suggests a weakness of the and though these statements Q print (v); the same shared Wlll never be executed). Conceptually, variable: present approach to the problem we will ig- xis v: =0; | P \v: = v+ 1; nore it for the time being increased by one when v is increased, and r Mutual exclusion: At most, one process s increased y at a time can be a The state of the system accumulating job ! b the value of v when the latter The conflict in the third case is caused by inside critical region. Termination: A process statistics can be defined by two integers Pnnted - Assuming that the desired in- mixing the sequence of operations of process will always com- variant holds before interaction plete a critical region within finite (initiallyequal to zero) " an takes Q on variable v with an operation of process a time. place we can Fair scheduling: process x the number jobs make the foll °wing assertions P same A can always of executed ' on the variable: enter r the number of jobs reported about the first case : _ a critical region within a finite time. The variable v should comment x-r - v; Dekker has shown that mutual exclusion represent the number Q print (v); of arbitrary of jobs executed but yet v:~ v+ 1; [x: +1] statements can be implemented not reported; that = by is, the relationship v x r comment x r = v; means of load and store operations only = - should remain - P v: = v + 1; (provided invariant after each sequence of operations Print r:= r+ v] that load and store operations on > \- a given on the shared variable v. comment x — r = 0 ; variable exclude one another in Q v: =Q; time). His solution Theexample illustrates two general char- v:~ °; is far too complicated acteristics of multiprogramming commentx r v; and inefficient to be of practical value [4], systems: — = performs 1) The correctness The implicit Process Q a sequence of opera- but it does illustrate the subtlety of the mu- criterion for concur- statements are enclosed in tions on the shared variable v. It is rent operations on a square assumed tual exclusion problem. shared variable is de- brackets to distinguish them from that the invariant holds before and after The by an invariant— inadequacy of the load-and-store ap- fined & relationship that actual statements. In this case, the in- this proach must be true after initialization sequence of operations. But while the to mutual exclusion inspired Dijk- of the vari- variant still holds after the process inter- operations are out, v stra able and continue hold carried the variable and Scholten to invent semaphores. to before and after action. may be in various intermediate states in Algorithm subsequent operations on the variable. 5 defines an implementation of In the second case, the invariant is also which the invariant is not satisfied. If other critical regions by 2) The invariant property a means of a semaphore of shared maintained by the interaction : concurrent statements are carriedout on the mutex.

Computing Surveys, Vol. 5, No. 4, December 1973 Computing Surveys, Vol. 5, No. December 1973

\

1S

4, *

238 Per Brinch Hansen Concurrent Programming Concepts

Algorithm 5 Critical regions implemented ess leaves its critical region and produces A semaphore is an elegant synchronizing lowing errors: if a pair of wait and signal by means of a semaphore another signal. tool for an ideal programmer who never operations are exchanged, if one or both of A more rigorous proof of the mutual ex- var integer; mutex: semaphored); makes mistakes. But unfortunately the con- them are missing, or if a semaphore is ini- v: depends on begin clusion the following observa- sequences of using semaphores incorrectly tialized incorrectly. A compiler is also un- 0; tions: can be quite serious. A programmer might aware of the correspondence between a v:= 1) Independent signal of how wait and by mistake write the wait and signal opera- shared variable v and the semaphore mutex cobegin operations cycle "P" are used they maintain the sem- tions in reverse order for a particular criti- used to gain exclusive access to v. Conse- aphore invariant: region: ...wait (mutex); v.— v 1; cal quently, the compiler cannot protest if a + sent 1 region implemented signal (mutex) ; 0 received + (mutex) critical by a semaphore signal(mutex) ; . . . wait ; by to end 2) When a semaphore is used to achieve mutex mistake refers another shared variable w (instead v). cycle "Q" mutual exclusion, a process always executes In that case, the system will sometimes per- of Indeed, a com- piler cannot give the ...wait (mutex); print (?;); a wait operation followed by a signal opera- mit three processes to be inside their "criti- programmer any assis- tance whatsoever v.— 0; signal (mutex); tion. At any given time, some processes may cal regions" at the same time. This is a in establishing critical re- gions correctly by end have a wait not yet time-dependent error that only reveals itself means of semaphores. executed operation, but Since coend the corresponding signal operation. the if other processes enter critical regions while semaphores alone do not enable a So programmerto end structure of the program shows that the in- the erroneous critical region is being exe- indicate whether a variable v should be private to a single process or semaphore mutex variant: cuted. The declaration of a shared by several processes, a compiler must has been extended with an integer constant Another serious error would be the fol- 0 sent received lowing: either forbid or permit any process inter- defining the initial number sig- of available is also maintained. action involving that variable. To forbid nals: 3) Finally, it is clear that the number of wait (mutex) ; . . . wait(mutex) ; process interaction is unrealistic (since it varmutex: semaphore (initial) processes that are inside their critical re- prevents us from building interactive multi- This one causes the process executing the programming systems consisting A semaphore is now characterized gions at any given time are those processes of co- by three completed critical region to wait forever at the end of operating ; to permit arbitrary integer components : that have a wait operation but processes) not yet it for a timing signal that will never be pro- process interaction is (because initial the number of signals initially the following signal operation, that disastrous is: duced. Since the incorrect process is unable of the danger of irreproducible program- available to leave its critical region, other processes ming errors). We must sent the number of signal inside received sent therefore conclude operations = — trying to enter their critical regions will semaphores not a completed that do enable compiler By combining these three invariants, we also be delayedforever. Notice, that the be- to give a programmer the received the effective assist- number of wait operations find that the first assumption about critical havior of the other processes is only in- ance in error detection that he ex- completed should regions is satisfied: fluenced by the erroneous process after the pect from an implementation of a high-level The semaphore invariant must therefore be latter has entered its incorrectregion. So the language. revised slightly: 0 Sj inside 1 error is clearly time-dependent. Such a sit- To improve this situation, I have sug- At most, one process at a time can be inside uation in which two or more processes are gested 0 received sent + initial region. a structured notation for shared vari- a critical waiting indefinitely for synchronizing con- ables For the semaphores used in Section 7, initial Assuming that and critical regions [2]. A shared vari- processes are scheduled ditions that will never be satisfied is called able v of type T is declared as follows: is zero. In Algorithm 5, initial is equal to fairly when they are inside their critical re- a deadlock. one. var v: shared T gions, we can also conclude that the state- These examples show how easy it is to To make a sequence of statements SI, S2, ments executedwithin the critical regions of cause a time-dependent error by means of a Concurrent processes can only refer to and Sn a critical region, we enclose a . . . , it by Algorithm 5 will terminate within a finite semaphore. Even if a semaphore is used change a shared variable v inside a struc- pair of wait and signal operations on a time. And if the scheduling of processes correctly it still does not provide us with a tured statement of the form: semaphore mutex initialized to one: waiting for timing signals in the semaphore satisfactory notation to indicate that the region var mutex: semaphore(1) queue is also fair, then aprocess can only be violation of the rule of disjointness with vdo Si ; S2 . . . Sn end delayed finite ; a number of times while respect to the variable v in Algorithm 5 is This notation indicates that the sequence of . . wait (mutex) Si; S2; . . . Sn; other processes enter critical regions ahead signal (mutex);. . . deliberate. statements SI, S2, . . Sn should have ex- of it ; so a process will always be able even- A semaphore is a general programming clusive access to the shared variable v. By The initial signal allows precisely one proc- tually to enter its critical region. The im- tool that can be used to solve arbitrary syn- explicitly associating a critical region with ess to enter its critical region. Once a proc- plementation of critical regions in Algo- chronizing problems. Hence a compiler can- the shared variable on which it operates the ess has consumed the available signal and rithm 5 is therefore correct. Notice that the not always assume that a pair of wait and programmertells the compiler that the shar- entered its critical region, no other process analysis of this concurrent program is stated signal operations on a semaphore initialized ing of this variable among concurrent proc- can enter a critical region associated with in terms of the implicit variables, received, to one delimits a critical region. In particu- esses is a deliberate exception to the rule of the same semaphore until the former proc- sent, and inside. lar, a compiler cannot recognize the fol- disjointness; at the same time, the compiler

Computing Surveys, No. December Vol. 1973 Computing Surveys, Vol. 5, No. December 1973

L

239

;

S, 4, 4, *

240 Per Brinch Hansen I imcurrent Programming Concepts 241 can check that a shared variable is used unbounded, they cannot be represented in a "Q" region w do region v do . . . end ing of concurrent processes is based on our only inside critical regions and can generate computer with a finite word length. On the end ability to execute their interactions strictly code that implements mutual exclusion cor- other hand, actual variables being bound to coend sequentially; Only disjoint processes can rectly. It is perfectly reasonably to use sem- a finite range can only represent the most proceed truly simultaneously. aphores in the underlying implementation recent past of a system's history. We must Process P can enter its region v at the same of this language feature, but at higher levels therefore find a function f(x, r) of the im- time that process Q enters its region w. of programming the explicit of sema- plicit variables x and r with a finite range When process P tries to enter its region w, use 9. CONDITIONAL CRITICAL REGIONS phores to achieve mutual exclusion is de- that can be represented by an actual vari- it will be delayed because Q is already batable. A similar notation for critical re- able v. In Algorithms 5 and 6 the function is inside its region w. And process Q will be We will now consider multiprogramming gions was developed independently by delayed trying to enter its region v because systems in which processes can wait until f(x, r) — x — r v Hoare [6]. — P is already inside its region v. certain conditions are satisfied by other Algorithm 6 shows the use of a shared For a semaphore, the invariant is a rela- The deadlock occurs because the two processes. The classic example is the ex- integer v and two critical regions to solve tionship among the implicit variables rep- processes enter their critical regions in op- change of messages between two processes the previous problem (See also Algorithm resenting all signals sent and received posite order and create a situation in which by means of a buffer of finite capacity as 5). throughout the lifetime of a semaphore: each process is waiting indefinitely for the defined by Algorithm 7. Here the sender Algorithm 6 Critical regions represented completion of a region within the other must be able to wait while the received sent initial buffer is full, by a structured notation 0 + process. It can be proved that a deadlock and the receiver must able to wait while the computer, these implicit variables cannot occur if all processes enter nested buffer is empty. var In a can v: shared integer; be represented by a single integer (equal to regions in the same (hierarchical) order The message buffer v is declared as a begin sent initial received) that remain [I]. A compiler might prevent such dead- sharedrecord consisting of two components: v:= + must 0; non-negative.— locks simply by checking that nested critical a sequence s of messages of some type T, cobegin a strong correct- regions refer to shared variables in the (lin- and an integer defining the number of cycle "P" People with interest in full proofs may helpful to ear) order in which these variables are de- messages currently stored in the sequence. region l end ness well find it de- . . . udou.'=v + . . . clare implicit variables and implicit state- clared in the program. Initially, the buffer is empty (full = 0) . end ments referring to them explicity in the It is an amusing paradox of critical re- Since the buffer has a finite capacity, the cycle "Q" program text. Implicit quantities have no gions that to implement one, we must ap- operations used to send and receive a mes- region vdo print (v) v:= . . . ; 0 on the execution their peal to the existence of simpler critical sage must maintain the following in- end . . effect of a program; buffer sole function is to facilitate program veri- regions (called wait and signal) . The im- variant : end fication by making assumptions about the plementation of wait and signal operations, coend 0 full capacity system's history explicit. in turn, requires the use of an arbiter a end — Implicit variables and statements should hardware implementation of still simpler Algorithm 7 Message buffer critical regions that guarantee exclusive It has been our persistent goal to look for be subject to the following restrictions: v shared access to a semaphore single var : record multiprogramming features that can be un- 1) Assertions about a program may refer by a processor in a multiprocessor system. s: sequenceof T; derstood in time-independent terms. Since to actual as well as implicit variables. This use of nested critical regions at all full: integer; the precise ordering of critical regions in 2) Expressions involving actual and im- continues levels of machine design we end time is unknown, a time-independent as- plicit variables may formally be "evalu- until reach the atomic n: at which nuclear states are m, T; sertion of their net effect an as- ated" and "assigned" to implicit variables. level, known to can only be be discrete and mutually comment send message m; sertion about a propertly of the associated 3) Expressions evaluated and assigned to exclusive. regionv capacity The main conclusion of must whenfull < do shared variable that remains constant in actual variables may only refer to actual this section put(m, s) — be that it is impossible to make as- ; short, an invariant I that must be true after variables. useful 1; Critical regions referring to different sertions about the effect of concurrent state- full:- full + initialization of the variable and before and ments end after each critical region operating it. shared variables can be executed simulta- unless operations on shared variables neously. The use of nested critical regions exclude one another in time. Mutual exclu- A relationship 7 that remains true at all sion comment receive message n; times in reflect the entire can, however, lead to a deadlock unless is necessary to reduce a virtual infinity must some way of possible time-dependent region v when full > 0 do history of processes referring precautions are taken. Consider, for exam- results to a the to the small number of possibilities. get(n,s); shared variable. So we find that the in- ple, the following program with two shared The use of invariant relationships simplifies pro- full:= full — 1; variant for Algorithms 5 expressed variables v and w of types T and T': the end and 6 is gram analysis further. Together, these men- in terms of the total number of jobs exe- var v: shared T' shared T; w: ; tal tools enable us to study concurrent pro- To indicate that the sending of message cuted and reported throughout the existence cobegin a grams in time-independent terms by an must be postponed until capacity, we of the multiprogramming system. region v do region w do end full < However, "P" . . . effort proportional to the number of critical will use the conditional critical region pro- since the range of such implicit variables is end regions used. So in the end, our understand- posed by Hoare [6] :

Computing Surveys, Vol. 5, No. December 1973 Computing Surveys, Vol. 5, No. December 1973

4, 4, 242 Per Brinch Hansen Concurrent Programming Concepts region capacity v whenfull < do . . . end and false by one or more "consumers," the Algorithm 8 Pipeline system The condition for receiving a message completion of the given critical region When the sender enters this conditional can var v:pipeline; tO, t1, . . . , tn: T; through sequence s (i) is the only be delayed a finite number of times by begin critical region, Boolean expression full other critical regions. full(i) 0 capacity is evaluated. If the expression is initialize (v) ; > < We will regions true, the sender completes the execution of use conditional critical to cobegin A sequence s (i) may temporarily contain the critical region by putting a message m solve the following problem [8]: A stream cycle "PO" more than reserved (i) messages. But the of data elements of type T produced by a produce(tO) of type T into the sequence s and increasing total number of messages stored in all se- process PO passes through a sequence of 1) by one. false, the send(tO, v, quences cannot exceed the capacity of the full But if the expression is processes PI, P2, Pn that sender leaves the critical region temporar- . . . , operate on end pipeline. So the system must maintain the the data elements in that order: ily and enters an anonymous queue associ- following invariant: cycle ated with the shared variable v. The sender PO PI P2 >Pn "Pi" -» -> (ti,v,i) full(i) 0 0 isg n will be allowed to reenter and complete the receive ; for Each pair of processes (Pi'-l Pi, update(ti) critical region as soon as the receiver has and available + 23 max(full(i), reserved (i)) where 1 n) a sequence send(ti, i 1 ) i a message (thus i is connected by v, + removed from the buffer (i) making capacity). s that can hold one or more data elements end capacity full < type s(n) = Another conditional critical region is used of T. The sequences 5(1),..., a common cycle It can be shown formally Algorithm to postpone the receiving of a message until are kept in store with a finite "Pn" that 9 (tn, v, n) ; maintains this invariant, 0. If 0, the receiver leaves the capacity. Algorithm 8 gives an overview of receive but the proof is full > full = pipeline (tn) ; tedious and adds nothing critical region temporarily and joins the this system. consume to one's informal end understanding of the anonymous queue. In that case, the critical The common store is declared as a vari- pipeline. region will able vof type pipeline (to be defined later) . j coend In Hoare's conditional critical regions, be continued when the sender has end processes put another message into the buffer and A process Pi receives a message ti of type T | can only be delayed at the begin- ning of a made 0. At this point, the receiver from its predecessor and sends an updated j Algorithm 9 Pipeline system (cont.) critical region. In practice, one full > message must be to will take a message n of type T from the to its successor by means of two able place synchronizing condi- procedures typepipeline shared record tions anywhere sequence s and decrease full by one. = within critical regions as s: array 1 . . nof shown in [2]. In general, a conditional critical region receive (ti,v,i) send(ti,v,i+l) sequenceof T ; The conditional critical region is an im- region v when Bdo Si ; S2 ; . . . ;Sn end Algorithm 9 defines the data type pipe- full, reserved: portant description tool for multiprogram- line and the procedures send A array 1 . . nof integer; ming systems. It remains to be seen whether is used to a and receive. delay the completion of critical pipeline available: integer; it also is a practical programming region until a shared variable v satisfies a is a shared record consisting of tool. The four components: an array of sequences s; end main difficulty of achieving an efficient im- a specific condition B (in addition to an in- procedure send(t: T; var v: pipeline; i: plementation variant 7) an array of integers defining the number of is the reevaluation of syn- . locations l-.n); chronizing conditions each time a critical When a process enters a conditional criti- full ("messages) in each sequence; an array of integers defining region v region is completed. cal region, B is the minimum a Boolean expression evalu- number of whenfull(i) < reserved(i) or available In simple cases, the reevaluation ated. If B is true, the critical region is com- store locations reserved perma- can be nentlyfor transmission of messages through > Odo reduced (but seldom eliminated) by the use pleted by executing the statements SI, S2, each sequence; and integer defining j put(t,s(i)); of semaphores. To do this, one must Sn; otherwise, the process leaves its an the asso- .. number of store locations are j full(i):=full(i) + 1; ciate a semaphore with each synchronizing critical region temporarily and enters a that generally available for transmission messages if full (i) > reserved (i) then condition. When a process makes a condi- queue associated with the shared variable of v. through all sequences (when available:^ available — 1; tion B true, the process must check whether All processes waiting for one condition or they have used up their reserved locations). end other processes are waiting for that another on the variable v enter the same end condi- Initially, all sequences are empty and the tion, and, if so, produce a signal that will queue. When a process completes a critical procedure (var pipeline has been divided into reserved and receive t: T; varv: pipe- enable one (and only one!) of them to con- region on v, the synchronizing conditions of n) generally available storage. sequence line; i: 1 . . tinue. As I have pointed out elsewhere, the waiting processes are reevaluated. If A s(i) [7] at (i) region v the efficiency of this scheme is bought at one of these conditions is satisfied, the cor- can always hold least reserved mes- the sages. When this amount has been used, the whenfull(i) > 0 do expense of increased program complexity. responding process is allowed to reenter and get(t,s(i)); Concurrent complete its critical region. sequence must compete with other sequences statements and critical re- for rest store. it full(i) > reserved (i) then gions seem well-suited to the design The scheduling waiting processes must use of the of the available So of of the available:= available + 1 ; small multiprogramming systems be fair following sense: a process condition for sending a message through dedicated in the If sequence s(i) end to user applications (but is waiting for a condition B that is repeat- is remember that full(i):=full(i) -1; they have not yet been implemented and edly made by or more "producers" full(i) received (i) or true one < available > 0 end used in practice). Although these concepts

Computing Surveys, Vol. 5, No. December 1973 Computing Surveys, Vol. 5, No. December 1973

243

;

; ;

;

4, 4, '*

244 Per Brinch Hansen < 'oncurrent Programming Concepts 245

are simple and well-structured, they do not comments on this paper. I am also grateful for the 9. Dijkstra, E. W. "The structure of The multi- design decisions made in The multiprogram- seem to be adequate for the design of large constructive criticism of Coen Bron, Peter Denning, programming system." Comm ACM 11, 5 ming system (See also 9). Recommends the (May 1968), 341-346. multiprogramming systems (such as oper- Brian Wichmann, Mike Woodger, and thereferees. use of monitors (called "secretaries") for fu- A brief description of the hierarchical struc- ture operating system design. ating systems). The main problem is that ture of The multiprogramming system. 11. Hoare, C. A. R. "A structured paging system." the use of critical regions scattered through- 10. Dijkstra, Computer Journal 16, 3 (August 1973), 209- E. W. "Hierarchical ordering of --J 14. out a program makes it difficult to keep BIBLIOGRAPHY sequential processes." Acta 2 Informatica 1, Illustrates track of how a shared variable is used by 1. (1970,115-138. the use of the monitor concept Brinch Hansen, P. Operating system princi- presented in (1) for the design of a demand concurrent processes. It has therefore re- ples. Prentice-Hall, Englewood New A more extensive motivation of the basic paging system. cently been suggested that one should com- Jersey (July 1973). bine shared variable An introductionto operating systems covering a and the possible sequential and concurrent processes, processor operations on it in a single, syntactic con- and store management, scheduling algorithms struct called a monitor [1, 10, 11]. It is, and resource protection. Describes the RC 4000 multiprogramming system in detail. In- however, too early to speculate about what troduces a programming language notation this approach may lead to. for monitors based on the class concept of Simula 67. 2. Brinch Hansen, P. "Structured multipro- gramming." Comm ACM 15, 7 (July 1972), 10. CONCLUSION 574-578. I A condensed presentation of the viewpoints have tried to show that the design of reli- described in the present paper. It suggests the able multiprogramming systems should be use of event queues within critical regions as guided by two simple a means of reducing the overhead of process principles that are scheduling. equally valid for sequential programming: 1) 3. Wirth, N. "The programming language Pas- It should be possible to understand a cal." Acta Informatica 1, 1 (1971), 35-63. program in time-independent terms by an A highly readable definition of a sequential effort proportional to its size. programming language that combines the al- 2) should be possible to state gorithmic notation of Algol 60 with far more It assump- general datastructures. tions about invariant relationships among 4. Dijkstra, E. W. "Cooperating sequential proc- program components, and have them esses." in Programming Languages, F. Genuys, checked automatically. (Ed.) Academic Press, New York, New York, I have also discussed some specific lan- 1968. guage The classical monograph that introduced con- features for multiprogramming. To current statements, semaphores, and critical avoid misunderstanding, I ask you to re- regions. It also contains Dekker's solution to gard these not as definite proposals but the mutual exclusion problem. merely as illustrations of a common theme. 5. Habermann, A. N. "Synchronization of com- municating processes." Comm ACM 15, 3 Better language concepts for multiprogram- (March 1972), 171-176. ming will undoubtedly be proposed by An axiomatic definition of the semaphore others. But I would expect any realistic operations, wait and signal. proposal to: 6. Hoare, C. A. R. "Towards a theory of parallel 1) distinguish clearly between disjoint programming." in Operating Systems Tech- niques, C. A. R. Hoare and R. H. Perrott, and interacting processes; (Eds), Academic Press, New York, New York, 2) associate shared data explicitly with 1973. operations defined on them; The original proposal for conditional critical 3) regions which includes an axiomatic definition ensure mutual exclusion of these op- of disjoint processes and critical regions. erations in time ; and 7. Brinch Hansen, P. "A comparison of two syn- I 4) include synchronizing primitives that chronizing concepts." Acta Informatica 1, 3 | permit partial or complete programmercon- (1972), 190-199. . trol of process scheduling. A comparison of the use of semaphores and conditional critical regions to solve a schedul- ing problem. 8. Dijkstra, E. W. "Information streams sharing ACKNOWLEDGEMENTS a finite buffer." Information Processing Letters 1, (1972), 179-180. I am indebted to my students Ram Rao, David A solution to the "pipeline"problem by means ' and Sankaran Srinivas for many helpful of conditional critical regions.

Computing Surveys, Vol. 5, No. 4, December 1973 Computing Surveys, Vol. No. 4, December 1973

Cliffs,

Smith,

S,