sign in sign up
<<
Home , Per Brinch Hansen

PART III HARNESSING PARALLELISM

The primary impetus behind the study of parallel programming, or concurrent programming as some would call it, was the necessity to write operating systems. While the first computers were simple enough to be considered sequential in nature, the advent of faster central processing units and the accompanying discrepancy in speed between the central processor and periph• eral devices forced the consideration of possible concurrent activity of these mechanisms. The was asked to control concurrent activity, and to control it in such a way that concurrent activity was maximized This desire for concurrent activity resulted in machine features such as the channel and the interrupt-the mechanism by which termination of an I/O activity is signaled The main problem with the interrupt, as far as the operating system designer was concerned, was its inherent unpredictability. Not knowing exactly when an interrupt could occur, the operating system had to be designed to expect an interrupt at any time. With the advent of multiprogramming and interactive, on-line processing, it became clear that the original motivation for parallelism-the control and exploitation of concurrent I/O-was only a special case, and some fundamen• tal concepts began to emerge for study and use in more general parallel programming situations. Thus emerged the general idea of a set of processes executing concurrently, with the need for synchronization and mutual exclu• sion (with respect to time) of some of their activities. The past 10-15 years have seen an enormous amount of activity in studying the general problem, a very small part of which is presented here. Now, the article An axiomatic proof technique for parallel programs in Part II presents a theory of parallel programming that can be quite useful in dealing with certain kinds ofparallelism. However, for many tasks it is far too general. In effect, using it is an attempt to ride the parallel programming horse bareback, relying on theory and skill but without any physical aids. Most riders need equipment, in the form of constructs, to facilitate riding the horse. As early as 1964-1965 PL/1 did furnish such equipment: tasks, event names, on-conditions, and so forth. But, this equipment did not sell well, partly 200 II HARNESSING PARALLELISM because it was not based on a good theory or methodology for understanding parallelism, and partly because of the odd way it was attached to the sequential part of the language. (For example, the procedure is confused with the task,' the on-unit is a bewildering mixture of executable statement and declaration.) So let us see what other equipment has been developed An important step was the invention of the by Edsger W. Dijkstra, which took place around 1965 [Dijkstra 68c*]. A semaphore s can be operated on by the operators V(s) and P(s), which the reader may know as signal(s) and wait(s). Semaphores were used in the "bottom layer" of the T.H.E. Multiprogramming System [Dijkstra 6&1*, 71d*] in order to hide the uglier primitive of the hardware, the interrupt. The monograph [Dijkstra 68c*] gave the field a firm foundation by introducing and analyzing critical sec• tions, , semaphores, message buffers, and deadlock. The article [Dijkstra 71 d*] provides more analysis, and also argues against earlier synchronization mechaniSms, such as wait and cause operating on event variables. The semaphore was recognized early as a rather primitive construct, and attempts were made to develop other synchronization and exclusion mechanisms. The critical region construct, region r do S is generally attri• buted to Tony (C.A.R.) Hoare and . Here, resource r is a group of shared variables. Any references to them must be within a region r do S statement, and only one process can execute a critical region for r at a time (others must wait their turn). And now we see what such equipment can do for us: the problem oj insuring that at most one process at a time can reference a particular shared variable is reduced to a syntactic, almost trivial exercise! To provide synchronization, Hoare extended the construct to the condi• tional critical region witb r when B do S; in addition to waiting its turn to use the resource r, a process had to wait until its guard B was true before executing S. Brinch Hansen, on the other hand, added queuing variables to provide for synchronization. Their articles on this topic, Towards a theory of parallel programming and Structured multiprogramming are reproduced in Part III. Besides reading two interesting viewpoints on synchronization, the reader will find good discussions of objectives for and difficulties with theories of parallel program• ming. Hoare's article also contains initial investigations into providing an axiomatic definition for the conditional critical region,' this work was extended by Susan Owicki in her PhD. thesis (see [Owicki 75*, 76a*]). The next tool to be developed in this line was the monitor. In [Dijkstra 71 d*], Dijkstra proposed taking out from N processes the critical sections that refer to a set of common variables and combining them to form a new (N + l)st process, called a secretary. Thus, all references to the shared variables were to be made in a single process, the secretary. The term monitor and a language notation for it (based on the 67 class concept) are due to Brinch Hansen, while the concept was then also developed by Hoare. II HARNESSING PARALLELISM 201

The last two articles in Part III allow for further comparison of their ideas; we present Hoare's 1974 article, Monitors: an operating system structuring concept and Brinch Hansen's 1975 article, The programming language . Again, both articles are worthwhile not only for their discussion of the construct-in this case the monitor-but also for their expression of a general philosophy and methodology for parallel programming. After reading these articles, the reader may wish to turn to Brinch Hansen's new book [Brinch Hansen 77*], which gives the complete Solo operating system written in Concurrent Pascal. Thus far, I have outlined the history of one line of equipment for riding the parallel programming horse. While this is the only line presented in detail in Part Ill, the reader might wish to at least know what else is available from other manufacturers. Petri nets, developed by Carl A. Petri in his 1962 thesis, have long been used by some as a tool in reasoning about parallel programming. The reader is referred to the Computing Surveys article [Peterson 77], which contains 93 further references to the literature. Message passing offers another viable approach to harnessing parallelism, and may very well be better suited for distributed networks. In fact, message passing was used in the time sharing operating system for the CDC 6600, which, with its ten peripheral processors, can be considered to be a distributed network [Harrison 67]. A better explanation of the message passing concept can be found in [Brinch Hansen 70*]. With respect to semaphores, A. Nico Habermann gave one of the first formalizations, using "auxiliary variables", in [Habermann 72]. Others have extended the semaphore concept to make it more flexible, but it remains too primitive. Habermann is also responsible, along with Roy H. Campbel~ for introduc• ing path expressions as a means of expressing synchronization requirements [Campbell 74]. By means of a regular expression over the names ofprocedures or statements, one describes all allowed execution sequences of those proce• dures or statements, and it is up to the implementation to assure that this requirement is met. Finally, 's new language should be mentioned [Wirth 77a*,77b*]. MODULA, which uses the monitor concept, has been designed with so-called real-time programming in mind. The article [Wirth 77b*] begins the development of a methodology for real-time programming.