Half-Sync/Half-Async An Architectural Pattern for Ef®cient and Well-structured Concurrent I/O Douglas C. Schmidt and Charles D. Cranor [email protected] and [email protected] Department of Computer Science Washington University St. Louis, MO 63130, (314) 935-7538 An earlier version of this paper appeared in a chapter in the book ªPattern Languages of Program Design 2º ISBN 0-201-89527-7, edited by John Vlissides, Jim Coplien, and Norm Kerth published by Addison-Wesley, 1996. SYNC USER PROCESS SYNC USER 2 LEVEL - Abstract PROCESS1 PROCESSES This paper describes the Half-Sync/Half-Async pattern, USER which integrates synchronous and asynchronous I/O models SYNC USER to support both programming simplicity and execution ef®- PROCESS3 ciency in complex concurrent software systems. In this pat- 1, 4: read(data) tern, higher-level tasks use a synchronous I/O model, which simpli®es concurrent programming. In contrast, lower-level SOCKET QUEUES tasks use an asynchronous I/O model, which enhances ex- LAYER SOCKET ecution ef®ciency. This pattern is widely used in operating 3: enqueue (data) systems such as UNIX, Mach, Windows NT, and VMS, as well as other complex concurrent systems. ASYNC PROTOCOL PROCESSING 1 Intent 2: interrupt KERNEL BSD UNIX The Half-Sync/Half-Async pattern decouples synchronous NETWORK INTERFACES I/O from asynchronous I/O in a system to simplify concurrent programming effort without degrading execution ef®ciency. Figure 1: BSD UNIX Software Architecture 2 Motivation available the process will sleep until the data arrives from the To illustrate the Half-Sync/Half-Async pattern, consider the network. software architecture of the BSD UNIX [1] networking sub- In the BSD architecture, the kernel performs I/O asyn- system shown in Figure 1. The BSD UNIX kernel coor- chronously in response to device interrupts. In contrast, dinates I/O between asynchronous communication devices user-level applications perform I/O synchronously. This sep- (such as network adapters and terminals) and applications aration of concerns into a ªhalf synchronous and half asyn- running on the OS. Packets arriving on communication de- chronousº concurrent I/O structure resolves the following vices are delivered to the OS kernel via interrupt handlers two forces: initiated asynchronously by hardware interrupts. These han- Need for programming simplicity: Programming an dlers receive packets from devices and trigger higher layer asynchronous I/O model can be complex because input and protocol processing (such as IP, TCP, and UDP). Valid pack- output operations are triggered by interrupts. Asynchrony ets containing application data are queued at the Socket layer. can cause subtle timing problems and race conditions when The OS then dispatches any user processes waiting to con- the current thread of control is preempted by an interrupt sume the data. These processes synchronously receive data handler. Moreover, interrupt-driven programs require ex- from the Socket layer using the read system call. A user tra data structures in addition to the run-time stack. These process can make read calls at any point; if the data is not data structures are used to save and restore state explicitly 1 when events occur asynchronously. In addition, debugging SYNC asynchronous programs is hard since external events occur SYNC TASK at different points of time during program execution. TASK 1 3 In contrast, programming applications with a synchronous I/O model is easier because I/O operations occur at well de- ®ned points in the processing sequence. Moreover, programs that use synchronous I/O can block awaiting the completion SYNC TASK LAYER TASK LAYER TASK SYNCHRONOUS 2 of I/O operations. The use of blocking I/O allows programs SYNCHRONOUS to maintain state information and execution history in a run- time stack of activation records, rather than in separate data 1, 4: read(data) structures. Thus, there is a strong incentive to use a syn- chronous I/O model to simplify programming. LAYER LAYER MESSAGE QUEUES QUEUEING QUEUEING Need for execution ef®ciency: The asynchronous I/O model maps ef®ciently onto hardware devices that are driven 3: enqueue(data) by interrupts. Asynchronous I/O enables communication and ASYNC computation to proceed simultaneously. In addition, context TASK switching overhead is minimized because the amount of in- 2: interrupt formation necessary to maintain program state is relatively EXTERNAL small [2]. Thus, there is a strong incentive to use an asyn- EVENT SOURCES TASK LAYER chronous I/O model to improve run-time performance. TASK LAYER ASYNCHRONOUS In contrast, a completely synchronous I/O model may be ASYNCHRONOUS inef®cient if each source of events (such as network adapter, terminal, or timer) is associated with a separate active object Figure 2: The Structure of Participants in the Half-Sync/Half- (such as a process or thread). Each of these active objects Async Pattern contain a number of resources (such as a stack and a set of registers) that allow it to block while waiting on its source ± it is inef®cient to dedicate a separate thread of of events. Thus, this synchronous I/O model increases the control to perform synchronous I/O for each source time and space required to create, schedule, dispatch, and of external events, and terminate separate active objects. ± the higher-level tasks in the system can be sim- pli®ed signi®cantly if I/O is performed syn- 3 Solution chronously. One or more tasks in a system must runinasingle- To resolve the tension between the need for concurrent thread of control, while other tasks may bene®t from programming simplicity and execution ef®ciency use the multi-threading. Half-Sync/Half-Async pattern. This pattern integrates syn- chronous and asynchronous I/O models in an ef®cient and ± For example, legacy libraries like X windows and well-structured manner. In this pattern, higher-level tasks Sun RPC are often non-reentrant. Therefore, mul- (such as database queries or ®le transfers) use a synchronous tiple threads of control cannot safely invoke these I/O model, which simpli®es concurrent programming. In library functions concurrently. However, to ensure contrast, lower-level tasks (such as servicing interrupts from quality of service or to take advantages of multiple network controllers) use an asynchronous I/O model, which CPUs, it may be necessary to perform bulk data enhances execution ef®ciency. Because there are usually transfers or database queries in separate threads. more high-level tasks than low-level tasks in a system, this The Half-Sync/Half-Async pattern can be used to pattern localizes the complexity of asynchronous processing decouple the single-threaded portions of an ap- within a single layer of a software architecture. Communi- plication from the multi-threaded portions. This cation between tasks in the Synchronous and Asynchronous decoupling enables non-reentrant functions to be layers is mediated by a Queueing layer. used correctly, without requiring changes to exist- ing code. 4 Applicability 5 Structure and Participants Use the Half-Sync/Half-Async pattern when Figure 2 illustrates the structure of participants in the Half- A system possesses the following characteristics: Sync/Half-Async pattern. These participants are described below. ± The system must perform tasks in respond to ex- ternal events that occur asynchronously, and Synchronous task layer (User processes) 2 ± The tasks in this layer perform high-level I/O oper- ations that transfer data synchronously to message External Async Message Sync queues in the Queueing layer. Unlike the Asyn- Event Source Task Queue Task chronous layer, tasks in the Synchronous layer are notification() EXTERNAL EVENT active objects [3] that have their own run-time read(msg) stack and registers. Therefore, they can block RECV MSG ASYNC PHASE work() while performing synchronous I/O. PROCESS MSG enqueue(msg) Socket layer Queueing layer ( ) ENQUEUE MSG read(msg) ± This layer provides a synchronization and buffer- PHASE DEQUEUE MSG ing point between the Synchronous task layer and QUEUEING work() the Asynchronous task layer. I/O events processed EXECUTE TASK by asynchronous tasks are buffered in message SYNC PHASE queues at the Queueing layer for subsequent re- PHASE trieval by synchronous tasks (and vice versa). Figure 3: Collaboration between Layers in the Half- Asynchronous task layer (BSD UNIX kernel) Sync/Half-Async Pattern ± The tasks in this layer handle lower-level events from multiple external event sources (such as net- messages. The key to understanding the pattern is to recog- work interfaces or terminals). Unlike the Syn- nize that Synchronous tasks are active objects. Thus, they chronous layer, tasks in the Asynchronous layer can make blocking read or write calls at any point in are passive objects that do not have their own run- accordance with their protocol. If the data is not yet avail- time stack or registers. Thus, they cannot block able tasks implemented as active objects can sleep until the inde®nitely on any single source of events. data arrives. In contrast, tasks in the Asynchronous layer are passive objects. Thus, they cannot block on read calls. External event sources (Network interfaces) Instead, tasks implemented as passive objects are triggered ± External devices (such as network interfaces and by noti®cations or interrupts from external sources of events. disk controllers) generate events that are received and processed by the Asynchronous task layer. 7 Consequences 6 Collaborations The Half-Sync/Half-Async pattern yields the following ben- e®ts: Figure 3 illustrates the dynamic collaboration among partici- pants in the Half-Sync/Half-Async pattern when input events Higher-level tasks are simpli®ed because they are arrive at an external event source (output event processing is shielded from lower-level asynchronous I/O. Complex similar). These collaborations are divided into the following concurrency control, interrupt handling, and timing is- three phases: sues are delegated to the Asynchronous task layer.