The Design and Implementation of the Reactor An Object-Oriented Framework for Event Demultiplexing (Part 2 of 2) Douglas C. Schmidt [email protected] http://www.cs.wustl.edu/schmidt/ Department of Computer Science Washington University, St. Louis 63130 An earlier version of this paper appeared in the September I/O-based and/or timer-based demultiplexing mechanisms. 1993 issue of the C++ Report. The following paragraphs describe the primary features of- fered by the Reactor framework: 1 Introduction Export Uniform Object-Oriented Interfaces: Appli- cations using the Reactor do not access the lower-level This is the second half of the third article in a series that de- I/O demultiplexing system calls directly. Instead, they in- scribes techniques for encapsulating existing operating sys- herit from a common Event Handler abstract base class tem (OS) interprocess communication (IPC) services using to form composite concrete derived classes (as illustrated object-oriented (OO) C++ wrappers. In the ®rst half of this in Figure 1). The Event Handler base class speci®es a article, a client/server application example was presented to uniform interface consisting of virtual methods that handle motivate the utilityof wrappers that encapsulate event demul- (1) synchronous input, output, and exceptions and (2) timer- tiplexing mechanisms [1]. Event demultiplexing is useful for based events. Applications create objects of these derived developing event-driven network servers that receive and classes and register them with instances of the Reactor. process data arriving from multiple clients simultaneously. Automate Event Handler Dispatching: When events The previous article also examined the advantages and dis- occur, the Reactor automatically invokes the appropri- advantages of several alternative I/O demultiplexing schemes ate virtual method event handler(s) belonging to the pre- such as non-blockingI/O,multipleprocess or thread creation, registered derived class objects. Since C++ objects are reg- and synchronous I/O demultiplexing (via the select and istered with the Reactor (as opposed to stand-alone sub- poll system calls). routines), any context information associated with an object This article focuses on the design and implementation of is retained between invocations of its methods. This is par- an object-oriented framework called the Reactor.The ticularly useful for developing ªstatefulº services that retain Reactor provides a portable interface to an integrated col- information in between client invocations. lection of extensible, reusable, and type-secure C++ classes Support Transparent Extensibility: The functionality select poll that encapsulate and enhance the and event of both the Reactor and its registered objects may be ex- demultiplexing mechanisms [2]. To help simplify network tended transparently without modifying or recompiling ex- programming, the Reactor combines the demultiplexing isting code. To accomplish this, the Reactor framework of synchronous I/O-based events together with timer-based employs inheritance, dynamic binding, and parameterized events. When these events occur, the Reactor automati- types to decouple the lower-level event demultiplexing and cally dispatches the method(s) of previously registered ob- service dispatching mechanisms from the higher-level appli- jects to perform application-speci®ed services. cation processing policies. Example low-level mechanisms This article is organized as follows: Section 2 describes include (1) detecting events on multiple I/O handles, (2) the primary features offered by the Reactor framework; handling timer expiration, and (3) invoking the appropriate Section 3 outlines the object-oriented design and implemen- method event handler(s) in response to events. Likewise, tation of the framework; Section 4 presents a distributed application-speci®ed policies include (1) connection estab- logging example that demonstrates how the Reactor sim- lishment, (2) data transmission and reception, and (3) pro- pli®es the development of concurrent, event-driven network cessing service requests from other participating hosts. applications; and Section 5 discusses concluding remarks. Increase Reuse: The Reactor's demultiplexing and dispatching mechanisms may be reused by many network ap- 2 Primary Features of the Reactor plications. By reusing rather than redeveloping these mech- anisms, developers are free to concentrate on higher-level The Reactor provides an object-oriented interface that sim- application-related issues, rather than repeatedly wrestling pli®es the development of distributedapplications that utilize with lower-level event demultiplexing details. In addition, 1 subsequent bug-®xes and enhancements are transparently shared by all applications that utilize the Reactor's com- ponents. Conversely, developers that access select and : Logging : Logging : Logging REGISTERED poll directly must reimplement the same demultiplexing Handler Handler Acceptor OBJECTS LEVEL and dispatching code for every network application. More- LEVEL : Event : Event : Event over, any modi®cations and improvements to this code must Handler Handler Handler APPLICATION be replicated manually in all related applications. APPLICATION Enhance Type-Security: The Reactor shields applica- 1: handle_input() tion developers from error-prone,low-level details associated with programming existing I/O demultiplexing system calls. :Timer : Handle These details involve setting and clearing bitmasks, handling Table : Signal LEVEL Queue LEVEL Handlers timeouts and interrupts, and dispatching ªcall-backº meth- FRAMEWORK FRAMEWORK : Reactor ods. In particular, the Reactor eliminates several subtle causes of errors with poll and select that involve the OS EVENT DEMULTIPLEXING INTERFACE misuse of I/O handles and fd set bitmasks. Reactor LEVEL Improve Portability: The also shields appli- LEVEL KERNEL cations from differences between select and poll that KERNEL impede portability. As illustrated in Figure 5, the Reactor Figure 1: The Reactor Inheritance Hierarchy exports the same interface to applications, regardless of the underlying event demultiplexingsystem calls. Moreover, the Reactor's object-oriented architecture improves its own in- contains support for multi-threading. In general, a single in- ternal portability. For example, porting the Reactor from stance of the Reactor is active in a thread at any point in a select-based OS platform to a poll-based platform re- time. However, there may be multiple different instances of quired only a few well-de®ned changes to the framework. Reactor objects running in separate threads in a process. The framework provides the necessary synchronization op- In addition to simplifying application development, the erations to prevent race conditions [4]. Reactor also performs its demultiplexing and dispatching tasks ef®ciently. In particular, its event dispatching logic improves upon common techniques that use select di- 3.1 Platform-Independent Class Components rectly. For instance, the select-based Reactor uses The following paragraphs summarize the salient charac- an ACE Handle Set class (described in Section 3.2) that teristics of the three platform-independent classes in the avoids examining fd set bitmasks one bit at a time in many Reactor:theACE Time Value, ACE Timer Queue, circumstances. An article in a future issue of the C++ Report and ACE Event Handler classes: will empirically evaluate the performance of the Reactor and compare it against a non-object-oriented solution written ACE Time Value: This class provides a C++ wrap- in C that accesses I/O demultiplexing system calls directly. per that encapsulates an underlying OS platform's date and time structure (such as the struct timeval data type on UNIX and POSIX platforms). The timeval structure 3 The Object-Oriented Design and Im- contains two ®elds that represent time in terms of seconds and microseconds. However, other OS platforms use differ- plementation of the Reactor ent representations, so the ACE Time Value class abstracts these details to provide a portable C++ interface. This section summarizes the object-oriented design of the The primary methods in the ACE Time Value class are Reactor framework's primary class components, focus- illustrated in Figure 2. The ACE Time Value wrapper ing on the interfaces and strategic design decisions. Where uses operator overloading to simplify time-based compar- appropriate, tactical design decisions and certain implemen- isons within the Reactor. Overloading permits the use of tation details are also discussed. Section 3.1 outlines the OS standard arithmetic syntax for relational expressions involv- platform-independent components; Section 3.2 covers the ing time comparisons. For example, the following code cre- platform-dependent components. ates two ACE Time Value objects constructed by adding The Reactor was originally modeled after a C++ wrap- user-supplied command-line arguments to the current time, per for select called the Dispatcher that is available and then displaying the appropriate ordering relationship be- with the InterViews distribution [3]. The Reactor frame- tween the two objects: work described here includes several additional features. For example, the Reactor operates transparently on top of int main (int argc, char *argv[]) both the System V Release 4 poll interface and select, { if (argc != 3) { which is available on both UNIX and PC platforms (via cerr << "usage: " << argv[0] the WINSOCK API). In addition, the Reactor framework