APPLYING THE PROACTOR PATTERN TO HIGH-PERFORMANCE WEB SERVERS James Hu Irfan Pyarali Douglas C. Schmidt [email protected],edu [email protected] [email protected] Department of Computer Science, Washington University St. Louis, MO 63130, USA This paper is to appear at the 10th International Confer- 1 INTRODUCTION ence on Parallel and Distributed Computing and Systems, IASTED, Las Vegas, Nevada, October 28-31, 1998. Computing power and network bandwidth on the Internet has increased dramatically over the past decade. High- speed networks (such as ATM and Gigabit Ethernet) and high-performance I/O subsystems (such as RAID) are be- ABSTRACT coming ubiquitous. In this context, developing scalable Web servers that can exploit these innovations remains a Modern operating systems provide multiple concurrency key challenge for developers. Thus, it is increasingly im- mechanisms to develop high-performance Web servers. portant to alleviate common Web server bottlenecks, such Synchronous multi-threading is a popular mechanism for as inappropriate choice of concurrency and dispatching developing Web servers that must perform multiple oper- strategies, excessive filesystem access, and unnecessary ations simultaneously to meet their performance require- data copying. ments. In addition, an increasing number of operating sys- Our research vehicle for exploring the performance im- tems support asynchronous mechanisms that provide the pact of applying various Web server optimization tech- benefits of concurrency, while alleviating much of the per- niques is the JAWS Adaptive Web Server (JAWS) [1]. JAWS formance overhead of synchronous multi-threading. is both an adaptive Web server and a development frame- This paper provides two contributions to the study of work for Web servers that runs on multiple OS platforms high-performance Web servers. First, it examines how syn- including Win32, most versions of UNIX, and MVS Open chronous and asynchronous event dispatching mechanisms Edition. impact the design and performance of JAWS, which is our Our experience [2] building Web servers on multiple OS high-performance Web server framework. The results re- platforms demonstrates that the effort required to optimize veal significant performance improvements when a proac- performance can be simplified significantly by leveraging tive concurrency model is used to combine lightweight con- OS-specific features. For example, an optimized file I/O currency with asynchronous event dispatching. system that automatically caches open files in main mem- In general, however, the complexity of the proactive con- ory via mmap greatly reduces latency on Solaris. Like- currency model makes it harder to program applications wise, support for asynchronous event dispatching on Win- that can utilize asynchronous concurrency mechanisms ef- dows NT can substantially increase server throughput by fectively. Therefore, the second contribution of this paper reducing context switching and synchronization overhead describes how to reduce the software complexity of asyn- incurred by multi-threading. chronous concurrent applications by applying the Proactor Unfortunately, the increase in performance obtained pattern. This pattern describes the steps required to struc- through the use of asynchronous event dispatching on ex- ture object-oriented applications that seamlessly combine isting operating systems comes at the cost of increased concurrency with asynchronous event dispatching. The software complexity. Moreover, this complexity is fur- Proactor pattern simplifies concurrent programming and ther compounded when asynchrony is coupled with multi- improves performance by allowing concurrent application threading. This style of programming, i.e., proactive pro- to have multiple operations running simultaneously with- gramming, is relatively unfamiliar to many developers ac- out requiring a large number of threads. customed to the synchronous event dispatching paradigm. This paper describes how the Proactor pattern can be This work was supported in part by Siemens Med and Siemens Cor- applied to improve both the performance and the design porate Research. of high-performance communication applications, such as 1 Web servers. Strategy, I/O Strategy, Protocol Pipeline, Protocol Han- A pattern represents a recurring solution to a software dlers,andCached Virtual Filesystem. Each framework is development problem within a particular context [3]. Pat- structured as a set of collaborating objects implemented us- terns identify the static and dynamic collaborations and in- ing components in ACE [5]. The collaborations among teractions between software components. In general, ap- JAWS components and frameworks are guided by a family plying patterns to complex object-oriented concurrent ap- of patterns, which are listed along the borders in Figure 1. plications can significantly improve software quality, in- An outline of the key frameworks, components, and pat- crease software maintainability, and support broad reuse terns in JAWS is presented below; Section 4 then focuses of components and architectural designs [4]. In particu- on the Proactor pattern in detail.1 lar, applying the Proactor pattern to JAWS simplifies asyn- chronous application development by structuring the de- Event Dispatcher: This component is responsible for co- multiplexing of completion events and the dispatching of ordinating JAWS’ Concurrency Strategy with its I/O Strat- their corresponding completion routines. egy. The passive establishment of connection events with The remainder of this paper is organized as follows: Sec- Web clients follows the Acceptor pattern [6]. New incom- tion 2 provides an overview of the JAWS server framework ing HTTP request events are serviced by a concurrency design; Section 3 discusses alternative event dispatching strategy. As events are processed, they are dispatched to strategies and their performance impacts; Section 4 ex- the Protocol Handler, which is parameterized by an I/O plores how to leverage the gains of asynchronous event dis- strategy. JAWS ability to dynamically bind to a particu- patching through application of the Proactor pattern; and lar concurrency strategy and I/O strategy from a range of Section 5 presents concluding remarks. alternatives follows the Strategy pattern [3]. Concurrency Strategy: This framework implements con- 2 JAWS FRAMEWORK OVERVIEW currency mechanisms (such as single-threaded, thread-per- request, or thread pool) that can be selected adaptively at Figure 1 illustrates the major structural components and run-time using the State pattern [3] or pre-determined at design patterns that comprise the JAWS framework [1]. initialization-time. The Service Configurator pattern [7] is JAWS is designed to allow the customization of various used to configure a particular concurrency strategy into a Web server at run-time. When concurrency involves multi- Reactor/Proactor Strategy Singleton ple threads, the strategy creates protocol handlers that fol- Memento I/O Strategy Cached Virtual low the Active Object pattern [8]. Framework Filesystem State I/O Strategy: This framework implements various I/O mechanisms, such as asynchronous, synchronous and re- Asynchronous Completion Token Tilde ~ active I/O. Multiple I/O mechanisms can be used simulta- Expander /home/... Protocol Acceptor neously. In JAWS, asynchronous I/O is implemented using Event Dispatcher Handler the Proactor pattern [9], while reactive I/O is accomplished Protocol through the Reactor pattern [10]. These I/O strategies may Filter utilize the Memento [3] and Asynchronous Completion To- Service Configurator Service Configurator ken [11] patterns to capture and externalize the state of a Adapter request so that it can be restored at a later time. Concurrency Protocol Pipeline Strategy Protocol Handler: This framework allows system devel- Framework Framework State opers to apply the JAWS framework to a variety of Web Pipes and Filters Active Object Strategy system applications. A Protocol Handler is parameter- ized by a concurrency strategy and an I/O strategy. These strategies are decoupled from the protocol handler using Figure 1: Architectural Overview of the JAWS Framework the Adapter pattern [3]. In JAWS, this component imple- Web server strategies in response to environmental factors. ments the parsing and handling of HTTP/1.0 request meth- These factors include static factors (e.g., number of avail- ods. The abstraction allows for other protocols (such as able CPUs, support for kernel-level threads, and availabil- HTTP/1.1, DICOM, and SFP [12]) to be incorporated eas- ity of asynchronous I/O in the OS), as well as dynamic fac- ily into JAWS. To add a new protocol, developers simply tors (e.g., Web traffic patterns and workload characteris- write a new Protocol Handler implementation, which is tics). then configured into the JAWS framework. JAWS is structured as a framework of frameworks.The 1Due to space limitations it is not possible to describe all the patterns overall JAWS framework contains the following compo- mentioned below in detail. The references provide complete coverage of nents and frameworks: an Event Dispatcher, Concurrency each pattern, however. 2 Protocol Pipeline: This framework allows filter operations Efficiency: The server must minimize latency, maximize to be incorporated easily with the data being processed by throughput, and avoid utilizing the CPU(s) unnecessarily. the Protocol Handler. This integration is achieved by em- Adaptability: Integrating new or improved transport