Motivation
Using Design Patterns and
Frameworks to Develop
Developing ecient, robust, extensible, portable,
and reusable communication software is
Object-Oriented Communication
hard
Systems
It is essential to understand successful tech-
niques that have proven e ective to solve
Douglas C. Schmidt
common development challenges
www.cs.wustl.edu/schmidt/
Design patterns and frameworks help to
capture, articulate, and instantiate these
Washington University, St. Louis
successful techniques
2 1
Observations
Design Patterns
Develop ers of communication software con-
front recurring challenges that are largely
Design patterns represent solutions to prob-
application-indep endent
lems that arise when developing software
within a particular context
{ e.g., service initialization and distribution, error
handling, ow control, event demultiplexing,
{ i.e., \Patterns == problem/solution pairs in a
concurrency control
context"
Patterns capture the static and dynamic
Successful develop ers resolve these chal-
structure and collab oration among key par-
lenges by applying appropriate design pat-
ticipants in software designs
terns
{ They are particularly useful for articulating how
and why to resolve non-functional forces
However, these patterns have traditionally
b een either:
Patterns facilitate reuse of successful soft-
1. Lo cked inside heads of exp ert develop ers
ware architectures and designs
2. Buried in source co de
3 4
Proxy Pattern
Graphical Notation
1: METHOD : BROKER CALL
: QUOTER 4: METHOD OBJECT PROXY RETURN PROCESS : CLASS 2: FORWARD THREAD REQUEST
3: RESPONSE
CLASS CLIENT CLASS TEMPLATE UTILITY : QUOTER CLASS NETWORK
CLASS CATEGORY INHERITS INSTANTIATES
SERVER
ABSTRACT CLASS
A CONTAINS USES
Intent: provide a surrogate for another
object that controls access to it
5 6
Frameworks
A framework is:
More Observations
{ \An integrated collection of comp onents that
collab orate to pro duce a reusable architecture
Reuse of patterns alone is not sucient
for a family of related applications"
{ Patterns enable reuse of architecture and de-
sign knowledge, but not co de directly
Frameworks di er from conventional class
libraries:
1. Frameworks are \semi-complete" applications
To be pro ductive, develop ers must also
reuse detailed designs, algorithms, inter-
2. Frameworks address particular application do-
mains
faces, implementations, etc.
3. Frameworks provide \inversion of control"
Application frameworks are an e ective
way to achieve broad reuse of software
Typically, applications are develop ed by in-
heriting from and instantiating framework
comp onents
7 8
Tutorial Outline
Di erences Between Class
Libraries and Frameworks
Outline key challenges for developing com-
NETWORKING
software APPLICATION munication SPECIFIC LOGIC
INVOKES MATH ADTS
Present the key reusable design patterns
framework comp onents in high-p erformance
EVENT USER and
eb clients and servers DATA W LOOP INTERFACE
BASE
{ Both single-threaded and various multi-threaded
are presented (A) CLASS LIBRARY solutions
ARCHITECTURE
{ The patterns and frameworks covered general-
ize to other communication software systems NETWORKING USER
MATH
e.g., ORBs, video-on-demand, medical imag- INTERFACE
APPLICATION ing CALL INVOKES SPECIFIC BACKS LOGIC EVENT
LOOP
Discuss lessons learned from using pat-
ADTS
and frameworks on pro duction soft-
DATABASE terns
ware systems
(B) APPLICATION FRAMEWORK
e.g., telecom, avionics, medical systems
ARCHITECTURE {
10 9
Concurrency vs. Parallelism
Stand-alone vs. Distributed
Application Architectures
SERVER PRINTER maxfdp1 read_fds COMPUTER CLIENT WORK FILE REQUEST CD ROM SYSTEM WORK WORK WORK CLIENT REQUEST REQUEST REQUEST (1) STAND-ALONE APPLICATION ARCHITECTURE CLIENT CLIENT
TIME CONCURRENT SERVER NAME SERVICE CYCLE SERVICE SERVICE SERVER DISPLAY
SERVICE CPU1 CPU2 CPU3 CPU4
FI LE CLIENT NETWORK SERVICE WORK REQUEST PRINT SERVICE WORK WORK WORK CD ROM CLIENT REQUEST REQUEST REQUEST
PRINTER FILE SYSTEM CLIENT CLIENT
(2) DISTRIBUTED APPLICATION ARCHITECTURE PARALLEL SERVER
11 12
Sources of Complexity
Sources of Complexity cont'd
Distributed application development exhibits
b oth inherent and accidental complexity
Accidental complexity results from limita-
tions with to ols and techniques, e.g.,
Inherent complexity results from funda-
mental challenges, e.g.,
{ Low-level to ols
{ Distributed systems
e.g., Lack of typ e-secure, portable, re-entrant,
and extensible system call interfaces and com-
Latency
p onent libraries
Error handling
{ Inadequate debugging supp ort
Service partitioning and load balancing
{ Widespread use of algorithmic decomp osition
{ Concurrent systems
Fine for explaining network programming con-
cepts and algorithms but inadequate for de-
Race conditions
veloping large-scale distributed applications
Deadlo ck avoidance
{ Continuous rediscovery and reinvention of core
concepts and comp onents
Fair scheduling
Performance optimization and tuning
14 13
OO Contributions
Concurrent Web Client/Server
Example
Communication software has traditionally
b een p erformed using low-level OS mech-
anisms, e.g.,
The following example illustrates a con-
{ fork/exec
current OO architecture for a high-p erformance
{ Shared memory
Web client/server
{ Signals
Key system requirements are:
{ So ckets and select
1. Robust implementation of HTTP proto col
{ POSIX pthreads, Solaris threads, Win32 threads
{ i.e., resilient to incorrect or malicious Web
clients/servers
OO design patterns and frameworks ele-
vate fo cus to application concerns, e.g.,
2. Extensible for use with other proto cols
{ Service functionality and p olicies
{ e.g., DICOM, HTTP 1.1, SFP
{ Service con guration
3. Leverage multi-pro cessor hardware and OS soft-
{ Concurrent event demultiplexing and event han-
ware
dler dispatching
{ e.g., Supp ort various concurrency mo dels
{ Service concurrency and synchronization
16 15
Web Server Software Architecture
General Web Client/Server Interactions HTTP HTTP Handler Handler 1: GET ~schmidt WWW HTTP/1.0 WWW Sock Sock HTTP Stream Stream CLIENT SERVER Handler 2: index.html HTTP Sock Acceptor PROTOCOL Stream HTML HANDLERS Event Sock PARSER Dispatcher Acceptor GUI DISPATCHER
REQUESTER
Event Dispatcher
GRAPHICS COMMUNICATION PROTOCOL
Encapsulates Web server concurrency and dis-
ADAPTER (E.G., HTTP) {
patching strategies
HTTP Handlers
OS KERNEL OS KERNEL
Parses HTTP headers and pro cesses requests
OS I/O SUBSYSTEM OS I/O SUBSYSTEM {
HTTP Acceptor NETWORK ADAPTERS NETWORK ADAPTERS
NETWORK
{ Accepts connections and creates HTTP Han-
dlers
17 18
Tactical Patterns
Design Patterns in the Web
Server Implementation
Proxy
{ \Provide a surrogate or placeholder for another
to control access to it" Thread object Thread-per Acceptor
Request Pool
Strategy
Active Connector
\De ne a family of algorithms, encapsulate each
Object { and make them interchangeable" Thread-per one,
Session Service Adapter
Half-Sync/ Configurator
\Convert the interface of a class into another
Half-Async {
client exp ects" Asynchronous interface Completion Reactor/ Double
Checked Singleton Token Proactor
Locking
\Ensure a class only has one instance and pro-
STRATEGIC PATTERNS {
vide a global p oint of access to it" TACTICAL PATTERNS
Abstract State State Strategy Adapter Singleton
Factory
{ \Allow an object to alter its b ehavior when its
internal state changes"
20 19
Concurrency Patterns
Event Handling Patterns
Reactor
{ \Decouples metho d execution from metho d in-
vo cation and simpli es synchronized access to
{ \Decouples synchronous event demultiplexing
shared resources by concurrent threads"
and event handler initiation dispatching from
service s p erformed in resp onse to events"
Half-Sync/Half-Async
Proactor
{ \Decouples synchronous I/O from asynchronous
I/O in a system to simplify concurrent pro-
{ \Decouples asynchronous event demultiplexing
gramming e ort without degrading execution
and event handler completion dispatching from
eciency"
service s p erformed in resp onse to events"
Double-Checked Lo cking Optimization Pat-
Asynchronous Completion Token
tern
{ \Eciently asso ciates state with the comple-
{ \Ensures atomic initialization of objects and
tion of asynchronous op erations"
eliminates unnecessary lo cking overhead on each
access"
21 22
Concurrency Architecture
Service Initialization Patterns
Patterns
Connector
Thread-p er-Request
{ \Decouples active connection establishment from
the service p erformed once the connection is
{ \Allows each client request to run concurrently
established"
in a separate thread"
Acceptor
{ \Decouples passive connection establishment
{ \Allows up to N requests to execute concur-
from the service p erformed once the connec-
rently within a pool of threads "
tion is established"
Thread-p er-Connection
Service Con gurator
{ \Allows each client connection to run concur-
{ \Decouples the b ehavior of network services
rently"
from p oint in time at which services are con-
gured into an application"
Suited for HTTP 1.1, but not HTTP 1.0
23 24
Alternative Web Server
Selecting the Server's
Concurrency Patterns
Concurrency Architecture
Problem
The following example illustrates the de-
sign patterns and framework comp onents
{ A very strategic design decision for high-p erformance
Web servers is selecting an ecient concur-
in an OO implementation of a concurrent
rency architecture
Web Server
The following are the key concurrency pat-
Forces
tern alternatives:
{ No single concurrency architecture is optimal
1. Reactive
{ Key factors include OS/hardware platform and
2. Thread-p er-request
workload
3. Thread-p er-connection
Solution
4. Synchronous Thread Pool
{ Understand key alternative concurrency pat-
terns
5. Asynchronous Thread Pool
26 25
Thread-p er-Request Web Server
Reactive Web Server
2: HANDLE INPUT 2: HANDLE INPUT HTTP 3: CREATE HANDLER HTTP 3: CREATE HANDLER Handler 4: ACCEPT CONNECTION Handler 4: ACCEPT CONNECTION 5: SPAWN THREAD 5: ACTIVATE HANDLER HTTP Handler HTTP HTTP HTTP Acceptor Acceptor Handler HTTP Reactor Handler Reactor
6: PROCESS HTTP REQUEST 6: PROCESS HTTP REQUEST
SERVER SERVER 1: CONNECT 1: CONNECT CLIENT CLIENT CLIENT CLIENT
CLIENT CLIENT
27 28
Handle-based Synchronous
Thread-p er-Connection Web
Thread Pool Web Server Server
3: SPAWN THREAD 2: CREATE, ACCEPT, Event PER CONNECTION AND ACTIVATE HTTP Dispatcher HTTP_HANDLER Handler HTTP HTTP HTTP Handler 2: ACCEPT CONNECTION Handler Handler HTTP Acceptor HTTP 3: MORPH INTO HANDLER Handler Reactor HTTP HTTP HTTP Acceptor Handler Acceptor
4: PROCESS HTTP REQUEST 1: HTTP REQUEST SERVER 4: PROCESS HTTP REQUEST CLIENT 1: HTTP REQUEST SERVER CLIENT CLIENT CLIENT CLIENT
CLIENT
29 30
Asynchronous Thread Pool Web
Queue-based Synchronous
Server
Thread Pool Web Server
2: HANDLE INPUT Message 3: ENQUEUE REQUEST I/O 1: INITIATE ASYNC ACCEPT Queue Completion 2: RUN EVENT LOOP Active 4: ACCEPT COMPLETES HTTP HTTP Port Object 5: QUEUE COMPLETION Handler HTTP Handler 4: DEQUEUE & Handler HTTP 6: DEQUEUE COMPLETION PROCESS Async Async Active Handler & PROCESS REQUEST HTTP Accept Object HTTP REQUEST Async Read Acceptor Handler Write Async Active Accept Active HTTP Object HTTP Object Reactor Handler Handler Proactor
5: PROCESS HTTP REQUEST 7: PROCESS HTTP REQUEST 1: HTTP REQUEST 3: HTTP REQUEST SERVER SERVER CLIENT CLIENT CLIENT CLIENT CLIENT
CLIENT
31 32
The ADAPTIVE Communication
Architecture of Our WWW Server
Environment ACE
SELF-CONTAINED MIDDLEWARE DISTRIBUTED JAWS ADAPTIVE APPLICATIONS WWW SERVER SERVICE WEB SERVER THE ACE ORB COMPONENTS TOKEN GATEWAY (TAO) SERVER SERVER svc_run svc_run svc_run svc_run LOGGING NAME TIME SERVER SERVER SERVER
SERVICE CORBA FRAMEWORKS ACCEPTOR CONNECTOR HANDLER HANDLER
ADAPTIVE SERVICE EXECUTIVE (ASX) PROCESS/ : Msg C++ THREAD LOG SERVICE SHARED : HTTP : Options MANAGERS MSG CONFIG- MALLOC WRAPPERS REACTOR/ Queue URATOR Processor PROACTOR SYNCH SPIPE SOCK_SAP/ FIFO MEM SYSV s WRAPPERS SAP TLI_SAP SAP MAP WRAPPERS
OS ADAPTATION LAYER C PROCESSES/ STREAM SOCKETS/ NAMED SELECT/ DYNAMIC MEMORY SYSTEM : HTTP APIS THREADS PIPES TLI PIPES IO COMP LINKING MAPPING V IPC : HTTP : HTTP Handler Handler PROCESS/THREAD COMMUNICATION VIRTUAL MEMORY Handler SUBSYSTEM SUBSYSTEM SUBSYSTEM GENERAL POSIX AND WIN32 SERVICES : HTTP Acceptor
: Reactor
A set of C++ wrapp ers and frameworks
based on common communication soft-
ware design patterns
33 34
Demultiplexing and Dispatching
Events
The Reactor Pattern
Problem
Intent
{ Web servers must pro cess several di erent typ es
of events simultaneously
{ \Decouples synchronous event demultiplexing
and event handler initiation dispatching from
service s p erformed in resp onse to events"
Forces
{ Multi-threading is not always available
This pattern resolves the following forces
for synchronous event-driven software:
{ Multi-threading is not always ecient
{ How to demultiplex multiple typ es of events
from multiple sources of events synchronously
{ Tightly coupling general event pro cessing with
and eciently within a single thread of control
server-sp eci c logic is in exible
{ How to extend application b ehavior without re-
quiring changes to the event dispatching frame-
Solution
work
{ Use the Reactor pattern to decouple generic
event pro cessing from server-sp eci c pro cess-
ing
36 35
Collab oration in the Reactor
Structure of the Reactor Pattern
Pattern
APPLICATION- APPLICATION- select (handles); INDEPENDENT DEPENDENT foreach h in handles { HTTP callback : if (h is output handler) Handler table[h]->handle_output () ; main Concrete Initiation if (h is input handler) program Event_Handler Dispatcher table[h]->handle_input (); if (h is signal handler) Reactor() table[h]->handle_signal (); Event_Handler HTTP INITIALIZE } Acceptor timer_queue->expire_timers (); handle_input() register_handler(callback) handle_output() REGISTER HANDLER handle_signal() get_handle() EXTRACT HANDLE handle_timeout() MODE n get_handle() n handle_events() START EVENT LOOP A 1 INITIALIZATION 1 1 select() FOREACH EVENT DO Initiation 1 Timer_Queue handle_input() Dispatcher DATA ARRIVES 1 n schedule_timer(h) handle_events() Handles handle_output() cancel_timer(h) OK TO SEND register_handler(h) expire_timers(h) remove_handler(h) handle_signal() n SIGNAL ARRIVES 1 1
MODE handle_timeout() TIMER EXPIRES remove_handler(callback) REMOVE HANDLER EVENT HANDLING handle_close()
CLEANUP
Participants in the Reactor pattern
37 38
An Integrated Reactive/Active
A Single-threaded Reactive Web
Web Server Server
REGISTERED svc_run REGISTERED svc_run 2: accept() OBJECTS 4: getq(msg) OBJECTS 5: recv(request) 3: make_handler() 5:svc(msg) svc_run 6: process(request) HTTP HTTP HTTP HTTP HTTP Acceptor Handler Handler Handler Handler HTTP Handler LEVEL Message Event Event HTTP HTTP Event Event Queue Event Handler LEVEL Handler Processor
APPLICATION Handler Handler Handler Handler Event APPLICATION Handler Event 2: recv_request(msg) 4: handle_input() Handler3: putq(msg) 1: handle_input()
1: handle_input() Initiation
LEVEL Dispatcher Initiation
LEVEL Dispatcher FRAMEWORK Reactor FRAMEWORK Reactor OS EVENT DEMULTIPLEXING INTERFACE OS EVENT DEMULTIPLEXING INTERFACE LEVEL KERNEL
LEVEL
KERNEL
39 40
The HTTP Handler Public
Handler Protected The HTTP
Interface
Interface
The HTTP Handler is the Proxy for commu-
nicating with clients
The following metho ds are invoked by call-
{ Along with Reactor, this class implements the
backs from the Reactor
asynchronous task part of Half-Sync/Half-Async
protected:
// Reusable base class.
// Reactor notifies when client's timeout.
template
virtual int handle_timeout const Time_Value &,
class HTTP_Handler :
const void *
public Svc_Handler { NULL_SYNCH> { // Remove from the Reactor. public: Reactor::instance ->remove_handler // Entry point into HTTP_Handler, called by this, READ_MASK ; // HTTP_Acceptor. } virtual int open void * { // Register with Reactor to handle client input. // Reactor notifies when HTTP requests arrive. Reactor::instance ->register_handler this, READ_M virtual int handle_input HANDLE ; // Register timeout in case client doesn't // Receive/frame client HTTP requests e.g., GET . // send any HTTP requests. int recv_request Message_Block &* ; Reactor::instance ->schedule_timer }; this, 0, ACE_Time_Value HTTP_CLIENT_TIMEOUT ; } 42 41 Integrating Multi-threading The Active Object Pattern Problem Intent { Multi-threaded Web servers are needed since { \Decouples metho d execution from metho d in- Reactive Web servers are often inecient, non- vo cation and simpli es synchronized access to scalable, and non-robust shared resources by concurrent threads" Forces This pattern resolves the following forces for concurrent communication software: { Multi-threading can be very hard to program { How to allow blo cking read and write op era- { No single multi-threading mo del is always op- tions on one endp oint that do not detract from timal the quality of service of other endp oints { How to simplify concurrent access to shared state Solution { Use the Active Object pattern to allow multi- { How to simplify comp osition of indep endent ple concurrent server op erations using an OO services programming style 44 43 Structure of the Active Object Pattern Collab oration in the Active loop { Pattern Client m = actQueue.remove() Object Interface dispatch (m) } get_request() 1: get_request() : Client : Activation : Represent- client : Scheduler set_request() Interface Queue ation head_request() 3: dispatch() m1() Scheduler INVOKE Activation CREATE METHOD cons(m1') dispatch() OBJECT future() VISIBLE get_request'() Queue RETURN RESULT CONSTRUCTION HANDLE TO set_request'() insert() METHOD OBJECT CLIENTS head_request'() 111 1remove() / INSERT IN insert(m1') 1 PRIORITY QUEUE 2: insert(get_request') 1 DEQUEUE NEXT remove(m1') 1 n METHOD OBJECT EXECUTION INVISIBLE SCHEDULING TO Resource Method CLIENTS Representation Objects EXECUTE dispatch(m1') RETURN RESULT reply_to_future() COMPLETION The Scheduler determines the sequence that Method Objects are executed 45 46 Using the Active Object Pattern Pro cessor Class The HTTP in the Web Server Pro cesses HTTP requests using the \Thread Po ol" concurrency mo del REGISTERED svc_run svc_run 4: getq(msg) OBJECTS Implement the synchronous task portion of the 5:svc(msg) { pattern svc_run Half-Sync/Half-Async HTTP_Processor : public Task { HTTP class Handler HTTP Message HTTP public: Singleton access point. Queue // Handler HTTP Processor HTTP_Processor *instance void ; LEVEL Event Handler static Event APPLICATION Handler Pass a request to the thread pool. Handler Event 2: recv_request(msg) // int put Message_Block * ; Handler 3: putq(msg) virtual // Event loop for the pool thread int svc int { 1: handle_input() virtual *mb = 0; // Message buffer. Initiation Message_Block LEVEL Dispatcher // Wait for messages to arrive. FRAMEWORK ;; { Reactor for getq mb ; // Inherited from class Task; Identify and perform WWW Server OS EVENT DEMULTIPLEXING INTERFACE // // request processing here... } LEVEL protected: KERNEL HTTP_Processor void ; // Constructor. 48 47 Using the Singleton Pattern Subtle Concurrency Woes with the Singleton Pattern The HTTP Processor is implemented as a Singleton that is created \on demand" Problem HTTP_Processor * { The canonical Singleton implementation has HTTP_Processor::instance void subtle \bugs" in multi-threaded applications { // Beware of race conditions! if instance_ == 0 instance_ = new HTTP_Processor; Forces return instance_; { Too much lo cking makes Singleton to o slow::: } { Too little lo cking makes Singleton unsafe::: Constructor creates the thread pool Solution HTTP_Processor::HTTP_Processor void { { Use the Double-Checked Lo cking optimization // Inherited from class Task. pattern to minimize lo cking and ensure atomic activate THR_NEW_LWP, num_threads ; initialization } 49 50 Using the Double-Checked The Double-Checked Lo cking Lo cking Optimization Pattern for Optimization Pattern the Web Server Intent \Ensures atomic initialization of objects and { if (instance_ == NULL) { unnecessary lo cking overhead on each eliminates mutex_.acquire (); access" if (instance_ == NULL) instance_ = new HTTP_Processor; mutex_.release (); } This pattern resolves the following forces: return instance_; 1. Ensures atomic initialization or access to ob- regardless of thread scheduling order jects, HTTP Processor 2. Keeps lo cking overhead to a minimum static instance() e.g., only lo ck on creation { static instance_ Mutex Note, this pattern assumes atomic mem- ory access:: : 51 52 Integrating Reactive and Multi-threaded Layers Half-Sync/Half-Async Pattern Problem Intent { Justifying the hybrid design of our Web server { \Decouples synchronous I/O from asynchronous can be tricky I/O in a system to simplify programming e ort without degrading execution eciency" Forces This pattern resolves the following forces { Engineers are never satis ed with the status for concurrent communication systems: quo ;- { How to simplify programming for higher-level communication tasks { Substantial amount of time is sp ent re-discovering the intent of complex concurrent software de- These are p erformed synchronously sign { How to ensure ecient lower-level I/O com- munication tasks Solution These are p erformed asynchronously { Use the Half-Sync/Half-Async pattern to ex- plain and justify our Web server concurrency architecture 54 53 Structure of the Collab orations in the Half-Sync/Half-Async Pattern Half-Sync/Half-Async Pattern SYNC SYNC External Async Message Sync TASK Event Source Task Queue Task TASK 1 3 notification() EXTERNAL EVENT read(msg) RECV MSG SYNC ASYNC PHASE work() TASK LAYER TASK PROCESS MSG SYNCHRONOUS 2 enqueue(msg) ENQUEUE MSG 1, 4: read(data) read(msg) PHASE DEQUEUE MSG QUEUEING LAYER work() MESSAGE QUEUES EXECUTE TASK QUEUEING 3: enqueue(data) SYNC PHASE ASYNC TASK 2: interrupt This illustrates input pro cessing output EXTERNAL ro cessing is similar EVENT SOURCES p TASK LAYER ASYNCHRONOUS 56 55 Joining Async and Sync Tasks in Using the Half-Sync/Half-Async the Web Server Pattern in the Web Server svc_run svc_run The following metho ds form the b oundary svc_run between the Async and Sync layers LEVEL SYNC TASK 4: getq(msg) 5:svc(msg) int HTTP_Handler::handle_input void { *mb = 0; Message HTTP Message_Block Queue Processor // Try to receive and frame message. if recv_request mb == HTTP_REQUEST_COMPLETE { LEVEL Reactor::instance ->remove_handler QUEUEING HTTP HTTP READ_MASK ; HTTP this, Handler Handler ->cancel_timer this ; Handler Reactor::instance 2: recv_request(msg) // Insert message into the Queue. Event Event 3: putq(msg) ->put mb ; Event HTTP_Processor HandlerHandler Handler } } 1: handle_input() // Task entry point. Message_Block *msg { LEVEL Reactor HTTP_Processor::put // Insert the message on the Message_Queue ASYNC TASK // inherited from class Task . putq msg ; } 58 57 Optimizing Our Web Server for Asynchronous Op erating Systems The Proactor Pattern Problem Intent { Synchronous multi-threaded solutions are not always the most ecient { \Decouples asynchronous event demultiplexing and event handler completion dispatching from service s p erformed in resp onse to events" Forces { Purely asynchronous I/O is quite powerful on This pattern resolves the following forces some OS platforms for asynchronous event-driven software: e.g., Windows nt 4.x { How to demultiplex multiple typ es of events from multiple sources of events asynchronously and eciently within a minimal numb er of threads { Good designs should be adaptable to new con- texts { How to extend application b ehavior without re- quiring changes to the event dispatching frame- work Solution { Use the Proactor pattern to maximize p erfor- mance on Asynchronous OS platforms 60 59 Structure of the Proactor Pattern APPLICATION- APPLICATION- INDEPENDENT DEPENDENT ration in the Proactor HTTP Async Collab o overlapped_result = GetQueuedCompleteStatus(); Handler Write overlapped_result->complete() Pattern Event_Handler HTTP Acceptor Asynchronous handle_accept() Proactive Operation Asynchronous Completion Completion handle_read_file() Async Initiator Processor Operation Dispatcher Handler handle_write_file() Async handle_timeout() Op Accept register (operation, handler, dispatcher) get_handle() Asynchronous open() operation initiated n cancel() A execute 1 n Operation performed 1 asynchronously Completion 1 Timer_Queue dispatch Dispatcher 1 Operation completes 1 n schedule_timer(h) handle_events() Handles cancel_timer(h) register_handle() expire_timer(h) Completion Handler handle event notified 1 1 Participants in the Proactor pattern 61 62 Client Connects to a Proactive Client Sends Request to a Web Server Proactive Web Server 4: connect Web Server 1: accept 1: GET connections /etc/passwd Web Server Web Browser 7: create Acceptor Web HTTP 4: parse request HTTP Browser Handler 2: accept Handler 6: 6: write (File, Conn., accept (Acceptor, 8: read (connection, 8: write Handler, Dispatcher) complete Dispatcher) 3: read Handler, Dispatcher) 5: read (File) complete complete 7: write complete Completion Operating File Completion Operating Dispatcher System System Dispatcher System 3: handle 5: accept 2: read complete events complete 63 64 The Acceptor Pattern Structuring Service Initialization Intent Problem { \Decouples passive initialization of a service from the tasks p erformed once the service is { The communication proto col used between clients initialized" and the Web server is often orthogonal to the initialization proto col This pattern resolves the following forces for network servers using interfaces like Forces so ckets or TLI: { Low-level connection establishment APIs are 1. How to reuse passive connection establishment tedious, error-prone, and non-p ortable co de for each new service { Separating initialization from use can increase 2. How to make the connection establishment co de software reuse substantially portable across platforms that may contain so ck- ets but not TLI, or vice versa Solution 3. How to ensure that a passive-mo de descriptor is not accidentally used to read or write data { Use the Acceptor pattern to decouple passive service initialization from run-time proto col 4. How to enable exible p olicies for creation, connection establishment, and concurrency 66 65 Structure of the Acceptor Pattern Collab oration in the Acceptor Pattern HTTP Handler acc : peer_acceptor_ sh: : Initiation HTTP HTTP Server Acceptor : SOCK Svc_Handler Dispatcher Acceptor Handler Acceptor open() INITIALIZE PASSIVE open() peer_stream_ peer_acceptor_ ENDPOINT register_handler(acc) open() CREATE & handle_input() REGISTER HANDLER get_handle() ACTIVATE EXTRACT HANDLE PHASE handle_events() ENDPOINT START EVENT LOOP INITIALIZATION FOREACH EVENT DO select() Event NOTIFIES handle_input() CONNECTION EVENT sh = make_svc_handler() Dispatcher CREATE, ACCEPT, accept_svc_handler (sh) AND ACTIVATE OBJECT activate_svc_handler (sh) register_handler(sh) PHASE REGISTER HANDLER SERVICE FOR CLIENT I/O get_handle() INITIALIZATION EXTRACT HANDLE handle_input() DATA EVENT PROCESS MSG svc() handle_close() CLIENT SHUTDOWN PHASE Acceptor is a factory that creates, con- SERVICE SERVER SHUTDOWN handle_close() PROCESSING nects, and activates a Svc Handler 67 68 The Acceptor Class Using the Acceptor Pattern in the The Acceptor class implements the Accep- Web Server tor pattern Reusable Factor HTTP // HTTP Handler Acceptor : Acceptor Handler Handler class public Service_Object // Subclass of Event_Handler. Svc Svc Svc { Handler Acceptor Handler Handler public: // Notified by Reactor when clients connect. virtual int handle_input void 1: handle_input() { The strategy for initializing a SVC_HANDLER. 2: sh = make_svc_handler() ACTIVE HTTP // *sh = new SVC_HANDLER; 3: accept_svc_handler(sh) CONNECTIONS Handler SVC_HANDLER sh->peer ; 4: activate_svc_handler(sh) peer_acceptor_.accept ; Svc sh->open PASSIVE LISTENER Handler } ... Reactor // protected: // IPC connection factory. SOCK_Acceptor peer_acceptor_; } 70 69 Putting the Pieces Together at Run-time The HTTP Acceptor Class Interface Problem { Prematurely committing ourselves to a partic- The HTTP Acceptor class accepts connec- ular Web server con guration is in exible and inecient tions and initializes HTTP Handlers class HTTP_Acceptor : public Acceptor Forces // Inherits handle_input strategy from Acceptor. { { Certain server con guration decisions can't be public: made eciently until run-time // Hook called automatically when HTTP_Acceptor // is dynamically linked. { Forcing users to pay for comp onents they don't virtual int init int argc, char *argv[] ; use is undesirable // Hook called automatically when HTTP_Acceptor is // dynamically unlinked. virtual int fini void ; Solution // ... } { Use the Service Con gurator pattern to assem- ble the desired Web server comp onents dynam- ically 71 72 Structure of the Service The Service Con gurator Pattern Con gurator Pattern Intent Concrete \Decouples the b ehavior of communication ser- { Service Object vices from the p oint in time at which these LAYER services are con gured into an application or APPLICATION system" Service Service Object Config This pattern resolves the following forces r highly exible communication software: fo suspend() 1 resume() 1 init() How to defer the selection of a particular typ e, { A fini() 1 or a particular implementation, of a service un- LAYER info() n very late in the design cycle til Service 1 CONFIGURATION Repository i.e., at installation-time or run-time { How to build complete applications by comp os- multiple indep endently develop ed services ing Event Handler 1 Reactor LAYER n How to optimize, recon gure, and control the { REACTIVE b ehavior of the service at run-time 73 74 Using the Service Con gurator Pattern in the Web Server Collab oration in the Service r Pattern Con gurato Reactive SERVICE WWW Server CONFIGURATOR TP RUNTIME svc : : Service : Service WWW Server Service main() : Reactor Service_Object Config Repository Object SHARED Service_Config() Service Service CONFIGURE OBJECTS FOREACH SVC ENTRY DO process_directives() Repository Object DYNAMICALLY LINK link_service() SERVICE init(argc, argv) TPR INITIALIZE SERVICE WWW Server register_handler(svc) REGISTER SERVICE Service MODE get_handle() Reactor EXTRACT HANDLE Config Service insert() STORE IN REPOSITORY CONFIGURATION Object run_event_loop() START EVENT LOOP handle_events() FOREACH EVENT DO handle_input() INCOMING EVENT SHUTDOWN EVENT handle_close() Existing Web server is based on Half-Sync/Half- MODE CLOSE SERVICE remove_handler(svc) unlink_service() pattern UNLINK SERVICE Async fini() remove() EVENT HANDLING Other versions could be single-threaded, could use other concurrency strategies, and other proto cols 76 75 Con guring the Web Server with The HTTP Acceptor Class the Service Con gurator Implementation // Initialize service when dynamically linked. The concurrent Web Server is con gured int HTTP_Acceptor::init int argc, char *argv[] and initialized via a con guration script { Options::instance ->parse_args argc, argv ; cat ./svc.conf dynamic TP_WWW_Server Service_Object * // Set the endpoint into listener mode. www_server.dll:make_TP_WWW_Server Acceptor::open local_addr ; "-p $PORT -t $THREADS" // Initialize the communication endpoint. Reactor::instance ->register_handler this, ACCEPT_MASK } Factory function that dynamically allo cates // Terminate service when dynamically unlinked. a Half-Sync/Half-Async Thread Pool Web int HTTP_Acceptor::fini void Server { // Unblock threads in the pool so they will extern "C" Service_Object *make_TP_WWW_Server void ; // shutdown correctly. HTTP_Processor::instance ->close ; Service_Object *make_TP_WWW_Server void { // Wait for all threads to exit. return new HTTP_Acceptor; Thread_Manager::instance ->wait ; // ACE dynamically unlinks and deallocates this object. } } 78 77 The OO Architecture of the JAWS Framework Main Program for Web Server Reactor/ProactorState Singleton Strategy I/O Strategy Cached Virtual Dynamically con gure and execute the Web Framework Filesystem Server { Note that this is totally generic! Asynchronous Completion Token Tilde ~ Expander main int argc, char *argv[] int /home/... Protocol Acceptor { Handler Event Dispatcher Service_Config daemon; Protocol Filter Strategy // Initialize the daemon and dynamically // configure the service. argc, argv ; daemon.open Adapter // Loop forever, running services and handling Concurrency reconfigurations. // Protocol Pipeline Strategy Framework Framework Active Object ; daemon.run_event_loop Streams Service Configurator /* NOTREACHED */ } www.cs.wustl.edu/~jxh/research/ 79 80 Applying Patterns to CORBA Web Server Optimization ORBs Techniques CLIENT SERVICE SERVANT Use lightweight concurrency CONFIGURATOR ABSTRACT Minimize lo cking FACTORY ACTIVE STRATEGY Apply le caching and memory mapping OBJECT THREAD-SPECIFIC Use \gather-write" mechanisms STORAGE CONNECTOR ACCEPTOR REACTOR Minimize logging WRAPPER FACADES Pre-compute HTTP resp onses OS KERNEL OS KERNEL Avoid excessive time calls Optimize the transp ort interface www.cs.wustl.edu/~schmidt/ORB-patterns.ps.gz 81 82 Drawbacks to Design Patterns Bene ts of Design Patterns Patterns do not lead to direct co de reuse Design patterns enable large-scale reuse of software architectures Patterns are deceptively simple Patterns explicitly capture exp ert knowl- Teams may su er from pattern overload edge and design tradeo s Patterns are validated by exp erience and Patterns help improve develop er commu- discussion rather than by automated test- nication ing Patterns help ease the transition to object- Integrating patterns into a software devel- oriented technology opment pro cess is a human-intensive ac- tivity 84 83 Suggestions for Using Patterns Lessons Learned using OO E ectively Frameworks Do not recast everything as a pattern Bene ts of frameworks { Instead, develop strategic domain patterns and reuse existing tactical patterns { Enable direct reuse of co de cf patterns { Facilitate larger amounts of reuse than stand- Institutionalize rewards for developing pat- alone functions or individual classes terns Drawbacks of frameworks Directly involve pattern authors with ap- { High initial learning curve plication develop ers and domain exp erts Many classes, many levels of abstraction { The ow of control for reactive dispatching is Clearly do cument when patterns apply and non-intuitive do not apply { Veri cation and validation of generic comp o- nents is hard Manage exp ectations carefully 85 86 Conferences and Workshops on Patterns and Framework Patterns Literature Bo oks Pattern Language of Programs Confer- ences { Gamma et al., \Design Patterns: Elements of Reusable OO Software" AW, 1994 { Septemb er, 1998, Monticello, Illinois, USA { Pattern Languages of Program Design series by AW, 1995 1997 { st-www.cs.uiuc.edu/users/patterns/patterns.html { Siemens, Pattern-Oriented Software Architec- ture, Wiley, 1996 The Europ ean Pattern Languages of Pro- gramming conference Sp ecial Issues in Journals { July, 1998, Kloster Irsee, Germany { Octob er '96 \Communications of the ACM" eds: Douglas C. Schmidt, Ralph Johnson, and Mohamed Fayad { www.cs.wustl.edu/~schmidt/patterns.html { Octob er '97 \Communications of the ACM" eds: Douglas C. Schmidt and Mohamed Fayad USENIX COOTS Magazines { April 27 30, 1998, Santa Fe, New Mexico { C++ Rep ort and JOOP, columns by Coplien, { www.usenix.org/events/co ots98/ Vlissides, Vinoski, Schmidt, and Martin 88 87 Obtaining ACE and JAWS The ADAPTIVE Communication Environ- ment ACE is an OO to olkit designed ac- cording to key network programming pat- terns { JAWS is b oth a Web server framework and a high-p erformance Web server All source co de for ACE and JAWS is freely available { www.cs.wustl.edu/~schmidt/ACE.html Mailing lists Newsgroup { comp.soft-sys.ace 89