Event-Driven Server Throughput
Total Page:16
File Type:pdf, Size:1020Kb
CSCI-1680 Network Programming II Rodrigo Fonseca Today • Network programming – Programming Paradigms – Programming libraries • Final project Low-level Sockets • Address Family AF_PACKET – Socket type: SOCK_RAW • See link-layer (Ethernet) headers. Can send broadcast on a LAN. Can get/create non-IP packets – Socket type: SOCK_DGRAM • See IP headers. Can get protocols other than TCP/UDP: ICMP, SCTP, DCCP, your own… • Can cook your own IP packets – Must have root privileges to play with these Building High Performance Servers e need for concurrency • How to improve throughput? – Decrease latency (throughput α 1/latency) – Hard to do! • Optimize code (this you should try!) • Faster processor (no luck here, recently) • Speed of light isn’t changing anytime soon… • Disks have to deal with things like inertia! – Do multiple things at once • Concurrency – Allows overlapping of computation and I/O – Allows use of multiple cores, machines Process 1 3.3 Single-process event-driven AcceptGet Read Find SendGet Read File The single-process event-driven (SPED) architecture ConnConn Request File HeaderConn Send Data uses a single event-driven server process to perform concurrent processing of multiple HTTP requests. The High-performanceProcess N Servers server uses non-blocking systems calls to perform asyn- AcceptGet Read Find SendGet Read File chronous I/O operations. An operation like the BSD ConnConn Request File HeaderConn Send Data UNIX select or the System V poll is used to check Common Patterns for I/O operations that have completed. Figure 4 depicts the SPED architecture. Figure 2: Multi-Process - In the MP model, each server ASPEDservercanbethoughtofasastatemachine process handles one request at a time. Processes execute that performs one basic step associated with the serving MulBple processes the processing stages sequentially. Process 1 3.3 Single-processMulBple Threads event-driven of an HTTP request at a time, thus interleaving the pro- cessing steps associated with many HTTP requests. In AcceptGet Read Find SendGet Read File The single-process event-driven (SPED) architecture ConnConn Request File HeaderConn Send Data each iteration, the server performs a select to check uses a singleAcceptGet event-drivenRead serverFind processSendGet Read to performFile for completed I/O events (new connection arrivals, com- concurrentConnConn processingRequest of multipleFile HTTPHeaderConn requests.Send Data The Process N pleted file operations, client sockets that have received server uses non-blocking systems calls to perform asyn- data or have space in their send buffers.) When an I/O AcceptGet Read Find SendGet Read File chronous I/O operations. An operation like the BSD event is ready, it completes the corresponding basic step ConnConn Request File HeaderConn Send Data UNIX select or the System V poll is used to check Figure 3: Multi-Threaded - The MT model uses a single and initiates the next step associated with the HTTP re- for I/O operations that have completed. Figure 4 depicts Single Process Event Driven with Helpers address space with multiple concurrent threads of execu- quest, if appropriate. the SPED architecture. Figure 2: Multi-Process - In the MP model, each server tion. Each thread handles a request. In principle, a SPED server is able to overlap the Send Header Send Header AcceptGet Single Process Event Driven Read Find ASPEDservercanbethoughtofasastatemachineAcceptGet Read Find CPU, disk and network operations associated with the processConnConn handlesRequest one request atFile a time. ProcessesRead File execute ConnConn Request File Read File Send Data that performs one basic step associatedSend with Data the serving serving of many HTTP requests, in the context of a sin- the processing stages sequentially. Send Header Send Header of an HTTPAcceptGet requestRead at a time,Find thus interleaving the pro- AcceptGet Read Find Since each processEvent has Dispatcher its own private address space, gle process and a single thread of control. As a result, ConnConn RequestEvent DispatcherFile Read File ConnConn Request File Read File Send Data cessingno synchronization steps associated is necessary with many to handle HTTPSend therequests.Data processing In the overheads of context switching and thread synchro- each iteration, the server performs2 a select to check Event Dispatcher of different HTTP requestsEvent Dispatcher.However,itmaybemore nization in the MP and MT architectures are avoided. AcceptGet Read Find SendGet Read File for completed I/O events (new connection arrivals, com- Figure 4:ConnConn SingleRequest Process EventFile DrivenHeaderConn -Send The Data SPED difficultHelper to 1 performHelper optimizations 2 in this Helper architecture k that However, a problem associated with SPED servers is that model uses a single process to perform all client process- pletedrely on file global operations, information, client sockets such as that a shared have received cache of many current operating systems do not provide suitable ingFigure and disk 4: activitySingle Process in an event-driven Event Driven manner. - The SPED datavalid orHelper URLs. have 1 space Figure in 2Helper their illustrates 2 send buffers.) the MP Helper architecture. When k an I/O support for asynchronous disk operations. Figure 5: Asymmetric Multi-Process Event Driven - The model uses a single process to perform all client process- event is ready, it completes the corresponding basic step In these operating systems, non-blocking read and AMPED3.2 modelMulti-threaded uses a single process for event-driven re- ingFigure and 3: disk Multi-Threaded activityFigures from in an event-driven - ThePai MT, et al., 1999 “Flash: An efficient and portable Web server” model manner. uses a single and initiates the next step associated with the HTTP re- write operations work as expected on network sock- questFigure processing,Multi-threaded 5: Asymmetric but has (MT) Multi-Process other servers, helper depicted Event processes Driven in to Figure han-- The 3, theaddress event-driven space with approach multiple of concurrent the SPED threads architecture of execu- quest, if appropriate. ets and pipes, but may actually block when used on disk dleAMPEDemploy some disk model multiple operations. uses independent a single process threads for event-drivenof control operat- re- withtion. multiple Each threadhelper handlesprocesses a request. (or threads) that handle In principle, a SPED server is able to overlap the files. As a result, supposedly non-blocking read opera- the event-driven approach of the SPED architecture questing processing, within a single but has shared other address helper space. processes Each to han- thread blocking disk I/O operations. By default, the main CPU, disk and network operations associated with the tions on files may still block the caller while disk I/O is with multiple helper processes (or threads) that handle dleperforms some disk all operations. the steps associated with one HTTP re- event-driven process handles all processing steps asso- serving of many HTTP requests, in the context of a sin- in progress. Both operating systems used in our experi- blockingSince diskeach processI/O operations. has its own By private default, address the space, main 4.1quest Performance before accepting characteristics a new request, similar to the MP ciated with HTTP requests. When a disk operation is gle process and a single thread of control. As a result, ments exhibit this behavior (FreeBSD 2.2.6 and Solaris event-drivenno synchronization process is handles necessary all to processing handle the steps processing asso- themodel’s overheads use of of a context process. switching and thread synchro- necessary (e.g., because a file is requested that is not Disk4.1 operations Performance-Thecostofhandlingdiskactivity characteristics 2.6). To the best of our knowledge, the same is true for ciatedof different with HTTP HTTP requests.requests2.However,itmaybemore When a disk operation is nizationThe in primary the MP difference and MT between architectures the MP are and avoided. the MT likely to be in the main memory file cache), the main varies between the architectures based on what, if any, most versions of UNIX. necessarydifficult to (e.g., perform because optimizations a file is in requested this architecture that is thatnot However,architecture, a problem however, associated is that all with threads SPED can servers share is global that server process instructs a helper via an inter-process circumstancesDisk operations cause-Thecostofhandlingdiskactivity all request processing to stop while Many UNIX systems provide alternate APIs that im- likelyrely on to global be in the information, main memory such file as a cache), shared the cache main of manyvariables. current The operating use of a systems single shared do not address provide space suitable lends communication (IPC) channel (e.g., a pipe) to perform adiskoperationisinprogress.IntheMPandMTmod-varies between the architectures based on what, if any, plement true asynchronous disk I/O, but these APIs are servervalid URLs. process Figure instructs 2 illustrates a helper thevia MP an architecture. inter-process supportitself easily for asynchronous to optimizations disk operations. that rely on shared state. the potentially blocking operation. Once the operation els,circumstances only the process cause allor threadrequest that processing causes tothe stop disk while ac- generally not integrated with the select operation. communication (IPC) channel (e.g., a pipe) to perform However,In these theoperating