Comet and Websocket Web Applications How to Scale Server-Side Event-Driven Scenarios
Total Page:16
File Type:pdf, Size:1020Kb
Comet and WebSocket Web Applications How to Scale Server-Side Event-Driven Scenarios Simone Bordet [email protected] Roma, March 5th, 2011 1 Agenda What are Comet web applications ? Impacts of Comet web applications WebSocket web applications The CometD project Questions & Answers Roma, March 5th, 2011 2 What are Comet & WebSocket Web Applications ? Roma, March 5th, 2011 3 Web 1.0 Applications Web Classic Interaction Request Pattern Bursts of requests for HTML, images, CSS, JS Navigation Mode Full page based Interaction with Server Reactive, changes happen on user click Resources download Roma, March 5th, 2011 4 Ajax Web Applications Web Classic + XMLHttpRequest (XHR) Request Pattern: changed Bursts (for classic) + Concurrent (for XHR) Requires synchronization in server code Navigation Mode: radically changed Sometimes full page, most often single page with partial changes Interaction with Server: changed Reactive as before Data download Roma, March 5th, 2011 5 Comet Web Applications Web Classic + Ajax + WebSocket Request Pattern, Navigation Mode, Interaction with Server Same as with XHR More Efficient Now messages can be server-initiated Messages do not require a response Uses JavaScript Same as with XHR Roma, March 5th, 2011 6 Rich User Interfaces Traditional Web sacrificed Rich UI for: Ubiquitous access Zero Install Heavy usage of client-side JavaScript and XHR changed the way we create and develop webapps They become rich, and raise expectations Server side events are needed To meet user raised expectations To implement new types of web applications Roma, March 5th, 2011 7 Comet Examples Roma, March 5th, 2011 8 Polling for Server Side Events Polling Strategy for Server-side Events Notification Simple to implement (“I can do that !”) Sensible latency Can look like a denial of service (DoS) attack to the poor server When poll interval is short When the number of clients is large Roma, March 5th, 2011 9 Long Polling for Server-side events Comet Strategy More difficult to implement correctly Minimal latency The “long poll” request is held by the server until: A server-side event arrives A timeout expires The client disconnects Roma, March 5th, 2011 10 Websocket for Server-side events WebSocket strategy More efficient than long poll True bidirectional web communication WebSocket Messages No more Request & Response Not widely deployed yet Messages can be initiated by the server Roma, March 5th, 2011 11 WebSocket From HTML 5 interface WebSocket API & Protocol { Two way communication attribute Function onopen; attribute Function onmessage; Developed by WHATWG attribute Function onerror; Browser vendors attribute Function onclose; Little server side input boolean send(in String data); In IETF WG void close(); Developing standard }; Slow progress Experimentally Deployed Chrome, Opera, Firefox Roma, March 5th, 2011 12 Impacts on the Server Roma, March 5th, 2011 13 Scaling Polling Scaling Polling 1000 clients, each polling every 5 seconds Request Rate: 200 requests/s Latency: 2.5 seconds in average, too high How can polling achieve, say, 250 ms latency ? Reduce the polling interval to 0.5 seconds Request Rate: 2000 requests/s, pretty high Limits Most likely, the server is CPU bound, then connection bound Roma, March 5th, 2011 14 Scaling Comet (Blocking) Scaling Comet (long polling) 1000 clients, long poll request held for 20 seconds Request Rate: 50 requests/s, good But yields 1000 concurrent requests ! Running out of threads in the pool On 64-bit JVM the thread stack size is 1 MB Latency Average latency approaches network latency Maximal latency is at least 3x network latency Limits Most likely, the server is memory bound Note that threads stack memory is outside Java heap Roma, March 5th, 2011 15 Scaling Comet Comet has huge impacts on server-side You cannot just deploy your Comet web application in a standard configuration You cannot deploy your Comet application behind Apache Httpd & other intermediaries Do not scale for the same reasons You need a new generation of servers Roma, March 5th, 2011 16 Asynchronous Servers Jetty Server explored these problems in the Servlet(TM) space for JMS messaging. The Jetty Continuations allow the Jetty Server to threadlessly suspend requests The continuation concepts have been incorporated in the new Servlet(TM) 3.0 specification Jetty 6 and Jetty 7 successfully deployed Comet applications worldwide Jetty 8 implements the Servlet(TM) 3.0 specification Roma, March 5th, 2011 17 Blocking Comet Server-side Event-driven Comet Web Application Roma, March 5th, 2011 18 Asynchronous Comet Server-side Event-driven Comet Web Application Roma, March 5th, 2011 19 Scaling Comet (Asynchronous) Comet (Asynchronous) 1000 clients, long poll request held for 20 seconds Wait for server-side events is threadless Request is re-run on event arrival or on long poll timeout Request Rate: 100 requests/s, good Only few concurrent active (threaded) requests Good latency, good request rate, no thread pool problems Limits Most likely, the server is connection bound, then CPU bound Scales a lot better than normal polling and blocking Comet Roma, March 5th, 2011 20 Scaling Comet (WebSocket) WebSocket 1000 clients, keepalive message every 20 seconds No need to reply, decreased bandwidth Wait for server-side events is threadless Request Rate: 50 requests/s, good Only few concurrent active (threaded) requests Optimal latency, good request rate, no thread pool problems Limits Most likely, the server is connection bound, then CPU bound Scales a slightly better than Comet with continuations due to better data density and reduced request rate Roma, March 5th, 2011 21 The Need of a Framework With Comet, we have an asynchronous bidirectional web Looks like messaging to me Writing server-side code based on continuations (or, in the future, with Servlet 3.0) is difficult It really is, you don't want to do it How are failures handled? Timeouts, dropouts, retries, backoffs, cross domain, etc. Web applications should be easy to write Can we abstract the gory details into a library ? Roma, March 5th, 2011 22 The CometD Project Roma, March 5th, 2011 23 The CometD Project We have now a scalable bidirectional web What do we need to write applications ? We don't want to care about: Which transport: long-polling, websocket limits: same-origin policy, two connection limits We want: A clean way to publish data to the server A clean way to receive data from the server A clean way to “emit” server-side events to clients Roma, March 5th, 2011 24 CometD JavaScript API var cometd = $.cometd; // jQuery style cometd.init('http://myserver/cometd'); cometd.subscribe('/my/channel', function(message) { var data = message.data; // Do something with the data }); cometd.publish('/my/channel', { chatText: 'hello!' }); cometd.disconnect(); Roma, March 5th, 2011 25 The CometD Project There are a lot of other details to take care of Transport Negotiation WebSocket, Long Polling over CORS, JSONP, etc. Authentication Network Failures With possible automatic retry Message Batching Message Acknowledgment Message serialization Etc. From these and other requirements and input, the CometD project was born Roma, March 5th, 2011 26 The CometD Project The CometD project delivers libraries that use the Comet technique (long poll) or WebSocket to transport Bayeux messages The Bayeux protocol specifies the format of the Bayeux messages It is based on JSON Libraries are available in JavaScript (client) Java (client and server) Perl & Python (less active) Roma, March 5th, 2011 27 Bayeux Bayeux is at its core a publish/subscribe messaging system Very similar to JMS publish/subscribe A Bayeux channel is similar to a JMS Topic You may subscribe to it You will be delivered messages published onto You can publish messages to a channel The server automatically delivers the messages to all channel subscribers Roma, March 5th, 2011 28 CometD JavaScript The JavaScript library features Common code with bindings for Dojo and jQuery Highly configurable Message batching Automatic reconnection Pluggable transports WebSocket, Long Polling and Callback Polling available Supports Cross-Origin servers Extensible Many extensions already available Roma, March 5th, 2011 29 CometD Java The Java library features Highly scalable (the client is based on Jetty asynchronous HTTP Client) Message batching Lazy messages A variety of listeners to be notified of relevant events in client and server Data filters to automatically convert data Extensions SecurityPolicy Roma, March 5th, 2011 30 Performance Roma, March 5th, 2011 31 DEMO Roma, March 5th, 2011 32 WebSocket Redux Slightly better maximal latency But don't run your pacemaker on it Slightly better data density If only that was a limitation! No help with hard problems Connection management Handling Failures You still need a framework like CometD Roma, March 5th, 2011 33 References WebSocket http://www.w3.org/TR/websockets http://www.ietf.org/id/draft-ietf-hybi-thewebsocketprotocol-01.txt CometD http://cometd.org Jetty http://eclipse.org/jetty Roma, March 5th, 2011 34 Questions & Answers Roma, March 5th, 2011 35.