Comet and WebSocket Web Applications How to Scale Server-Side Event-Driven Scenarios
Simone Bordet [email protected]
Roma, March 5th, 2011 1 Agenda
What are Comet web applications ?
Impacts of Comet web applications
WebSocket web applications
The CometD project
Questions & Answers
Roma, March 5th, 2011 2 What are Comet & WebSocket Web Applications ?
Roma, March 5th, 2011 3 Web 1.0 Applications
Web Classic Interaction
Request Pattern Bursts of requests for HTML, images, CSS, JS
Navigation Mode Full page based
Interaction with Server Reactive, changes happen on user click Resources download
Roma, March 5th, 2011 4 Ajax Web Applications Web Classic + XMLHttpRequest (XHR) Request Pattern: changed Bursts (for classic) + Concurrent (for XHR) Requires synchronization in server code Navigation Mode: radically changed Sometimes full page, most often single page with partial changes Interaction with Server: changed Reactive as before Data download
Roma, March 5th, 2011 5 Comet Web Applications
Web Classic + Ajax + WebSocket
Request Pattern, Navigation Mode, Interaction with Server Same as with XHR More Efficient Now messages can be server-initiated Messages do not require a response Uses JavaScript Same as with XHR
Roma, March 5th, 2011 6 Rich User Interfaces
Traditional Web sacrificed Rich UI for: Ubiquitous access Zero Install
Heavy usage of client-side JavaScript and XHR changed the way we create and develop webapps They become rich, and raise expectations
Server side events are needed To meet user raised expectations To implement new types of web applications
Roma, March 5th, 2011 7 Comet Examples
Roma, March 5th, 2011 8 Polling for Server Side Events
Polling Strategy for Server-side Events Notification Simple to implement (“I can do that !”) Sensible latency
Can look like a denial of service (DoS) attack to the poor server When poll interval is short When the number of clients is large
Roma, March 5th, 2011 9 Long Polling for Server-side events
Comet Strategy More difficult to implement correctly Minimal latency
The “long poll” request is held by the server until: A server-side event arrives A timeout expires The client disconnects
Roma, March 5th, 2011 10 Websocket for Server-side events
WebSocket strategy More efficient than long poll True bidirectional web communication
WebSocket Messages No more Request & Response Not widely deployed yet Messages can be initiated by the server
Roma, March 5th, 2011 11 WebSocket
From HTML 5 interface WebSocket API & Protocol { Two way communication attribute Function onopen; attribute Function onmessage; Developed by WHATWG attribute Function onerror; Browser vendors attribute Function onclose; Little server side input boolean send(in String data); In IETF WG void close(); Developing standard }; Slow progress
Experimentally Deployed Chrome, Opera, Firefox
Roma, March 5th, 2011 12 Impacts on the Server
Roma, March 5th, 2011 13 Scaling Polling
Scaling Polling 1000 clients, each polling every 5 seconds Request Rate: 200 requests/s Latency: 2.5 seconds in average, too high
How can polling achieve, say, 250 ms latency ? Reduce the polling interval to 0.5 seconds Request Rate: 2000 requests/s, pretty high
Limits Most likely, the server is CPU bound, then connection bound
Roma, March 5th, 2011 14 Scaling Comet (Blocking)
Scaling Comet (long polling) 1000 clients, long poll request held for 20 seconds Request Rate: 50 requests/s, good But yields 1000 concurrent requests ! Running out of threads in the pool On 64-bit JVM the thread stack size is 1 MB
Latency Average latency approaches network latency Maximal latency is at least 3x network latency Limits Most likely, the server is memory bound Note that threads stack memory is outside Java heap
Roma, March 5th, 2011 15 Scaling Comet
Comet has huge impacts on server-side
You cannot just deploy your Comet web application in a standard configuration
You cannot deploy your Comet application behind Apache Httpd & other intermediaries Do not scale for the same reasons
You need a new generation of servers
Roma, March 5th, 2011 16 Asynchronous Servers
Jetty Server explored these problems in the Servlet(TM) space for JMS messaging.
The Jetty Continuations allow the Jetty Server to threadlessly suspend requests
The continuation concepts have been incorporated in the new Servlet(TM) 3.0 specification
Jetty 6 and Jetty 7 successfully deployed Comet applications worldwide Jetty 8 implements the Servlet(TM) 3.0 specification
Roma, March 5th, 2011 17 Blocking Comet
Server-side Event-driven Comet Web Application
Roma, March 5th, 2011 18 Asynchronous Comet
Server-side Event-driven Comet Web Application
Roma, March 5th, 2011 19 Scaling Comet (Asynchronous)
Comet (Asynchronous) 1000 clients, long poll request held for 20 seconds Wait for server-side events is threadless Request is re-run on event arrival or on long poll timeout Request Rate: 100 requests/s, good Only few concurrent active (threaded) requests Good latency, good request rate, no thread pool problems
Limits Most likely, the server is connection bound, then CPU bound Scales a lot better than normal polling and blocking Comet
Roma, March 5th, 2011 20 Scaling Comet (WebSocket)
WebSocket 1000 clients, keepalive message every 20 seconds No need to reply, decreased bandwidth Wait for server-side events is threadless Request Rate: 50 requests/s, good Only few concurrent active (threaded) requests Optimal latency, good request rate, no thread pool problems Limits Most likely, the server is connection bound, then CPU bound Scales a slightly better than Comet with continuations due to better data density and reduced request rate
Roma, March 5th, 2011 21 The Need of a Framework
With Comet, we have an asynchronous bidirectional web Looks like messaging to me
Writing server-side code based on continuations (or, in the future, with Servlet 3.0) is difficult It really is, you don't want to do it How are failures handled? Timeouts, dropouts, retries, backoffs, cross domain, etc.
Web applications should be easy to write Can we abstract the gory details into a library ?
Roma, March 5th, 2011 22 The CometD Project
Roma, March 5th, 2011 23 The CometD Project
We have now a scalable bidirectional web What do we need to write applications ?
We don't want to care about: Which transport: long-polling, websocket limits: same-origin policy, two connection limits
We want: A clean way to publish data to the server A clean way to receive data from the server A clean way to “emit” server-side events to clients
Roma, March 5th, 2011 24 CometD JavaScript API var cometd = $.cometd; // jQuery style cometd.init('http://myserver/cometd'); cometd.subscribe('/my/channel', function(message) { var data = message.data; // Do something with the data }); cometd.publish('/my/channel', { chatText: 'hello!' }); cometd.disconnect();
Roma, March 5th, 2011 25 The CometD Project
There are a lot of other details to take care of Transport Negotiation WebSocket, Long Polling over CORS, JSONP, etc. Authentication Network Failures With possible automatic retry Message Batching Message Acknowledgment Message serialization Etc.
From these and other requirements and input, the CometD project was born
Roma, March 5th, 2011 26 The CometD Project The CometD project delivers libraries that use the Comet technique (long poll) or WebSocket to transport Bayeux messages
The Bayeux protocol specifies the format of the Bayeux messages It is based on JSON
Libraries are available in JavaScript (client) Java (client and server) Perl & Python (less active)
Roma, March 5th, 2011 27 Bayeux
Bayeux is at its core a publish/subscribe messaging system Very similar to JMS publish/subscribe
A Bayeux channel is similar to a JMS Topic You may subscribe to it You will be delivered messages published onto
You can publish messages to a channel The server automatically delivers the messages to all channel subscribers
Roma, March 5th, 2011 28 CometD JavaScript
The JavaScript library features Common code with bindings for Dojo and jQuery Highly configurable Message batching Automatic reconnection Pluggable transports WebSocket, Long Polling and Callback Polling available Supports Cross-Origin servers Extensible Many extensions already available
Roma, March 5th, 2011 29 CometD Java
The Java library features Highly scalable (the client is based on Jetty asynchronous HTTP Client) Message batching Lazy messages A variety of listeners to be notified of relevant events in client and server Data filters to automatically convert data Extensions SecurityPolicy
Roma, March 5th, 2011 30 Performance
Roma, March 5th, 2011 31 DEMO
Roma, March 5th, 2011 32 WebSocket Redux
Slightly better maximal latency But don't run your pacemaker on it
Slightly better data density If only that was a limitation!
No help with hard problems Connection management Handling Failures
You still need a framework like CometD
Roma, March 5th, 2011 33 References
WebSocket http://www.w3.org/TR/websockets http://www.ietf.org/id/draft-ietf-hybi-thewebsocketprotocol-01.txt
CometD http://cometd.org
Jetty http://eclipse.org/jetty
Roma, March 5th, 2011 34 Questions & Answers
Roma, March 5th, 2011 35