Comet and WebSocket Web Applications How to Scale Server-Side Event-Driven Scenarios

Simone Bordet [email protected]

Roma, March 5th, 2011 1 Agenda

What are Comet web applications ?

Impacts of Comet web applications

WebSocket web applications

The CometD project

Questions & Answers

Roma, March 5th, 2011 2 What are Comet & WebSocket Web Applications ?

Roma, March 5th, 2011 3 Web 1.0 Applications

Web Classic Interaction

 Request Pattern  Bursts of requests for HTML, images, CSS, JS

 Navigation Mode  Full page based

 Interaction with Server  Reactive, changes happen on user click  Resources download

Roma, March 5th, 2011 4 Web Applications Web Classic + XMLHttpRequest (XHR)  Request Pattern: changed  Bursts (for classic) + Concurrent (for XHR)  Requires synchronization in server code  Navigation Mode: radically changed  Sometimes full page, most often single page with partial changes  Interaction with Server: changed  Reactive as before  Data download

Roma, March 5th, 2011 5 Comet Web Applications

Web Classic + Ajax + WebSocket

 Request Pattern, Navigation Mode, Interaction with Server  Same as with XHR  More Efficient  Now messages can be server-initiated  Messages do not require a response  Uses JavaScript  Same as with XHR

Roma, March 5th, 2011 6 Rich User Interfaces

Traditional Web sacrificed Rich UI for:  Ubiquitous access  Zero Install

Heavy usage of client-side JavaScript and XHR changed the way we create and develop webapps  They become rich, and raise expectations

Server side events are needed  To meet user raised expectations  To implement new types of web applications

Roma, March 5th, 2011 7 Comet Examples

Roma, March 5th, 2011 8 Polling for Server Side Events

Polling Strategy for Server-side Events Notification  Simple to implement (“I can do that !”)  Sensible latency

 Can look like a denial of service (DoS) attack to the poor server  When poll interval is short  When the number of clients is large

Roma, March 5th, 2011 9 Long Polling for Server-side events

Comet Strategy  More difficult to implement correctly  Minimal latency

 The “long poll” request is held by the server until:  A server-side event arrives  A timeout expires  The client disconnects

Roma, March 5th, 2011 10 Websocket for Server-side events

WebSocket strategy  More efficient than long poll  True bidirectional web communication

 WebSocket Messages  No more Request & Response  Not widely deployed yet  Messages can be initiated by the server

Roma, March 5th, 2011 11 WebSocket

From HTML 5 interface WebSocket  API & Protocol {  Two way communication attribute Function onopen; attribute Function onmessage; Developed by WHATWG attribute Function onerror;  Browser vendors attribute Function onclose;  Little server side input boolean send(in String data); In IETF WG void close();  Developing standard };  Slow progress

Experimentally Deployed  Chrome, Opera, Firefox

Roma, March 5th, 2011 12 Impacts on the Server

Roma, March 5th, 2011 13 Scaling Polling

Scaling Polling  1000 clients, each polling every 5 seconds  Request Rate: 200 requests/s  Latency: 2.5 seconds in average, too high

How can polling achieve, say, 250 ms latency ?  Reduce the polling interval to 0.5 seconds  Request Rate: 2000 requests/s, pretty high

Limits  Most likely, the server is CPU bound, then connection bound

Roma, March 5th, 2011 14 Scaling Comet (Blocking)

Scaling Comet (long polling)  1000 clients, long poll request held for 20 seconds  Request Rate: 50 requests/s, good  But yields 1000 concurrent requests ! Running out of threads in the pool On 64-bit JVM the thread stack size is 1 MB

Latency  Average latency approaches network latency  Maximal latency is at least 3x network latency Limits  Most likely, the server is memory bound  Note that threads stack memory is outside Java heap

Roma, March 5th, 2011 15 Scaling Comet

Comet has huge impacts on server-side

You cannot just deploy your Comet in a standard configuration

You cannot deploy your Comet application behind Apache Httpd & other intermediaries  Do not scale for the same reasons

You need a new generation of servers

Roma, March 5th, 2011 16 Asynchronous Servers

Jetty Server explored these problems in the Servlet(TM) space for JMS messaging.

The Jetty Continuations allow the Jetty Server to threadlessly suspend requests

The continuation concepts have been incorporated in the new Servlet(TM) 3.0 specification

Jetty 6 and Jetty 7 successfully deployed Comet applications worldwide Jetty 8 implements the Servlet(TM) 3.0 specification

Roma, March 5th, 2011 17 Blocking Comet

Server-side Event-driven Comet Web Application

Roma, March 5th, 2011 18 Asynchronous Comet

Server-side Event-driven Comet Web Application

Roma, March 5th, 2011 19 Scaling Comet (Asynchronous)

Comet (Asynchronous)  1000 clients, long poll request held for 20 seconds  Wait for server-side events is threadless  Request is re-run on event arrival or on long poll timeout  Request Rate: 100 requests/s, good  Only few concurrent active (threaded) requests  Good latency, good request rate, no thread pool problems

Limits  Most likely, the server is connection bound, then CPU bound  Scales a lot better than normal polling and blocking Comet

Roma, March 5th, 2011 20 Scaling Comet (WebSocket)

WebSocket  1000 clients, keepalive message every 20 seconds No need to reply, decreased bandwidth  Wait for server-side events is threadless  Request Rate: 50 requests/s, good  Only few concurrent active (threaded) requests  Optimal latency, good request rate, no thread pool problems Limits  Most likely, the server is connection bound, then CPU bound  Scales a slightly better than Comet with continuations due to better data density and reduced request rate

Roma, March 5th, 2011 21 The Need of a Framework

With Comet, we have an asynchronous bidirectional web  Looks like messaging to me

Writing server-side code based on continuations (or, in the future, with Servlet 3.0) is difficult  It really is, you don't want to do it How are failures handled?  Timeouts, dropouts, retries, backoffs, cross domain, etc.

Web applications should be easy to write  Can we abstract the gory details into a library ?

Roma, March 5th, 2011 22 The CometD Project

Roma, March 5th, 2011 23 The CometD Project

We have now a scalable bidirectional web What do we need to write applications ?

We don't want to care about:  Which transport: long-polling,  limits: same-origin policy, two connection limits

We want:  A clean way to publish data to the server  A clean way to receive data from the server  A clean way to “emit” server-side events to clients

Roma, March 5th, 2011 24 CometD JavaScript API var cometd = $.cometd; // jQuery style cometd.init('http://myserver/cometd'); cometd.subscribe('/my/channel', function(message) { var data = message.data; // Do something with the data }); cometd.publish('/my/channel', { chatText: 'hello!' }); cometd.disconnect();

Roma, March 5th, 2011 25 The CometD Project

There are a lot of other details to take care of  Transport Negotiation WebSocket, Long Polling over CORS, JSONP, etc.  Authentication  Network Failures With possible automatic retry  Message Batching  Message Acknowledgment  Message serialization  Etc.

From these and other requirements and input, the CometD project was born

Roma, March 5th, 2011 26 The CometD Project The CometD project delivers libraries that use the Comet technique (long poll) or WebSocket to transport Bayeux messages

The Bayeux protocol specifies the format of the Bayeux messages  It is based on JSON

Libraries are available in  JavaScript (client)  Java (client and server)  Perl & Python (less active)

Roma, March 5th, 2011 27 Bayeux

Bayeux is at its core a publish/subscribe messaging system  Very similar to JMS publish/subscribe

A Bayeux channel is similar to a JMS Topic  You may subscribe to it  You will be delivered messages published onto

You can publish messages to a channel  The server automatically delivers the messages to all channel subscribers

Roma, March 5th, 2011 28 CometD JavaScript

The JavaScript library features  Common code with bindings for Dojo and jQuery  Highly configurable  Message batching  Automatic reconnection  Pluggable transports WebSocket, Long Polling and Callback Polling available  Supports Cross-Origin servers  Extensible Many extensions already available

Roma, March 5th, 2011 29 CometD Java

The Java library features  Highly scalable (the client is based on Jetty asynchronous HTTP Client)  Message batching  Lazy messages  A variety of listeners to be notified of relevant events in client and server  Data filters to automatically convert data  Extensions  SecurityPolicy

Roma, March 5th, 2011 30 Performance

Roma, March 5th, 2011 31 DEMO

Roma, March 5th, 2011 32 WebSocket Redux

Slightly better maximal latency  But don't run your pacemaker on it

Slightly better data density  If only that was a limitation!

No help with hard problems  Connection management  Handling Failures

You still need a framework like CometD

Roma, March 5th, 2011 33 References

WebSocket  http://www.w3.org/TR/websockets  http://www.ietf.org/id/draft-ietf-hybi-thewebsocketprotocol-01.txt

CometD  http://cometd.org

Jetty  http://eclipse.org/jetty

Roma, March 5th, 2011 34 Questions & Answers

Roma, March 5th, 2011 35