The Functional Web Warp: A Haskell Web Server Michael Snoyman • Suite Solutions oughly two years ago, I began work on about this runtime is its lightweight threads. As the Yesod Web framework. I originally a result of this feature, Warp simply spawns a R intended FastCGI for my deployment strat- new thread for each incoming connection, bliss- egy, but I also created a simple HTTP server for fully unaware of the gymnastics the runtime is local testing, creatively named SimpleServer. performing under the surface. Because Yesod targets the Web Application Part of this abstraction involves converting Interface (WAI), a standard interface between synchronous Haskell I/O calls into asynchro- Haskell Web servers and Web applications, it nous system calls. Once again, Warp reaps the was easy to switch back ends for testing and benefits by calling simple functions like recv production. and send, while GHC does the hard work. It didn’t take long before I started getting Up through GHC 6.12, this runtime sys- feature requests for SimpleServer: slowly but tem was based on the select system call. This surely, features such as chunked transfer encod- worked well enough for many tasks but didn’t ing and sendfile-based file serving made their scale for Web servers. One big feature of the way in. The tipping point was when Matt Brown GHC 7 release was a new event manager, written of Soft Mechanics made some minor tweaks by Google’s Johan Tibell and Serpentine’s Bryan to SimpleServer and found that it was already O’Sullivan. This new runtime uses different sys- the fastest Web server in Haskell (see Figure 1). tem calls (epoll, kqueue, and so on) depending After that, he and I made some modest improve- on what’s available on the target operating sys- ments and released the code as Warp. tem. Additionally, Tibell and O’Sullivan made Very little code in Warp itself is geared extensive enhancements to the data structures toward speed. For the most part, it simply the manager uses: it now uses a radix trie for builds on the shoulders of giants — by relying storing callbacks and a priority search queue for on underlying libraries that perform extremely timeouts. well, Warp can achieve a lot in fewer than 500 The end result: Haskell programs can easily lines of code. Let’s explore how Warp uses each scale to thousands of simultaneous connections. of these libraries, what makes them so powerful, Programmers can write their code against a very and how they fit together. simple API, spawning new lightweight threads using forkIO and calling blocking functions Glasgow Haskell Compiler inside them. The first library isn’t really a library at all: the Glasgow Haskell Compiler (GHC) is the standard Enumerator Haskell compiler. It has all the optimizations A recent move in the Haskell community you’d expect of an industrial-strength com- has been adopting the enumerator pattern. piler, such as loop unrolling, extensive inlin- This pattern allows for processing streams of ing, unboxing, and fusion. It even lets users data in a deterministic manner. This is espe- specify their own optimizations via rewrite cially important for Web servers, which must rules. In addition, it provides a very sophisti- quickly release scarce resources such as file cated multi threaded runtime. One great thing descriptors. MAY/JUNE 2011 1089-7801/11/$26.00 © 2011 IEEE Published by the IEEE Computer Society 81 IC-15-03-funweb.indd 81 4/6/11 3:52 PM The Functional Web 81,701 body and sent to the application, or will be part of the next request. Enumeratees also play an impor- 64,028 tant role in Warp. They ensure that the application consumes the entire request body before continuing with the next request, and that the application doesn’t consume more 35,272 35,811 bytes than it should for the request body. They also convert the response body from a stream of Builders (dis- 18,654 cussed next) to a stream of bytes with chunked transfer encoding applied. 3,237 3,416 3,417 4,660 Blaze-Builder The simplest way to represent a string in Haskell is as a list of Uni- PHP Snap Node Yesod Warp Goliath Tornado code characters. This has two major Winstone Happstack performance issues: it’s expensive to append data to a list, and the rep- Figure 1. Pong benchmark. Requests/second (higher is better). resentation of the list has a lot of overhead. Historically, two different John Millikin (unaffiliated) wrote and Iteratee: it receives a stream solutions have existed, one solving the enumerator package that WAI of data from an Enumerator and each issue: and Warp use. In this package, the sends a new stream of data to an central datatype is Iteratee. An Iteratee. • Use difference lists instead of Iteratee is a data consumer, receiv- Warp’s entire I/O system is built actual lists during data con- ing chunks of data and performing on top of the Enumerator datatype. struction, and produce only the some action with them. Iteratee is Once Warp establishes a connection final output list at the end. This an instance of Monad, making it easy and starts a new handler thread, it exploits the fact that append- to compose two Iteratees together produces an Enumerator from the ing to a difference list is an O(1) to build up more complicated actions. client socket and pipes that data into operation. (For those not familiar, a Monad an Iteratee. This Iteratee is where • Represent our data using a packed is a container that encapsulates a all request parsing occurs. format such as ByteString or the computation’s side effects. Haskell Enumerator’s built-in chunking newer Text datatype. programmers can easily combine behavior also works perfectly for different monadic values to build up Warp as well. The Enumerator opti- The blaze-html package, by more powerful computations.) mizes the size of its requested buf- Jasper Van der Jeugt of Ghent The flip side ofIteratee is fers, currently set at 4,096 bytes. University and Simon Meier of ETH Enumerator, a data producer. An The consuming Iteratee, on the Zurich, sought to solve both issues Enumerator will feed data into an other hand, has no concept of these during HTML content construction. Iteratee until either the Enumerator chunks’ size. Instead, it simply con- The idea is to work around the has run out of data or the Iteratee sumes as many bytes as it wants. central concept of a Builder, a no longer accepts more. A simple If there isn’t enough buffered con- value that knows how it should fill example of the interaction between tent to complete an operation (for up a memory buffer. Internally, a these two is file input and output: example, the chunk terminated in Builder is a difference list of these enumFile is an Enumerator that reads the middle of an HTTP header), then buffer-filling actions. Combining data from a file and streams it into an control automatically returns to the these two points, we end up with a Iteratee, whereas iterHandle is an Enumerator to provide more output. packed representation of data with Iteratee that consumes a stream of If too much data was provided, the efficient append operations. And, bytes and sends them to a handle. remainder is left in the Enumerator just as important, we’re guaranteed A third datatype, an Enumeratee, to be consumed by the next action. that the bytes will be copied precisely is a combination of Enumerator It will either be part of the request once into our final buffer. 82 www.computer.org/internet/ IEEE INTERNET COMPUTING IC-15-03-funweb.indd 82 4/6/11 3:52 PM Warp: A Haskell Web Server It quickly became apparent that Instead, I ended up writing a request and returns a response. The the Builder abstraction would be an Enumeratee that would take a Request datatype contains informa- useful outside the context of HTML stream of Builders and use them to tion such as the requested path, query generation. The Yesod Web frame- fill up buffers. When a buffer filled, strings, request headers, and remote work immediately used it for gen- the Enumeratee would wrap it in a host/port. One thing noticeably lack- erating CSS, JavaScript, and JSON. Byte String and send it down the ing from this list is the request body. Meier split off the Builder datatype pipeline to the Iteratee. This meant To understand why, consider the fol- and its associated functions into a that the code produced optimally lowing type signature: separate blaze-builder package. sized ByteStrings, with minimal WAI and Warp rely heavily on buffer copying, and used constant type Application = Request → blaze-builder for constructing memory. Meier has since taken the Iteratee ByteString IO Response responses. Applications always send code, improved it, and released it as their response bodies to the server blaze-builder-enumerator. The Application returns its in the form of Builders. This lets It turns out that the exact same Response inside an Iteratee, so Warp efficiently append the body to requirements exist when writing a it consumes the request body from the response headers, meaning that, Web server. The application can give there. As mentioned previously, for many common responses, Warp the server chunks of data of any size, Warp performs all its operations uses only a single memory buf- and the server wants to concatenate inside the Iteratee monad; this fer and makes a single system call. these into optimally sized buffers means that calling the Application As a nice finishing touch, blaze- to minimize system-call overhead, is simply another step in that pro- builder provides a helper function without using large amounts of cess.