The Functional Web Yaws: Yet Another Web Server
Steve Vinoski • Verivue
he modestly named Yaws — “Yet Another bytecode files and other parts of the Yaws sys- Web Server” — is an open source Erlang Web tem installed elsewhere under /usr/local. T server known for its reliability, stability, Command-line options for the generated config- and scalability. It started roughly a decade ago as ure script let you choose alternative installation the brainchild of legendary Erlang programmer locations. This approach is suitable for running Claes “Klacke” Wikström, who also invented sev- Yaws as a stand-alone Web server. Alternatively, eral important Erlang features, including Erlang you can build Yaws using rebar: term storage (ets), Distributed Erlang, the Mnesia database, and the Erlang bit syntax. Yaws is a rebar compile general-purpose HTTP 1.1 Web server consisting of a relatively simple core surrounded by imple- The result of this approach, which builds mentations of various features, such as response Yaws and installs it within its own build area, streaming, AJAX support, websockets support, can also run stand-alone but is best-suited for Common Gateway Interface (CGI) and Fast CGI applications that embed Yaws as an internal (FCGI) support, and application embedding. Web server. While it’s perfectly capable of performing regu- lar file serving, Yaws is most useful for applica- Runtime Configuration tions that generate and serve dynamic content. Yaws configuration controls details such as where it looks for files being served, what ports Configure, Make, Install it listens on, and what applications it loads to To get started with Yaws, you can either obtain handle requests. Two configuration portions an official release from its primary website, exist. Server configurations apply to individual http://yaws.hyber.org, or you can retrieve the virtual servers. A single Yaws instance can latest sources from the Yaws github.com source host multiple virtual servers on the same or repository: different IP addresses. Server configuration controls features such as on which port and IP git clone git://github.com/klacke/yaws.git address the server listens for requests, the root path in the filesystem from which Yaws serves You can build Yaws in two different ways: files, the types of scripts the server can execute, either using GNU autotools and make, or via the and the application modules, or appmods, that rebar Erlang build tool (see https://github.com/ are assigned to various URI paths. Appmods are basho/rebar). To use the autotools approach: very useful for dynamic Yaws Web applications, so I’ll present more details about them later. autoconf Global configuration applies to the running ./configure Yaws instance as a whole. It specifies directo- make && sudo make install ries where logs are deposited, from which Erlang bytecode is loaded, and from which include files This will build Yaws and by default install are retrieved for use in dynamically compiled it into /usr/local/bin/yaws, with Erlang beam application code. It also sets size and timeout
90 Published by the IEEE Computer Society 1089-7801/11/$26.00 © 2011 IEEE IEEE INTERNET COMPUTING
IC-15-04-funw.indd 90 6/7/11 3:08 PM Yaws: Yet Another Web Server
limits for the Yaws internal file
Hello, World!
feature is the ability to embed Erlang tion tells Yaws where to find the doc- code within HTML. When a request ument root for that server, and Yaws arrives for a file with a “.yaws” suf- then uses request URI paths to find And here is the same content repre- fix, Yaws processes any Erlang code requested files relative to that root. sented in ehtml: it finds within the file betweenJULY/AUGUST 2011 91
IC-15-04-funw.indd 91 6/7/11 3:08 PM The Functional Web
out(Arg) -> Uri = yaws_api:request_url(Arg), means that it will receive all requests NewId = create_new_id(), sent to that server. Segs = [Uri#url.path, NewId], The result of the out/1 function NewPath = string:join(Segs, "/"), typically supplies the status, head- NewUri = Uri#url{path = NewPath}, ers, and body for the client response. NewLoc = yaws_api:format_url(NewUri), For example, the appmod in Figure 2 Json = json2:encode( returns a response suitable for a POST {struct, [{"location", NewLoc}]}), request creating a new resource. [{status, 201}, This out/1 function creates a new {header, {"Location", NewLoc}}, resource and a URI to identify it. It {content, "application/json", Json}]. first extracts the target URI from the request Arg, which returns a Yaws Figure 2. Application module. This appmod out/1 function returns a response url record instance. It then calls the suitable for a POST request creating a new resource. It first obtains the target create_new_id/0 function to gener- URI and augments a copy of it with an identifier for the new resource. It then ate a new identifier to use in the new creates a JSON object with a “location” key to store the new resource URI, URI (the details of create_new_id/0 and also returns the new URI as the value of the response Location header. are unimportant here). It then copies The result specifies HTTP status and theLocation header, and includes the the target URI record but sets its path JSON object string as the response body. component to that of the new URI path string. To create the response values of the query parameters “title” • various sections of the path por- body, it uses the Yaws json2 module and “name” appearing within the URI tion of the request URI, to create a JSON object whose “loca- used to access the resource. Within • the request body (such as for tion” key contains the URI string for the code, the two calls to the yaws_ HTTP POST, PUT, and PATCH), if the new resource. Finally, it returns api:queryvar/2 function return the any, and a list of tuples specifying the HTTP values of the two query parameters, • query parameters, if any. response: a 201 status code indicat- which are then used to complete the ing the creation of a new resource, ehtml returned from the out/1 func- The arg instance is key to pro- a Location HTTP header specifying tion. Accessing the resource via the gramming Yaws applications. Once the URI for the new resource, and the URI http://example.org/example. Yaws passes it to your code, the code content body as a JSON string speci- yaws?title=Example&name=Joe can then pass it back to various Yaws fied with the “application/json” media substitutes the text “Example” for functions to extract information from type. Before writing the response the title and the name “Joe” for the it to help fulfill the request. to the client connection socket, Yaws variable part of the paragraph text. augments this with other typical This example, though contrived (and Appmods headers, such as Date, Server, and potentially unsafe, given that the Not all Web resources can be repre- Content-Length. query parameter values are used sented only in HTML, of course, so Appmods are infinitely flexible. To unescaped), shows how an applica- not all applications can use ehtml handle requests and form responses, tion can call any Erlang functions or “.yaws” pages. Yaws applications they can call into databases, invoke to dynamically create any content it wanting to deal with common for- backend services, implement caches, wants to return to its clients. mats such as XML and JSON can do or communicate with replicas on so via appmods. An appmod is con- other Erlang nodes. You can imple- Arg Data ceptually quite simple: it’s an Erlang ment them as regular functions or You probably noticed the parameter module that exports an out/1 func- as calls into instances of standard called Arg passed to the out/1 func- tion. You configure Yaws to associate Erlang behaviors such as gen_server tion. This is an instance of the Yaws the module with a portion of a URI and gen_fsm. arg record, a structure that encapsu- path. When the configured server lates everything Yaws knows about receives a request containing that Yapps the current request, including path, Yaws invokes the appmod’s A single drawback to appmods, out/1 function, passing it an arg though, is that to be deployed, they • the client connection socket, instance representing the request. require configuration changes to • the client IP address and port, For example, registering an appmod associate the appmod with a URI • request headers, in a given server under the path “/” path. If an application is developed
92 www.computer.org/internet/ IEEE INTERNET COMPUTING
IC-15-04-funw.indd 92 6/7/11 3:08 PM Yaws: Yet Another Web Server
by a single developer or small team, and Perl, that support these eliminating many of the decou- this isn’t a big problem. However, protocols. pling, evolvability, and scalabil- consider the fact that Erlang/OTP is • Websockets. Yaws clients can ity benefits it provides. Despite equipped with a modular application send an HTTP upgrade request to these flaws, some Yaws users still packaging and deployment system ask Yaws to switch the protocol see these protocols as useful, so that allows applications from inde- over to websockets, which allows Yaws supports them. pendent developers to be deployed HTTP-independent bidirectional • Reverse proxy. This experimental together within a single Erlang communication between client code allows Yaws to be configured node. Under this system, it would and server. At the time of this to proxy requests from clients be nice if Web applications could be writing, the websockets specifi- and send them to entirely sepa- stitched together from independently cation was still in draft form and rate Web origin servers that Yaws developed component applications and not finalized, so Yaws support is effectively hides from clients. could be deployed into already-running currently considered experimen- • Embedding. As mentioned earlier, Yaws instances without requiring tal and subject to change. Once applications can embed Yaws as configuration file changes. the specification is completed, an internal Web server rather A yapp, short for “Yaws applica- we’ll update Yaws to conform than just being appmods or other tion,” is similar to an appmod but to it. components that a stand-alone encompasses a whole Erlang applica- • Response streaming. This allows Yaws controls. tion rather than just a single module. Yaws applications to essentially This means the way the application take over the client connection A number of these features were starts, its supervision tree, and the from Yaws to stream responses donated by users who required the way it’s upgraded at runtime are to the client, which is useful, for functionality for their production all independent of Yaws. To enable example, for long-polling appli- systems, and they’ve been present in yapp registration, Yaws provides a cations (Comet applications) or for the Yaws source code repository for yapp module that allows registra- serving video streams. The appli- several years. tion under URI paths via a webpage cation out/1 function returns an without the need for configuration indication to Yaws that it wants Performance file changes. But consistent with to stream a response to the cli- At this point, most articles like this appmods, each yapp simply provides ent from a given Erlang process one would provide graphs and charts one or more out/1 functions that (which is like a very lightweight showing the results of executing Yaws calls when it decides to dis- thread), after which Yaws sends a benchmarks against whatever Web patch a request to the yapp. message to that process to tell it server the article describes. Despite that it now owns the client socket. their popularity, such benchmarks But Wait! There’s More! Once the application’s process is are never as generally useful as they Features like file serving, ehtml, finished with the socket, it sends might seem. They’re typically con- “.yaws” pages, appmods, and yapps a message back to Yaws to return trived, rarely represent real-world aren’t all that Yaws provides. It also socket ownership. loads, are often nearly impossible offers various other features that • JSON-RPC and SOAP. These two to reproduce, depend highly on the this column space doesn’t allow me protocols are tunneled through tuning of the underlying platform, to describe in detail, including but HTTP, letting clients interact with and are sometimes even rigged to not limited to remote resources in a fashion dif- make the subject Web server appear ferent than what HTTP allows. superior. While there’s no doubt • Arg rewriting. Applications can JSON-RPC and SOAP both allow that benchmarks are a very useful register modules that can exam- remote procedure calls (RPCs) to tool for developers of servers and ine and modify the request arg remote resources, and SOAP also frameworks to measure, assess, and record instances before Yaws dis- allows interaction with remote improve the throughput and latency patches requests for them, which resources via non-RPC XML mes- of their own code, too many users allows applications to affect how sage exchange. Both approaches misinterpret such benchmarks by that dispatching occurs. are fundamentally flawed in mistakenly equating the fastest with • CGI and FCGI support. This lets that they treat HTTP as a trans- being the best. Yaws fulfill client requests via port protocol rather than the Writing a simple Web server in calls to external programs and application protocol that it actu- Erlang isn’t difficult because the scripting languages, such as PHP ally is, thereby bypassing and support for HTTP header parsing is
JULY/AUGUST 2011 93
IC-15-04-funw.indd 93 6/7/11 3:08 PM The Functional Web
built into the Erlang runtime and For such applications, the server- But don’t take my word for it; if you’re is activated by setting a flag on a side application itself, and not interested in using Yaws and want to socket representing a client connec- the underlying Web server, is be sure it performs well, I advise you tion. The Erlang runtime also pro- often the bottleneck in terms of to benchmark it yourself, as only you vides socket event handling. While throughput and latency because know best what your application will this is quite convenient for applica- of calls it makes into databases look like and what sorts of systems tions, it also means that well-written or backend services to obtain it will need to deploy on, integrate Erlang Web servers don’t greatly dif- response data. with, and depend on for data. I also fer from each other as far as perfor- • Let’s face it — while everyone likes advise you to follow the sage advice mance goes because they’re all using to think their applications require of Mark Nottingham of Yahoo as far these same facilities. At first glance, the utmost in performance and as HTTP load testing goes; on his blog this might seem problematic, but it scalability, few actually do. Most he provides an excellent set of rules really isn’t, for at least the following dynamic applications will never for how to best obtain useful and reasons: even approach the performance meaningful results from such testing limits of Yaws or any other Erlang (see www.mnot.net/blog/2011/05/18/ • Developers tend to choose Erlang Web server. http_benchmark_rules). for its reliability, stability, con- currency support, and ease of That being said, my observations Yaws Going Forward development. Naturally, they of production systems show Yaws Klacke and I actively maintain want reasonable performance, performance and scalability to be and enhance Yaws, occasionally but they’re not necessarily seek- quite good, especially considering the accepting patches for fixes and new ing to give up these other impor- number of features it provides. It holds features from users. We watch devel- tant characteristics to maximize up well against other Erlang Web opments in areas such as websockets performance. frameworks that advertise themselves and work to keep Yaws up-to-date • Developers normally don’t choose as being lightweight, including some and also keep an eye out for problem an Erlang Web server for static file that lack robust support for HTTP 1.1 areas in the code or documentation serving, but rather, as mentioned or fail to provide important opera- that need attention. Klacke typically earlier, for dynamic applications. tional features such as access logging. creates two or three official releases per year, and users are always wel- AdvertiSer informAtion • july/AuguSt 2011 come to obtain the Yaws source code from github.com and build it them- selves. Users communicate with Advertising Personnel Klacke, other users, and me using Marian Anderson: Sr. Advertising Coordinator Email: [email protected] the Yaws mailing list (see https:// Phone: +1 714 821 8380 | Fax: +1 714 821 4010 lists.sourceforge.net/lists/listinfo/ erlyaws-list). Sandy Brown: Sr. Business Development Mgr. Email: [email protected] Phone: +1 714 821 8380 | Fax: +1 714 821 4010 ur ultimate goal is to keep Yaws Advertising Sales Representatives (Display) O as reliable, stable, well-performing, Western US/Pacific/Far East: Eric Kincaid and feature-rich as its long-time Email: [email protected] users have come to expect. Phone: +1 214 673 3742; Fax: +1 888 886 8599 Steve Vinoski is a member of the technical Eastern US/Europe/Middle East: Ann & David Schissler staff at Verivue in Westford, Mass. He’s a Email: [email protected], d[email protected] Phone: +1 508 394 4026; Fax: +1 508 394 4926 senior member of IEEE and a member of the ACM. You can read Vinoski’s blog at
Advertising Sales Representatives (Classified Line/Jobs Board) http://steve.vinoski.net/blog/ and contact Greg Barbash him at [email protected]. Email: [email protected] Phone: +1 914 944 0940 Selected CS articles and columns are also available for free at http:// ComputingNow.computer.org.
94 www.computer.org/internet/ IEEE INTERNET COMPUTING
IC-15-04-funw.indd 94 6/7/11 3:08 PM