A Sense of Time for Node.js: Timeouts as a Cure for Event Handler Poisoning Anonymous Abstract—The software development community has begun to new Denial of Service attack that can be used against EDA- adopt the Event-Driven Architecture (EDA) to provide scalable based services. Our Event Handler Poisoning attack exploits web services. Though the Event-Driven Architecture can offer the most important limited resource in the EDA: the Event better scalability than the One Thread Per Client Architecture, Handlers themselves. its use exposes service providers to a Denial of Service attack that we call Event Handler Poisoning (EHP). The source of the EDA’s scalability is also its Achilles’ heel. Multiplexing unrelated work onto the same thread re- This work is the first to define EHP attacks. After examining EHP vulnerabilities in the popular Node.js EDA framework and duces overhead, but it also moves the burden of time sharing open-source npm modules, we explore various solutions to EHP- out of the thread library or operating system and into the safety. For a practical defense against EHP attacks, we propose application itself. Where OTPCA-based services can rely on Node.cure, which defends a large class of Node.js applications preemptive multitasking to ensure that resources are shared against all known EHP attacks by making timeouts a first-class fairly, using the EDA requires the service to enforce its own member of the JavaScript language and the Node.js framework. cooperative multitasking [89]. An EHP attack identifies a way to defeat the cooperative multitasking used by an EDA-based Our evaluation shows that Node.cure is effective, broadly applicable, and offers strong security guarantees. First, we show service, ensuring Event Handler (s) remain dedicated to the that Node.cure defeats all known language- and framework- attacker rather than handling all clients fairly. Our analysis level EHP attacks, including real reported EHP attacks. Second, identifies many EHP vulnerabilities among previously-reported we evaluate its costs, finding that Node.js incurs reasonable npm vulnerabilities, and a sample of npm modules suggests overheads in micro-benchmarks, and low overheads ranging from that EHP vulnerabilities may be an epidemic (xIV). 1.02x (real servers) to 1.24x (middleware). Third, we show that Node.cure offers strong security guarantees against EHP attacks In xV we define criteria for EHP-safety, and use these for the majority of the Node.js ecosystem. criteria to evaluate different EHP defenses (xVI). The general solution we propose is to relax but not eliminate the coop- erative multitasking constraint of the EDA. Our prototype, I. INTRODUCTION Node.cure (xVII), makes timeouts a first-class member of the Node.js framework, and is the first framework we know of that Web services are the lifeblood of the modern Internet. To does so. If an Event Handler in Node.cure blocks, Node.cure handle the demands of millions of clients, service providers will emit a TimeoutError. Although cooperative multitasking replicate their services across many servers. Since every server is ultimately maintained, Node.cure thus informs every Event costs time and money, service providers want to maximize the Handler when it ought to yield. number of clients each server can handle. Over the past decade, this goal has led the software community to seriously consider Our findings on EHP-safety have been corroborated by a paradigm shift in their software architecture — from the One the Node.js community (xIX). We have developed a guide for Thread Per Client Architecture (OTPCA) used in Apache to practitioners on building EHP-proof systems (PR preliminarily the Event-Driven Architecture (EDA) championed by Node.js1. approved), updated the Node.js documentation to warn devel- Although using the EDA may increase the number of clients opers about the perils of particular APIs (two PRs merged), each server can handle, it also exposes the service to a new and submitted a pull request to improve the safety of these class of Denial of Service (DoS) attack unique to the EDA: APIs (PR preliminarily approved). Event Handler Poisoning (EHP). In our evaluation of Node.cure (xVIII), we describe the The OTPCA and the EDA differ in the resources they strong security guarantees against EHP that Node.cure offers, dedicate to each client. In the OTPCA, each client is assigned and demonstrate this by showing that Node.cure defeats rep- a thread, and the overhead associated with each thread limits resentatives of each of the classes of EHP attack. Our cost the number of concurrent clients the service can handle. In the evaluation on micro- and application benchmarks shows that EDA, a small pool of Event Handler threads is shared by all Node.cure introduces acceptable monitoring overhead, as low clients. The per-client overhead is thus smaller in the EDA, as 0% on real server applications. improving scalability. The first steps toward this paper were presented at a Though researchers have investigated application- and workshop2. There we introduced the idea of EHP attacks and language-level security issues that affect many services, we are argued that some previously-reported security vulnerabilities the first to examine the security implications of using the EDA were actually EHP vulnerabilities. We also proposed the “C- from a software architecture perspective. In xIII we describe a WCET Principle”, which we have evaluated and found wanting 1The EDA paradigm is also known as the reactor pattern. 2We omit the reference to meet the double-blind requirement. (xVI-C). Here we offer the complete package: we refine system call3), and pending events are handled sequentially our previous definition of EHP, increase our survey of EHP by an Event Loop. The Event Loop may offload expensive vulnerabilities in the wild, and describe a prototype that offers Tasks to the queue of a small, fixed-size Worker Pool, whose strong security guarantees using timeouts. Workers execute Tasks and generate “Task Done” events for the Event Loop when they finish. One common use of the In summary, our work makes the following contributions: Worker Pool is to provide quasi-asynchronous (blocking but 1) We are the first to define Event Handler Poisoning (EHP) concurrent) access to the file system. (xIII), a DoS attack against the EDA. EHP attacks con- sume Event Handlers, a limited resource in the EDA. Our definition of EHP accommodates all known EHP attacks. 2) After evaluating the solution space for EHP defenses (xVI), we describe a general antidote for Event Handler Poisoning attacks: first-class timeouts (xVII). We demonstrate its effectiveness with a prototype for the popular Node.js EDA framework. To the best of our knowledge, our prototype, Node.cure, is the first EDA framework to incorporate first- class timeouts. 3) We evaluate the security guarantees of timeouts (strong), their costs (small), and their effectiveness (quite) in the Node.js ecosystem (xVIII). 4) We released a guide on EHP-safe techniques for new developers in the EDA, which has been accepted. Our PR Fig. 1. This is the AMPED Event-Driven Architecture. Incoming events from clients A and B are stored in the event queue, and the associated Callbacks improving the safety of Node.js APIs has been accepted (CBs) will be executed sequentially by the Event Loop. The Event Loop may and our PRs documenting unsafe APIs were merged (xIX). generate new events, to be handled by itself later or as Tasks offloaded to the Worker Pool. Alas, B has performed an EHP attack on this service. His evil input (third in the event queue) will eventually poison the Event Loop, and II. BACKGROUND the events after his input will not be handled (xIII-D). In this section we contrast the EDA with the OTPCA (xII-A). We then explain our choice of EDA framework for Application developers must adopt the EDA paradigm study (xII-B), and finish by introducing formative work on when developing EDA-based software. Developers define the Algorithmic Complexity attacks (xII-C). set of possible Event Loop Events for which they supply a set of Callbacks [60]. During the execution of an EDA-based application, events are generated by external activity or internal A. Overview of the EDA processing and routed to the appropriate Callbacks. There are two main architectures for scalable web servers, We refer to the Event Loop and the Workers as Event the established One Thread Per Client Architecture (OTPCA) Handlers. Events for the Event Loop are supplied by the and the upstart Event-Driven Architecture (EDA). In the OT- operating system or the framework, while events for the PCA style, exemplified by the Apache Web Server [61], each Worker Pool (i.e. Tasks) are supplied by the Event Loop. We client is assigned a thread or a process, with a large maximum say that the Event Loop “executes a Callback” in response to number of concurrent clients. The OTPCA model isolates an event, and that a Worker “performs a Task”. A Callback clients from one another, reducing the risk of interference. is executed synchronously on the Event Loop, and a Task is However, each additional client incurs memory and context performed synchronously on a Worker, asynchronously from switching overhead. In contrast, by multiplexing multiple client the Event Loop’s perspective. connections onto a single-threaded Event Loop and offering a small Worker Pool for asynchronous tasks, the EDA approach In the EDA, each Callback and Task is guaranteed atom- reduces a server’s per-client resource use. While the OTPCA icity: once scheduled, it runs to completion. Because there are and EDA architectures have the same expressive power [71], relatively few Event Handlers, cooperative multitasking [89] in practice EDA-based servers can offer greater scaling [84].