TESLA: A Transparent, Extensible Session-Layer Architecture for End-to-end Network Services Jon Salz Alex C. Snoeren MIT Laboratory for Computer Science University of California, San Diego [email protected] [email protected] Hari Balakrishnan MIT Laboratory for Computer Science [email protected] Session-layer services for enhancing functionality and ² Encryption services for sealing or signing ¤ows. improving network performance are gaining in impor- ² General-purpose compression over low-bandwidth tance in the Internet. Examples of such services in- links. clude connection multiplexing, congestion state shar- ² ing, application-level routing, mobility/migration sup- Traf£c shaping and policing functions. port, and encryption. This paper describes TESLA, a These examples illustrate the increasing importance of transparent and extensible framework allowing session- session-layer services in the Internet—services that oper- layer services to be developed using a high-level ¤ow- ate on groups of ¤ows between a source and destination, based abstraction. TESLA services can be deployed and produce resulting groups of ¤ows using shared code transparently using dynamic library interposition and and sometimes shared state. can be composed by chaining event handlers in a graph structure. We show how TESLA can be used to imple- Authors of new services such as these often imple- ment several session-layer services including encryption, ment enhanced functionality by augmenting the link, net- SOCKS, application-controlled routing, ¤ow migration, work, and transport layers, all of which are typically im- and traf£c rate shaping, all with acceptably low perfor- plemented in the kernel or in a shared, trusted interme- mance degradation. diary [12]. While this model has suf£ced in the past, we believe that a generalized, high-level framework for session-layer services would greatly ease their develop- 1 Introduction ment and deployment. This paper argues that Internet end hosts can bene£t from a systematic approach to de- Modern network applications must meet several increas- veloping session-layer services compared to the largely ing demands for performance and enhanced function- ad-hoc point approaches used today, and presents TESLA ality. Much current research is devoted to augmenting (a Transparent, Extensible Session Layer Architecture), the transport-level functionality implemented by stan- a framework that facilitates the development of session- dard protocols as TCP and UDP. Examples abound: layer services like the ones mentioned above. Our work with TESLA derives heavily from our own ² Setting up multiple connections between a source previous experience developing, debugging, and deploy- and destination to improve the throughput of a sin- ing a variety of Internet session-layer services. The earli- gle logical data transfer (e.g., £le transfers over est example is the Congestion Manager (CM) [4], which high-speed networks where a single TCP connec- allows concurrent ¤ows with a common source and des- tion alone does not provide adequate utilization [2, tination to share congestion information, allocate avail- 15]). able bandwidth, and adapt to changing network condi- tions. Other services include Resilient Overlay Networks ² Sharing congestion information across connections (RON) [3], which provides application-layer routing in sharing the same network path. an overlay network, and the Migrate mobility architec- ² Application-level routing, where applications route ture [23, 24], which preserves end-to-end communica- traf£c in an overlay network to the £nal destination. tion across relocation and periods of disconnection. ² End-to-end session migration for mobility across Each these services was originally implemented at the network disconnections. kernel level, though it would be advantageous (for porta- bility and ease of development and deployment) to have new network services. This paper describes the design them available at the user level. Unfortunately this tends and implementation of TESLA, a generic framework for to be quite an intricate process. Not only must the im- development and deployment of session-layer services. plementation specify the internal logic, algorithms, and TESLA consists of a set of C++ application program in- API, but considerable care must be taken handling the terfaces (APIs) specifying how to write these services, details of non-blocking and blocking sockets, interpro- and an interposition agent that can be used to instantiate cess communication, process management, and integrat- these services for use by existing applications. ing the API with the application’s event loop. The end re- sult is that more programmer time and effort is spent set- ting up the session-layer plumbing than in the service’s 2 Related Work logic itself. Our frustrations with the development and implementation of these services at the user level were Today’s commodity operating systems commonly al- the prime motivation behind TESLA and led to three ex- low the dynamic installation of network protocols on plicit design goals. a system-wide or per-interface basis (e.g., Linux ker- First, it became apparent to us that the standard BSD nel modules and FreeBSD’s netgraph), but these exten- sockets API is not a convenient abstraction for program- sions can only be accessed by the super-user. Some ming session-layer services. Sockets can be duplicated operating systems, such as SPIN [5] and the Exoker- and shared across processes; operations on them can be nel [12], push many operating system features (like net- blocking or non-blocking on a descriptor-by-descriptor work and £le system access) out from the kernel into basis; and reads and writes can be multiplexed using sev- application-speci£c, user-con£gurable libraries, allow- eral different APIs (e.g., select and poll). It is undesir- ing ordinary users £ne-grained control. Alternatively, able to require each service author to implement the en- extensions were developed for both operating systems to tire sockets API. In response, TESLA exports a higher allow applications to de£ne application-speci£c handlers level of abstraction to session services, allowing them to that may be installed directly into the kernel (Plexus [14] operate on network ¤ows (rather than simply socket de- and ASHs [27]). scriptors) and treat ¤ows as objects to be manipulated. Operating systems such as Scout [21] and x- Second, there are many session services that are re- kernel [16] were designed explicitly to support sophis- quired ex post facto, often not originally thought of by ticated network-based applications. In these systems, the application developer but desired later by a user. For users may even rede£ne network or transport layer proto- example, the ability to shape or police ¤ows to conform col functions in an application-speci£c fashion [6]. With to a speci£ed peak rate is often useful, and being able TESLA, our goal is to bring some of the power of these to do so without kernel modi£cations is a deployment systems to commodity operating systems in the context advantage. This requires the ability to con£gure session of session-layer services. services transparent to the application. To do this, TESLA In contrast to highly platform-dependent systems such uses an old idea—dynamic library interposition [9]— as U-Net [26] and Alpine [11], TESLA does not attempt taking advantage of the fact that most applications today to allow users to replace or modify the system’s network on modern operating systems use dynamically linked li- stack. Instead, it focuses on allowing users to dynami- braries to gain access to kernel services. This does not, cally extend the protocol suite by dynamically compos- however, mean that TESLA session-layer services must ing additional end-to-end session-layer protocols on top be transparent. On the contrary, TESLA allows services of the existing transport- and network-layer protocols, to de£ne APIs to be exported to enhanced applications. achieving greater platform independence and usability. Third, unlike traditional transport and network layer TESLA’s modular structure, comprising directed services, there is a great diversity in session services graphs of processing nodes, shares commonalities with as the examples earlier in this section show. This im- a number of previous systems such as x-kernel, the Click plies that application developers can bene£t from com- modular router [19], and UNIX System V streams [22]. posing different available services to provide interesting Unlike the transport and network protocols typically con- new functionality. To facilitate this, TESLA arranges for sidered in these systems, however, we view session-layer session services to be written as event handlers, with a protocols as an extension of the application itself rather callback-oriented interface between handlers that are ar- than a system-wide resource. This ensures that they are ranged in a graph structure in the system. subject to the same scheduling, protection, and resource We argue that a generalized architecture for the de- constraints as the application, minimizing the amount of velopment and deployment of session-layer functionality effort (and privileges) required to deploy services. will signi£cantly assist in the implementation and use of To avoid making changes to the operating system or to the application itself, TESLA transparently interposes it- Input flow self between the application and the kernel, intercepting from upstream Zero or more Flow handler output flows and modifying the interaction between the application to downstreams and the system—acting as an interposition agent [18]. It uses dynamic library interposition [9] to modify the interaction between the application and the system. This technique is popular with user-level £le systems such as Figure 1: A ¤ow handler takes as input one network ¤ow IFS [10] and Ufo [1], and several libraries that provide and generates zero or more output ¤ows. speci£c, transparent network services such as SOCKS [20], Reliable Sockets [29], and Migrate [23, 24]. Each Upstream of these systems provides only a speci£c service, how- ever, not an architecture usable by third parties.