XORP: An Open Platform for Network Research Mark Handley Orion Hodson Eddie Kohler ICSI Center for Internet Research, Berkeley, California mjh, hodson, kohler @icir.org f g ABSTRACT searchers to experiment with and deploy new work on. Routing Network researchers face a significant problem when deploying toolkits exist, but typically they implement a subset of IP rout- software in routers, either for experimentation or for pilot deploy- ing functionality and are rarely used in production environments— ment. Router platforms are generally not open systems, in either routing and forwarding research requires access to real production the open-source or the open-API sense. In this paper we discuss the traffic and routing information to be validated. Similarly, open- problems this poses, and present an eXtensible Open Router Plat- source-based testbed networks such as CAIRN [1] provide valu- form (XORP) that we are developing to address these issues. Key able tools for the researcher, but they rarely provide a realistic test goals are extensibility, performance and robustness. We show that environment and are usually limited to a small number of sites due different parts of a router need to prioritize these differently, and to cost. A recent spate of research in open, extensible forwarding examine techniques by which we can satisfy these often conflicting paths is moving in the right direction [6, 11], but a truly extensible, goals. We aim for XORP to be both a research tool and a stable production-quality router would need routing daemons, forwarding deployment platform, thus easing the transition of new ideas from information bases, management interfaces, and so on in addition to the lab to the real world. a forwarding path. How then can we enable a pathway that permits research and ex- perimentation to be performed in production environments whilst 1 VALIDATING INTERNET RESEARCH minimally impacting existing network services? In part, this is the A yawning gap exists between research and practice concerning same problem that Active Networks attempted to solve, but we be- Internet routing and forwarding disciplines. The savvy researcher lieve that a much more conservative approach is more likely to see has the tools of theory and simulation at hand, but validating results real-world usage. in the real world is hard. Why should this be so? We envision an integrated open-source software router platform, For network applications research, we have access to languages, running on commodity hardware, that is viable as a research and APIs, and systems that make development and deployment easy. as a production platform. The software architecture should be de- For end-to-end protocol research, we have access to open source signed with extensibility as a primary goal and should permit ex- operating systems, such as Linux and FreeBSD. End-to-end proto- perimental protocol deployment with minimal risk to existing ser- cols can be simulated and implemented in these systems. And since vices using that router. Internet researchers needing access to router these operating systems are used in both research and production software could then share a common platform for experimentation environments, migration from the research to the production envi- deployed in places where real traffic conditions exist. Researchers ronment is feasible. TCP SACK provides an excellent example [8]. working on novel router hardware could also use the mature soft- Unfortunately the same cannot be said of router software. Router ware from this platform to test their hardware in real networks. In vendors do not provide API’s that allow third party applications these ways, the loop between research and realistic real-world ex- to run on their hardware. Thus, even conducting a pilot study in perimentation can be closed, and innovation can take place much a production network requires the router vendor to implement the more freely. protocol. Unless the router vendor perceives a reward in return for the effort, they are unlikely to invest resources in the protocol im- 1.1 Alternatives plementation. Similarly, customers are unlikely to request a feature unless they have faith in existing research results or can experi- Having motivated the need for an open router on which net- ment in their own environment. A catch-22 situation exists of not work research can be deployed, we discuss the alternatives in more being able to prototype and deploy new experimental protocols in detail—simulations and network testbeds. any kind of realistic environment. Even when vendors can be con- First, we note that it has not always been so difficult to deploy vinced to implement, it is not uncommon for initial implementa- experimental work on the Internet. Prior to the advent of the World tions of a protocol to be found wanting, and the path to improving Wide Web, the Net was predominantly non-commercial. Most of its the protocols is often difficult and slow. Finally, network operators users came from universities and other research labs. Whilst there are almost always reluctant to deploy experimental services in pro- was a tension between conducting research and providing a net- duction networks for fear of destabilizing their existing (hopefully working service, researchers could have access to the network to money-making) services. run experiments, develop and test new protocols, and so forth. With Thus, we believe the difficulty in validating Internet research is the advent of the Web, the network grew rapidly and commercial largely attributable to the absence of open Internet routers for re- ISPs emerged. Even the academic parts of the network became re- luctant to perform networking experiments for fear of disrupting regular traffic. In the commercial parts of the network, where inter- First Workshop on Hot Topics in Networks, Princeton, New Jersey, October 28–29, 2002 esting scaling phenomena started to emerge, it was nearly impos- Permission to make digital or hard copies of all or part of this work for any sible to do any form of experimentation. Growth was so rapid that purpose is granted without fee provided that copies bear this notice and the it was all ISPs could do to keep up with provisioning. These prob- full citation on the first page. lems were recognised, and two main solutions emerged: network Copyright c 2002 International Computer Science Institute testbeds and network simulators. Network testbeds rarely resulted in good network research. One present an integrated router user interface to the network op- notable exception was DARTnet [3], which used programmable erator, which reduces the likelihood of acceptance. routers that network researchers had access to. Among its achieve- Researchers could develop a new open-source router, designed ments, DARTnet demonstrated IP multicast and audio and video • conferencing over IP. It worked well because the network users from the ground up for extensibility, robustness, and perfor- were also the network researchers and so there was less tension mance. It would take time to build such a router, and then to involved in running experiments. convince the network operator community that it met strin- Over recent years, the majority of network research that involved gent robustness standards, but the resulting router would make testing protocols has taken place in network simulators such as ns. a more useful research platform than any of the other choices. Among the desirable properties of simulators is the complete con- We have chosen this last path, which we call the Extensible Open trol they provide over all the parameters of a system, and so a large Router Platform, or XORP. Our design addresses the four major range of scenarios can be examined. Within the research commu- challenges in building an open-source router: traditional router fea- nity the ns simulator has been particularly successful1. Many re- tures, extensibility, performance, and robustness. searchers have contributed to improve ns itself, but an even greater number have used it in their research. Many published results are Features. Real-world routers must support a long feature list, in- supported by publicly available simulation code and scripts. This cluding routing protocols, management interfaces, queue man- has allowed for the direct comparison of contemporary networking agement, and multicast. algorithms and allowed for independent verification of results. It could therefore be argued that ns has increased the rigor of network Extensibility. Every aspect of the router should be extensible, from research. routing protocols down to details of packet forwarding. The Conversely, it could equally well be argued that simulators make router must support multiple simulatenous extensions as long it too easy to run experiments and are responsible for numerous as those extensions don’t conflict. APIs between router com- papers that bear little, or no, relationship to real networks. Accord- ponents should be both open and easy to use. ingly there is understandable doubt about any claims until they’ve been demonstrated in the real world. Performance. XORP isn’t designed for core routers, at least not Even in skilled hands, simulators have limits. When work re- initially. However, forwarding performance is still important: quires access to real traffic patterns, or needs to interact with real that is the purpose of a router. Scalability in the face of rout- routing protocols, or relates to deployed implementations warts- ing table size or number of peers is also critical. and-all, there is no substitute for real-world experimentation. Robustness. Real-world routers must not crash or misroute pack- ets. A fragile router faces an extremely difficult deployment 2 ARCHITECTURE AND REQUIREMENTS path. Extensibility makes robustness even more difficult to We’ve hopefully convinced you that Internet research would be achieve: extensions are inherently experimental, yet their use better off if changes and extensions could be provisionally deployed. should not compromise the router’s robustness (except per- No simulation or testing environment can provide similarly useful haps for a subset of packets intended for the extension).