1 JumboGen: Dynamic Jumbo Frame Generation For Network Performance Scalability David Salyers, Yingxin Jiang, Aaron Striegel, Christian Poellabauer Department of Computer Science and Engineering University of Notre Dame Notre Dame, IN. 46530 USA Email: {dsalyers, yjiang3, striegel, cpoellab}@nd.edu Abstract— Network transmission speeds are increasing at a there is a fixed amount of overhead processing per packet, significant rate. Unfortunately, the actual performance of the it is a challenge for CPUs to scale with increasing network network does not scale with the increases in line speed. This speeds [2]. In order to overcome this issue, many works on is due to the fact that the majority of packets are less than or equal to 64 bytes and the fact that packet size has not scaled with high performance networks, as in [3] and [4], have advocated the increases in network line speeds. This causes a greater and the use of a larger MTU length or jumbo frames. These works greater load to be placed on routers in the network as routing show that the performance of TCP and the network in general decisions have to be made for an ever increasing number of can significantly improve with the use of jumbo-sized packets. packets, which can cause increased packet delay and loss. In Unfortunately, the benefits of jumbo frames are limited for order to help alleviate this problem and make the transfer of bulk data more efficient, networks can support an extra large several reasons. First, the majority of packets transferred are MTU size. Unfortunately, when a packet traverses a number of 64 bytes or less. In this case, a larger MTU would not increase different networks with different MTU sizes, the smallest of the the efficiency of the network since the smaller MTU is already MTU sizes will determine the size of the packet that arrives at significantly greater than the packet size. The second factor the destination, which will limit the effectiveness of large MTU that limits the use of jumbo frames is the lack of end-to- packets. In this paper we present our solution, JumboGen, an approach end support. If at any point during the packet transmission that will allow for a higher utilization of larger packet sizes on the jumbo frame passes through a node that does not support a domain-wise basis, instead of on an end-to-end basis. Smaller jumbo frames, the packet will be fragmented into many smaller packets are encapsulated into a larger packet for transmission packets via IP fragmentation. The destination will then receive across the domain. At the egress point, the original packets are these small fragments instead of the original jumbo frame. removed from the jumbo-sized packet and forwarded toward their final destination such that no nodes besides the ingress and Given that network speeds will be increasing for the fore- egress are aware any conversion took place. Through simulations, seeable future, a reduction in the volume of packet processing we show that the dynamic creation of jumbo packets for the is critical. However, the legacy nature of the current Internet domain does not adversely affect the network in terms of link dictates that attempts to impose end-to-end solutions for larger utilization or fairness while decreasing the number of packets packet sizes will be plagued by fragmentation, thus removing processed by core routers. The decrease in the number of packets seen by the core routers will lessen their load and improve their their benefit. In this paper, we propose the JumboGen approach ability to scale to greater and greater network speeds. which allows for the incremental deployment and efficient utilization of jumbo-size packets. JumboGen dynamically aggregates smaller packets at the I. INTRODUCTION ingress to a domain for domain-wise transmission as a single Network (LAN and WAN) speeds are continuously increas- large JumboGen packet. At the egress point, the original ing and are such that computer networks can now transfer packets are de-queued, shaped, and transmitted toward their billions of bits, and beyond, of information every second. final destination. The end result of JumboGen is a significant However, in these high speed networks, the number of packets reduction in the number of packets, hence significantly im- becomes a major bottleneck for the performance of the routers proving the scalability of the core of the network. due to the overhead associated with each packet. There are two However, before JumboGen can be implemented numerous primary reasons for this bottleneck. First, as network speeds issues need to be addressed. These issues include, how to have increased, the common MTU size has not. Second, since determine which packets can be aggregated together, RED the majority of packets transferred over the network are 64 Queue effects on JumboGen packets, release of encapsulated bytes or less in length [1], a larger MTU would not help reduce packets, and partial deployment issues. The remainder of the the number of those types of packets. These two facts have paper is organized as follows: Section II presents an overview caused the number of packets the routers in the network need of the JumboGen approach and presents solutions to important to process to increase substantially. issues, JumboGen packet assembly, RED interactions, schedul- For example, consider a 10 Mb/s network that can handle ing, and deployment. Next, Section III discuses the details N packets every second. If the network is upgraded to 10 Gb/s of the simulation studies performed with JumboGen. Then, the routers would be expected to handle 1000N packets. Since background and related works are discussed in Section IV. 2 B D A 4,5 C 6 7 C D C 4 B A J IP JumboGen B 1,2,3 De-Aggregator A JumboGen Aggregator D Standard Packets JumboGen Packets Standard Packets Fig. 1. JumboGen operation Overview Finally, Section V presents conclusions for the paper. may be difficult to discern. Thus, JumboGen includes mechanisms to ensure that only packets sharing the same II. JUMBOGEN DETAILS egress point will be combined into the same JumboGen packet. Fig. 1 shows an overview of the JumboGen approach. The • Packet Encapsulation: The structure of the JumboGen basic flow of events is described below: packet is such that the method is computationally simplis- 1) Packets arrive at an ingress node to the domain. tic with little overhead. Furthermore, the packet structure 2) At the ingress node, the packets are sorted into queues must be easily modifiable to allow for pruning of ag- based on egress node, while starting a Jumbo Frame gregated packets to support RED-like behavior on a fine Aggregation Timer (JFAT) for the queue. granularity. 3) Packets that are being sent through the same egress point • Structure of Packet Queues: JumboGen incorporates an are combined into the same JumboGen packet, subject efficient and fast method for determination of packet to MTU. placement at the ingress. 4) Once the JFAT for the queue has expired, the JumboGen • RED Interactions: Since JumboGen packets consist of packet is released towards the egress point. many smaller encapsulated packets, the loss of a Jumbo- 5) The JumboGen packet is routed through the core of Gen packet has more significant implications for network the network. Routing is provided by using the standard dynamics. Thus, the JumboGen approach includes con- routing mechanism of the network1. siderations for RED and the interactions with JumboGen 6) The JumboGen packet arrives at the egress node and the and non-JumboGen packets. original packets are separated out. 7) Finally, the original packets are forwarded onto their final destination. A. Aggregation with Shared Egress Points There are two main benefits to this approach. The first is that If packets are randomly aggregated without consideration JumboGen lowers the number of packets that the core routers for their egress points, it is likely that when the JumboGen are responsible for processing, thus allowing the network to packet arrives at an egress node, many of the encapsulated better scale as line speeds increase. The second beneficial packets will be routed back across the domain. This can aspect to the use of JumboGen is that data is more efficiently add a significant amount of delay and is unacceptable from transferred over the network. The increase of efficiency is due a network routing perspective. Hence, in order to aggregate to the reduced number of physical layer headers used (due to packets together at the ingress node, the egress node for the a lower number of packets) to transfer the same amount of packet should be known. data over the network. Unfortunately, in standard IP intra-domain routing (ie. While this process seems relatively simple there are a OSPF, IS-IS) only the next hop is known at any given link number of non-trivial issues that need to be addressed. Issues on the packets path to the destination. However, it is possible that need to be addressed include: to discern which packets will have the same egress point, even if the specific egress point can not be determined. • Aggregation with Shared Egress Points: Since IP based If the network supports MPLS [5], aggregation of packets routing provides only the next hop, the egress point can be performed by matching MPLS labels. Since the same 1Information from the header of an encapsulated packet (MPLS or IP) is MPLS label indicates that packets are in the same Forward simply copied to the actual JumboGen packet Equivalence Class (FEC) and will be routed along the same 3 Packet Queues Filter Addr: 129.154.190 Parent Link: IP Header NULL Filter Addr: Filter Addr: { 129.154.160 129.154.200 QTime Parent Link: Parent Link: NULL NULL avgLen nPkt pLen1 pLen2 Filter Addr: Filter Addr: JumboGen Header 129.154.150 129.154.195 Parent Link: Parent Link: NULL 129.154.160 {pLen(n-1) pLen(n) Packet 1 Fig.