Washington University in St. Louis Washington University Open Scholarship All Computer Science and Engineering Research Computer Science and Engineering Report Number: WUCSE-2011-101 2011 The Forest Overlay Network Logan Stafman Forest is an overlay network designed for large real-time distributed systems. In particular, we’re interested in virtual worlds that provide high-quality interaction over a constantly changing pattern of communication. Forest is suitable for applications in which many entities send data to a large set of constantly changing entities. By using tree-structured channels, we can create logically isolated private networks which support both unicast and multicast routing. In this paper, we will discuss the core components of Forest and measure the performance of the control elements of the network. We will also provide examples of control sequences and the roles... Read complete abstract on page 2. Follow this and additional works at: https://openscholarship.wustl.edu/cse_research Part of the Computer Engineering Commons, and the Computer Sciences Commons Recommended Citation Stafman, Logan, "The Forest Overlay Network" Report Number: WUCSE-2011-101 (2011). All Computer Science and Engineering Research. https://openscholarship.wustl.edu/cse_research/55 Department of Computer Science & Engineering - Washington University in St. Louis Campus Box 1045 - St. Louis, MO - 63130 - ph: (314) 935-6160. This technical report is available at Washington University Open Scholarship: https://openscholarship.wustl.edu/ cse_research/55 The Forest Overlay Network Logan Stafman Complete Abstract: Forest is an overlay network designed for large real-time distributed systems. In particular, we’re interested in virtual worlds that provide high-quality interaction over a constantly changing pattern of communication. Forest is suitable for applications in which many entities send data to a large set of constantly changing entities. By using tree-structured channels, we can create logically isolated private networks which support both unicast and multicast routing. In this paper, we will discuss the core components of Forest and measure the performance of the control elements of the network. We will also provide examples of control sequences and the roles played by control elements in those sequences to help maintain fast, reliable data delivery. Department of Computer Science & Engineering 2011-101 The Forest Overlay Network Authors: Logan Stafman Corresponding Author: [email protected] Abstract: Forest is an overlay network designed for large real-time distributed systems. In particular, we’re interested in virtual worlds that provide high-quality interaction over a constantly changing pattern of communication. Forest is suitable for applications in which many entities send data to a large set of constantly changing entities. By using tree-structured channels, we can create logically isolated private networks which support both unicast and multicast routing. In this paper, we will discuss the core components of Forest and measure the performance of the control elements of the network. We will also provide examples of control sequences and the roles played by control elements in those sequences to help maintain fast, reliable data delivery. Type of Report: Other Department of Computer Science & Engineering - Washington University in St. Louis Campus Box 1045 - St. Louis, MO - 63130 - ph: (314) 935-6160 Introduction Forest is an overlay network designed for large distributed real-time applications such as online virtual worlds. In order to benefit from Forest, an application must involve many entities sharing real-time status with a constantly changing subset of other entities. The status updates issued by clients can be sent up to tens of times per second. In order to keep high-quality interaction, these packets must be delivered non-stop even as communication patterns change. A tree-structured channel in Forest is called a comtree, or communications tree. This channel supports both unicast and multicast packet delivery. Each comtree can be viewed as a private network that is logically isolated and can be provisioned to meet bandwidth requirements of the application. For example, in the case of a virtual world, each comtree might be a separate world; clients from one world cannot interact with or affect the worlds of clients in other worlds. Clients can join and leave comtrees easily by sending requests to the Comtree Controller. Forest routers use tables to manage their links, routes, and comtree links, and interfaces. For example, when a client (or, in the comtree sense, leaf) joins the network, the link table of the router must be updated to include the new link. When a client joins a data comtree, the comtree link table is updated accordingly. The routers together keep a global view of the topology of the Forest network. Addressing in Forest Forest addresses are 32 bit numbers. The high order bit distinguishes between unicast and multicast addresses; a 1 here identifies this address as a multicast address. Unicast addresses have two parts, the zip code and the local address. The zip code is 15 bits long and the local address is 16 bits long, allowing 2.1 billion addresses on a single comtree. Note that addresses in different comtrees are independent of one another, so we have many more unicast addresses available. Unicast addresses that are in the same zip code must form a single connected subtree of the overall comtree in order to keep routing scalable. This means that from any node outside of a zip code, there must exist a single link by which nodes in that zip code can be reached. With this restriction, routers can maintain a single route for foreign zip codes, allowing local routers to forward packets to local endpoints. In the example above, an endpoint in zip code 2 is attempting to send a packet to an endpoint in zip code 3. Due to the fact that zip code 3 is a subtree, there exists one link from zip code 2 to zip code 3. 2 Packet Format Packets on the Forest network have some additional information to allow for quick routing. Below is a description of a forest packet. • Version number. First four bits – currently at 1. • Packet Length. Specified in bytes, the field is 12 bits long. 1400 is maximum size. • Flags. This is an 8 bit field. Currently, the only flag is the route request flag set by a router to request an endpoint. • Comtree. This is a 32 bit field which identifies the comtree on which this packet should be processed. • Source Address. This 32 bit field specifies the unicast forest address of the sender. • Destination Address. This 32 bit field specifies the unicast or multicast forest address of the receiver. • Header Error Check. This is a 32 bit check sum for the header. • Payload Error Check. This is a 32 bit check sum for the payload (excluding the header). Topology File Format Each Forest Network has a specific topology that must be described in a topology file. This topology outlines everything about the instance of Forest, including routers, their interfaces, predefined links to control elements, and definitions for all of the predefined comtrees (though comtrees can be added dynamically as well). Below is a description of the topology file format. Routers: The router section contains each router, the range of legal Forest addresses it can assign to clients, its physical location (in order to select the closest router to a client), its Forest Address, and a list of its interfaces. Each interface includes an IP address, the links that interface is in charge of, and maximum bit rates and packet rates allowed on that interface. Leaf Nodes: While clients can connect dynamically, the links for the control elements must be predefined in the topology. Each leaf node has a name, an IP address, a Forest address, and a location. Links: Any backbone links (e.g. links connecting routers) should be defined in this section. In addition, links to predefined leaf nodes should be listed here. Each link has the two nodes it connects, its length, its maximum packet rate, and its maximum bitrate. Comtrees: This section contains a listing of each of the comtrees in this instance of Forest. Both data and control comtrees should be defined here. Each comtree should have a comtree number, a root node, and the upstream and downstream packet rates for backbones and for leaf links. In addition, all predefined comtree links should be defined here; each set of comtree links must be a subset of the links previously defined in the links section. Links to controllers must be present on control comtrees. 3 Packet Forwarding Unicast Forwarding Each router typically holds routes for each foreign zip (e.g. the first half of the Forest address), as well as routes to all local addresses. These routes to foreign addresses are acquired using a self-learning process similar to that used in Ethernet LANs. If a router receives a packet for which it has no route, it will forward the packet on each outgoing link and set the Route Request Flag, asking any endpoint that already has a route to the destination address to respond with a Route Reply Packet. Above, a client on the left wants a route to Forest address 2.3. Assuming router E has a route to 2.3, the packet is flooded across the Forest network, until router E replies with a Route Reply packet and turn off the Route Request flag. If an earlier router in the path (e.g. router C) has a route to 2.3 already, then the network will not be flooded, and C will simply respond with the route to the destination. Local addresses are known to access routers when an endpoint is added to the Forest Network. Routers have the option, but are not required, to process Route Request packets by storing the route to the requesting router.