The GINI Router:

A Routing Element for User-level Micro Internet

by

Weiling Xu

DEPARTMENT OF COMPUTER SCIENCE

MCGILL UNIVERSITY, MONTREAL

JUNE,2004

A THESIS

SUBMITTED TO MCGILL UNIVERSITY

IN PARTIAL FULFILMENT OF THE REQUlREMENTS OF THE

DEGREE OF MASTER OF SCIENCE

© Weiling Xu 2004 Library and Bibliothèque et 1+1 Archives Canada Archives Canada Published Heritage Direction du Branch Patrimoine de l'édition

395 Wellington Street 395, rue Wellington Ottawa ON K1A ON4 Ottawa ON K1A ON4 Canada Canada

Your file Votre référence ISBN: 0-494-06476-5 Our file Notre référence ISBN: 0-494-06476-5

NOTICE: AVIS: The author has granted a non­ L'auteur a accordé une licence non exclusive exclusive license allowing Library permettant à la Bibliothèque et Archives and Archives Canada to reproduce, Canada de reproduire, publier, archiver, publish, archive, preserve, conserve, sauvegarder, conserver, transmettre au public communicate to the public by par télécommunication ou par l'Internet, prêter, telecommunication or on the Internet, distribuer et vendre des thèses partout dans loan, distribute and sell th es es le monde, à des fins commerciales ou autres, worldwide, for commercial or non­ sur support microforme, papier, électronique commercial purposes, in microform, et/ou autres formats. paper, electronic and/or any other formats.

The author retains copyright L'auteur conserve la propriété du droit d'auteur ownership and moral rights in et des droits moraux qui protège cette thèse. this thesis. Neither the thesis Ni la thèse ni des extraits substantiels de nor substantial extracts from it celle-ci ne doivent être imprimés ou autrement may be printed or otherwise reproduits sans son autorisation. reproduced without the author's permission.

ln compliance with the Canadian Conformément à la loi canadienne Privacy Act some supporting sur la protection de la vie privée, forms may have been removed quelques formulaires secondaires from this thesis. ont été enlevés de cette thèse.

While these forms may be included Bien que ces formulaires in the document page count, aient inclus dans la pagination, their removal does not represent il n'y aura aucun contenu manquant. any loss of content from the thesis. ••• Canada Abstract (English)

Most routers, to our knowledge, are difficult for researchers and students without root privileges to implement or modify, because their implementation is in kemel space. GINI is an innovative platform on conventional PCs, which brings the whole functionality of a router into user space. GINI is able to create a complex virtual network consisting of

User-Mode Linux (UML), virtual switches and routers. GINI is far more than a simulation engine, because packets on a virtual link may experience real networks by going through UDP tunnels. This thesis also presents a new architecture for building flexible and configurable routers in GIN!. They have a modular structure and are easy to extend. Our evaluation shows that the GINI router has a much better performance than

UML to be a router. Since the GINI routers are lightweight and configurable, it is easy to run hundreds of them by a simple script on one or more PCs, without crashing them.

11 Résumé (Français)

La plupart des routeurs sont, à notre connaissance, difficiles à appliquer ou à modifier pour des chercheurs et des étudiants sans privilèges de racine, parce que leur implantation s'effectue dans l'espace de grain. GINI est une plateforme innovatrice sur les PCs conventionnels, qui apporte la fonctionnalité entière d'un routeur dans l'espace d'utilisateur. GINI peut créer un réseau virtuel complexe constitué de Linux D'utilisateur­ mode (UML) , de commutateurs et routeurs virtuels . GINI est bien plus que de la

émulation, parce que les paquets sur un lien virtuel peuvent traverser de vrais réseaux en passant par des tunnels UDP. Cette thèse présente également une nouvelle architecture pour la construction de routeurs virtuels flexibles et configurables. Ils ont une structure modulaire et sont faciles à prolonger. Notre évaluation prouve que le routeur de GINI est un routeur bien plus performant qu'UML. Comme les routeurs de GINI sont légers et configurables, il est facile de lancer des centaines d'eux par un script simple sur un ou plusieurs PCs, sans affecter le fonctionnement de ceux-ci.

111 Acknowledgements

l would like to thank my supervisor Prof. Muthucumaru Maheswaran for his direction, assistance, and guidance. His recommendations and suggestions have been invaluable for the project and for software improvement.

l am deeply grateful to Shah Asaduzzaman for giving me many use fui suggestions. l also wish to thank Arindam Mitra, who is the adrninistrator of ANRL lab and assisted me with setting up the environment for UML. Special thanks should be given to my colleagues in ARNL lab who helped me in many ways.

Finally, words alone cannot express the thanks l owe to my parents Mingqian Xu and

Junwen Zhang, and my husband Yuan Xu, for their encouragement and assistance.

IV Table of Contents

Chapter One: Introduction ...... 1 1.1 Motivation for the Thesis ...... 1 1.2 Thesis Outline ...... 5 Chapter Two: Related Work and GINI ...... 7 2.1 Re1ated Work ...... 7 2.1.1 Virtual Router Improved ...... 7 2.1.2 Click ...... 8 2.1.3 Virtual Router ...... 8 2.1.4 Extensible Open Router Platfonn & Zebra ...... 9 2.1.5 An Implementation of a Teaching Network with User-Mode Linux & Virtual Network User-Mode Linux ...... 9 2.2 Elements of GINI ...... Il Chapter Three: Background Knowledge ...... 12 3.1 Overview of User-Mode Linux (UML) ...... 12 3.1.1 What is UML? ...... 12 3.1.2 UML Networking ...... 13 3.1.3 The Switch Daemon & the GINI Switch ...... 14 3.1.4 Desired GINI Router ...... 17 3.2 Hardware Router andthe GINI Router...... 18 3.2.1 Hardware Router Architecture ...... 20 3.2.2 What Can be Imp1emented in the GINI Router? ...... 22 Chapter Four: Design of the GINI Router...... 24 4.1 Modular Structure ...... 24 4.2 How Modules ColIaborate? ...... 26 Chapter Five: Implementation of the GINI Router...... 29 5.1 Essential Global Data Structures ...... 29 5.2 Common Routines Shared by AlI Modules ...... 34 5.3 Organization of Files ...... 38 5.4 CLI Module ...... 39 5.5 UML_ADAPTER Module ...... 41 5.6 IP Module ...... 46 5.7 ICMP Module ...... 50 5.8 FRAGMENT Module ...... 52 5.9 ARP Module ...... 53 5.10 QOS Module ...... 56 5.10.1 InternaI Structure ...... 56 5.10.2 Worst Case Weighted Fair Queuing + ...... 59 5.10.3 Message Flows ...... 61 Chapter Six: Profiling Tooi in the GINI Router ...... 63 6.1 Overvievv ...... 63 6.2 Statistical Type ...... 65 v Chapter Seven: Customization of the GINI Router...... 68 7.1 Add a New Module ...... 68 7.2 Remove a Module ...... 69 Chapter Eight: Evaluation ...... 71 8.1 Experimental Setup ...... 71 8.2 Packet Delay ...... 72 8.3 Forwarding Rates ...... 76 8A InternaI Overload Behaviour of the GINI Router ...... 82 8.5 Delay Breakdown to Modules ...... 84 8.6 Additional Overhead ...... 87 8.7 Differentiated Services ...... 88 8.8 Configurations ofNetwork ...... 90 Chapter Nine: Conclusion ...... 96 9.1 Summary and Discussion of Present Work ...... 96 9.2 Future Work ...... 98 Appendix A - User-Mode Linux Manua!...... 99 A.1 Installation & Preparation ...... 99 A.l.I DownIoad & Install UML RPM ...... 99 A.l.2 Configure & Recompile the Kemel of Host Computer to Support UML. .... 100 A.I.3 Build UML Kemel from Source ...... 102 AJA Prepare the Root File System ...... 103 A.2 Launch UML and Set up Its Network by Using the GINI Switch ...... 104 A.3 Permanently Change and Save Network Configuration in COW File ...... 106 A.4 Shutdown UMLs and GINI Switches ...... 107 A.S Set up a Virtual Network for Two UMLs on Different Hosts ...... 108 A.6 Set up UML Network By Using Multicast ...... 109 A. 7 Mount Host Machine Filesystem in UML ...... 109 A.8 Set up a Virtual Network with Multiple LANs by Using UML as Router ...... 109 Appendix B - Bibliography ...... 112

VI List of Tables

Table 5-1 Connection Table ...... 42

Table 5-2 Message Types and Their Arguments Supported in UML_ADAPTER ...... 44

Table 5-3 Routing Table ...... 47

Table 5-4 MTU Table ...... 47

Table 5-5 Message Types and Their Arguments Supported in IP ...... 49

Table 5-6 Message Types and Their Arguments Supported in ICMP ...... 51

Table 5-7 Message Types and Their Arguments Supported in FRAGMENT...... 53

Table 5-8 ARP Table ...... 53

Table 5-9 Message Types and Their Arguments Supported in ARP ...... 54

Table 5-10 Queue Information in QOS Module ...... 58

Table 5-11 Message Types and Their Arguments Supported in QOS ...... 62

Table 6-1 Statistical Types ...... 67

Table 8-1 Hosts' Linux Version and Network Information ...... 72

Table 8-2 Maximum Loss-free Injection Rate: the GINI Router vs. UML ...... 80

Table 8-3 InternaI Overload Behaviour of the GINI Router...... 83

Table 8-4 Additional Overhead: the GINI Router ...... 88

Table 8-5 Comparison of Service Received by Packets with Different Priorities ...... 89

Vil List of Figures and Illustrations

Figure 1-1 An Illustration of GIN!...... 3

Figure 3-1 UML Kernel ...... 13

Figure 3-2 Network Scenario Using the GINI Switch ...... 15

Figure 3-3 Network Scenario Using the GINI Router ...... 17

Figure 3-4 Architecture ...... 20

Figure 3-5 Input Ports ...... 21

Figure 3-6 Output Ports ...... 22

Figure 4-1 Packet Flows in the GINI Router ...... 27

Figure 5-1 Main Routine ...... 35

Figure 5-2 Sendout Routine ...... 37

Figure 5-3 Message Sent Out by CLI ...... 39

Figure 5-4 Message Flows in UML_ADAPTER...... 43

Figure 5-5 urnCadaptecfromDevice Routine ...... 45

Figure 5-6 urnCadaptectoDevice Routine ...... 46

Figure 5-7 Message Flows in IP ...... 48

Figure 5-8 Message Flows in ICMP ...... 51

Figure 5-9 Message Flows in FRAGMENT ...... 52

Figure 5-10 Message Flows in ARP ...... 54

Figure 5-11 InternaI Structure ofQOS ...... 56

Figure 5-12 Message Flows in QOS ...... 61

V III Figure 8-1 Packet Delay: the GINI Router vs.UML...... 73

Figure 8-2 Packet Delay over Hops ...... 74

Figure 8-3 Actual Injection Rate by Client: GINI Switch vs. GINI Router vs. UML ... 77

Figure 8-4 Network Bandwidth: the GINI Switch vs. the GINI Router vs. UML ...... 78

Figure 8-5 Packet Loss: the GINI Switch vs. the GINI Router vs. UML...... 79

Figure 8-6 Maximum Injection Rate & Network Bandwidth: GINI Router vs.UML .... 81

Figure 8-7 Effect of the Number ofHops ...... 82

Figure 8-8 Delay Break Down to Modules in the GINI Router ...... 84

Figure 8-9 Config_J_2 ...... 90

Figure 8-10 Switch ...... 91

Figure 8-11 Mcast ...... 91

Figure 8-12 ConfiR-2_3,ConfiR-3_4,ConfiR- 4_5, ConfiR-5_6 ...... 92

Figure 8-13 ConfiR-x_x+ 1 ...... 93

Figure 8-14 ConfiR-2_3_2 ...... 94

Figure 8-15 ConfiR-1_4 ...... 95

IX List of Symbols, Abbreviations and Nomenclature

ARP Address Resolution Protocol

CU Command Line Interface

ICMP Internet Control Message Protocol

IGMP Internet Group Management Protocol

IP Internet Protocol

MTU Maximum Transmission Unit

OSPF Open Shortest Path First

QoS Quality of Service

RED Random Early Detect

RIP Routing Information Protocol

SNMP Simple Network Management Protocol

TCP Transmission Control Protocol

TOS Type of Service

UML User-Mode Linux

VNUML Virtual Network User-Mode Linux

WF2Q+ Worst Case Weighted Fair Queuing+

x Chapter One: Introduction

1.1 Motivation for the Thesis

Routers are machines that route Internet packets from hop to hop, so that two machines on different networks across geographical locations can communicate. Routers are the core part of the Internet, and an important part in the study and research on networking.

With the rising popularity of the Internet, the amount of the traffic transmitted over the networks has increased tremendously and will continue to do so as more network applications are being developed and deployed. As the gateway of networks, routers are facing the great challenge of providing higher performance and more capabilities.

Significant research effort has been put in developing faster and better routers. Various routing algorithms and different buffer management schemes have been proposed and studied. Moreover, as multimedia or real-time data begins to partake the Internet transmission, Quality of Service (QoS) [KuR02] functionality on routers becomes a desirable tèature.

There are two ways of implementing routers. One is to use special hardware, e.g. Cisco

Routers. Given that most of the research efforts don't involve hardware modification, it becomes very expensive and inconvenient to use them. The other method is to use software on computers, e.g. routing utilities provided by operating systems. However, in most cases, any modification of the routing operation would require changes made to ho st machine's OS kernel, and kernel implementation is significantly harder. For researchers, new development or ideas on routers take longer to implement. AIso, for most of the network programming courses, it is impossible to set up a router for each student to do an assignment on. So, students only get to learn the theory, but they do not have the opportunity to do any project on routers.

There are sorne software routers that allow limited modification in user-space, e.g.

VRIM [ApC02], Click [KohOl], VR [ChJ02] and XORP [HaH02]. However, aIl ofthem make changes to ho st machines' routing function in the kernel. As a result, they need root privileges and a dedicated computer for each router.

This thesis tries to address the problem by providing a platform to help in building a virtual network with many nodes (even hundreds) inside one or more Linux machines, without involving the investment and management complexity needed to create them using real equipment. Our new platform GINI emulates micro-scale Internet and runs the network elements that normally run in kernel space in user space, as illustrated in Figure

1-1. User-Mode Linux (UML) [Dik03] machine can be considered as a virtual Linux machine composed by many user space processes running on a real Linux machine. It can be run by non-root user; any modification within it does no harm to the host machine.

These merits make it perfect as a virtua1 end node. Other e1ements in GINI virtual network are routers, switches and links, which can also be emulated by programs running in the user space. One can easily connect those GINI elements together according to sorne topologies, which can be generated by using to01s like Tiers [Doa96] or GT-ITM

[CaD97]. The actual packet transport between the network elements in GINI is provided by the host's networking APIs such as socket calls. The packets traverse down the protocol stack in the source UML machine and transmitted by the host machine to the

2 destination UML machine where they traverse up the protocol stack. When the packets reach host machine, they are wrapped in aIl network headers and are in a "wire" format.

The host machine transports these wire format packets as data in sockets to the other

GINI network elements. Because the packets are in wire format, the GINI elements can operate on them with realism matching that happens in the actual Internet.

Host (Linux) Machine

UML Machine Other GINI UML Machine Elements User (Routers, Level Switches, links)

Kernel Level

Figure 1-1 An Illustration of GINI

By usmg GINI, students in a c1ass can implement and test their own routers independently and concurrently; researchers can use the p1atform as a quick too1 to do proof of concept. For instance, a new buffer management mechanism can be implemented, whose performance is compared with other existing management techniques.

3 The biggest contribution of the thesis is that it presents a barebone GINI router with a multi-threaded modular architecture, which allows thorough experimentation with router functionalities in an easy and co st-effective way.

GINI's design began with these principles:

1) Without dedicated machines and hardware: using a dedicated machine or

hardware for each router is not economical. It would be perfect if students can use

the computers in public lab and use our framework to build their own routers. It is

also economical for several routers running on the same machine.

2) Without root privilege: considering that students usually have no root privilege

to access OS kemel of computers in public computer labs, on our platform,

students are allowed to do any modification of the router within their user space.

In user space, programmers can debug their implementations easily using normal

user level debuggers. It is also easier to distribute and install modified routers and

forwarding daemons as normal user level program packages.

3) No harm to OS kernel's functionality: since public lab computers are multi­

purpose, kemel settings, especially networking configuration, should be kept

intact an the time. Any risk of damaging the host computers will be definitely

prevented by public lab administrators, but they would be happy to let researchers

use our safe platform.

4) Packets can go through a real network instead of a simulation program: a

simulation program can't exactly reflect the reality, it will be better to be able to

measure the performance of a real network.

4 5) Standards Compliant: it should confonn to RFC 1812 "Requirements for IP

Version 4 Routers" [Bak95], RFC 792 "Internet Control Message Protocol"

[Pos8I], RFC 791 "Internet Protoco1" [Pot81], RFC 2475 "An architecture for

differentiated services" [BIB98], and various others as necessary.

6) Extensible and Programmable: with extensible router architecture, researchers

and students can use our clear code framework and well-documented manual to

build their own customized routers easily. For example, different components of a

router can be replaced, modified and experienced with.

7) Lightweight: to create a large network with many hops, like the Internet, our

routers should not be CPU consuming. So that users can launch hundreds of

routers on each one among a limited number of computers.

8) Command Line Interface: It enab1es us ers to initialize routers, configure

routers while running, and check routers' internaI statistics for analysis and

diagnosis.

1.2 Thesis Oudine

We have designed and implemented the complete GINI experimental platform on top of our software router framework and User-Mode Linux (UML) [Dik03]. With the above principles in mind, we first examine sorne related projects in Chapter 2 that implement software routers and then introduce our own solution with its comparative advantages. In

Chapter 3, we will introduce User-Mode-Linux (UML), especially UML's networking.

The critical part of our work is to develop a virtual router to connect UMLs together. We

5 mention it as a virtual router because it is not a real hardware router; instead, it is a software router. Similarly, one can consider UML as a virtual machine instead of a real computer. Before our design, we explore what's inside a hardware router. After gaining solid background knowledge, we propose the design of our GINI router in Chapter 4.

The GINI router's implementation detail is discussed in Chapter 5. In Chapter 6, we de scribe the implementation of an internaI measurement facility in the GINI router. In

Chapter 7, we show the extensibility of the GINI router, and how to modify it with the assistance of templates. In Chapter 8, we analyze the results of well-designed experiments that use the GINI router and UML as routers. We compare their performance

(packet delay and link bandwidth) under normal and overload circumstances, on a variety of networks with different topologies. Chapter 9 is a conclusion of our CUITent work according to our evaluation, followed by directions to future works, which can be achieved based on GIN!.

6 Chapter Two: Related Work and GINI

The objective in this chapter is to briefly touch upon the most important contributions that are directly related to this thesis. This literature review includes software routers, and each work is presented together with a discussion of its advantage and disadvantage. Our solution for GINI is presented in the second section, by learning from others' strang points and introducing novel alternatives to offset their weaknesses.

2.1 Related Work

Since a hardware router is co st consummg, a software/virtual router, a pro gram emulating the function of a hardware router, is preferable for teaching purposes. There are a number of projects that develop software routers. We study sorne of them, especially those used for teaching and research purposes.

2.1.1 Virtual Router Improved

Virtual Router Improved (VRIM) [ApC02] has five components: Virtual Router

Clients, Virtual Router Servers, Firewall, Load balancer and Web/FTP/Application

Servers. It handles arbitrary topologies, which are specified by students from virtual router clients. The packets are actually routed and distributed across multiple physical servers. Its support of multiple clients allows students to use different topologies at the same time. However, it has three weaknesses: Students can not do anything other than

7 modifying the topology file; it needs dedicated physical machines; except of the vÎltual client, most components are not implemented totally in user space; it uses Libpcap/libnet to capture packets, which is suspected oflosing packets [ApC02].

2.1.2 Click

Click [KohOl] is much more extensible and configurable than VRIM [ApC02]. A

Click router is assembled from packet processing modules called elements, which are usually small and implement a simple functionality. As a result, it supports a wide variety of tasks through composition. Configurations are written in a declarative language that supports user-defined abstractions. With help of its optimization tools, it can achieve performance near the limits of high-end PC hardware. The Click kemel driver mns as a kemel thread under Linux 2.2, looping over the router's task queue and runs each task. It uses Linux /proc file system to communicate with its user processes. It also has sorne disadvantages. Since it has a kemel thread, the Linux kemel has to be patched for support of Click's polling device driver. As a result, the host machine's default routing won't work any more, which means, it needs dedicated machines too. In addition, it takes sorne time to leam its configuration language.

2.1.3 Virtual Router

Vitual Router - A User Space Router for the Linux Operating System [ChJ02] uses a modified Linux kemel to capture the packets relevant to the user space routers into a privileged de multiplexer, while the other packets are still passed through the usual kemel modules. To send the packet, the user space router retums the IP packet together with

8 next hop address information back to the privileged demultiplexer process that actually injects the packet back to the network. Again it has sorne disadvantages. Though the original ho st routing can handle irrelevant packets instead of being skipped as Click

[KohOl], its demultiplexer is still in kernel space; the system admin has to take effort to define rules to prevent vicious users to capture the packets of other users. In addition, their initial implementation of IP filter using Netfilter does not capture aIl packets that should be received by the kernel [ChJ02].

2.1.4 Extensible Open Router Platform & Zebra

Extensible Open Router Platform (XORP) [HaH02] & Zebra [Zab03] are different from other routing software that often only have one process that provides aIl of the routing protocol functionalities. They both use a multi-process architecture with one process per routing protocol, plus extra processes for management. This is appreciable because multi-processing provides a clearer modular structure and each module can be upgraded independently of the others. However, for a machine with a single processor, it cause much overhead. In that case, multi-threading will be a good alternative.

AlI of the above software routers have a common disadvantage, that is, they have part of their Implementation within the OS kernel space, which makes them useless for researchers or students using the public facility without root privileges.

2.1.5 An Implementation of a Teaching Network with User-Mode Linux & Virtual Network User-Mode Linux

An Implementation of a Teaching Network with User-Mode Linux: a Detailed Case

Study How-To [EwaOl] bypasses the above-mentioned weakness by using UML virtual 9 machines. In aIl, the virtual net consists of 20 virtual hosts sitting on 10 subnets connected via one virtual router to their campus network (and hence out to the Internet).

One of the virtual machines is used as a router and it can be configured as a firewall, effectively sandboxing the virtual network users within the virtual labo Building such a network, involving such subnetting is quite complex. Therefore, they have written sorne shell scripts to build the network almost automatically. The merits and shortcoming of it will be discussed later.

Virtual Network User-Mode Linux (VNUML) [VNU04] is a newly born virtualization tool. Its idea is almost the same as the above work using UML, but it is more convenient to modify configurations. The VNUML tool is made of two main components: the

VNUML language used for describing emulations in XML; and the interpreter of the language (VNUML command), that builds and manages the emulation, hiding aIl the complex details ofUML from users.

The advantage ofusing UML is obvious: one doesn't need root privilege to run UML.

Due to similarities between UML and Linux, students can play with the virtual machine and gain experience of a Linux kernel without the risk of damaging the host machine' s

OS. Because of this amazing advantage, we chose to use UML in our project as virtual end nodes at the edge of our virtual Internet.

However, the projects above confine themselves by using a TUN/TAP device to connect the UMLs, which needs root privileges. Furthermore, they use UML to act as a virtual router, which is very hard to modify. Though UML is an open source system, it is still too complicated for students to figure out how to modify its routing part. In addition, each UML consists of many processes. It is almost impossible to build a complex virtual

10 network with hundreds of UMLs. Because, as stated above, this reqmres lots of configuration work.

2.2 Elements of GINI

Our goal is to develop a useful tool that can be used to emulate general Linux based network scenarios. It is aimed to help in testing network applications and services over complex test beds made of many nodes using one or more Linux machines.

As illustrated in Figure 1-1, GINI is composed of virtual end nodes, networks and routers. UML has the merit of requiring no root privilege to launch it, and no risk of damaging ho st machine's setup, which makes it perfect for virtual end nodes in GIN!.

There are two differences between GINI and VNUML [VNU04]. Firstly, GINI doesn't use TUN/TAP that needs root privileges. There is an existing tool called "The Switch

Daemon", which can be used to conne ct UMLs by a non-root user. The GINI switch extends that tool to enable a virtual network to be created on several hosts. GINI doesn't use UMLs as routers to connect different virtual networks, for this approach has many drawbacks as stated in the above section. Instead, we develop our own GINI router in the user space to achieve all the princip les outlined in Chapter 1.

Il Chapter Three: Background Knowledge

We will present the design of the GINI router in the next chapter based on sorne essential background knowledge in this chapter. Since UML plays an important role in

GINI, it is necessary to have sorne fundamental knowledge of UML and especially of

UML networking. The first section describes how to use the GINI switch to conne ct multiple UMLs and how to use the GINI router to connect intermediate networks. As a software router, the GINI router resembles a hardware route but is not exactly the same.

A hardware router is illustrated in the second section. The difference between them is also discussed.

3.1 Overview of User-Mode Linux (UML)

3.1.1 What is UML?

UML is a safe secure way of running Linux versions and Linux processes. By using it, one can mn buggy software and experiment with new Linux kemels or distributions without risking ho st machine's Linux setup. UML provides a virtual machine that may have more virtual resources than the physical host computer. Disk storage for the virtual machine is entircly contained inside a single file on the host machine. It is also possible to make virtual machine only access selected hardware. With properly crafted access

12 limits, nothing perfonned on the virtual machine can change or damage the real computer, or its software.

How does UML work? Nonnally, the Linux Kemel talks straight to the hardware

(video card, keyboard, hard drives, etc), and any programs that run ask the kemel to operate the hardware. The UML Kemel is different. As illustrated in the Figure 3-1, instead of talking to the hardware, it talks to a 'real' Linux kemel (referred as the 'host kemel' trom now on), like any other program. Programs can then run inside User-Mode

Linux as if they were running under a nonnal kemel.

Process 2 Process 3

Process 1 User-Mode Linux

Linux Kemel

Hardware

Figure 3-1 UML Kernel

3.1.2 UML Networking

There are currently several transport types available for a UML virtual machine to exchange packets with other UMLs and hasts: Ethertap, TUN/TAP, Multicast, Switch

Daemon, Slip, Slirp, Pcap. Ethertap and TUN/TAP are mostly used to access the host machine's networking like in VNUML [VNU04], and they use tap devices on a host, which can only be brought up by a root user. The Switch Daemon and Multicast are used

13 for purely virtual networks and can be run by a non-root user. Slip, Slirp and Pcap are rarely used.

Among these transport types, we finally chose The Switch Daemon because of three reasons: it requires no root privilege; it is used to build purely virtual networks; it has better performance than Multicast.

3.1.3 The Switch Daemon & the GINI Switch

The Switch Daemon provides a mechanism for creating a totally virtual network. By default, it provides no connection to the ho st network. Though there is a way by attaching it to a tap device on the host, one can not use it as a non-root user. However, we don't want to confine ourselves to the scenario of running all UMLs on only one machine, for it is not scalable. As a result, we choose another tool that extends the Switch Daemon, called UML Switch2 [Ein03]. We prefer to calI it UML Virtual Switch to better describe its function and to be consistent with other naming. It allows two switches on different hosts to communicate with each other through a UDP tunnel, as shown in Figure 3-2. It is also configurable through a configuration file.

14 UML 1 UML2 UML3 UM~ UM~ UM~ 192.168.0.1-192.168.0.2 192.168.0.3 192.168.0.4 192.168.0.5 192.168.0.6 Host 1 Host 2

Figure 3-2 Network Scenario Using the GINI Switch

In order to setup virtual networks by using the GINI switch, the first thing to do is to mn the GINI switch. It can be mn as a background process on a host as follows:

umCvirtuaCswitch configurationJile

The configuration file can contain comments (starting with #; and ending at the line

end), strings (between Il characters), and \ to protect the next character. Cornrnands start at the beginning of a new line and may have an optional argument.

Use fuI commands implemented are listed below:

1) socket name Creates the UNIX domain socket for the User-Mode Linux

instances.

2) sockowner passwd-entry Change the ownership of the control socket to the

corresponding UNIX user.

3) port number Port number to listen for connections from the GINI switch

mnning on other hosts (and defining the port used for sending packets to other 15 hosts). The prograrn enforces sorne kind of security by only accepting packets

originating frorn the same port as it listens.

4) remote hostname Host name or IP address of remote hosts to send broadcast

Ethemet packets.

5) pidfile file Create a lock file containing the process-id of the running process.

6) logfile file Log all messages to this file.

7) hub Act as a hub instead of a switch.

8) fork Should be the last cornmand in the configuration file. Start polling for

network packets on the virtual switch. Has the effect of converting the pro gram to

adaemon.

9) debug --Increase debugging level by one.

Launch a UML with bring up its network interface, which is connected to the GINI switch. This is done by describing it on the 'linux' command line mentioned before. The general format is:

ethn=daemon, ethernet address, socket type, control socket, data socket

One can specify the Ethemet address if the one that will be assigned to it isn't acceptable for sorne reason. The rest of the arguments describe how to communicate with the switch. Socket type is usually 'unix'. Control socket should be the same as socket name specified in the backend switch's configuration file. Data socket can be left empty.

A step-by-step tutorial for how to setup a virtual network by using the GINI switch can be found at our UML rnanual in Appendix A.2 and A.5

16 3.1.4 Desired GINI Router

Though the GINI switch allows us to create a UML network easily ev en on several host machines, it is merely a switch/hub working at the data link layer that can only connect nodes within one network. To connect nodes on different networks, a router working at the network layer should be used. The scenario is shown in Figure 3-3 .The

UMLs are on different networks instead of on the same network as shown in Figure 3-2.

Here we use UML itself as a router just for illustration. As we have already discussed in

Chapter 2, using UML as a router has many drawbacks, so we have to develop our own

GINI router, which is supposed to replace the two 'umlr1' in Figure 3-3.

UDP Socket 1------port: 1107

Figure 3-3 Network Scenario Using the GINI Router

17 We don't simply modify the GINI switch to be our GINI router. Because the GINl switch has only one thread, which is different from our preferred multi-threaded structure.

So it is hard to extend.

In the above configuration, it is the GINI switch that connects everything including

UMLs and routers. Rence, our desired router should be able to communicate with switches. After going through the GINI switch's source code and several portions of the

UML code, we designed an API named VPL library, which can be used to connect to and exchange packets with the switch. It has been developed by my supervisor. The VPL

Library contains source code files (vpl.c/vpl.h) and examples (inject.c and retrieve.c). To use it, one can simply include its header file and use its routines.

To be weIl prepared for our design of the GINI router, we should explore what are the essential components inside a real router (usually hardware).

3.2 Hardware Router and the GINI Router

A router is an Intermediate System that operates at the network layer of the OSI reference model [KuR02]. Routers may be used to connect two or more IP networks, or an IP network to an Internet connection.

A hardware router consists of a computer with at least two network interface cards supporting the IP protocol [Pot81]. The router receives packets from each interface via a network interface and forwards the received packets to an appropriate output network interface. Received packets have aIllink layer protocol headers removed, and transmitted packets have a new link protocol header added prior to transmission.

18 The router uses the information held in the network layer header (i.e. IP header) to dccide whether to forward each rcceived packet, and which network interface to use to send the packet. Most packets are forwarded based on the packet's IP destination address, along with routing information held within the router in a routing table. Routers are often used to connect networks that use different types of links. The maximum packet lengths

(i.e. the maximum transmission unit (MTU) [MoD90]) are different for different types of network. A router may therefore use IP to provide segmentation of packets into a suitable size for transmission on a network. Before a packet is forwarded, the processor checks

MTU of the specified interface. Packets larger than the interface's MTU must be fragmented by the router into two or more smaller packets. If a packet is received which has the Don't Fragment (DF) bit set in the packet header, the packet is not fragmented, but instead discarded. In this case, an error message is retumed to the sender, informing it of the interface's MTU size.

The routing table lists known IP destination addresses with the appropriate network interface to be used to reach that destination. A default entry may be specified to be used for aH addresses not explicitly defined in the table.

A router forwards packets from one IP network to another IP network. It determines the IP network from the logical AND of an IP address with the associated sub network address mask. One exception to this mIe is when a router receives an IP packet to a network broadcast address. In this case, the router discards the packet. Forwarding broadcast packet can lead to severe storms of packets.

19 Associated protocois perfonn network error reporting (ICMP [Pos8I]), Ethernet addresses resolution (ARP [Plu82]) and remote monitoring of the router operation

(network management).

3.2.1 Hardware Router Architecture

A high level view of a generic router architecture [KuR02] is shown in the Figure 3-4.

Four components of a router can be identified, which are described in details below.

Input Port Output Port

• • • • Output Port

Figure 3-4 Architecture

Here is the explanation of what these components do:

1) Input Ports. The input port perfonns severai functions. lt pcrforms the physical

layer functionality (shown in Figure 3-5) ofterminating an incoming physicailink

to a router. It performs the data link layer functionality needed to interoperate 20 with the data link layer functionality on the other side of the incoming link. It also

performs a lookup and forwarding function so that a datagram forwarded into the

switch fabric of the router emerges at the appropriate output port. Control packets

(e.g., packets carrying routing protocol information such as RIP, OSPF or IGMP

[KuR02]) are forwarded from the input port to the routing processor. In practice,

multiple ports are often gathered together on a single line card within a router.

Data Processing Switch Line (protocols and TermiJlafion decapsulation) Fabric

Figure 3-5 Input Ports

2) Switch Fabric. The switch fabric connects the router's input ports to its output

ports. This switch fabric is completely contained within the router - a network

inside a network router! Switching can be via memory, via a bus or via an

interconnection network based on the hardware support.

3) Output Ports. An output port stores the datagrams that have been forwarded to it

through the switch fabric, and then transmits the datagrams on the outgoing link.

The output port thus performs the reverse data link and physical layer

functionality as the input port (shown Figure 3-6). A consequence of output port

queuing is that a packet scheduler at the output port must choose one packet

21 among those queued for transmission. This selection might be do ne on a simple

basis such as first-come-first-served (FCFS) scheduling, or a more sophisticated

scheduling discipline such as weighted fair queuing (WFQ) [BeZ97], which

shares the outgoing link "faidy" among the different end-to-end connections that

have packets queued for transmission. Packet scheduling plays a crucial role in

providing Quality of Service (QoS) [B1B98] guarantees.

Switch Data Processing Une --t~ (protocols and Termination Fabric decapsulation)

Figure 3-6 Output Ports

4) Routing Pro cess or. The routing processor executes the routing protocols (e.g.

Static-route, RIP, OSPF or IGMP), maintains the routing tables, and performs

network management functions (e.g. SNMP [KuR02]) within the router.

3.2.2 What Can be Implemented in the GINI Router?

To simulate hardware routers, our intention is to implement as much functionality as possible in the GINI router. However, sorne of them are not relevant to virtual routers connecting virtual hosts. For instance, we may not need line termination at the physical

22 layer, because both the GINI switch and the VPL library use an Ethernet packet tonnat.

This me ans the packets exchanged with the GINI switch are at virtual data link layer instead ofvirtual physicallayer. The reason why we call it 'virtual' data link layer is that the packet is sent at application layer actually. It is easier to understand if the packet is considered to be sent though a UDP tunnel. Though the packet has an Ethernet fonnat, this infonnation is only meaningful to nodes in our virtual network, not a real ho st machine. We also find the switch fabric unnecessary, because the switch can be done by copying a packet or its pointer from the input buffer to the output buffer within the software router.

As a result, the functions that should be implemented in the GINI router are:

1) Data link processing (Input & Output Ports)

2) Lookup, forwarding, queuing (Input Ports)

3) Queuing: buffer management (Output Ports)

4) Implementation of the routing protocols (Routing Processor)

5) Network management functions (Routing Processor)

Furthennore, GINI router is complying with the following Internetworking standards

(Requests For Comments (RFC)).

RFC 1812 "Requirements for IP Version 4 Routers" [Bak95]

RFC 792 "Internet Control Message Protocol (ICMP)" [Pos81]

RFC 791 "Internet Protocol (IP)" [Pot81]

RFC 2475 "An architecture for differentiated services" [BlB98]

RFC 1191 "Path MTU discovery" [MoD90]

23 Chapter Four: Design of the GINI Router

We started the design of the GINI router with a modular structure, dividing desired functionalities into modules. Then we explain the mechanisms to bring modules to work together. Packets of different types go through different modules in the GIN! router, which is illustrated by packet flows shown in Figure 4 -1.

4.1 Modular Structure

UVR is completely composed of modules. Currently implemented modules are CLI,

UML_ADAPTER, IP, ARP, ICMP, FRAGMENT, and QOS.

We divide the router into these modules because of the following reasons: firstly, each module is functionally independent so that it is easy to understand; secondly, the GINI router doesn't have too many small modules as is the case with 'components' in Click

[KohOl] in order to reduce configuration complexity; thirdly, the modules are not too big to keep them flexible for modification and extension.

Here is the description of each module.

1) CLI. The Command Line Interface module accepts the requests typed by user

on command line, and then sends the request to other modules. There are two

kinds of requests: execution and status requcsts. A status request tells the

destination module to report its status on the screen. An execution request asks the

module to change its own configuration. Plcase refer to the Section 5.4 for a

24 whole description of the commands. CU plays an essential role as a commander

in the router. CU can initialize the router either by a configuration file or

commands input by a user; under its supervision, aIl modules work properly and

gracefully together.

2) UML_ADAPTER. It creates connections and maintains their information; it

receives packets from or sends them out of the router. A connection, also

mentioned as an interface, can be considered as an Ethemet card on our virtual

router. Data link processing is done here. The UML_ADAPTER has several

threads: threads to receive packets from each connection and one more thread to

send packets to aIl connections.

3) IP. It processes IP packets, performs Checksum, TTL Expire or other checking,

looks up its Routing Table for the packet's next hop IP address and output

interface. Routing protocols can be implemented here; currently we only have

static-route.

4) ARP. It accepts ARP requests and sends ARP responses; it also resolves a

packet's next-hop IP address to MAC address.

5) ICMP. It replies ICMP requests destined to the router; it sends TTL Expire,

DF-Set and ICMP Redirect messages.

6) FRAGMENT. It divides a large packet into fragments, which are small enough

to be sent through the next-hop network.

7) QOS. If it is enabled (by defining QOS macro), aIl IP packets will be sent to it.

It implements buffer management. There are two threads within the module:

Classifier and Scheduler. Classifier is responsible for getting packets from the

25 module's FIFO Queue and classifying them into different internaI queues

according to their priorities (specified in TOS bit). If the destination module's

(UML_ADAPTER) FIFO Queue is not full, Scheduler will select an internaI

queue to get a packet from according to an algorithm (currently, we implement

Priority Queuing [KuR02] and Worst Case Weighted Fair Queuing + (WF2Q+)

[BeZ97]) and forward the packet to the UML_ADAPTER. Otherwise, Scheduler

will wait until there is space in the destination module's FIFO Queue.

4.2 How Modules Collaborate?

It will be easier to understand if we consider the GINI router as a factory, packets as products and modules as workrooms. Each module contains one or more threads, which can be considered as workmen. And each module has a FIFO Queue, which can be considered as an input box for products (packets). Workmen (threads) in one workroom

(module) always perform one kind oftask in collaboration - get a product (packet) from their room's input box (FIFO Queue), process it differently according to the label put by the last workman who processes it in another workroom (module), decide which room

(module) the product (packet) should to forwarded to, replace the label on it and put in into the destination room's (module) input box (FIFO Queue). Continue with another product (packet) by repeating the same things.

Figure 4 -1 illustrates packet flows between modules in the GINI router.

26 Packets From ,--- UVS 01

IP -1

1 1 r--- 1 1 1 1 1 1 o 1 (command 1 line interface)

1: [ FIFO 6~eu~ ] ('";\ 1 1 (unused now) 1 ï-~-..J :0 1r------1 ~I f'~o pyeu~ ~ ~~&~IF:O~§U~~u~r~ ~ E'~O :Q~eye ~ 1 1 I1 ____ J

1 ICMP FRAGMENT _...1 1 QOS , 1 L______--==-_-==-_-==-_-==-_-=_I ______,

Figure 4-1 Packet Flows in the GINI Router

AIl packets are received at UML_ADAPTER, and processed along different paths (as

Shown in Figure 4-1, presented by arrow headed lines of different patterns). Which path a packet will take depends on what the packet is. If a packet is not dropped by the router, it will finally go back to UML_ADAPTER and be sent to the proper GINI switch.

In Figure 4-1, the typical path is the path numbered 1: a normal IP packet, which needs to be forwarded by the router, is received by UML_ADAPTER, decapsulated (form a

27 data link layer packet) here; in IP module, sorne checks are performed and routing infonnation is found; in ARP, its Ethemet address of next-hop is resolved; finally, it is encapsulated and sent out by UML_ADAPTER.

If the packet is an ICMP request, it will be forwarded to ICMP module by IP

(numbered 2). If the packet is an ARP request, it will be sent directly to ARP module by

UML_ADAPTER (numbered 5). If the packet needs fragmentation, it will be forwarded to FRAGMENT by IP (numbered 3). If QoS is enabled, an IP packets will be sent to

QOS module for rearrangement according to their priority before forwarded to

UML_ADAPTER (numbered 4).

28 Chapter Five: Implementation of the GINI Router

We implemented the GINI router in a single process with multiple threads; each module has one or more threads to perform independent tasks. As a result, it is lightweight, compared to multi-process programs. The GINI router is written in C and each module has its own files. This overview of implementation includes the essential global data structures and common routines, which are used for modules to communicate with each other. In section 5.3, we explain the file organization of the GINI router's code.

Sections 5.4 to 5.10 explain the implementation ofindividual modules in the GINI router in details; in each section, we talked about what a module is used for, how messages of each type are processed within the module, what information is maintained and in what form it is stored, and how many threads are used and what are their tasks. Furthermore, there is a manual in section 5.4, which explains how to manipulate the GINI router through CLI.

5.1 Essential Global Data Structures

The important structures defined in 'uml_router.h', as shown below, are essential for communication between modules. Ail modules include this file within its own header file.

29 struct packet { struct ( unsigned char dst[61i unsigned char src[61i unsigned char prot[21i ) headeri unsigned char data[15001i Ji The 'packet' structure is also defined in the GINI switch, which is the actual format of packets going through our virtual network. It has an Ethemet format. 'dst' me ans the destination interface's MAC address. 'src' means the source interface's MAC address.

'prot' means the protocol (IP/ ARP). AlI the fields in 'header' are filIed in by

UML_ADAPTER, for it does the data link processing. 'data' may contain IP, ICMP or

ARP header, each filled in by the corresponding module.

30 struct messager struct{ int src_module_num; int dst_module_num; int hb; FLAG type; unsigned char arg[ARG_MAX_SIZE]; #ifdef MEASURE

struct timeval t; #endif } header; struct{

BYTE src_int_ip[4]; BYTE src_int_mac[6]; int dst_int_num; BYTE bNexthop[4]; #ifdef DEBUG struct timeval ttotal; int tf; #endif } packet_frame; struct packet sPacket; }; 'Message' is only for the router's interna1 use, which is the actual container going between modules. A packet is wrapped into a 'message' when it is received by

UML_ADAPTER. There are three sub-stmctures within 'message' structure: 'header',

'packeCframe' and 'packet'.

The 'header' structure includes the destination module's number and arguments need to be passed to that module. 'header' should be filled in every time before being forwarded 31 to another module, except the fields 'src_module_num' and 't'. 'src_module_num' specifies the sending module's number. 'dsCmodule_num' specifies the destination module's number. 'hb' indicates the priority of a message. If it is true, it will be sent to the destination module's high priority FIFO Queue, otherwise to the low priority queue.

ARP packets and those sent from CL! have high priority. An IP packets is originally set with low priority by UML_ADAPTER, and reset by IP according to the packet's TOS bits. 'type' indicates the message type. The destination module treats a message differently according to its type. If MEASURE is defined, 't' records time at the beginning of 'sendout' function, which is used for ca1culation in the destination module.

'PackeCframe' contains information carried by the message along, which will be used by certain modules. 'src_int_num' records input interface number, filled in by

UML_ADAPTER. 'src_inCip' records input interface's IP address, used by

ARP,IP,UML_ADAPTER, filled in by UML_ADAPTER. 'src_inCmac' records input interface's MAC address, used by ARP, filled in by UML_ADAPTER. 'dst_inCnum' indicates output interface number, used by UML_ADAPTER, filled in by IP/ARP.

'bNexthop' indicates output interface's IP address, used by ARP, filled in by IP. 'ttotal' and 'tf' 1S only used by UML_ADAPTER. IfDEBUG is defined, 'ttotaI' records the time when the packet is received to count the packet's total time in router. 'tf' is set to 1, only if 'ttotal' has a value set. With this, packets generated inside the router won't be counted.

32 struct module { pthread_mutex_t array_mutexi pthread_cond_t nonempty; void (*module threadJ (int *idJ i /*For high priority queue */ unsigned int hArrayMaxi struct message *hArray[MOD_ARRAY MAX]i int hWcount i int hCsize; /*For low priority queue*/ unsigned int lArrayMaxi struct message *lArray[MOD_ARRAY_MAX]i int lWcounti int lCsize; Ji These variables in a module can be accessed by other modules. They are initialized in function 'iniCmodule' of file 'uml_router.c'.

The following three variables are used for the buffer management of a module' s FIFO

Queues, which follows the 'multiple producers and single consumer' paradigm.

'array_mutex' is the mutex of a module's FIFO Queues. To read from or write to a module's queues, its mutex should be seized first. 'nonempty' is the condition of a module's queues. When a module finds its both queues are empty, it will release its mutex and wait for the condition to become true again. When another module puts something into the module's queues, it signaIs the condition. 'nonfull' is the other condition of a module's queues. It is true only if a module's both queues are full. Unlike 'nonempty', this condition is only used in the case described as following. Before QOS sends a packet to the proper FIFO queue of destination module (UML_ADAPTER), it checks this

33 condition of UML_ADAPTER queues. IF it is true, QOS will release its mutex and wait for this condition to become truc. When UML_ADAPTER gcls something out of its queues, it will signal the condition. Since other modules don't check this condition, they simply put packets into the queues oftheir destination module even if they are full.

'module_thread' is the start function of a module's main thread. It is defined in a module's own header file. If a module has more than one thread (e.g. QOS and

UML_ADAPTER), the other threads are created by this thread.

The following variables are related to a module's FIFO high priority queue.

'hArrayMax' is the maximum length of a module's high priority queue. 'hArray' is a module's high priority FIFO Queue. It stores message pointers instead of the messages themselves. 'hWcount' indicates where to write in a module's high priority queue, it is increased circularly (increase 1 mod hArrayMax) by other modules when they put messages. Correspondingly, the variable 'hRcount' is used to store where to read from the queue. It is defined in the module's own file, because only the module itself needs to read messages from its queue. It is increased circularly by the module.' hCsize' indicates the current length of a module' s high priority queue. The variables for low priority queue are similar to those ofhigh priority queue, so we don't repeat them.

5.2 Common Routines Shared by AIl Modules

Although the modules provide quite different functionalities, they have sorne similar routines. AlI modules share the same main routine, which is shown in Figure 5-1, the difference between them lies in the specific functions used for different types of messages.

34 -Initialize variables, -initialize measurement variables if MEASURE defined

False Seize mutex?

True

FIFO Queues empty? ~--True

-Release mutex False -Wait nonempty condition

igh priority True False ueue empty.?

-Release mutex -Gather statistics if MEASURE defined

True

essage MEASURE_REPORT MEASURE_CLEAN Type?

-Cali Other types -Cali prinCmeasureJeport() measure_init() -Other routines to display measurement to reset -Use sendout() to results measurement send the packet L_____ -=-~~ ____ ~ __

Figure 5-1 Main Routine

35 The code of this main routine is contained in file 'template.c'. The routine has an endless loop: first of an, a module's main thread blocks until it seizes the mutex of the module's FIFO Queues. Then it gets a message out of the module's high priority FIFO

Queue if the queue is not empty. Otherwise, it gets a message from the low priority queue.

Then it caUs the proper function according to the message's type (message type is stored in variable 'message.header.type'). In the called function, it processes the message, and then it either forwards the message to another module or discards the message. If the message should be forwarded, it refills the message's header, and caUs 'sendout' routine, which is illustrated in Figure 5-2. If both queues are empty, it releases the mutex and waits 'nonempty' condition.

36 Start

-Set src_module_num to itself -If defines MEASURE, count passing time, and reset header.t to current Ume

True False

False Seize the mutex 0 estination module?

True True

he high priorit The low priority queue of destination queue of destination module is full? module is full?

False False True True -Signal the nonempty -Signal the nonempty condition of condition of destination module destination module -Put the message into -Put the message the queue into the queue

-Display error -Packet thrown -Exit

easure >---True defined?

False

End

Figure 5-2 Sendout Routine

37 In 'sendout' routine, the module's mam thread selzes the mutex of destination module's FIFO Queues, and puts the message into the queue with the same priority as the message (message priority is stored in variable 'message.header.hb'), then signaIs

'nonempty' condition.

5.3 Organization of Files

The GINI router is written in C. Here is how the files are organized. Each module has its own code file '. c' and header file '.h'. The main function is in 'umLrouter.c', which initializes the router and creates the threads for modules and then tums itself to be CL!.

Global definitions are in 'umLrouter.h'. And VPL library files 'vpl.c' and 'vpl.h' are used. Finally, one can use the Makefile to obtain an executable pro gram 'umLrouter'.

To run the router, one may use a configuration file used to initialize it. Though the same commands can be input after the router is started, it is a good measure to put them into a file for reuse. There are also two template files ('template.c' and 'template.h') that will be very useful for people who want to modify the router. They contain the code necessary to create a module, which is compatible to our framework (Similar code is found in other modules except CL!) but without any other functionality. So that one can focus on the special part of the extended router.

38 5.4 CLI Module

As mentioned before, the CU module wraps all the modules and gives a seamless interface.

When the router is initialized, CLI starts a loop to execute the user's command inputs.

Sorne of them are used to print information; others are used to change the configuration.

When the user enters a command, CLI will create one or more control messages of a corresponding type, which are sent to other modules, as shown in Figure 5-3.

CONNLIS~ ~ARP_PRINT CONN_ADD ]L ADAPTID CONN_DEL - (QOS_QUEUE_LIST) -CONN_MTU QOS ~(QOS_QUEUE_WEIGHT) CLI ~ROUTE .PRIN~ ~(MEASURE_REPORT) ROUTE.ADD IP ~(MEASURE_CLEAN)~ ROUTE DEL ~MTU]RINT

Figure 5-3 Message Sent Out by CLI

Here is a list of all the commands in the GINI router. The type of the corresponding messages is mentioned in parenthesis.

1) No control messages:

a) read Process a list of commands from ; it has

the same effect as putting 'filename' as a parame ter when launching the

GINI router.

b) help Display the commands list.

c) quit Quit router

39 2) Control messages sent to UML_ADAPTER:

a) connection Print Conneetion Table to sere en (CONN_LIST).

b) connection add [mtu] Add an entry

speeified by the parameters to Conneetion Table (CONN_ADD). 'mtu' is

not mandatory. If 'mtu' is not speeified, it will be set by default to 1500.

e) connection delete Delete an entry speeified by

'eonneetion index', whieh is the ID of the connection to be deleted from the

Conneetion Table (CONN_DEL).

d) connection mtu Change the MTU for a eonnection,

this eommand will later be forwarded to the IP module, which stores the

MTU Table. 'index' is the ID of the eonnection whose MTU needs to be

changed in Connection Table (CONN_MTU).

3) Control messages sent to IP:

a) route Display Routing Table on screen (ROUTE_PRINT).

b) route add Add

the specified routing entry to Routing Table (ROUTE_ADD).

c) route dei ete Delete an entry from Routing Table, 'route

index' is the ID of the entry to be deleted (ROUTE_DELETE).

d) connection mtu Print MTU Table to screen. This command will be

forwarded to IP module, which stores the MTU Table (MTU_PRINT).

4) Control messages sent to ARP:

a) arp Display ARP Table on screen (ARP_PRINT)

5) Control messages sent to QOS:

40 a) qos [onloff) Tum on/off QOS module. If QoS is tumed off, IP packets

will bypass QOS and be sent directly to UML_ADAPTER, and they are

treated as low priority packets in aIl modules whatever their TOS values are.

The following commands are Effective only if WF2Q+ [BeZ97] is enabled by defining WFQ in 'umLrouter.h'

b) qos queue Display weight information Table for queues

c) qos queue set Change weight value for a queue,

'queue index' is ID of the queue whose weight needs to be changed

(QOS_QUEUE_WEIGHT).

6) Control messages sent to aIl modules other than CLI:

The following commands are effective only if measurement lS enabled by defining MEASURE in 'umLrouter.h'

a) measure report Make every module print its own statistics values to screen

(MEASURE_REPORT).

b) measure clean Make every module reset to its own statistics values

(MEASURE_CLEAN).

5.5 UML_ADAPTER Module

The UML_ADAPTER module is responsible for creating connections, and for receiving and sending packets. It has several threads: each connection/interface has a dedicated thread to receive packet one at a time, process it, and forward it to either the IP

41 or ARP module according the packet's type. There is another thread that sends a packet from the module's FIFO Queue to any interface. UML_ADAPTER also cncapsulates and decapsulates Ethemet packets.

It stores information about connections within the Connection Table. By usmg

'connection' command, we list this table with ail the currently available interfaces listed.

Interface ID IP address Mac address 1 192.168.3.1 FE:FD:OO:OO:OO:01 2 192.168.1.1 FE:FD:OO:OO:OO:02 ...

Table 5-1 Connection Table

Intemally, the information is stored in a linked list of interfaces.

struct interface { int ndi /* ID of the interface*/ BYTE bMacAddress[6]i/* MAC address of the interface */ BYTE bIPaddress[4]i /* IP address of the interface */ int iDescriptori/* file descriptor of the interface */ struct vpl_data sVPLdatai /* VPL library structure for the interface*/ struct interface *beforei /* point to previous interface in the chain*/ struct interface *nexti /* point to next interface in the chain*/ pthread_t threadIDi /* thread ID assigned to this interface*/ Ji

42 Figure 5-4 gives a briefview ofmessage flows going through the UML_ADAPTER:

------,~PACKET_IP~ uml_ virtual_swi tdi """>--~----,,-:------PACKET_ARP~ frornDevice

~-CONN_LIST ~ toDevice

,----CONN_ADD-­ ._ ..... _._--:.:::;. MTU_ADD~

T""'ll:-----CONN_DEL -----I~<,~:::::~~:;7"::~~~--_------­ ROUTE_DEL~ "'-----CONN _MTU--~/>«:------,------, __ " " ...... MTU_DEL~ ...... (MEASURE_REPORT)~ ...... -'-, ARP_DEL~

uml_virtual switch

Figure 5-4 Message Flows in UML_ADAPTER

Here a list of the message types and corresponding arguments supported in

43 Type Argument PACKET_UML_ADAPTER BYTE bMacAddr[6]; I*Packet to be sent out*1 I*Destination Ethernet address to be filled *1 BYTE prot[2}; I*Type to be filled in packet*1 CONN_LIST NIA I*Command to print ail interfaces' information*1 CONN_ADD char pcSocket[50]; 1* String name of socket file*1 I*Command to add a BYTE bMacAddr[6}; I*lnterface's MAC address*1 connection*1 BYTE bIPaddr[4}; I*lnterface's IP address*1 int mtu; I*lnterface's MTU*I CONN_DEL int nd; I*Command to delete a I*The number of the interface to delete*1 connection*1 CONN_MTU int nm; I*Command to change the I*The number of the interface to change*1 MTU value of a connection*1 int iMTU; I*new value of MTU *1

Table 5-2 Message Types and Their Arguments Supported in UML_ADAPTER

UML_ADAPTER has one thread for each connection to receive packets, whose routine is shown in Figure 5-5.

44 -Set thread cancel type to die as saon as cancelled

True

acket is destined to its own MAC address or broadcast? True False

-Generate a message and wrap the packet into it -Record information for the interface from which the packet is received

-Fill in header of the -Fi Il in header of the message message -Send the message to -Send the message ta IP module ARP module

Figure 5-5 urnCadapter_frornDevice Routine

However, there is only one thread in the UML_ADAPTER module to send an the packets in the FIFO queues no matter to what interface. Its routine is illustrated in Figure

5-6.

45 -Remove an interface from linked list -Send ROUTE_DEL & -Fil! in destination MAC address MTU_DEL messages to and protocol in Ethernet header IP of the -Cancel thread, close socket and free for the interfa interface False True -Send MTU_ADD message to IP -Fil! in source () -For each interface, MAC address 0 fil! in source MAC -Connect to socket in Ethernet address in Ethernet -Create a new header of the header and fil! in interface packet source IP and MAC 0 -Add the interface into I----.! -Send the address in ARP linked list packet -Send MTU_ADD message to IP -Create a thread the interface

Figure 5-6 umCadapter_toDevice Routine

5.6 IP Module

The IP module verifies the checksum and TTL expiry condition, and th en looks up routing information for an IP packet according to the Routing Table. 1t maintains two tables: Routing Table and MTU Table.

The Routing Table is used to store routing information for packets.

46 ID Network IP Addr Netmask Next-hop IP Addr Output Interface 0 192.168.3.0 255.255.255.0 0.0.0.0 1 1 192.168.1.0 255.255.255.0 192.168.3.1 1 ...

Table 5-3 Routing Table

The Routing Table consists ofroute entries as defined below: struct route_entry { FLAG fEntryStatus; /*indicate entry used or not*/ BYTE bNetwork[4]; /* Network IF address*/ BYTE bNetmask[4]; /* Netmask*/ BYTE bNexthop[4]; /* Next-hop IF address*/ int psInterface; /* output interface*/ }; The MIU Table contains the information ofMTU of different networks.

Interface ID MTU 1 500 2 1000 ...

Table 5-4 MTU Table

The MIU Table consists ofMTU entries as defined below: struct mtu entry{ FLAG mEntryStatus; /*indicate entry used or not*/ int int_num; /* interface ID*/ int mtu; /* MTU value*/ };

47 The message flows going through IP are illustrated and explained below.

- PACKET - IP - '~;~~::~:::------__ PACKET _ l CMP _ECIIO _REQUEST ------ROUTE_ADD \'~:;:~'<'-"''- '--'- PACKET_lCMP 3TL_EXPIRED~ ~ \\\ '-'"

\, ---- \ (ÇQ)----MTU]RINT ------.. \\\\""'", PACKET_ICMP_DFFLAG_SET~ -MTU_ADD IP \\,\\ --MTU_DEL ,PACKET]RAGMENT~AGMEî!D @D- (MEASURE_REPORT) \\ ~(MEASURE_CLEAN)

Figure 5-7 Message Flows in IP

Message Type Argument PACKET_IP NIA I*Packet that needs to be routed*1 ROUTE_PRINT NIA I*Command to print Routing Table*1 ROUTE_ADD BYTE pbNwork[4]; I*network address*1 I*Command to add a route BYTE pbNmask[4]; I*netmask address*1 into Routing Table*1 BYTE pbNhop[4]; I*next-hop IP address*1 int psi nt; /*output interface*1 ROUTE_DEL int num; I*Command to delete a route /*The number of entry to delete if the message from Routing Table*/ is from CLI. Or the interface number related with which the entries to be deleted, if it is from UML_ADAPTER*/

48 MTU_PRINT NIA I*Command to print MTU Table*1 MTU_ADD int pslnt; I*interface number*1 I*Command to add an entry int mtu; I*MTU value*1 into MTU Table*1 MTU_DEL int pslnt; I*interface number*1 I*Command to dei ete an entry from MTU T able*1

Table 5-5 Message Types and Their Arguments Supported in IP

The IP module has only one thread. Bere are the routines for different message types:

1. PACKET_IP, 'message.packeCframe.src_inCip' is required

If packet is destined to the router itself, caU process_self_ip yackeCroute 0

Ifpacket is an ICMP request, change message and send it to ICMP as

Else free message

Else caU route_ipyacketO

Ifpacket's TTL expired, change message and send it to the ICMP module

Else route packet

If packet's IP addresses of source and next-hop are on the same

network, create a new message and send it to ICMP as

Ifpacket's next-hop network's MTU is smaller than its size

49 If the DF flag is set in packet's IP header, which means the

packet can't be divided into fragments, change message and send it

to the ICMP module as PACKET_ICMP_DF _FLAG_SET

Else send it to FRAGMENT as PACKET_FRAGMENT

Eise send it to the ARP module as PACKET_ARP_OTHER

2. ROUTE_PRINT, calI route-printO to print the Routing Table to screen.

3. ROUTE_ADD, caU route_addO to add an entry to the Routing Table.

4. ROUTE_DEL

Iffrom CU, caU route_delO to delete an entry from the Routing Table.

Else if from UML_ADAPTER, caU route_deCint() to dei ete the entries related to a specifie interface from the Routing Table.

Else print 'error' on the screen.

5. MTU_PRINT, caU mtu-printO to print the MTU Table to screen.

6. MTU_ADD, caU mtu_addO to add an entry to the MTU Table.

7. MTU_DEL, caU mtu_deleteO to delete an entry from the MTU Table.

5.7 ICMP Module

The ICMP module creates the ICMP header for an ICMP packet, whose IP header is already filled in by IP module.

The message flows going through ICMP are iUustrated and explained below.

50 @-PACKELICMP_ECIIO_REQUEST~ ... ". @-PACKET_ICMP_TTL_EXPIRED ...... ~ ...... @-PACKET_ICMP_REDIRECT ~ ...... ~.~;~=:::;); @-PACKET_ICMP_DF]LAG_SET ...... -_ ...... -·_ .. @D- (MEASURE_REPORT) -----. @D-- (MEASURECLEAN)

Figure 5-8 Message Flows in ICMP

Message Type Argument PACKET_ICMP _ECHO_ int iLength; REQUEST I*The packet's data length*/ PACKET_ICMP _ TTL_ int iPrevBytes; EXPIREO I*Length of internet header and 64 bit data*/ BYTE bPrevBytes[MAX_IPREVLENGTH_ICMP] I*Buffer to store internet header and 64 bit data*/ PACKET_ICMP _REOIRECT BYTE pbGateway[4] I*New next-hop IP address*/ int iPrevBytes; I*Length of internet header and 64 bit data*/ BYTE bPrevBytes[MAX_1 PREVLENGTH_ICMP] I*Buffer to store internet header and 64 bit data*/ PACKET_ICMP _OF _FLAG_ int iPrevBytes; SET I*Length of internet header and 64 bit data*/ BYTE bPrevBytes[MAX_1 PREVLENGTH_ICMP] /*Buffer to store internet header and 64 bit data*/ int imtu; /*Next-hop MTU*/

Table 5-6 Message Types and Their Arguments Supported in ICMP

51 The ICMP module has only one thread. Here are the routines for different message types:

1. PACKET_ICMP_ECHO_REQUEST, caU icmp_sendechoreplyO, which changes the

message to 'ICMP Echo Reply' and sends it to ARP as PACKET_ARP_OTHER.

2. PACKET_ICMP_TTL_EXPlRED, caU icmpjendttlexpiredO, which changes the

message to 'ICMP TTL Expired' and sends it to ARP module as type

3. PACKET_ICMP_REDlRECT, caU icmp_sendredirectO,which changes the message

to 'ICMP Redirect' and sends it to ARP module as type PACKET_ARP_OTHER

4. PACKET_ICMP _DF _FLAG_SET, caU icmp_senddfflagsetO,which changes the

message to 'ICMP DF Flag Set' and sends it to ARP module as type

5.8 FRAGMENT Module

FRAGMENT module divides a large packet into smaller pieces, which are allowed to go through the next-hop network. The message flows going through the FRAGMENT module are illustrated and explained below.

@-PACKET_FRAGMENT~ "",., @D-(MEASURE_REPORT) ... " .... , .. "." .. , ... , PACKET_ARP _OTHER-Q.~V FRAGMENT CfQ)-(MEASURE__ CLEAN)~

Figure 5-9 Message Flows in FRAGMENT

52 Message Type Argument PACKET_FRAGMENT int iMTU; /*IP packet which needs ta be fragmented*/ /*Size of the fragments*/

Table 5-7 Message Types and Their Arguments Supported in FRAGMENT

FRAGMENT has only one thread. Here is the routine for the messages oftype

PACKET_FRAGMENT: caU ipJragmentO, which sends fragmented messages to ARP module as type PACKET_ARP_OTHER.

5.9 ARP Module

ARP automaticaUy updates ARP Table and uses it to resolve IP address to Mac address.

ID IP address MAC address .. - 1 192.168.3.27 FD:FE:OO:OO:OO:OO ...

Table 5-8 ARP Table

The ARP Table consists of ARP entries defined below: struct arp entry { FLAG fEntryStatusi /*indicate entry used or not*/ BYTE bIP [4] ;

BYTE bMac [6] i

} i

53 If a packet's next-hop IP address can't be resolved, it will be put into a buffer until its address can be resolved. The buffer is composed of several entries defined below: struct arp_buffer_entry { FLAG fEntryStatusi /*indicate entry used or not*/ struct message *sMessagei

} i The message flows going through ARP are illustrated and explained below.

Figure 5-10 Message Flows in ARP

Message Type Argument PACKET_ARP_OTHER NIA 1* IP packet whose address needs to be resolved *1 PACKET_ARP NIA I*ARP request or reply packet*1 ARP_PRINT NIA I*Command to print ARP Table*1 ARP_DEL BYTE pdAddr[4]; I*Command to delete the entries related to a I*Entries containing this IP interface in ARP Table*1 address should be deleted*J

Table 5-9 Message Types and Their Arguments Supported in ARP

54 ARP has only one thread. Here are the routines for different message types:

caU process_arp~acketO, In

which 'message.packeCframe.src_inCmac' and 'message.packeCframe.src_int_ip'

are required.

If it' s an 'ARP reply', send aIl buffered messages related to the resolved

address, free original message,

If gQos=l, send them to as

PACKET_UML_ADAPTER

Else send them to QOS as PACKET_QOS.

If it' s an 'ARP request', change it to 'ARP reply', send it to UML_ADAPTER,

asPACKET_ADAPTER

2. PACKET_ARP_OTHER, caU arp_resolve~acketO, which reqmres

'message.packet_frame.src_inCmac' and 'message.packeCframe.bNexthop'

If its next-hop IP address can't be resolved, buffer it, and create an 'ARP

request' message and send it to as

PACKET_UML_ADAPTER

If its next-hop IP address can be resolved

IfgQos=l, send it to UML_ADAPTER as PACKET_UML_ADAPTER

Else send it to QOS as PACKET_QOS.

3. ARP_PRINT, caU arp~rintO to print the ARP Table to screen.

4. ARP_DEL, cali arp_deleteO to deI ete an entry from ARP Table.

55 5.10 QOS Module

The QOS module implements Quality of Service (QoS) in the GINI router, which

rearranges packets according to their priorities. The module can be enabled or disabled

simply through a CU commando If it is enabled, aIl IP packets should pass it before

being sent by the UML_ADAPTER module. Otherwise, the module will be skipped.

5.10.1 Internal Structure

Figure 5-11 is an illustration of internaI structure of the QOS module.

ARf"'---H~ t--Ir--.UML ADAPTER

QOS

Figure 5-11 InternaI Structure of QOS

56 Here 1S the structure of each FIFO queue: struct queue{ struct message *q[QOS_MAX_QUEUE_SIZEJ; int q_wcounti int q_ rcount i int q_sizei #ifdef WFQ double q_sti double q_fti #endif Ji The queue stores message pointers. 'q_wcount' and 'q_rcount' indicates respectively where to write and read in the queue. 'q_size' is the current length of the queue. IfWorst

Case Weighted Fair Queuing + (WF2Q+) [BeZ97] is enabled by defining WFQ, 'q_st'/

'q_ft' is the starting / finishing time of the queue (explained in Section 5.10.2).

QOS has two threads: Classifier and Scheduler. The Classifier splits a single incoming stream of traffic into multiple outgoing ones within the router according to sorne pre­ defined rules, e.g. Type of Service (TOS) bits that is used in our implementation, source and destination IP address, etc. Multiple queues are available to store packets classified as belonging to different traffic. The Scheduler is responsible for scheduling the departure of packets from different queues based on same service criteria. Currently,

Worst Case Weighted Fair Queuing + (WF2Q+) [BeZ97] and Priority Queuing [KuR02] are implemented. Whenever the Scheduler wants to choose a packet to forward, it will check if the FlFO Queue of the packet's destination module (UML_ADAPTER) is full, it only forwards the pack et when that queue has sorne space. In this way, we control the output rate of QOS. Packets rnay be droppcd by Classifier when a queue is full. We

57 simply drop packets at the tail of queues, but there are alternative policies available, e.g.

Random Early Detection [KuR02].

TOS is the second byte in the IP header of a packet. Within this byte: bit 0 (lowest bit) for reserved, bit 1-4 for type of service, and bit 5-7 for precedence. Possible settings for type of service are: minimal cost: Ox02, reliability: Ox04, throughputOx08, 10w delay:OxlO. Possible settings for special precedence range from priority (Ox20) to net control (OxeO), only root user can use higher value. An example to set this value:

ping 192.168.3.27 -Q OxeO Iiset TOS precedence value to the highest

In our implementation, packets are classified according to their precedence values into different queues shown below.

TOS Queue ID Queue Priority Queue Weight (precedence) (Default) OxcO-OxeO 3 High 0.4 Ox80-0xaO 2 Medium 0.3 Ox40-0x60 1 Low 0.2 OxOO-Ox20 0 Best Effort 0.1

Table 5-10 Queue Information in QOS Module

Queue Weight is only used by WF2Q+. To use it, WFQ should be defined in file

'uml_router.h'. Otherwise, Priority Queuing [KuR02] will be used.

Priority Queuing algorithm always serves higher priority queue first. It is simple but not fair. Because packets in the lowest queue may suffer starvation if higher priority queues are always non-empty.

58 5.10.2 Worst Case Weighted Fair Queuing +

Faimess is an important criteria to consider when comparing scheduling algorithms.

Faimess Index (FI) is used to measure faimess properties of different algorithms in allocation services to any two sessions i and j, continuously backlogged during the interval (t1, t2). It is defined by Golestani as following:

Wi (tl, t2) represents the service received by session i during time interval (t1, t2) and ri is its reserved rate. The smaller this value is, the fairer an algorithm Is.

Unlike Priority Queuing [KuR02], Worst Case Fair Weighted Fair Queuing + (WF2Q+)

[BeZ97] is a fair scheduling algorithm. Compared to similar algorithms, Weighted Fair

Queuing and Worst Case Fair Weighted Fair Queuing [BeZ97], WF2Q+ has smaller FI and uses a new virtual time function, which achieves low complexity and high accuracy.

With WF2Q+, each queue is associated with a weight, such that the sum of the weights of aIl queues is no larger than a predefined value W, i.e., ifthere are n queues, then

weight[l] +weight[2]+ ... +weight[n]<=W

(ln our router, default is 0.4+0.3+0.2+0.1=1)

A queue's weight specifies how much share of the capacity of the output link a queue is entitled to receive. For example, if one queue has weight l, and another queue has weight 2, then the second queue will get twice as mu ch bandwidth as the first queue.

Each queue is associated two variables Sri] and F[i], that represent the starting time and the finishing time of the packet at the he ad of the queue of queue i.

Finally, a global variable V, called system virtual time, is associated to the system. 59 The WF2Q+ algorithm can be then described as foIlows:

Initialization: Set aIl variable: V, S[i], and F[i], for an queues i, to 0

Enqueue: Assume a packet of queue i arrives. Then execute the following pseudo

code:

enqueue (p, queue[i])i

if (queue[i] .size = =1) { //compute starting and finishing time for packet at the head of the queue of queue i S [il =max (V, F[i])i F [il =S[i] + length (p)/weight[ili //update system virtual time V=max (min {j} (S [j)), V)i //there should be at least one backlogged queue } Where min_ {j} (S[j]) represents the minimum of starting times among aIl backlogged queues. A queue is called backlogged if it has at least one packet in the queue. queue[i].size indicates how many packets are in the queue.

Dequeue: Consider the set S of all backlogged queues, such that their starting times are no larger than the system virtual time V, i.e., S[j] <=V, for any queue j in S. Then select the queue i in S that has the minimum finishing time F[i]. Finally, execute the following pseudo code:

60 pO= dequeue (queue[i]); Send (pO); p=head (queue[i]);

If (p) {

S [il =F [il ; F[i] =S[i] + length (p)/weight[i]; ) //update system virtual time if (there is at least one backlogged queue) V=max (min {j} (S[j]), V+ length (pO)/W); else V= V + length (pO) /W; If there is no packet to dequeue, which means there is no packet in aU queues, do initialization above in order to prevent virtual times becoming infinite.

IfW=C, where C represents the link capacity, then the weight of a queue represents the minimum bandwidth that the queue is guaranteed to receive.

5.10.3 Message Flows

The message flows going through the QOS module are iUustrated and explained below.

@>--PACKET_QOS--~ .. ".",., .... " ... ®--(QOS_QUEUE_LIST """ ~ QOS'''··,·· PACKET_UML_ADAPTER ~QOS_QUEUE_WEIGIIT-+ -+ ®-- (MEASURE_REPORT) QML_ADAPTED @D-- (MEASURE_CLEAN)

Figure 5-12 Message Flows in QOS

61 Message Type Argument PACKET_QOS NIA I*Packet to be arranged*1 QOS_ QUEUE_ WEIGHT int qn; I*queue number *1 I*Command to set queue weight*/ int iWeight; /*queue weight*1 QOS_ QUEUE_LIST NIA /*Command to print queue weight info*1 Table 5-11 Message Types and Their Arguments Supported in QOS

Here are the routines for different message types in QOS:

1. PACKET_QOS, caU qos_classificatonO, which puts it into a proper queue and

Schedu1er thread will send it to the UML_ADAPTER module as type

PACKET_UML_ADAPTER.

IfWfQ is enabled by defining WFQ:

2. QOS_QUEUE_WEIGHT, caU qos_seCqueue_weightO to set weight for a queue.

3. QOS_QUEUE_LIST, caU qos_queue_listO to display aB queues' weight information.

62 Chapter Six: Profiling Tool in the GINI Router

Measurement is a desirable feature provided by the GINI router, which reveals the run­ time information collected by individual modules. The code of this facility is included in templates, so that new modules can automatically include it. In addition, one can easily add more statistical types for whatever interests him.

6.1 Overview

To use internaI measurement facility, one has to define macro MEASURE in

'uml_router.h'. Measurement results are stored in statistical variables maintained by each module. Their values can be displayed by command MEASURE_REPORT, and be reset by MEASURE_CLEAN. Statistical variables of interest are given below. For aB variables the minimum, maximum and average values can be measured.

1) Travel time (in Ilsec) for messages of different statistical types

2) Length ofhigh priority queue

3) Length of low priority queue

4) Waiting time (in Ilsec) by the sender module to have a space in receiver's queue

plus queuing delay.

5) Waiting time (in Ilsec) by the sender module to have a space in receiver's queue

plus queuing delay.

63 Fundamental code for measurement facility is included in template files, so that a new module created by templates will automatically have this feature. Here is a description of the code:

1. Define statistical variables in header file

2. Define macro for statistical types in header file

3. Initialize these statistical variables

4. In endless loop, whenever getting out a message to process:

1. Count and update Waiting Time for high/low priority messages.

Waiting Time =Current time- Message 's time field (message.header.t)

11. Count and update high/low priority queue length.

5. Before using 'sendout' function to forward a message to another module, set the

function's argument 'ty' to a specifie statistical type

6. In 'sendout' function:

1. Save a message's time field (message.header.t) temporarily for measurement

within the module and update it to CUITent time for other modules to count.

11. Send the message

lll. Count and update Travel Time for messages of the statistical type specified.

Travel Time =Current time- Message 's time field (message.header.t)

7. Provide functions to display or reset statistical variables.

64 6.2 Statistical Type

Statistical type is used for counting Travel Time of different kinds of messages, for which it is useful to see the delay in each module. Statistical type is different from message type. Message type tells a module how to process the message, while statistical type tells the module to update the Travel Time variable for messages of the same kind.

Usually, one message type has one corresponding statistical type, e.g.

P ACKET_FRAFMENT and ST AT_PACKET _FRAGMENT. Sometimes, several message types may share one statistical type, if these messages' Travel Time is similar.

For example, aIl message types of ICMP are combined into one statistical type.

Sometimes, one message type may have several statistical types. For instance,

ARP_OTHER messages may experience quite different Travel Time, which depends whether the next-hop address of a message can be resolved or not. Obviously, if the address is not resolved, the message will be buffered and take much longer time going through the module. So it is better to divide this message type into two:

STAT_NON_BUFFERD_PACKET_ARP_OTHER and STAT

_BUFFERD_PACKET_ARP_OTHER.

One can easily add a statistical type by simply defining its macro in the header file.

To use it, one can pass it together with the message in the 'sendout' function and add a line of code in measure_report 0 to display it.

Here is a list of aH statistical types currently defined:

65 Statistical Type Description -- ARP STAT_NON_BUFFERD_PACKE /*PACKET_ARP _OTHER, which is not buffered T _ARP_ OTH ER_H and has a high priority*/ STAT_NON_BUFFERD_PACKE /*PACKET_ARP _ OTHER, which is not buffered T _ARP_ OTHER_L and has a low priority */ STAT_BUFFERD_PACKET_ARP /*PACKET_ARP _ OTHER, which is buffered and

- OTHER_H has a high priority */ STAT_BUFFERD_PACKET_ARP /*PACKET_ARP _OTHER, which is buffered and

- OTHER_L has a low priority */ STAT _PACKET _ARP /*PACKET_ARP, which is an ARP request*/ FRAGMENT STAT_PACKET_FRAGMENT_H /*PACKET_FRAGMENT, interval from the packet (high priority) being received to its last fragment being sent*/ STAT_PACKET _FRAGMENT_L /*PACKET_FRAGMENT, interval from the packet (Iow priority) being received to its last fragment being sent* / UML_ADAPTER STAT_PACKET _IN /*PACKET_ARP or PACKET_IP, created in the module*/ STAT_PACKET_OUT /*PACKET_ UML_ADAPTER*/ IP STAT_PACKET_IP_ TO_ICMP /*PACKET_IP, which is sent to ICMP*/

STAT_PACKET_IP_ TO- /*PACKET_IP, which is sent to FRAGMENT*/ FRAGMENT STAT_PACKET_IP_ TO_ARP /*PACKET_IP, which is forwarded to ARP*/ -- ICMP STAT_PACKET _ICMP _H /* Ali messages with high priority */ STAT_PACKET _ICMP _L /*AII messages with low priority*/

66 QOS 0 /*Message going through best effort queue */ 1 /*Message going through low priority queue*/ 2 /*Message going through medium priority queue*/ 3 /*Message going through high priority queue*/ _. Table 6-1 Statistical Types

67 Chapter SeveD: Customization of the GINI Router

The GINI Router is easy to extend because of its modular structure. This chapter shows how to add a new module or remove a module in the GINI router. Our templates facilitate creating a new module compliant to other modules in the GINI router and make the programming of arbitrary software router a pleasant exercise.

7.1 Add a New Module

Here are the steps to add a new module into the GINI router:

1. Create files for the new module by using templates:

1. Copy and rename 'template.c' and 'template.h' to the name ofnew module.

11. Replace 'xxx' and 'XXX' in template files with the name ofmodule.

2. Modify 'uml_router.h' (as indicated in the file):

1. Define the new module's macro.

11. Increase macro MODULE_NUM by 1

111. Define the message types supported in the module, which can be used by other

modules.

3. Modify 'umCrouter.c' (as indicated in the file):

1. Include the new module's header file.

11. Initialize the new module's variables, for instance, the module's high/low

priority queue's maximum length, its thread's start function, etc.

68 4. Modify the code of existing modules to redirect messages to the new module. Please

note that sending a message from one module to another is actually donc by copying

its pointer instead of the message itself. To avoid memory leaking, whenever a

module drops a message without forwarding it further, it must free the message. It

can forward a message to another module only if it knows for sure that the message

is recognizable by that module. In most cases, messages containing packets are

created by and freed by UML_ADAPTER module. Command messages are created

by CU, and freed by the modules receiving them.

7.2 Remove a Module

Currently implemented modules are essential ones, which are not supposed to be removed, especially CLI and UML_ADAPTER. Sometimes, in order to improve performance, one may not want messages to route through sorne modules or may want to strip down sorne facilities. One can use a global variable like the one we use for QOS module, which can be switched on and off by a CLI commando One can also use a global macro and use '#ifdef' to separate the code, e.g. we enable measurement by defining

MEASURE. If one decides to remove a module, he should follow the steps below, which is almost a reverse procedure of adding a new module:

1. Exclude the module's files.

11. Modify 'uml_router.h' (as indicated in the file);

a) Delete definition ofthe module's macro

b) Decrease MODULE_NUM by 1

69 c) Delete the message types merely supported by the module

111. Modify 'umCrouter.c' (as indicated in the file):

a) Delete inclusion of the module's header file.

b) Delete initialization code for the module's variables.

IV. Modify the code of modules to redirect messages to other module instead of the

deleted module.

70 Chapter Eight: Evaluation

This chapter evaluates the GINI router's performance, compared to that of UML as a router, on several network topologies/configurations. Experimentation inc1udes testing packet deIay, bandwidth and overload behaviour. The tools we used are Ping, Iperf

[TiQ99], and the GINI router's internaI measurement facility. Satisfyingly, we find that the GINI router is much more efficient than UML in both normal and overload situation.

We also compare the time a packet spends in each module, which shows that the

UML_ADAPTER module is the bottleneck in the GINI router. The fact validates the correctness of our choice to put the QOS module in front ofUML_ADAPTER. The GINI router provides sorne facilities, which may cause sorne additional overhead. Our test results show that CLI, measurement and QoS induce only little overhead. We also test

QoS functionality by using WF2Q+ [BeZ97] and find that the QOS module can successfully differentiate packets of different priorities by providing different portion of bandwidth to them.

8.1 Experimental Setup

The experimental setup consists of a total of two Intel PCs running multiple UMLs,

GINI switches and GINI routers in a local rcal network. Each machine has a 2.4GHZ

Intel® Pentium® 4 CPU, 512KB cache, 512 MB memory, 160 GB hard drive, and a 100

71 Mbits/sec Ethemet Controller. Their Linux version and network information is listed below:

Host 1 Host 2 Linux Version 2.4.22-10mdk (gcc 2.4.21-0.13mdk (gcc version 3.3.1 (Mandrake version 3.2.2 (Mandrake Linux 9.2 3.3.1-2mdk)) Linux 9.1 3.2.2-mdk» Host Name atom.mcgillplanetlab.org seyfert.mcgillplanetlab.org Interface Eth1 EthO IP Address 192.197.121.6 192.197.121.7 Netmask 255.255.255.224 255.255.255.224 MTU 1500 1500

Table 8-1 Hosts' Linux Version and Network Information

During each test, we adopt several topologies/configurations to set up virtual networks, where are aIl illustrated in Section 8.8. The first digit in a configuration's name is the number ofrouters; the second digit is the number ofswitches. For example, confi~L2 has one router and 2 switches. Among them, Confi~2_3_2 resides on both hosts, the others reside on Host 1. The UML version that we use is 2.4.19.5. The file system is stored in the file 'rooCfs.rh-7.2-full.pristine.20020312'.

8.2 Packet Delay

Figure 8-1 shows the round trip time (RTL) for a packet measured by Ping, on different configurations of networks. During this test, we build networks using these configurations (illustrated in Section 8.8) by increasing the number ofrouters from 1 to 5. n In each configuration, we use the GINI router and UML as router to compare their performance. The result represents the average of 30 pings. There is no ARP request, and the measurement and QoS are tumed off in the GINI router.

~~~~2.052

[SI mIL as Router o>=! Config_4_5 .Our Router .r-! +> (1j ~ Config_3_4 .r-! lH >=! (3 Confi g_2_3

o 0.5 1 1.5 2 2.5 rrt(msec)

Figure 8-1 Packet Delay: the GINI Router vs.UML

We also performed the same experiment on networks using the configurations 'Switch' and 'Mcast' (illustrated in Section 8.8). Their results are 0.315 msec and 0.594 msec respectively. So the RTT for a packet usmg one GINI router (inc1uding two GINI switches) is comparable to that using Mcast.

Figure 8-1 shows that, for an configurations, the RTT for a packet using the GINI routers is almost half of that using UMLs. Furthermore, the RRT for a packet,only increases approximately 0.1 msec by increasing one GINI router; while it increases more than 0.2 msec by increasing one UML. Since aU the other settings are the same except of

73 the routers, these figures reflect the performance gain of the GINI router over UML.

UML has multiple processes, which consume CPU resource to support other functionalities. However, the GINI router has a single process and takes off the extra burden taken by UML, hence it achieves better performance.

Figure 8-2 shows the RTT for a packet measured by Ping with varying number of routers. We launch tens of GINI routers on one computer by using Config_x_x+ 1

(illustrated in Section 8.8). The number of routers is varied from one to sixtY hops. Since the maximum TTL set in the GINI router is 64, we can only ping smaller than 64 hops.

We will be able to ping hundreds ofhops by increasing this number (up to 255). It is hard to test on UMLs this time, because it is not trivial to configure them as routers, and it is also impossible to run so many UMLs on one computer. Since the GINI routers and switches are configurable through configuration files, we can easily create aIl the files and launch them by a script. The other settings are the same as above. The only exception is that we tum off CLI. Since its loop to wait for user input consumes sorne CPu.

10 9 8

~ 7 (,,) (J,) (/J 6 El '--" 5 E- 0::: 4 0::: 3 2 1

1 10 20 30 40 50 60 l Number of Routers Figure 8-2 Packet Delay over Hops 74 The result is approximately linear. Ten GINI routers cost about 1.5 msec delay. But it incrcases a little slower (1 msec) from one router to 10 routers. This number conforms to what is shown in Figure 8-1.

As shown in Figure 8-1, the RTT for 60 hops is only 9 msec, which is really fast.

Imagine that a packet going through 60 hops may traverse the global world in real networks. However this configuration does not include cross traffic of different packet flows going through routers, which may cause additional packet delay. Furthermore, with so many routers launched, the CPU usage is as low as 30%.

According to our test results, a conventional PC is capable of running up to 150 GINI routers (90% CPU is user load and 10% CPU is system 10ad that is used to maneuver threads). In contrast, 10 UMLs cost 30% CPU. Based on the fact, we estimate a PC can run only 30 UMLs at most. Furthermore, UML takes long time (50sec) to start-up.

During this time, it consumes as high as 50% CPU. By contrast, the start-up time of the

GINI router is negligible.

We also test the RTT of a packet on a network ofusing configuration 'Confi!L2_3_2'

(illustrated in Section 8.8), which is on two hosts. In this case, packets go through a local network between two hosts. We create a virtuallink by connecting two GINI switches on different hosts with a UDP tunnel. There are three networks: 192.168.2.0 is the network including the virtuallink. Bach router acts as a gateway for the other two networks.

The pinging result is 1.33 msec, compared to the result (0.616 msec) of Confi!L2_3.

The higher delay is caused by the physica1 link between the hosts and one additional

GINl switch. Since a GINI switch causes about 0.315 msec delay, the physical link should cause 0.399 msec delay. This estimated number is validated by the result wh en we

75 ping directly from Host 1 to Host 2, which is 0.39 msec. The number is low because the physical1ink is on a lOO Mbits/sec local network.

8.3 Forwarding Rates

We characterize the performance of a router by measuring the rate at which it can forward 470-byte packets over a range of packet injection rates. We previously tried a combination of two network measurement tools: Packit [PacOO] and Nettimer [LaBOl], since the first is good for packet injection and the latter is useful to measure bandwidth.

Finally we choose Iperf [TiQ99], because it has both functions within the same tool, which makes it achieve better performance than using two tools. Iperf can be fUn in two modes: Client and Server. !ts Client injects packets, while its Server receives and calculates their information. Furthermore, its Client can create UDP streams of specified bandwidth; its Server can measure intermediate bandwidth, packet loss, delay jitter, and print periodic reports at specified intervals.

We compile Iperf on a UML to obtain an executable program, then fUn Iperf in Server mode on one UML and fUn Iperfin Client mode on another UML. For example:

[UML1] .jiperf -8 -u -1 470 [UML2] .jiperf -c 192.168.0.144 -u -b lm -1 470 On UMLI Iperf is fUn as Server to capture UDP packets of size 470 bytes sent to default interface (ethO); on UML2, Iperf is fUn as Client to send UDP packets of size 470 bytes to sever with a specified injection rate of 1 Mbits/sec.

Figure 8-3 shows the actual injection rate of packets by Iperf Client whcn varying the specified injection rate. Iperf Client calculates the actual injection rate of packets into a 76 network. In this experiment, we use configuration 'ConfiR-l_2' for the GINI router and

UML, and use configuration 'Switch' for THE GINI SWITCH (illustrated in 8.8). There is no ARP request, and measurement and QoS are tumed off in the GINI router.

Specified Injection Rate (Mbits/sec)

'------~~~~-~--~-~~~------~--~-~-~.~------

Figure 8-3 Actual Injection Rate by Client: GINI Switch vs. GINI Router vs. UML

Ideally, the actual injection rate should be equal to the specified injection rate, but in

Figure 8-3, we find this only happens when the specified injection rate is very low. This is because the actual injection rate is limited by the size of the UDP buffer at the Client.

The default size is 64. Client does generate packets at the rate specified, but it won't send them until the buffer has sorne space. If the rate at which a network gets packets is slower than the specified injection rate, the buffer will get full. That causes the actual injection rate of packets to be slower than the specified rate. Obviously, the actual injection rate is higher for a network that gets packets faster. Since the actual injection rate for the GINI router is always higher than that for UML, and they use the same topology, we can conclude the GINI router is more efficient than UML. 77 Figure 8-4 shows the network bandwidth vs. the actual injection rate of packets in the same experiment. The network bandwidth indicates the forwarding rate of a network, which is calculated by Iperf Server.

,------~.~-_._----- .... _--~-~------_._---~~ --._..

o o 5 10 15 Actual Injection Rate (Mbits/sec)

------_._-_.. _------

Figure 8-4 Network Bandwidth: the GINI Switch vs. the GINI Router vs. UML

Ideally, the network bandwidth should be equal to the actual injection rate of packets, which means the results should be linear. But Figure 8-4 shows that this only happens when the actual injection rate is low. The network bandwidth increases slower and slower as the actual injection rate rises. The reason is that the network is subject to packets loss, which will be discussed later. In Figure 8-4, the order from highest network bandwidth to lowest is the GINI switch, the GINI router, and UML. The result again shows that the GINI router is faster than UML.

Figure 8-5 shows the packet 10ss in the same experiment, which is counted by Iperf

Server. We find that the GINI router has less packet loss than UML. Packets may be dropped at the Server's UDP buffer or at the routers' buffers when they are full. Also, the

78 packets may take so long time to reach Server that they aren't counted. The faster the router processes packets, the less possibility that the above situation will occur. This result testifies that the GINI router is faster than UML.

70 60

cf2. 50 '-"----- (fJ 40 ---- (fJ --+- UVS 0 .....:J 30 \lit UVR +-"

Figure 8-5 Packet Loss: the GINI Switch vs. the GINI Router vs. UML

Table 8-2 shows the maximum loss-free injection rate. In this experiment, we vary the configurations (illustrated in Section 8.8) ofnetworks using the GINI router or UML. The maximum loss-free injection rate is defined as the maximum specified injection rate at which the packet loss is less than 1%. We find that the maximum loss-free injection rate of the GINI router is about 1.4 times higher than that of UML, which means the GINI router can normally carry heavier loads than UML. In addition, we have also tested the rate for the GINI switch in configuration 'Switch' (illustrated in Section 8.8). The result is 4 Mbits/sec, which is the same as that for the GINI router in 'Confi!S-1_2'. From this aspect, the GINI router is comparable to the GINI switch.

79 Configuration The GINI Router UML (Mbits/sec) (Mbits/sec) Config_1_2 4 2.7 Config_2_3 3.4 2.4 Config_3_4 3.3 2.4 Config_4_5 3.2 2.2 Config_5_6 2.8 2.1

Table 8-2 Maximum Loss-free Injection Rate: the GINI Router vs. UML

Figure 8-6 illustrates the maximum values of actual injection rate and network bandwidth in the same experiment. Those numbers give us an idea of the best performance of each network. Figure 8-6 shows that the best performance of the GINI router is still higher than that of UML. The maximum network bandwidth of one hop of the GINI router is 5.18 Mbits/sec, which is close to that of the GINI switch (5.99

Mbits/sec according to our test). The fact that the maximum network bandwidth is more than half of the maximum actual injection rate is true for the GINI router but not for

UML. The result indicates the GINI router is more efficient than UML. As the number of routers increases, aIl maximum values decrease. Because those values are related to a network's complexity. The more complex a network is, the worse performance it achieves.

80 Figure 8-6 Maximum Injection Rate & Network Bandwidth: GINI Router vs.UML

We perform another experiment for more hops of the GINI router, using configuration

'Config_x_x+ l' (illustrated in Section 8.8). The result is shown in Figure 8-7. Figure (a) shows the maximum actual injection rate and corresponding network bandwidth. Figure

(b) shows the maximum network bandwidth and corresponding actual injection rate.

81 -_.~~----~~- -~~~~~~----, a. Maximum Actual Injection Rate b. Maximum Network Bandwidth

12 r------~~-__. 7 r------~

-+-Actual --II- Actual 10 ~+------I 6 1--+----1 Injection Injection

Rate ~ 5 1---'\--\-----1 Rate Cl Networt Cl) -+-Network

o o 10 20 30 40 50 60 10 20 30 40 50 60 Number of Hops Number of Hops

Figure 8-7 Effect ofthe Number ofHops

In both figures, as the number ofhops increases, the margin between two lines is reduced. This happens because there is less packet loss when the actual injection rate is lower, and the actual injection rate is lower for a network with more hops. That also explains why Figure Cb) has a larger gap than Figure Ca).

8.4 InternaI Overload Behaviour of the GINI Router

When a network begins to drop packets due to a high actual injection rate, it is overloaded. We have already seen externally the overload behaviour of both GINI router and UML in the previous section, and find that the GINI router obtains better overload performance than UML. In this section, we use the measurement facility in the GINI router to see the GINI router's internaI overload behaviour. Table 8-3 shows the result for

82 a network using configuration 'Config_L2' (illustrated in Section 8.8) to forward 470- byte UDP packets at the specified injection rate of 10 Mbits/sec. measurement and QoS are turned on and there is no ARP request in this experiment.

Maximum Travel Maximum Packet Time (JI sec) Queue Length Dropped UML_ADAPTER 224195 49 0 IP 710 0 0 ARP 837 0 0 QOS 204812 50 486

Table 8-3 InternaI Overload Behaviour of the GINI Router

The Maximum Travel Time is the maximum time that a packet traverses in a module.

We find that this value is extremely high for UML_ADAPTER and QOS module, which means sorne packets experience quite a long delay in queues of the modules. We also find that both modules' FIFO Queues are full (queue capacity: 50) sometimes. The QOS module won't send any message to UML_ADAPTER module until the receiver's FIFO

Queue has sorne space. The fullness of the UML_ADAPTER module's FIFO Queue directly causes the overflow of QOS module's internaI queues, which makes the

Classifier thread in QOS module to drop 486 packets. Obviously, the UML_ADAPTER module is the bottleneck in the GINI router, which leads us to conc1ude that

UML_ADAPTER module is the bottleneck in the GINI router and placing QOS before it is a wise choice.

83 8.5 Delay Breakdown to Modules

Figure 8-8 breaks the packet delay of different kinds of messages down to modules by using Ping and measurement facility in the GINI router. The topology used lS

'Confi~I_2' (illustrated in Section 8.8). QoS, measurement and CU are hlrned on.

a. Regular IP packet 1II UMLj,DAPTER h.1P packet need ARP request -total: 60 usee 9 UMLj.DAPTER (Receiver) -total: 391usee -nt: 1.185 msee (receiver) -nt: 0.640 msee I2IIP I2IIP 1II2 1il19 o ARP o ARP

OQOS OQOS

ISIlThIT._ADAPTER 1II UML_ADAPTER (Sender) (sender) o 332 '---'----'---'

e.1P packet need fragmentation I;IUML_ADAPTER d.leMP eeho requestpacket 1;1 UML_ADAPTER -total: 127 usee (Receiver) -total: 88 usee (Receiver) -nt: 1.2~ msee ISIIP -nt: 0347 ~ee ~IP 15114 02T o FRAGMENT OICMP

o ARP o ARP 030 I2IQOS 9QOS

lIIUML_ADAPTER !SI UML_ADAPTER (Sender) (Sender) 022 nll

Figure 8-8 Delay Break Down to Modules in the GINI Router

Figure (a) is in the situation that a regular IP packet goes along the path:

Queue 0 (QOS) ~

STAT_PACKET_OUT (UML_ADAPTER). 84 Besides QOS, UML_ADATER costs most of the time, which shows again that UML

_ADAPTER (Sender thread) 1s the slowest place in the GINI router. On the contrary,

UML_ADAPTER (Receiver thread) is the fastest. The reason for it is that aH interfaces share only a single thread to send packets, while each interface has a thread to receive packets. AlI threads have the same chance to run. So more threads gain better performance. QOS also costs much because WF2Q+ [BeZ97] has a complexity of °(N).

The total cost to traverse a GINI router once is 60 Ilsec, compared with 640 Ilsec, of

RTT. This number does not include the queuing delay before a packet goes to the GINI router. Let U be the time spent at the end nodes of UMLs, S be the time to traverse a

GINI switch, R be the time to traverse a GINI router, C be the time going through a connection. We have the two equations below:

2U+ 2S+ 4C = 315 2U + 4S + 2R + 8C = 640 Since 315 and 640 are the RTT of a packet going through a network of using configuration 'Confi~1_2' and 'Switch' respectively. The solution of these equations is

R = 5+U. Since R is 60 Ilsec, U is 55 Ilsec and (S+2C) is 102.5 Ilsec. Briefly speaking, the packet delay mostly resides on switches and connections.

Figure (b) is in the situation that a packet whose next-hop address can't be resolved goes along the path:

STAT_PACKET_IN (UML_ADAPTER) -7

STAT_PACKET_IP_TO_ARP (lP) -7

STAT_BUFFERD_PACKET_ARP_OTHER_L(ARP)-7

Queue 0 (QOS) -7

STAT_PACKET_OUT (UML_ADAPTER)-7 85 The packet spends most of the time in ARP module, because it is buffered until its next-hop Ethemet address is resolved. Packets' average Travel Time (332 Ilsec) in the

ARP module includes the interval from sending ARP requests to receiving an ARP reply.

Figure (c) is in the situation that a packet requiring fragmentation goes along the path:

STAT_PACKET_IN (UML_ADAPTER)-7

STAT_PACKET_IP_TO_FRAGMENT (IP)-7

STAT_PACKET_FRAGMENT_L (FRAGMENT) -7

STAT_NON_BUFFERD_PACKET_ARP_OTHER_L(ARP)-7

Queue 0 (QOS) -7

STAT_PACKET_OUT (UML_ADAPTER) -7

Packets average Travel Time in the Fragment module is 30 Ilsec, which is counted from receiving a packet to sending its last fragment. A larger packet has longer Travel

Time. In this experiment, a packet is divided into 3 fragments.

Figure (d) is in the situation that an ICMP packet goes along the path:

STAT_PACKET_IN (UML_ADAPTER) -7

STAT_PACKET_IP_TO_ICMP (lP)-7

STAT_PACKET_ICMP_L (lCMP)-7

STAT_NON_BUFFERD_PACKET_ARP_OTHER_L(ARP)-7

Queue 0 (QOS) -7

STAT_PACKET_OUT (UML_ADAPTER)

Packets' average Travel Time in ICMP module is 13 Ilsec, which is shorter than that oflP, QOS and UML_ADAPTER (Sender) module.

86 8.6 Additional Overhead

Table 8-4 shows the overhead of using additional functions in the GINI router.

Here is a list of them:

{- CLI: it can be switched on by defining CLeON macro in the global header file

(umLrouter.h). Ifit is tumed off, there is no loop waiting for users' input.

{- Measurement: it can be switched on by defining MEASURE macro in the global

header file (umLrouter.h). If it is tumed off, no code of it will be executed.

{- QoS: it can be switched on/off during ron-time through a CU commando If it is

switched off, QOS module will be bypassed. AlI packets are treated as having the

same priority.

{- ARP request: it is implemented in the ARP module in the GINI router, but it only

occurs at the first time a packet is sent. After that, the information to resolve

addresses is stored in aIl GINI routers and UMLs along the path. Hence, other

packets having the same source and destination IP addresses don't need it anymore.

{- Fragmentation: it is implemented in the FRAGMENT module in the GINI router,

but Packets won't be forwarded to the module unless the size of packets is larger

than their next-hop network's MTU.

We use Config_L2. Overhead is counted as follows:

Overhead = (N-X) / N

Where N is the RTT of packets without using any ab ove functions, X is the RTT of packets using an individual function. We find that CU and Measurement cause very little overhead (less than 1%), so does QoS. This means one can use those use fuI facilities

87 without sacrificing much performance. Though ARP request and Fragmentation cause more overhead, they are not frequently used.

RTT (msec) Overhead (%) None 0.554 0 CU 0.566 0.022 Measurement 0.592 0.069 QoS (WFzQ+) 0.606 0.094 ARP request 1.052 0.899 Fragmentation 1.086 0.960

Table 8-4 Additional Overhead: the GINI Router

8.7 Differentiated Services

We set up a network of confiR-l_2 using the GINI router. Four Iperf Clients are launched on one UML to generate simultaneous packet flows to the other UML. Each flow sends 470-byte packets at the specified injection rate of 1 Mbits/sec for 2 minutes, and packets from each Iperf Client are assigned different priority values. Clients are run by a script as background processes. Though, it is impossible to make aIl Clients begin to send packets at the same time, we can reduce the effect caused by the order of running

Clients by running them for longer time (2 minutes). QoS and measurement are turned on, and WFQ is enabled. There is no ARP request. Table 8-5 shows the statistics counted both by Iperf and the GINI router' s internaI measurement.

88 WFZQ+ OxEO OxAO Ox60 OxOO ( Weight) (0.4) (0.3) (0.2) (0.1) Packet Loss (%) 35% 38% 39% 40% Network Bandwidth (Kbits/sec) 575 560 544 535 Packet Sent 28814 28296 28240 27484 Packets Dropped 0 27 418 880 Maximum Travel Time (lJsec) 18424 35356 69857 393863

Table 8-5 Comparison of Service Received by Packets with Different Priorities

Packet Loss and Network bandwidth are counted by Iperf Server. Packet Dropped,

Packet Sent and Maximum Travel Time are counted by QOS module in the GINI router.

Packet Sent indicates how many packets have been forwarded by the QOS module.

Packet Dropped means how many packets have been dropped by the Classifier thread because of the fullness of queues in QOS. Maximum Travel Time is the maximum time a packet spends in QOS. According to the table, the higher priority a packet has, the better service it receives. We find that the Maximum Travel Time ofbest-effort packets (OxOO) is abnormally high, because sorne packets wait too long to be treated, in order to give priority to higher priority packets. One may be curious about why packet loss is so high but not many packets are dropped in the GINI router. Actually, there are sorne other places causing packet loss: a packets maybe dropped at the receiving buffer of the Server, if it is full; a packet may be considered 10st, if it reaches the Server too late. The sum of aU Network Bandwidth values in Table 8-5 is 2.14 Mbits/sec, which is much less than the loss-free injection rate of the network (4 Mbits/sec according to Table 8-2). It is because of high packet 10S8. 89 The result seems not as what we expected, since the weights of queues from highest priority to lowest are 0.4, 0.3, 0.2 and O.l respectively. The Network Bandwidth of thc highest priority packets (OxEO) should be 4 times of that of best-effort packets (OxOO).

The reason why there is not much difference between them is that QoS is not implemented in the GINI switch. In 'ConfifCL2', the number of GINI switches is twice as many as that of the GINI router. If the GINI router is not much slower than the GINI switch, any differentiation in the GINI router won't have a significant effect. And this 1S true because Section 8.5 shows that the packet delay in the GINI router is only half as that in one GINI switch and two connections.

8.8 Configurations of Network oUML1 UML2 o

iiii~ Switch(umI1.ctl) Switch(umI2.ctl) lA MTU:500 MTÙ:1000

Router/UML 11

Figure 8-9 Config... J_2

90 UML6

ethO: 192.168.0.145 (FD:F :00:00:00:00)

ç;;:;t;:;:::;j'witch ( um 11 .cU) MTU:1500

UML7

Figure 8-10 Switch

UML6

ethO: 192.168.0.145 (FD:F :00:00:00:00)

UML7

Figure 8-11 Mcast

9l Figure 8-12 Confi~2_3,Confi~3_ 4,Confi~4_5, Confi~5_6

92 oUML1

~--+--"A Switch(umI1.ctl) MTU:500

ethO: 192.168.3.1 __~ (FD:FE:00:00:00:01) eth1: 192.168.4.1 (FD:FE:00:00:00:02) ~-+--.....,. Switch(umI4.ctl) MTÙ:1500 ethO: 192.168.4.2 11111 (FD:FE:00:00:00:01) eth1: 192.168.5.1 1 (FD:FE:00:00:00:02) ~tCh/Router P~

~----r-., Switch(umILctl) MTU:1500 ethO: 192.168.L2 ~j~ (FD:FE:00:00:00:01) I! eth1: 192.168.1.1 (FD:FE:00:00:00:02) mI2.ctl) :1000

ethO: 192.168.1.16 o (FD:FE:OO:OO:OO:OO)

UML2

93 Host 1 192.197.121.6

itch(umI1.ctl) ç;;;;t:;::;=:;:01 MTU:500

Router1

Router2

Host 2 UML5 1 7.121.7

94 UML1 UML2

ç;.;:;:;±'~:;ël Switch(umI1.ctl) ~~~JJ Switch(umI2.ctl) MTU:500 ~ MTU:1000

Switch(umI3.ctl) Switch(umI5.ctl) MTU:1500 MTU:1500

ethO: 10.10.0.25 ethO: 10.10.2.25 (FD:FE:OO:OO:OO:OO) (FD:FE:OO:OO:OO:OO) .....--le...:...... ,o .....--IL...... :....,o

UML3 UML5

Figure 8-15 Confi~l_4

95 Chapter Nine: Conclusion

9.1 Summary and Discüssion of Present Work

This work involves the design and implementation of GINI to build a virtual Linux network without root privilege. We are not aware of any prevlous such implementation.

As a core part, the GINI router is weIl designed and achieves aH of our goals. The most outstanding features of the GINI router are shown below:

1) Completely at the user level: it is a functional software router that can receive

and send packets in a virtual network correctly. Since nothing is hidden in lower

level, students have the freedom of modifying any part of the router. This

amazing advantage gives them a better understanding of routers.

2) High-Performance: our performance analysis shows that the GINI router

obtains lower packet delay, higher forwarding rate and less packet loss than UML

as a router in networks of the same topologies. And its overload performance is

also satisfying compared to UML. Its facilities like CLI and measurement are

very useful and only cause little overhead.

3) Scalable: it is light-weight so that the virtual network built from it is scalable up

to 150 routers on a conventional Pc.

4) Configurable: it has a set of commands, which can either be stored in a

configuration file to initialize a router, or input by user through CU after the

96 router is started. AIl configuration files can be created by a simple script, which

may be used to launch hundreds of routers.

5) Extensibility and Programmability: its clear modular structure is ideal for

extensibility. Students choose the module that interests them to irnplement, while

leave the other parts intact. Teachers can take out the codes of sorne parts and ask

students to implement them. AIso, students can add their new modules by simply

providing their own modules with the assistance of our templates and registering

them in the global files. Furthermore, our code framework is weIl documented.

Each module has a detailed description in its header file for other modules to refer.

6) Measurable: if the measurement facility is enabled, statistics are generated in

individual modules for debugging and performance testing. It can be displayed by

the modules though a CU commando Currently, we implement a few

general/specific statistic types in each module. More types can be easily added.

7) Quality of Service (QoS) [KuR02): QoS can be rarely found in other existing

software routers, but it is a desirable feature. In our router, we implement two

scheduling algorithms for QoS: Worst Case Weighted Fair Queuing + (WF2Q+)

[BeZ97] and Priority Queuing [KuR02]. Our evaluation shows that GINI router

can provide differentiated service to packets of different priorities. To make it

more effective, QoS should also be provided by other components of the network

(e.g. The GINI switch).

97 9.2 Future Work

There are sorne other interesting experiments that can be performed on GIN!. For instance, mapping virtual nodes on different physical nodes enables us to measure the performance of a real network and its components like a physlcallink or a device.

More modules can be added, for example, RIP and SNMP.

Furthermore, a visual central interface for GINI is desirable. Within the tool, one will be able to configure and launch aU the routers, switches, and end nodes of a virtual network and watch mn-time results of measurement though a GUI interface. Templates can be replaced by more powerful wizards, which makes extension even easier.

98 Appendix A - User-Mode Linux Mannal

In this manual, '[hast] #' indicates the command line on hast machine, while '[uml] #'

indicates the command line on User-Mode Linux virtual machine.

A.1 Installation & Preparation

It can be installed by using rpm, which is the easiest way but needs root privileges (in

Section A.1.I). Ifyou fail ta launch UML after installing its RPM, you have ta configure

and recompile the kernel of hast computer ta support UML (A.I.2). You can also choose

ta compile it from source code (A. 1.3), which needs no root privileges. After the

installation ofUML, you need a file system ta boat UML from (A.I.4).

A.1.I Download & Install UML RPM

This method need 'root' privileges.

1. Download installation file 'usecmode_linux-2.4.1 9.5um-O.i386.rpm' from

'http://user-mode-linux.sourforge.net/dl-sf.html'

2. Install this RPM.

[hast] # rpm -ivh user_made_linux-2.4.19.5um-O.i386.rpm

[hast] # rpm -ql user_made_linux-2.4.19.5um-O.i386

fusrlbin/jailtest

99 /usr/bin/linux <- executable binary that is really the UML kemel, the uml OS

/usrlbin/tunctl

/usr/bin/uml_mconsole

/usr/bin/umLmoo

/usr/bin/umLnet

/usrlbin/uml_switch

/usr/lib/uml/config

/usrllib/uml/modules-2.2. tar

/usr/lib/uml/modules-2.4. tar

/usr/lib/uml/port-helper

By installing this RPM, you have accomplished two things. First, you have installed

the executable kernel (/userlbin/linux), this is the UML Kemel. Second, you have

installed various UML utilities.

A.1.2 Configure & Recompile the Kernel of Host Computer to Support UML

This method again need 'root' privileges.

1. Download linux kemel file 'linux-2.4.19.tar.gz' from

'http://www.kernel.org/pub/linux/kemel/v2.4', extract it.

[hast} # tar zxvf linux-2.4.19.tar.gz

[hast} # cd linux-2.4.19

2. Configure the kerne1 options. This is where you select aIl the features you want to

compile into the kernel.

100 [hast] linux-2.4. /9# make xconfig

This command will pop up the configuration window. Under "Code maturity lcvel options", choose 'Yes' for "Prompt for development and/or incomplete code/drivers", this will allow you to choose [EXPERIMENTAL] options. Under "Block Deviees", choose

'Yes' for "Loopback device support". Under "File Systems", choose 'Yes' for "/dev file system support [EXPERIMENTAL)". Click "Save and Exit" to save your configuration.

3. Make dependencies. After saving your configuration above (it is stored in the

".config" file) you have to build the dependencies for your chosen configuration.

[hast] linux-2.4.19# make dep

4. Make the kemel. You can now compile the actual kemel. The resulting kemel file is

"archli386/boot/bzlmage"

[hast] linux-2.4.19# make hz/mage

5. Make the modules ..Modules are parts of the kemel that are loaded on the tIy, as they are needed. They are stored in individual files (e.g. ext3.0).

[hast] linux-2.4.19# make madules

6. Install the modules. You need 'root' priority to do it. This will copy aIl the modules to a new directory, "/lib/rnodules/a.b.c" where a.b.c is the kemel version

[hast] linux-2.4.19# make madules_install

In case you want to re-compile, you must also issue a couple "make" commands that cIean intermediate files. Note that "make mrproper" deletes your .config file.

101 A.1.3 Build UML Kernel from Source

UML may need sorne utilities, whose default path is /usr/lib. A good choice is to

change this default path to user's directory in its source code then compile a new kemel,

and distribute this kemel to students.

1. Make a directory and unpack the linux kemel into it

[hast] # cd /usr/src

[hast] /usr/src# mkdir uml

[hast] /usr/src# capy .llinux-2.4.19.tar.gz .luml

[hast] /usr/src# cd uml

[hast] /usr/src/uml# tar zxvflinux-2.4.19.tar.gz

2. Download uml kemel patch file 'uml-patch-2.4.l9-49.bz2' from

'http://sourceforge.net/project/showfiles.php?group_id=429" and save it to

'/usr/src/uml/linux-2.4.19'

3. Apply the patch.

[hast] /usr/src/uml# cd linux-2.4.19

[hast] /usr/src/uml/linux-2.4.19# cat uml-patch-2.4.19-49.bz2 / bunzip2 - / patch -pl

4. Configure UML kemel

[hast] /usr/src/umf/linux-2.4.19# make xcanfig ARCH=um

This command will pop up the configuration window, Click "Save and Exit" to save

your configuration

5. To finish with, the result is a file called 'linux' in the top directory ofyour source tree

[hast] /usr/src/uml/linux-2. 4.19# make linux ARCH=um

102 During compilation, if there is an error message "Id can't find le", that is because it

can't find the file 'libc.a' in folder '/usr/lib'. You can solve this problem by installing the

corresponding rpm. In our case, since we use mandrake 9.1, we install 'glibc-stactic­

devel'.

6. Make the modules

[host] /usr/src/uml/linux-2.4.19# make modules A RCH=um

With the keme1 not booted,

[host] /usrlsrc/uml/linux-2.4.19# mkdir mnt

7. Mount the root file system

[host] /usr/src/uml/linux-2.4.19# mount root-fs.rh-7.2-full.pristine.20020312 mnt-o

loop

8 Install modules in root file system

[host] /usr/src/uml/linux-2.4.19# make modules_install INSTALL MOD PATH= . /mnt

ARCH=um

4. Unmount the filesystem

[host] /usr/src/uml/linux-2.4.19# umount mnt

A.1.4 Prepare the Root File System

There are a number available for download from the UML website. There are also

several too1s, which can be used to generate UML-compatible file system images from

media.

103 1. Download root-filesystem file 'roocts.rh-7 .2-full.pristine.20020312.bz2' (or

'root_fs.rh-6.2-full.pristine.20020312.bz2')from

'http://sourceforge.net/project/showfiles.php?group_id=429'.

2. Extract it on command line:

[host) # bunzip2 root-fs.rh-7.2-full.pristine.20020312.bz2

A.2 Launch UML and Set up Its Network by Using the GINI Switch

In this section, we will launch one GINI swÏtch and two UMLs, and make them ping from each other.

1. Download 'umI_router' foIder from our server, please don't change the structure of

the folder; otherwise, you may encounter sorne problem.

2. Create a GINI switch instance for UMLs on the same host

A. Modify the file 'umLvirtuaI_switch1.conf:

[host} umCrouter # vi umCrouterlumCvirtuaCswitchl.conf

The content of'umI_virtuaI_switch1.conf Ïs shown below:

logfile .IumLvirtuaLswitch1.log

pidfile .IumLvirtuaLswitch1.pid

socket .Iumll.ctl

sockowner Weilng ---Change to your usemame

fork

B. Run executable file 'umLvirtuaLswitch' as backend for UMLs.

[host] # ./umCvirtuaCswitch umCvirtuaCswitchl.conf

104 C. Useful debug information is stored in 'umLvirtuaLswitchl.log', you can check it.

3. Run a UML with bring up a network interface ethO connected to the backended

switch. Make sure the file 'rooCfs.rh-7.2-full.pristine.20020312' is in the CUITent

directory, type:

[host} # Linux rootJs.rh-7.2-full.pristine.20020312 ethO=daemon" unix, ./umll. ctl

When you execute the 'linux' command, you should see in your terminal a new operating system booting up. You should hopefully end up with a login prompt. Just login with the user 'root' and the password 'root' giving you access to the operating system, it will ask you to change password, type the old password again, then type your new password twice.

Where 'fs_rooCrh7.2_cow' is the private copy-on-write (COW) file and 'rooCfs.rh-7.2- full.pristine.20020312' is the existing shared filesystem. The cow file need not exist. If it doesn't, the driver will create and initialize it. Once the COW file has been initialized, it can be used on its own on the command line: ubdO=roocfs_rh7.2_cow. Using COW files is the only way to share a filesystem between two UMLs. COW files are read-write, while the shared file is read-only.

Where we launched the command 'linux', specifically the part ethO=daemon. 'daemon' here is the GINI switch, which we have already ron before. Our goal is to build a virtual network by using a daemon program on the host that simulates a switchbox to the connected user-mode-linux instances.

4. Run another UML with its ethO connected to the same switch.

105 [host] # linux root-fi.rh-7.2-fitll.pristine.20020312 ethO=daemun" unix, ./umll. ctl

5. Since both UMLs use the same filesystem, they have the same IP/ MAC address (IP:

192.168.0.144/ MAC: FE: FD:OO:OO:OO:OO). You have to assign different values in

the second UML to make each UML has unique IP/MAC addresses. The IP address

should share the same network part, a good example is 192.168.0.145.

[umI2] # ifconfig ethO 192.168.0.145 up

[uml2]# ifconfig ethO hw ether FE: FD::00:00:00:01

6. Now, these two UMLs can talk to each other

[uml2} # ping 192.168.0.144 -ping from the second UML to the first one

A.3 Permanently Change and Save Network Configuration in COW File

Instead of use ifconfig each time, you can modify the script files to make yOUf change be effective upon future re-boot.

1. Change IP address.

[uml2} # vi /etclsysconfig/network-scripts/ifcfg-ethO

Here is an example:

DEVICE=ethO

ONBOOT=YES

IPADDR= 10.10.0.25

GATEWAY=1O.10.0.1

NETMASK=255.255.255.0

106 NETWORK= 10.10.0.0

BROADCAST= 10.1 0.0.255

MTU=1000

In this file, you can change its IP address, as weIl as gateway, MTU, etc.

2. Change static routing information.

[umI2] # vi /etclsysconfig/static-routes

Here 1S an example:

any net 192.168.1.0 netmask 255.255.255.0 gw 0.0.0.0 dey ethO

any net 10.1 0.2.0 netmask 255.255.255.0 gw 192.168.1.1 devethO

3. Ifyou want to make modified configuration effect without re-boot.

[uml}# /etclrc.d/init.d/network service restart

A.4 Shutdown UMLs and GINI Switches

1. Shut down an UMLs, in each UML's command hne, type:

[uml] # hait

2. Kill aIl Processes:

[host]# killall umCvirtuaCswitch

3. Delete switches' log files:

[hostlumCrouter# rm -1 *.Iog

107 A.5 Set up a Virtual Network for Two UMLs on Different Hosts

1. Create one GINI switch instance on each host

A. Modify the configuration files of switches.

[host IJ umCrouter # vi umCrouterlumCvirtuaCswitch4.conf

Other lines are the same as in Section A.2, two new lines are added:

port 1234 ---Change to any number unused and greater than 1000

remote 192.197.121.6 ---Change to the IP address of remote host which has

the UMLs you want to connect

[host2J umCrouter # vi umCrouter/umCvirtuaCswitch7.conf

Two new lines added is shown below:

port 1234 --- Should be same as above.

remote 192.197.121.7 ---Change to the IP address of remote host which has

the UMLs you want to connect

B. Launch switches as backend for UMLs.

[host1J # ./umCvirtuaCswitch umCvirtuaCswitch4.conf

[host2J # ./umCvirtuaCswitch umCvirtuaCswitch7.conf

2. Launch two UMLs with each connected to one switch.

[hostiJ # linux root-fi.rh-7. 2-full.pristine. 200203 J 2 ethO=daemon" unix, ./umI4. ctf

[host2J # linux root-fi.rh-7.2:full.pristine. 200203 J 2 ethO=daemon" unix, ./umI7. ctl

3. Change IP/MAC address in UML where necessary.

4. Ping from one UML to the other to see if it works. 108 A.6 Set up UML Network By Using Multicast

The simplest way to set up a virtual network between multiple UMLs is to use the

Mcast, which is illustrated in Section 8.8. Make sure the file 'root_fs.rh-7.2- full.pristine.20020312' is in the CUITent directory,

[hast} # linux ubdO=.fs cow_6, rootJs.rh-7.2-full.pristine.20020312 ethO=mcast

Similarly, you can run more UMLs, but you have to make each UML has unique

IPIMAC address.

A.7 Mount Host Machine Filesystem in UML

1. To begin with, make sure that hostfs is available inside the virtual machine with

[uml] # cat /proc(filesystems

hostfs should be listed.

2. Now all you need to do is run mount:

[uml] # mkdir /mnt/host

[uml] # mount none /mnt/host -t hostfs

A.8 Set up a Virtual Network with Multiple LANs by Using UML as Router

We use Config_L2, which is illustrated in Section 8.8.

109 'umI1.ctl' is a switch on network 1 (ex. 192.168.3.0), 'UML1' is a no de on the same

network; 'umI2.ctl' is a switch on network 2 (ex. 192.168.1.0), 'UML2' is a node on the

same network; 'UML Il' is a router to connect the two networks.

1. Modif)r configuration files and launch switches as backend.

[hast} umCrauter # ./umCvirtuaCswitch umCvirtuaCswitchl.canf

[hast} umCrouter # ./umCvirtuaCswitch umCvirtuaCswitch2.canf

2. Prepare COW files. Launch a UML instance of the COW file first, then modify

according to the procedure stated in section A.3 and the tables below:

A. Change script file' ifcfg-ethO' /'ifcfg-eth1 '

COW File Device IPADDR GATEWAY MTU ONBOOT Name (NETWORKIBROADCAST change accordingly) fs_cow_1 ethO 192.168.3.27 192.168.3.1 500 yes fs_cow_2 ethO 192.168.1.16 192.168.1.1 1000 Yes fs_cow_11 ethO 192.168.3.1 NIA 500 Yes eth1 192.168.1.1 NIA 1000 Yes

B. Empty the 'static-routes' files in 'fs_cow_l' and 'fs_cow_2', Change the file

'static-routes' in 'fs_cow_11' accord to the table below:

Destination Gateway Genmask Iface 192.168.3.0 0.0.0.0 255.255.255.0 ethO 192.168.1.0 0.0.0.0 255.255.255.0 eth1 127.0.0.0 0.0.0.0 255.0.0.0 10

3. Launch 2 UMLs as end nodes.

110 [host] umCrouter # linux ubdO=fs_cow_l,rootJs.rh-7.2-full.pristine.20020312 ethO=daemon" unix,. 1/um1. ct!

[host] umCrouter # linux ubdO=fs_ cow_2,rootJs.rh-7.2-full.pristine.20020312 ethO=daemon" unix,. 1/um2. ctl

4. Launch 1 UML as a router

[host] umCrouter # linux ubdO=fs_cow_11,rootJs.rh-7.2-full.pristine.20020312 ethO=daemon" unix,. 1/umll. ct! eth1 =daemon" unix,. 1/umI2. ctl

5. Change MAC address

[um111]# ifconfig ethO hw ether FE:FD:OO:OO:OO:01

[um111]# ifconfig eth1 hw ether FE:FD:OO:OO:OO:02

6. Enable ip_forward function in router UMLll.

[um111]# echo 1 > /procisys/net/ipv4/ipJorward

7. Ping from UMLI to UML2 to see ifit works.

III Appendix B - Bibliography

[ApC02] G. Appenzeller, M. Casado and V. Vijayaraghavan, "VRIM: The Virtual

Router Improved", Course Project Report, Stanford University, 2002,

http://klamath.stanford.edu/vns/vns.html.

[Bak95] F.Baker, "Requirements for IP Version 4 routers", RFC 1812, Internet

Engineering Task Force, September 1995, ftp://ftp.ietf.org/rfc/rfc1812.txt.

[BeZ97] Jon C.R. Bennett and H. Zhang, "Hierarchical Packet Fair Queuing

Algorithms", IEEE/A CM Transactions on Networking, 5(5): 675-689,Oct 1997.

[BlB98] S.Blake, D.Black, M.Carlson, E.Davies, Z.Wang, and W.Weiss, "An

architecture for differentiated services", RFC 2475, internet Engineering Task

Force, December 1998, ftp://ftp.ietf.org/rfc/rfc2475.txt.

[CaD97] K. Calvert, M. Doar, and E. Zegura, "Modeling Internet Topology", IEEE

Transactions on Communications, pages 160-163, December 1997.

[ChJ02] P. Chau and L. Jin, "Virtua1 Router- A User Space Router for the Linux

Operating System", Course Project Report, Stanford University, 2002,

http://www.stanford.edu/c1ass/cs344/staff/group4/.

[Dik03] J. Dike, "User-Mode Linux", Home Page, 2003, http://user-mode-

linux. source forge .net/.

[Doa96] M. Doar, "A Better Model for Generating Test Networks", Proc. of IEEE

GLOBECOM, Nov. 1996.

[Ein02] Z. Einstiegsseite, "UML Switch2". UML Switch2, Homepage, 2003,

112 http://www.uxu.ch/uxu/ software/umLswttch2.

[EwaOl] W. M. Ewan, "An implementation of a tcaching network with UML: A

detailed Case Study How-To", Technical Report, School ofComputing, CPIT,

New Zealand, 200 l, http://user-mode-linux.sourceforge.net/case-studies.html.

[HaH02] M. Handley, O. Hodson, E. Kohler, "XORP Goals and Architecture", XORP

Homepage, lCSl Center for Internet Research, Berkeley, California, 2002,

http://www.xorp.org/papers/xorp-hotnets.pdf

[KohOl] E. Kohler, "The Click Modular Router", PhD thesis, M.LT., 2001,

http://www .pdos.1cs.mit.edu/c1ick/.

[KuR02] J. F. Kurose, K.W. Ross, "Computer Networking: A Top-Down Approach

Featuring the Internet", Addison-Wesley, 2002, www.aw.com.cs.

[LaBOI] K. Lai, M. Baker, "Nettimer: A TooI for Measuring Bottleneck Link

Bandwidth", Technica1 Report, Stanford University, 2001,

http://mosquitonet.stanford.edu/~laik/projeets/nettimer/publications/usits2001

/main.html.

[MoD90] lMogu1, S.Deering, "Path MTU Diseovery", RFC 1191, Internet Engineering

Task Force, November 1990, ftp://ftp.ietforg/rfc/rfeI191.txt.

[PacOO] Free Software Foundation Ine., "Paekit- Network auditing too1", Paekit

Homepage, 2000, http://www.gnu.org/direetory/network/seeuritylPackit.html.

[P1u82] David C.P1ummer, "Ethernet Address Resolution Protocol: Or converting

network protoeol addresses to 48.bit Ethernet address for transmission on

Ethernet hardware", RFC 826, Internet Engineering Task Force, November

1982, ftp://ftp.ietforglrfc/rfc826.txt.,

113 [Pos8I] J. Postel, "Internet Control Message Protocol", RFC 792, Internet

Engineering Task Force, September 1981, ftp:l/ftp.ietf.org/rfc/rfc0792.txt.

[Pot81] J.Postel, "Internet Protocol", RFC 791, Internet Engineering Task Force,

September 1981, ftp://ftp.ietf.org/rfc/rfe0791.txt.

[Ti099] A. Tirumala, F. Qin, J. Dugan, 1. Ferguson, K. Gibbs, Iperf homepage,

University of Illinois, 1999, http://dast.nlanr.netlProjects/lperf/.

[VNU04] "Virtual Network User-Mode Linux", Telematies Engineering Department,

Teehnical University of Madrid, Spain, VNUML Homepage, 2004,

http://jungla.dit.upm.es/~vnuml/.

[Zeb03] IP Infusion Ine, "GNU Zebra- routing software", Homepage, 2003,

http://www.zebra.orgidocs.html.

114