Micro Transport Protocol (µTP) in Java
Ivan Iljkic Zürich, Switzerland Student ID: 06-920-854 – Communication Systems Group, Prof. Dr. Burkhard Stiller HESIS T
Supervisor: Daniel Dönni, Thomas Bocek, Andri Lareida ACHELOR Date of Submission: January 1, 2006 B
University of Zurich Department of Informatics (IFI) Binzmühlestrasse 14, CH-8050 Zürich, Switzerland ifi Bachelor Thesis Communication Systems Group (CSG) Department of Informatics (IFI) University of Zurich Binzmühlestrasse 14, CH-8050 Zürich, Switzerland URL: http://www.csg.uzh.ch/ Abstract
Transfer of large files like P2P file sharing traffic and software updates are less sensible to long delays and low bandwidths. But especially P2P file sharing clients gain an unfair advantage over applications like VoIP when managing multiple TCP connections. LED- BAT is an experimental algorithm that aims to manage background traffic in order to free up bandwidth for more interactive applications. An implementation of the algorithm is already in use in some P2P file sharing clients, known as µTP (Micro Transport Protocol).
This paper discusses UTP4J, an Java implementation of µTP for scientific projects. The goal is to provide an easy-to-integrate, configurable and stable library for experimental purposes. It covers the API, architecture and implementation details of UTP4J. Further, an evaluation of UTP4J’s behavior is given. This includes experiments with the congestion control mechanism during concurrent TCP traffic and ways to make µTP more aggressive under the influence of concurrent traffic.
i ii Zusammenfassung
Die Ubertragung¨ von grossen Dateien in P2P-File-Sharing-Applikationen und grossen Software Updates ist oftmals weniger sensibel was lange Delays und begrenzte Bandbre- iten angeht. Aber Speziell P2P-Clients erhalten einen unfairen Vorteil gegenuber¨ anderen Applikationen wie VoIP, wenn sie mehrere TCP-Verbindungen unterhalten. LEDBAT ist ein experimenteller Algorithmus, der sich zum Ziel gemacht hat, Background-Traffic zu managen, um gegebenenfalls mehr Bandbreite fur¨ interaktivere Applikationen freizugeben. Eine Implementation, bekannt als µTP (Micro Transport Protocol) des Algorithmus ist schon bei einigen P2P-Clients im Einsatz.
Diese Arbeit stellt UTP4J vor, eine auf Java basierende Implementation von µTP fur¨ wis- senschaftliche Zwecke. Das Ziel ist es, eine einfach zu integrierende, konfigurierbare und stabile Bibliothek fur¨ experimentelle Zwecke zur Verfugung¨ zu stellen. Das Dokument beschreibt die API, Architektur und Implementationsdetails von UTP4J. Desweiteren wird das Verhalten von UTP4J evaluiert. Dies umfasst experimente mit dem Staukon- trollmechanismus w¨ahrend nebenl¨aufigem TCP-Verkehr und Wege, UTP4J aggressiver zu machen.
iii iv Acknowledgments
I would like to thank Prof. Dr. Burkhard Stiller and the Communication Systems Group for giving me the possibility of working on such an exciting topic.
I would also like to thank my supervisors, Danni, Thomas and Andri for all the patience they had and all their valuable input which made this work possible.
v vi Contents
Abstract i
Zusammenfassung iii
Acknowledgments v
1 Introduction 1
1.1 Managing Background Traffic ...... 1
1.2 Buffering Delay ...... 2
1.3 LEDBAT Essentials ...... 3
1.4 µTP ...... 3
1.5 ThesisOutline...... 3
2 Related Work 5
3 Requirements 7
4 Design 9
4.1 API Design ...... 9
4.2 Architecture ...... 10
4.3 Design Decisions ...... 10
vii viii CONTENTS
5 Implementation 13
5.1 Connection Setup ...... 13
5.2 Writing Implementation ...... 13
5.3 Algorithm ...... 14
5.3.1 Waitingtime ...... 14
5.3.2 Retransmission Timeout ...... 15
5.3.3 Clock Drift Correction ...... 15
5.3.4 Configuration ...... 16
5.4 Reading Implementation ...... 16
5.5 ReceivingPackets...... 17
5.6 DataTypes ...... 17
5.7 Timestamping...... 17
6 Evaluation 19
6.1 Test Environment Setup ...... 19
6.2 TCP Reference ...... 20
6.3 OptimalParameters ...... 20
6.4 OptimalTargetDelay ...... 23
6.5 µTP competing with concurrent traffic ...... 26
6.6 µTP Aggresivity ...... 28
6.7 Compatibility to libutp ...... 29
7 Summary and Conclusions 31
8 Future Work 33
Bibliography 35
Abbreviations 37
List of Figures 37 CONTENTS ix
List of Tables 39
List of Listings 41
A Important Configuration Parameters 45
B Installation Guidelines 47
C Contents of the CD 49 x CONTENTS Chapter 1
Introduction
Internet traffic can be roughly categorized into two different groups. The first group is the so called foreground traffic and its Quality of Experience (QoE) is sensible to long delays and limited bandwidth. Such traffic is caused by interactive applications like VoIP, media streaming and online gaming. For example, the International Telecommunications Union recommends a 150ms one-way delay to be acceptable [1]. On the other hand, there is also traffic, that does not suffer from bad connections. This kind of traffic usually is not interactive and thus not sensitive to long delays or limited bandwidth. Traffic caused by P2P applications or large software updates belong into this group of background traffic.
1.1 Managing Background Traffic
TCP is the internet’s most common transmission protocol and seeks to share the available bandwidth evenly across all connections. However, one would want foreground traffic to be prioritized over background traffic and when there is no foreground traffic, the internet link should be still fully utilized. An even worse scenario is, when a background application, like a Bit-torrent client, manages multiple TCP connections. In this case, the Bit-torrent client would gain an unfair advantage over a more interactive application which only holds one TCP connection at a time and therefore its QoE would decrease. There are still options to limit the bandwidth utilization of a P2P client, but these settings require manual interaction each time when bandwidth is needed for more important applications and when the bandwidth can be fully used again. This solution is suboptimal [2]:
The user needs to configure his client after installing it
Knowledge about the available bandwidth is required for all networks
Figuring out the optimal numbers is difficult. A wrong guess may waste bandwidth or not be sufficient.
1 2 CHAPTER 1. INTRODUCTION
A mechanism to negate the disadvantages is desired. Ideally, it should all be done dynam- ically with a short response time. The mechanism should detect foreground traffic when it is present and reduce background traffic to the point where it no longer interferes with foreground traffic. On the opposite, when foreground traffic disappears, the mechanism should utilize the full available bandwidth.
A good way to detect the competitive traffic is to keep an eye on the congestion situ- ation in the network. Typically, network devices have a buffer where packets temporary reside until the device processes the packet. When the buffer is full, routers usually will start to drop incoming packets. TCP’s way to detect congestion is based on this principle. Every time TCP registers a packet was lost, it will halve its sending rate. However, at this point, the network is already congested. And after TCP has slowed down because of a lost packet, it still will start to gradually increase its sending rate until the next packet loss occurs due to a full buffer somewhere in the network.
1.2 Buffering Delay
Yet a better way to detect competitive traffic is to look at the buffer utilization and react before it is completely filled. This can be achieved by measuring the one-way delay of an existing connection. There is no point in interpreting a delay value without a reference. An application can open a connection which will be routed through an already congested part of the network. In this case, the delays will always be high and it is difficult to tell whether the network is congested or the delays are high because of an other reason. Finding a generally accepted reference value is also difficult, as there are transmissions mediums with different delay characteristics.
There are three factors which cause delays (δ) in a network transmission [3, 4]. The first one is the time the data spends traveling trough the transmission medium (τ). This can be calculated as the the length of the medium divided by the velocity the signal trav- els through the medium. The second is the routing delay (ρ). That is the time, a router spends to analyze the packet headers and figuring out where to forward the packet. And finally there is the buffering delay (β): the time, a packet spends waiting in buffers.
If all packets are routed through the same routers, τ and ρ will stay constant. The only influenceable part by a sender is the buffering delay β. If a router has to process more packages, β will increase and vice versa. The problem can be reduced now. In order to track the congestion situation of a network, the sender needs to periodically measure the one-way delay and find out the values for τ and ρ.
The LEDBAT algorithm [4] (for Low Extra Delay Background Transport) does this by keeping track of the minimum delay δmin. Assuming, that every once in a while, a packet will travel through the network without any buffering delay, LEDBAT concludes that this must be the base delay δbase = τ + ρ = δmin since β = 0. With this value, LEDBAT can estimate the buffering delay βn for each packet sent by comparing its delay with the base 1.3. LEDBAT ESSENTIALS 3 delay: βn = δn − δbase = (τ + ρ + βn) − (τ + ρ)
1.3 LEDBAT Essentials
LEDBAT uses, just like TCP, a congestion window (CWND) to control its send rate. CWND is defined as the amount of data, that the sender is allowed to send in one round cwnd trip time (RTT). Thus the send rate can be defined as sendrate = RTT . Each outgoing packet has an attached time stamp. The receiver compares the incoming time stamp with a local one and returns the difference in the acknowledgment (ACK). The sender inter- prets this difference as a delay measurement of the packet that was just ACKed. With the obtained data from the ACKs, LEDBAT can easily keep track of the base delay δbase and consequently compute the buffering delays from the packets it sends.
Since the send rate is directly proportional to the congestion window, it can be con- trolled by either shrinking or growing CWND. LEDBAT introduces a target buffering delay (Congestion Target) and tries to never experience a higher buffering delay than this target value. If the delays are below the target delay, the congestion window is increased and vice versa. b u f f e r i n g delay = delay − base delay o f f t a r g e t factor = (ctrgt − b u f f e r i n g delay)/ctrgt window factor = bytes acked /cwnd cwnd += max possible gain * window factor * o f f t a r g e t f a c t o r Listing 1.1: LEDBAT algorithm to calculate CWND
1.4 µTP
µTP [2] is an implementation of LEDBAT and a Bit Torrent standard. It is build on top UDP and therefore implements its own mechanisms to guarantee an ordered and reliable delivery of packets. µTP defines its own framing scheme too, which consists fields for sequence and ACK numbers, timestamps and an header extension system. There are two reference implementations in C/C++ [5, 3].
1.5 Thesis Outline
After an introduction to background traffic, delays and µTP, the related work to these topics is presented. The next chapter describes the functional and non-functional require- ments of the implementation, followed by an introduction to the API design. An overview of the architecture and the design decisions also belongs to that chapter. Afterwards, the implementation is explained in detail. In the following chapter, the testing setup is pre- sented and the performance of the library in comparison to TCP is discussed. After the summary, an overview of future work is given. 4 CHAPTER 1. INTRODUCTION Chapter 2
Related Work
Congestion control algorithms are widely researched. Standard TCP can use Explicit Congestion Notification (ECN) [17] to avoid congestion. The ECN flags can be set on a IP packet by routers before their buffers overflow. But this requires the ECN support of every node in the network. Some implementations of TCP use a delay based conges- tion control algorithm. TCP Santa Cruz [18] computes a sum of most recent one way delays to estimate the sum of packets queued at the bottleneck. TCP Vegas [19] measures RTTs and computes a theoretical throughput. If the actual throughput is less than the computed value, Vegas will reduce its send rate and vice versa. Both TCP modifications follow the same approach: avoid congestion but still be as fast as possible. LEDBAT [4] is also a delay based congestion algorithm. If a network is congested, LEDBAT tries to reduce its throughput until a delay threshold is reached.
The LEDBAT algorithm is published as an experimental RFC by the IETF. Some re- search on LEDBAT is available [20, 21, 22, 23, 24]. µTP, as specified in [2], implements the algorithm. There are software libraries (libutp [5] and libtorrent [3]) designed for the use in torrent clients written in C/C++. An experimental implementation in python [16] is available. Currently, only one Java implementation of the algorithm is known [25] but does not meet well established software quality standards.
5 6 CHAPTER 2. RELATED WORK Chapter 3
Requirements
The requirement of this thesis is to implement and evaluate a µTP [2] library in Java which meets well established software quality standards. Furthermore, the compatibility with the reference implementation in [5] must be assured. The implementation should meet severel non-functional requirements:
Documentation The source code must be well documented using Java Doc.
Unit Tests Unit tests must be implemented to assure the correct functionality of the code.
Publication The library must be available to the public.
7 8 CHAPTER 3. REQUIREMENTS Chapter 4
Design
4.1 API Design
Designing µTP sockets was considered, but dealing with sockets is cumbersome thus one of the main goals was to have a simple API. Java’s NIO seemed to be most suitable. Its abstract channels correspond very well to the concept of two connected end points and offer a set of methods for all desired operations with very clear semantics. Unlike the standard NIO channels, UTP4J’s channels won’t block the program flow but return future objects so the program can continue immediately. The use of abstract classes is necessary to hide implementation details. A programmer will only need to know about a hand full of classes to use UTP4J efficiently.
Just like Java’s NIO, UTP4J offers a server class UtpServerSocketChannel and a socket channel UtpSocketChannel. Both share a set of the well known operations open(), close() and bind(port) to assign a local port to the channel or respectively the server. Ad- ditionally, the server class requires an accept() method and the channel needs a con- nect(SocketAddress) method. Writing and reading operations require the programmer to pass a ByteBuer argument that contains the data to read from, respectively the memory space to write to.
Most of the methods return future objects to underline the asynchronous character of the library. But unlike the asynchronous NIO implementations that came with Java SE7, the future objects do not block the program when get() is invoked [11]. Its the choice of the programmer to either wait for completion by calling block() or do something else. The futures allow the programmer to probe them for completion with the invocation of isDone(). In case of a failed operation, the future objects can be used to retrieve the exception by calling getException().
Read futures also define a method setListener(UtpReadListener). UtpReadListener is an abstract type and can be inherit from by the programmer. After the reading operation is completed, the listener’s abstract method actionAfterReading() will be invoked. It is
9 10 CHAPTER 4. DESIGN possible for the listener to run in a separate thread if desired. This has the advantage, that listener operations which would take a long time wont block the channel.
4.2 Architecture
The implementation is separated from the interface classes in UTP4J. This allows to hide public implementation methods from the programmer. Each bound port will have its own instance of UtpRecieveRunnable. Its purpose is to listen on the socket for incoming pack- ets. The interface UtpPacketRecievable which is implemented by UtpSocketChannelImpl and UtpServerSocketChannelImpl defines a method that is used by UtpRecieveRunnable to pass incoming packets. The server and channel classes can both receive packets. This is important, since the programmer may open a channel directly, or open a server and wait for incoming SYN’s. If a channel was returned by a call to the servers accept() method, the server will keep a reference to the channel, since the channel wont have a reference to the UtpRecieveRunnable. All packet dispatching in this case is done by the server based on the connection ID’s in the µTP packet headers. When opening a channel directly, the channel will create it’s own instance of UtpRecieveRunnable.
Channel operations like write and read are implemented in separate classes. UtpWrit- ingRunnable handles the sending. A separate implementation of the the algorithm can be found in UtpAlgorithm. The algorithm can be configured by changing the values in UtpAlgCon guration. As UTP4J uses a sliding window for its flow control, there is a class OutPacketBuer that keeps track of the packets that are on fly and not yet ACKed. The handling of the delay samples and clock drift correction is implemented in MinimumDe- lay. Reading and ACKing of incoming data is implemented in UtpReadingRunnable and SkippedPacketBuer.
4.3 Design Decisions
µTP uses UDP to transmit its packets. So the first design decision that had to be made was regarding the question, which Java API to use to send UDP packets. The two options were: The UDP sockets from java.net [12] and the UDP channels from java.nio [13]. The choice was made to use UDP sockets because of performance reasons [6]. Put that aside, the library still would need to take payload data from the ByteBuer which is provided by the programmer, wrap the µTP headers around the payload, serialize the µTP packet and put it in a ByteBuer to call DatagramChannel.write(). Again, the DatagramChannel needs to take the µTP packet and wrap UDP headers around the µTP packet.
UTP4J should support blocking and non-blocking IO. In order to do this, the use of Java’s Selector API was considered [14] and inheritation from Java’s abstract channel classes. It is possible to develope an asynchronous library with the use of selectors. But 4.3. DESIGN DECISIONS 11 using Java’s selectors would require a custom SelectorProvider implementation. Unfor- tunately, this is not a easy task and the providers are platform dependent. If desired by programmers, custom selector providers can be set by system properties. However, configuring UTP4J with system properties was considered a bad idea since it is developed to be easy to use. It anyway would cause more harm than good in case a project needed a custom provider for its own reasons since it would require the developers implement yet another one which fulfills UTP4J’s and their own requirements.
As shown, it was not practicable to have an own selector provider. But due to the fact, that UTP4J was developed to be non-blocking per default and only offer an option to enable blocking mode, the idea was dropped. Instead, the library now provides future objects for a fully asynchrounus mode. 12 CHAPTER 4. DESIGN Chapter 5
Implementation
5.1 Connection Setup
When using UtpSocketChannel.connect() a SYN packet is sent to the specified address in the method arguments and a connect future object is returned. To handle a potential lost of the SYN packet, a separate Task is will be executed every second which will resend the SYN packet until the connection is set up. This task must be executed in a separate Thread in order not to block the program. Once the connection setup is finished, the Thread will be interrupted. Proper thread synchronization mechanisms are implemented, since the connection setup will be done by another Thread as described in section 5.5.
5.2 Writing Implementation
Since UTP4J is designed to be asynchronous, separate threads need to handle the IO operations. The thread UtpWritingRunnable handles a writing operation. It will also keep the the writing future updated. The thread will run in a loop and only stop under the following circumstances:
The writing operation successfully ends. This is the case when the ByteBuer is empty. However, it will not stop immediately, all packets must be acknowledged by the receiver.
The runnable detects a timeout. This is the case, when the last ACK that was received was too long ago. The timeout limit is specified in section 5.3.
It is gracefully interrupted by the user with a call to the UtpSocketchannel.close() method.
An IOException is thrown by the java.net components.
13 14 CHAPTER 5. IMPLEMENTATION
In all cases, the future object will be updated accordingly. Incoming ACK packets are stored in a queue (cf. section 5.5) and will be processed by the runnable and the the algorithm (cf. section 5.3). Considerations to have two different threads handling the writing operation were made. One thread was supposed to handle the ACK packets and update the algorithm accordingly, the other should query the algorithm whether there are packets to resend which timed out or if it can send the next packet. Such an implementation would require a very good synchronization of the threads and must be tested in order to avoid deadlocks and handle timeouts properly. Of course it is possible to write unit tests with the help of proper testing frameworks but the number of test cases would be large. Also, this would require to mock the class MicroSecondTimeStamp to behave like in real time program execution down to µs accuracy. The time to write tests to assure the correct functionality was estimated too high and the idea was dropped. Instead, the approach of a single thread was chosen. The writing runnable looks roughly as follow: while(writing) { while(algorithm .canSendNextPacket()) { p = getNextPacket() sendPacket(p) algorithm .markPacketOnFly(p) } waitAndProcessAcks (); List timedoutPackets = algorithm.getTimedOutPackets(); foreach(p : timedoutPackets) { sendPacket(p) } } Listing 5.1: Writing algorithm
Timestamps are required in the µTP specification [2]. They should be set just before the packet is serialized and passed to the UDP socket. When a packet was sent, the algorithm must be informed, that the packet is on fly, as it keeps track of RTTs, delays and manages the sliding window. To avoid a busy wait, the thread waits for incoming ACKs before it moves on with execution. This waiting time is specified in section 5.3. The purpose of this waiting time implementation is to reduce the CPU utilization and avoid large packet bursts that can not be processed as fast by routers. Some packets might get lost. These packets must be resend.
5.3 Algorithm
The algorithm manages all relevant data. It keeps track of RTTs, the RTO and buffering delays. With that data, it calculates the window size and thus the send rate.
5.3.1 Waiting time
The waiting time calculation for the write thread (cf. section 5.2) is done by the algorithm. The length of the waiting time depends on three factors. If there are some packets that 5.3. ALGORITHM 15 already timed out, the waiting time will be 0 in order to continue immediately and resend these packets. In the case that the window is full, the algorithm calculates the time to the next timeout. The write thread will continue either after that times has passed or an ACK arrives. If none of these two conditions is met, a predefined waiting time is returned (see section 5.3.4). The behavior is the same like in the case when the window is full: The thread waits until that time has passed or an ACK arrives.
5.3.2 Retransmission Timeout
The algorithm measures the round trip time for each packet. These values are used to predict future round trip times in order to be able to provide a good RTO. Since the round trip times may vary a lot, they must be smoothed and the variance must also be considered for the RTO:
r t t v a r += 0.25 * ( abs ( r t t − p a c k e t r t t ) − r t t v a r ) r t t += 0.125 * ( p a c k e t r t t − r t t ) r to = max(MIN RTO, rtt + rtt v a r * 4) Listing 5.2: Calculation of the retransmission timeout.
It is important, that only packets that were send only once are used to update the RTO since it is not possible to tell which packet was ACKed.
5.3.3 Clock Drift Correction
It is difficult to synchronize clocks down to µs accuracy on two different machines. How- ever, timestamps used in the packet headers are platform dependent and thus meaningless as an absolute value. They can only be used for comparison. Since the base delay is as- sumed to be constant anyway (cf. section 1.3), µTP uses the smallest difference returned by an ACK packet as the base delay. But when clocks run at different rates, µTP’s delay measurements will experience a systematic error. This phenomenon is called clock drift. There are only two possible situations: the local clock runs faster than the remote clock and the remote clock runs faster than the local clock. Luckily, only the latter is a problem since the faster local clock will just cause more frequent base delay updates and the drift will be corrected automatically by the smaller base delay. Faster remote clocks artificially increase the delay measurements and µTP will react to this by interpreting higher delays as a sign of congestion in the network. To correct for faster remote clocks, the receiver must timestamp the ACK packets too and the sender must calculate the base delay for the ACK packets on the return path. Whenever the sender registers an constant monotone increase in his delay measurements, it should check if the buffering delays from the ACK packets decrease by the same amount. If that is the case, the sender increases his base delay by the same amount the base delay from the receiver has decreased. 16 CHAPTER 5. IMPLEMENTATION
5.3.4 Configuration
The algorithm can be configured for experimenting purposes or to meet user requirements. An overview of the most important configuration parameters can be found in the appendix.
5.4 Reading Implementation
The read operation runs in its own thread. Incoming packets will be queued in a Block- ingQueue. Each packet is treated differently. A packet is considered expected if it arrives in order. An unexpected packet, is a packet that arrived out of order. When it is unex- pected, it will be buffered. The buffer is named in listing 5.3 inBuffer. Its responsibility is to manage packets that arrived out of order. An unexpected packet must trigger a selective ACK to notify the sender, that the packet was received, but some packets were expected to arrive before this one. An expected packet must not be buffered. Its payload can be put in to the ByteBuer that contains the payload and later returns to the appli- cation. If inBuffer is empty, the packet will be ACKed. If not, all packets that are now (with the new packet) correctly sequenced and gap-less will be retrieved from inBuffer and their payload (in correct order and gap-less) can be put in the ByteBuer. If inBuffer is now empty, a simple ACK will be returned indicating that all packets up to this ACK number arrived correctly. However, if there are still packets in the buffer, a selective ACK is needed. while(receiving) { p = queue.getPacket() if (isExpected(p)) { byteBuffer.put(p.payload) if (inBuffer.isEmpty()) { currentAckNumber = p.sequenceNumber ack(currentAckNumber) } e l s e { packets = inBuffer.getAllCorrectlySequencedPackets() foreach(s : packets) { currentAckNumber = s .sequenceNumber byteBuffer.put(s.payload) } if (inBuffer.isEmpty()) { ack(currentAckNumber) } e l s e { selectiveAck(currentAckNumber) } } } e l s e { if (alreadyAcked(p) { ack(currentAckNumber) } e l s e { inBuffer.bufferPacket(p) selectiveAck(currentAckNumber) } } 5.5. RECEIVING PACKETS 17
} Listing 5.3: Reading implementation.
5.5 Receiving Packets
To receive packets, UTP4J need a thread that listens on incoming packets on a UDP socket in a loop (UtpRecieveRunnable). Every time a packet arrives, it will pass this packet to an UtpPacketRecievable. As described in sections 5.2 and 5.4, there are separate threads that handle packets. Though, there are packets, that are not processed by these threads. The listener thread should return as soon as possible to his listening position. When passing a packet, the server, respectively the receiver, analyze the packet and decide what to do. This runs still in the listener thread. If it is an ACK packet or a data packet, it will be put in to a queue. The reader or the writer will consume the packet, similar to a producer/consumer scenario. However, SYN, RST and FIN packets will be handled in the listener thread, as they do not require a lot of processing time. This is a compromise between the listener thread’s responsibility only to pass packets and the need of an additional task to process the additional packets.
5.6 Data Types
Since Java does not support unsigned numeric types but µTP requires the header data to be unsigned, UTP4J must always convert all data that is put in the packet headers. There are many ways to do this. However, the implementation should allow for a fast conversion between signed Java friendly types and unsigned µTP data types. The µTP sequence number header field is a 16bit unsigned integer number. In order to process the sequence number correctly, UTP4J transforms the unsigned short in an Java integer type with bitwise AND and shift operators. When setting the sequence number in a packet header it must be transformed back to an unsigned short type. This can be achieved with a bitwise & 0xFFFF operator and typecasting to int, respectively to short. When serializing the µTP packet to a byte array (the serialized µTP packet is the payload of a Datagram packet), shift operators must be used. For the sake of code readability, unsigned types are always commented as such.
5.7 Timestamping
The preferable times tamp resolution is down to µs. Java offers two time stamp methods: System.currentTimeMillis() returns a signed 64bit integer with ms precision, and Sys- tem.nanoTime() returns a signed 64bit integer with ns precision, but the accuracy is not guaranteed to be down to ns. The documentation says, that System.nanoTime() should be only used to measure relative time, not absolute. 18 CHAPTER 5. IMPLEMENTATION
µTP timestamps should be 32bit long and the accuracy should go down to µs. µTP can handle arbitrary timestamps, as long as they are monotone growing (with the exception of numeric overflow). But UTP4J also needs some mechanism to measure the elapsed time (for timeout detection for example). There is also the need to implement a time stamp class. On initialization, MicroSecontsTimeStamp stores two timestamps: one initMillis of System.currentTimeMillis() and another one initNanos of System.nanoTime(). Whenever the library needs a time stamp, it just returns initMillis*1000 + (System.nanoTime() - initNanos)/1000. When the time stamp needs to be converted to an µTP time stamp, it is transformed to an unsigned type (see section 5.6). However, it must be guaranteed, that no overflow will occur. This is done by a modulo operation with MAX_UINT. Chapter 6
Evaluation
6.1 Test Environment Setup
All tests were conducted in the same test environment specified in this section. Figure 6.1 shows the setup. Two routers (devices 1 and 2) were connected by their WAN ports and served as bottlenecks. To each of the routers, two notebooks (devices 3, 4, 5 and 6) were connected by a 100Mbit LAN. All connections and data transfers went trough the bottleneck, this means, device 4 was never connected to device 3 and device 5 was never connected to device 6. For reference values with TCP the unix program netcat [9] was used. The operating system was the same on all notebooks, namely Ubuntu 12.04 with linux kernel 3.2.0-51. If not specified, the version of the source code was always 6e5319a5fb9e75e38abd33ad458e3cef125ddc79 from the UTP4J github repository [7] and the test file size was 142155468 bytes.
Device 1 ZyXel NBG-416N Device 2 Asus Wireless Router WL 500G Deluxe Device 3 Lenovo T61, 2GB Ram, Intel Core 2 Duo T9300 2 2.5Ghz ∗ 2 Device 4 Lenovo T61, 4GB Ram, Intel Core 2 Duo T7500 @ 2.2GHz ∗ 2 Device 5 Lenovo T61, 2GB Ram, Intel Core 2 Duo T7500 @ 2.2Ghz ∗ 2 Device 6 Lenovo T61, 2GB Ram, Intel Core 2 Duo T7500 @ 2.2GHz ∗ 2
The development of an automated test executor was practicable as it made it possible to run a large number of tests over night and collect the results afterwards. Algorithm parameters can be provided in a CSV file. The test executor will configure the algo- rithm according to these parameters. Logging of the CPU utilization and transmission rate was done automatically. To measure the CPU utilization, the return values of the method com.sun.management.OperatingSystemMXBean.getProcessCpuLoad() were av- eraged. During a test run, this method was called every 50ms. There is also an option to enable the logging of extended transmission statistics for all algorithm variables. All data in this section is based on these logs except for the tests conducted in section 6.2.
19 20 CHAPTER 6. EVALUATION
Device 4 Device 5
WAN Device 2
LAN LAN Device 1
Device 6 Device 3
Figure 6.1: Test environment setup
test case test case s kB/s s kB/s 59.22 3328 60.42 3262 61.09 3226 59.31 3323 59.94 3288 58.01 3397 59.67 3303 58.66 3359 60.22 3272 59.29 3324
Table 6.1: Measurement of TCP transmission speed in the testing environment.
6.2 TCP Reference
In order to compare UTP4J’s performance with TCP, reference test have been conducted. The program used to test TCP’s performance in the test environment was netcat. A file was transmitted 10 times from device 4 to device 5. The file size was 201822192 Bytes. Every transmission had its time measured with the unix command $ time `netcat targetip targetport < lename`. On device 5, the command $ netcat -l localport > /dev/null/ was used to receive the file. The average transmission rate is 3308kB/s with a σ of 49kB/s. Table 6.2 shows the results.
6.3 Optimal Parameters
During the implementation phase, UTP4J was slower than netcat, so exploratory experi- ments were conducted to figure out the reasons. Using netcat with the option -u causes 6.3. OPTIMAL PARAMETERS 21
rate [kB/s] CPU [%] Burst / Wait avg σ avg σ 5 / 8ms 3533.5 191.6 23.89 0.11 5 / 10ms 3571.7 113.0 23.94 0.38 5 / 12ms 3579.2; 82.6 23.92 0.56 5 / 28ms 3551.5 78.3 23.89 0.34
Table 6.2: More detailed comparison of feasible parameters.
the program to transmit the data with UDP. Surprisingly, the transmitted file’s size on device 5 was less than 40% of the file size on device 4. Further investigation with iptraf [8] (a network admin tool for unix) showed an outgoing rate of almost 90000Mbit/s on the Ethernet port of device 4. Almost 60% of the data was dropped during the transmission because the routers could not process packets as fast as device 4 sent them. A theoretical analysis of the algorithm in 5.1 has shown, that the algorithm will send packets until its window is filled. While sending, ACK packets will be received and stored in the packet queue. As soon as the window was full, the program left the send loop and processed the ACK packets one by one. Each ACK packet increased the window and the amount of bytes on the wire was reduced. The algorithm could send even more data during the next execution of the send loop since it benefited from the larger window. Locally, the send rate was very high and caused the routers to drop packets. To work around this, a mechanism was implemented to reduce the time the algorithm spends in that loop to smoothen the send rate (cf. section 5.3).
The new parameters had to be evaluated experimentally. Different test configurations of the algorithm were defined in a CSV file. Each test was conducted 10 times. Trans- mission rate and CPU utilization were logged for each execution. 75 combinations of MAX BURST and WAIT BETWEEN BURSTS were tested. All other parameters were fixed according to the µTP specification [2]. A constant packet size of 1472 Bytes was used. Figure 6.2 shows the results.
An increase of the transmission rate and CPU utilization can be seen when switching from MAX BURST = 2 to MAX BURST = 3. Using a MAX BURST of 3 gives stable results, but the σ of both rate and CPU will become higher when using MAX BURST > 3. Optimal Configuration of the library with one of the tested parameters should lead to an high transmission rate and low CPU utilization. Since there are TCP reference values available from section 6.2, all combinations of parameters that yield to a significantly lower transmission rate must be discarded, despite their lower CPU utilization. Table 6.2 gives a detailed comparison of the results of four feasible parameter combinations. They offer the highest rates and a low CPU utilization. Further tests were always conducted with MAX SEND = 5 and WAIT BETWEEN BURSTS = 28ms as they offer the lowest CPU utilization and a reasonable rate with low σ. 22 CHAPTER 6. EVALUATION
30 4000 rate CPU
28 3500
26
3000
24
2500 22
20 2000
18 Achieved Rate [kB/s] 1500 Process CPU Utilisation [%]
16
1000
14
500 12
10 0 1/4000 1/10000 1/16000 1/22000 1/30000 2/4000 2/10000 2/16000 2/22000 2/30000 3/4000 3/10000 3/16000 3/22000 3/30000 4/4000 4/10000 4/16000 4/22000 4/30000 5/4000 5/10000 5/16000 5/22000 5/30000
Figure 6.2: Combinations of different MAX BURST and WAIT BETWEEN BURST and their results. 6.4. OPTIMAL TARGET DELAY 23
6000
5000
4000
3000 Transmission rate [kB/s]
2000
1000
0 5 7 10 15 20 25 30 50 75 100 target delay [ms]
Figure 6.3: Results of different target delays
6.4 Optimal Target Delay
The target delay is an important factor in µTP’s configuration as its transmission rate directly depends on a comparison of measured buffering delays and the target delay to determine the window. To assure the correct functionality of UTP4J, different target delays were tested in the testing environment without any concurrent traffic. An overview of the achieved results can be found in figure 6.3. Such results were not expected. Even when setting the target delay down to 5ms, the rate had no significant drop in comparison to the higher target delay samples. Further investigation was required so an extended logging mechanism was implemented in the library which allows to obtain all algorithm variables during a test execution. Every time an ACK packed arrives, the algorithm variables will be updated and can be logged. With the help of this data, an analysis of the window behavior was made. Figures 6.4 and 6.5 show a comparison of delay samples and the window growth with a target delay of 10ms respectively 100ms. In the first example, the window will have a stable value after a short time and the delay samples oscillate around the target value. The 100ms target delay example shows an growing window, since the delay samples stay well below 100ms. Such a window behavior was expected but nevertheless this can not explain the observations made earlier with figure 6.3. However, one can observe the increasing delays in figure 6.5. Packet RTTs depend on buffering delays and as explained in section 1.3, the send rate is also dependeand on the RTT. Figures 6.6 and 6.7 show the same transmissions. But this time, the window is divided by the smoothed RTT that is used to compute the RTO (cf. 5.3.2).
The measured transmission rate is the file size divided by the time it took UTP4J to transmit it. It is clearly evident that the linear fits are higher than the measured rate. It is important to know, that the measuring of the transmission rate involves only the 24 CHAPTER 6. EVALUATION
target delay measured rate linear fit difference in percent w/o overhead 10ms 3559kB/s 3846kB/s 287kB/s 7.46% 1.63% 100ms 3559kB/s 3848kB/s 289kB/s 7.51% 1.68%
Table 6.3: Differences between measured rate and linear fit.
Comparison between MaxWindow and OurDelay 600000 120000 our delay max window target delay
500000 100000
400000 80000 s] µ
300000 60000 OurDelay [ MaxWindow [B]
200000 40000
100000 20000
0 0 0 5000000 10000000 15000000 20000000 25000000 30000000 35000000 40000000 time [µs]
Figure 6.4: Comparison of delay samples and window growth during a transmission with a target delay of 10ms.
payload. Packet headers are not considered. In a MTU of 1500 bytes, OSI layers 3 and 4 protocol headers (IP, UDP) and µTP headers must fit in as well, which reduces the space available for payload to 1452 Bytes. Ethernet frame headers also need to be transmitted. The IEEE 802.3 [10] specification of Ethernet frames uses 42 Bytes for its header data. The overhead makes 5.83% of the transmitted data. The remainings can be explained with OSI layer 1 overhead, collision avoidance with the ACK packets and errors in measurement. Table 6.3 summarises the comparison between the fit and the measured rate.
The analysis of this experiment concludes, that the target delay has a almost no influence on the throughput because of the buffer bloat. However, it is possible to control the buffer utilization of the routers. While the window in figure 6.4 never exceeded 100kB, the window in figure 6.5 was higher than 300kB at times. All this data was buffered in the routers and would have hindered other traffic, leading to early packet losses and high delays. 6.4. OPTIMAL TARGET DELAY 25
Comparison between MaxWindow and OurDelay 600000 120000 our delay max window target delay
500000 100000
400000 80000 s] µ
300000 60000 OurDelay [ MaxWindow [B]
200000 40000
100000 20000
0 0 0 5000000 10000000 15000000 20000000 25000000 30000000 35000000 40000000 time [µs]
Figure 6.5: Comparison of delay samples and window growth during a transmission with a target delay of 100ms. The discontinuity at 25s is caused by a packet loss.
5000
4500
4000
3500 rate [kB/s]
3000
2500
max_window/RTT measured rate linear fit 2000 0 5000000 10000000 15000000 20000000 25000000 30000000 35000000 40000000 time [µs]
Figure 6.6: Calculated transmission rate according to the window and the smoothed RTT. Target delay was at 10ms. 26 CHAPTER 6. EVALUATION
5000
4500
4000
3500 rate [kB/s]
3000
2500
max_window/RTT measured rate linear fit 2000 0 5000000 10000000 15000000 20000000 25000000 30000000 35000000 40000000 time [µs]
Figure 6.7: Calculated transmission rate according to the window and the smoothed RTT. Target delay was at 100ms. 6.5 µTP competing with concurrent traffic
µTP’s main purpose is to be less aggressive than TCP. This section investigates the behavior of UTP4J when TCP traffic occurs. Device 3 (cf. figure 6.1) was used to transmit a data file to device 6. During this transmission, concurrent TCP traffic was manually turned on. Device 4 transmitted the same data file to device 5. The only varying parameter was the target delay. TCP was expected to be slower when UTP4J used a higher target delay and vice versa. Table 6.4 summarises the results. Up to a target delay of 20ms TCP has no noticeable slowdown, considering that it normally would be slowed down by 50% if the concurrent traffic would also be TCP. After that 20ms threshold, TCP is hindered much more. Nevertheless, TCP always has finished its transmission before UTP4J. To analyze UTP4J’s behavior, the extended logging functionality was used. Examples with 10ms target delay and 100ms target delay are showed in figures 6.8 and 6.9. When the target delay is low like shown in figure 6.8, UTP4J will reduce its window even down to 0. In order to avoid a socket timeout and to obtain a buffering delay sample, every second a packet will be send when the window drops down to 0. Such a behavior was expected. Using a higher target delay will cause UTP4J to be more aggressive. As shown in figure 6.9, UTP4J is sending during the TCP transmission which leads to a few lost packets. The characteristic steep increases of the delay samples after a RTO were observed in other tests as well. Since both, UTP4J and TCP will lose packets when the buffer is already full, both reduce their window and send less which frees the router’s buffer. Hence the delay samples appear very low shortly after a packet loss. It took TCP 53.3s to finish its transmission while under the influence of µTP with an 6.5. µTP COMPETING WITH CONCURRENT TRAFFIC 27
target delay µTP avg rate TCP time TCP rate increase [ms] [kB/s] [s] [kB/s] [%] 2.5 1633 42.7 3249 1.76 5 1672 42.3 3277 0.93 7.5 1692 42.5 3263 1.36 10 1672 43.3 3202 3.18 15 1713 42.3 3278 0.91 20 1713 44.3 3130 5.35 25 1692 45.1 3074 7.06 30 1713 46.3 2996 9.43 35 1559 48.4 2864 13.40 50 1713 48.2 2876 13.05 75 1713 51.7 2683 18.87 100 1713 53.3 2600 21.38
Table 6.4: TCP performance when competing with µTP.
Comparison between MaxWindow and OurDelay 600000 120000 our delay max window target delay
500000 100000
400000 80000 s] µ
300000 60000 OurDelay [ MaxWindow [B]
200000 40000
100000 20000
0 0 0 10000000 20000000 30000000 40000000 50000000 60000000 70000000 80000000 90000000 time [µs]
Figure 6.8: UTP4J’s behavior with concurrent TCP traffic. Target delay was at 10ms. 28 CHAPTER 6. EVALUATION
Comparison between MaxWindow and OurDelay 600000 120000 our delay max window target delay
500000 100000
400000 80000 s] µ
300000 60000 OurDelay [ MaxWindow [B]
200000 40000
100000 20000
0 0 0 10000000 20000000 30000000 40000000 50000000 60000000 70000000 80000000 90000000 time [µs]
Figure 6.9: UTP4J’s behavior with concurrent TCP traffic. Target delay was at 100ms. target delay of 100ms. This is, as shown in table 6.4 21.4% more than usual. With the help of the UTP4J logs, one can calculate the rate of µTP during that period since the number of bytes already send is also logged. All values in the logs are timestamped. The rate is calculated from 10s into the transmission to 64s. 42010kB were transmitted in 54.0s, hence the rate is 778kB/s or 21.9% of the usual UTP4J throughput.
6.6 µTP Aggresivity
UTP4J can be configured to be more aggressive. To make UTP4J as aggressive as TCP means to make the congestion control packet loss driven. To achieve that, an high target delay must be set which never will cause an negative delay factor (cf. listing 1.1). As shown in table 6.4, the target delay directly influences the TCP throughput and increases UTP4J’s throughput. Another factor might be the minimum RTO, which is per default 500ms. Reducing this value, will cause UTP4J to resend lost packets faster. Another option is the increase of the maximum gain. TCP and UTP4J transmissions were started simultaneously. Table 6.5 shows the results. It took TCP 42s to transmit the test file without any competitive traffic. Thus, when both protocols have the same throughput, then it will take TCP 84s to send the file.
As seen in table 6.5, UTP4J’s aggressivity is very sensible to the gain. Its reasonable to assume that both will lose packets when the buffer is full. TCP’s window will grow exponentially while UTP4J’s window will grow linearly. When the first packet loss occurs, TCP has already send much more data than UTP4J. After some time, TCP will adapt its congestion threshold. When this threshold is reached, TCP’s window will also grow 6.7. COMPATIBILITY TO LIBUTP 29
min RTO target delay gain TCP time [ms] [ms] [B] [s] 250 3000 500 50.012s 250 4000 500 54.536s 250 5000 500 56.453s 250 6000 500 61.147s 250 7000 500 62.464s 250 8000 500 62.746s 250 8000 500 61.484s 250 9000 500 67.434s 250 10000 500 68.455 250 20000 500 81.179
Table 6.5: TCP and UTP4J competing. linearly. Making UTP4J’s window growing as fast as TCP’s is only possible by increasing the gain, since the delay factor and the window factor are always smaller than 1.
6.7 Compatibility to libutp
To ensure the compatibility to the reference implementation [5], the source code (ver- sion 7c4f19abdfe0f781311cdc68e147084961d67424 from the master branch) was compiled. There is a test folder that contains a runnable example. This was used together with the provided libutp config file to open up a connection to an UTP4J channel and send the data and vice versa. Both connections could be successfully opened and closed. All data was transmitted and received correctly. However, the test was conducted on the same device as this was no performance comparison to libutp. Its only purpose was to ensure that libutp and UTP4J can interact with each other. 30 CHAPTER 6. EVALUATION Chapter 7
Summary and Conclusions
The purpose of this Work was to implement an easy-to-integrate, configurable and stable µTP library. This goal was achieved by the UTP4J library. It can be published on Maven Central (as soon as all criterias are met) and it can be integrated in existing projects for scientific purposes. The library has been tested excessively. Unit tests are available. Also, a few thousand test runs were made during the evaluation phase. Most of these results had to be discard as they failed because of software bugs, deadlocks and other erratic behavior that had to be fixed. The implementation offers a dynamic calculation of the RTO, selec- tive ACKs and a fast resend mechanism of lost packets. It can detect and correct clock drift that causes wrong delay measurement. Its API is designed to run in asynchronous mode. This means, programs do not have to wait for the IO operations to be completed. This is achieved by providing a set of future objects and a listener interface. For evalua- tion purposes, a powerful logging system was implemented and an automatic test executor which can be configured to run a series of predefined tests with different configuration setups. However, the library still can be improved and must be evaluated in a real world environment. For that purpose, its source code is published under the Apache License 2.0.
The performance evaluation has shown, that UTP4J is as fast as TCP, but comes with a much higher CPU consumption. Attempts to reduce the CPU consumption were made with limited success. Experiments with the congestion control of UTP4J showed the de- sired effects and the results can be explained within the test environment. When properly configured, UTP4J will not hinder competing TCP traffic and even stops sending until more bandwidth is available again. It is shown, that with the right configuration, UTP4J can be as aggressive as TCP. Since UTP4J is compatible with the reference implementa- tion, it also could be used in a Java based P2P torrent client.
31 32 CHAPTER 7. SUMMARY AND CONCLUSIONS Chapter 8
Future Work
CPU load is still high and thus not yet satisfying. Further optimization of the source code is a possible way to do it once the hotspots have been identified. A CPU profiling with suitable tools would discover methods which consume a lot of CPU time. The number of expensive system calls for the time stamp implementation has been reduced already, but there are other areas which can be improved, like the copying of byte arrays.
Java’s java.net and java.nio packages do not offer an interface to set IP header options. This is the reason for the lack of a path MTU discovery algorithm. Packet sizes are configurable and the library can both send and receive packets of arbitrary size. If an implementation of such methods will be provided in upcoming Java releases, such a mech- anism could be easily implemented. The outbuffer implementation holds each outgoing packets in a DTO to store some meta data about the packet. Starting with a low packet size, the library can increase the size of a few test packets with a set DF (Don’t Fragment) flag. Routers will drop these packets instead of fragmenting them. If all test packets will be ACKed, the algorithm can increase the packet size and repeat the process until the packet size reaches the path MTU.
Dynamic target delay estimation is a possibility to improve UTP4J further. LEDBAT is a sender only algorithm and a change to the target delay has no influence on the receiving end. This can be done by recording last delay samples before a packet was dropped. The average (or any other combination) of these samples could be the new target delay.
33 34 CHAPTER 8. FUTURE WORK Bibliography
[1] SERIES G: TRANSMISSION SYSTEMS AND MEDIA, DIGITAL SYSTEMS AND NETWORKS; International telephone connections and circuits - Gen- eral; Recommendations on the transmission quality for an entire interna- tional telephone connection; One-way transmission time, ITU-T Recommen- dation G.114, https://www.itu.int/rec/dologin_pub.asp?lang=e&id=T-REC-G. 114-199303-S!!PDF-E&type=items, Last access: 10.2013. [2] A. Norberg: ”uTorrent Transport Protocol”, http://www.bittorrent.org/beps/ bep_0029.html, Last access: 10.2013. [3] A. Norberg: ”uTP”, http://www.libtorrent.org/utp.html, Last access: 10.2013. [4] S. Shalunov, G. Hazel, BitTorrent Inc et al.: ”Low Extra Delay Background Transport”, IETF, RFC 6817, http://tools.ietf.org/html/rfc6817, Last access: 10.2013. [5] G. Hazel, J. Chroboczek: ”libutp”, Version: 7c4f19abdfe0f781311cdc68e147084961d67424, https://github.com/bittorrent/libutp, Last access: 10.2013. [6] B. Sigoure: ”Java NIO and overengineering”, http://blog.tsunanet.net/2010/08/ java-nio-and-overengineering.html, Last access: 10/2013. [7] I. Iljkic: ”UTP4J”, Version: c72db6bffdeef07abfe436e16ab14c4e622e7802, https:// github.com/iiljkic/utp4j/, Last access: 11.2013. [8] G. P. Java: ”iptraf”, Version 3.0.0, http://iptraf.seul.org/, Last access: 11.2013. [9] G. Giacobbi: ”netcat”, Version: 0.7.1, http://netcat.sourceforge.net/, Last ac- cess: 11.2013. [10] IEEE Standard for Ethernet, IEEE Standard 802.3, 2012.
[11] Package java.util.concurrent, Java Platform SE7, http://docs.oracle.com/ javase/7/docs/api/java/util/concurrent/package-summary.html, Last access: 11.2013.
[12] Package java.net, Java Platform SE7, http://docs.oracle.com/javase/7/docs/ api/java/nio/channels/Selector.html, Last access: 11.2013. [13] Package java.nio, Java Platform SE7, http://docs.oracle.com/javase/7/docs/ api/java/nio/package-summary.html, Last access: 11.2013.
35 36 BIBLIOGRAPHY
[14] Class Selector, Java Platform SE7, http://docs.oracle.com/javase/7/docs/api/ java/nio/channels/Selector.html, Last access: 11.2013.
[15] ”The Central Repository”, Apache Software Foundation, http://search.maven. org/, Last access: 11.2013.
[16] M.I. Andreica, N. Tapus, J. Pouwelse, ”Performance evaluation of a Python imple- mentation of the new LEDBAT congestion control algorithm,” Automation Quality and Testing Robotics (AQTR), 2010 IEEE International Conference on , vol.2, no., pp.1,6, 28-30 May 2010.
[17] K. Ramakrishnan, S. Floyd, D. Black et al.: ”The Addition of Explicit Congestion Notification (ECN) to IP”, IETF, RFC 3168, http://www.ietf.org/rfc/rfc3168. txt, Last access: 11.2013.
[18] C. Parsa, J.J. Garcia-Luna-Aceves: ”Improving TCP Congestion Control Over Inter- nets with Heterogeneous Transmission Media”, Computer Engineering Department, University of California, Santa Cruz, 2000, http://users.soe.ucsc.edu/~chris/ sctcp/, Last access: 11.2013.
[19] L. Brakmo, S. O’Malley, L. Peterson, ”TCP Vegas: New techniques for congestion detection and avoidance”, Proceedings of SIGCOMM ’94, pages 24-35, August 1994.
[20] A.J. Abu, S. Gordon, ”A Dynamic Algorithm for Stabilising LEDBAT Congestion Window,” Computer and Network Technology (ICCNT), 2010 Second International Conference on , vol., no., pp.157,161, 23-25 April 2010.
[21] A.J. Abu, S. Gordon, ”Impact of Delay Variability on LEDBAT Performance,” Ad- vanced Information Networking and Applications (AINA), 2011 IEEE International Conference on , vol., no., pp.708,715, 22-25 March 2011.
[22] G. Carofiglio, L. Muscariello, D. Rossi, S. Valenti, ”The Quest for LEDBAT Fairness,” Global Telecommunications Conference (GLOBECOM 2010), 2010 IEEE , vol., no., pp.1,6, 6-10 Dec. 2010.
[23] C. Chirichella, D. Rossi, C. Testa, T. Friedman, A. Pescape, ”Inferring the buffering delay of remote BitTorrent peers under LEDBAT vs TCP,” Peer-to-Peer Computing (P2P), 2012 IEEE 12th International Conference on ,vol., no., pp.77,78, 3-5 Sept. 2012.
[24] D. Ros, M. Welzl, ”Assessing LEDBAT’s Delay Impact,” Communications Letters, IEEE , vol.PP, no.99, pp.1,4, 0.
[25] Peter Lipay, ”Java UTP Socket Library”, http://sourceforge.net/projects/javautp/files, Last access: 10.2013- Abbreviations
QoE Quality of Experience P2P Peer to Peer VoIP Voice over IP TCP Transmission Control Protocol LEDBAT Low Extra Delay Background Transport cwnd Congestion Window RTT Roundtrip time SACK Selective Ack MTU Maximum Transmission Unit DTO Data Transfer Object ECN Explicit Congestion Notification
37 38 ABBREVIATONS List of Figures
6.1 Test environment setup ...... 20
6.2 Combinations of different MAX BURST and WAIT BETWEEN BURST andtheirresults...... 22
6.3 Results of different target delays ...... 23
6.4 Comparison of delay samples and window growth during a transmission with a target delay of 10ms...... 24
6.5 Comparison of delay samples and window growth during a transmission with a target delay of 100ms. The discontinuity at 25s is caused by a packetloss...... 25
6.6 Calculated transmission rate according to the window and the smoothed RTT. Target delay was at 10ms...... 25
6.7 Calculated transmission rate according to the window and the smoothed RTT. Target delay was at 100ms...... 26
6.8 UTP4J’s behavior with concurrent TCP traffic. Target delay was at 10ms. 27
6.9 UTP4J’s behavior with concurrent TCP traffic. Target delay was at 100ms. 28
39 40 LIST OF FIGURES List of Tables
6.1 Measurement of TCP transmission speed in the testing environment. . . . 20
6.2 More detailed comparison of feasible parameters...... 21
6.3 Differences between measured rate and linear fit...... 24
6.4 TCP performance when competing with µTP...... 27
6.5 TCP and UTP4J competing...... 29
41 42 LIST OF TABLES Listings
1.1 LEDBAT algorithm to calculate CWND ...... 3 5.1 Writingalgorithm...... 14 5.2 Calculation of the retransmission timeout...... 15 5.3 Reading implementation...... 16
43 44 LISTINGS Appendix A
Important Configuration Parameters
Description of other parameters can be found in the source code.
SKIP_PACKETS_UNTIL_ACK This integer value determines how many packets the receiver can receive before sending an ACK packet. Default is 2. AUTO_ACK_SMALLER_THAN_ACK_NUMBER A boolean value, when set to true, all packets up to the specified ACK-Number in the ACK packet will be ACKed. It’s highly recommended to set this value to true, especially when SKIP_PACKETS_UNTIL_ACK is greater than 1. MINIMUM_DELTA_TO_MAX_WINDOW_MICROS Having low delay samples will result in an increasing window. But when there is a rate limitation for some reason, the max window might never be utilized fully but still would grow. To prevent the window from growing infinitely, UTP4J must keep track of the last time stamp when the window was full. If this time stamp is older than the value specified here, the gain will be set to 0. MINIMUM_DIFFERENCE_TIMESTAMP_MICROSEC LEDBAT assumes that packets will take the same path in a network. But µTP uses UDP to transport its packets and they might take another route which leads to varying base delays, resulting in wrong buffering delay measurements. To adapt to this situation, the base delay will be updated regularly. It can not be older than the value specified here. PACKET_SIZE_MODE µTP uses dynamic packet sizes per default. It is possible to change to constant packet sizes by changing this value. MAX_CWND_INCREASE_PACKETS_PER_RTT This is the maximum possible increase (see section 1.3). C_CONTROL_TARGET_MICROS The target delay can be changed by modifying this value.
45 46 APPENDIX A. IMPORTANT CONFIGURATION PARAMETERS
SEND_IN_BURSTS To avoid large bursts of packets in short time, this value is set to true. It will force the write thread to stop sending after a predefined number of packets. When this value is set to false, packets will be send continuously until the window is full. MAX_BURST_SEND When SEND_IN_BURSTS is true, the write thread can not send more packets than specified by this integer value at a time. MIN_SKIP_PACKET_BEFORE_RESEND µTP implements selective ACKs. Packets that were lost can either trigger a retrans- mission timeout (RTO) or a fast resend. This value specifies the minimum number of packets, that must be ACKed past the lost packet to trigger a fast resend. The default value is 3. Appendix B
Installation Guidelines
The library can be used with Java 1.6 but the pom file must be slightly modified to exclude the examples package, as there is a class that uses Java 1.7 features. Between the
47 48 APPENDIX B. INSTALLATION GUIDELINES Appendix C
Contents of the CD
The CD contains following folders:
LaTeX Source The source of the written report.
Presentations The final and intermediate presentations.
Printable files The PDF and PS versions of the report.
Source UTP4J’s source code.
HTML Page The HTML page of UTP4J.
Experiment Data Results of the experiments with plotscripts.
49