Bachelor thesis Computing Science

Radboud University

Performance comparison of DNS over HTTPS to unencrypted DNS

Author: First supervisor/assessor: Jeroen Wijenbergh Dr. Veelasha Moonsamy s4792459 [email protected]

Daily supervisor: Dr. Roland van Rijsdijk-Deij [email protected]

Second assessor: ir. D.W.C. Dani¨elKuijsters [email protected]

October 3, 2019 Abstract To encrypt DNS, DNS over HTTPS has been proposed. This protocol has additional overhead that can not be found in DNS over UDP. In this study, we will analyze the performance impact of this extra overhead. To do this, we have extended the established DNS measurement tool Flamethrower with DNS over HTTPS capability. This imple- mentation is used to query a server that we have set up. This server runs Dnsdist and Unbound to provide DNS and DNS over HTTPS on a single host. We will query two of these servers with a different number of concurrent TCP and HTTP/2 streams and analyze the results. These results will help us in answering what performance impact DNS over HTTPS has as compared to unencrypted DNS. The results that we will obtain in this research is that for all of our tests DNS over HTTPS has worse performance as compared to UDP. The performance difference is less noticeable when caching is disabled. Contents

1 Introduction 4 1.1 Motivation ...... 5 1.2 Overview of thesis ...... 6

2 Preliminaries 7 2.1 Network Stack ...... 7 2.1.1 Application layer ...... 8 2.1.2 Transport layer ...... 8 2.1.3 Network layer ...... 8 2.1.4 Data Link layer ...... 8 2.1.5 Physical layer ...... 9 2.1.6 Layer encapsulation/decapsulation ...... 9 2.1.6.1 Importance of encapsulation/decapsulation ...... 9 2.2 Transport layer protocols ...... 9 2.2.1 Transmission Control Protocol (TCP) ...... 10 2.2.1.1 Structure ...... 10 2.2.1.2 Handshake ...... 12 2.2.1.3 Message exchange ...... 12 2.2.1.4 Positives and drawbacks ...... 13 2.2.2 User Datagram Protocol (UDP) ...... 13 2.2.2.1 Structure ...... 13 2.2.2.2 Differences between UDP and TCP ...... 14 2.3 Application Layer protocols ...... 14 2.3.1 Hypertext Transfer Protocol (HTTP) ...... 14 2.3.1.1 Structure ...... 15 2.3.1.2 Encryption ...... 16 2.3.1.3 HTTP standards ...... 16 2.3.2 Transport Layer Security (TLS) ...... 16 2.3.2.1 Goals of TLS ...... 16 2.3.2.2 Encryption ...... 17 2.3.2.3 Authentication ...... 17 2.4 DNS ...... 19

1 2.4.1 Distributed system ...... 19 2.4.1.1 Stub resolver ...... 19 2.4.1.2 Recursive resolver ...... 20 2.4.1.3 Authoritative servers ...... 20 2.4.2 Structure ...... 20 2.4.3 Recent work in encrypting DNS ...... 24 2.4.3.1 DNS over HTTPS (DoH) ...... 24

3 Research 28 3.1 Scope ...... 28 3.2 Methodology ...... 29 3.2.1 Software ...... 29 3.2.1.1 Client ...... 29 3.2.1.2 Resolver ...... 40 3.2.2 Hardware ...... 44 3.2.2.1 Client ...... 44 3.2.2.2 Server ...... 44 3.2.3 Performance measurement ...... 45 3.2.3.1 Dataset ...... 45 3.2.3.2 Tests ...... 45 3.2.3.3 Running the tests ...... 46 3.2.3.4 Analyzing the output ...... 48 3.2.3.5 Overview ...... 49 3.3 Results ...... 50 3.3.1 TCP connection delay ...... 50 3.3.1.1 Caching enabled ...... 50 3.3.1.2 Caching disabled ...... 51 3.3.2 TLS connection delay ...... 51 3.3.2.1 Caching enabled ...... 52 3.3.2.2 Caching disabled ...... 53 3.3.3 Round-trip time of DNS and DoH ...... 53 3.3.3.1 Caching enabled ...... 54 3.3.3.2 Caching disabled ...... 59 3.4 Reflecting on the results and limitations ...... 65

4 Related Work 66 4.1 DNS Caching ...... 66 4.2 Mozilla DoH measurements ...... 67 4.3 DNS, DoT and DoH page load time and resolution times ...... 68 4.4 Methodology for testing DNS in a lab environment ...... 69

5 Conclusions 70 5.1 Future work ...... 70

2 Bibliography 71

A Appendix 75 A.1 TCP connection delay measurements ...... 76 A.1.1 Caching enabled ...... 76 A.1.2 Caching disabled ...... 78 A.2 TLS connection delay measurements ...... 80 A.2.1 Caching enabled ...... 80 A.2.2 Caching disabled ...... 81 A.3 Round-trip time measurements ...... 82 A.3.1 Caching enabled ...... 82 A.3.2 Caching disabled ...... 88

3 Chapter 1

Introduction

When you want to visit a website on the internet, your browser needs to know the IP address of that website. These IP addresses are hard to remember. To address this prob- lem, two researchers (Paul Mockapetris and Jon Postel) introduced the Domain Name System (DNS). This architecture maps hostnames (e.g. google.com) to an IP address (e.g. 216.58.211.110 ), which makes it so that you only have to enter the hostname into your browser to gain access to the website. This is the main benefit of DNS, but there are more; an important one is that DNS also allows a website to easily transfer their hostname to another server by changing the DNS settings. Furthermore, while we gave the example of using DNS to obtain the IP address for a website, DNS is also used for email. The DNS architecture was introduced in 1983 [41] [43] and has seen little change since then. As a result, this architecture has a few problems. One of these problems is that the DNS messages are unencrypted. To add encryption to DNS, extensions have been proposed, for example: • DNS over HTTPS (DoH) [35] • DNS over TLS (DoT) [38] • DNSCURVE [16] • DNSCrypt [4] In this thesis, we want to focus on the DNS over HTTPS protocol and how it compares to regular DNS. The reason why we chose to study this protocol will be discussed in section 1.1.

As the title of this thesis already suggests, we want to explicitly focus on the per- formance impact of using DNS over HTTPS. What we exactly mean with performance will be discussed in chapter 3.

4 The following will be our research question: How much performance impact does DNS over HTTPS have as compared to using unencrypted DNS? We also have the following sub-research questions: • How do the HTTP methods POST and GET compare with regards to perfor- mance? • To what extent do TCP connections impact the performance of DNS over HTTPS? • To what extent do HTTP/2 streams impact the performance of DNS over HTTPS? • To what extent does caching impact the performance of DNS over HTTPS? To get help with this thesis, we have interned at the NLnet Labs foundation. They helped us with deciding on a scope for this thesis. This scope will be defined in section 3.1 and there we will discuss our research questions in more detail. The methodology we used for testing this performance has also been discussed with NLnet Labs and can be found in section 3.2. To get a better idea of why we chose to research this topic, we will now discuss our motivation.

1.1 Motivation

As stated before, DNS messages are still unencrypted. To understand why this is a problem, we need to look at a possible attack scenario. DNS messages (on top of the hostname) provide information about the client that is sending the message (e.g. your browser). A third-party that listens for traffic between you and the DNS server can see any DNS message. This means that this third-party can see what websites you are visiting on the internet. This problem not only exists in theory, it is actively abused in practice [34].

When you encrypt these DNS messages, the content of them can not be seen by parties that are not involved in the DNS hierarchy. This includes eavesdroppers on a network. One of the most popular and newest protocols to achieve this encryption is the DNS over HTTPS protocol. The protocol was first published in May 2017 [36], and was subse- quently standardized by the Internet Engineering Task Force in August 2018 [35]. Even though this protocol is new, it is already being used in the DNS resolvers from Cloud- flare [26] and Google [33]. Firefox, a popular internet browser, is also testing DNS over HTTPS support [44] [45] [46]. Because the adoption of this protocol moves quicker than the other protocols that were mentioned (DNS over TLS, DNSCURVE, DNSCrypt), we have decided to focus this thesis on this protocol. Due to the recency of this protocol, we must investigate if it can replace unencrypted DNS. To achieve this, as we have pre-

5 viously alluded to in our research questions, we will investigate the performance of DNS over HTTPS as compared to unencrypted DNS. See the next subsection for an overview of the thesis.

1.2 Overview of thesis

In this thesis, we will first describe the different topics that are needed to follow this thesis. This is defined in the Preliminaries section (chapter 2). After this we will describe our Research and the results (chapter 3). We will also discuss Related Work (chapter 4) and finally give a Conclusion (chapter 5).

6 Chapter 2

Preliminaries

In this section we will give a brief overview of the preliminaries that are needed to read this thesis. We will first give a broad overview of the network stack. Understanding this network stack helps in understanding the basics of networking. After this, we will go into more detail of the protocols that are important to know. At last, we will describe DNS and DNS encryption.

2.1 Network Stack

In computer networking, each protocol belongs to a so-called “layer”. We have two models that define these layers, the OSI model and the TCP/IP model:

Application layer Application layer Presentation layer Transport layer Session layer Network layer Transport layer Data Link layer Network layer Physical layer Data Link layer Table 2.2: TCP/IP model Physical layer

Table 2.1: OSI model

The OSI model is seen as a purely theoretical model, while the TCP/IP model can be seen as a practical model [40]. Because of this, we will only discuss the layers that are used in the TCP/IP model. The protocols of all these layers are defined as a network stack (also called protocol stack). Notice that the Application layer is the top layer and is encapsulated all the way down to the Physical Layer. To get a good grasp on what these layers exactly represent, we will give an overview of

7 each layer and a short description of the different protocols that these layers have. After this, we will give an explanation on why it is important that networking is actually split in different layers.

2.1.1 Application layer This layer defines the protocols that do the process to process communication. These protocols utilize the transport layer protocols to do the communication itself. Some examples of application layer protocols are: • File Transfer Protocol (FTP): Used for transferring files between process. • Simple Mail Transfer Protocol (SMTP): Used for sending and receiving mail be- tween hosts. DNS is an architecture that also resides in the application layer. We will explain DNS in section 2.4.

2.1.2 Transport layer The transport layer defines the protocols that make the actual communication between hosts happen. The two most popularly used protocols in this layer are: • User Datagram Protocol (UDP) • Transmission Control Protocol (TCP) The difference between these two protocols will be explained in section 2.2.2 and sec- tion 2.2.1.

2.1.3 Network layer The network layer defines protocols that are used to route packets between (or within) networks. Some protocols of this layer are: • Internet Control Message Protocol (ICMP): Used to send error messages. • IPv4/IPv6: Protocol that delivers packets from the source host to the destination host based on IP addresses. • Open Shortest Path First (OSPF): A routing protocol.

2.1.4 Data Link layer The Data Link layer defines protocols that are used for the delivery of packets (called “frames” in this layer) over links. With links we have two flavours: wireless (WLAN) or wired (LAN). The most popular protocol in this layer is the Ethernet protocol.

8 2.1.5 Physical layer The physical layer defines the protocols that transports individual bits from one node 1 to another node [40]. For example Bluetooth [2] is a protocol that is on the physical layer.

2.1.6 Layer encapsulation/decapsulation To see more clearly that all these layers are encapsulated from the Application Layer to the Physical Layer, see Figure 2.1: To form the application layer these layers must all be

Figure 2.1: Layer encapsulation of the TCP/IP model [22] encapsulated. On the receiving end these layers are decapsulated so that the different layers can be processed.

2.1.6.1 Importance of encapsulation/decapsulation Networking is a complex problem, in order to make it easier for developers the network stack has been split in these layers. Each layer only provides a part of the complete networking stack. When we want to extend the network stack with a certain functionality we can do so by adding it to the correct layer.

2.2 Transport layer protocols

In the last section, we have described the network stack in a global overview. For this thesis it is crucial to know DNS and DNS over HTTPS in detail. Because these two protocols operate on top of the transport layer (the application layer), it is important to know the two transport protocols that can be used for DNS: • Transmission Control Protocol (TCP)

1Node is an endpoint.

9 • User Datagram Protocol (UDP)

2.2.1 Transmission Control Protocol (TCP) As described in section 2.1.2, TCP is a protocol that is used for communication. This protocol is used for “connection-oriented transport”. This means, that before two clients can send data they must first establish a connection. This is done by a handshake.

2.2.1.1 Structure In order to understand the TCP handshake we will first go over the structure of a TCP packet. To see what a TCP packet looks like see Figure 2.2:

Figure 2.2: A TCP packet [8]

The Data represents the message. The rest of the fields is the header. We will go over the important fields of this header.

Port numbers In the TCP header there are two fields: Source port 2 number and Destination port number. As the name suggests, the Source port number is the port used by the sender of the TCP packet. The Destination port number is the port where the message will be delivered.

Sequence and Acknowledgement number

2A port is “used to distinguish between different services that run over transport protocols” [21]. A list of standard ports is published by the Internet Assigned Numbers Authority (IANA) [21].

10 It is important to know what the sequence number and acknowledgement number rep- resent. The sequence number can be seen as an identifier for the TCP packet. This is so that the client and the server know which packet is which. This can be used for re-ordering packets. The acknowledgement number is a number that is sent back when a packet is received. This number is set to the next packet that the receiver is expected to receive from the sender. For example when a sequence number is set to x and every packet is in order then the client sends back an acknowledgement number of x+1.

Flags TCP also has some flags that can be set, these are [47]: • URG: If this flag is set then the TCP packet is meant to be urgent, this indicates that the receiver should look at the urgent pointer in the TCP packet header. • ACK: If this flag is set then the receiver should look at the Acknowledgement number field of the TCP packet header. • PSH: This flag is set when the data needs to be immediately transferred to the application. • RST: When this flag is set it indicates that a reset packet 3 is send to the receiver. • SYN: This flag is used in the connection establishment handshake. We will discuss this in section 2.2.1.2. • FIN: This flag is used in the connection termination handshake. We will discuss this in section 2.2.1.2. Window size The window size specifies how much data a receiver is willing to receive.

Checksum The checksum can be used to verify if the TCP packet has been corrupted or not.

Urgent pointer The urgent pointer signifies where the urgent data ends.

Options TCP has parameters that are set in the options field of the TCP header. We will discuss two important ones: • Maximum segment size (MSS): Defines the maximum amount of data that can be sent in one packet (also called “segment”) [47].

3A reset packet is a packet that is send when an unexpected packet has been received from a client. This packet has no payload.

11 • Window scale: An option that can be used to increase the maximum window size [47]. It is possible that a packet exceeds the maximum segment size. This means that the payload must be split in multiple packets. Because of this, we need to send multiple packets in the right order. If a packet gets lost, TCP retransmits the packet with the same sequence number. The receiver can then re-order these packets by looking at the sequence numbers.

Padding Padding is used to fill the remaining space of the packet. This padding has no further use case.

2.2.1.2 Handshake TCP has two different handshakes, it has a handshake for connection establishment and connection termination.

Connection establishment In order to explain the TCP handshake we will give an example scenario. Suppose Bob wants to communicate with Alice through TCP. The first step is to create a TCP packet with the SYN bit set and the sequence number set to an arbitrary number, let’s call this number a here. When Alice receives this number, she will send back a packet with the SYN and ACK bit set. Her packet will also need the acknowledgement number set to a+1 and will need a different sequence number, let’s call this b. At last Bob will send a packet with the ACK bit set and with acknowledgement number b+1 and sequence number a+1. The handshake is now done.

Connection termination We have seen that the connection establishment handshake is 3-way, first a SYN, then SYN-ACK and at last the ACK. The handshake for termination has 4 messages [40]. When a client wants to close the connection it sends a tcp message with the FIN bit set. When the server receives this message it sends back a tcp message with the ACK bit set, which indicates that the server has received the previous FIN message. The server then agrees with the shutdown and does the same thing that the client has done (sending a FIN message). The client then sends back an ACK message to confirm that it has received the FIN message from the server.

2.2.1.3 Message exchange Now that the handshake is done, we can send messages. Whenever we send a mes- sage we give the message a sequence number. When the server receives this message, it will send an ACK with as acknowledgement number the next packet it expects to receive. Whenever we want to close the connection, we issue the connection termination handshake.

12 2.2.1.4 Positives and drawbacks To conclude this section, we will give a short summary of the benefits of TCP as compared to other transport protocols (e.g. UDP): • It is reliable, packets are always received due to the extensive retransmission schemes that TCP uses. • Packets can always be put in the right order by the receiver. Some drawbacks of TCP: • Additional overhead as compared to UDP. • Hard to implement properly.

2.2.2 User Datagram Protocol (UDP) In order to understand the User Datagram Protocol (UDP), we will first go over the structure of an UDP packet.

2.2.2.1 Structure An UDP packet looks like the following:

Figure 2.3: An UDP packet [3]

We see that an UDP packet is simply a trimmed down version of a TCP packet. This is

13 because UDP doesn’t have re-ordering of packets or retransmitting schemes. This means that the UDP packet then also doesn’t need all the special flags and acknowledgemen- t/sequence numbers that TCP has. As a result, this indicates that one UDP packet is the whole message that needs to be send. Because of this, the UDP packet has a length field that represents the total length of the packet (header and payload).

2.2.2.2 Differences between UDP and TCP Unlike TCP, UDP is not “connection oriented”. This means that UDP does not have an initialization or closing handshake. It also does not have a retransmission scheme. This makes UDP a very simple protocol as compared to TCP. Because the UDP packet is much smaller and no handshake needs to be done, it is faster than TCP. Because of this, UDP is often used in services that require speed over reliability. Another benefit is that UDP is easier to implement as compared to TCP.

2.3 Application Layer protocols

In the last section, we have seen transport layer protocols that are important in order to understand DNS. In this section we will give a description of two application layer protocols that are important to understand DNS over HTTPS: • Hypertext Transfer Protocol (HTTP) • Transport Layer Security (TLS)

2.3.1 Hypertext Transfer Protocol (HTTP) The Hypertext Transfer Protocol (HTTP) is an application layer protocol that is used for communication on the World Wide Web (WWW). HTTP is used on top of the TCP protocol.

14 2.3.1.1 Structure The structure of a HTTP request message is the following:

Figure 2.4: Structure of a HTTP message [40]

For each section of this message we will explain what it means.

Request line The HTTP request line specifies the HTTP method, the URL and the version of HTTP that was used. We will talk more about the HTTP version in section 2.3.1.3 The HTTP method specifies in what way we want to send the HTTP message. The request methods that are important to know for this thesis are, GET and POST.

GET The GET method has no body/content. When we want to send a certain parameter to the HTTP server, we often do this with a parameter in the URL.

POST The POST method does have a body. When we want to send a parameter we can just put this in the body of the request. The URL specifies to what endpoint we want to send the HTTP message. For example if we want to make a request to Google the URL will be http://www.google.com. The http part of the URL signifies that the protocol used is HTTP. Header lines The HTTP headers specify the metadata that we want to send to the receiver. For example when we want to inform that our body is a JSON message then we simply add Content-Type: Application/Json to the HTTP headers. This way the receiver can see what type of data we have sent and can parse it accordingly. Two other important parameters to know is path and authority. To understand these parameters, we will give an example: If we have as URL http://www.jwijenbergh.com/index.html then

15 the authority is www.jwijenbergh.com and the path is /index.html.

2.3.1.2 Encryption To add encryption to HTTP, HTTPS is used. HTTPS encrypts traffic by using a HTTP connection in a TLS tunnel. In section 2.3.2, we will explain how TLS works.

2.3.1.3 HTTP standards Nowadays there are two popular HTTP standards: • HTTP/1.1 [32] • HTTP/2 [25] Everything we talked about can be done with HTTP/1.1, but for DoH HTTP/2 is required [35]. This is why it is interesting to look at some of the improvements [25]: • To reduce the packet size, HTTP/2 can use header compression using an algorithm called HPACK. • HTTP/2 also brings a new request method that is called a Server push. • HTTP/2 can use multiple streams over one TCP connection. If we want to rapidly send requests, we do not have to setup a TCP connection for each request.

2.3.2 Transport Layer Security (TLS) As the name suggests, the Transport Layer Security is a protocol that encrypts the trans- port layer (over TCP). TLS operates one layer above the transport layer (application layer according to the TCP/IP model). There are different revisions of the TLS protocol: • TLS 1.1 [29] • TLS 1.2 [30] • TLS 1.3 [48] Because TLS 1.2 is the most used protocol nowadays, we will follow this revision with our explanation of the protocol. When we refer to TLS in this section we are talking about TLS 1.2. At the end of this section, we will talk about TLS 1.3 and what advantages it brings over TLS 1.2. In the next section, we will talk about the goals that TLS wants to realize.

2.3.2.1 Goals of TLS TLS has the following goals [30] (in order of importance):

16 1. Cryptographic security: It should be used to create a secure tunnel between two ends. 2. Interoperability: Applications utilizing TLS should be able to work with other applications that also utilize TLS. 3. Extensibility: It should be easy to extend the TLS protocol with additional fea- tures. 4. Relative efficiency: It should not be CPU intensive. We will further explain how the Cryptographic security goal is realized. We do this by first giving a broad overview of the encryption that is used. We will then explain how this encryption is negotiated and used in the protocol.

2.3.2.2 Encryption TLS relies on public-key cryptography and symmetric key cryptography. We will explain these two terms and show what use case they have in TLS. In public-key cryptography two keys are used for encrypting and decrypting data: a private key and a public key. As the name suggests, the public key is known publicly. Anyone who wants to see a public key can do so. The private key is only known to the party that owns the key, it is important that this key is not given to an outsider. The keys are setup such that data that is encrypted with the public key can be decrypted with the private key (and vice versa). This encryption is also known as assymetric en- cryption. In TLS, public-key cryptography is used to agree upon a symmetric key for using symmetric encryption. Symmetric encryption is encryption where the same cryptographic key is used for en- cryption and decryption of data. In TLS this key is called the session key. The session key is the key that is used for encrypting and decrypting traffic that is sent over the transport layer. What algorithms are used for encryption and establishing session keys is determined in the cipher suite of the TLS connection. How this cipher suite is established, we will discuss in the next section. But before we do that, we will also discuss how TLS provides authentication 4.

2.3.2.3 Authentication TLS provides authentication by using Certificate Authorities.A Ceritificate Authority (CA) is a trusted third party that hands out certificates. This certificate provides a guarantee of ownership for a certain public key. The format that is used in TLS of such a certificate is specified in standard X.509 [27].

4Authentication is “the ability to prove that a user or application is genuinely who that person or what that that application claims to be” [15]

17 Connection Establishment TLS uses a handshake (like TCP) to agree on the following attributes: • The cipher-suite • The session key The handshake works as the following: The client sends a ClientHello message to the server. This ClientHello message specifies the cipher-suite, the TLS version, the com- pression methods that are used and a list of supported protocols if Application-Layer Protocol Negotiation (ALPN) 5 is used. When the server receives this ClientHello mes- sage, the server sends a ServerHello message which has the cryptographic protocols that are being used (an agreement of the cipher-suite). This ServerHello message also contains a session id (optional, only send if the server resumes a previous session), the certificate of the server and the public key of the server. The client then needs to verify the certificate. It does by using the public key of the Certificate Authority (which is stored locally in the clients browser). If the certificate is verified (meaning trust has been established), the client can send the next message in the handshake. The client sends a ClientKeyExchange. This ClientKeyExchange contains the shared secret key that is generated using the public key of the server. At last the client and the server both send a ChangeCipherSpec and Finished message which indicates that every message from now on is encrypted and that the handshake has been terminated. An overview of this description is given below:

Figure 2.5: The TLS handshake [23]

5Application-Layer Protocol Negotiation (ALPN) is a scheme used to determine what protocol is used over a TLS-tunnel

18 TLS 1.3 improvements The newest version, TLS 1.3 comes with a few improvements over TLS 1.2 [48]: • The handshake is now shorter. Instead of needing two round-trip times for TLS 1.2, TLS 1.3 only needs one. • It introduces “zero-rtt” (zero round-trip time): A client that reconnects to a server has no need for a prior handshake. To achieve this, session caches or session tickets are used.

2.4 DNS

Now that we have explained the necessary prerequisites in order to understand DNS and DNS over HTTPS, we can explain DNS and DNS over HTTPS in detail. We will do this by first giving an explanation of the DNS architecture adn then continuing on with DNS over HTTPS.

As said, DNS is an architecture to map hostnames to IP addresses. We call DNS an “architecture” here because DNS is actually two things: 1. It is a “A distributed database implemented in a hierarchy of DNS servers” [40] 2. It is a protocol that can query this “distributed database” [40]. We will first explain how this “distributed database” is formed and then give the DNS protocol that is used to query this database.

2.4.1 Distributed system The DNS database is called distributed, because there is more than one item involved that is used to get the IP address of a hostname: • Stub resolver • Recursive resolver • Authoritative servers We will go over each of these components and explain their role in the DNS architecture.

2.4.1.1 Stub resolver The stub resolver is the client that that sends the initial DNS message. It does this by forwarding this message to the recursive resolver that is configured in the application/- operating system.

19 2.4.1.2 Recursive resolver A DNS (recursive) resolver is a server that is responsible for receiving a DNS message from a client (e.g. an internet browser) and forwarding it to the authoritative servers. This DNS resolver is often by default the server from your Internet Service Provider. This resolver does not always forward the query to the authoritative servers. This only needs to be done if the hostname for the DNS message is not stored in the cache of the resolver. The cache is where domain names are stored (once they are received from the authoritative servers) with a time to live setting. This time to live specifies how long a domain name must be stored in the cache before it expires. The benefit of this cache is that queries can be answered quickly, because the resolver does not have to contact the authoritative servers if the domain name is still valid in the cache.

2.4.1.3 Authoritative servers There are different authoritative servers: • Root servers • Top-level domain servers • Name servers Root servers The root servers are responsible for finding the appropriate top level domain server for a query. There a 13 root servers that are distributed across the world.

Top-level domain servers Before we explain what a top-level domain server is, we need to know what the top-level domain represents. The top-level domain is the last part of a hostname. When we have as hostname google.com then the top-level domain is com. For every top-level domain there is a top-level domain server. This server knows what authoritative server to contact for each domain.

Name servers The authoritative name server knows the DNS specific settings (the IP addresses) for a set of domains. This server is responsible for giving the IP address for the domain. It will forward the answer back to the recursive resolver. The resolver then gives the IP address to the client.

2.4.2 Structure A DNS message has the following structure [42]:

20 Header Question Answer Authority Additional

Table 2.3: DNS high-level structure We will give an overview for each part of this DNS message.

Header The header itself also has an underlying structure which looks like the following [42]:

Figure 2.6: Structure of a DNS Header

We will explain what each field is used for. The ID field is used as an identifier to know which reply belongs to which request. The QR field is set to 0 when the DNS message is a request (query) and is set to 1 when it represents a reponse. The OPCODE field indicates what type of query the message is. The important OPCODE to know is 0, which indicates a standard DNS query. AA( Authoritative Answer) is a bit that is set to 1 when “the responding name server is an authority for the domain name in question section”. The TC bit indicates whether or not the DNS response is truncated 6 . The RD bit represents if recursion is desired. The RA bit is set to 1 when support for recursion is available from the name server. The Z is a set of zeros. The RCODE (response code) can have the following values [42]: • 0: No error reported • 1: Format error • 2: Server failure • 3: Name error • 4: Not implemented

6Truncating happens when the DNS message doesn’t fit in a packet, often TCP will then be used instead of UDP to transport the message.

21 • 5: Refused The count values represent the number of entries in each section: • QDCOUNT: Question section count • ANCOUNT: Answer section count • NSCOUNT: Authority section count • ARCOUNT: Additional section count What each of these sections mean will be discussed next.

Question This section is used for the query. It represents what the actual question to the resolver is. This section is formatted like the following:

Figure 2.7: DNS Answer section structure [42]

The QNAME represents the domain name where we want to get the IP-address for. The QTYPE represents the type of query. The QCLASS represents the class of the query, this is set to IN for internet.

22 Answer, Authority & Additional The last 3 sections all have the same structure. This structure is also known as a Resource record:

Figure 2.8: DNS resource record structure [42]

We will yet again elaborate what these fields mean. The NAME field is used to store the domain name. The TYPE field contains the resource record type code. The important resource record types to know are the following: • A: This resource record means that it is the answer to an IPv4 address. • AAA: This resource record means that it is the answer to an IPv6 address. • MX: This resource record contains the server that is responsible for mail. • CNAME: This resource record indicates that the hostname has an alias (also known as CNAME). This CNAME should be queried to get the IP-address. The CLASS field specifies the class of the dns query. The TTL (time-to-live) is a value that states how long the resource record should be stored in the cache. The RDLENGTH specifies the length of the RDATA field. Finally, the RDATA field is “a variable length string of octets that describe the resource” [42].

We have seen that the format for the Answer, Authority & the Additional section are the same. So when is each section used? • The Answer section contains answers to the question. • The Authority section contains the name server(s) that are responsible for answer- ing the question. This can also contain evidence that a record does not exist (when DNSSEC 7 is used).

7DNSSEC is an extension to DNS that provides origin authentication and integrity for DNS records [24]

23 • The additional section contains “additional” answers for domain names that we did not ask for. This is useful if the server thinks we will query this domain name in the future. The additional section also contains TSIG (used for DNS transaction authentication) and OPT (used for EDNS 8) records.

2.4.3 Recent work in encrypting DNS As described in chapter 1, DNS traffic is still unencrypted. To encrypt DNS traffic there are two standards published by the Internet Engineering Task Force (IETF): • DNS over TLS (DoT) [38] • DNS over HTTPS (DoH) [35] Because this thesis focuses on the latter protocol, we will give an overview of this pro- tocol.

2.4.3.1 DNS over HTTPS (DoH) DNS over HTTPS (DoH) is an extension that aims to encrypt the DNS traffic. It does this by utilizing HTTPS to encrypt the connection. The minimum required HTTP ver- sion is HTTP/2. Before we give an example of a DNS request, we will give an explanation on how these requests are formed.

Request There are two different ways to create a DoH request. The first method is using the HTTP GET method.

GET As we have seen in HTTP (section 2.3.1), a GET request has no body, so we need to parameterize the DNS request. To do this an encoding called base64url 9 is used. When we have obtained the encoded dns packet it needs to be added to an URL parame- ter. This is done by appending ?dns=encoding to the url of the request (where encoding describes the encoded dns packet). To make this more clear we will give an example of sending a DNS query for hostname example.com to the DoH server doh.jwijenbergh.com.

8EDNS are extension mechanisms for DNS [28] 9Base64url is an encoding that is essentially base64 encoding, but with the alphabet changed to allow for the encoding of URLs.

24 Example 1: Constructing a DoH GET packet

The first thing we need is the dns message in base64url encoding. For www.example.com this is AAABAAABAAAAAAAAA3d3dwdleGFtcGxlA2NvbQAAAQAB. We can now construct the header:

:method = GET :scheme = https :authority = doh.jwijenbergh.com :path = /dns-query?dns=AAABAAABAAAAAAAAA3d3dwdleGFtcGxlA2NvbQAAAQAB accept = application/dns-message

These headers are then send to the server doh.jwijenbergh.com. The accept header field is set to application/dns-message which indicates that we expect a dns message as reponse.

POST The second method is using the HTTP POST request method. This method does have a body, which means that we do not have to encode the DNS packet. The DNS packet is simply used as body for the POST request. We will now give the same example as the GET method and see how these two methods differ.

25 Example 2: Constructing a DoH POST packet

We first need to obtain the DNS query in DNS wire format (which will be the body of the request). This is the following 33 bytes [35]:

00 00 01 00 00 01 00 00 00 00 00 00 03 77 77 77 07 65 78 61 6d 70 6c 65 03 63 6f 6d 00 00 01 00 01

Now we just need the headers of the DNS message. These will be the fol- lowing:

:method = POST :scheme = https :authority = doh.jwijenbergh.com :path = /dns-query accept = application/dns-message content-type = application/dns-message content-length = 33

Notice that with the POST method the content-length is provided in the headers. The header also contains the content-type, which needs to be set to application/dns-message to indicate that a dns message is contained in the body. Also notice that the POST method has the path /dns-query without any parameters. This whole message (Headers + body) is then send to the server doh.jwijenbergh.com.

Response The response is then simply the dns response stored in the body of a HTTP response. The example that the DoH specification gives is the following [35]:

26 Example 3: DoH response

Suppose we queried yet again for wwww.example.com. Suppose the response has an address of 2001:db8:abcd:12:1:2:3:4 and a TTL of 3709 seconds, then the HTTP response is the following [35]: :status = 200 content-type = application/dns-message content-length = 61 cache-control = max-age=3709

61 bytes represented by the following hex encoding 00 00 81 80 00 01 00 01 00 00 00 00 03 77 77 77 07 65 78 61 6d 70 6c 65 03 63 6f 6d 00 00 1c 00 01 c0 0c 00 1c 00 01 00 00 0e 7d 00 10 20 01 0d b8 ab cd 00 12 00 01 00 02 00 03 00 04

27 Chapter 3

Research

In this section, we will describe how we answered our research question. We do this by first giving an overview of what we want to measure to test the performance of DNS over HTTPS. This is defined in the section called Scope. Next we will how we accomplished the Scope in our Methodology section. At last we will give the Results that were obtained by applying the Methodology.

3.1 Scope

In chapter 1 it was stated that our main research question was the following: How much performance impact does DNS over HTTPS have as compared to using unencrypted DNS? This research question on its own might be a bit too vague for explaining our scope. Because of this, we want to give a more accurate description of what we are measuring. This is given below.

Because the DNS architecture is big and complex, we needed to figure out what parts we wanted to measure and how we were going to do this. The NLnet Labs foundation helped us with this. We had three meetings with them and after that interned at their building in Amsterdam for one month. These three meetings were brainstorm sessions where we discussed the different possibilities of measuring DNS and DoH. We quickly came to the conclusion that we wanted a setup that looked like the following (see next page):

28 Figure 3.1: Experimental setup

We wanted to measure the performance of path A, meaning the performance between the client and the resolver. Here, the cloud represents the “Rest” of the DNS architecture (authoritative servers). To accurately measure this performance, we needed to be able to set up our client and resolver. The sub-research questions that were mentioned in chapter 1 gives us an idea of what kind of client we want to use. In the next section (section 3.2), we will elaborate on what features we wanted our client and server to have (these are based on our sub- research questions). After we have discussed this software and hardware, we will present the measurements that we ran.

3.2 Methodology

We will split the methodology into three parts: Software, Hardware and Measurements.

3.2.1 Software This section contains all the software we used to measure DNS and DoH performance. This software has a clear separation: Client- and Server software.

3.2.1.1 Client To measure the performance of DNS and DoH, we needed to find a program that could send queries over these protocols and could then measure the round-trip time. We also needed some extra options that this tool was able to provide: • Have a high-resolution clock for accurate measurements. This was brought up during the NLnet labs meetings. We needed to have a high-resolution clock so that we could see the performance differences in detail. • The ability to configure the total outgoing queries per second. This was so that we do not overload the network. • The ability to configure the number of HTTP/2 streams. We wanted this so we could compare the different configuration options that DoH can have. Streams is one of them. • The ability to read from a query dataset and send the queries in this dataset. We wanted the results to be easily reproducible.

29 • The ability to send the results to an output file for further analyzing. We wanted detailed metrics that we could analyze the way we want. When looking for a tool with NLnet Labs, we found two options that came close to our requirements: 1. DNSPERF by DNS-OARC [6] 2. Flamethrower by DNS-OARC [9] The problem with these tools was that they both had no DoH support; but because Flamethrower already had support for the similar DNS over TLS protocol, we decided to use this tool and extend it with DoH support1.

How Flamethrower works Flamethrower is a program that can send queries over DNS (UDP/TCP) and DNS over TLS (and also experimental support for DNS over QUIC). Flamethrower works by hav- ing a specified number of traffic generators that are firing once every second. Each of these “traffic generators” will have a batch of queries to send. The batch amount can be configured. The maximum number of queries 2 can be configured to a fixed amount, or can be configured to change over time. The results of Flamethrower can be output to a JSON file for further analysis. This JSON file will contain the minimum, average and maximum round-trip time for each batch of queries in ms. The traffic generators are called “generators” because they can generate unique requests. Flamethrower has the following options to generate traffic: • static: This generator is used by default. It generates queries by returning a fixed query name and query type. • file: This generator can read a file to use as query names and query types. • numberqname: Adds random numbers to the query name that has been set. • randompkt: Generates packet of random size, the number of packets and the max- imum size can be specified. • randomqname: Generates random query names. • randomlabel: Generate queries using random labels added to the query name that has been set. The number of queries to generate, the maximum length of a label and the maximum number of labels can be configured. We will use a dataset in our measurement. Because of this, we use the file option. Flamethrower expects a query file as input. This query file is a file in which each line is

1Even though DNSPERF had experimental DNS over TLS support, it has not been merged because of performance reasons [7] 2this is different from the batch amount, the batch amount is per traffic generator, the maximum number of queries is in total

30 a query name separated with a query type. An example query file is given below: Example 4: A query file

google.com A ru.nl A jwijenbergh.com A

Now that we have explained how Flamethrower works, we can talk about how we ex- tended the existing code to add the ability to send traffic using DoH.

Extending Flamethrower To explain how we extended the existing code for DoH support, we will first give an overview of how the code is structured. We will do this by going over each file (exclud- ing header files) and give a brief explanation on what it does: • main.cpp: This is the main file of the program, it parses the command-line argu- ments and starts the traffic generators. • metrics.cpp: This file collects the metrics (such as round-trip time, error codes). • query.cpp: This file provides functions to get the correct DNS packet for UD- P/TCP and TLS. • tcpsession.cpp: This file contains the code for sending and receiving a DNS over TCP request • tcptlssession.cpp: This file contains the code for sending and receiving a DNS over TLS request. • trafgen.cpp: This file contains functions for sending and receiving an UDP re- quest. It also has functions that call the appropriate session files for TCP (tcpses- sion/tcptlssession). The UDP and TCP sockets are added in these functions and are non-blocking sockets 3 that are provided by the Libuv [17] library. Now that we have seen how Flamethrower’s code has been structured, we can look at how our DoH implementation builds on top of that. After we have discussed this DoH implementation, we will describe how we extended the Flamethrower code to have more statistics. The full Flamethrower code can be found on our GitHub [11]. The hand- shake time branch is the full implementation (DNS over HTTPS support and extended statistics). The master branch is our implementation with only DNS over HTTPS. We made the extended statistics have a separate branch because we are planning on send- ing the DNS over HTTPS changes without the extended statistics to DNS-OARC by making a pull request.

3non-blocking sockets are sockets where you never have to wait for an operation to complete.

31 DoH Implementation We needed to change two things in the command-line parameters to implement DoH. We first needed to implement the ability to change the protocol to “https” (command-line -P option). All we needed to do for this is change the available options in main.cpp and parse it accordingly (including setting the default port for “https”).

Secondly, we needed to be able to distinguish between a GET and a POST request. We added the -M option to the parameters. The default value for this parameter is GET. This parameter is set as an enumeration, which can be found in the file http.h: 1 enum class HTTPMethod { 2 POST, 3 GET, 4 }; Listing 3.1: The enumeration for distinguishing between a GET/POST request in http.h

Because, in DoH, we are dealing with HTTPS URLs, we needed to parse the target differently: 1 struct http_parser_url parsed={}; 2 std::string url= raw_target_list[i]; 3 if(url.rfind("https://", 0)!= 0) { 4 url.insert(0,"https://"); 5 } 6 int ret= http_parser_parse_url(url.c_str(), strlen(url.c_str()), 0, &parsed); 7 if(ret!= 0) { 8 std::cerr<<"could not parse url:"<< url<< std::endl; 9 return1; 10 } Listing 3.2: Code for url parsing in main.cpp

The http parser parse url parses the URL and stores the result in a http parser url struct. This function is from a third-party library called llhttp [13]. This code is run for every target that is specified in the command-line. Because we now have more data to pass to other functions, we needed to store this data. We have done this by defining a Target structure that has been saved in the target.h file: 1 struct Target { 2 http_parser_url* parsed; 3 std::string address; 4 std::string uri; 5 }; Listing 3.3: The Target structure in target.h

As you can see, the structure contains the parsed URL, the IP-address and the unparsed URL (called uri here).

32 The next step was to change the start tcp session in the trafgen.cpp. This function takes care of sending DNS over TCP/TLS messages using a TCP socket. Sending DoH messages needed some changes in this function. We first allowed for the variable tcp session to also be an instance of the class httpssession (which needed to be implemented): 1 if(_traf_config->protocol== Protocol::TCP) { 2 _tcp_session= std::make_shared(_tcp_handle, malformed_data, ,→ got_dns_message, connection_ready); 3 } else if(_traf_config->protocol== Protocol::TCPTLS) { 4 _tcp_session= std::make_shared(_tcp_handle, malformed_data, ,→ got_dns_message, connection_ready, malformed_data); 5 } else{ 6 _tcp_session= std::make_shared(_tcp_handle, malformed_data, ,→ got_dns_message, connection_ready, malformed_data, current_target, ,→ _traf_config->method); 7 } Listing 3.4: Updated tcp session variable in trafgen.cpp

To send the DoH messages, we needed to prepare the packet one-by-one. In the existing code, the TCP and TLS batch were sent all at once (due to pipelining). To make this distinguishment, we added an if statement that checks which protocol is activated: 1 auto connection_ready=[this]() { 2 /** SEND DATA**/ 3 uint16_t id{0}; 4 std::vector id_list; 5 \textit{\textbf{Creating the DNS over HTTPS request}}\\ 6 \textit{\textbf{Sending the DNS over HTTPS request}}\\ 7 for(inti=0;i< _traf_config->batch_count; i++) { 8 if (_free_id_list.empty()) { 9 // out of ids, have to limit 10 break; 11 } 12 if (_rate_limit &&!_rate_limit->consume(1)) 13 break; 14 id= _free_id_list.back(); 15 _free_id_list.pop_back(); 16 assert(_in_flight.find(id)== _in_flight.end()); 17 id_list.push_back(id); 18 // might be better to do this after write(in WriteEvent) but it needs to be ,→ available 19 // by the time DataEvent fires, and we don’t wanta race there 20 _in_flight[id].send_time= std::chrono::high_resolution_clock::now(); 21 22 // Send one by one with DoH 23 if(_traf_config->protocol== Protocol::HTTPS) { 24 auto qt= (_traf_config->method== HTTPMethod::GET) 25 ? _qgen->next_base64url(id_list[i]) 26 : _qgen->next_udp(id_list[i]); 27 _tcp_session->write(std::move(std::get<0>(qt)), std::get<1>(qt)); 28 _metrics->send(std::get<1>(qt), 1, _in_flight.size());

33 29 } 30 } 31 32 if (id_list.size()== 0) { 33 // didn’t send anything, probably due to rate limit. close. 34 _tcp_handle->close(); 35 return; 36 } 37 38 if(_traf_config->protocol!= Protocol::HTTPS) { 39 auto qt= _qgen->next_tcp(id_list); 40 41 // async send the batch. fires WriteEvent when finished sending. 42 _tcp_session->write(std::move(std::get<0>(qt)), std::get<1>(qt)); 43 44 _metrics->send(std::get<1>(qt), id_list.size(), _in_flight.size()); 45 } 46 47 }; Listing 3.5: Sending a DoH packet in trafgen.cpp

As you can see, we distinguished the different HTTP methods (lines 22-24). For a GET request, we needed an encoded packet. When the method was a POST a simple UDP packet is needed. To implement this next base64url function we needed to add this to the file query.cpp. This function looks like the following: 1 QueryGenerator::QueryTpt QueryGenerator::next_base64url(uint16_t id) 2 { 3 WireTpt w= _wire_buffers[_reqs++ % _wire_buffers.size()]; 4 size_t len{w.second}; 5 auto buf= std::make_unique(len); 6 memcpy(buf.get(), w.first, w.second); 7 uint16_t _id= ntohs(id); 8 memcpy(buf.get(), &_id, sizeof(_id)); 9 std::string encoded= base64_encode((unsigned char*) buf.get(), len); 10 size_t encoded_len= encoded.size(); 11 auto encoded_buf= std::make_unique(encoded_len); 12 memcpy(encoded_buf.get(), encoded.c_str(), encoded_len); 13 return std::make_tuple(std::move(encoded_buf), encoded_len); 14 } Listing 3.6: The function for generating a base64url encoded DoH packet in query.cpp

Lines 3-8 are creating a regular UDP packet, the rest of the lines encodes this buffer by calling the base64 encode. This function is from a is from a library called cpp-base64 [10]. This library has been adapted to fit base64url encoding.

To create the httpssession.cpp file, we built on top of the already existing DNS over TLS code (tcptlssession.cpp). The only change in this setup is that Application- Layer Protocol Negotiation (ALPN) needed to be set up for negotiating the HTTP/2

34 protocol. Furthermore, instead of sending a plain DNS message directly over a TLS tunnel, HTTP/2 code needed to be implemented. To do this a library called LIBNGHTTP2 [12] was used. We will go over the implementation and explain what code we changed to get DNS over HTTPS to work.

First of all, we needed to change how TLS is set up. As said, we needed to add code to get ALPN to work: 1 gnutls_datum_t alpn; 2 alpn.data=(unsigned char*)"h2"; 3 alpn.size=2; 4 ret= gnutls_alpn_set_protocols(_gnutls_session, &alpn, 1, GNUTLS_ALPN_MANDATORY); 5 if (ret!= GNUTLS_E_SUCCESS) { 6 std::cerr<<"GNUTLS failed to set ALPN:"<< gnutls_strerror(ret)<< std::endl; 7 return false; 8 } Listing 3.7: Added ALPN code in the setup() function in httpssession.cpp

The data is set to h2 which indicates that the TLS connection is used for the HTTP/2 protocol. gnutls alpn set protcols uses the current TLS session to set the ALPN property as mandatory.

Next we needed to change how data was sent, this is done in the write function: 1 void HTTPSSession::write(std::unique_ptr data, size_t len) 2 { 3 int32_t stream_id; 4 http2_stream_data *stream_data= create_http2_stream_data(std::move(data), len); 5 std::string uri= stream_data->uri; 6 const struct http_parser_url *u= stream_data->u; 7 std::string current_path= stream_data->path; 8 if(_method== HTTPMethod::GET) { 9 nghttp2_nv hdrs[]={ 10 MAKE_NV2(":method","GET"), 11 MAKE_NV(":scheme", &uri[u->field_data[UF_SCHEMA].off], u->field_data[UF_SCHEMA] ,→ .len), 12 MAKE_NV(":authority", stream_data->authority.c_str(), stream_data->authority. ,→ size()), 13 MAKE_NV(":path", current_path.c_str(), current_path.size()), 14 MAKE_NV2("accept","application/dns-message"), 15 }; 16 //print_headers(stderr, hdrs, ARRLEN(hdrs)); 17 stream_id= nghttp2_submit_request(_current_session->session, NULL, hdrs, ARRLEN( ,→ hdrs), NULL, stream_data); 18 } else{ 19 nghttp2_nv hdrs[]={ 20 MAKE_NV2(":method","POST"), 21 MAKE_NV(":scheme", &uri[u->field_data[UF_SCHEMA].off], 22 u->field_data[UF_SCHEMA].len),

35 23 MAKE_NV(":authority", stream_data->authority.c_str(), stream_data->authority. ,→ size()), 24 MAKE_NV(":path", current_path.c_str(), current_path.size()), 25 MAKE_NV2("accept","application/dns-message"), 26 MAKE_NV2("content-type","application/dns-message"), 27 MAKE_NV("content-length", std::to_string(len).c_str(), std::to_string(len).size ,→ ()) 28 }; 29 //print_headers(stderr, hdrs, ARRLEN(hdrs)); 30 nghttp2_data_provider provider; 31 provider.read_callback= post_data; 32 stream_id= nghttp2_submit_request(_current_session->session, NULL, hdrs, ARRLEN( ,→ hdrs), &provider, stream_data); 33 } 34 if (stream_id< 0) { 35 std::cout<<"Could not submit HTTP request:"<< nghttp2_strerror(stream_id); 36 } 37 38 stream_data->stream_id= stream_id; 39 40 if(session_send()!= 0) { 41 std::cerr<<"failed to send"<< std::endl; 42 } 43 } Listing 3.8: Sending a DoH request in httpssession.cpp

This function has an if statement that checks what HTTP method is used. The MAKE NV and MAKE NV2 functions are used to construct the relevant structs that are used for storing the HTTP headers. These headers are then given to the nghttp2 submit request function, which executes the necessary callbacks that were set up. However, for a POST request, we need to first add the data to the body of the request, this is done by the function post data: 1 static ssize_t post_data(nghttp2_session *session, int32_t stream_id, uint8_t *buf, ,→ size_t length, uint32_t *data_flags, nghttp2_data_source *source, void *user_data ,→ ){ 2 auto stream_data= static_cast( ,→ nghttp2_session_get_stream_user_data(session, stream_id)); 3 size_t nread= std::min(stream_data->data.size(), length); 4 memcpy(buf, stream_data->data.c_str(), nread); 5 *data_flags= NGHTTP2_DATA_FLAG_EOF; 6 return nread; 7 } Listing 3.9: Copying the POST data into a buffer for nghttp2 to use, httpssession.cpp

We see that this is simply copying the stream data data pointer into a buf variable, which is internally used by the nghttp2 library. The callback that sends the DoH data is defined by:

36 1 static ssize_t send_callback(nghttp2_session *session, const uint8_t *data, size_t ,→ length, int flags, void *user_data) 2 { 3 HTTPSSession *class_session= (HTTPSSession *)user_data; 4 class_session->send_tls((void*) data, length); 5 return (ssize_t)length; 6 } Listing 3.10: The callback for sending the HTTP/2 data in httpssession.cpp

All this function does is simply using the send tls function to send the data. It then returns the length that has been sent. The send tls function is simply the old write function that is used for the tcptlssession.cpp file.

Now that we have given an overview of sending data, we need to take a look at receiving data. Receiving data is done in the receive data function: 1 void HTTPSSession::receive_data(const char data[], size_t _len) 2 { 3 _pull_buffer.append(data, _len); 4 switch(_tls_state) { 5 case LinkState::HANDSHAKE: 6 do_handshake(); 7 break; 8 case LinkState::DATA: 9 char buf[2048]; 10 for(;;){ 11 ssize_t len= gnutls_record_recv(_gnutls_session, buf, sizeof(buf)); 12 if (len> 0) { 13 receive_response(buf, len); 14 } else{ 15 if(len== GNUTLS_E_AGAIN) { 16 // Check if we don’t have any data left to read 17 if(_pull_buffer.empty()) { 18 break; 19 } 20 continue; 21 } else if(len== GNUTLS_E_INTERRUPTED) { 22 continue; 23 } 24 break; 25 } 26 } 27 break; 28 case LinkState::CLOSE: 29 break; 30 } 31 } Listing 3.11: Receiving HTTP/2 data in httpssession.cpp

37 This function was yet again taken from the tcptlssession.cpp file. However, changes needed to be made to receive the data properly. When an error occurs (if(len ≤ 0)) we needed to check which error it was. If GNUTLS E AGAIN has happened, we only continue reading data if the buffer is not empty (meaning there is data to read). We also needed to continue reading data if the GNUTLS E INTERRUPTED error has been set. If we did not include this portion (as it was done in the original function), we were having issues when we received data that was not HTTP/2 data (such as TLS session tickets). The rest of the httpsession.cpp is the same principle as the tcptlssession.cpp file.

Adding statistics In the existing Flamethrower code, each batch of queries will have the following output in the file that we have specified with the -o flag: 1 { 2 "in_flight": 1, 3 "period_bad_count": 0, 4 "period_net_errors": 0, 5 "period_r_count": 10, 6 "period_response_avg_ms": 89.03025109999999, 7 "period_response_max_ms": 89.042695, 8 "period_response_min_ms": 89.020198, 9 "period_s_count": 10, 10 "period_tcp_connections": 1, 11 "period_timeouts": 0, 12 "pkt_size_avg": 56, 13 "responses":{ 14 "NOERROR": 10 15 }, 16 "run_id":"7ffdeb62a110", 17 "runtime_s": 112.72691444700001, 18 "trafgen_id":"40169" 19 } Listing 3.12: Flamethrower default JSON file output

For our measurements (see section 3.2.3), we wanted to have the round-trip time for every query (not just an average, minimum and maximum). On top of this, we needed to analyze the delay for setting up a TLS and TCP connection. To realize these statistics, we extended the Flamethrower code. We will discuss how we did this by going over each changed file (excluding header files).

In metrics.cpp, we added the variables period tcp latency handshake, period tls latency handshake and period response arr ms. The first two vari- ables are for the TCP and TLS connection delays respectively. The last variable, period response arr ms, is an array that has every round-trip time for the current batch stored in it. To fill this array of round-trip times, we needed to add every la- tency when the receive function is called in metrics.cpp. To set the TCP and TLS connection delays, we added the following two functions:

38 1 void Metrics::set_tcp_handshake(const std::chrono::high_resolution_clock::time_point & ,→ tcp) 2 { 3 auto now= std::chrono::high_resolution_clock::now(); 4 auto tcp_latency= now- tcp; 5 double tcp_latency_ms= tcp_latency.count() * Metrics::HR_TO_MSEC_MULT; 6 _period_tcp_latency_handshake= tcp_latency_ms; 7 } 8 9 void Metrics::set_tls_handshake(const std::chrono::high_resolution_clock::time_point & ,→ tls) 10 { 11 auto now= std::chrono::high_resolution_clock::now(); 12 auto tls_latency= now- tls; 13 double tls_latency_ms= tls_latency.count() * Metrics::HR_TO_MSEC_MULT; 14 _period_tls_latency_handshake= tls_latency_ms; 15 } Listing 3.13: Functions for setting the TLS and TCP connection delay in metrics.cpp

As can be seen above, these two functions are almost identical. They set the TCP and TLS latency by computing the difference of the initial time point as compared to the current time. To compute this initial time point we needed to add the appropriate code to httpssession.cpp and trafgen.cpp. We will first discuss the changes that have been made in trafgen.cpp and then continue on with the changes in httpssession.cpp. For trafgen.cpp, we needed to keep track of the starting time of the TCP and TLS connection. Changes for this were made in the start tcp session function. Before our Libuv socket calls the connect function (which initiates a TCP connection), we get the current time with std::chrono::high resolution clock::now() and store the returned value in a variable called tcp handshake time. When the Libuv socket is at a ConnectEvent (which means that a TCP connection has been established) the following happens: 1 // SOCKET: on connect 2 _tcp_handle->on([this](uvw::ConnectEvent &event, uvw::TcpHandle &h) { 3 _tls_handshake_time= std::chrono::high_resolution_clock::now(); 4 _tcp_session->on_connect_event(); 5 _metrics->set_tcp_handshake(_tcp_handshake_time); 6 _metrics->tcp_connection(); 7 8 // start reading from incoming stream, fires DataEvent when receiving 9 _tcp_handle->read(); 10 }); Listing 3.14: The ConnectEvent for the Libuv socket with extended statistics in trafgen.cpp

39 In line 3, we see that we save the TLS time point in a variable called tls handshake time. This is done immediately after a TCP connection has been established because line 4 initiates the TLS tunnel. Line 5 is the function that we have discussed in Listing 3.13. Now that we have the starting latency of the TLS tunnel, we need to call the set tls handshake from Listing 3.13. To implement this, we have created the following callback: 1 auto got_handshake=[this]() { 2 _metrics->set_tls_handshake(_tls_handshake_time); 3 }; Listing 3.15: The callback for in trafgen.cpp

This callback is then passed to httpssession.cpp and is called whenever the TLS handshake finishes (line 5): 1 void HTTPSSession::do_handshake() 2 { 3 int err= gnutls_handshake(_gnutls_session); 4 if (err== GNUTLS_E_SUCCESS) { 5 _got_handshake(); 6 . 7 . 8 . Listing 3.16: The TLS handshake callback in httpssession.cpp

3.2.1.2 Resolver In this section, we will explain what software we used to set up the resolver. We wanted the same server software that provides DoH as well as unencrypted DNS because this would assure us that the actual DNS resolving had the same implementation for both protocols. Like the client tool, we also wanted some additional requirements: • The software must be used in production • Must be highly configurable • High performance We found that Dnsdist was the best to use for our case. Dnsdist is a DNS load balancer made by PowerDNS [5]. A load balancer means that the software acts as a middle layer between the client and a (set of) server(s). The load balancer chooses the best server for each client. Dnsdist allows forwarding of DNS over HTTPS and regular DNS to a DNS resolver. This means that in this case, Dnsdist only forwards the traffic to a specified server, but doesn’t do any resolving itself. That’s why we needed a DNS resolver on top of that. We used Unbound by NLNetLabs for this. Unbound is “a validating, recursive, caching DNS resolver. It is designed to be fast and lean and incorporates modern fea- tures based on open standards” [19]. It was made by NLnet Labs which made it useful for us to use.

40 Deploying across multiple servers We wanted to run our tests simultaneously for DNS and DoH. This means that we needed two of the same setups. We wanted to easily deploy the server software across these two instances. For this we used Docker. Docker is a container software that al- lowed us to write a configuration that would set up and install Dnsdist and Unbound. This configuration looks like the following: 1 version:’3’ 2 3 networks: 4 dns_network: 5 ipam: 6 driver: default 7 config: 8 - subnet: 172.20.0.0/24 9 10 services: 11 dnsdist: 12 build:’./dnsdist’ 13 tty: true 14 volumes: 15 -./dnsdist/config/dnsdist.conf:/etc/dnsdist/dnsdist.conf:ro 16 - /etc/letsencrypt/:/etc/letsencrypt/:ro 17 ports: 18 -’53:53/tcp’ 19 -’53:53/udp’ 20 -’443:443/tcp’ 21 -’853:853/tcp’ 22 networks: 23 dns_network: 24 ipv4_address: 172.20.0.5 25 depends_on: 26 - unbound 27 unbound: 28 image:’secns/unbound’ 29 environment: 30 - DO_IPV6=no 31 - DO_IPV4=yes 32 - DO_UDP=yes 33 - DO_TCP=yes 34 - VERBOSITY=0 35 - NUM_THREADS=4 36 - SO_RCVBUFF=0 37 - SO_SNDBUF=0 38 - SO_REUSEPORT=no 39 - EDNS_BUFFER_SIZE=4096 40 - MSG_CACHE_SIZE=4m 41 - RRSET_CACHE_SIZE=4m 42 - CACHE_MIN_TTL=86400 43 - CACHE_MAX_TTL=86400 44 - CACHE_MAX_NEGATIVE_TTL=86400

41 45 - HIDE_IDENTITY=no 46 - HIDE_VERSION=no 47 - STATISTICS_INTERVAL=0 48 - STATISTICS_CUMULATIVE=no 49 - EXTENDED_STATISTICS=no 50 networks: 51 dns_network: 52 ipv4_address: 172.20.0.6 Listing 3.17: Docker compose file for setting up the DNS/DoH resolver in Dockerfile

Here the unbound Docker image is provided by Docker hub [20], however, we have adapted this image for the latest unbound version (1.9.3). The Dnsdist image we made ourselves, because the DoH support in Dnsdist is not in the release version yet. This Docker image is the following: 1 # Use alpine 2 FROM alpine as builder 3 4 # Set working dir 5 WORKDIR /dnsdist 6 7 # Update 8 RUN apk--update upgrade 9 10 # Install libraries 11 RUN apk add \ 12 boost-dev \ 13 fstrm-dev \ 14 gnutls-dev \ 15 lua5.3-dev \ 16 libedit-dev \ 17 libuv-dev \ 18 openssl-dev \ 19 net-snmp-dev \ 20 protobuf-dev \ 21 libsodium-dev \ 22 re2-dev \ 23 yaml-dev \ 24 wslay-dev \ 25 zlib-dev 26 27 # Install needed utilities for building 28 RUN apk add \ 29 cmake \ 30 curl \ 31 g++ \ 32 gnupg \ 33 make \ 34 perl 35 36 ENV H2O_VERSION 2.2.6 37 ENV DNSDIST_VERSION 1.4.0-rc2

42 38 39 # Get libh2o from source 40 RUN curl-LO https://github.com/h2o/h2o/archive/v${H2O_VERSION}.tar.gz&&\ 41 tar xzf v${H2O_VERSION}.tar.gz && \ 42 rm v${H2O_VERSION}.tar.gz 43 44 # Install libh2o 45 RUN cd h2o-${H2O_VERSION} && \ 46 cmake \ 47 -DCMAKE_INSTALL_PREFIX=/usr \ 48 -DCMAKE_INSTALL_LIBDIR=/usr/lib \ 49 -DWITHOUT_LIBS=off \ 50 -DBUILD_SHARED_LIBS=off \ 51 .&&\ 52 make && \ 53 make libh2o && \ 54 make install && \ 55 cd.. && rm-r h2o-${H2O_VERSION} 56 57 # Get archive and verify 58 ## create directory 59 RUN mkdir-v-m 0700-p /root/.gnupg 60 61 ## import keys 62 RUN curl-O"https://dnsdist.org/_static/dnsdist-keyblock.asc"&&\ 63 gpg2--import dnsdist-keyblock.asc 64 65 ## Verify and extract archive 66 RUN curl-O"https://downloads.powerdns.com/releases/dnsdist-${DNSDIST_VERSION}.tar.bz2 ,→ {,.asc,.sig}"&&\ 67 gpg2--verify dnsdist-${DNSDIST_VERSION}.tar.bz2.sig && \ 68 tar xjf dnsdist-${DNSDIST_VERSION}.tar.bz2 && \ 69 rm *.asc *.sig dnsdist-${DNSDIST_VERSION}.tar.bz2 && \ 70 rm-r /root/.gnupg 71 72 # build 73 RUN cd dnsdist-${DNSDIST_VERSION} && \ 74 ./configure--sysconfdir=/etc/dnsdist--mandir=/usr/share/man \ 75 --enable-dns-over-https--enable-dns-over-tls--with-libsodium--enable-dnstap--with- ,→ re2--with-net-snmp && \ 76 make-j2 && \ 77 make install && \ 78 cd.. && rm-r dnsdist-${DNSDIST_VERSION} 79 80 FROM alpine 81 82 COPY./config/dnsdist.conf /etc/dnsdist/dnsdist.conf 83 COPY--from=builder /usr/local/bin /usr/local/bin/ 84 COPY--from=builder /usr/lib /usr/lib 85 COPY--from=builder /usr/share/man/man1 /usr/share/man/man1/ 86 87 RUN addgroup-S dnsdist && \ 88 adduser-S-D-G dnsdist dnsdist

43 89 90 CMD ["dnsdist","--verbose"] Listing 3.18: Docker image for Dnsdist version 1.4.0-rc2 in docker-compose.yml

The complete Docker configuration can be found at our GitHub [14]. To get the servers to have the same initial configuration we used snapshots that were provided by our hosting service (Digitalocean).

We have now described our methodology for setting up the client and the server. We will show in the next section on what hardware we ran this software.

3.2.2 Hardware In this section, we will give the hardware specifications for the client and the server.

3.2.2.1 Client To run our Flamethrower application, we needed a client. The client that we used to run Flamethrower on is a laptop with the following specifications:

• Processor: Intel Core i5-6200U • Graphics Card: Intel integrated graphics • RAM: 8GB DDR3 • OS: NixOS Linux 19.03 • Connection: Wireless • Location: Radboud University Nijmegen, Netherlands

3.2.2.2 Server To host the resolver (Dnsdist + Unbound), a VPS 4 was rented from Digitalocean with the following specifications: • Processor: 2 vCPUs • RAM: 4GB • Transfer: 4TB • Location: Amsterdam, Netherlands

4A VPS is a virtual private server, it is a virtual machine hosted on a server

44 3.2.3 Performance measurement This section will contain all the information needed for measuring the performance of DoH and unencrypted DNS. We will first talk about the dataset we used to run tests on, next we will talk about exactly what tests we ran against this dataset. At last, we will describe our automated way of running these tests.

3.2.3.1 Dataset For our measurements, we decided that we needed a dataset of the most popular domain names. We used the Alexa top websites dataset. This download was accessed and down- loaded on Tuesday 17 September [1]. This file contained the top 1 Million domain names sorted by popularity. For the measurements, we used the top 10,000 domain names. To change this data to the query file format that Flamethrower uses (see Example 4), we added the query type A to each line of the dataset.

3.2.3.2 Tests As said in section 3.2.2.1, the tests were performed on a public network (the network of the Radboud University). To not disturb the network, we needed to limit the total out- going queries per second. Our Flamethrower extension supported 100 outgoing queries per second by default for DoH. We decided to use this number as the number of queries that we wanted to use for each Flamethrower client that we had running. With this limitation in mind, we came up with the following tests that we wanted to run:

Test Queries per second Concurrent traffic generators Number of queries per traffic generator (HTTP/2 streams) 1 100 1 100 2 100 2 50 3 100 4 25 4 100 5 20 5 100 10 10 6 100 20 5 7 100 25 4 8 100 50 2 9 100 100 1

Table 3.1: The tests for measuring DNS and DoH performance

These tests were based on our sub-research questions that were mentioned in chapter 1: • How do the HTTP methods POST and GET compare with regards to perfor- mance? • To what extent do TCP connections impact the performance of DNS over HTTPS? • To what extent do HTTP/2 streams impact the performance of DNS over HTTPS? • To what extent does caching impact the performance of DNS over HTTPS?

45 Now that we have shown the tests that we ran, we will explain how we ran these tests using scripts.

3.2.3.3 Running the tests To run these tests automatically, we wrote simple Bash scripts that gave the command- line arguments to our Flamethrower implementation. The script that could accept these command-line arguments and start the appropriate flamethrower process is the following: 1 echo"Doing udp on dns1.jwijenbergh.com and https GET on dns2.jwijenbergh.com" 2 echo"flamethrower command:-q $1-Q $2-c $3-o $4" 3 build/flame-n 1-q $1-Q $2-c $3-P udp-f~/measuring/queryfile dns1.jwijenbergh.com ,→ -o $4/udp.json &>/dev/null & 4 build/flame-n 1-q $1-Q $2-c $3-P https-M GET-f~/measuring/queryfile dns2. ,→ jwijenbergh.com-o $4/https-get.json &>/dev/null & 5 6 wait 7 echo"First round done" 8 echo"Waiting 10 secs..." 9 sleep 10s 10 11 12 echo"Doing udp on dns1.jwijenbergh.com and https POST on dns2.jwijenbergh.com" 13 echo"flamethrower command:-q $1-Q $2-c $3-o $4" 14 15 build/flame-n 1-q $1-Q $2-P udp-f~/measuring/queryfile dns1.jwijenbergh.com &>/dev ,→ /null & 16 build/flame-n 1-q $1-Q $2-P https-M POST-f~/measuring/queryfile dns2.jwijenbergh. ,→ com-o $4/https-post.json &>/dev/null & 17 18 wait 19 echo’Done!’ Listing 3.19: Bash script for running flamethrower in send.sh

Here $1 represents the total number of queries per generator, for HTTPS this is the number of HTTP/2 streams. $2 is the maximum queries per second, which for these tests is set to 100. $3 is the total number of concurrent traffic generators. The dataset that we described in section 3.2.3.1 is located at ~/measuring/queryfile.

Each of these tests (Table 3.1) we ran with caching on and off. The tests were executed two times (we will call each time a round) with the following two resolver configurations:

46 Configuration UDP resolved on DNS over HTTPS resolved on 1 dns1.jwijenbergh.com dns2.jwijenbergh.com 2 dns2.jwijenbergh.com dns1.jwijenbergh.com

Table 3.2: The two different resolver configurations used for measuring DNS and DoH performance

Round 1 and 4 used resolver configuration 1 and round 2 and 3 used resolver configura- tion 2. This brings the total number of tests we ran to: 2 (two resolver configurations) × 2 (caching on and caching off) × 2 (two times for every configuration) × 9 (the tests specified in Table 3.1) = 36.

The script that loops send.sh for each round is given below: 1 #!/usr/bin/env bash 2 3 if[-d"$1"]; then 4 echo"DIRECTORY ALREADY EXISTS, EXITING..." 5 exit 6 fi 7 8 mkdir-p $1 9 10 # Array declarations 11 declare-a concurrent_generators=(1 2 4 5 10 20 25 50 100 10) 12 declare-a batch_counts=(100 50 25 20 10 5 4 2 1) 13 declare-a test_names=("test1""test2""test3""test4""test5""test6""test7""test8"" ,→ test9") 14 15 total_tests=${#test_names[@]} 16 17 for (( i=0;i<${total_tests}; i++ )); 18 do 19 mkdir-p"$1/${test_names[$i]}" 20 echo"STARTING EXPERIMENT" $((i+1)) 21 ./scripts/send.sh"${batch_counts[$i]}" 100"${concurrent_generators[$i]}""$1/${ ,→ test_names[$i]}" 22 echo"WAITING..." 23 sleep 10s 24 done Listing 3.20: Bash script for running the tests in start tests.sh

To enable caching, we first filled the cache by sending all the 10,000 domain names to the resolver. This script was called fillcache.sh and is given below: 1 #!/usr/bin/env bash 2

47 3 echo"Filling cache...." 4 build/flame-n 1-Q 100-c 2-P udp-f~/measuring/queryfile dns1.jwijenbergh.com &>/dev ,→ /null & 5 build/flame-n 1-Q 100-c 2-P udp-f~/measuring/queryfile dns2.jwijenbergh.com &>/dev ,→ /null & 6 wait 7 echo"Done filling the cache" Listing 3.21: Bash script for filling Unbound cache in fillcache.sh

These bash scripts can be found on our GitHub [14]. To ensure that the messages stayed in cache we set the following options for Unbound in our resolver docker-compose.yml file (as mentioned in section 3.2.1.2). 1 - MSG_CACHE_SIZE=256M 2 - RRSET_CACHE_SIZE=512M 3 - CACHE_MIN_TTL=86400 4 - CACHE_MAX_TTL=86400 5 - CACHE_MAX_NEGATIVE_TTL=86400 Listing 3.22: Settings to enable caching for Unbound

This allowed for the query names to stay in the Unbound cache for 24 hours. Even though Dnsdist also has an option to store data in its cache, this is disabled by default. We only used the Unbound cache when necessary. To disable the Unbound cache, we changed the cache sizes to 0, which means that the cache has no size to store any query names.

3.2.3.4 Analyzing the output These results are then output to a file with the -o flag in Flamethrower. In our imple- mentation of Flamethrower, this file has the following output for each batch of queries: 1 { 2 "in_flight": 1, 3 "period_bad_count": 0, 4 "period_net_errors": 0, 5 "period_r_count": 10, 6 "period_response_arr_ms":[ 7 89.042695, 8 89.037795, 9 89.035502, 10 89.033074, 11 89.030431, 12 89.028888, 13 89.02617099999999, 14 89.02474, 15 89.023017, 16 89.020198 17 ], 18 "period_response_avg_ms": 89.03025109999999,

48 19 "period_response_max_ms": 89.042695, 20 "period_response_min_ms": 89.020198, 21 "period_s_count": 10, 22 "period_tcp_connections": 1, 23 "period_timeouts": 0, 24 "pkt_size_avg": 56, 25 "responses":{ 26 "NOERROR": 10 27 }, 28 "run_id":"7ffdeb62a110", 29 "runtime_s": 112.72691444700001, 30 "tcp_handshake": 21.315645999999997, 31 "tls_handshake": 18.697547999999998, 32 "trafgen_id":"40169" 33 } Listing 3.23: Flamethrower JSON file output

This file is a JSON file. We parse this JSON file with a python script that also plots graphs using the library Matplotlib [18]. The full python script can be found on our GitHub [14].

3.2.3.5 Overview Before we talk about the results that were obtained, we will give an overview similar to Figure 3.1:

Figure 3.2: Experimental setup with our Flamethrower and Dnsdist applications

This overview now contains all of the applications that we have discussed in section 3.2.1 and section 3.2.2.

49 3.3 Results

As we have mentioned in the last section, we will now share the results we got from running the tests in Table 3.1. The first statistic we will analyze is the delay of a TCP and TLS connection. This will be given in section section 3.3.1 and section 3.3.2 respectively. After this, we will continue with the round-trip times for DNS and DoH. For the TCP and TLS latencies we will give a figure for the two resolvers combined. For the round-trip time measurements, we will have a simple bar graph for each resolver and combine the two resolvers into a single figure. The combination of the two resolvers will also be used to plot a distribution function that will be explained later on. As we have mentioned in the previous section, the result for each resolver configuration has been run two times (two rounds). In this section, we have combined the rounds for each resolver configuration. The full results for each round can be found in Appendix A.

3.3.1 TCP connection delay We will go over the results in two separate sections: Caching enabled and Caching disabled. We ran the tests for HTTPS POST and HTTPS GET, which means that we will give the TCP latencies for each HTTP method.

3.3.1.1 Caching enabled

HTTPS GET HTTPS POST

Figure 3.3: Average time for establishing a TCP connection on dns1.jwijenbergh.com and dns2.jwijenbergh.com caching enabled

Here (Figure 3.3), we see that when the total number of concurrent generators increases, the time it takes for a TCP connection to be set up also increases. It seems to be that our Dnsdist application is having a hard time keeping up with the concurrent TCP connections. The default number of TCP threads for our Dnsdist resolvers is 10. If all these threads are busy then a TCP connection is queued in Dnsdist.

50 This can be seen in the graphs because the TCP delay starts to increase starting at Test 5. Test 5 is the first test that has 10 or more concurrent traffic generators. Test 6, 7, 8 and 9 all have increasingly more than 10 traffic generators, which can be observed by an increased TCP latency. We could have increased the maximum number of TCP threads, however, this would have also increased the resource usage by our server. When looking at the tests that all have 10 or fewer concurrent generators (Test 1, Test 2, Test 3 and Test 4 ), we see that these tests are closer in terms of milliseconds.

3.3.1.2 Caching disabled We will now show the same figure, but with caching disabled in Unbound:

HTTPS GET HTTPS POST

Figure 3.4: Average time for establishing a TCP connection on dns1.jwijenbergh.com and dns2.jwijenbergh.com caching disabled

We see that we get similar results as in section 3.3.1.1, however, Tests 5-7 do not have the same delay as can be seen in Figure 3.3. We also observe that Test 8 and Test 9 have a smaller TCP delay than with caching turned on. A possible explanation for these two observations is that with caching turned off, the DoH queries take a longer time (this will be discussed in section 3.3.3). This can change the TCP delay because the Dnsdist server needs to wait more for Unbound to respond, which allows the Dnsdist resolver to empty the TCP connection queue faster. Now that we have seen what the delay is for establishing a TCP connection, we will talk about the TLS connection delay.

3.3.2 TLS connection delay Like the previous section, we will yet again split this section into two parts: caching enabled and caching disabled. We will, again, have a separate graph for each HTTP

51 method.

3.3.2.1 Caching enabled With caching turned on we got the following results:

HTTPS GET HTTPS POST

Figure 3.5: Average time for establishing a TLS tunnel on dns1.jwijenbergh.com and dns2.jwijenbergh.com caching enabled

We see similar results as in section 3.3.1. We see yet again that when the number of concurrent generators increases, the time it takes for a TLS tunnel to be set up is also increased. This is expected because TLS operates over (the already established) TCP connection to create the encrypted tunnel. This means that the same TCP queue will be filled in Dnsdist.

52 3.3.2.2 Caching disabled When we turned caching off, the following results were obtained:

HTTPS GET HTTPS POST

Figure 3.6: Average time for establishing a TLS tunnel on dns1.jwijenbergh.com and dns2.jwijenbergh.com caching disabled

For Tests 1-7 we see the same pattern as when we measured the TCP delay with caching disabled (see section 3.3.1.2). However, for Test 8, we now see a performance decrease. In the next section, we will talk about the round-trip time for DNS and DNS over HTTPS queries over the established TCP/TLS connection.

3.3.3 Round-trip time of DNS and DoH Equivalent to the previous two sections, we will present the results for caching enabled and caching disabled. We will give a bar plot that plots the mean for the round-trip times of DNS and DNS over HTTPS. On top of this, we will give the absolute difference that has been obtained by subtracting the mean DNS over HTTPS round-trip time and the mean UDP round-trip time. To get more info on how these round-trip times are distributed, we will give a graph that plots the empirical cumulative distribution function (ECDF). This graph is read as follows: the y-axis represents the probability of the protocol having the corresponding value or smaller on the x-axis. A better performing protocol (meaning less round-trip time) will be closer to the top of the graph.

53 3.3.3.1 Caching enabled We have obtained the following results when caching is enabled:

(a) Average round-trip time

(b) Average round-trip time difference as compared to UDP

Figure 3.7: Round-trip time measurements for dns1.jwijenbergh.com with caching enabled

54 (a) Average round-trip time

(b) Average round-trip time difference as compared to UDP

Figure 3.8: Round-trip time measurements for dns2.jwijenbergh.com with caching enabled

55 (a) Average round-trip time

(b) Average round-trip time difference as compared to UDP

Figure 3.9: Round-trip time measurements for dns1.jwijenbergh.com and dns2.jwijenbergh.com with caching enabled

56 When we look at the results for dns1.jwijenbergh.com (Figure 3.7), we see that for ev- ery test UDP is the fastest protocol. The same result is obtained for dns2.jwijenbergh.com (Figure 3.8). HTTPS POST and HTTPS GET perform similarly except for Test 5. With this test, we see that HTTPS POST has less round-trip time as compared to HTTPS GET. A possible explanation for this is that the HTTPS GET packets still need to be decoded at the receiver, which adds additional latency. On top of this, HTTPS POST packets are smaller than HTTPS GET packets [35]. However, because we do not see this performance difference as clearly with the rests of our tests, an external factor can also be the case (such as packet loss). To see what tests perform the best and which tests perform the worse, we have combined the results from both of our resolvers (see Figure 3.9). In this figure, we can observe that Test 4 has the least average round-trip time and Test 8 has the most average round- trip time (when we take the average of the HTTPS GET and HTTPS POST round-trip times). When looking at the combined results from both of our resolvers (Figure 3.9), we notice that Tests 6-8 clearly have the highest average round-trip times. These tests all have more than 10 TCP connections, which means that this can yet again be attributed to the fact that our Dnsdist TCP queue is set to 10. Test 1-4 are similar when we look at the round-trip time. To get more info on how these round-trip times are distributed we will give a figure that plots the empirical cumulative distribution function (ECDF) for each test. This figure can be found on the next page and will be discussed next.

57 Figure 3.10: Test 1 Figure 3.11: Test 2 Figure 3.12: Test 3

Figure 3.13: Test 4 Figure 3.14: Test 5 Figure 3.15: Test 6

Figure 3.16: Test 7 Figure 3.17: Test 8 Figure 3.18: Test 9

Figure 3.19: ECDF graph with caching enabled on dns1.jwijenbergh.com and dns2.jwijenbergh.com averaged

58 As we have mentioned in the introduction of section 3.3.3, we are looking for the protocol that has a curve that is closest to the top of each plot. When looking at these graphs, we notice that UDP has this property. For HTTPS, the tests that have less than 10 concurrent generators (Test 1, 2, 3, 4 ) have a similar curve. These tests have the steepest curve for HTTPS, which means that these tests perform the best. For Test 5, we see once again that HTTPS POST performs better than HTTPS GET.

3.3.3.2 Caching disabled We will now discuss the same graphs, but with caching disabled. We will start once again with a simple bar plot of the mean round-trip time and continue with the empirical cumulative distribution function.

59 (a) Average round-trip time

(b) Average round-trip time difference as compared to UDP

Figure 3.20: Round-trip time measurements for dns1.jwijenbergh.com with caching disabled

60 (a) Average round-trip time

(b) Average round-trip time difference as compared to UDP

Figure 3.21: Round-trip time measurements for dns2.jwijenbergh.com with caching disabled

61 (a) Average round-trip time

(b) Average round-trip time difference as compared to UDP

Figure 3.22: Round-trip time measurements for dns1.jwijenbergh.com and dns2.jwijenbergh.com with caching disabled

62 We have yet again combined the results of both resolvers in a single graph (Figure 3.22). In the combined results, we observe a performance increase in Test 4 and Test 5 when HTTPS is used (except for HTTPS GET on Test 5 ). The HTTPS GET performance decrease as compared to HTTPS POST is still present in Test 5. However, it is not as drastic as we have described in section 3.3.3.1. When we compare the rest of the tests to the results that we have seen when caching was enabled, we observe that Test 6 and Test 7 have less performance decrease as compared to UDP. The difference for the two resolvers (Figure 3.9) in Test 1, Test 2, Test 3 and Test 8 between UDP and HTTPS is larger (a performance decrease) than with caching enabled. To gain more insight into what the distribution looks like, we will discuss the empirical distribution functions for all these tests.

63 Figure 3.23: Test 1 Figure 3.24: Test 2 Figure 3.25: Test 3

Figure 3.26: Test 4 Figure 3.27: Test 5 Figure 3.28: Test 6

Figure 3.29: Test 7 Figure 3.30: Test 8 Figure 3.31: Test 9

Figure 3.32: ECDF graph with caching enabled on dns1.jwijenbergh.com and dns2.jwijenbergh.com averaged

64 These results are different than when caching is enabled (see Figure 3.19). We can see here that the distribution for HTTPS is now closer to that of UDP. This can be seen easily in Test 4. In this test, HTTPS comes slightly above UDP. This phenomenon can also be seen in Test 5, but only with HTTPS POST. The test that has the worst distribution is Test 8. Furthermore, Test 1 and Test 2 both have a clearly worse distribution than with UDP. Test 3 comes closer but is still under the curve of UDP. The remaining tests (Test 6, Test 7 and Test 9 ) also perform worse than with UDP.

3.4 Reflecting on the results and limitations

To wrap up this section, we will give an overview of our results and discuss them in regard to our main research question that we have defined in chapter 1 and section 3.1. The main research question was the following: • How much performance impact does DNS over HTTPS have as compared to using unencrypted DNS? In this section, we have seen that when caching was enabled we got a worse distribution for HTTPS and a worse average round-trip time than with UDP. When caching was disabled we got the same results, except for Test 4 and Test 5 where DNS over HTTPS performed better. However, as we have said in section 3.2, our Flamethrower application also needs to set up a TCP and TLS connection. The latency for this combined is at least 30 milliseconds for Test 4 and Test 5 as can be seen in Figure 3.3 and Figure 3.5. When we add this latency to the round-trip time difference for these two tests, we notice that DNS over HTTPS is still slow as compared to UDP. Even though this TCP and TLS latency is not for every query but for every batch of queries, it is still an added latency that is not present when UDP is used. In our methodology, there are still some limitations that can be found: • We only test DNS over HTTPS with one client implementation and one server implementation. It can be that Dnsdist does not have the most performant imple- mentation of DNS over HTTPS. The same holds for our Flamethrower implemen- tation. • The tests are not executed on multiple network conditions. If we would have emulated different network conditions (such as introducing packet loss), we might have seen better performance for DNS over HTTPS. This is due to the fact that DoH runs over TCP, which makes it a more reliable connection than UDP. • Our tests were run on a public network, which makes it more error-prone than a test performed in a lab environment.

65 Chapter 4

Related Work

In this section, we will go over the research that has been done about this topic in the past. We will start with related DNS work and then continue with DNS over HTTPS research. This section will have work that has been done in the area of measuring DNS performance. We will start with a paper that talks about DNS caching. After that, we will talk about the performance of encrypted DNS protocols (DNS over HTTPS and DNS over TLS).

4.1 DNS Caching

In 2002, an analysis [39] was done on DNS and associated TCP traffic. This traffic was collected from the MIT Laboratory for Computer Science (data from January to December 2000) and the Korea Advanced Institute of Science and Technology (data from May 2001). They had two research questions [39]: • What performance, in terms of latency and failures, do DNS clients perceive? • How does varying the TTL and degree of cache sharing impact caching effective- ness? They found that 23% of DNS lookups from the MIT trace received no answer from the DNS server. And 13% of DNS lookups from the same trace resulted in an error, where “most of these errors indicate that the desired name does not exist.” [39] They further stated that “DNS servers also appear to retransmit overly aggressively” [39]. This ac- counted for more than half of all DNS query packets. They also looked at changes in DNS performance over time: “The percentage of TCP connections made to names with low TTL values increased from 12% to 25% between January and December 2000” [39]. A possible explanation for this was that there was an increase in DNS deployment. La- tency wise, they found that the “median name resolution latency was less than 100ms” [39]. However, the “latency of the worst 10% grew substantially between January and December 2000” [39].

66 In the MIT dataset, they analyzed the hit rate of the DNS cache, this was between 80% and 86%. This data included parallel TCP connections, which indicates that the A-record caching did not seem particularly effective.

4.2 Mozilla DoH measurements

In August 2018, Mozilla posted a blog about the DNS over HTTPS protocol [44]. They measured the performance impact of using DoH by using data from 25,000 Firefox Nightly (footnote) users. They found that “Most queries were around 6 milliseconds slower” as compared to traditional DNS. They further stated that “the slowest DNS transactions performed much better with the new DoH based system than the tradi- tional one - sometimes hundreds of milliseconds better”. They posted the following graph:

Figure 4.1: DoH performance results by Mozilla, August 2018 [44]

They stated possible reasons why DoH in some cases is even faster. One of these rea- sons was because they used one resolver. This eliminated the users who were using unmaintained or overloaded resolvers. Another reason was because of the “modern loss recovery” that HTTP uses, this allowed it “to better operate on very busy or low-quality networks”.

In April 2019, Mozilla posted another blog about the DoH protocol [46]. In this blog, they referenced their test that was rolled out in November 2018 [45]. This test wanted to also look at performance, but in particular, the goal was to see the impact of getting “less localized DNS responses that could slow the browsing experience”. The results were that DNS over HTTPS had “minimal impact or clearly improved the total time it takes to get a response from the resolver and fetch a web page.”

67 4.3 DNS, DoT and DoH page load time and resolution times

In July 2019, five researchers from Princeton University published a paper where they “measure the effect of DoH and DoT on name resolution performance and content deliv- ery” [37]. They build a libcurl and getdns library to measure the performance (resolution time) of unencrypted DNS, DoT, and DoH. They also measure the page load time by using unencrypted DNS, DoT and DoH. They do this by running a headless browser called selenium to load the websites. They then inspect the load times by inspecting the HTTP archive objects (HARs). Their choice of resolvers was the following: Cloudflare, Google, Quad9 and a local university resolver. Because the local resolver only supported unencrypted DNS, it was used as a baseline.

For resolution times they found that unencrypted DNS “performs better than DoT and DoH for most lookups across all resolver” [37]. They also stated that DoH performs slightly better on the slowest queries (on Quad9 and Cloudflare) than unencrypted DNS. They say this is because of HTTP caching. When they compared DoT with DoH they reported differences between resolvers: • Cloudflare DoT and DoH “perform equally for the majority of requests, though DoH begins to outperform DoT for requests that take longer than ≈50ms” [37]. • With Google, the response time for DoT is smaller than DoH for queries that take less than ≈ 100ms. They thought this was because “this could be due to the experimental nature Google’s DoH service, as it may have a different caching backend than the DoT or Do54 Google recursors” [37]. For page load time, they found that “Cloudflare DoT outperforms all other non-Cloudflare configurations, including the default Do53 recursor” [37]. They also noted that all 3 pro- tocols (unencrypted DNS, DoH, DoT) on Cloudflare “performed indistinguishably from each other” [37]. They thought this was because “the additional latency introduced by DoT and DoH is minimal compared to the overall page load time” [37].

They also looked at how different network conditions affected the protocols. They emu- lated cellular 4G and 3G networks. On cellular 4G they found that “Do53 significantly outperforms DoT and DoH in terms of resolution time”. They found that on cellular 3G DoT and DoH resolution times “are substantially longer” than unencrypted DNS resolution times. Further, they noted that “the fastest DoH and DoT lookups take ≈ 470ms, whereas the fastest Do53 lookups take ≈ 150ms. When it comes down to page load time, they noted that on cellular 4g networks “DoT performs indistinguishable from Do53”. But in a lossy 4G network, DoT performs “slightly better than Do53”. For DoH page load time they noted that “DoH performs indistinguishable from Do53 on the lossy 4G network, but it performs worse on the standard 4G network”. For page load time on 3G, DoT and DoH “are no longer able to perform as well as Do53”.

68 4.4 Methodology for testing DNS in a lab environment

In July 2017 a research titled Making DNS Servers Resistant to Cyber Attacks: An Em- pirical Study on Formal Methods and Performance [31]. In their research, they measured the performance of different DNS servers. One of these servers they measured was IRON- SIDES. According to their paper IRONSIDES “is an open-source Domain Name System (DNS) server designed using formal methods to reduce DNS vulnerabilities to cyber at- tacks” [31]. They wanted to know if performance was sacrificed by using IRONSIDES. They found that “IRONSIDES performs quite well compared to other DNS server, both proprietary and open-source” [31]. Their measurements were done on a closed network (a network that is not connected to the internet). Their setup looked like the following:

Figure 4.2: Experimental test environment [31]

The *PERF are the performance measurement tools that they used (dnsperf and res- perf). INETSIM “is an open-source software suite designed to simulate common internet services in a lab environment” [31]. INETSIM allowed them to simulate the DNS archi- tecture, it does this by returning “a mock address to the resolver under test” [31]. The server1..N contained the DNS servers that they tested for performance. The Router in this picture was not connected to the internet, it was only “used for downloading software” [31].

69 Chapter 5

Conclusions

In this thesis, we have looked at the performance impact of using DNS over HTTPS (DoH) as compared to unencrypted DNS. We have done this by first going over the preliminaries for reading this thesis (chapter 2). After this, we have given the scope (section 3.1) and the methodology (section 3.2) for answering the research question. We have given the results (section 3.3) of the methodology and later on, we have given a reflection and limitations of our research (section 3.4). At last, we have looked at related work (chapter 4) for DNS and DoH.

We found that when a TCP and TLS connection has been set up, DoH is slower than unencrypted DNS for most cases. When caching was disabled, we saw a few cases where DoH had a round-trip time that was smaller than the round-trip time of UDP. However, as we have mentioned in section 3.4, the added cost of creating a TCP and TLS con- nection outweigh the performance increase that DoH can have when caching is disabled. With caching enabled, the performance benefit of UDP becomes even more apparent. To address the limitations that were found in section 3.4, we will give future work that can be done to improve this research.

5.1 Future work

It would be interesting if our methodology is applied over multiple network conditions to see what performance DoH has in these conditions. To get more accurate measurements, one can also simulate our research in a closed lab environment. Another improvement is to look at different client and server implementations of DoH to see how they perform compared to the implementation that we have made (the client) and used (the resolver). To see how DoH compares to other encryption protocols (e.g. DNS over TLS), it can also be interesting to use the same methodology with these protocols.

70 Bibliography

[1] Alexa Top Websites Dataset. http://s3.amazonaws.com/alexa-static/top-1m. csv.zip. Accessed: 2019-09-17.

[2] Bluetooth R Low Energy Physical Layer. https://microchipdeveloper.com/ wireless:ble-phy-layer. Accessed: 2019-09-04. [3] Chapter 10. user datagram protocol (udp) and ip fragmentation. https://notes. shichao.io/tcpv1/ch10/. Accessed: 2019-09-04. [4] DNSCrypt Frequently Asked Questions. https://dnscrypt.info/faq. Accessed: 2019-04-29. [5] dnsdist Overview - dnsdist documentation. https://dnsdist.org/. Accessed: 2019-10-03. [6] Dnsperf GitHub repository. https://github.com/DNS-OARC/dnsperf. Accessed: 2019-09-22. [7] Dnsperf-tcp GitHub repository. https://github.com/Sinodun/dnsperf-tcp. Ac- cessed: 2019-09-22. [8] Exploring the anatomy of a data packet. https://www.techrepublic.com/ article/exploring-the-anatomy-of-a-data-packet/. Accessed: 2019-09-04. [9] Flamethrower GitHub repository. https://github.com/DNS-OARC/flamethrower. Accessed: 2019-09-22. [10] GitHub - cpp-base64 library. https://github.com/ReneNyffenegger/cpp- base64. Accessed: 2019-10-02. [11] GitHub - jwijenbergh Flamethrower fork. https://github.com/jwijenbergh/ flamethrower. Accessed: 2019-09-23. [12] GitHub - nghttp2. https://github.com/nghttp2/nghttp2/. Accessed: 2019-10- 02. [13] GitHub - nghttp2 third-party url-parser library. https://github.com/nghttp2/ nghttp2/tree/master/third-party/url-parser. Accessed: 2019-10-02.

71 [14] GitHub - RadboudBachelorThesis. https://github.com/jwijenbergh/ RadboudBachelorThesis. Accessed: 2019-10-03. [15] Identification and authentication. https://www.ibm.com/support/ knowledgecenter/en/SSFKSJ_7.5.0/com.ibm.mq.sec.doc/q009740_.htm. Accessed: 2019-09-04. [16] Introduction to DNSCurve. https://dnscurve.org/. Accessed: 2019-09-23. [17] Libuv GitHub repository. https://github.com/libuv/libuv. Accessed: 2019-09- 12. [18] Matplotlib: Python plotting. https://matplotlib.org/. Accessed: 2019-09-23. [19] NLnet Labs - Unbound - About. https://nlnetlabs.nl/projects/unbound/ about/. Accessed: 2019-09-23. [20] secns/unbound - Docker Hub. https://hub.docker.com/r/secns/unbound. Ac- cessed: 2019-10-03. [21] Service Name and Transport Protocol Port Number Registry. https: //www.iana.org/assignments/service-names-port-numbers/service-names- port-numbers.xhtml. Accessed: 2019-10-01. [22] TCP/IP Five-Layer Software Model Overview. https://microchipdeveloper. com/tcpip:tcp-ip-five-layer-model. Accessed: 2019-09-04. [23] What Happens in a TLS Handshake? https://www.cloudflare.com/learning/ ssl/what-happens-in-a-tls-handshake/. Accessed: 2010-09-04. [24] R. Arends, R. Austein, M. Larson, D. Massey, and S. Rose. Dns security in- troduction and requirements. Technical Report 4033, RFC Editor, March 2005. http://www.rfc-editor.org/rfc/rfc4033.txt. [25] M. Belshe, R. Peon, and M. Thomson. Hypertext transfer protocol version 2 (HTTP/2). Technical Report 7540, RFC Editor, May 2015. http://www.rfc- editor.org/rfc/rfc7540.txt. [26] Cloudflare. DNS over HTTPS - Cloudflare. https://developers.cloudflare. com/1.1.1.1/dns-over-https. Accessed: 2019-04-29. [27] D. Cooper, S. Santesson, S. Farrell, S. Boeyen, R. Housley, and W. Polk. Internet x.509 public key infrastructure certificate and certificate revocation list (crl) profile. Technical Report 5280, RFC Editor, May 2008. http://www.rfc-editor.org/ rfc/rfc5280.txt. [28] J. Damas, M. Graff, and P. Vixie. Extension mechanisms for dns (edns(0)). Tech- nical Report 75, RFC Editor, April 2013.

72 [29] T. Dierks and E. Rescorla. The Transport Layer Security (TLS) Protocol Version 1.1. Technical Report 4346, RFC Editor, April 2006. http://www.rfc-editor. org/rfc/rfc4346.txt. [30] T. Dierks and E. Rescorla. The Transport Layer Security (TLS) Protocol Version 1.2. Technical Report 5246, RFC Editor, August 2008. http://www.rfc-editor. org/rfc/rfc5246.txt. [31] Barry S. Fagin, Bradley Klanderman, and Martin C. Carlisle. Making DNS servers resistant to cyber attacks: An empirical study on formal methods and perfor- mance. In 2017 IEEE 41st Annual Computer Software and Applications Conference (COMPSAC). IEEE, jul 2017. [32] Roy T. Fielding, James Gettys, Jeffrey C. Mogul, Henrik Frystyk Nielsen, Larry Masinter, Paul J. Leach, and Tim Berners-Lee. Hypertext transfer protocol – http/1.1. Technical Report 2616, RFC Editor, June 1999. http://www.rfc- editor.org/rfc/rfc2616.txt. [33] Google. Public DNS - Google Developers. https://developers.google.com/ speed/public-dns/. Accessed: 2019-04-29. [34] Christian Grothoff, Matthias Wachs, Monika Ermert, and Jacob Appelbaum. NSA’s MORECOWBELL: knell for DNS. Unpublished technical report, 2017. [35] P. Hoffman and P. McManus. DNS Queries over HTTPS (DoH). Technical Report 8484, RFC Editor, October 2018. [36] Paul Hoffman and Patrick McManus. DNS Queries over HTTPS. Technical Report draft-hoffman-dns-over-https-00, IETF Secretariat, May 2017. http://www.ietf. org/internet-drafts/draft-hoffman-dns-over-https-00.txt. [37] Austin Hounsel, Kevin Borgolte, Paul Schmitt, Jordan Holland, and Nick Feamster. Analyzing the costs (and benefits) of dns, dot, and doh for the modern web. CoRR, abs/1907.08089, 2019. [38] Z. Hu, L. Zhu, J. Heidemann, A. Mankin, D. Wessels, and P. Hoffman. Specification for dns over transport layer security (tls). Technical Report 7858, RFC Editor, May 2016. [39] Jaeyeon Jung, E. Sit, H. Balakrishnan, and R. Morris. DNS performance and the effectiveness of caching. IEEE/ACM Transactions on Networking, 10(5):589–603, oct 2002. [40] James Kurose. Computer networking : a top-down approach. Pearson, Boston, 2017. [41] P. Mockapetris. Domain names: Concepts and facilities. Technical Report 882, RFC Editor, November 1983. http://www.rfc-editor.org/rfc/rfc882.txt.

73 [42] P. Mockapetris. Domain names - implementation and specification. Technical Report 13, RFC Editor, November 1987. http://www.rfc-editor.org/rfc/ rfc1035.txt. [43] P. V. Mockapetris. Domain names: Implementation specification. Technical Report 883, RFC Editor, November 1983. [44] Mozilla. Firefox nightly secure dns experimental results. https: //blog.nightly.mozilla.org/2018/08/28/firefox-nightly-secure-dns- experimental-results/, August 2018. Accessed: 2019-04-29. [45] Mozilla. Next Steps in DNS-over-HTTPS Testing. https://blog. mozilla.org/futurereleases/2018/11/27/next-steps-in-dns-over-https- testing/imental-results/, November 2018. Accessed: 2019-04-29. [46] Mozilla. DNS-over-HTTPS (DoH) Update – Recent Testing Results and Next Steps. https://blog.mozilla.org/futurereleases/2019/04/02/dns-over- https-doh-update-recent-testing-results-and-next-steps/, April 2019. Accessed: 2019-04-29. [47] Jon Postel. Transmission Control Protocol. Technical Report 7, RFC Editor, September 1981. http://www.rfc-editor.org/rfc/rfc793.txt. [48] E. Rescorla. The transport layer security (tls) protocol version 1.3. Technical Report 8446, RFC Editor, August 2018.

74 Appendix A

Appendix

This appendix contains the graphs from section 3.3 for each round. The description for each round can be found in section 3.2.3.3.

75 A.1 TCP connection delay measurements

A.1.1 Caching enabled

HTTPS GET - Round 1 (on dns2) HTTPS POST - Round 1 (on dns2)

HTTPS GET - Round 2 (on dns1) HTTPS POST - Round 2 (on dns1)

HTTPS GET - Round 3 (on dns1) HTTPS POST - Round 3 (on dns1)

76 HTTPS GET - Round 4 (on dns2) HTTPS POST - Round 4 (on dns2)

77 A.1.2 Caching disabled

HTTPS GET - Round 1 (on dns2) HTTPS POST - Round 1 (on dns2)

HTTPS GET - Round 2 (on dns1) HTTPS POST - Round 2 (on dns1)

HTTPS GET - Round 3 (on dns1) HTTPS POST - Round 3 (on dns1)

78 HTTPS GET - Round 4 (on dns2) HTTPS POST - Round 4 (on dns2)

79 A.2 TLS connection delay measurements

A.2.1 Caching enabled

HTTPS GET - Round 1 (on dns2) HTTPS POST - Round 1 (on dns2)

HTTPS GET - Round 2 (on dns1) HTTPS POST - Round 2 (on dns1)

HTTPS GET - Round 3 (on dns1) HTTPS POST - Round 3 (on dns1)

HTTPS GET - Round 4 (on dns2) HTTPS POST - Round 4 (on dns2)

80 A.2.2 Caching disabled

HTTPS GET - Round 1 (on dns2) HTTPS POST - Round 1 (on dns2)

HTTPS GET - Round 2 (on dns1) HTTPS POST - Round 2 (on dns1)

HTTPS GET - Round 3 (on dns1) HTTPS POST - Round 3 (on dns1)

81 HTTPS GET - Round 4 (on dns2) HTTPS POST - Round 4 (on dns2)

A.3 Round-trip time measurements

A.3.1 Caching enabled

Round 1 Round 2

Round 3 Round 4

82 Round 1 Round 2

Round 3 Round 4

83 Figure A.1: Test 1 Figure A.2: Test 2 Figure A.3: Test 3

Figure A.4: Test 4 Figure A.5: Test 5 Figure A.6: Test 6

Figure A.7: Test 7 Figure A.8: Test 8 Figure A.9: Test 9

Figure A.10: ECDF graph round 1

84 Figure A.11: Test 1 Figure A.12: Test 2 Figure A.13: Test 3

Figure A.14: Test 4 Figure A.15: Test 5 Figure A.16: Test 6

Figure A.17: Test 7 Figure A.18: Test 8 Figure A.19: Test 9

Figure A.20: ECDF graph round 2

85 Figure A.21: Test 1 Figure A.22: Test 2 Figure A.23: Test 3

Figure A.24: Test 4 Figure A.25: Test 5 Figure A.26: Test 6

Figure A.27: Test 7 Figure A.28: Test 8 Figure A.29: Test 9

Figure A.30: ECDF graph round 3

86 Figure A.31: Test 1 Figure A.32: Test 2 Figure A.33: Test 3

Figure A.34: Test 4 Figure A.35: Test 5 Figure A.36: Test 6

Figure A.37: Test 7 Figure A.38: Test 8 Figure A.39: Test 9

Figure A.40: ECDF graph round 4

87 A.3.2 Caching disabled

Round 1 Round 2

Round 3 Round 4

88 Round 1 Round 2

Round 3 Round 4

89 Figure A.41: Test 1 Figure A.42: Test 2 Figure A.43: Test 3

Figure A.44: Test 4 Figure A.45: Test 5 Figure A.46: Test 6

Figure A.47: Test 7 Figure A.48: Test 8 Figure A.49: Test 9

Figure A.50: ECDF graph round 1

90 Figure A.51: Test 1 Figure A.52: Test 2 Figure A.53: Test 3

Figure A.54: Test 4 Figure A.55: Test 5 Figure A.56: Test 6

Figure A.57: Test 7 Figure A.58: Test 8 Figure A.59: Test 9

Figure A.60: ECDF graph round 2

91 Figure A.61: Test 1 Figure A.62: Test 2 Figure A.63: Test 3

Figure A.64: Test 4 Figure A.65: Test 5 Figure A.66: Test 6

Figure A.67: Test 7 Figure A.68: Test 8 Figure A.69: Test 9

Figure A.70: ECDF graph round 3

92 Figure A.71: Test 1 Figure A.72: Test 2 Figure A.73: Test 3

Figure A.74: Test 4 Figure A.75: Test 5 Figure A.76: Test 6

Figure A.77: Test 7 Figure A.78: Test 8 Figure A.79: Test 9

Figure A.80: ECDF graph round 4

93