TLS Decryption in Passive Monitoring System with Server Private Key

UMEÅ UNIVERSITY June 4, 2021 Department of Computer Science Report

5DV199 TLS Decryption in passive monitoring system with server private key

Degree Project: Bachelor of Science in Computing Science

Supervisor Jerry Eriksson Employer Polystar Contents

1 Introduction 1 1.1 Background ...... 1

2 Related Work 2

3 TLS Fundamentals 3 3.1 TLS Handshake Protocol ...... 3 3.2 TLS Record Protocol ...... 4 3.3 TLS hello extensions ...... 4 3.4 Cipher suites ...... 5 3.5 Forward secrecy ...... 5 3.6 The PRF algorithm ...... 6 3.6.1 Key Block ...... 7 3.7 Record Payload In CBC Mode ...... 7

4 Implementation delimitations 8

5 Proposed solution 9 5.1 Obtain premaster secret ...... 9 5.2 Derive session keys ...... 10 5.3 Decryption of traﬃc ...... 10

6 Experimental result 11

7 Discussion 13 7.1 Conclusion and Recommendations ...... 13 7.2 Future Work ...... 14 7.3 Personal Reﬂections ...... 14 Abstract

Many network operators need to be able to ensure that customers get the level of service they pay for. To avoid bandwidth and server performance bottlenecks, and easily troubleshoot network problems, the network providers need to be able to see what payload data is sent. Modern networks encrypt data when sending it between nodes that makes passive monitoring more complex. A prevalent encryption mechanism on an IP-based network is TLS that needs to be decrypted. This article’s purpose is to check if it is possible to decrypt TLS traffic in a passive monitoring system with the server’s private key. This is done by implementing a decryptor in a passive monitoring system in the programming language Java. The implemented solution intercepts the traffic, takes out relevant data from the traffic, and derives the session key from that data. How this is done is dependent on what cipher suite is used for the session. Because of delimitations and lack of time the solution is only able to decrypt the cipher suite TLS_RSA_WITH_AES_128_CBC_SHA256. The result showed that it is possible to decrypt TLS traffic and should be possible for more than the specified cipher suite. But there exists a major problem that’s called forward secrecy. This is used in the key exchange algorithm called Diffie–Hellman and makes it impossible to decrypt with only server private key. The conclusion is that it is possible but because of forward secrecy, it is not recommended. TLS 1.3 only uses cipher suites with the key exchange algorithm Diffie–Hellman and the forward secrecy functionality is important for security. Acknowledgement

I want to thank my friends William, Magnus, and Tobias for all the fun times over these 3 years of studying. I also want to thank them for our nice GeoGuesser breaks under the course. It was fun breathers between all coding and writing. I also want to thank Jerry for the help with this article. TLS Decryption Introduction

1 Introduction

Passive network monitoring is an essential tool to ensure quality and reliability for telecommunication networks. Passive monitoring is a technique used to capture traffic from a network by copying traffic. By monitoring traffic between network nodes, the network operator can ensure that its customers get the level of service they pay for. The network providers can also avoid bandwidth and server performance bottlenecks, easily troubleshoot network problems, and more to ensure quality for the users. Traditionally, the signaling between core network nodes has been non-encrypted and the separation of the telecommunication domain from the public internet has been deemed sufficient from a security and privacy point of view. This is no longer the case, modern networks encrypt data when sending it between nodes which makes passive monitoring much more complex. Because of this, many network providers are not able to monitor the network where the encryption applies. Transport Layer Security (TLS) is a cryptographic protocol de- signed to provide communications security over a computer network and is a prevalent encryption mechanism on IP-based network interfaces that need to be decrypted. This thesis work aims to mitigate the problem of encrypted signaling on the above-mentioned interfaces by providing an approach on how decryption of signaling can be done inside a passive monitoring system. This helps network operators ensure the quality of service to the customers. This is a relevant problem to Polystar that is a company selling products to network providers for monitoring traffic. The TLS traffic prevalent for Polystar is SIP-trunking that uses TLS for encryption. A more detailed explanation about this is in the Section 1.1. The suggested method is tested to verify that it works and check if there exists any problem with the presented solution. The research question for this thesis is,

• can TLS traﬃc eﬀectively be decrypted in a passive monitoring system with only the server’s private key?

A program is implemented in Java for the decryption of TLS data. The program is tested towards test data and should be able to decrypt all the data with the server’s private key. The implemented program is integrated into Polystars system to see if it can work in real use.

1.1 Background One of the IP-based networks is Voice Over IP(VoIP)[1] that is used for the delivery of voice communications and multimedia sessions over internet protocol networks. Before VoIP, voice has been transmitted over the public switched telephone network (PSTN). The investment over IP-based networks (public and private) produced the interest in transmitting voice over IP. VoIP converts the user’s voice to a digital signal. SIP trunking [2] is a VoIP technology based on Session Initiation Proto- col (SIP)[3] that delivers telephone services and uniﬁed communications to

Emil Käck 1 June 4, 2021 TLS Decryption Related Work customers. SIP trunking provides network access to multiple clients simul- taneously. SIP is an application-layer control protocol that can establish, modify, and terminate multimedia sessions that include voice, video, and messaging applications [3]. It can also invite participants to already existing sessions, e.g. multicast conferences. Media can be added to an existing session. It incorporates many elements of the Hypertext Transfer Protocol(HTTP) and Simple Mail Transfer Protocol(SMTP). SIP works in conjunction with sev- eral other protocols that specify and carry the session media. It is often used with Real-time Transport Protocol (RTP) for transporting real-time data, Real-Time streaming protocol (RTSP) for controlling delivery of streaming media, Media Gateway Control Protocol (MEGACO) for controlling gateway to the Public Switched Telephone Network (PSTN), and the Session Description Protocol (SDP) for describing multimedia session. SIP uses a resource called SIP or SIPS URI(Uniform Resource Identi- ﬁer)[4] that provide a simple and extensible mean for identifying a communication resource. A SIPS URI speciﬁes that the resource is contacted securely. This means that TLS is to be used between user agent client (UAC) and the domain that owns the URI, this means that in the monitoring system the data received is encrypted with a TLS Cipher suite[5] making it impossible to monitor without access to the private key.

2 Related Work

The article Passive, Transparent, and Selective TLS Decryption for Network Security Monitoring [6] introduces a new solution for monitoring the network to ensure quality. The solution is based on studies which show that passive man-in-the-middle (MitM) proxies for monitoring reduces connection security and potentially introduces additional attack vectors to the network. It also lets users retain privacy for chosen connections. The solution is that the cooperative approach in which end-hosts as cryptographic end-points provide TLS key material to NMS (network monitoring system) for decryption. The problem before is that the encrypted data prevented the NMS from analyzing the data. By giving the NMS the needed key material it can decrypt the data and handle it. The result the paper present is that the runtime increased by a factor of 2.5 compared to analyzing cleartexts only. The complexity analysis shows that it needs less computational resources than MitM proxies. The article also concluded that it could decrypt 99.9% of all observed TLS connections if it has a 40ms delay for the data. The important part is by making the end-host receiving all key-material to filter it and send it to the NMS, it can decrypt almost all TLS data and decrease the security risks. This is related to this article thesis because it solves the problem but differently, in the article, it is more implemented in the endpoint while this thesis focuses on only analyzing the traffic with no help from the endpoints. In many cases, the companies that own the server don’t want other companies implementing programs in their server.

Emil Käck 2 June 4, 2021 TLS Decryption TLS Fundamentals

3 TLS Fundamentals

TLS (Transport layer security)[5] is used to provide privacy and data integrity between two communicating applications. It is divided into two lay- ers, the TLS Record Protocol and the TLS Handshake Protocol. The TLS Record Protocol provides connection security that has two properties. The ﬁrst is that the connection is private. It uses symmetric cryptography for data encryption. The second is that the connection is reliable, message transport includes a message integrity check (MAC) to control that it is the correct application sending the data. The TLS Handshake Protocol allows the server and client to authenticate each other and negotiate an encryption algorithm and cryptographic keys before the application protocol transmits or receives its ﬁrst byte of data.

3.1 TLS Handshake Protocol Before the data can be sent over TLS, the client and server exchange all the information required by both sides [5]. A typical connection example is illus- trated in Figure 1. First, a client sends a ClientHello message that includes the highest TLS version it supports, a random number, a list of supported cipher suites, and the suggested compression methods. It also includes extensions that can change diﬀerent parts of the handshake process and other things. After this, the server responds with a ServerHello message that contains the chosen parameters from the ClientHello. These include protocol version, cipher suite, compression method, and extensions. The message also contains a random number. These values sent with ServerHello are used over the TLS session. The server also sends a Certificate message that is used as identiﬁcation so the client knows it is communication with the right endpoint. It also sends a ServerKeyExchange message that is sent if Certificate does not contain enough data to allow the client to exchange a premaster secret (used for cipher suites using the DHE and ECDHE key exchange algorithms). Lastly, the server sends a ServerHelloDone message to indicate to the client it is done with handshake negotiation. When the server is done the client responds with a ClientKeyExchange message that may contain a PreMasterSecret, public key, or nothing depending on what cipher suite is selected. The client and server then use the random numbers and PreMasterSecret to compute a secret called master secret. All keys used for encryption/decryption can be derived from the master secret. After the ClientKeyExchange, the client sends a ChangeCipherSpec record telling the server that all messages sent from the client onward are going to be authenticated and encrypted. Lastly, the client sends a Finished message that holds the hash and Message Authentication Code (MAC) over the previous handshake messages. The server tries to decrypt this message to verify the hash and MAC. In the last part of the handshake, the server sends a ChangeCipherSpec and Finished message that has the same purpose that the client had. If all goes through, the client and server have now safely exchanged all needed information to start sending the application data. This handshake process

Emil Käck 3 June 4, 2021 TLS Decryption TLS Fundamentals is often used but can be diﬀerent in some cases.

Figure 1: Shows a simpliﬁed illustration of the full TLS 1.2 handshake.

3.2 TLS Record Protocol The TLS Record Protocol is a layered protocol. It includes ﬁelds for length, description, and content (the encrypted data and more) [5]. It takes messages to be transmitted, fragments the data into manageable blocks, optimally compresses the data, applies a MAC, encrypts, and transmits the result. The data received are then decrypted, veriﬁed, decomposed, reassembled, and then delivered to a higher level. There exist four protocols that use the record protocol and they are • the handshake protocol, • the alert protocol, • the change cipher spec protocol, and • the application data protocol. The Handshake protocol and ChangeCipherSpec have already been described under section TLS Handshake Protocol. The Application protocol includes the encrypted data sent between the two endpoints. The Alert protocol conveys the severity of the message and description of the alert.

3.3 TLS hello extensions If the client wants to extend the functionality from servers, it can send data in the extensions ﬁeld in ClientHello [5]. The extensions used under a

Emil Käck 4 June 4, 2021 TLS Decryption TLS Fundamentals session are determined by the server, in the ServerHello message it sends the extensions it supports of the requested extensions by the client. If one extension is sent to the client that the client didn’t send to the server the session is aborted. These extensions can alter the handshaking process and decryption process. For example, the extension extended_master_secret changes the way the cryptographic keys are generated [7]. Instead of taking server random and client random to calculate the master secret it takes the hello messages and concatenates them in the order of arrival.

3.4 Cipher suites Cipher suites are a set of algorithms that help secure a network connection [5]. A cipher suite often contains a key exchange algorithm, an authentication algorithm, a bulk encryption algorithm, and a message authentication code algorithm. An example for a cipher suite is the following:

TLS_RSA_WITH_AES_128_CBC_SHA256.

The first value in the cipher suite defines the protocol that the cipher suite is for, it iss either TLS or SSL. After that comes the key exchange algorithm. This is represented by RSA in the example and outlines how keys are exchanged by the client and server. The next one is the authentication algorithm that is also represented by RSA. It is a digital signature that shows the type of certificate and helps the client verify that it communicates with the server and no man in the middle. The next part is the bulk data encryption algorithm that is represented by AES_128_CBC in the example. It is a symmetric algorithm that is used to encrypt and decrypt a large number of data [8]. There are two types of bulk ciphers, a stream cipher, and a block cipher. For this cipher suite, AES is a block cipher, which means it works on blocks of bytes. AES operates on a block of 16 bytes at a time and encrypts/decrypts that block. The number 128 in the cipher suite determine the size of the key used for encryption/decryption. TLS 1.2 supports key sizes of 128 and 256. The last part of the block cipher is the mode of operation. In this example it is CBC. It is used to provide confidentiality or authenticity. With only the block cipher the data would be secure but not the connection, you would not be able to tell if the correct endpoint is sending the data. The last part of the cipher suite is the message authentication code (MAC) algorithm that is represented by SHA256 in the example. A MAC algorithm provides data origin authentication, as well as data integrity, by producing a MAC tag on data sent [9]. TLS 1.2 uses HMAC that needs a hash function and a secret cryptographic key. The hash function used is the one set in the cipher suite (in this example SHA256). The secret key is a symmetric key used in the TLS session.

3.5 Forward secrecy Forward secrecy or perfect forward secrecy is a feature of speciﬁc key agree- ment protocols that gives assurances that session keys are not compromised

Emil Käck 5 June 4, 2021 TLS Decryption TLS Fundamentals even if long-term secrets used in the session key exchange are compromised [10]. The long-term secret is often the private key of the server. This is often done by generating a unique session key for every session a user ini- tiates. If the session key is compromised it only aﬀects the session it is created for and not data transferred in future or past sessions. This is used in Diffe-Hellman key exchange algorithm for TLS. TLS 1.3 only supports key exchange algorithms that have forward secrecy.

3.6 The PRF algorithm The pseudo random function (PRF) algorithm is used in two places, to generate the master secret and derive all session keys.

Figure 2: Taken from the article The Transport Layer Security (TLS) Protocol Version 1.2 [5]. Shows how master secret is derived using PRF algorithm

Figure 3: Taken from the article The Transport Layer Security (TLS) Protocol Version 1.2 [5]. Show how key block is derived usin PRF algorithm

Figure 2 and Figure 3 show what arguments the PRF takes to calculate the two values [5]. The PRF algorithm takes three values, the ﬁrst one is called secret, the next one label, and lastly the seed. With these arguments, it creates an output of arbitrary length. All cipher suites for TLS 1.2 and documents published before use a PRF function based on HMAC. This is determined by the hash function in the cipher suite, in this case, SHA-256. This means that the HMAC function generates a random hash with a length of 256-bits. For calculating the master secret it needs the premaster secret that is derived using the key exchange algorithm, the label "master secret" and the client and server random received from ClientHello and ServerHello message. After the master secret is derived with the PRF function it is then used to derive the key block. For deriving the key block the function takes the master secret, the label "key expansion" and the same server and client random used in the master secret.

Emil Käck 6 June 4, 2021 TLS Decryption TLS Fundamentals

Figure 4: Taken from the article The Transport Layer Security (TLS) Protocol Version 1.2 [5]. Shows the data expansion function

The PRF function for TLS 1.2 uses an expansion function shown in Figure 4. It uses the hash function determined in the cipher suite to expand a secret and seed into an arbitrary quantity of output. How A() is calculated is shown in Figure 5. The P_hash function iterates depending on how many bytes are needed. For example, if the hash function used is SHA-256 then each iteration generates 32-bytes. Then to calculate the master secret that is always 48-bytes it needs to iterate 2 times to get all bytes.

Figure 5: Taken from the article The Transport Layer Security (TLS) Protocol Version 1.2 [5]. Shows how A() is calculated in data expansion

Figure 6: Taken from the article The Transport Layer Security (TLS) Protocol Version 1.2 [5]. Shows how data expansion is used with PRF

Figure 6 shows how the P_hash is used for the PRF function.

3.6.1 Key Block

The key block is generated using PRF as shown in Figure 3. The master secret is expanded into a sequence of bytes. These bytes are split into keys. These keys are the client write MAC key, a server write MAC key, a client write encryption key, and a server write encryption key. The MAC keys are used for veriﬁcation of the encrypted data and the encryption key is used for encrypting/decrypting the data. The server keys are used when the server is sending the data and client keys for when the client sends the data. Also, some AEAD ciphers may additionally require a client write IV and a server write IV that is also needed for decryption.

3.7 Record Payload In CBC Mode The record payload has a diﬀerent structure depending on what mode of operation is used [5]. The structure of the payload is shown in Figure 7. This is how the data block is structured before encrypted. Hello extensions

Emil Käck 7 June 4, 2021 TLS Decryption Implementation delimitations can also change the structure of the block. With CBC mode the block has a Initialization Vector (IV) that is not encrypted and is needed for the decryption to work. The block contains the content that is what the server/client wanted to send, it has a MAC that is used for conﬁrming the right endpoint is sending the data and lastly padding used to force the length of the plaintext to be an integer multiple of the block cipher’s block length.

Figure 7: Taken from the article The Transport Layer Security (TLS) Protocol Version 1.2 [5]. Shows the structure of the decrypted TLS data

4 Implementation delimitations

TLS has many different cipher suites and they use different methods for decryption. This report focuses only on the cipher suites using the encryption algorithm AES because it is the most used one and there is not enough time to implement more. This can easily be extended in the future to more decryption algorithms. It also only supports AES in CBC mode because of time constraints. Another delimitation is the key exchange algorithm, there exist many different key exchange algorithms used to exchange cryptographic keys that are used under a session. There are key exchange algorithms that use forward secrecy which means that it is impossible to decrypt the traffic even if you have the server’s private key. Both client and server create secret variables for each session that is used to derive all keys for the session. These variables are not sent over the network which makes it impossible to monitor the traffic with only the server’s private key. For this reason, the thesis focuses on the key exchange algorithm RSA that does not use forward secrecy. There exist other key exchange algorithms that do not use forward secrecy but because of time constraints, only RSA is used. Lastly, the solution only supports TLS version 1.2. These different versions of TLS do different things in different stages of the TLS process, because of this it may take too much time to implement a solution for all the different versions. The solution implements TLS 1.2 because it is the most used one.

Emil Käck 8 June 4, 2021 TLS Decryption Proposed solution

5 Proposed solution for passive TLS decryption with server private key

A description of the process of passively decrypting TLS traﬃc at a passive monitoring system is presented. The most important part is obtaining the premaster secret from the messages sent between the endpoint, after receiving the secret the system needs to generate the session keys and lastly decrypt the data using the session keys. The implementation only supports the cipher suite TLS_RSA_WITH_AES_128_CBC_SHA256.

5.1 Obtain premaster secret When a session starts the endpoints need to establish the session keys needed for encryption/decryption and other security. In this implementation, the monitoring system is comparable to a man in the middle system. The monitoring system can access all data sent between the endpoints and for this reason, it can’t handle TLS computations similar to the endpoints. This solution only implements the key exchange algorithm RSA because of the time constraint and because it doesn’t have forward secrecy. With forward secrecy, it is impossible to derive the premaster secret with only the server’s private key. For example Diﬃe–Hellman key exchange (DHE) [11] uses forward secrecy in the key exchange making it impossible for a monitoring system to get the session keys. Here is an example of how DHE works:

1. Server generates generator (g) and prime modulus (p) for exampleg = 3 andp = 17.

2. Server sendsg andp to client

3. Server picks secret number (x) for examplex = 54 and calculates X = gxmod(p) = 554mod(17) = 15

4. Server sendsX to client

5. Client picks secret number (y) for exampley = 24 and calculates Y = gymod(p) = 524mod(17) = 16

6. Client sendsY to server

7. With the sent values server can calculate K = Y xmod(p) = 1654mod(17) = 1 and client Xymod(p) = 1524mod(17) = 1

8. Server and client have now calculated premaster secret = 1

The values marked in red in the DHE example are values that are never sent between the two endpoints. They are essential for the calculation of the premaster secret and because the values are never sent between the server and client the monitoring system never gets access to them. For this reason, a monitoring system can’t decrypt TLS data for a session using key exchange algorithms with forward secrecy. These values are uniquely generated for each session so they can’t be stored in the monitoring system.

Emil Käck 9 June 4, 2021 TLS Decryption Proposed solution

The key exchange algorithm RSA (Rivest-Shamir-Adleman) does not use forward secrecy [5]. When RSA is used the client generates a 48-byte premaster secret, encrypts it using the public key from the server’s cer- tiﬁcate, and sends the result in an encrypted premaster secret message. This message is a variant of ClientKeyExchange message and is sent as a ClientKeyExchange message. Because the monitoring system has access to the server’s private key it can read the encrypted data, decrypt it with the private key, and then the monitoring system has access to the premaster secret. An important thing that is necessary to know about is that some server systems change their private key regularly that impacts the monitoring system. If they change private keys the monitoring system must also change to the new one to be able to decrypt the RSA data.

5.2 Derive session keys The next step in the process after the monitoring system has derived the premaster secret is to generate the session keys [5]. But before that, a master secret needs to be generated using the premaster secret. For all key exchange methods, the same algorithm is used to covert the premaster secret into the master secret. The algorithm is called PRF and is described in section 3.6. In this solution, it is implemented to handle the hello extension extended_master_secret. When the extension is used it changes the parameters used to derive the master secret. Instead of the third argument being ClientHello.random + ServerHello.random it uses session_hash that are all handshake messages sent or received, starting at the ClientHello up to and including the ClientKeyExchange messages. This is hashed with the hash function determined by the cipher suite before used as an argument. The parameter "master secret" is also changed to "extended master secret". The extension doesn’t change the process of deriving the session keys. When the master secret is derived then the same algorithm PRF is used to derive all session keys needed for the decryption that is also described in section 3.6. When the keys are generated they are stored in a cache until the application data is sent and then used for decrypting the data. The premaster secret is removed so that there is less risk of security issues.

5.3 Decryption of traffic After deriving the session keys it is possible to decrypt the traffic. Section 3.7 shows and describes the data blocks structure. The test traffic used for testing the implementation used an extension called encrypt_then_mac that changes the structure of the payload [12]. If the extension is not used then the MAC is encrypted in the payload but with the extension, it doesn’t encrypt it. It puts it at the back of the payload not encrypted so that the server/client can verify the mac before decrypting the content. After verification, it can start the decryption process. In Java there exists a class called Cipher [13] that is used for encryption/decryption with cipher suites. This is used for decrypting the encrypted payload using the AES cipher. For decryption with AES in CBC mode, you give the decrypter the encrypted payload, the

Emil Käck 10 June 4, 2021 TLS Decryption Experimental result client_write_key/server_write_key depending on if the server or client sent the traﬃc, and lastly the IV that are at the beginning of the payload. And with that, the traﬃc is decrypted and can be used/sent where it needs to be.

6 Experimental result

To verify that the implementation can decrypt TLS traffic, it is tested on an HTTP request. Because SIP-trunking traffic could not be acquired that used the key exchange algorithm RSA the test is conducted on an HTTP request. What sort of traffic is used does not affect the TLS decryption, the only requirement is that TLS is used. The implementation can decrypt traffic using TLS 1.2 and the cipher suite TLS_RSA_WITH_AES_128_CBC_SAH256. Figure 9 shows the traffic in wireshark that is not decrypted. Wireshark is used because it doesn’t show up in the environment it is implemented in if not decrypted. The traffic is an HTTP request and a response. Figure 9 is the request for a website and the Figure 10 shows the response with a body.

Figure 8: HTTP traﬁc that is not decrypted in wireshark F

Figure 9: Decrypted HTTP request data

Emil Käck 11 June 4, 2021 TLS Decryption Experimental result

Figure 10: Decrypted HTTP response data

Table 1 shows the key exchange algorithms used and if it is possible to do decrypt payload data if it is used for the passive monitoring system. An important note is that for TLS 1.3 the Diﬀe-Hellman algorithms is the only one used.

Table 1: TLS key exchange algorithm if used can still decrypt traﬃc.

Algorithm Possible RSA Yes Diﬃe–Hellman No ECDH Yes SRP Yes PSK Yes

Emil Käck 12 June 4, 2021 TLS Decryption Discussion

7 Discussion

The purpose of this report is to see if it is possible to decrypt TLS traffic in a passive monitoring system with the server’s private key. Network providers need to monitor the network to ensure quality to their customers but if the traffic is encrypted the providers can’t do that. For that reason, this article checks if it is possible to decrypt TLS traffic effectively meaning how much traffic is possible to decrypt. The result shows that it is possible to decrypt the traffic to some extent. The implementation is only for one cipher suite and with specific extensions because of the time restriction, but it shows that it is possible. The traffic is a HTTP request and response that would only be random text if not decrypted. Table 1 shown in the result is the key exchange algorithms used in TLS 1.2. The table shows that if the key exchange algorithm is used in a cipher suite, if it is possible to decrypt that traffic with only a private server key. The reason why Diffie–Hellman can’t be decrypted is because of forward secrecy making it impossible to decrypt the TLS traffic with only the server’s private key. Another important part is that in TLS 1.3 the only key exchange algorithms used are different versions of Diffie–Hellman so for this scenario it is not possible to decrypt any traffic for TLS 1.3. The reason why it works for this specific cipher suite is pretty simple. The most important part of the TLS session is the key exchange. If the monitoring system can’t derive the session keys it is impossible to decrypt the traffic. With RSA it encrypts the premaster secret needed for deriving the session keys. It encrypts it with the server’s public key so when it is sent to the server only the server can decrypt it with its private key. In this implementation, the monitoring system has access to the server’s private key so it can decrypt the premaster secret and use it to derive all session keys. After the session keys have been derived no apparent problems occurred that make it impossible to decrypt the traffic. Because of the delimitations that needed to be taken the implementation could only support one cipher suite. It doesn’t invalidate the result but may alter the reliability. Yes, it can decrypt TLS traffic but to what extent? Because of all the different cipher suites used and different algorithm com- binations used in all of them, it is hard to tell. The result of this article only says that it is possible for one cipher suite. Through theoretical con- clusions, we could say that it would be possible for more. For example, the cipher suite TLS_RSA_WITH_AES_256_CBC_SHA256 should also work because the only difference is the key length used for AES.

7.1 Conclusion and Recommendations Yes, it is possible to decrypt TLS traﬃc with only a server private key in a passive monitoring system. The problem is forward secrecy used in the most used key exchange algorithms for security reasons. Another problem is that in some servers they regularly change the server private key to avoid attacks if the key is compromised. Because TLS 1.3 only supports key exchange algorithms using forward secrecy it is not a recommended solution

Emil Käck 13 June 4, 2021 TLS Decryption Discussion for future versions. For these reasons, it is recommended to implement a similar solution as described in related work. If the server owner permits the implementation then it should be possible to decrypt most TLS traﬃc. It is also a lot of work because of all the diﬀerent algorithms used that change the process and extensions.

7.2 Future Work This article focuses on the concept if it is possible to decrypt TLS traffic in a passive monitoring system with only the server private key. Because of this, it doesn’t have a high focus on how effective this solution is or if there exist more things that need to be taken into account. For example, if security gets compromised using this method. Security is a big thing with traffic and the purpose for adding TLS encryption to the traffic. If this solution compromises the security it should not be used and another solution should be implemented. Another interesting thing may be having a deeper look at the algorithms used in TLS 1.2. Because the focus of this article is to check if it is possible and not all different problems that may occur there may be problems with some algorithms. For example in this article found that forward secrecy is a big problem, there may exist more problems in other algorithms not checked in this report.

7.3 Personal Reﬂections There was a lot of decoding needed for the implementation. Because of this, the implementation took longer than expected and at one time I didn’t think that the solution would be ﬁnished. Because of all this, the implementation was only able to support one cipher suite. It was an interesting project and gave a lot of insight into how TLS works and how the monitoring systems work. It would be more interesting to look into similar solutions like the one discussed in related work. This solution will probably not work in the future because of forward secrecy that many servers will probably adopt in the future.

Emil Käck 14 June 4, 2021 TLS Decryption References

References

[1] Upkar Varshney et al. “Voice over IP”. In: Communications of the ACM 45.1 (2002), pp. 89–96. [2] Ivan Gaboli and Virgilio Puglia. “SIP Trunking the route to the new VoIP services”. In: 2010 ITU-T Kaleidoscope: Beyond the Internet?- Innovations for Future Networks and Services. IEEE. 2010, pp. 1–8. [3] Jonathan Rosenberg et al. SIP: session initiation protocol. 2002. [4] Francois Audet. The use of the SIPS URI Scheme in the Session Ini- tiation Protocol (SIP). Tech. rep. RFC 5630, October, 2009. [5] Tim Dierks and Eric Rescorla. “The transport layer security (TLS) protocol version 1.2”. In: (2008). [6] Florian Wilkens et al. “Passive, Transparent, and Selective TLS De- cryption for Network Security Monitoring”. In: arXiv preprint arXiv:2104.09828 (2021). [7] Karthikeyan Bhargavan et al. “Transport Layer Security (TLS) session hash and extended master secret extension”. In: IETF RFC 7627 (2015). [8] Bulk ciphers. 2021. url: https://tex.stackexchange.com/questions/ 3587/how- can- i- use- bibtex- to- cite- a- web- page (visited on 05/14/2021). [9] SP NIST. “800-56A Rev 2”. In: Recommendation for Pair-Wise Key Es- tablishment SchemesUsing Discrete Logarithm Cryptography–May (2013). [10] Henk CA Van Tilborg and Sushil Jajodia. Encyclopedia of cryptography and security. Springer Science & Business Media, 2014. [11] Nan Li. “Research on Diﬃe-Hellman key exchange protocol”. In: 2010 2nd International Conference on Computer Engineering and Technol- ogy. Vol. 4. IEEE. 2010, pp. V4–634. [12] Peter Gutmann. “Encrypt-then-MAC for Transport Layer Security (TLS) and Datagram Transport Layer Security (DTLS)”. In: Request for Com- ments 7366 (2014). [13] Class Cipher. 2021. url: https://docs.oracle.com/javase/7/ docs/api/javax/crypto/Cipher.html (visited on 05/17/2021).

Emil Käck 15 June 4, 2021