View metadata, citation and similar papers at core.ac.uk brought to you by CORE

provided by Publikationsserver der Universität Tübingen

Performance Comparison of VPN Solutions

Lukas Osswald, Marco Haeberle, and Michael Menth University of Tuebingen, Chair of Communication Networks, Tuebingen, Germany Email: [email protected], {marco.haeberle,menth}@uni-tuebingen.de,

I.INTRODUCTION Overlay network

192.168.0.1 192.168.0.2 Virtual Private Networks (VPN) is the state-of-the-art method to build secure connections between remote hosts over

public networks. In times of high-speed connections to the 10.0.0.2 10.0.1.2 internet, a need for personal information security and business cases, like cloud computing, high data throughput and a stable connection are increasingly important. Virtual 1 Virtual server 2

OpenVPN [9] is an open source VPN solution which can be Figure 1. Testbed setup. deployed on a wide range of platforms, including common op- erating systems and public cloud platforms. For cryptography, OpenVPN relies on OpenSSL which enables TLS support. Both virtual hosts are running Ubuntu 18.04 with IPsec is a protocol suite consisting of two protocols (Encapsu- kernel 5.3.5. Unfortunately for us, WireGuard will be upstream lated Security Payload, Header) and two modes only starting with kernel 5.6, thus we use the kernel module (Tunnel, Transport). Depending on configuration, IPsec offers with version 0.0.20190913 [5]. For configuring IPsec, we use confidentiality and authenticity [2]. WireGuard [6] describes strongSwan in version 5.6.2. OpenVPN is used in version itself as a modern, fast, and secure VPN tunnel because 2.4.7. The network cards, Intel X710 with 10 Gb/s, are passed it uses modern cryptographic algorithms and implements a through directly to the virtual hosts. The host system is new cryptographic API called Zinc. The developer puts big equipped with an Intel(R) Xeon(R) Gold 6134 CPU @ 3.20 emphasis on minimalism which manifests in its small code GHz, the virtual machines use 4 cores and 16 GB RAM. Our base, easy configuration, lack of a key distribution mechanism, tool set consists of nuttcp 8.1.4 for measuring throughput, and a fixed cipher suite. It is frequently covered by media [8] ping for measuring ping time and mpstat 11.6.1 for measuring because of its upcoming merge into Linux Kernel 5.6 and its detailed CPU utilization. performance claims [7]. Benchmarks of VPN solutions have been discussed in related work, but the data is quite old or uses other se- tups [1][4][3]. Furthermore, we noticed that the benchmarks from the WireGuard whitepaper seem unrealistic, even if we III.RESULTS AND DISCUSSION take protocol overhead into account [7]. In this work, we have decided to conduct VPN benchmarks ourselves. In the following paragraphs we describe our setup and look at three We test the VPN systems in different configurations. For heavily used VPN solutions: OpenVPN, IPsec and WireGuard. IPsec, we test all combinations of protocols and modes as well as various cryptographic algorithms. We also collect data which shows that route-based IPsec processing is equal to the policy-based processing in terms of throughput. For Open- VPN, we use the same cipher suites except for ChaCha20- II.TESTSETUP because it is not supported in the currently available version of OpenVPN. Furthermore, we test several methods to increase throughput as suggested by OpenVPN [10]. - The testbed is shown in Figure 1. It consists of two virtual Guard only offers one cipher suite for configuration, thus there hosts which are connected directly via a 10 Gb/s link. They is only one test case. For AES-based , we also have are configured to be in different subnets. We measure network a look at the performance gain from hardware acceleration throughput, ping time, and retransmission count of TCP traffic through AES-NI. At last, we look at the effect of CPU pinning with and without CPU pinning. on network throughput. IPsec/AES-128_AES-NI_TRN_PB 3776.9096 IPsec/AES-128_AES-NI_TRN_PB 3440.5988 IPsec/AES-128_AES-NI_TUN_PB 3716.5202 IPsec/AES-128_AES-NI_TUN_PB 3352.1761 IPsec/AES-128_TRN_PB 1261.9964 IPsec/AES-128_TRN_PB 1089.1608 IPsec/AES-128_TUN_PB 1215.3741 IPsec/AES-128_TUN_PB 1062.7542 IPsec/AES-256_AES-NI_TRN_PB 3659.4616 IPsec/AES-256_AES-NI_TRN_PB 3337.5392 IPsec/AES-256_AES-NI_TUN_PB 3597.1829 IPsec/AES-256_AES-NI_TUN_PB 3172.7567 IPsec/AES-256_TRN_PB 1072.3199 IPsec/AES-256_TRN_PB 1078.2479 IPsec/AES-256_TUN_PB 718.5759 IPsec/AES-256_TUN_PB 1059.5369 IPsec/CHAPOLY_TRN_PB 2683.6437 IPsec/CHAPOLY_TRN_PB 2333.7271 IPsec/CHAPOLY_TUN_PB 2650.1884 IPsec/CHAPOLY_TUN_PB 2227.3481 OpenVPN/AES-128 607.1265 OpenVPN/AES-128 540.2972 OpenVPN/AES-128_AES-NI 752.6029 OpenVPN/AES-128_AES-NI 632.4011 OpenVPN/AES-256 577.2823 OpenVPN/AES-256 495.9365 OpenVPN/AES-256_AES_NI 731.1624 OpenVPN/AES-256_AES_NI 751.3413 OpenVPN/No_cipher 812.5908 OpenVPN/No_cipher 702.814 OpenVPN/Optimized_AES-128_AES-NI 838.8096 OpenVPN/Optimized_AES-128_AES-NI 729.9893 OpenVPN/Optimized_no_cipher 862.2184 OpenVPN/Optimized_no_cipher 776.2278 Wireguard/CHAPOLY 3213.0385 Wireguard/CHAPOLY 4489.2696

0 500 1000 1500 2000 2500 3000 3500 0 1000 2000 3000 4000 Bandwidth in Mb/s Bandwidth in Mb/s

Figure 2. Network throughput without CPU pinning, CPU turbo boost Figure 3. Network throughput with CPU pinning enabled and CPU turbo enabled, and 4 CPU cores. boost disabled, and 3 CPU cores.

Figure 2 shows the benchmark results of tests without CPU pinning. All IPsec variants with hardware accelerated REFERENCES AES-based encryption surpass WireGuard’s throughput. The [1] .J.C. Pena and J. Evans. “Performance evaluation of importance of using AES-NI for AES-based encryption is software virtual private networks (VPN)”. In: Proceed- also shown well by the data, as well as a small superiority ings 25th Annual IEEE Conference on Local Computer of AES-128 compared to AES-256. Also, there is a small, Networks. LCN 2000 (2000). but visible throughput difference in favor of transport mode [2] S. Kent and K. Seo. Security Architecture for the because of the smaller protocol overhead. OpenVPN does Internet Protocol. RFC4301. 2005. not compete with both systems in any configuration we [3] Shaneel Narayan ; Kris Brooking ; Simon de Vere. tested. Even deactivating encryption has no large effect on “Network Performance Analysis of VPN Protocols: performance. This shows that encryption of data is not the An Empirical Comparison on Different Operating Sys- bottleneck for OpenVPN. The throughput of WireGuard is tems”. In: 2009 International Conference on Networks significantly larger than the ChaCha20-Poly1305 variants of Security, Wireless Communications and Trusted Com- IPsec, presumably because WireGuard uses parallel workers puting (2009). to encrypt and decrypt data [7]. IPsec is implemented in the [4] P. Rybar´ I. Kotuliak and P. Truchly.´ “Performance xfrm part of the Linux kernel and does not take advantage of comparison of IPsec and TLS based VPN technolo- parallelism. Thus, WireGuard’s performance is correlated to gies”. In: 2011 9th International Conference on Emerg- CPU performance. ing eLearning Technologies and Applications (ICETA) Next we pin the cores of the virtual machines to physical (2011). cores. The results are shown in Figure 3. CPU pinning enables [5] Jason A. Donenfeld. Wireguard is now in Linus’ tree. WireGuard to increase its throughput by about 40 percent 2020. URL: https://lists.zx2c4.com/pipermail/wireguard/ because it can use the CPU more efficiently (less context 2020-January/004906.html. switches) even though we have to decrease the number of CPU [6] Jason A. Donenfeld. WireGuard website. 2020. URL: cores by one and disable turbo boost. The disabled turbo boost https://www.wireguard.com/. can be noticed in an overall drop in performance of IPsec and [7] Jason A. Donenfeld. WireGuard Whitepaper. 2020. OpenVPN test cases. URL: https://www.wireguard.com/papers/wireguard.pdf. [8] Sebastian Gruner.¨ Wireguard in Linux-Kernel IV. CONCLUSION eingepflegt. 2020. URL: https : / / www . golem . de / We conclude that IPsec with AES-based encryption is a very news / vpn - technik - wireguard - in - linux - kernel - good choice regarding performance in a virtualised environ- eingepflegt-1912-145437.html. ment without CPU pinning. In non virtualised environments, [9] OpenVPN Inc. OpenVPN website. 2020. URL: https : WireGuard surpasses IPsecs performance by about 30 percent. //.net/. In general, it should be noted that the choice of the right VPN [10] OpenVPN Inc. Optimizing performance on gigabit net- system should be multifactorial, resulting in use cases for all works. 2020. URL: https : / / community. openvpn . net / three systems. openvpn/wiki/Gigabit Networks Linux.