Building an Application-Aware Ipsec Policy System
Total Page:16
File Type:pdf, Size:1020Kb
Building an Application-aware IPsec Policy System Heng Yin Haining Wang Department of Computer Science The College of William and Mary Williamsburg, VA 23187 {hyin,hnw}@cs.wm.edu Abstract ability of the network-layer, such as IP spoofing [25] and IP fragmentation attacks [16], to sabotage end-to- As a security mechanism at the network-layer, the IP se- end communications. The IP security (IPsec) proto- curity protocol (IPsec) has been available for years, but col [17, 18, 19, 27] is a suite of protocols that secure data its usage is limited to Virtual Private Networks (VPNs). communications on the Internet at the network-layer. The end-to-end security services provided by IPsec have IPsec provides packet-level source authentication, data not been widely used. To bring the IPsec services into confidentiality and integrity, and supports perfect for- wide usage, a standard IPsec API is a potential solution. ward security. There are two major protocols in the IPsec However, the realization of a user-friendly IPsec API in- protocol suite: the Authentication Header (AH) protocol volves many modifications on the current IPsec and In- and the Encapsulation Security Payload (ESP) protocol. ternet Key Exchange (IKE) implementations. An alter- The AH protocol provides source authentication and data native approach is to configure application-specific IPsec integrity, while the ESP protocol provides data confiden- policies, but the current IPsec policy system lacks the tiality and authentication. Internet Key Exchange (IKE) knowledge of the context of applications running at up- [13, 21] is the default key agreement protocol for the es- per layers, making it infeasible to configure application- tablishment of IPsec security associations (SAs), doing specific policies in practice. mutual authentication and choosing cryptographic keys. In this paper, we propose an application-aware IPsec However, while IPsec has been available for years, its policy system on the existing IPsec/IKE infrastructure, in usage is limited to the deployment of Virtual Private Net- which a socket monitor running in the application con- works (VPNs). Currently, the IPsec policy is not aware text reports the socket activities to the application policy of the specific security requirements of Internet appli- engine. In turn, the engine translates the application poli- cations. It is static and coarse-grained, providing all cies into the underlying security policies, and then writes or nothing security protection to Internet applications. them into the IPsec Security Policy Database (SPD) via Thus, in comparison with the success of security proto- the existing IPsec policy management interface. We im- cols and techniques deployed at the transport and appli- plement a prototype in Linux (Kernel 2.6) and evaluate cation layers such as SSL/TLS and PGP, IPsec has been it in our testbed. The experimental results show that rarely used to provide end-to-end security protection for the overhead of policy translation is insignificant, and Internet applications. The major obstacles to the wide us- the overall system performance of the enhanced IPsec age of IPsec beyond VPNs (i.e., providing end-to-end se- is comparable to those of security mechanisms at up- curity services to Internet applications) are listed as fol- per layers. Configured with the application-aware IPsec lows. policies, both secured applications at upper layers and legacy applications can transparently obtain IP security • enhancements. IPsec only provides rudimentary policy manage- ment support [8, 14], and IPsec lacks the knowl- edge of application context. Thus, the policy selec- 1 Introduction tor can only be the tuple of source/destination ad- dresses, port numbers, and transport protocol type. Network-layer security protection is essential to Inter- The trust relationship and security policy between net communications. No matter how secure the upper- two participants must be set up in advance. Due to layer protocols are, adversaries can exploit the vulner- dynamic port numbers and unpredicted destination USENIX Association 14th USENIX Security Symposium 315 IP addresses, configuring fine-grained application- icy system has the following advantages: (1) without any specific policies beforehand is infeasible in prac- modification, Internet applications can transparently ob- tice. Note that many applications, including passive tain end-to-end security protection at the network-layer FTP data connections, DCOM/CORBA-based ap- by simply configuring application-specific policies; and plications, and RTP-based streaming applications, (2) the application-aware policy system is built on the ex- even negotiate source/destination ports at runtime. isting IPsec policy configuration interface, and no mod- ifications on the current IPsec and IKE implementations • There is no standard IPsec API, and implement- are needed. Thus, it is much easier for IPsec vendors to ing a user-friendly IPsec API requires many modi- support and deploy it. fications on the current IPsec and IKE implementa- We implement a prototype of the proposed IPsec pol- tions. For instance, the current two-phase IKE pro- icy system in Linux (Kernel 2.6), and evaluate the ef- tocol only performs host-oriented authentication, ficacy of our prototype in the testbed. The experi- whereas user-oriented authentication is one basic mental results show that the overhead of policy transla- requirement of a secured application (see more dis- tion is insignificant and overall system performance of cussion in Section 2). Moreover, the wide usage of the enhanced IPsec is comparable to those of security upper-layer security mechanisms such as SSL/TLS mechanisms at upper layers. More importantly, our ex- and SSH greatly dampens the demand for IPsec periments demonstrate that the application-aware IPsec API. policy system provides network-layer security enhance- ments (e.g., packet-level integrity protection) for secured • IPsec needs Public Key Infrastructure (PKI) to per- applications and provides a variety of network-layer se- form identity authentication in the public network curity services for legacy applications. environment, whereas PKI itself is not widely de- Note that IPsec is not a panacea to resolve all network- ployed yet. The lack of scalable authentication layer security problems. For example, IPsec can- mechanisms is another major impediment to the not thwart the bandwidth-exhaustion Denial of Service wide use of IPsec. (DoS) attacks [31] and IKE itself is vulnerable to frag- The good news about IPsec is: (1) IPsec and IKE have mentation flooding attacks [16]. Seeking solutions to been implemented on nearly all modern operating sys- these problems is outside the scope of this paper. Al- tems, and have been applied to the VPNs scenario ma- though IPsec is able to protect broadcast and multicast turely now; and (2) while no standard IPsec API exists, traffic, the current key management protocols for IPsec each implementation does have its own policy manage- can only work in one-to-one mode for now. Therefore, ment interface. our proposed scheme applies to unicast communication In this paper, we explore an approach other than the only. The scenarios of broadcast and multicast are also IPsec API to facilitating the wide usage of IPsec. We at- beyond the scope of our research. tempt to provide application-aware IPsec service by in- The remainder of this paper is organized as follows. troducing application context into IPsec policy model, Section 2 presents the background of this research and while leaving the IPsec and IKE implementations in- related work. Section 3 details the application-aware tact. We propose an application-aware IPsec policy sys- IPsec policy system. We describe the implementation of tem as middleware to provide Internet applications with our prototype in Linux (Kernel 2.6) in Section 4. In Sec- network-layer security protection. In order to bring ap- tion 5, we conduct a series of experiments on our testbed plication context into the IPsec policy model, a socket to evaluate the proposed scheme. Finally, the paper con- monitor detects the socket activities of applications and cludes with Section 6. reports them to the application policy engine. Then, the application policy engine automatically generates the 2 Background and Related Work fine-grained IPsec policy in accordance with the applica- tion policy, and writes them into the IPsec Security Pol- Since IPsec is a layer-3 security protocol, it must be icy Database (SPD) via the existing IPsec policy man- implemented in the kernel space. There are two aux- agement interface. 1 To ease policy configuration, we iliary databases that IPsec consults with: Security Pol- also propose an application specification language. Be- icy Database (SPD) and Security Association Database ing simple, uniform, and extendable, the proposed spec- (SAD). Generally speaking, a security policy determines ification language is essential to configure and distribute what kinds of services are to be offered to an IP flow, and application-specific policies easily, and hence, reduces a security association (SA) specifies how to process it. the management burden. Although the SAs can be manually created, in most In addition to the feasibility issue, compared with a cases they are created automatically by the key manage- potential standard IPsec API, the application-aware pol- ment program. The IKE [13] is the current IETF stan- 316 14th USENIX Security Symposium USENIX Association dard for key establishment and SA parameter negotia- used to send a particular outgoing packet. Microsoft has tion. IKEv2 [15] has been proposed to replace the origi- integrated IPsec in its Windows 2000 and Windows XP nal IKE protocol for simplicity and strong DoS protec- products, but no official IPsec API has been published tion. Both are two-phase protocols. During the first yet. A home-brewed IPsec API library [3] for Windows phase, the two key management daemons authenticate 2000 and above versions has been implemented by ma- each other and establish a secure channel between them nipulating the local policy repository. Because of their (i.e IKE SA).