140 JOURNAL OF ELECTRONIC SCIENCE AND TECHNOLOGY, VOL. 11, NO. 2, JUNE 2013

Security-State Adjustable Gateway with Threat-Based Configuration

Chin-Fu Kuo, Yung-Feng Lu, and Chi-Ying Chen

Abstract⎯This paper proposes a configurable secure International Data Corporation highlights that the security gateway architecture which allows the system is the greatest challenge for the adoption of cloud[1]. Hence, administrators to dynamically configure the security security is a huge concern for cloud users, and it becomes mechanisms upon deployment or during the run-time. an important issue that provides a secure way to use the Rather than allowing the system administrators to turn cloud. To improve the mutual trust between users and the on or off individual security mechanisms, the proposed cloud provider, the cloud security alliance (CSA)[2],[3] has architecture allows the administrators to configure the developed a security guide that identifies many areas for gateway based on the security threats to be overcome. concern in cloud computing. The cloud network and The current common architecture leads to tremendous perimeter security is one of the most challenging issues. administration overhead and increases the chance of Hence, protecting the information from illegal access has misconfiguration vulnerability. We propose a novel become an inevitable issue for IT product vendors and software architecture to aid the product designers to developers. avoid the misconfiguration vulnerability and the In this paper, we are concerned with how to configure end-users to ease the administration overhead. The the security mechanisms for security gateways in a software architecture makes use of the threats to the systematic manner. Network gateways, one of the popular gateways and the occurrence relation between the IT products, are designed to bridge network traffic between threats to configure the security software components on the Internet and private networks. The major responsibility the gateways. With the software architecture, the of the gateways is to protect private networks from being end-users can focus on determining the desired security accessed without appropriate permission. Although there do features rather than the software configuration. exist numerous security mechanisms, there is no universal Moreover, the architecture allows the product designers standard for configuring security mechanisms on gateways. or security service to incrementally revise the software A gateway that is set up as a network router for an configuration when new threats appear. enterprise private network often requires strict security

Index Terms⎯Common criteria, gateway, operation policies. However, a gateway for home usage might not system, security, threat. have proper security measures. In addition, the majority of the security mechanisms are computationally intensive. Enforcing unnecessary security mechanisms may 1. Introduction compromise the system performance and network Cloud computing is a combination of various throughput. The challenge is how to configure a network computing entities, globally separated, but electronically gateway so that its security features are met without connected. As the geography of computation is moving compromising its performance. towards corporate server rooms, it brings more issues In the current gateway architecture, individual security including security, such as the transmission security. There software components are designed to provide particular is a survey regarding the use of cloud services made by security mechanisms. Designing each software component to provide particular security mechanisms has its own Manuscript received February 9, 2013; revised March 23, 2013. This merits. However, it leads to additional administration work was supported by National Science Council under Grant No. NSC overhead and it may also cause misconfiguration 101-2218-E-025-001, NSC 100-2221-E-390-012, and NSC 101-2221- vulnerability. E-390-007. Instead of proposing new security features, this paper C.-F. Kuo is with the Department of Computer Science and Information Engineering, National University of Kaohsiung, Kaohsiung focuses on the providing of a threat-based security (Corresponding author e-mail: [email protected]) configuration architecture for gateways to ease the Y.-F. Lu is with the Department of Computer Science and Information administration overhead and reduce the chance of Engineering, National Taichung University of Science and Technology, misconfiguration vulnerability. Fig. 1 illustrates the Taichung (e-mail: [email protected]). threat-based security configuration architecture for the C.-Y. Chen is with the Department of System Research, Trend Micro, Taipei. gateways. The novel architecture serves two purposes: 1) Digital Object Identifier: 10.3969/j.issn.1674-862X.2013.02.005 reducing the software integration and configuration

KUO et al.: Security-State Adjustable Gateway with Threat-Based Configuration 141 overhead for product designers during the design phase and integrate several open source security components on a 2) providing an intuitive configuration interface for the Linux platform. However, the designed architecture is not end-users. From product designers’ perspective, the limited to the software components used in this paper. architecture provides a systematic manner to integrate the Other (open source or commercial) software can also be software components to overcome the threats and to integrated on such an architecture. A series of experiments properly configure the software components. The are conducted to evaluate the capability of the proposed threat-based configuration architecture shown in the dashed methodology. box in Fig. 1 separates the software configuration into four The rest of this paper is organized as follows. Section 2 parts: the common criteria document, threat dependency describes the proposed security gateway based on common graph, state transition graph, and software configuration. criteria. Section 3 describes the design of the gateway and The common criteria document defines the threats that the discusses the implementation issues of the security gateway. gateway should overcome. Section 4 evaluates the performance of the system and Given the threats defined by the common criteria demonstrates the effectiveness of the proposed architecture. document, we propose a threat dependency graph to model Our work is summarized and the future work is discussed in the occurrence relation among the threats. The threat Section 5. dependency graph helps the designers identify the set of threats to be overcome in order to provide certain security 2. Security Gateway Based on features and reduce the chance of misconfiguration vulnerability. We also define the system security states, Common Criteria each of which defines the set of the threats that the system Gateways in the paper are referred to the devices can overcome. The change of the security states during the defined by RFC1009[5], in which a gateway is an IP-level run-time are modelled by the state transition digraph. Given router to connect two or more networks. In June 1993, the the common criteria document and security states, the United States, the United Kingdom, Germany, France, designers can find the software components to overcome Canada, and Netherlands started to develop an evaluation the threats with minimal efforts. The four parts of the standard for a multi-national marketplace. This standard is threat-based security configuration are determined during known as the “Common Criteria for Information the design phase and are constructed by domain experts. Technology Security Evaluation” (CCITSE), and it is From end-users’ perspective, the architecture allows the usually referred to as the “Common Criteria” (CC). end-users to configure the gateways by selecting the desired In CC, a product which is subjected to evaluation is security features, as shown on the left-hand side of Fig. 1. called a target of evaluation (TOE). As shown in Fig. 2, After the desired security features are selected, the software there is a process for the evaluation of CC. Our TOE is a components on the gateways are configured according to gateway that is designed to protect the private networks the flow shown in the dashed box in Fig. 1 to provide the against security threats from the external network. For the selected features. There is no need for the end-users to sake of presentation, we will use terms “TOE” and configure the individual software components on the “gateway” interchangeably in the paper. The security gateways. When the new threats are discovered, the measures of a TOE are defined by two documents: the designer shall revise the threat dependency graph, state protection profile (PP) and security target (ST). PP allows transition diagraph, and software configuration. The the consumers and developers to compile standardized sets end-users shall download the revised threat and state of security requirements to meet their needs. On the other information to reconfigure their own gateways. By this hand, ST specifies the functional requirements and manner, the change of misconfiguration vulnerability can assurance security for the product developers. The be reduced. evaluators use ST as the basis for evaluation. There are The idea of threat dependency graph is related to the many research have adopted CC to help them verify their attack tree approach developed by Schneier[4]. The attack security requirements. Details of CC usages are given in tree approach is intended for penetration testing where there [6]−[9]. is a little background information about the system to be T hreat-B ased Security Configuration Security gatew ay Security features Fram ework tested. The basic idea is a combination of the work

P ro v id e a tru ste d p a th Common Criteria Documents ra pnec raph hreatG ependence T D

b e tw e e n th e u se rs a n d th e h p ra ig D n sitio n ra T te ta S breakdown structure from project management and the n ratio u fig n o C are ftw o S re m o te sy ste m s. familiar tree representation of a logical proposition. Provide a m eans to re c o rd a u d it tra il o f However, the occurrence of threats in the gateways may not se c u rity -re la te d e v e n ts. . . always have such relation. When a system is vulnerable to . End users Provide secure session eavesdropping, the system is vulnerable to both the e sta b lish m e n t u sin g encryption functions. unauthorized access to the security configuration data and spoof. We extend the tree structure to a directed acyclic C ustom er organization Product designers/ graph to model the occurrence relation among the threats. (e.g., US Department of Defense ) A d m in istrato rs To demonstrate the capability of the architecture, we Fig. 1. Threatbased security configuration architecture.

142 JOURNAL OF ELECTRONIC SCIENCE AND TECHNOLOGY, VOL. 11, NO. 2, JUNE 2013 Table 1: Threat description of the gateway[9],[14] Threat Description Threats for the gateway OSs T.NOAUTH An unauthorized entity may attempt to bypass the security of the TOE so as to access and use security functions and/or non-security functions provided by the TOE. T.SELPRO An unauthorized entity may read, modify, or destroy security critical configuration data. Threats for the private networks T.AUDIT An unauthorized entity may deny doing some Fig. 2. Evaluation process of CC. behaviors if the TOE lack for an audit mechanism. T.MEDIATE An unauthorized entity may send impermissible T. i information through the TOE which results in the exploitation of resources on the internal network. T. k T. l T.RESOURCE An unauthorized entity executes commands, sends T. j data, or performs other operations that make system resources unavailable to system users. Fig. 3. Threat dependency graph. Threats for the communication channel T.EAVESDROP An unauthorized entity obtains user data by 2.1 Security Threats eavesdropping on communications lines. This To define the security threats for a security gateway, we threat is relevant when the system must exchange user data with a remote system, and the have studied several PPs and STs for the definition of confidentiality of that data is important. threats for various TOEs. The target products include T.SPOOF An attacker tricks users into interacting with operating systems (OSs)[10], firewalls[11],[12], and intrusion spurious system services. For instance, an attacker [13] may modify protocol headers such that a user detection systems . The threats for the security gateway believes the communication is coming from a are summarized in Table 1. The threats listed here are not source that is different from where it was actually sent. newly discovered but are categorized to aid the design of T.MASQUERADE A cracker captures the interactive session of an the threat-based configuration architecture. In the following authorized user. The cracker now appears as a sections, a symbol “T.threat” represents a threat named legitimate user and can perform any action allowed to that user, including reading or “threat”. modifying sensitive data. The threats are classified into three categories: the threats for the gateway OSs, the threats for the private 2.2 Threat Dependency Graph networks, and the threats for the communication channel. A threat dependency graph is an acyclic directed graph The threats for the gateway OSs may cause abnormal in which each node represents a threat on the gateway and operations in the underlying OSs. Two threats, T.NOAUTH each directed edge represents an attacking scenario from and T.SELPRO, are listed in this category. The threats for the source threat to the destination threat. We call a threat the private networks are the ones that may cause disordered T.a a preceding threat of a threat T.b if there exists a path from T.a to T.b. Essentially, an edge in the threat operations on the private networks and information leakage. dependency graph represents that failing to overcome the T.AUDIT, T.MEDIATE, and T.RESOURCE are the threats source threat leads to the failure of overcoming the listed in this category. The threats for the communication destination threat. In other words, to fully overcome one channels are the ones that may compromise the threat T, the system must overcome all the preceding confidentiality of the communication channels when threats of T.a and T.b. network packets are not encrypted during transmission. Fig. 3 shows a simple threat dependency graph with T.EAVESDROP, T.SPOOF, and T.MASQEURADE are four threats: T.i, T.j, T.k, and T.l, and three directed edges the threats in this category. (i.e., attacking scenarios). The edges in Fig. 2 illustrate that Although the listed threats could occur individually, T.k may occur when T.i or T.j occurs on the system. their occurrences are not fully independent. For example, Similarly, T.l may occur when T.k has been observed on T.EAVESDROP might occur when an unauthorized user is the system. To protect the system from T.k, the system able to eavesdrop on the communication media and obtain needs the security mechanisms to prevent all preceding the authentication data. Hence, the user can bypass the threats of T.k (i.e., T.i and T.j) and T.k rather than T.k only. security mechanisms to modify critical configuration data, To construct the threat dependency graph for the which causes T.NOAUTH. Note that such a scenario occurs security gateway, we studied the attack scenarios collected even when the system enforces a flawless authentication by the CERT® coordination Center, Snort rules database[15], mechanism, i.e., no user can log in the system without and earlier work in the literature (e.g., [16] and [17]). We giving proper identity information. In short, when the threat selected the attacking scenarios in which the threats related T.EAVESDROP occurs on the system, it is very likely that to the gateway are involved. Fig. 4 shows the threat threat T.NOAUTH also occurs on the system. We call such dependency graph system from T.EAVSDROP. On the a sequence of attacking events of threats as an attacking other hand, T.AUDIT is the threat that requires most scenario. overhead to overcome. It is because the system has to

KUO et al.: Security-State Adjustable Gateway with Threat-Based Configuration 143 overcome all eight threats in the graph. T.EAVESDROP Note that the threat dependency graph shown in Fig. 4 T.NOAUTH may not cover all possible threats on a security gateway. This is because the graph is compiled based on the T.SPOOF observed attacking scenarios. Any attacking scenarios not T.SELPRO observed in the literature are not covered by the graph. T.MEDIATE However, the graph can be revised incrementally when a T.MASQUERADE new threat or an attack scenario has been discovered. The product designer or security service providers can revise the T.AUDIT T.RESOURCE threat dependency graph by analyzing the relationship between the new threat and any of the existing threats. For Fig. 4. Threat dependency graph for the security gateway. example, if failing to overcome the new threat T.new could Service of Gateway lead to the failure of overcoming one of the existing threat (T.EAVESDROP, T.SPOOF, T.old, and a new edge from T.new to T.old will be added to T.MASQUERADE) the graph. Secure Gateway (T.AUDIT, T.RESOURCE, T.MEDIATE)

Secure OS 3. System Design and Implementation (T.NOAUTH, T.SELPRO)

3.1 System Architecture Hardware We propose a layered software architecture for threat based security gateways. The advantage of this architecture Fig. 5. System architecture of the security-enhanced gateway. is the following. The security functionalities of each layer are verified without any concern for the software overcome T.SELPRO. There are two main types of components in the other layers. Its correct functionalities implementation for access control: the discretionary access can be used in other above layers. As a result, the system control (DAC) and mandatory access control (MAC). The designer can reduce the overhead of selecting and DAC grants the access permission based on the user integrating security software components on the gateway. identity and ownership. Other security-relevant information Each layer is designed to overcome the threats listed in one are ignored. On the other hand, the MAC grants the access threat category defined in Section 2.2. The software at each permission based on the subjects and objects specified in layer will be chosen to overcome the threats of the layer. In security policies. With MAC, users can confine user the proposed architecture, there are three software layers: programs and system daemons to the minimum privilege the secure OS, secure gateway, and service of gateway, as required to complete their tasks. By specifying appropriate shown in Fig. 5. The bottom layer is the secure operating security policies, the probability in which the user programs system layer, which is responsible for providing the basic and system daemons might cause harm (e.g., buffer OS protection to the gateway. The top layer is the gateway overflow or configuration errors) is minimized. Most OSs, [21] service layer, which is on the top of the secure OS layer and such as FreeBSD , implement through DAC. is responsible for providing a secure private network. It is ·Secure gateway layer: Three security modules are responsible for providing secure communication channels. selected for the secure gateway layer: iptables[22], Snort[15], For the rest of this section, we describe the selected and iproute2[23]. Iptables is part of the netfilter framework software components in each layer. Rather than in the Linux kernel. We use iptables to implement the implementing new security modules in each layer, existing network address translation (NAT1) protocol and build a open source security modules are integrated to demonstrate packet filtering firewall at the packet level. Iptables is used the effectiveness of our design. However, the gateway is in our gateway to overcome T.MEDIAT. Iptables is also not limited to use the software chosen in this paper. Other one of the minimal requirements for our security gateway. commercial or open source software can also be used in this The Snort[15] is an intrusion detection system (IDS) that architecture as long as the software overcomes the threats could inspect the traffic flow passing through the gateway defined in the paper. to protect the private networks. By logging the packets or ·Secure OS layer: The secure OS layer is the minimal only the packet header, the administrator can trace or requirement for our security gateway. The mechanisms analyze the attacks later. Hence, T.AUDIT is overcome. In adopted to implement the secure OS layer are pluggable addition, when certain attacks occur, Snort signals can authentication modules (PAM)[18] and security-enhanced alarm to the system. Linux (SELinux)[19],[20]. Iproute2[23] consists of traffic control mechanisms for PAM provides the authentication mechanism and is classifying, prioritizing, sharing, and limiting traffic of used in our gateway to overcome T.NOAUTH. In addition, either direction. Iproute2 allows the end users to control the we choose SELinux as a part of the secure OS layer to traffic such that the bandwidth is not over-consumed by

144 JOURNAL OF ELECTRONIC SCIENCE AND TECHNOLOGY, VOL. 11, NO. 2, JUNE 2013 certain hosts. Iproute2 also allows the end users to assign threats. State S7 has the maximal security level which the priority to each host. Therefore, T.RESOURCE. is overcomes all eight threats of the gateway. But note that the overcome by iproute2. index of the security state does not imply the security level ·Service of gateway layer: The widely used IPv4 has of the gateway. A greater index does not imply a higher no authentication and encryption mechanisms on the header security level. For example, it is difficult to compare the and content of IP (Internet protocol) datagrams. Therefore, security level of states S2 and S3 because they could the datagrams could be read, modified, or forged during overcome different threats and attacking scenarios. transmission. This defect may cause threats Table 2: State configuration T.EAVESDROP, T.SPOOF, and T.MASQUERADE. IPSec is one of many mechanisms to resolve this defect. State Overcame threats Software configuration S None None There are two major protocols in IPSec: the IP 0 S1 T.MEDIATE, iptables and IPSec with DES authentication header (AH)[24] and IP encapsulated security T.EAVESDROP, enabled payload (ESP) protocol[25]. AH authenticates IP datagrams and T.SPOOF S2 T.MEDIATE, iptables and IPSec with advanced to provide data integrity and source authentication; ESP T.EAVESDROP, encryption standard (AES) enabled encrypts IP datagrams to prevent eavesdropping. Therefore, T.SPOOF, all communication channels of different applications and T.MASQUERADE S T.MEDIATE, iptables, IPSec with DES enabled, operated over the IP protocol could be protected. 3 T.EAVESDROP, and PAM T.SPOOF, 3.2 Security State and State Transition and T.NOAUTH

The security state of a gateway is a set of threats that S4 T.MEDIATE, iptables, IPSec with AES enabled, the gateway can overcome. For each threat in the threat set, T.EAVESDROP, and PAM T.SPOOF, its preceding threats must also be included in the set. T.MASQUERADE, Otherwise, the gateway is not able to be fully protected and T.NOAUTH S T.MEDIATE, iptables, IPSec with AES enabled, against the threats and the threat set is not a valid security 5 T.EAVESDROP, PAM, state. Consider the threat dependency graph in Fig. 6. The T.SPOOF, and SELinux threat set {T.i, T.j, T.k} is a valid security state because all T.MASQUERADE, T.NOAUTH, preceding threats of threats T.i, T.j, and T.k are included in and T.SELPRO the set. However, the threat set {T.i, T.k} is not a valid S6 T.MEDIATE, iptables, IPSec with AES enabled, security state because T.j is a preceding threat of threat T.k T.EAVESDROP, PAM, but not included in the threat set. T.SPOOF, SELinux, and iproute2 T.MASQUERADE, There are eight security states in our security gateway. T.NOAUTH, Table 2 shows the states, the threats that the system can T.SELPRO, overcome, and the corresponding security software and T.RESOURCE S7 T.MEDIATE, iptables, IPSec with AES enabled, configuration for each state. State S0 is the initial state of T.EAVESDROP, PAM, SELinux, iproute2, and the system and activates no security software. State S1 has T.SPOOF, Snort the minimal security level which overcomes threats T.MASQUERADE, T.NOAUTH, T.MEDIATE, T.EAVESDROP, and T.SPOOF. When the T.SELPRO, system is in state S1, the system activates iptables and IPSec T.RESOURCE, with data encryption standard (DES) to overcome the and T.AUDIT

S3 S5

Overcoming Overcoming Overcoming Ignoring T.NOAUTH T.MASQUERADE T.SELPRO T.RESOURCE Ignoring Ignoring Ignoring Overcoming T.NOAUTH T.RESOURCE T.MASQUERADE T.SELPRO Overcoming T.MEDIATE, T.EAVESDROP, and T.SPOOF S 0 S1 S4 S6 Ignoring T.MEDIATE, T.EAVESDROP, and T.SPOOF

Overcoming Ignoring Overcoming T.NOAUTH T.AUDIT T.MASQUERADE Ignoring Overcoming Ignoring T.NOAUTH T.AUDIT T.MASQUERADE

S2 S7 Fig. 6. State transition digraph.

KUO et al.: Security-State Adjustable Gateway with Threat-Based Configuration 145

The state transition digraph of the gateway is shown in Therefore, a network-based IDS such as Snort is more Fig. 6. Each node represents a security state and each edge suitable for our gateway. One benefit of Snort is that its represents the state transition from the source state to the online signature database is updated constantly by the destination state. The initial system state is state S0. The user/security community. The database contains signatures annotation on each edge represents the changing of the for various attacks. By updating the new signatures, Snort threat set. The system may overcome or ignore certain could respond to new attacks quickly. threats. For example, when the gateway is in state S5, its ·Communication channel protection: There are two security state could be changed to S6 by changing the ways to protect the communication channels. One way is to software configuration to ignore another threat, provide authentication and encryption mechanisms on the T.RESOURCE. application layer. For example, the secure socket layer The state transition digraph aids the end users to reduce protocol (SSL)[26] has been proposed to protect a the runtime administration overhead. With the state communication channel between a web browser and a web transition digraph, the end users only focus on determining server. However, the disadvantage of this approach is that the security features that the system should overcome. The each application must be tailored to support secure security mechanisms on the system are automatically communication channels. The other way is to adopt IPSec configured based on the software configuration shown in on the network layer. The IPSec serves as a transparent Table 2. For instance, when the system starts, the system is mechanism to authenticate and encrypt IP datagrams in the initial state S0. If the end user decides that the system flowing through the secure gateway, so that all hosts in the should be protected from the T.NOAUTH attacks, the end private network could benefit from it without any user can request the system to migrate to state S3. And the modification. software on the system is, then, configured according to Several challenges occur during our system integration: Table 2. (The system can also be protected from threat ·Integrating the IPSec and firewall: There are several T.NOAUTH by migrating to state S4, S5, S6, or S7. However, challenges when IPSec has to work with firewalls. One it will over consume more resources and compromise the challenge is that the firewall should allow the IPSec traffic system performance. The performance of the system at to pass through the firewall. The firewall has to be different states will be studied in Section 4.) configured to allow traffic using ports 50 and 500. Because The state transition digraph also aids the end users to the port 50 is used by ESP and the port 500 is used for the eliminate the misconfiguration vulnerability. When the key exchange. The other challenge is about NAT. IPSec system is in state S4 and the end user decides to ignore authenticates IP datagrams to ensure that they are not T.NOAUTH attacks but overcome T.SELPRO attacks, the modified during transmission, but NAT rewrites the header system state should be migrated to state S5. Without the of the datagrams when they go pass through the gateway. state transition digraph, the end user may decide to As a result, IPsec authentication fails since IP datagrams deactivate PAM and activate SELinux. However, without are modified. A resolution to this problem is that the activating PAM, the system is not able to overcome threat sending gateway passes the IPSec datagrams without doing T.SELPRO. This is because T.NOAUTH has a path to NAT, and the receiving gateway also passes the IPSec T.SELPRO in the threat dependency graph and must be datagrams without doing NAT. The process is shown in overcome. Fig. 7. · Integrating the IDS and firewall: An important 3.3 Implementation Challenges consideration in the installation of IDS is the time to In this section, we present the alternatives for analyze the packets. If the IDS analyzes the packets before implementing the proposed architecture and discuss the they are processed by the firewall, the IDS analyzes all implementation decisions we made. We also present the packets directed to the gateway. On the other hand, an IDS challenges for integrating several existing security behind the firewall will only analyze packets that pass mechanisms, and how we handle those challenges. The through the firewall. We choose the former setting. The alternatives for the implementation of IDS and the reason is that the system will get more information for the communication channel protection are discussed as follows. state of the environment, avoiding a false sense of safety. ·IDS: There are two main types of IDS. One is the This approach introduces additional performance overhead network-based IDS and the other is the host-based IDS. A to the system. Unfortunately, it is extremely difficult to network-based IDS installed on the edge of the networks quantify the increased overhead. This is because the scans all of the incoming/outgoing traffic of the private amount of increased overhead highly depends on the networks for suspicious attacks; the host-based IDS number and type of rules set in the firewall. In addition, installed on a host machine scans security vulnerabilities although more firewall rules may filter more packets to only on the host machine. IDS is used on our gateway to reduce the run-time overhead on the IDS, it also introduces protect the private network rather than one host machine. additional overhead on the firewall.

146 JOURNAL OF ELECTRONIC SCIENCE AND TECHNOLOGY, VOL. 11, NO. 2, JUNE 2013 4.2 Results and Analysis We present the results that show the tradeoff between the security states and performance. In the evaluation Fig. 7. System configuration with both NAT and IPSec. results, the 95% confidence interval of each data point in Table 3: Software versions for the experimental environment our results is no greater than 2.4% of its data values. · Average throughput: The average network Module Version SELinux kernel 2.6.3 with SELinux throughput for different security states are shown in Fig. 9.

Linux kernel 2.6.3 The average network throughputs were no more than 25 PAM 0.77 Mb/sec. This was caused by the overhead imposed by Iptables 1.2.9 IPSec. The overhead of authentication and IP datagrams IPSec Kernel 2.6.3 with native IPSec encryption backlogged the network packets and reduced the Iproute2 2.4.7 Snort 1.9 average network throughput. In Fig. 9, the average throughput increases when the system state migrates from S1 to S2 and from S3 to S4 (i.e., 4. Performance Evaluation the system is able to overcome more threats). This is In this section, we evaluate the effectiveness and because that AES used in S2 and S4 is more efficient than performance of the proposed threat-based security gateway. DES used in S1 and S3. Although AES provides more functionalities and has better performance than DES does, 4.1 Workload Generation and Performance Metrics we chose AES and DES for different states for the purpose To evaluate the performance of a threat-based security of demonstration. In practice, the system designer can use gateway, we set up an experimental environment which AES for both states to reduce the overhead. Moreover, the was isolated from the Internet to avoid interference, as difference of the average network throughput between S1 shown in Fig. 8. The security gateway was installed on an and S3 (or S2 and S4) was small because PAM introduced x86-based machine which has an Intel Pentium III 700 negligible overhead when there was a new secure shell MHz CPU and 512 MB RAM (random access memory). (SSH) session generated. In order to overcome additional Two security gateways were connected via a wire-speed threats from S4 to S7, additional security software layer-3 switch. Each security gateway was connected to a components were activated and the average network private network which has one or more hosts. The software throughput was decreased gradually. installed on the gateways is listed in Table 3. We adopted Iperf[27] to generate network traffic and measure network performance. In our setting, there were an Iperf client and an Iperf server. The Iperf client sent Fig. 8. Experimental environment for performance evaluation. (transmission control protocol (TCP) or user datagram protocol (UDP)) packets to the Iperf server. The TCP traffic 225 5 was used to measure the throughput of the connection. 21.9 21.6 220 19 The experiments also verified the effectiveness and 0 17.88 17.6 17.5 16.1 performance tradeoff of state transition. Attacking packets 115 were generated by using hping[28]. We used hping to forge 5 110 malicious packets by stuffing the payloads with the 0

signatures specified in the Snort database. Packets were (Mb/sec) Throughput Average sent by hping at the rate of 100 packets per second to 55 Average throughput (Mb/sec) (Mb/sec) throughput Average trigger attacking alerts of Snort. When Snort detected attack 00 S S S S S S S events, it sent alert messages to the end-user. So, the S1 1 S2 2 S33 S44 S55 S66 S77 State configuration end-user can modify the desired security features of the State Configuration gateway, and then the state of the gateway could be Fig. 9. Average throughput for different state configurations. automatically adjusted. The performance metrics are the average network Table 4: Average CPU workload throughput and average CPU workload. The average Security state Average CPU workload (%) network throughput is the amount of transferred data in S1 39.8 mega bits per second (Mb/sec) on the network connection S2 37.9 between the Iperf client and Iperf server in average; the S3 40.3 S4 38.2 average CPU workload is the ratio of the time when the S5 39.6 CPU is busy. We measured performance for different S6 40 security states. S7 100

KUO et al.: Security-State Adjustable Gateway with Threat-Based Configuration 147 ·Average CPU workload: The average CPU workload and received UDP packets. Fig. 11 shows the traffic flow in for different security states are shown in Table 4. The the gateway when the Snort is active. The Snort copies the average CPU workloads of states S1 and S3 (or S2 and S4) incoming packets to its own buffer space and performs the are similar. It shows that overcoming T.NOAUTH needed inspection. When the gateway is overloaded, the Snort may little overhead. Moreover, it is clear that enabling AES in have to drop/ignore packets. However, when the Snort states S2 and S4 could reduce the CPU workload. The drops a packet, the original packet still arrives on the reason is that AES is more efficient than DES. Overcoming gateway. T.SELPRO increased the CPU workload from S4 to S5. The The average number of packets analyzed by the Snort average CPU workloads of states S5 and S6 are similar. It and the received UDP packets are shown in Fig. 12. When shows that overcoming T.RESOURCE imposed ignorable the system was in state S5, the number of the received UDP effect on the CPU workload. When the system was in state packets increased steadily until the UDP sending rate was

S7, the CPU workload was 100%. This was because the 35 Mbits/sec. When the sending rate was greater than 35 Snort was activated in S7. Incoming packets were analyzed Mbits/sec, the number of the received UDP packets began by the Snort to check for suspicious activities. The to decrease. The reason is as follows. Before the peak, the checking process involved pattern-matching with thousands interrupt service routine had enough CPU resources to of build-in attacking signatures. This process was handle the incoming packets. However, when the UDP computationally intensive. sending rate reached 35 Mb/sec, the system was fully ·Threat detection and state transition: The throughput loaded by the incoming packets. After the peak, the number variation of the security gateway as the system state of the received UDP packets began to decrease. The reason changes is shown in Fig. 10. The upper part shows the is that the buffer of the network interface card was corked change of network throughput over time; the bottom part and the incoming packets were dropped. The same result shows the attacks generated by hping over time. In order to could be observed when the state of the system was in state demonstrate the state transition, the default state of the S6. The difference was that the peak appeared earlier, because the IPSec was turned on in the state S . This gateway was set as S1 and the Snort was turned on. In the 6 phenomenon also affected the number of packets analyzed figure, Si+Snort represents the software configuration when by the Snort. There was also a peak point for the number of the system was in the state Si and the Snort was activated. Afterward, the Snort detected attack events. If attack events packets analyzed by the Snort. Either in state S2 or S6, the were detected, alarm messages were sent to the end-user. number of analyzed packets increased until the peak was Then, the state of the security gateway was changed reached, and began to decrease after the peak. This is automatically based on the security features of the end-user. because the priority of interrupt service routine is higher When no attack occurred, the throughput was about 16 than that of the Snort. Therefore, when the system is Mb/sec. At time 5, a T.MASQUERADE attack was issued overloaded, the resource is mostly consumed by the and detected by the Snort. The state of the security gateway interrupt service routine and forces the Snort to drop packets without analyzing. This is the reason why the peak was manually changed from S1 to S4 to overcome T.MASQUERADE. It is clear that the throughput was of the number of analyzed packets by the Snort appeared earlier than that of the number of received UDP packets. increased to around 20 Mb/sec because AES was enabled. (Note that AES is a more efficient algorithm.) At time 13, a

T.RESOURCE attack was added in and detected by the 25 Snort. The state of the gateway was manually changed from

S4 to S7 to overcome T.RESOURCE. We used iproute2 to 20 restrict the total traffic of the private network to be 10

Mb/sec. The throughput was further maintained around 10 15 Mb/sec. At time 19, when there was no attack of

T.MASQUERADE, the state of the gateway was still S7. At 10 Throughput (Mb/sec) Throughput time 25, when there was no attack belonging to S +SnortS +Snort T.RESOURCE and T.MASQUERADE, the state of the S1+Snort 4 S7 S7 1 5 gateway was changed from S7 to S1+Snort, which was the default state. The throughput was back to around 16 Mb/sec again. The results demonstrate how the threat-based 0 12345678910111213141516171819202122232425262728293031 security gateway can react to the change of system security NO ATTACK features. When the system is in safe states, unnecessary T.MASQUERADE T.RESOURCE security mechanisms are turned off to improve the system T.MASQUERADE+ T.RESOURCE performance. However, when the system is under threats, 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 Time necessary security mechanisms are turned on to protect the (sec) system. The tradeoff is the system performance. The average number of packets analyzed by the Snort Fig. 10. Throughput with manual state changing.

148 JOURNAL OF ELECTRONIC SCIENCE AND TECHNOLOGY, VOL. 11, NO. 2, JUNE 2013 [2] Security Guidance for Critical Areas of Focus in Cloud TCP/IP Computing, Cloud Security Alliance, 2012. Snort protocol [3] M. Pan, P. Li, Y. Song, Y. Fang, and P. Lin, “Spectrum statck clouds: a session based spectrum trading system for multi-hop cognitive radio networks,” in Prof. of Int. Conf. Copied Computer Communications, Orlando, 2012, pp. 1557−1565. packets [4] B. Schneier, “Attack trees,” Dr Dobb’s Journal, vol. 24, no. 12, pp. 21−19, Dec. 1999. Incoming Outcoming packets Driver packets [5] R. Braden and J. Postel, RFC 1009 (Requirements for Internet gateways), 1987. [6] Y. Wu, Y. Suhendra, and H. Guo, “A gateway-based access control scheme for collaborative clouds,” in Prof. of the 7th Gateway Int. Conf. on Internet Monitoring and Protection, Stuttgart, Fig. 11. Flow of packets in the gateway. 2012, pp. 54−60. [7] Common criteria for information technology security evaluation. [Online]. Available: http://www. commoncriteriaportal.org/cc/ [8] K. V. Dolan, P. A. Wright, and R. R. Montequin. U.S. Department Defense application-level firewall protection profile for medium robustness environments, version 1.0. Technical Report. [Online]. Available: http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix= html&identifier=ADA395046 [9] K. V. Dolan, P. A. Wright, and R. R. Montequin, B. Mayer, L. Gilmore, and C. Hall. U.S. government traffic-filter firewall protection profile for medium robustness environments. Technical Report. Available: http://www.dtic.mil/dtic/tr/fulltext/u2/a395046.pdf [10] National Security Agency. Protection profile for single-level Fig. 12. Average number of analyzed packets by Snort and the operating systems in environments requiring medium received UDP packets. robustness, version 1.22. Technical Report. [Online]. Available: http://www.niap-ccevs.org/pp/ 5. Conclusions pp_os_sl_mr_v1.22.pdf [11] T. Alexander, R. Hawkins, and T. Kelly. Security assurance This paper explores the design and implementation of a cases: motivation and the state of the art. [Online]. Available: security gateway based on CC, and proposes a threat-based http://www-users.cs.york.ac.uk/~rda/York%20CESG%20sec software-configuration architecture. The architecture serves urity%20case%20report%20i1_1.pdf two purposes: 1) reducing the integration and configuration [12] T. Kaneko, S. Yamamoto, and H. Tanaka, “Proposal on overhead for the product designers and administrators and 2) countermeasure decision method using assurance case and providing a user-friendly manner for the end-users to common criteria,” in Proc. of the 6th Int. Conf. on Project configure the gateway. We identify potential threats for Management, Honolulu, 2012, pp. 331−336. security gateways, derive the relationship among the threats, [13] A network infrastructure security product for attack define the system security states, and integrate several open detection, analysis and response, version 2.11. Technical source security software components to demonstrate our Report. [Online]. Available: http://www. design. The performance evaluation shows that when the commoncriteriaportal.org/files/epfiles/st_vid2013-st.pdf system security state changes, the network throughput [14] Common criteria for information technology security significantly changes. In addition, when the Snort is turned evaluation, version 2.1. Technical Report. [Online]. off, 60% of the CPU workload can be released. If it is not Available: http://www.niap-ccevs.org/ required to overcome T.AUDIT, the released CPU time can Documents_and_Guidance/ cc_docs/cc_users_guide.pdf be used to increase the throughput or execute other critical [15] M. Roesch, “Snort-lightweight intrusion detection for operations. networks,” in Proc. of the 13th USENIX conference on System administration, Berkeley, 1999, pp. 229−238. References [16] B. Hatch and J. Lee, Hacking Linux Exposed, New York: McGraw Hill, 2003. [1] F. Gens. New IDC IT cloud services survey: Top benefits [17] W. Stallings, Network Security Essentials, Upper Saddle and challenges. [Online]. Available: http://blogs.idc.com/ River: Prentice Hall, 2002. ie/?p=730 [18] Pluggable authentication modules. [Online]. Available:

KUO et al.: Security-State Adjustable Gateway with Threat-Based Configuration 149 http://www.kernel.org/pub/linux/libs/pam Information Engineering (CSIE), National University of [19] S. Smalley, C. Vance, and W. Salamon. Implementing Kaohsiung (NUK), Kaohsiung in 2006. Currently, he is an selinux as a linux security module. Technical Report. associate professor with CSIE, NUK, Kaohsiung. His research [Online]. Available: http://www.nsa.gov/research/ interests include real-time process scheduling, resource _files/publications/implementing_selinux.pdf management, cloud computing, and network QoS. [20] SELinux Project. [Online]. Available: http://selinuxproject.

org/page/Main_Page

[21] FreeBSD. [Online]. Available: http://www.openbsd.org/ Yung-Feng Lu received the B.S. and M.S. [22] H. Welte, J. Kadlecsik, M. Josefsson, and P. McHardy. Iptables. [Online]. Available: http://www.netfilter.org degrees from the Department of Electronic [23] Iproute2. [Online]. Available: http://snafu.freedom.org/ Engineering, National Taiwan University of linux2.2/iproute Science and Technology, Taipei in 1998 and [24] S. Kent and R. Atkinson, RFC 2402 (IP Authentication 2000, respectively, and the Ph.D. degree Header), 1998. from the Department of Computer Science [25] S. Kent and R. Atkinson, RFC 2406 (IP Encapsulating and Information Engineering, National Security Payload), 1998. Taiwan University, Taipei in 2010. [26] Secure sockets layer. [Online]. Available: http://www. Dr. Lu is currently an assistant professor with the Department openssl.org/ of Computer Science and Information Engineering, National [27] C. H. Hsu and U. Kremer, “Iperf: a framework for automatic Taichung University of Science and Technology, Taichung. He construction of performance prediction models,” in Proc. of received the Best Paper Award from the IEEE/IFIP International Workshop on Profile and Feedback-Directed Compilation, Conference on Embedded and Ubiquitous Computing (EUC) in Paris, 1998, doi: 10.1.1.40.2777. 2008, and the Best Paper Award from the 7th Workshop on [28] Hping: Netwrok packet analyzer and reassembler. [Online]. Wireless, Ad Hoc, and Sensor Networks (WASN), in 2011, Available: http://www.hping.org/ respectively. His research interests include networking, embedded systems, databases, storage, neural networks, cloud computing, and information security.

Chin-Fu Kuo received his B.S. and M.S. degrees from the National Chung Cheng Chi-Ying Chen received the B.S. and M.S. degrees from the University, Chiayi in 1998 and 2000, Department of Computer Science and Information Engineering, respectively. He received the Ph.D. degree National Taiwan University, Taipei in 2002 and 2004, respectively. in the computer science and information He is currently a senior engineer with the Trend Micro. His engineering from National Taiwan research interests include networking, embedded systems, cloud University, Taipei in 2005. He joined the computing, and information security. Department of Computer Science and (Photograph not available at the time of publication)