An Architectural Metrics Scorecard Based Approach to Intrusion Detection System Evaluation for Wireless Network by Rupinder Singh & Dr

View metadata, citation and similar papers at core.ac.uk brought to you by CORE

provided by Global Journal of Computer Science and Technology (GJCST)

Global Journal of Computer Science and Technology Network, Web & Security Volume 12 Issue 11 Version 1.0 June 2012 Type: Double Blind Peer Reviewed International Research Journal Publisher: Global Journals Inc. (USA) Online ISSN: 0975-4172 & Print ISSN: 0975-4350

An Architectural Metrics Scorecard Based Approach to Intrusion Detection System Evaluation for Wireless Network By Rupinder Singh & Dr. Jatinder Singh Khalsa College, Amritsar, Punjab, India Abstract - Wireless IDS architectural metrics are used to compare the intended scope, architecture of wireless IDS, and how they match the deployment architecture. These metrics can be used to evaluate the architectural efficiency of a wireless IDS and can help in designing efficient wireless IDS. Wireless IDS analyze wireless specific traffic including scanning for external users trying to connect to the network through access points and play important role in security to wireless network. Design of wireless IDS is a difficult task as wireless technology is advancing every day, Architectural metrics can play an important role in the design of wireless IDS by measuring the areas concern with the architecture of a wireless IDS. In this paper we describe a set of architectural metrics that are relevant to wireless IDS. A “scorecard” containing the set of values is used as the centerpiece of testing and evaluating a wireless IDS. Evaluation of a wireless IDS can be done by assigning score to various architectural metrics concern with wireless IDS. We apply our architectural metrics scorecard based evaluation approach to three popular wireless IDS Snort-wireless, AirDefense Guard, and Kismet. Finally we discuss the results and the opportunities for further work in this area. Keywords : Architectural Metrics, Wireless, Metrics, IDS, and Scorecard. GJCST-E Classification: C.2.0

An Architectural Metrics Scorecard Based Approach to Intrusion Detection System Evaluation for Wireless Network

Strictly as per the compliance and regulations of:

© 2012. Rupinder Singh & Dr. Jatinder Singh. This is a research/review paper, distributed under the terms of the Creative Commons Attribution-Noncommercial 3.0 Unported License http://creativecommons.org/licenses/by-nc/3.0/), permitting all non-commercial use, distribution, and reproduction inany medium, provided the original work is properly cited.

An Architectural Metrics Scorecard Based Approach to Intrusion Detection System Evaluation for Wireless Network

Rupinder Singh α & Dr. Jatinder Singh σ

Abstract - Wireless IDS architectural metrics are used to Lord Kelvin said “If you cannot measure it, you compare the intended scope, architecture of wireless IDS, and cannot improve it”. This fact also applies to wireless 2012 how they match the deployment architecture. These metrics network security issues. An activity cannot be managed June can be used to evaluate the architectural efficiency of a if it cannot be measured, this is a widely accepted wireless IDS and can help in designing efficient wireless IDS. management principle and security falls under this 1 Wireless IDS analyze wireless specific traffic including scanning for external users trying to connect to the network rubric. Metrics can be an effective tool for security through access points and play important role in security to providers to discern the effectiveness of various wireless network. Design of wireless IDS is a difficult task as components of their security programs. Metrics can play wireless technology is advancing every day, Architectural an important role in the designing of wireless IDS. metrics can play an important role in the design of wireless Security Metrics that are related to wireless network are IDS by measuring the areas concern with the architecture of a hard to generate because the discipline itself is still in wireless IDS. In this paper we describe a set of architectural the early stages of development. There is not yet a metrics that are relevant to wireless IDS. A “scorecard” common vocabulary and not many documented best containing the set of values is used as the centerpiece of practices to follow [1]. testing and evaluating a wireless IDS. Evaluation of a wireless IDS can be done by assigning score to various architectural This paper provides an architectural metrics metrics concern with wireless IDS. We apply our architectural scorecard based approach to evaluate Intrusion metrics scorecard based evaluation approach to three popular Detection Systems that are currently popular for wireless

wireless IDS Snort-wireless, AirDefense Guard, and Kismet. in the commercial sector. We describe a testing ) DDDD E

Finally we discuss the results and the opportunities for further methodology we developed to evaluate wireless IDS by ( work in this area. assigning score to various architectural metrics concern Keywords : Architectural Metrics, Wireless, Metrics, IDS, with it. The approach followed in this paper do not and Scorecard. compare wireless IDS against each other, but against a set of architectural metrics concern with wireless IDS. I. Introduction The generalized approach of this paper will new and exciting world has been opened by allow systems with any wireless requirements to tailor wireless. Its technology is advancing every day evaluation of ID technologies to their specific needs. Aand its popularity is increasing. The biggest Since evaluation is against a static set of architectural concern with wireless, however, has been its security, metrics the evaluation may be extended for other for some time wireless has had very poor, if any, security metrics like logistical metrics, performance metrics, on a wide-open medium. Along with improved quality metrics etc. The standard approach of encryption schemes, a new solution to help combat this comparison used in this paper also gives us scientific problem is the Wireless Intrusion Detection System repeatability. (WIDS). An Intrusion Detection System (IDS) is a device or software application that monitors network and/or II. Snort, Airdefense Guard and Kismet system activities for malicious activities or policy Wireless Ids violations and produces reports to a Management In order to explain architectural metrics Station (Wikipedia, 2012). A wireless IDS performs this scorecard based evaluation approach to wireless IDS, exclusively for the wireless network. This system Global Journal of Computer Science and Technology Volume XII Issue XI Version I we choose three wireless IDS namely Snort-wireless, monitors traffic on network looking for threats and AirDefense Guard, and Kismet as these are one of the alerting personnel to respond. most popular and works on different technology.

a) Snort Wireless IDS Author α : Assistant Prof., P.G. Deptt. of Computer Science and Snort is an open source network intrusion Applications, Khalsa College, Amritsar, Punjab, India. Author σ : Principal, Golden College of Engg. & Tech., Gurdaspur, detection and prevention system (IDS/IPS) that Punjab, India. E-mails : rupi_singh76@yaho o.com, combines the benefits of signature, protocol, and [email protected] anomaly-based inspection, and is the most widely

©2012 Global Journals Inc. (US) deployed IDS/IPS technology worldwide. With millions of Kismet identifies networks by passively collecting downloads Snort has become the de facto standard for packets and detecting standard named networks, IDS/IPS [4]. Snort-wireless allows for custom rules to be detecting hidden networks, and inferring the presence of created based on framing information from a wireless nonbeaconing networks via data traffic [8]. Kismet packet. It also contains rules to attempt to find rogue wireless IDS without sending any loggable packets is access points, war drivers, and ad hoc networks. able to detect the presence of both wireless access Snort-wireless works by implementing a points and wireless clients, and associate them with detection engine that allows registering, warning, and each other. Unlike most other wireless network responding to attacks previously defined. Snort-wireless detectors. Kismet has the ability to log all sniffed is available under GPL (General Public License) and packets and save them in a tcpdump /Wireshark or runs under Windows and GNU/Linux. It is among the Airsnortcompatible fileformat. Kismet also captures PPI most widely used, has a number of predefined headers. Kismet also has the ability to detect default or 2012 signatures and continuously updated. Snort wireless "not configured" networks, probe requests, and June can be configured in three modes namely sniffer, packet determine what levels of wireless encryptions is used on logger, and network intrusion detection. In addition to all a given access point. Kismet also supports logging of 2

of these basic Snort-wireless features, Snort-wireless the geographical coordinates of the network if the input can be set up to send real-time alerts. This provides with from a GPS receiver is additionally available [12].

the ability to receive alerts in real time, rather than having to continuously monitor Snort system. Snort is III. Architectural Metrics Scorecard like a vacuum that takes packets and allows to do Based Approach different things. a) Developing Scorecard b) AirDefense Guard Wireless IDS Centerpiece of testing and evaluating wireless AirDefense Guard is a wireless IDS that IDS will be a “scorecard” containing the set of provides advanced intrusion detection for wireless LANs architectural metrics and their definitions. Each metric based on signature analysis, policy deviation, protocol can have low (+), average (++), or high (+++) score, assessment policy deviation and statistically anomalous where higher scores will be interpreted as more behavior. AirDefense Guard is able to respond to favorable ratings. attacks with Active Defense technology, which interfaces The architectural metrics used are general )

DDDD with the access points to disconnect the attackers characteristics that are relevant to architecture of a E

( connection to the WLAN. wireless IDS. The method used for observing each AirDefense can be used to identity theft. This is architectural metric value can be either analysis (source done by stealing an authorized MAC address, an code analysis) or open source material (such as intruder has full access to the network. However, specifications, white papers or reviews provided by AirDefense tracks the digital fingerprints vendor-specific vendors or users). We use open source material to characteristics and personal trademarks of authorized analyze each architectural metrics for wireless IDS. We users to identify intruders in the network. AirDefense can examine publicly available research papers, reports, be used to detect Denial-of-Service (DoS) attacks. product documentation, published conference material AirDefense is able to quickly recognize the early signs (proceedings) and other material available for public and protocol abuses of a DoS attack that jams the review. airwaves and shuts down a wireless LAN. AirDefense b) Architectural Metrics for a Wireless IDS can also be used to detect Man-in-the-Middle attacks. Architectural metrics are used to compare the Posing as an access point, intruders can force intended scope and architecture of wireless IDS and workstations to disassociate from authorized access how they match the deployment architecture. These points and route all traffic through the intruder. The metrics evaluate the architectural efficiency of a Wireless intruder can then gain access to the network by posing IDS [13]. The metrics defined in this area are shown in as an authorized user and simultaneously operating on Table 1. Other Architectural metrics that may be multiple channels. AirDefense detects man-in-the- included are: Anomaly Based, Autonomous Learning, middle attacks and ensure that access points only

Global Journal of Computer Science and Technology Volume XII Issue XI Version I Host/OS Security, Interoperability, Package Contents, operate on set channels and proper protocols are used. Process Security, Signature Based, and Visibility [6]. c) Kismet Wireless IDS

Kismet is an 802.11 layer2 wireless network detector, sniffer, and intrusion detection system. Kismet will works with any wireless card that supports raw

monitoring mode, and can sniff 802.11b, 802.11a, 802.11g, and 802.11n traffic. Kismet also supports plugins that allows sniffing other media such as DECT.

© 2012 Global Journals Inc. (US) Table 1 : Selected architectural metrics for a wireless Table 2 : Scorecard for Snort wireless, AirDefense, and IDS Kismet wireless IDS. + : Low score ; ++ : Average score ; +++ : High Score Architectural Description Architectural Snort AirDefense Kismet Metrics Metrics wireless Guard Adju stable The difficulty of altering the sensitivity Sensitivity of a wireless IDS in order to achieve a Adjustable balance between false positive and Sensitivity +++ ++ ++ false negative error rates at various times and for different environments. Required data Requ ired Data The amount of disk space needed to Storage + ++ +++ Storage store logs and other application data. Capacity

Capacity 2012 Load Balancing

Load Balancing It measures the ability of a wireless June Scalability +++ +++ ++ Scalability IDS to partition traffic into

independent, balanced sensor loads. 3 Multiple Sensor The cardinality of sensors supported. Multiple Sensor Support Support +++ +++ +++ Reor dering and It can be used to find an attack that has Stream been artificially fragmented and Reassembly transmitted out of order. Reordering and Stream +++ +++ +++ State Tracking This metric is useful in hardening Reassembly wireless IDS against storms of random traffic used to confuse it. State Tracking Data Pool This metric is used to define the data +++ +++ +++ Selectability source to be analyzed for intrusions. Data Pool Selectability +++ +++ +++ Syste m Maximal data input rate that can be System Throughput processed successfully by the wireless ) DDDD Throughput +++ ++ ++ E

IDS. (

c) Architectural Metrics Scorecard Based Approach not done in all situations, but when it is detected the In this section we will apply above mentioned potential for false positives is very low [16]. So, kismet approach to popular wireless IDS Snort-wireless, gets average score for metric adjustable sensitivity. As AirDefense Guard, and Kismet. We choose these three described in [17] Air defense guard delivered a false for evaluation as they are one of the most widely used positive for a Nets tumbler scan that turned out to be and have different ways of working. Below with table 2 one of test laptops pinging an AP. Air Defense we describe how scores to architectural metrics related acknowledged that its Nets tumbler, signature needs to these three wireless IDS are assigned. some tweaking. So, it gets average score for metric Architectural metric Adjustable Sensitivity can adjustable sensitivity. Architectural metric Required Data Storage be assigned score depending on the following criteria: Capacity can be assigned score depending on the Low Score (+): No Adjustability. following criteria: Average Score (++): Adjustability via static methods. Low Score (+): Large capacity storage needed High Score (+++): Intelligent, dynamic Adjustability. to store log and other files. Snort-wireless makes use of the SSL Dynamic Average Score (++): Medium capacity storage Preprocessor (SSLPP), which decodes SSL and TLS needed to store log and other files. traffic and optionally determines if and when Snort- High Score (+++): Low capacity storage Global Journal of Computer Science and Technology Volume XII Issue XI Version I wireless should stop inspection of it. Encrypted traffic is needed to store log and other files. Databases are used ignored by Snort-wireless for both performance reasons with Snort wireless to store log and alert data. Logging and to reduce false positive and false negative error rate data to files in the disk is fine for smaller applications. [15]. So, Snort-wireless gets a high score (+++) for However, keeping log data in disk files is not metric adjustable sensitivity. Kismet wireless provides appropriate when there are multiple Snort-wireless alerts based on fingerprints (specific nets tumbler sensors or there is need to keep historical data as well. versions). In an attempt to disclose the SSID of a Databases also allow to analyze data generated by network, Nets tumbler sends out unique packets. This is Snort-wireless sensors. Snort-wireless uses rules stored

©2012 Global Journals Inc. (US) in text files that can be modified by a text editor. Rules only problem with this approach is that the data in the are grouped in categories. Rules belonging to each database is not strictly “real-time”. There is a certain category are stored in separate files. These files are then delay which depends upon frequency of uploading data included in a main configuration file called snort.conf. using SCP to the centralized database server. This Alerts are also stored in log files or databases where arrangement is shown in Figure 1 [7]. they can be viewed later on by security experts. Snort wireless needs a large database as its rules grows and gets a for this metric. Airdefense guard makes use of average data storage. Kismet wireless makes use of predefine rules and therefore needs less storage to store files. Architectural metric Load Balancing Scalability 2012 can be assigned score depending on the following June criteria:

4 Low Score (+): No load balancing scalability.

Average Score (++): Low load balancing scalability. High Score (+++): Highly capable of partitioning traffic into independent, balanced sensor loads. Figure 1 : Distributed Snort-wireless installation with the

If the bandwidth being passed by the network help of tools like SCP and Barnyard [7] interface associated with a Snort-wireless instance is Snort wireless gets a +++ score for this metric. greater than it can handle, more instances of Snort- The Air Defense solution is based on a Distributed wireless can be launched and the traffic can be load Collaborative Intelligence Architecture (DCIA), pioneered balanced across the instances. An Adaptive load by Air Defense, to provide the most comprehensive balancing architecture for snort is discussed in [18]. So, wireless intrusion protection. DCIA uses a dedicated snort wireless gets a +++ score for this metric. network of sensors and embedded client based agents Motorola AirDefene guard wireless IDS clients use a sophisticated load-balancing algorithm when too that continuously monitor the airwaves and wireless ) activity for attacks and policy violations. In addition, the DDDD E many clients attempt to connect to a particular access

( sensors use an intelligent channel scanning algorithm to point. The clients use a beacon element to perform pre- detect traffic across the RF spectrum. So, Air Defense emptive roaming and load balancing, thereby moving guard also gets a +++ score. Like Snort-wireless and from a heavily loaded AP to one that is less loaded. Air defense Kismet wireless also have a support to Kismet wireless is not as capable as Snortwireless and AirDefense Gurad for load balancing scalability. multiple sensors. Architectural metric Reordering and Stream Architectural metric Multiple Sensor Support Reassembly can be assigned score depending on the can be assigned score depending on the following criteria: following criteria: Low Score (+): Very less number of sensors Low Score (+): No capability to find an attack

supported. that has been artificially fragmented and transmitted out

Average Score (++): Average number of of order. sensors supported. Average Score (++): Very less capability to find

an attack that has been artificially fragmented and High Score (+++): Large number of sensors supported. transmitted out of order. A corporate environment probably have multiple High Score (+++): Highly capable to find an

locations and there is need to install Snort-wireless attack that has been artificially fragmented and

sensors. There are multiple ways to setup and install transmitted out of order. Snort-wireless in the enterprise as a distributed IDS. One The open source IDS Snort wireless implement

Global Journal of Computer Science and Technology Volume XII Issue XI Version I method is to connect multiple sensors to the same target-based analysis with the frag3 preprocessor.

centralized database. All data generated by these Frag3 is able to reassemble overlapping fragments

sensors is stored in the database. A user then uses a using the same policy as the destination host. A user

web browser to view this data and analyze it. configures the IDS to apply specific fragmentation

In an alternate mechanisms, Snort-wireless reassembly policies for individual hosts or networks.

sensors do not have a direct connection to the database Then, when the Snort sees overlapping fragments

server. The sensors may be configured to log to local bound for any of these hosts, it knows the appropriate

files. These files can then be uploaded to a centralized reassembly policy to apply—allowing both Snort and the server on a periodic basis using utilities like SCP. The destination host to reassemble the fragments identically.

© 2012 Global Journals Inc. (US) Snort wireless gets a +++ score as it is able to find an given network, on any given day, Snort can fire attack that has been artificially fragmented and thousands of alerts. Snort wireless makes use of ACID, transmitted out of order. AirDefense Guard and Kismet SGUIL, SnortSnarf, Snort_stat.pl, Swatch tools to wireless are also capable for out of order attaks. analyze intrusion data. AirDefense guard and kismet Architectural metric State Tracking can be wireless like Snort-wireless are also able to properly assigned score depending on the following criteria: select data pool. Low Score (+): No capability to detect storms Architectural metric System Throughput can be of random traffic used to confuse wireless IDS. assigned score depending on the following criteria: Average Score (++): Less capability to detect Low Score (+): Wireless IDS can successfully storms of random traffic used to confuse wireless IDS. process less data input rate. High Score (+++): High capability to detect Average Score (++): Wireless IDS can

storms of random traffic used to confuse wireless IDS. successfully process average data input rate. 2012

Snort wireless gets a high score for metric state High Score (+++): Wireless IDS can June

tracking as Snort wireless provides many configuration successfully process high data input rate.

and command line options to detect storms of random If Snort has to work with a high speed 5 traffic that can be specified in the snort configuration file. connection, then there is need to use unified logging Table 3 describes such commands. AirDefense guard and a unified log reader such as barnyard. This allows and Kismet wireless are also able to track state and gets Snort-wireless to log alerts in a binary form as fast as a +++ score. possible while another program performs the slow Architectural metric Data Pool Selectability can actions, such as writing to a database. AirDefense be assigned score depending on the following criteria: Guard and Kismet wireless process less data input rate Low Score (+): Poor capability to detect the as compare to snort wireless and both gets ++ score data source to be analyzed for intrusion. for the metric system throughput. Figure 2 shows score Average Score (++): Average capability to of Snort-wireless, Airdefense and Kismet IDS. detect the data source to be analyzed for intrusion. High Score (+++): Highly capable to detect IV. Conclusion and Future Work the data source to be analyzed for intrusion Unwanted activities on a wireless network can Snort wireless gets a +++ score for metric be detected by a wireless IDS. Architectural design of a ) DDDD data pool selectablity as Snort is a very complex pattern E wireless IDS is a difficult task as the technology of ( matcher geared toward detecting patterns of network design of wireless network is changing at a pace which attack traffic. On any brings additional challenges in the design of wireless Table 3 : Snort configuration commands IDS. This paper provides an architectural metrics scorecard based approach that can be used for Comm and Description evaluating a wireless IDS in order to find out the areas in which wireless IDS is weak and needs improvement. Enab le_decode_ Enables the dropping of bad packets Depending upon the requirements of the system these drops identiﬁed by decoder (only applicable metrics me given priorities and appropriate wireless IDS in inline mode). may be selected after developing the scorecard. Enab le_tcpopt_e Enables the dropping of bad packets In this paper we define various architectural xperimental_dro with experimental TCP option. (only metrics concern with wireless IDS and a scorecard ps applicable in inline mode). method to evaluate a wireless IDS by assigning scores to various architectural metrics. We use our evaluation Enab le_tcpopt_o Enables the dropping of bad packets methodology to test popular wireless IDS Snort-wireless, bsolete_ with obsolete TCP option. (only Air Defense Guard, and Kismet. This paper defines Drops applicable in inline mode). commonly used architectural metrics that are important to a wireless IDS, but a lot is required to be done to find

Enable_tcpopt_tt Enables the dropping of bad packets out more ones like anomaly based, autonomous cp_drops with T/TCP option. (only applicable in learning, Host/OS security, interoperability, package Global Journal of Computer Science and Technology Volume XII Issue XI Version I

inline mode). contents, process security, signature based, visibility etc. More architectural metrics and their definitions can

Enable_tcpopt_d Enables the dropping of bad packets be defined as lessons are learned while evaluating a rops with bad/truncated TCP option (only wireless network. Future work also includes applying the

applicable in inline mode). evaluation methodology to other metrics concern with

wireless IDS like logistical metrics, performance metrics, Enable_ipopt_dr Enables the dropping of bad packets quality metrics etc. ops with bad/truncated IP options (only applicable in inline mode). ©2012 Global Journals Inc. (US)

Snort-Wireless AirDefense Guard Kismet

3 ++:2, +++:4

, 0 :

+ 2

e r o

c 1 S 2012 0 June Adjustable Required data Load Balancing Multiple Sensor Reordering and State Tracking Data Pool System Sensitivity Storage Capacity Scalability Support Stream Selectability Throughput 6 Reassembly

Wireless IDS Logistic Metrics

Figure 2 : Graph showing score of Snort-wireless, Air Defense and Kismet wireless IDS

10. J. Gómez, C. Gil, N. Padilla1, R. Baños, and C. References Références Referencias Jiménez, “Design of a Snor – Based Hybrid 1. Rupinder Singh, Dr. Jatinder Singh, “A Metrics- Intrusion Detection System,” IWANN 2009, Part II, Based Approach to Intrusion Detection System LNCS 5518, 2009. Evaluation for Wireless Network,” International 11. Reijo Savola, “On the Feasibility of Utilizing Security Journal of Education and Applied Research Metrics in Software Intensive Systems,” IJCSNS, (IJEAR)Vol.1, Issue 1, Ver. 1: Jul.- Dec., 2011, ISSN : VOL. 10 No.1, January 2010. 2249-4944. 12. http://en.wikipedia.org/wiki/Kismet )

DDDD E 2. http://downloads.visionid.ie/wireless/Dedicated_Dist 13. Snehal Boob and Priyanka Jadhav, “Wireless

( ributed_Sensing_The_Right_Approach_to_Wireless_ Intrusion Detection System,” International Journal of Intrusion_Prevention.pdf Computer Applications (0975–8887) Volume 5– 3. Motorola Enterprise WLAN Design Guide Volume No.8, August 2010. November 2008, Available at: http://www.scantexas. 14. http://www.snort.org/ com/asset/support/Motorola%20Enterprise%20WLN 15. SNORT Users Manual 2.9.1, the Snort Project %20Design %20Guide.pdf September 20, 2011. Available at: 4. Stephen Northcutt, “Snort 2.1 Intrusion Detection,” http://www.snort.org/. assets/166/snort_manual.pdf Second Edition, Shroff Publishers, ISBN: 81-7336- 16. http://www.kismetwireless.net/documentation.html 894-9. 17. http://www.networkcomputing.com/wireless/229622 5. G. A. Fink, B. L. Chappell, T. G. Turner, and K. F. 63?printer_friendly=this-page O’Donoghue, “A Metrics - Based Approach to 18. M. Alam, Qasim Javed, M. Akbar, “Adaptive load Intrusion Detection System Evaluation for balancing architecture for snort,”. http://www. Distributed Real – Time Systems,” WPDRTS April geocities.ws/raza_nust/incc.pdf 2002, Ft. Lauderdale, Florida. 6. Harrykar Freelance, “HARRYKAR’S TECHIES BLOG Snort, IDS, IPS, NSM, hacking andbeyond,” 31 May 09. 7. Rafeeq Ur Rehman,“Intrusion Detection System with Snort Advanced IDS Techniques Using Snort, Global Journal of Computer Science and Technology Volume XII Issue XI Version I Apache, MySQL, PHP, and ACID,” Prentice Hall, ISBN 0-13-1407 33-3. Available at: http://ptgmedia. pearsoncmg.com/images/013147333/downloads/ 0131407333.pdf 8. http://www.kismetwireless.net/ 9. SNORT Users Manual 2.9.0, Snort Project, March 2011