Proceedings of the 3rd Cyberspace Research Workshop
15 November 2010 Shreveport, LA
Editors
Jean Gourd Louisiana Tech University
Vir V. Phoha Louisiana Tech University
S.S. Iyengar Louisiana State University
The Cyberspace Research Workshop is hosted by the Center for Secure Cyberspace (CSC), a collaboration between Louisiana Tech and Louisiana State Universities. Funding for the CSC is made possible by a grant from the Louisiana Board of Regents Support Fund. LEQSF(2007-12)-ENH-PKSFI-PRS-03 GENERAL CHAIR Lt. Gen. Robert J. Elder, USAF, Ret.
CONFERENCE CHAIR Les Guice
PUBLICATION & PROCEEDINGS CHAIRS Jean Gourd Vir V. Phoha
PUBLICITY CHAIRS FOR ANNOUNCEMENT & WEB Christian Duncan Jean Gourd
LOCAL ARRANGEMENT CHAIRS Brenda Brooks Jean Gourd
TECHNICAL PROGRAM COMMITTEE V. V. Phoha, S. S. Iyengar, Chair Chair D. Ali G. Allen T. Atkison K. Balagani S. Dua C. Duncan J. Gourd Md. E. Karim T. Kosar A. Lakhotia A. Ray T. Roberts R. Selmic P. Wahjudi J. Walczyk J. Zhang CRW'10: 3rd Cyberspace Research Workshop 15 November 2010
MESSAGE FROM THE GENERAL CHAIR
Welcome to the 3rd annual Cyberspace Research Workshop. This event is an opportunity to discuss opportunities to advance cyberspace research in innovative ways. Past workshops have been conducted in conjunction with CIC-sponsored Cyberspace Symposiums; this year we are holding the event alongside the Air Force Global Strike Command's global conference, giving us a different perspective to think about the future of cyberspace and its role in United States national security and economic development. We look forward to a day of exciting discourse on the impact of cyberspace on America's future, in particular, areas where research is needed to maintain United State's competitiveness as a global power. Many thanks for your participation! I look forward to discussing this topic with you and hearing your perspectives on the role of cyberspace in America's future.
Sincerely,
Dr. Robert Elder, Lt General, USAF (retired) CRW'10: 3rd Cyberspace Research Workshop 15 November 2010
TABLE OF CONTENTS
Strategic Methods in Adversarial Classifier Combination...... 1 Anshuman Singh and Arun Lakhotia
Method for Cyber-Security Risk Assessment and Cost-Efficient Management in Wireless Sensor Networks...... 8 Mehmet Sahinoglu
Pollination in MAIDS: Detecting and Combating Passive Intrusions in a Multi-Agent System...... 14 Jeremy Kackley, James Jacobs, Paulus Wahjudi and Jean Gourd
Micro-Aerial Vehicle and Sensor Networks Laboratory Development and Applications.21 Miguel Gates, Christopher Barber, Rastko Selmic, Christian Duncan and Jinko Kanno
Application of Context to Fast Contextually Based Spatial Authentication Utilizing the Spicule and Spatial Autocorrelation...... 27 Greg Vert, Jean Gourd and S.S. Iyengar
Network Maneuver Commander - Resilient Cyber Defense...... 35 Paul Beraud, Alen Cruz, Suzanne Hassell, Juan Sandoval and Jeffrey J. Wiley
Zero Knowledge Trust Propagation in Airborne Networks...... 44 Stephan White, David Irakiza, Abena Primo and Joseph M. Kizza
GSM Security Threats and Countermeasures...... 48 Saravanan Bala, Tanvir Ahmed , Samuel Kasimalla and Travis Atkison
Securing a Wireless Network: WPA Attacked and Defended...... 52 Hend Nuhait, Chetan Sakpal and Travis Atkison
Defending Against Cross Site Scripting...... 56 Harry May, Apurba Dhungana, Jundong Chen and Travis Atkison CRW'10: 3rd Cyberspace Research Workshop 15 November 2010
A Different Approach to Network Security: The Best Defense is a Good Offense...... 60 Miguel Gates, Umesh Dhital, Timothy Lindsay and Travis Atkison
A Call for the Consideration of Individual Differences in Cyber Security...... 65 John Buckner and Tilman Sheets
Building Secure, Resilient Architectures for Cyber Mission Assurance...... 72 Harriet Goldman Strategic methods in adversarial classifier combination ∗
Anshuman Singh and Arun Lakhotia University of Louisiana at Lafayette {axs6222,arun}@louisiana.edu
ABSTRACT a whole. In this paper, we focus on strategic design of com- We present an analytical method of configuring performance plex security detection systems or more technically, multiple parameters of individual classifiers in a multiple classifier classifier systems. system (MCS) such that system as a whole incurs minimum Strategic design considers cost sensitive decisions by ad- misclassification costs. We use game theory for this analysis versary and classifier systems. Strategic decision making is as it captures the strategic interdependence between the ad- considered an art and generally exercised by security experts versary and defender using the MCS. We consider the prim- based on their experience. Instead, the focus is mostly on itive combinations of MCS - the OR, AND and SELECT for computational issues in solving security problems. Though, our analysis. The analysis is based on cost-sensitive classifi- this approach is effective in the short run, in the long run cation where the objective is not just to maximize the detec- it inevitably leads to a race where more often than not ad- tion rate or minimize the false positive rate but to minimize versary is ahead and the defender is forced to take a reac- the total expected costs. This is more practical since in MCS tive approach. A more proactive approach is needed where like malware detectors the false positives can lead to high re- strategic decisions are not left to experts based on their ex- sponse costs and even if the detection rate is high enough, perience but are determined in a more reasoned, scientific the strategy of minimizing the expected costs is better than and a formal way. maximizing the detection rate. Strategic design of multiple classifier systems involves choos- ing the optimal combination method that is robust against adversarial inputs. Adversarial inputs are inputs that have 1. INTRODUCTION been modified so that they are misclassified by the secu- Many organizations deploy detection systems like malware rity system. At a finer level of granularity, strategic design detectors, intrusion detectors, vulnerability detectors, spam of classifier combinations involves determining the optimal detectors etc. to secure their networks. It is not uncom- performance parameters for component classifiers so that mon to find, for example, a malware detector being used in appropriate classifiers can be chosen using required level of conjunction with intrusion detector or an intrusion detec- training. In an adversarial setting, the optimal performance tor being used with spam detector. The individual detec- parameters of component classifiers system will depend on tors may further be composed of many specialized detectors. the choice of evasion or obfuscation method used by the For example, most malware detectors are composed of pack- adversary. This strategic interaction can be modeled using ing detectors, x-ray scanners, checksum matchers, behavior game theory. So far, statistical decision theory has been matchers etc. that work together to correctly classify an used in designing optimal classifier systems. We consider a input [13]. game-theoretic modeling and analysis for designing adver- There are choices in designing a particular detector and sarial classifier combinations. each one of the its component detectors. The design choices The paper is organized as follows. Section 2 briefly de- make a decision on the tradeoff between detection accuracy scribes related works in adversarial classification and learn- and cost (efficiency, development cost and time, research ing. Section 3 describes the cost-sensitive models of classifier cost and time, ease of use etc.). These design choices are and adversary. Basic concepts of multiple classifier systems generally exercised on individual detectors or classifiers but and game theory are also described. Section 4 presents the rarely exercised keeping in mind the security of the system as game-theoretic analysis for configuring primitive combina- tions of classifiers. Finally, in section 5, we summarize the ∗This research was supported in part by funds from Air Force Office of Scientific Research grant AF9550-09-1-0715. paper.
2. RELATED WORK Adversarial classification was first presented in [6]. In Permission to make digital or hard copies of all or part of this work for this paper, the interaction between adversary and classifier personal or classroom use is granted without fee provided that copies are is modeled as an extensive game where both classifier and not made or distributed for profit or commercial advantage and that copies adversary are cost-sensitive and classification function and bear this notice and the full citation on the first page. To copy otherwise, to feature change function are assumed to be public informa- republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. tion. The adversary first decides the minimum-cost feature Copyright 20XX ACM X-XXXXX-XX-X/XX/XX ...$10.00. change strategy followed by the classifier adapting the clas-
1 sification function to the possibility of feature change by the Also called hit rate, recall and sensitivity. adversary. The Nash equilibrium is then computed which Positives correctly classified T P T P constitutes the optimal classification function for the classi- tp rate = = = Total positives T P + F N P fier and minimum cost feature change function for adversary. The situation when adversary doesn’t know the classifi- cation function in advance was given in [10]. This paper False positive rate. presented a new learning paradigm suitable for adversar- Also called false alarm rate. ial problems, called adversarial classifier reverse engineering Negatives incorrectly classified F P F P (ACRE), where the goal is not to learn the entire decision fp rate = = = surface and adversary’s success is measured relative to a cost Total Negatives F P + T N N model. The adversary tries to learn instances that are not labeled malicious in polynomial number of queries. Cost Matrix. Adversarial classification and adversarial learning has been Given a classifier C : X Ω, where Ω = 1, 2,...,m the topic of two very recent doctoral dissertations [2, 12]. is the class space, the performance→ of C can{ be described} Design of pattern recognition systems for adversarial clas- using an m m matrix Ccost = [cij ] where cij is the cost of sification tasks is studied in [2, 5, 4, 3]. In [2], an analysis assigning class× j to an instance of input with true class i for of different attacks on different stages of a pattern recogni- i, j = 1, 2,...,m. The cost of correct classification is zero, tion systems (data preprocessing, feature extraction, model i.e. cii = 0, i = 1, 2,...,m. training etc.) is given. Then, a methodology of evaluating the robustness of a classifier at design phase is proposed. Classification related costs. Finally, the use of multiple classifier systems is proposed for Costs related to classification for security applications com- design of robust systems for adversarial pattern recognition. prises of the following three types [9]: Multiple classifier systems can also be attacked depending on their architecture, and this possibility is studied in our 1. Operational Cost: This involves the amount of com- work. In [12], a case-study of how learners can be manip- puting resources needed to make the classification de- ulated by poisoning the training data is presented. It also cision. presents another case study where evasion using minimum 2. Damage Cost: It characterizes the damage done when queries of already trained classifier is presented. the classifier misses an attack. Most of the works above study single classifiers in adver- sarial environment. These works suggest that there is no 3. Response Cost: It is cost of responding when there is solution yet to the problem of architectural vulnerabilities a positive classification (or an alarm) by the classifier of complex classification systems that can be attacked to irrespective of whether it is correct or not. the the advantage of the adversary. This is precisely the problem addressed in this paper. Cost-sensitive adversary. A cost-sensitive adversary will have to incur following types of cost: 3. BACKGROUND Feature-change cost (Obfuscation cost): This is the • cost of making changes to the features in the input Classifier. for evading classifiers. A classifier C is a function C : X Ω, where X = → Learning cost: This is the cost of making changes to x1,x2,...,xn is the input space and Ω = ω1,ω2,...,ωm • { } { } the feature change function when a modified input gets is the class space. The set Ω is discrete and finite for crisp correctly classified. label classification. It is the set [0, 1] for soft labels repre- senting probabilities of being in a class. Multiple classifier systems. Binary Classifier (Detector). Multiple classifier systems are designed for accuracy greater than that of the component classifiers. The two major com- A binary classifier Cb is a function Cb : X B, where → bination methods known in literature for designing multiple X = x1,x2,...,xn is the input space and B = +, is the class{ space. A binary} classifier is also called a{ detector−} . classifier systems are fusion and selection.
Confusion Matrix. Fusion combination. Given a classifier C : X Ω, where Ω = 1, 2,...,m is The fusion combination consists of classifiers connected the class space, the performance→ of C can be described{ using} together in parallel so that for any given input all classifiers are run, and the outputs are combined using some deci- an m m matrix Cconf = [nij ] where nij is the number of instances× of input with true class i that were classified as sion function. Decisions of individual classifiers are fused class j for i, j = 1, 2,...,m [7]. when entire feature space is the input to all classifiers and We consider boolean classifiers in the following definitions. error rate of all classifiers are almost identical. The most commonly used fusion function is the majority vote. More Also, given the confusion matrix, Cconf of a boolean clas- precisely, given classifiers = C1,C2,...,C and a com- sifier, T P = n11, F P = n21, F N = n12, T N = n22, n bination function f : 2C C Ω,{ the fusion combination} of a P = n11 + n12 and N = n21 + n22. multiple classifier system→ is defined as:
True positive rate. CMCS (x)= f(C1(x),C2(x),...,Cn(x))
2 Selection combination. 4. CONFIGURINGPRIMITIVE COMBINA- The selection combination consists of classifiers connected TIONS together using lightweight classifiers called selectors. Each classifier is an expert on a part of the feature space. Selector In section 4.1, we present the cost model that will be decides which part of the feature space needs to be used for used in the game-theoretic analysis of the primitive clas- classification and the input is directed to the corresponding sifier combinations. It can shown that majority voting used in fusion combination can be implemented using primitive classifier. Given classifiers C1,C2,C3 with C1 as selector, combinations functions boolean AND and boolean OR (See selection combination function can be written as: appendix). So we analyze the primitive combinations of OR C2(x) if C1(x)=+ and AND in addition to selection. In sections 4.2, 4.3 and CSEL(x)= 4.4, we present the game-theoretic analysis of SELECT, OR (C3(x) if C1(x)= − and AND combinations to obtain optimal detection rates and false positive rates that minimize the expected cost. Game theory. “Game theory concerns the behaviour of decision makers 4.1 Costmodel whose decisions affect each other” [1]. Game theory is a gen- We now give the cost matrix of adversary and multiple eralization of decision theory. Decision theory is essentially classifier system (MCS). The damage cost when MCS misses one person game theory. an attack is given by d. The response cost when there is an alarm or a positive classification by MCS is r. ϕ is the Normal form game.. feature change cost for the adversary. The learning cost for the adversary when the classifier correctly detects an attack It is a tuple < N,Ai, i> where N is a set of players is λ. The cost matrix of MCS and adversary is summarized indexed by i, Ai is the set of actions for player i where in figure 1. i N and i is the preference relation of player i defined ∈ Let pD denote the detection rate (true positive rate) of the on the set A = j∈N Aj , the outcomes of the game. In most cases, the preference× relation can be represented by a von classifier, i.e. the probability that classifier correctly detects true class of the input. Since the classifier can also give false Neumann Morgenstern utility function ui : A . → R positives, we denote the false positive rate by pF . A classi- Extensive form game.. fier can be configured to operate at a specific combination of (p ,p ) values on its Receiver Operating Characteristics Since order of moves is relevant in an extensive form game, D F (ROC) curve, which specifies the permissible combinations the concept of terminal history is used to describe the se- for the classifier. An ROC curve represents p as an increas- quence of actions that lead to an outcome. An extensive D ing concave function of p . We assume that the ROC curve form game is a tuple < N,Z,P, > where N is a set of F i is given by the power function p = pr , with 0