A Multi-Resolution Approach for Worm Detection and Containment∗ Vyas Sekar Yinglian Xie Michael K. Reiter Hui Zhang [email protected] [email protected] [email protected] [email protected] Carnegie Mellon University Abstract used for detection and containment. Future attacks can evade detection mechanisms which depend on scanning Despite the proliferation of detection and contain- rates, signatures, and other attack-specific features. ment techniques in the worm defense literature, sim- Interestingly, one of the earliest known scan- ple threshold-based methods remain the most widely de- detection heuristics, threshold-based detection based on ployed and most popular approach among practitioners. the number of unique destinations contacted, is applica- This popularity arises out of the simplistic appeal, ease ble across a wide spectrum of worm attacks. Threshold- of use, and independence from attack-specific properties based mechanisms are very popular and are one of the such as scanning strategies and signatures. However, most widely-deployed worm defenses [12] due to their such approaches have known limitations: they either simplicity and ease of deployment. The strength and ro- fail to detect low-rate attacks or incur very high false bustness of the mechanism lies in its minimal set of as- positive rates. We propose a multi-resolution approach sumptions about the nature of attacks – scanners contact to enhance the power of threshold-based detection and many unique destinations. By adopting a metric which rate-limiting techniques. Using such an approach we is invariant across scanning attacks, independent of the can not only detect fast attacks with low latency, but scanning strategy and content signatures, this approach also discover low-rate attacks – several orders of mag- has the ability to be attack-agnostic. nitude less aggressive than today’s fast propagating at- However, threshold-based detection mechanisms cur- tacks – with low false positive rates. We also outline a rently lack the accuracy and effectiveness of attack- multi-resolution rate limiting mechanism for throttling specific approaches. Having only a single fixed thresh- the number of new connections a host can make, to con- old for a metric (such as the number of unique desti- tain the spread of worms. Our trace analysis and sim- nation addresses contacted), typically measured over a ulation experiments demonstrate the benefits of a multi- single time window a few seconds long, network admin- resolution approach for worm defense. istrators must make a choice in the selection of the detec- tion threshold. The choice is between a high threshold that can detect only very high-rate attacks but has low 1. Introduction false positive rates, and a low threshold that can detect low-rate stealthy attacks but that may have a very high Worms pose a significant threat to the depend- false positive rate. This fundamental inflexibility limits ability of existing and future networking infrastruc- the practical applicability of threshold-based approaches ture. Defending against such self-propagating attacks to high-rate attacks. A natural question is: can we re- in an automated fashion is a challenging task, and has tain the attack-agnostic properties of threshold-based de- sparked much interest in the research community. Ex- tection, but provide detection capabilities comparable to isting approaches for worm defense (e.g., [3, 7, 13, 18]) attack-specific approaches? have been shown to be effective for very fast, non- Our solution is a multi-resolution approach for de- polymorphic, random scanning worms. However, they tecting and containing worms, without depending on leave open to attackers opportunities to circumvent the attack-specific scanning properties and signatures. The defense mechanisms by exploiting the very assumptions key insight behind the multi-resolution approach is the ∗This work was partially supported by NSF grant number CNS- following simple yet powerful observation. While the 0433540, and by KISA and MIC of Korea. short term connection patterns of normal end-hosts may be bursty, involving a large volume of traffic and con- sumption and avoid overloading network and router re- nections to many unique destination addresses, hosts ex- sources. Second, such approaches can curb the internal hibit lower average connection rates when observed over spread of worms that exploit topological locality. longer timescales. As a result, we find that connection Chen and Tang [2] propose worm detection and con- metrics, such as the traffic volume and the number of tainment based on connection failure rates. Jung et al. distinct destinations contacted, grow as a concave func- use sequential hypothesis testing [6, 13] to detect scan- tion of the size of the time window (i.e., the second ners by tracking failed connection attempts. Our ap- derivative with respect to the time window size is nega- proach is agnostic to the scanning strategy since it does tive). This suggests that using multiple resolutions with not rely on failed connections. different detection thresholds at different time granular- Several worm containment methods have been sug- ities will be an effective solution to detect a wide range gested in the literature, including rate-limiting, quaran- of attack rates with low false positive rates. tine, and signature-based filtering. Moore et al. [11] Our traffic analysis (Section 3) confirms this intu- study the limits on the responsiveness of content- ition, and indicates the potential benefits of a multi- filtering and address-blacklisting as containment mea- resolution approach. We provide a systematic frame- sures, while Wong et al. [18] discuss the effectiveness of work (Section 4) for realizing these benefits, by bal- rate-limiting mechanisms. Zou et al. [20] present an an- ancing the inherent tradeoff between the false positive alytical framework for reasoning about worm propaga- rate and the detection latency (and hence the potential tion in the presence of defense mechanisms. Williamson damage caused by infected hosts). We define the se- proposed the virus throttle [17] based on the observa- curity cost of a system, in terms of the false positive tion that the number of connections to previously un- rates and detection latencies, and outline an optimiza- contacted hosts is fairly low. While the class of contain- tion framework for selecting parameters optimally for a ment measures we evaluate have been proposed earlier multi-resolution detection system. in these contexts, our contribution is the design and eval- Our multi-resolution approach for containment (Sec- uation of a multi-resolution approach for rate limiting. tion 5) draws upon a similar insight in the nature of end- Multi-resolution analysis in spatial and temporal di- host behavior. Locality in destination address selection mensions, using Fourier and wavelet analysis, has been suggests that throttling connections to new destinations suggested for anomaly detection (e.g., [1, 4]). Calculat- that have not been contacted previously, will achieve the ing the number of unique destinations contacted over desired containment capability without disrupting the multiple time resolutions necessarily involves taking activity of normal hosts. Our evaluations demonstrate unions of the set of destinations contacted over multiple that a multi-resolution approach achieves enhanced con- time bins. Signal analysis techniques are not suitable in tainment capabilities over traditional approaches. this context as they cannot capture the semantics of such a union operation for multi-resolution analysis. 2. Related Work 3. Motivation Prior work has focused on understanding differ- ent worm propagation models (e.g., [10, 16]). Many In threshold based anomaly detection, the traffic techniques have been proposed for detecting worm monitor identifies abnormal activity by measuring spe- outbreaks using either large-scale monitoring infras- cific traffic metrics and flagging suspicious observa- tructures (e.g., [19]) or locally deployed honeypots tions which exceed a pre-set threshold within a spe- (e.g., [5]). There are also several systems for efficient cific time window. Commonly used metrics for de- and fast worm signature generation (e.g., [3, 7, 14]). tecting abnormal host behavior include the total traffic There has been surprisingly little work on detec- volume (number of packets or flows) and the number tion of stealthy, low-rate, scanning attacks. Staniford of unique destination addresses contacted (regardless of et al. describe a mechanism for detecting stealthy port whether the connection was successful or not). Despite scans [15] arising outside the network, by using a his- their widespread deployment, threshold-based mecha- torical probability model for different types of traffic. nisms suffer from an inherent inflexibility arising from Our work focuses on detecting and throttling infected the conflicting goals in threshold selection. A large (i.e, hosts inside a local network similar to [9, 13]. There are conservative) threshold that accounts for normal traffic two compelling reasons for deploying such capabilities. bursts will not be able to detect low-rate attacks, while a First, rate limiting can reduce wasteful bandwidth con- small (i.e., aggressive) threshold will result in high false 40 40 positive rates where even small bursts of legitimate ac- Day 2 99th percentile 35 Day 4 35 99.1 percentile tivity will be flagged as potentially anomalous. Day 6 99.3 percentile 30 30 99.5 percentile With respect to worm detection, the metric of inter- 25 25 est is the number of unique destination addresses con- 20 20 tacted. If the number of unique destination addresses 15 15 contacted by a benign host grows as a linear function 10 10 Outdegree (# hosts contacted) 5 Outdegree (# hosts contacted) 5 of the time window, then a single-resolution approach 0 0 0 50 100 150 200 250 300 350 400 450 500 0 50 100 150 200 250 300 350 400 450 500 operating with a fixed threshold is sufficient, as it will Window size in seconds Window size in seconds uniquely identify the (malicious) scanning rates we can (a) Growth of 99.5th per- (b) Growth of different statis- detect.