DOCSLIB.ORG

Detection of Botnets Using Combined Host- and Network-Level Information Yuanyuan Zeng, Xin Hu, Kang G

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Detection of Botnets Using Combined Host- and Network-Level Information Yuanyuan Zeng, Xin Hu, Kang G Detection of Botnets Using Combined Host- and Network-Level Information Yuanyuan Zeng, Xin Hu, Kang G. Shin University of Michigan, Ann Arbor, MI 48109-2121, USA {gracez, huxin, kgshin}@eecs.umich.edu 1. INTRODUCTION botnet at the network level and the malicious behavior Botnets have now become one of the most serious each bot exhibits at the host level, we propose a C&C security threats to Internet services and applications; protocol-independent detection framework that incor- they can mount Distributed-Denial-of-Service (DDoS) porates information collected at both the host and the attacks, spamming, phishing, identity theft, and other network levels. The two sources of information com- cyber crimes. plement each other in making detection decisions. Our To control a botnet, a botmaster needs to use a C&C framework first identifies suspicious hosts by discovering channel to issue commands and coordinate bots’ ac- similar behaviors among different hosts using network- tions. Although IRC- and HTTP-based C&C have been flow analysis, and validates the identified suspects to be adopted by many past and current botnets, both of malicious or not by scrutinizing their in-host behavior. them are vulnerable to a central-point-of-failure. That Since bots within the same botnet are likely to receive is, once the central IRC or HTTP server is identified the same input from the botmaster and take similar and removed, the entire botnet will be disabled. actions, whereas benign hosts rarely demonstrate such To counter this weakness, attackers have recently shifted correlated behavior, our framework looks for flows with toward a new generation of botnets utilizing decentral- similar patterns and labels them as triggering flows. It ized C&C protocol such as P2P. This C&C infrastruc- then associates all subsequent flows with each trigger- ture makes detection and mitigation much harder. A ing flow on a host-by-host basis, checking the similarity well-known example is the Storm worm (a.k.a. Nuwar, among those associated groups. If multiple hosts be- W32.Peacomm, and Zhelatin) [1] which spreads via email have similarly in the trigger-action patterns, they are spam and is known to be the first malware to seed a grouped into the same suspicious cluster as likely to be- botnet in a hybrid P2P fashion. Storm uses peers as long to the same botnet. Whenever a group of hosts HTTP proxies to relay C&C traffic and hides the bot- is identified as suspicious by the network analysis, the masters well behind the P2P network. A recent spam- host-behavior analysis results, based on a history of bot Waledac, which appeared at the end of 2008, also monitored host behaviors, are reported. A correlation spreads via spam emails and forms its botnet using a algorithm finally assigns a detection score to each host C&C structure similar to the Storm botnet. Some re- under inspection by considering both network and host searchers pointed out that Waledac is the new and im- behaviors. proved version of the Storm botnet [2]. Our contributions are three-fold. First, to the best To date, most botnet-detection approaches operate of our knowledge, this is the first framework that com- at the network level; a majority of them target tra- bines both network- and host-level information to de- ditional IRC- or HTTP-based botnets [3, 4, 5, 6, 7, tect botnets. The benefit is that it completes a detec- 8] by looking for traffic signatures or flow patterns. tion picture by considering not only the coordination We are aware of only one approach [9] designed for behavior intrinsic to each botnet but also each bot’s in- protocol- and structure-independent botnet detection; host behavior. For example, it can detect botnets that it depends on network traffic analysis and is unlikely appear stealthy in network activities with the assistance to have a complete view of botnets’ behavior. We thus of host-level information. Moreover, we extract features need the finer-grained host-by-host behavior inspection from NetFlow data to analyze the similarity or dis- to complement the network analysis. On the other similarity of network behavior without inspecting each hand, since bots behave maliciously system-wide, gen- packet’s payload, thus preserving privacy. Second, our eral host-based detection can be useful. One such way detection relies on the invariant properties of botnets’ is to match malware signatures, but it is effective in de- network and host behaviors, which are independent of tecting known bots only. To deal with unknown bot in- the underlying C&C protocol. It can detect both tra- filtration, in-host behavior analysis [10, 11, 12, 13, 14] is ditional IRC and HTTP, as well as recent hybrid P2P needed. However, since in-host mechanisms are vulner- botnets. Third, our approach is evaluated by using sev- able to host-resident malware, host-based approaches eral days of real-world NetFlow data from a core router alone can hardly provide reliable detection results and of a major campus network containing benign and bot- thus we need external, hard-to-compromise (i.e., network- net traces, as well as multiple benign and botnet data level) information for detection of bots’ malicious be- sets collected from virtual machines. Our preliminary havior. results show that the proposed framework can detect Considering the required coordination within each different types of botnets with low false-alarm rates. 1 H o s t A n a l y z e r H o s t A n a l y z e r I n H o s t M o n i t o r I n H o s t M o n i t o r I n H o s t I n H o s t N e t w o r k A n a l y z e r S u s p i c i o n L e v e l C a m p u s R o u t e r S u s p i c i o n L e v e l G e n e r a t o r G e n e r a t o r N e t F l o w D a t a F l o w A n a l y z e r C l u s t e r i n g C l u s t e r i n g R e s u l t s S u s p i c i o n > L e v e l a n d N e t w o r k F e a t u r e s D e t e c t i o n R e p o r t S u s p i c i o n > L e v e l a n d N e t w o r k F e a t u r e s C o r r e l a t i o n E n g i n e Figure 1: System architecture 2. SYSTEM ARCHITECTURE connected via a virtual network to monitor and collect Figure 1 shows the architecture of our system which their activities at the Registry, file system, and network primarily consists of three components: host analyzer, stack. Table 1 shows the details of these botnet traces, network analyzer, and correlation engine. each containing 4 bot instances. IRC-rbot and IRC- As almost all of current botnets target Windows ma- spybot’s traces were generated by running their mod- chines, our host analyzer is designed and implemented ified source code in the virtual network. We obtained for Windows platforms. The host analyzer is deployed the binaries of HTTP-based BobaxA and BobaxB, and at each host and contains two modules: in-host moni- hybrid-P2P-based Storm and Waledac from public web tor and suspicion-level generator. The former monitors sites. The IRC- and HTTP-based botnets’ network- run-time system-wide behavior taking place in the Reg- level traces were captured within a controlled environ- istry, file system, and network stack on a host. The lat- ment and transformed from packet data to flow data ter generates a suspicion-level by applying a machine- in our experiment. Since Storm and Waledac botnets learning algorithm based on the behavior reported at were still active in the wild when we collected data, each time window and computes the overall suspicion- to make it more realistic, we carefully configured the level using a moving average algorithm. The host ana- firewall setting and connected virtual machines to the lyzer sends the average suspicion-level along with a few outside network so that the bots actually joined the real network feature statistics to the correlation engine, if Storm and Waledac botnet and the campus router cap- required. The network analyzer also contains two mod- tured all of the bots’ traffic. We also collected 5-day ules: flow analyzer and clustering. The flow analyzer NetFlow data from a core router in our campus net- takes the flow data from a router as input and searches work which covered the flows generated by Storm and for trigger-action botnet-like flow patterns among dif- Waledac instances and all other hosts in the network. ferent hosts. It then extracts a set of features that can The 5-day data consists of three sets: (1) 2-day data best represent those associated flows and transforms that contains 48-hour Storm traces; (2) 1-day data in- them into feature vectors. Those vectors are then fed cluding Waledac; and (3) other 2-day data.
Load more

Recommended publications
  • Botnets, Cybercrime, and Cyberterrorism: Vulnerabilities and Policy Issues for Congress
    Order Code RL32114 Botnets, Cybercrime, and Cyberterrorism: Vulnerabilities and Policy Issues for Congress Updated January 29, 2008 Clay Wilson Specialist in Technology and National Security Foreign Affairs, Defense, and Trade Division Botnets, Cybercrime, and Cyberterrorism: Vulnerabilities and Policy Issues for Congress Summary Cybercrime is becoming more organized and established as a transnational business. High technology online skills are now available for rent to a variety of customers, possibly including nation states, or individuals and groups that could secretly represent terrorist groups. The increased use of automated attack tools by cybercriminals has overwhelmed some current methodologies used for tracking Internet cyberattacks, and vulnerabilities of the U.S. critical infrastructure, which are acknowledged openly in publications, could possibly attract cyberattacks to extort money, or damage the U.S. economy to affect national security. In April and May 2007, NATO and the United States sent computer security experts to Estonia to help that nation recover from cyberattacks directed against government computer systems, and to analyze the methods used and determine the source of the attacks.1 Some security experts suspect that political protestors may have rented the services of cybercriminals, possibly a large network of infected PCs, called a “botnet,” to help disrupt the computer systems of the Estonian government. DOD officials have also indicated that similar cyberattacks from individuals and countries targeting economic,
    [Show full text]
  • A the Hacker
    A The Hacker Madame Curie once said “En science, nous devons nous int´eresser aux choses, non aux personnes [In science, we should be interested in things, not in people].” Things, however, have since changed, and today we have to be interested not just in the facts of computer security and crime, but in the people who perpetrate these acts. Hence this discussion of hackers. Over the centuries, the term “hacker” has referred to various activities. We are familiar with usages such as “a carpenter hacking wood with an ax” and “a butcher hacking meat with a cleaver,” but it seems that the modern, computer-related form of this term originated in the many pranks and practi- cal jokes perpetrated by students at MIT in the 1960s. As an example of the many meanings assigned to this term, see [Schneier 04] which, among much other information, explains why Galileo was a hacker but Aristotle wasn’t. A hack is a person lacking talent or ability, as in a “hack writer.” Hack as a verb is used in contexts such as “hack the media,” “hack your brain,” and “hack your reputation.” Recently, it has also come to mean either a kludge, or the opposite of a kludge, as in a clever or elegant solution to a difficult problem. A hack also means a simple but often inelegant solution or technique. The following tentative definitions are quoted from the jargon file ([jargon 04], edited by Eric S. Raymond): 1. A person who enjoys exploring the details of programmable systems and how to stretch their capabilities, as opposed to most users, who prefer to learn only the minimum necessary.
    [Show full text]
  • The Botnet Chronicles a Journey to Infamy
    The Botnet Chronicles A Journey to Infamy Trend Micro, Incorporated Rik Ferguson Senior Security Advisor A Trend Micro White Paper I November 2010 The Botnet Chronicles A Journey to Infamy CONTENTS A Prelude to Evolution ....................................................................................................................4 The Botnet Saga Begins .................................................................................................................5 The Birth of Organized Crime .........................................................................................................7 The Security War Rages On ........................................................................................................... 8 Lost in the White Noise................................................................................................................. 10 Where Do We Go from Here? .......................................................................................................... 11 References ...................................................................................................................................... 12 2 WHITE PAPER I THE BOTNET CHRONICLES: A JOURNEY TO INFAMY The Botnet Chronicles A Journey to Infamy The botnet time line below shows a rundown of the botnets discussed in this white paper. Clicking each botnet’s name in blue will bring you to the page where it is described in more detail. To go back to the time line below from each page, click the ~ at the end of the section. 3 WHITE
    [Show full text]
  • CONTENTS in THIS ISSUE Fighting Malware and Spam
    MARCH 2008 Fighting malware and spam CONTENTS IN THIS ISSUE 2 COMMENT EVASIVE ACTION Home (page) renovations Pandex has attracted very little attention from the media and generated little 3 NEWS discussion between malware Botherders herded researchers and among the 29A folds general populace. Chandra Prakash and Adam Thomas provide an overview of the Pandex operation and take an in-depth look at VIRUS PREVALENCE TABLE 3 the underlying code that has allowed this malware to evade detection for so long. 4 MALWARE ANALYSIS page 4 Pandex: the botnet that could PACKING A PUNCH In the fi nal part of the series on exepacker 9 FEATURE blacklisting, Robert Neumann takes a look at how all the processing and analysis techniques are put Exepacker blacklisting part 3 into practice in a real-life situation. page 9 15 CONFERENCE REPORT AVG TURNS 8 Black Hat DC and CCC 24C3 John Hawes gets his hands on a preview version of the latest offering from AVG. 18 PRODUCT REVIEW page 18 AVG Internet Security 8 22 END NOTES & NEWS This month: anti-spam news and events, and Ken Simpson considers the implications of rising spam volume despite increasing accuracy of content fi lters. ISSN 1749-7027 COMMENT ‘It is hoped that within all sizes of business. It is hoped that the comment facility will promote discussion among visitors and that the comment facility in some cases the more knowledgeable of VB’s readers will promote will be able to guide and assist those less well versed in discussion among the complexities of anti-malware technologies.
    [Show full text]
  • An Analysis of the Asprox Botnet
    An Analysis of the Asprox Botnet Ravishankar Borgaonkar Technical University of Berlin Email: [email protected] Abstract—The presence of large pools of compromised com- motives. Exploitable vulnerabilities may exist in the Internet puters, also known as botnets, or zombie armies, represents a infrastructure, in the clients and servers, in the people, and in very serious threat to Internet security. This paper describes the way money is controlled and transferred from the Internet the architecture of a contemporary advanced bot commonly known as Asprox. Asprox is a type of malware that combines into traditional cash. Many security firms and researchers are the two threat vectors of forming a botnet and of generating working on developing new methods to fight botnets and to SQL injection attacks. The main features of the Asprox botnet mitigate against threats from botnets [7], [8], [9]. are the use of centralized command and control structure, HTTP based communication, use of advanced double fast-flux service Unfortunately, there are still many questions that need to networks, use of SQL injection attacks for recruiting new bots be addressed to find effective ways of protecting against the and social engineering tricks to spread malware binaries. The threats from botnets. In order to fight against botnets in future, objective of this paper is to contribute to a deeper understanding of Asprox in particular and a better understanding of modern it is not enough to study the botnets of past. Botnets are botnet designs in general. This knowledge can be used to develop constantly evolving, and we need to understand the design more effective methods for detecting botnets, and stopping the and structure of the emerging advanced botnets.
    [Show full text]
  • Detecting Botnets Using File System Indicators
    Detecting botnets using file system indicators Master's thesis University of Twente Author: Committee members: Peter Wagenaar Prof. Dr. Pieter H. Hartel Dr. Damiano Bolzoni Frank Bernaards LLM (NHTCU) December 12, 2012 Abstract Botnets, large groups of networked zombie computers under centralised control, are recognised as one of the major threats on the internet. There is a lot of research towards ways of detecting botnets, in particular towards detecting Command and Control servers. Most of the research is focused on trying to detect the commands that these servers send to the bots over the network. For this research, we have looked at botnets from a botmaster's perspective. First, we characterise several botnet enhancing techniques using three aspects: resilience, stealth and churn. We see that these enhancements are usually employed in the network communications between the C&C and the bots. This leads us to our second contribution: we propose a new botnet detection method based on the way C&C's are present on the file system. We define a set of file system based indicators and use them to search for C&C's in images of hard disks. We investigate how the aspects resilience, stealth and churn apply to each of the indicators and discuss countermeasures botmasters could take to evade detection. We validate our method by applying it to a test dataset of 94 disk images, 16 of which contain C&C installations, and show that low false positive and false negative ratio's can be achieved. Approaching the botnet detection problem from this angle is novel, which provides a basis for further research.
    [Show full text]
  • The Ecology of Malware
    The Ecology of Malware Jedidiah R. Crandall Roya Ensafi Stephanie Forrest University of New Mexico University of New Mexico University of New Mexico Dept. of Computer Science Dept. of Computer Science Dept. of Computer Science Mail stop: MSC01 1130 Mail stop: MSC01 1130 Mail stop: MSC01 1130 1 University of New Mexico 1 University of New Mexico 1 University of New Mexico Albuquerque, NM 87131-0001 Albuquerque, NM 87131-0001 Albuquerque, NM 87131-0001 [email protected] [email protected] [email protected] Joshua Ladau Bilal Shebaro Santa Fe Institute University of New Mexico 1399 Hyde Park Road Dept. of Computer Science Santa Fe, New Mexico 87501 Mail stop: MSC01 1130 [email protected] 1 University of New Mexico Albuquerque, NM 87131-0001 [email protected] ABSTRACT General Terms The fight against malicious software (or malware, which includes Security everything from worms to viruses to botnets) is often viewed as an “arms race.” Conventional wisdom is that we must continu- Keywords ally “raise the bar” for the malware creators. However, the mul- titude of malware has itself evolved into a complex environment, malware ecology, malware analysis, worms, viruses, botnets and properties not unlike those of ecological systems have begun to emerge. This may include competition between malware, fa- 1. INTRODUCTION cilitation, parasitism, predation, and density-dependent population regulation. Ecological principles will likely be useful for under- Modern malware defense involves a variety of activities and is standing the effects of these ecological interactions, for example, quickly becoming unsustainable. So many malware samples are carrying capacity, species-time and species-area relationships, the collected from the wild each day that triaging is necessary to deter- unified neutral theory of biodiversity, and the theory of island bio- mine which samples warrant further analysis.
    [Show full text]
  • How the Pursuit of Truth Led Me to Selling Viagra®
    How The Pursuit of Truth Led Me To Selling Viagra® Vern Paxson EECS Department, University of California International Computer Science Institute Lawrence Berkeley National Laboratory Berkeley, California USA August 13, 2009 Outline: • This is a broad, retrospective talk about network security Data • Specifically, 2 decades’ worth of Internet measurement: What the data tells us about the lay of the land … what’s changed … and what in fact doesn’t change (“invariants”) • A personal (ivory tower research) view: From general network characterization ⇒ manual attacks ⇒ worms ⇒ bots ⇒ spam Why all this leads to selling Viagra First, some acknowledgments: • ICSI: Mark Allman, Christian Kreibich, Robin Sommer, Nicholas Weaver • LBL: Craig Leres, Jim Rothfuss, Dwayne Ramsey, Brian Tierney, et al • UC San Diego: Stefan Savage, Chris Kanich, Kirill Levchenko, Brandon Enright, Geoff Voelker Part I Pursuit of Truth + Phobia of Being Fooled = Thirst for Data Three Invariants: Growth, Explosive Onset, & Diversity • Sep 1988: I apply to grad school 56,000 Internet hosts (3.3 MB/day) • Sep 1990: I enroll in grad “special topics” course on networking & start measuring traffic at LBL 313,000 Internet hosts (9.5 MB/day) • Oct 21 1991: I join Prof. Ferrari’s Tenet group 617,000 Internet hosts (17.5 MB/day) • May 11, 1994: My paper Growth Trends in Wide Area TCP Connections accepted for publication ≈ 3,000,000 Internet hosts (130 MB/day) “Our data suggests a very recent explosion in commercial use of the Internet …” “… relatively new information-retrieval protocols such as Gopher and World-Wide Web exhibited explosive growth” Data courtesy of Rick Adams = 80% growth/year Data courtesy of Rick Adams & David C.
    [Show full text]
  • The Waledac Protocol: the How and Why
    The Waledac Protocol: The How and Why Greg Sinclair Chris Nunnery Brent ByungHoon Kang iDefense/ University of North University of North University of North Carolina at Charlotte Carolina at Charlotte Carolina at Charlotte [email protected] [email protected] [email protected] [email protected] Abstract Waledac communication allowing the defender community to better understand the Waledac command Peer to Peer (P2P) botnets are a growing structure. In addition to providing the necessary occurrence in the malware community. The Waledac information, we released a decryption tool on a private botnet represents a new, more challenging trend in the security list to aid defenders in their decryption tasks. P2P botnet evolution. The Waledac infrastructure has Antivirus vendors have used our research to understand evolved key aspects of the P2P architecture and the nature of Waledac [4]. devolved others. This evolution/devolution has resulted In this paper we will provide a concise description in a more formidable botnet. As a result, the Waledac of the Waledac architecture from a network topology botnet is harder to infiltrate and harder to enumerate. and communication perspective. The Waledac This paper explains the various aspects of the Waledac architecture refines the Storm botnet’s architecture to botnet infrastructures to give defenders a better simplify aspects of the topology in order to increase understanding of the botnet in order to protect efficiency and security. The revisions made by the themselves and others. Waledac infrastructure do contain vulnerabilities that defenders can exploit as we will explain. 1. Introduction The remaining sections of this paper explain how Waledac operates and defensive techniques which can Botnets continue to grow in their effectiveness and be deployed by defenders to protect their infrastructure robust architecture.
    [Show full text]
  • IBM X-Force Threat Insight Quarterly 2 X-Force Threat Insight Quarterly IBM Security Solutions
    IBM Security Solutions May 2011 IBM X-Force Threat Insight Quarterly 2 X-Force Threat Insight Quarterly IBM Security Solutions Contents About the report 2 About the Report The IBM X-Force® Threat Insight Quarterly is designed to highlight some of the most significant threats and challenges 3 Evolution: From Nuisance to Weapon facing security professionals today. This report is a product of IBM Managed Security Services and the IBM X-Force 8 Prolific and Impacting Issues of Q1 2011 research and development team. Each issue focuses on specific challenges and provides a recap of the most significant recent 16 References online threats. IBM Managed Security Services are designed to help an organization improve its information security, by outsourcing security operations or supplementing your existing security teams. The IBM protection on-demand platform helps deliver Managed Security Services and the expertise, knowledge and infrastructure an organization needs to secure its information assets from Internet attacks. The X-Force team provides the foundation for a preemptive approach to Internet security. The X-Force team is one of the best-known commercial security research groups in the world. This group of security experts researches and evaluates vulnerabilities and security issues, develops assessment and countermeasure technology for IBM security products, and educates the public about emerging Internet threats. We welcome your feedback. Questions or comments regarding the content of this report should be addressed to [email protected]. 3 X-Force Threat Insight Quarterly IBM Security Solutions Evolution: From Nuisance to Weapon One of the more notable examples here is Brain3, a boot sector infector which originated in Pakistan and released in 1986, was Creeper, Wabbit, Animal, Elk Cloner, Brain, Vienna, Lehigh, one of the first examples of malware that infected PC’s running Stoned, Jerusalem.
    [Show full text]
  • [Recognising Botnets in Organisations] Barry Weymes Number
    [Recognising Botnets in Organisations] Barry Weymes Number: 662 A thesis submitted to the faculty of Computer Science, Radboud University in partial fulfillment of the requirements for the degree of Master of Science Eric Verheul, Chair Erik Poll Sander Peters (Fox-IT) Department of Computer Science Radboud University August 2012 Copyright © 2012 Barry Weymes Number: 662 All Rights Reserved ABSTRACT [Recognising Botnets in Organisations] Barry WeymesNumber: 662 Department of Computer Science Master of Science Dealing with the raise in botnets is fast becoming one of the major problems in IT. Their adaptable and dangerous nature makes detecting them difficult, if not impossible. In this thesis, we present how botnets function, how they are utilised and most importantly, how to limit their impact. DNS Dynamic Reputations Systems, among others, are an innovative new way to deal with this threat. By indexing individual DNS requests and responses together we can provide a fuller picture of what computer systems on a network are doing and can easily provide information about botnets within the organisation. The expertise and knowledge presented here comes from the IT security firm Fox-IT in Delft, the Netherlands. The author works full time as a security analyst there, and this rich environment of information in the field of IT security provides a deep insight into the current botnet environment. Keywords: [Botnets, Organisations, DNS, Honeypot, IDS] ACKNOWLEDGMENTS • I would like to thank my parents, whom made my time in the Netherlands possible. They paid my tuition, and giving me the privilege to follow my ambition of getting a Masters degree. • My dear friend Dave, always gets a mention in my thesis for asking the questions other dont ask.
    [Show full text]
  • Cyber Risk – Common Threats Part 1 of 2
    Cyber Risk – Common Threats Part 1 of 2 Table of Contents Threats to Information Systems ..................................................................................................... 2 Malware .......................................................................................................................................... 4 Viruses ............................................................................................................................................. 5 Virus Examples ................................................................................................................................ 6 Worms ............................................................................................................................................. 8 Brief Virus and Worm History ......................................................................................................... 9 Downloaders ................................................................................................................................. 11 Attack Scripts ................................................................................................................................ 13 Botnet -1 ....................................................................................................................................... 15 Botnet -2 ....................................................................................................................................... 17 IRCBotnet Example ......................................................................................................................
    [Show full text]