Atc Proceedings

Behavior-based Spyware Detection Engin Kirda and Christopher Kruegel Secure Systems Lab Technical University Vienna {ek,chris}@seclab.tuwien.ac.at Greg Banks, Giovanni Vigna, and Richard A. Kemmerer Department of Computer Science University of California, Santa Barbara {nomed,vigna,kemm}@cs.ucsb.edu Abstract damage or to spread to other systems. Instead, spyware Spyware is rapidly becoming a major security is- programs monitor the behavior of users and steal private sue. Spyware programs are surreptitiously installed on a information, such as keystrokes and browsing patterns. user’s workstation to monitor his/her actions and gather This information is then sent back to the spyware dis- private information about a user’s behavior. Current anti- tributors and used as a basis for targeted advertisement spyware tools operate in a way similar to traditional anti- (e.g., pop-up ads) or marketing analysis. Spyware pro- virus tools, where signatures associated with known spy- grams can also “hijack” a user’s browser and direct the ware programs are checked against newly-installed ap- unsuspecting user to web sites of the spyware’s choos- plications. Unfortunately, these techniques are very easy ing. Finally, in addition to the violation of users’ privacy, to evade by using simple obfuscation transformations. spyware programs are also responsible for the degrada- This paper presents a novel technique for spyware de- tion of system performance because they are often poorly tection that is based on the characterization of spyware- coded. like behavior. The technique is tailored to a popular A number of anti-spyware products, whose goal is the class of spyware applications that use Internet Explorer’s identification and removal of unwanted spyware, have Browser Helper Object (BHO) and toolbar interfaces to been developed. These tools are mostly based on the monitor a user’s browsing behavior. Our technique uses a same technology used by anti-virus products. That is, composition of static and dynamic analysis to determine they identify known spyware instances by comparing whether the behavior of BHOs and toolbars in response the binary image of these programs with a number of to simulated browser events should be considered mali- uniquely-characterizing signatures. These signatures are cious. The evaluation of our technique on a representa- manually generated by analyzing existing samples of tive set of spyware samples shows that it is possible to spyware. As a consequence, these anti-spyware tools reliably identify malicious components using an abstract suffer from the same drawbacks as signature-based anti- behavioral characterization. virus tools, including the need for continuous updating of their signature set and their inability to deal with simple Keywords: spyware, malware detection, static analysis, obfuscation techniques [4]. dynamic analysis. This paper presents a novel spyware detection technique that overcomes some of the limitations of exist- 1 Introduction ing anti-spyware approaches. Our technique is based on an abstract characterization of the behavior of a popu- Spyware is rapidly becoming one of the major threats lar class of spyware programs that relies on Internet Ex- to the security of Internet users [20, 23]. A compre- plorer’s Browser Helper Object (BHO) and toolbar inter- hensive analysis performed by Webroot (an anti-spyware faces to monitor a user’s browsing behavior. More pre- software producer) and Earthlink (a major Internet Ser- cisely, our technique applies a composition of static and vice Provider) showed that a large portion of Internet- dynamic analysis to binary objects to determine if a com- connected computers is infected with spyware [1], and ponent monitors users actions and reports its findings to that, on average, each scanned host has 25 different spy- an external entity. This characterization is independent ware programs installed [7]. of the particular binary image and therefore can be used Different from other types of malware, such as viruses to identify previously unseen spyware programs, and, in and worms, the goal of spyware is generally not to cause addition, it is resilient to obfuscation. USENIX Association Security ’06: 15th USENIX Security Symposium 273 The main contributions of this paper are as follows: other malware-detection tools, such as anti-virus products and many network-based intrusion detection sys- • We introduce a novel characterization of the behav- tems [17, 19]. Recently, researchers have tried to over- ior of spyware components that are implemented as come these limitations by proposing behavior-based mal- Browser Helper Objects or toolbars. ware detection techniques. These techniques attempt to characterize a program’s behavior in a way that is inde- • We present novel static and dynamic analysis tech- pendent of its binary representation. By doing this, it is niques to reliably identify malicious behavior in possible to detect entire classes of malware and to be re- Browser Helper Objects and toolbar components. silient to obfuscation and polymorphism. • We present experimental results on a substantial For example, in [5] the authors characterize different body of spyware and benign samples that demon- variations of worms by identifying semantically equiv- strate the effectiveness of our approach. alent operations in the malware variants. Another approach is followed in [11], which characterizes the be- The remainder of this paper is structured as follows. havior of kernel-level rootkits. In this case, the authors In Section 2, we present related work in the field of use static analysis to determine if a loadable kernel mod- behavior-based malware detection in general and spy- ule is accessing kernel memory locations that are typi- ware detection in particular. Section 3 provides some cally used by rootkits (e.g., the system call table). background information on Browser Helper Objects and There are many advantages to representing malware toolbars. It also shows how they are exploited by spy- in an abstract way. For instance, by using behavioral ware programs to monitor user behavior and to hijack characterizations to detect malicious applications, one browser actions. In Section 4, we describe our abstract can obviate the need for a large data base of signatures characterization of spyware-like behavior. In Section 5, to identify each known piece of malware. Another im- we motivate the use of static and dynamic analysis for portant benefit is that the characterization is resilient to spyware detection, and we provide the details of our malware variants and allows for the detection of pre- technique in sections 6 and 7. Section 8 discusses possi- viously unseen malware instances. An example of us- ble limitations of our proposed system. In, Section 9 we ing behavior characterization is Microsoft’s Strider Gate- provide an experimental evaluation of the effectiveness keeper [24]. This tool monitors auto-start extensibility of our technique. Finally, Section 10 briefly concludes points (ASEPs) to determine if software that will be exe- and outlines future work. cuted automatically at startup is being surreptitiously installed on a system. 2 Related Work Our approach is similar to the one pursued by Strider Gatekeeper, because our goal is to model spyware-like Spyware is difficult to define. There are many types of behavior. Consequently, our behavioral classifications spyware that behave in different ways and perform ac- are not specifically tailored to a single spyware program, tions that represent different levels of “maliciousness.” but, instead, they are able to detect entire classes of spy- For example, “adware” programs that present targeted ware applications. However, our technique is more pow- advertisements to the user are considered less malicious erful than the one used by Strider Gatekeeper, because it than other forms of spyware, such as key-loggers, which identifies a more general behavior pattern, which is the record every single key pressed by the user. Regardless acquisition of private information and the leaking of this of the type of privacy violation performed, spyware is information outside the boundaries of the application. In generally undesirable code that the user wants to remove addition, our technique uses a combination of static and from his/her system. dynamic analysis, which allows for a very precise char- The increasing sensitivity of consumers to the spyware acterization of the behavior of an application in reaction problem prompted a number of anti-spyware commer- to browser events, thus, reducing the chance of false pos- cial products. For example, both AdAware [2] and Spy- itives. Bot [22] are popular tools that are able to remove a large The use of static analysis to analyze the behavior of number of spyware programs. Recently, Microsoft re- malware has been proposed previously in [5]. However, leased a beta version of an anti-spyware tool, aptly called the technique proposed by Christodorescu et al. is tai- Windows AntiSpyware [14]. lored to detect different variations of the same malware Current spyware detection tools use signatures to de- (e.g., different versions of the NetSky worm). Our tech- tect known spyware, and, therefore, they suffer from nique, instead, focuses on abstract, spyware-like behav- the drawback of not being able to detect previously un- ior and is not limited to detecting just one spyware pro- seen malware instances. This is a deficiency shared by gram and its variants. 274 Security ’06: 15th USENIX Security Symposium USENIX Association The technique presented in this paper, however, is not web-based interfaces, browser plug-ins are another pop- completely general. We focus explicitly on one type of ular mechanism to collect sensitive data and monitor user spyware, that is, malware that exploits the hooks pro- actions. Internet Explorer plug-ins, and in particular vided by Microsoft’s Internet Explorer to monitor the Browser Helper Objects (BHOs) and toolbars, as men- actions of a user. This is done by using the Browser tioned previously, are used in the majority of spyware Helper Object (BHO) interface or by acting as a browser programs as a mechanism to access information about a toolbar object.

Load more