Detecting Kernel-Level Rootkits Through Binary Analysis Abstract 1 Introduction Most intrusions and computer security incidents fol- low a common pattern where a remote user scans a Rootkits are tool sets used by intruders to modify the target system for vulnerable services, launches an at- perception that users have of a compromised system. tack to gain some type of access to the system, and, In particular, these tools are used by attackers to eventually, escalates her privileges. These privileges hide their actions from system administrators. Orig- are then used to create backdoors that will allow the inally, rootkits mainly included modified versions of attacker to return to the system at a later time. In system auditing programs (e.g., ps or netstat on a addition, actions are taken to hide the evidence that Unix system). However, for operating systems that the system has been compromised in order to prevent support loadable kernel modules (e.g., Linux and So- the system administrator from noticing the security laris), a new type of rootkit has recently emerged. breach and implementing counter measures (e.g., re- These rootkits are implemented as kernel modules, installing the system). and they do not require modification of user space binaries to conceal malicious activity. Instead, the The tools used by an attacker after gaining admin- rootkit operates within the kernel, modifying critical istrative privileges includes tools to hide the presence data structures such as the system call table or the of the attacker (e.g., log editors), utilities to gather in- list of currently-loaded kernel modules. formation about the system and its environment (e.g., network sniffers), tools to ensure that the attacker can regain access at a later time (e.g., backdoored This paper presents a technique that exploits bi- servers), and means of attacking other systems. Com- nary analysis to ascertain, at load time, if a mod- mon tools have been bundled by the hacker commu- ule’s behavior resembles the behavior of a rootkit. nity into “easy-to-use” kits, called rootkits [3]. Through this method, it is possible to provide addi- tional protection against this type of malicious mod- Even though the idea of a rootkit is to provide ification of the kernel. Our technique relies on an ab- all the tools that may be needed after a system has stract model of module behavior that is not affected been compromised, rootkits focus in particular on by small changes in the binary image of the module. backdoored programs and tools to hide the attacker Therefore, the technique is resistant to attempts to from the system administrator. Originally, rootkits conceal the malicious nature of a kernel module. mainly included modified versions of system auditing programs (e.g., ps or netstat for Unix systems) [9]. These modified programs (also called trojan horses) Keywords: Rootkits, Binary Analysis, Kernel Hard- do not return any information to the administrator ening. that involves specific files and processes used by the 1 intruder. Such tools, however, are easily detected us- The rest of the paper is structured as follows. Sec- ing file integrity checkers such as Tripwire [7]. tion 2 discusses related work on rootkits and rootkit detection. Section 3 presents our approach to the de- Recently, a new type of rootkit has emerged. These tection of kernel-level rootkits. Then, Section 4 pro- rootkits are implemented as loadable kernel modules vides an experimental evaluation of the effectiveness (LKMs). A loadable kernel module is an extension and efficiency of our technique. Finally, Section 5 dis- to the operating system (e.g., a device driver) that cusses possible limitations of the current prototype can be loaded into and unloaded from the kernel at while Section 6 briefly concludes. runtime. Solaris and Linux are two popular operat- ing systems that support this type of runtime kernel extension. By implementing a rootkit as a kernel module, it 2 Related Work is possible to modify critical kernel data structures (such as the system call table, the list of active pro- Kernel-level rootkits have been circulating in the un- cesses, or the list of kernel modules) or intercept re- derground hacker community for some time and in quests to the kernel regarding files and processes that different forms [6]. In general, there are different are created by an intruder [10, 14, 15]. Once the ker- means that can be used to modify kernel behavior. nel is infected, it is very hard to determine if a system The most common way of modifying the kernel is has been compromised without the help of hardware by inserting a loadable kernel module. The module extensions, such as the TCPA chip [12]. Therefore, it has access to the symbols exported by the kernel and is important that mechanisms are in place to detect can modify any data structure or function pointer kernel rootkits and prevent their insertion into the that is accessible. Typically, these kernel-level root- kernel. kits “hijack” entries in the system call table and pro- vide modified implementations of the corresponding In this paper, we present a technique for the de- system call functions [10, 14]. These modified system tection of kernel-level rootkits in the Linux operating calls often perform checks on the data passed back to system. The technique is based on static analysis a user process and can thus efficiently hide informa- of loadable kernel module binaries, in particular be- tion about files and processes. An interesting varia- havioral specifications and symbolic execution. The tion is implemented by the adore-ng rootkit [15, 16]. analysis allows the kernel to determine if the module In this case, the rootkit does not touch the system being loaded includes evidence of malicious intent. call table but hijacks the routines used by the Vir- The contribution of this approach is twofold. First, tual File System (VFS), and, therefore, it is able to by using static analysis, our technique is able to deter- intercept (and modify) calls that access files in both mine if a kernel module is malicious before the kernel the /proc file system and the root file system. module is actually loaded into the kernel and exe- A related technique injects malicious code directly cuted. This is a major advantage, because once the into existing kernel modules instead of providing a kernel image has been modified it may become in- complete rootkit module. While this solution is in feasible to perform dynamic analysis of the module’s principle similar to the insertion of a rootkit kernel actions in a reliable way. Second, the technique is module, it has the advantage that the modification applied to the binary image of a module and does will survive a kernel reboot procedure if the modified not require access to the module’s source code. Be- module is automatically loaded in the kernel standard cause of this, the technique is widely applicable and configuration. On the other hand, this technique re- it is possible to analyze the behavior of device drivers quires the modification of a binary that is stored on and other closed source kernel components that are the file system, and, therefore, it may be detected distributed in binary form only. using integrity checkers. 2 Another way to modify the behavior of the kernel 3 Rootkit Detection is to access kernel memory directly from user space through the /dev/kmem file. This technique (used, for The idea for our detection approach is based on the example, by SucKIT [13]) requires the identification observation that the runtime behavior of regular ker- of data structures that need to be modified within nel modules (e.g., device drivers) differs significantly the kernel image. However, this is not impossible; from the behavior of kernel-level rootkits. We note in particular, well-known data structures such as the that regular modules have different goals than root- system call table are relatively easy to locate. kits, and thus implement different functionality. The main contribution of this paper is that we show Kernel-level rootkits can be detected by utilizing that it is possible to distinguish between regular mod- a number of different techniques. The most basic ules and rootkits by statically analyzing kernel mod- include searching for modified kernel modules on disk, ule binaries. The analysis is performed in two steps. searching for known strings in existing binaries, or First, we have to specify undesirable behavior. Sec- by searching for configuration files associated with ond, each kernel module binary is statically analyzed specific rootkits. The problem is that when a system for the presence of instructions sequences that imple- has been compromised at the kernel level, there is no ment these specifications. guarantee that these tools will return reliable results. Currently, our specifications are given informally, This is also true for signature-based rootkit detection and the analysis step has to be adjusted appropriately tools such as chkrootkit [11] that rely on operating to deal with new specifications. Although it might be system services to scan a machine for indications of possible to introduce a formal mechanism to model known rootkits. behavioral specifications, it is not necessary for our To circumvent the problem of a possibly untrusted detection prototype. The reason is that a few general operating system, rootkit scanners such as kstat [4], specifications are sufficient to accurately capture the rkscan [2], or St. Michael [8] follow a different ap- malicious behavior of all LKM-based rootkits. Nev- proach. These tools are either implemented as ker- ertheless, the analysis technique is powerful enough nel modules with direct access to kernel memory, or that it can be easily extended. This may become nec- they analyze the contents of the kernel memory via essary when rootkit authors actively attempt to evade /dev/kmem.