Efficient Monitoring of Untrusted Kernel-Mode Execution Abhinav Srivastava and Jonathon Giffin School of Computer Science, Georgia Institute of Technology fabhinav,[email protected] Abstract system call monitors ineffective, full kernel malware in- stances invoke kernel functionality with function calls Recent malware instances execute completely in the from a driver rather than with system calls from a pro- kernel as drivers; they do not contain any user-level ma- cess. In order to observe the behavior of kernel malware, licious processes. This design evades the system call security software instead must monitor the function-call monitoring used by many software security solutions, in- interface between drivers and the core kernel. cluding malware analyzers and host-based intrusion de- Previous research efforts focused on the interaction tectors that track only user-level processes. To trace the of drivers with the kernel through control and non- behavior of kernel malware instances, we design and im- control data. SBCFI [31], Autoscopy [32], and Hook- plement a hypervisor-based system called Gateway that Safe [49] protected control data (function and code monitors kernel APIs invoked by drivers. Gateway cre- pointers) allocated on heap, while RAD [10] and Stack- ates a hardened, non-bypassable monitoring interface Ghost [15] protected control data (return addresses) by isolating drivers in an address space separate from present on the stack. Sentry [40], Gibraltar [3], and Se- the kernel. To overcome the performance degradation mantic Integrity [30] protected a kernel’s non-control introduced by switches between these separate address data [8] from malicious modifications. Though these spaces, our design rewrites binary kernel and driver systems monitor and prevent illegal use of kernel data, code at runtime and generates new code on demand to they do not track use of kernel code by malicious drivers, optimize the address space transition speed. Our exper- which is equally important. A malicious kernel module imental measurements show performance overheads of can create illegitimate kernel control flows by directly 10% or better, with many overheads less than 1%. Our invoking arbitrary kernel functionality via calls, jumps, security evaluation shows that Gateway is able to mon- or returns to arbitrary addresses inside the kernel code. itor all kernel APIs invoked by malicious drivers across To complement the extensive previous research on the its non-bypassable interface. protection of kernel data, we investigate the use of ker- nel code by malicious drivers. We address the invisibility of full kernel malware 1. Introduction with a new system that monitors all control-flow inter- actions of drivers with the core kernel. We design and implement a security monitor called Gateway that mon- Recently developed malware instances push entire itors all kernel APIs invoked by drivers, passes their in- functionality out of malicious user-space processes vocations asynchronously to an arbitrary policy enforce- down into kernel drivers or modules. For example, ment utility, and ensures complete mediation of direct srizbi consists of a single kernel-mode driver imple- accesses from drivers to kernel code. Note that Gate- menting a full spam client with an HTTP based com- way’s non-bypassable interface applies to code entry mand and control infrastructure [21,45], and the lvtes points, and attacks targeting kernel data require other keylogger contains only a kernel-mode driver that inter- techniques to monitor or detect. Gateway protects itself cepts users’ keystrokes [7]. These malware instances from kernel-mode malware by utilizing a higher privi- are called full-kernel malware [23]. Existing host-based lege software layer created by a hypervisor. Gateway’s IDSs, honeypots, and other security utilities often mon- monitoring provides the foundation on which security itor process’ behavior at the system-call interface to de- software can be built in the same manner that system call tect malware and software attacks. To render existing monitoring provided the foundation for behavior-based interface, our design does not compromise security but application-level security software. provides significant improvements to performance. Security monitors, including Gateway, require the Gateway monitors kernel APIs invoked by drivers on monitored interface to be non-circumventable. Com- both the slow and fast paths. On the slow path, API modity operating systems publish interfaces to be used invocations from drivers cause page faults that are inter- by drivers and loadable modules to request services cepted by Gateway inside the hypervisor and logged for from the kernel. Operationally, this interface design higher-level security tools. Since the guest system’s ex- is similar to the system call interface, which is used ecution does not reach the hypervisor on the fast path, by userspace applications to request services from the the code on our transition pages logs API information in core kernel. However, unlike the system-call interface protected memory inside the guest VM. Gateway then boundary, which is explicitly trusted (all inputs are ver- periodically reads the logged information from the hy- ified) and enforced by the hardware, there is no mecha- pervisor. nism to enforce the implicitly trusted boundary between To demonstrate the feasibility of our ideas, we de- the core kernel and its drivers. veloped a prototype security monitor. Gateway provides The lack of a memory barrier inside operating sys- protection to a fully virtualized Linux 2.6 kernel run- tems allows kernel-mode malware to directly invoke ar- ning inside a guest VM created by the Xen hypervi- bitrary kernel functionality by simply calling or jump- sor, which utilizes the virtualization extensions present ing to arbitrary kernel code addresses. We thus harden in recent x86 hardware [4]. Gateway isolates all com- the existing interface so that it becomes fully non- modity Linux drivers present in the guest VM, performs bypassable. Gateway creates distinct virtual memory re- on-demand dynamic binary rewriting and runtime code gions for commodity monolithic kernels and their dri- generation, and monitors kernel APIs invoked by drivers vers in the same way that kernels manage distinct re- as control flows spanning the memory barrier occur. We gions for higher-level application software. This de- tested Gateway’s monitoring ability with two full-kernel sign isolates drivers in a different memory region and malware instances: the lvtes keylogger and a syn- requires them to invoke kernel code through published thetic kernel-mode bot. Our results show that Gateway entry points. To keep driver code and core kernel code is able to monitor and log all APIs invoked by malicious isolated throughout the execution of the system, Gate- drivers. We empirically show the significant difference way prevents DMA-capable devices from corrupting the between the kernel APIs invoked by legitimate drivers kernel’s code via malicious DMA writes [5]. and malware, information useful to full-kernel malware The creation of the non-bypassable interface via ad- detection or analysis. We also verified Gateway’s abil- dress space separation diminishes overall system perfor- ity to enforce the non-bypassable interface by testing it mance. Control-flows between drivers and the core ker- with a synthetic malware instance that tries to execute nel result in page faults managed by our hypervisor-level kernel functionality not provided by the published inter- software. The transition from execution in a guest vir- face. Gateway’s overhead on various workloads varies tual machine (VM) into the hypervisor is a world switch between 0% and 10%, with many measurements below (slow path) with significant performance overhead. We 1%. address this challenge by reducing the number of world Gateway’s API monitoring facilitates creation of switches when crossing the barrier between the kernel other security tools that restrict the use of kernel code and the drivers. Our solution establishes fast address by drivers. For example: an intrusion detection sys- space switching (fast path) by using on-demand runtime tem (IDS) can be built that enforces a policy restricting code generation and kernel and driver code rewriting. drivers’ uses of the kernel’s raw memory manipulation During execution, Gateway creates transition pages— functions. In another example, kernel-level anomaly de- read-only code pages shared by both the kernel and tectors similar to system-call based detectors can be de- driver address spaces—that contain short instruction se- veloped to identify malicious drivers by differentiating quences that switch address spaces without invoking the the patterns of kernel functions invoked by legitimate hypervisor. We overwrite control-flow instructions at drivers from those of the malware. Ptrace [22] allows runtime in the kernel’s and drivers’ code pages to redi- user-mode security tools to monitor system calls invoked rect control-flows spanning the memory barrier into the by processes for such anomaly detection. Like ptrace, transition pages. As we only generate transition code Gateway itself does not attempt to distinguish between for control flows correctly spanning the non-bypassable benign and malicious API invocations, but it rather pro- vides the mechanism and information enabling higher- level security software (anomaly and misuse detectors) safely. They used segmentation and programming lan- to make such decisions. guage