Extending .NET Security to Unmanaged Code

International Journal of Information Security manuscript No. (will be inserted by the editor) Patrick Klinkoff · Engin Kirda · Christopher Kruegel · Giovanni Vigna Extending .NET Security to Unmanaged Code Abstract The number of applications that are down- implementation is the first solution to secure unmanaged loaded from the Internet and executed on-the-fly is in- code in .NET. creasing every day. Unfortunately, not all of these ap- · · plications are benign, and, often, users are unsuspecting Keywords .NET Security Unmanaged Code and unaware of the intentions of a program. To facilitate Sandboxing and secure this growing class of mobile code, Microsoft introduced the .NET framework, a new development and runtime environment where machine-independent byte- 1 Introduction code is executed by a virtual machine. An important feature of this framework is that it allows access to na- With the growth of the Internet, applications are increas- tive libraries to support legacy code or to directly invoke ingly downloaded from remote sources, such as Web sites, the Windows API. Such native code is called unmanaged and executed on-the-fly. Often, little or no knowledge ex- (as opposed to managed code). Unfortunately, the exe- ists about the author or her intentions. Therefore, users cution of unmanaged native code is not restricted by the are susceptible to executing potentially malicious pro- .NET security model, and, thus, could provide the at- grams on their computers. Malicious programs contain tacker with a mechanism to completely circumvent the code that executes in any unauthorized or undesirable framework’s security mechanisms if the user decides to way. grant execute permission to the .NET application. To secure users and increase the proliferation of mo- The approach described in this paper uses a sand- bile code, Microsoft recently introduced a new devel- boxing mechanism to prevent an attacker from executing opment and runtime framework called .NET [5]. This malicious, unmanaged code that is not permitted by the framework leverages the previous experiences gathered security policy. Our sandbox is implemented as two secu- with the Java virtual machine concepts and includes a rity layers, one on top of the Windows API and one in the fine-grained security model that allows one to control kernel. Also, managed and unmanaged parts of an appli- the level of access associated with software built upon cation are automatically separated and executed in two .NET. These applications are referred to as composed of different processes. This ensures that potentially unsafe managed code. The model significantly limits the dam- code can neither issue system calls not permitted by the age that can be caused by malicious code. To address .NET security policy nor tamper with the memory of the the important problem of backward compatibility and .NET runtime. Our proof-of-concept implementation is legacy code support, .NET also offers a mechanism to transparent to applications and secures unmanaged code tie in native libraries. These libraries, however, execute with a generally acceptable performance penalty. To the outside of the .NET security model, and therefore are best of our knowledge, the presented architecture and called unmanaged code. As a consequence, the usage of this feature in .NET applications may allow an attacker P. Klinkoff, E.Kirda, and C. Kruegel to completely circumvent the framework’s security mech- Secure Systems Lab / DSG-ASG anisms, leading to the unrestricted execution of arbitrary Technical University Vienna code. This security problem is important because the use E-mail: [pk,ek,chris]@seclab.tuwien.ac.at · of unmanaged code will probably be common in future G. Vigna Windows .NET applications. Millions of lines of legacy Department of Computer Science native Windows code exist that will need to be integrated University of California, Santa Barbara and supported over the next decade. Also, software en- E-mail: [email protected] gineering research [10] has shown that it is not realistic 2 Patrick Klinkoff et al. to expect existing applications to be entirely rewritten declared and used in the runtime. An important prop- from scratch in order to take advantage of the features erty of the .NET framework is that it is type-safe. Type of a new language. safety ensures that memory accesses are performed only This paper describes our approach to extend the cur- in well-defined ways, and no operation will be applied rent .NET security model to native (unmanaged) code to a variable of the wrong type. That is, any declared invoked from .NET. To this end, we use a sandboxing variable will always reference an object of either that mechanism that is based on the analysis of Windows API type or a subtype of that type. In particular, type safety and system call invocations to enforce the .NET security prevents a non-pointer from being dereferenced to access policy. Our approach ensures that all unmanaged code memory. Without type safety, a program could construct abides by the security permissions granted by the frame- an integer value that corresponds to a target address, and work. then use it as a pointer to reference an arbitrary location Our primary contributions are as follows: in memory. Type safety is typically achieved by a com- bination of static and dynamic techniques. During the – Extension of existing sandboxing methods to .NET just-in-time (JIT) compilation process, a verifier checks unmanaged code invocations. whether the code fulfills all necessary (but not sufficient) – Two-step authorization of system calls by placing the requirements for type safety. Those operations that can- security layer in the Windows API and the enforce- not statically be proven to be type-safe are augmented ment mechanisms in a loadable kernel driver. with dynamic checks. In addition to type safety, .NET – Separation of untrusted native library and trusted also provides memory safety, which ensures that a pro- managed code into two separate processes by way of gram cannot access memory outside of properly allocated .NET remoting. objects. Languages such as C are neither type-safe nor memory-safe. Thus, arbitrary memory access and type We implemented a proof-of-concept prototype imple- casts are possible, potentially leading to security vul- mentation of the architecture we propose and tested it nerabilities such as buffer overflows. Also, as arbitrary against two real-world, popular libraries: Sleepycat Soft- memory access is possible, programs can rewrite parts of ware’s Berkeley Database [25] and the OpenGL graph- their own code or jump to code constructed on-the-fly. ics library [22]. Our evaluation shows that our system is As a result, it is typically almost impossible to provide well-suited to operate in real-world environments. strong security guarantees about programs written in an The paper is structured as follows. The next section unsafe language. provides an overview of the .NET framework and its The runtime environment can enforce a variety of security-relevant components. Section 3 introduces the security restrictions on the execution of a program by design of our proposed system. Section 4 discusses the relying on type and memory safety. This makes it pos- evaluation of the security and performance of the sys- sible to run multiple .NET programs with different sets tem and shows that our approach is viable. Section 5 of permissions in the same process (on the same virtual presents related work. Finally, Section 6 outlines future machine). To specify security restrictions, the CLI de- work and concludes the paper. fines a security model that is denoted as Code Access Security (CAS) [9]. CAS uses evidence provided by the program and security policies configured on the machine to generate permission sets associated with the applica- 2 Overview of the .NET Framework tion. Security relevant operations (for example, file access) create corresponding permission objects, which are Microsoft’s .NET framework is an implementation of the tested with respect to the granted permission set. If the Common Language Infrastructure (CLI) [6], which is the permission is not found in the granted set, the action is open, public specification of a runtime environment and not permitted and a security exception is thrown. Oth- its executable code. The architecture of the .NET frame- erwise, the operation continues. Evidence is a term that work is depicted in Figure 1. Multiple programming lan- describes the information used by the runtime to deter- guages can be used with the .NET Framework (e.g., mine the identity (and hence, the trust level) associated VB.NET, C#, C++, etc.). The idea is that programs with a program. It exists in two forms. The first is in- in any of these languages are compiled into a Common herent evidence provided by the software. This includes Intermediate Language (CIL), which is hardware-neutral the assembly content, publisher certificate, or a strong byte-code. The bottom layer of the architecture consists 1 name . The second form is runtime-based evidence, such of the Common Language Runtime (CLR). The CLR is as the location or URL the assembly is loaded from. That a virtual machine that interprets and executes the .NET is, different permissions can be assigned to programs de- byte-code. The .NET class libraries (also written in CIL) pending on the author of the code or the location from are built on top of the CLR and provide functionality where the code is downloaded. such as networking, I/O, streams, or serialization. A part of the CLI specification describes the Com- 1 A strong name is a cryptographic signature authenticat- mon Type System (CTS), which defines how types are ing the publisher. Extending .NET Security to Unmanaged Code 3 Fig. 1 Architecture of the .NET Framework Programs that are written in the common interme- executed with the same restrictions as managed .NET diate language are referred to as managed code.Man- code.

Load more