<<

Venerable Variadic Vulnerabilities Vanquished Priyam Biswas, Purdue University; Alessandro Di Federico, Politecnico di Milano; Scott A. Carr, Purdue University; Prabhu Rajasekaran, Stijn Volckaert, Yeoul Na, and Michael Franz, University of California, Irvine; Mathias Payer, Purdue University https://www.usenix.org/conference/usenixsecurity17/technical-sessions/presentation/biswas

This paper is included in the Proceedings of the 26th USENIX Security Symposium August 16–18, 2017 • Vancouver, BC, Canada ISBN 978-1-931971-40-9

Open access to the Proceedings of the 26th USENIX Security Symposium is sponsored by USENIX Venerable Variadic Vulnerabilities Vanquished

Priyam Biswas1, Alessandro Di Federico2, Scott A. Carr1, Prabhu Rajasekaran3, Stijn Volckaert3, Yeoul Na3, Michael Franz3, and Mathias Payer1

1Department of Computer Science, Purdue University {biswas12, carr27}@purdue.edu, [email protected] 2Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano [email protected] 3Department of Computer Science, University of California, Irvine {rajasekp, stijnv, yeouln, franz}@uci.edu

Abstract stractions and high degree of control left to the devel- oper. However, these languages guarantee neither Programming languages such as and C++ support vari- nor memory safety, and bugs may lead to memory cor- adic functions, i.e., functions that accept a variable num- ruption. Memory corruption attacks allow adversaries to ber of arguments (e.g., ). While variadic func- take control of vulnerable applications or to extract sen- tions are flexible, they are inherently not type-safe. In sitive information. fact, the semantics and parameters of variadic functions Modern operating systems and compilers implement are defined implicitly by their implementation. It is left several defense mechanisms to combat memory corrup- to the programmer to ensure that the caller and callee fol- tion attacks. The most prominent defenses are Address low this implicit specification, without the help of a static Space Layout Randomization (ASLR) [47], stack ca- type checker. An adversary can take advantage of a mis- naries [13], and Data Execution Prevention (DEP) [48]. match between the argument types used by the caller of a While these defenses raise the bar against exploitation, and the types expected by the callee to sophisticated attacks are still feasible. In fact, even violate the language semantics and to tamper with mem- the combination of these defenses can be circumvented ory. Format string attacks are the most popular example through information leakage and code-reuse attacks. of such a mismatch. Indirect function calls can be exploited by an adver- Stronger defense mechanisms such as Control Flow sary to divert execution through illegal paths. CFI re- Integrity (CFI) [6], protect applications by restricting stricts call targets according to the function prototype their control flow to a predetermined control-flow graph which, for variadic functions, does not include all the ac- (CFG). While CFI allows the adversary to corrupt non- tual parameters. However, as shown by our case study, control data, it will terminate the process whenever the current CFI implementations are mainly limited to non- control-flow deviates from the predetermined CFG. The variadic functions and fail to address this potential attack strength of any CFI scheme hinges on its ability to stati- vector. Defending against such an attack requires a state- cally create a precise CFG for indirect control-flow edges ful dynamic check. (e.g., calls through function pointers in C or virtual calls ++ We present HexVASAN, a compiler based sanitizer to in C ). Due to ambiguity and imprecision in the analy- effectively type-check and thus prevent any attack via sis, CFI restricts adversaries to an over-approximation of variadic functions (when called directly or indirectly). the possible targets of individual indirect call sites. The key idea is to record metadata the call site and We present a new attack against widely deployed mit- verify parameters and their types at the callee whenever igations through a frequently used feature in C/C++ that they are used at runtime. Our evaluation shows that Hex- has so far been overlooked: variadic functions. Variadic VASAN is (i) practically deployable as the measured functions (such as printf) accept a varying number of overhead is negligible (0.45%) and (ii) effective as we arguments with varying argument types. To implement show in several case studies. variadic functions, the programmer implicitly encodes the argument list in the semantics of the function and has to sure the caller and callee adhere to this im- 1 Introduction plicit contract. In printf, the expected number of argu- ments and their types are encoded implicitly in the for- C and C++ are popular languages in systems program- mat string, the first argument to the function. Another ming. This is mainly due to their low overhead ab- frequently used scheme iterates through parameters until

USENIX Association 26th USENIX Security Symposium 183 a condition is reached (e.g., a parameter is NULL). List- tizer to detect type mismatches during program testing. ing 1 shows an example of a variadic function. If an ad- We have implemented HexVASAN on top of the versary can violate the implicit contract between caller LLVM compiler framework, instrumenting the compiled and callee, an attack may be possible. code to record the types of each argument of a variadic In the general case, it is impossible to enumerate the function at the call site and to check the types when- arguments of a variadic function through static analysis ever they are retrieved. Our prototype implementation techniques. In fact, their number and types are intrinsic is light-weight, resulting in negligible (0.45%) overhead in how the function is defined. This limitation enables (or for SPEC CPU2006. Our approach is general as we show facilitates) two attack vectors against variadic functions. by recompiling the FreeBSD base system and effective as First, attackers can hijack indirect calls and thereby call shown through several exploit case studies (e.g., a format variadic functions over control-flow edges that are never string vulnerability in sudo). taken during any legitimate execution of the program. We present the following contributions: Variadic functions that are called in this way may inter- • Design and implementation of a variadic function pret the variadic arguments differently than the function sanitizer on top of LLVM; for which these arguments were intended, and thus vio- late the implicit caller-callee contract. CFI countermea- • A case study on large programs to show the preva- sures specifically prevent illegal calls over indirect call lence of direct and indirect calls to variadic func- edges. However, even the most precise implementations tions; of CFI, which verify the type signature of the targets of indirect calls, are unable to fully stop illegal calls to vari- • Several exploit case studies and CFI bypasses using adic functions. variadic functions. A second attack vector involves overwriting a variadic function’s arguments directly. Such attacks do not - 2 Background olate the intended control flow of a program and thus bypass all of the widely deployed defense mechanisms. Variadic functions are used ubiquitously in C/C++ pro- Format string attacks are a prime example of such at- grams. Here we introduce details about their use and im- tacks. If an adversary can control the format string plementation on current systems, the attack surface they passed to, e.g., printf, she can control how all of the provide, and how adversaries can abuse them. following parameters are interpreted, and can potentially leak information from the stack, or read/ to arbitrary memory locations. #include The attack surface exposed by variadic functions is #include significant. We analyzed popular software packages, int add(int start, ...) { such as Firefox, Chromium, Apache, CPython, nginx, int next, total = start; OpenSSL, Wireshark, the SPEC CPU2006 benchmarks, va_list list; and the FreeBSD base system, and found that variadic va_start(list, start); functions are ubiquitous. We also found that many of do{ the variadic function calls in these packages are indirect. next = va_arg(list, int); We therefore conclude that both attack vectors are realis- total += next; tic threats. The underlying problem that enables attacks } while (next != 0); on variadic functions is the lack of type checking. Vari- va_end(list); adic functions generally do not (and cannot) verify that return total; the number and type of arguments they expect matches } the number and type of arguments passed by the caller. int main(int argc, const char *argv[]) { We present HexVASAN, a compiler-based, dynamic san- printf("%\n", add(5, 1, 2, 0)); itizer that tackles this problem by enforcing type checks return 0; for variadic functions at run-. Each argument that is } retrieved in a variadic function is type checked, enforc- ing a strict contract between caller and callee so that (i) a Listing 1: Example of a variadic function in C. The maximum of the passed arguments can be retrieved and function add takes a non-variadic argument start (ii) the type of the arguments used at the callee are com- (to initialize an accumulator variable) and a series patible with the types passed by the caller. Our mecha- of variadic int arguments that are added until the nism can be used in two operation modes: as a runtime terminator value 0 is met. The final value is returned. monitor to protect programs against attacks and as sani-

184 26th USENIX Security Symposium USENIX Association 2.1 Variadic functions are passed on the stack. If a variadic function accepts five mandatory arguments and a variable number of variadic Variadic functions (such as the printf function in the C arguments, then all but one of these variadic arguments standard library) are used in C to maximize the flexibil- will be passed on the stack. The variadic function itself ity in the interface of a function, allowing it to accept a moves the arguments into a va_list variable using the number of arguments unknown at compile-time. These va_start . The va_list type is defined as follows: functions accept a variable number of arguments, which do not necessarily have fixed types. An example of a typedef struct{ variadic function is shown in Listing 1. The function add unsigned int gp_offset; accepts one mandatory argument (start) and a varying unsigned int fp_offset; void *overflow_arg_area; number of additional arguments, which are marked by void *reg_save_area; ... the ellipsis ( ) in the function definition. } va_list[1]; The C standard defines several macros that portable programs may use to access variadic arguments [33]. va_start allocates on the stack a reg_save_area to stdarg.h, the header that declares these macros, defines store copies of all variadic arguments that were passed an opaque type, va_list, which stores all information re- in registers. va_start initializes the overflow_arg_area quired to retrieve and iterate through variadic arguments. field to point to the first variadic argument that was In our example, the variable list of type va_list is ini- passed on the stack. The gp_offset and fp_offset fields tialized using the va_start macro. The va_arg macro are the offsets into the reg_save_area. They represent retrieves the next variadic argument from the va_list, the first unused variadic argument that was passed in a updating va_list to point to the next argument as a side general purpose register or floating point register respec- effect. Note that, although the programmer must specify tively. the expected type of the variadic argument in the call to The va_arg macro retrieves the first unused vari- va_arg, the C standard does not require the compiler to adic argument from either the reg_save_area or verify that the retrieved variable is indeed of that type. the overflow_arg_area, and either it increases va_list variables must be released using a call to the the gp_offset/fp_offset field or moves the va_end macro so that all of the resources assigned to the overflow_arg_area pointer forward, to point to the list are deallocated. next variadic argument. printf is an example of a complex variadic Other architectures. Other architectures may imple- function which takes a format string as its first argument. ment variadic functions differently. On 32-bit x86, for This format string implicitly encodes information about example, all variadic arguments must be passed on the the number of arguments and their type. Implementa- stack (pushed right to left), following the cdecl calling printf tions of scan through this format string several convention used on GNU/Linux. The variadic function times to identify all format arguments and to recover itself retrieves the first unused variadic argument directly the necessary space in the output string for the specified from the stack. This simplifies the implementation of types and formats. Interestingly, arguments do not have the va_start, va_arg, and va_end macros, but it generally to be encoded sequentially but format allow out- makes it easier for adversaries to overwrite the variadic of-order access to arbitrary arguments. This flexibility is arguments. often abused in format string attacks to access arbitrary stack locations. 2.3 Variadic attack surface 2.2 Variadic functions ABI When calling a variadic function, the compiler statically type checks all non-variadic arguments but does not en- The C standard does not define the calling convention force any restriction on the type or number of variadic - for variadic functions, nor the exact representation of the guments. The programmer must follow the implicit con- va_list structure. This information is instead part of the tract between caller and callee that is only present in the ABI of the target platform. code but never enforced explicitly. Due to this high flex- x86-64 ABI. The AMD64 System V ABI [36], which ibility, the compiler cannot check arguments statically. is implemented by x86-64 GNU/Linux platforms, dic- This lack of safety can lead to bugs where an adver- tates that the caller of a variadic function must adhere to sary achieves control over the callee by modifying the the normal calling conventions when passing arguments. arguments, thereby influencing the interpretation of the Specifically, the first six non-floating point arguments passed variadic arguments. and the first eight floating point arguments are passed Modifying the argument or arguments that control the through CPU registers. The remaining arguments, if any, interpretation of variadic arguments allows an adversary

USENIX Association 26th USENIX Security Symposium 185 to change the behavior of the variadic function, causing string argument may point to a writable memory area the callee to access additional or fewer arguments than at compile time. An attacker may use memory corrup- specified and to change the interpretation of their types. tion to redirect the format string pointer to an attacker- An adversary can influence variadic functions in sev- controlled area and fall back to a regular format string eral ways. First, if the programmer forgot to validate the exploit. input, the adversary may directly control the arguments to the variadic function that controls the interpretation of arguments. Second, the adversary may use an arbitrary 3 Threat model memory corruption elsewhere in the program to influ- ence the argument of a variadic function. Programs frequently use variadic functions, either in the program itself or as part of a shared library (e.g., printf Variadic functions can be called statically or dynam- in the ). We assume that the program ically. Direct calls would, in theory, allow some static contains an arbitrary memory corruption, allowing the checking. Indirect calls (e.g., through a function pointer), adversary to modify the arguments to a variadic function where the target of the variadic function is not known, do and/or the target of an indirect function call, targeting a not allow any static checking. Therefore, variadic func- variadic function. tions can only be protected through some form of run- time checker that considers the constraints of the call site Our target system deploys existing defense mecha- and enforces them at the callee side. nisms like DEP, ASLR, and a strong implementation of CFI, protecting the program against code injection and control-flow hijacking. We assume that the adversary 2.4 Format string exploits cannot modify the metadata of our runtime monitor. Pro- tecting metadata is an orthogonal engineering problem Format string exploits are a perfect example of corrupted and can be solved through, e.g., masking (and-ing every variadic functions. An adversary that gains control over memory access), segmentation (for x86-32), protecting printf printf the format string used in can abuse the the memory region [9], or randomizing the location of function to leak arbitrary data on the stack or even re- sensitive data. Our threat model is a realistic scenario for to arbitrary memory corruption (if the pointer to the current attacks and defenses. target location is on the stack). For example, a format string vulnerability in the smbclient utility (CVE-2009- 1886) [40] allows an attacker to gain control over the 4 HexVASAN design Samba file system by treating a filename as format string. Also, in PHP 7.x before 7.0.1, an error handling function HexVASAN monitors calls to variadic functions and in zend execute API.c allows an attacker to execute arbi- checks for type violations. Since the semantics of how trary code by using format string specifiers as class name arguments should be interpreted by the function are in- (CVE-2015-8617) [1]. trinsic in the logic of the function itself, it is, in general, Information leaks are simple: an adversary changes impossible to determine the number and type of argu- the format string to print the desired information that re- ments a certain variadic function accepts. For this rea- sides somewhere higher up on the stack by employing the son, HexVASAN instruments the code generated by the desired format string specifiers. For arbitrary memory compiler so that a check is performed at runtime. This modification, an adversary must have the target address check ensures that the arguments consumed by the vari- encoded somewhere on the stack and then reference the adic function match those passed by the caller. target through the %n modifier, writing the number of al- The high level idea is the following: HexVASAN ready written bytes to that memory location. records metadata about the supplied argument types at The GNU C standard library (glibc) enforces some the call site and verifies that the extracted arguments protection against format string attacks by checking if match in the callee. The number of arguments and their a format string is in a writable memory area [29]. For types is always known at the call site and can be encoded format strings, the glibc printf implementation opens efficiently. In the callee this information can then be used /proc/self/maps and scans for the memory area of the to verify individual arguments when they are accessed. format string to verify correct permissions. Moreover, a To implement such a sanitizer, we must design a meta- check is performed to ensure that all arguments are con- data store, a pass that instruments call sites, a pass that sumed, so that no out-of-context stack slots can be used instruments callers, and a runtime library that manages in the format string exploit. These defenses stop some at- the metadata store and performs the run-time type verifi- tacks but do not mitigate the underlying problem that an cation. Our runtime library aborts the program whenever adversary can gain control over the format string. Note a mismatch is detected and generates detailed informa- that this heavyweight check is only used if the format tion about the call site and the mismatched arguments.

186 26th USENIX Security Symposium USENIX Association source1.c source2.cpp source3.c tures with the type information produced by our instru- mentation, and with a counter to track the index of the last argument that was read from the list. A second data C frontend C++ frontend C frontend structure, the variadic call stack (VCS), allows callers of IR IR IR variadic functions to store type information of variadic HexVASAN HexVASAN HexVASAN arguments until the callee initializes the va_list. instrumentation instrumentation instrumentation Each variadic call site is instrumented with a call to IR IR IR pre call, that prepares the information about the call Compile Compile Compile site (a variadic call site descriptor or VCSD), and a Object file call to post call, that cleans it up. For each vari- Object file Object file va start adic function, the calls are instrumented with list init, while va copy, whose purpose is to clone a va list, is instrumented through list copy. The two hexvasan.a output.elf run-time functions will allocate the necessary data struc- tures to validate individual arguments. Calls to va end Figure 1: Overview of the HexVASAN compilation are instrumented through list end to free up the corre- pipeline. The HexVASAN instrumentation runs right sponding data structures. after the C/C++ frontend, while its runtime library, Algorithm 1 summarizes the two phases of our anal- hexvasan.a, is merged into the final executable at link ysis and instrumentation pass. The first phase identifies time. all the calls to variadic functions (both direct and indi- rect). Note that identifying indirect calls to variadic func- tions is straight-forward in a compiler framework since, 4.1 Analysis and Instrumentation even if the target function is not statically known, its type We designed HexVASAN as a compiler pass to be run is. Then, all the parameters passed by that specific call in the compilation pipeline right after the C/C++ fron- tend. The instrumentation collects a set of statically input: a module m available information about the call sites, encodes it in /* Phase 1 */ the LLVM module, and injects calls to our runtime to foreach function f in module m do perform checks during program execution. foreach variadic call c with n arguments in f do Figure 1 provides an overview of the compilation vcsd.count ← n; pipeline when HexVASAN is enabled. Source files are foreach argument a of type t do ++ first parsed by the C/C frontend which generates the in- vcsd.args.push(t); termediate representation on which our instrumentation end runs. The normal compilation then proceeds, generating emit call to pre call(vcsd) before c; instrumented object files. These object files, along with emit call to post call() after c; the HexVASAN runtime library, are then passed to the end linker, which creates the instrumented program binary. end /* Phase 2 */ 4.2 Runtime support foreach function f in module m do foreach call c to va start(list) do The HexVASAN runtime augments every va_list in the emit call to list init(&list) after c; original program with the type information generated by end our instrumentation pass, and uses this type information foreach call c to va copy(dst,src) do to perform run-time type checking on any variadic argu- emit call to list copy(&dst,&src) after c; ment accessed through va_arg. By managing the type in- end formation in a metadata store, and by maintaining a map- foreach call c to va end(list) do ping between va_lists and their associated type infor- emit call to list free(&list) after c; mation, HexVASAN remains fully compatible with the end platform ABI. This design also supports interfacing be- foreach call c to va arg(list,type) do tween instrumented programs and non-instrumented li- emit call to check arg(&list,type) before c; braries. end The HexVASAN runtime manages the type informa- tion in two data structures. The core data structure, called end Algorithm 1: The instrumentation process. the variadic list map (VLM), associates va_list struc-

USENIX Association 26th USENIX Security Symposium 187 site are inspected and recorded, along with their type in input : a type t and an initial hash value h a dedicated VCSD which is stored in read-only global output: the final hash value h data. At this point, a call to pre call is injected before h = hash(h, typeID(t)); the variadic function call (with the newly created VCSD switch typeID(t) do as a parameter) and, symmetrically, a call to post call case AggregateType is inserted after the call site. /* union, struct and pointer */ The second phase identifies all calls to va start and foreach c in componentTypes(t) do va copy, and consequently, the va list variables in the h = hashType(c, h); program. Uses of each va list variable are inspected in end an architecture-specific way. Once all uses are identified, case FunctionType we inject a call to check arg before dereferencing the h = hashType(returnType(t), h); argument (which always resides in memory). foreach a in argTypes(t) do h = hashType(a, h); 4.3 Challenges and Discussion end end When designing a variadic function call sanitizer, several endsw issues have to be considered. We highlight details about h = hash(h, typeID(t)); the key challenges we encountered. return h Multiple va lists. Functions are allowed to create Algorithm 2: Algorithm describing the type hashing multiple va_lists to access the same variadic arguments, function hashType. typeID returns an unique identifier either through va_start or va_copy operations. Hex- for each basic type (e.g., 32-bit integer, double), type VASAN handles this by storing a VLM entry for each of aggregate type (e.g., struct, union...) and functions. individual va_list. hash is a simple hashing function combining two inte- Passing va_lists as function arguments. While un- gers. componentTypes returns the components of an ag- common, variadic functions are allowed to pass the gregate type, returnType the return type of a function va_lists they create as arguments to non-variadic func- prototype and argTypes the type of its arguments. tions. This allows non-variadic functions to access vari- adic arguments of functions higher in the call stack. Our design takes this into account by maintaining a list map keeping a global map of type-unique id pairs. Due to (VLM) and by instrumenting all va_arg operations, re- the information loss during the translation from C/C++ gardless of whether or not they are in a variadic function. to LLVM IR, our type system does not distinguish be- tween signed and unsigned types. The required meta- Multi-threading support. Multiple threads are sup- data is static and immutable and we mark it as read-only, ported by storing our per-thread runtime state in a thread- protecting it from modification. However, the VCS still local variable as supported on major operating systems. needs to be protected through other mechanisms. Metadata format. We use a constant data structure per Handling floating point arguments. In x86-64 ABI, variadic call site, the VCSD, to hold the number of ar- floating point and non-floating point arguments are han- guments and a pointer to an array of integers identifying dled differently. In case of floating point arguments, their type. The check arg function therefore only per- the first eight arguments are passed in the floating point forms two memory accesses, the first to load the number registers whereas in case of non-floating point the first of arguments and the second for the type of the argument six are passed in general-purpose registers. HexVASAN currently being checked. handles both argument types. To uniquely identify the data types with an integer, we decided to build a hashing function (described in Algo- Support for aggregate data types. According to rithm 2) using a set of fixed identifiers for primitive data AMD64 System V ABI, the caller unpacks the fields of types and hashing them in different ways depending on the aggregate data types (structs and unions) if the argu- how they are aggregated (pointers, union, or struct). ments fit into registers. This makes it hard to distinguish The last hash acts as a terminator marker for aggre- between composite types and regular types – if unpacked gate types, which allows us to, e.g., distinguish between they are indistinguishable on the callee side from argu- {struct{ int }, int} and {struct {struct{ int, ments of these types. HexVASAN supports aggregate int }}}. Note that an (unlikely) hash collision only re- data types even if the caller unpacks them. sults in two different types being accepted as equal. Such Attacks preserving number and type of arguments. a hashing mechanism has the advantage of being deter- Our mechanism prevents attacks that change the num- ministic across compilation units, removing the need for ber of arguments or the types of individual arguments.

188 26th USENIX Security Symposium USENIX Association Format string attacks that only change one modifier can the destructor, leveraging the exception handling mech- therefore be detected through the type mismatch even if anism to make HexVASAN exception-safe. Functions the total number of arguments remains unchanged. like longjmp can be instrumented to purge the portions Non-variadic calls to variadic functions. Consider the of HexVASAN’s data structures that correspond to the following code snippet: discarded stack area. We did not observe any such calls in practice and leave the implementation of handling - typedef void (*non_variadic)(int, int); ceptions and longjump across variadic functions as future engineering work. void variadic(int, ...) {/* ...*/} int main() { non_variadic function_ptr = variadic; 5 Implementation function_ptr(1, 2); } We implemented HexVASAN as a sanitizer for the LLVM compiler framework [31], version 3.9.1. We In this case, the function call in main to function_ptr have chosen LLVM for its robust features on analyzing appears to the compiler as a non-variadic function call, and transforming arbitrary programs as well as extract- since the type of the function pointer is not variadic. ing reliable type information. The sanitizer can be en- Therefore, our pass will not instrument the call site, lead- abled from the C/C++ frontend () by providing the ing to potential errors. -fsanitize=vasan parameter at compile-time. No an- To handle such (rare) situations appropriately, we notations or other source code changes are required for would have to instrument all non-variadic call sites too, HexVASAN. Our sanitizer does not require visibility of leading to an unjustified overhead. Moreover, the code whole source code (see Section 4.3), but works on indi- above represents undefined behavior in C [27, 6.3.2.3p8] vidual compilation units. Therefore link-time optimiza- and C++ [26, 5.2.10p6], and might not work on certain ar- tion (LTO) is not required and thus fits readily into exist- chitectures where the calling convention for variadic and ing build systems. In addition, HexVASAN also supports non-variadic function calls are not compatible. The GNU handlers. C compiler emits a warning when a function pointer is HexVASAN consists of two components: a static in- cast to a different type, therefore we require the devel- strumentation pass and a runtime library. The static in- oper to correct the code before applying HexVASAN. strumentation pass works on LLVM IR, adding the nec- Central management of the global state. To allow the essary instrumentation code to all variadic functions and HexVASAN runtime to be linked into the base system li- their callees. The support library is statically linked to braries, such as the C standard library, we made it a static the program and, at run-time, checks the number and library. Turning the runtime into a shared library is pos- type of variadic arguments as they are used by the pro- sible, but would prohibit its use during the early process gram. In the following we describe the two components initialization – until the dynamic linker has processed all in detail. of the necessary relocations. Our runtime therefore ei- Static instrumentation. The implementation of the ther needs to be added solely to the C standard library static instrumentation pass follows the description in (so that it is initialized early in the startup process) or Section 4. We first iterate through all functions, looking the runtime library must carefully use weak symbols to for CallInst instructions targeting a variadic function ensure that each symbol is only defined once if multiple (either directly or indirectly), then we inspect them and libraries are compiled with our countermeasure. create for each one of them a read-only GlobalVariable C++ exceptions and longjmp. If an exception is raised of type vcsd t. As shown in Listing 2, vcsd t is com- while executing a variadic function (or one of its callees), posed by an unsigned integer representing the number the variadic function may not get a chance to clean up the of arguments of the considered call site and a pointer to metadata for any va_lists it has initialized, nor may the an array (another GlobalVariable) with an integer el- caller of this variadic function get the chance to clean up ement for each argument of type t. type t is an inte- the type information it has pushed onto the VCS. Other ger uniquely identifying a data type obtained using the functions manipulating the thread’s stack directly, such hashType function presented in Algorithm 2. At this as longjmp, present similar issues. point a call to pre call is injected before the call site, C++ exceptions can be handled by modifying the with the newly create VCSD as a parameter, and a call to LLVM C++ frontend (i.e., clang) to inject an object post call is injected after the call site. with a lifetime spanning from immediately before a vari- During the second phase, we first identify all va_start, adic function call to immediately after. Such an object va_copy, and va_end operations in the program. In the IR would call pre_call in its constructor and post_call in code, these operations appear as calls to the LLVM in-

USENIX Association 26th USENIX Security Symposium 189 struct vcsd_t { trinsics llvm.va_start, llvm.va_copy, and va_end. We unsigned count; instrument the operations with calls to our runtime’s type_t *args; list_init, list_copy, and list_free functions respec- }; tively. We then proceed to identify va_arg operations. Although the LLVM IR has a dedicated va_arg instruc- thread_local stack vcs; tion, it is not used on any of the platforms we tested. thread_local map> vlm; cation of va_arg is therefore platform-specific. On x86- 64, our primary target, we identify va_arg by recogniz- void pre_call(vcsd_t *arguments) { ing accesses to the gp_offset and fp_offset fields in the vcs.push_back(arguments); va_list } x86-64 version of the structure (see Section 2.2). void post_call() { The fp_offset field is accessed whenever the program vcs.pop_back(); attempts to retrieve a floating point argument from the } list. The gp_offset field is accessed to retrieve any other void list_init(va_list *list_ptr) { types of variadic arguments. We insert a call to our run- vlm[list_ptr] = { vcs.top(), 0 }; time’s check_arg function before the instruction that ac- } cesses this field. Listing 2 shows (in simplified C) how the code in List- void list_free(va_list *list_ptr) { ing 1 would be instrumented by our sanitizer. vlm.erase(list_ptr); } Dynamic variadic type checking. The entire runtime is implemented in plain C code, as this allows it to be void check_arg(va_list *list_ptr, type_t type) { linked into the standard C library without introducing pair &args = vlm[list_ptr]; a dependency to the standard C++ library. The VCS is unsigned index = args.second++; implemented as a thread-local stack, and the VLM as assert(index < args.first->count); a thread-local hash map. The pre_call and post_call assert(args.first->args[index] == type); } functions push and pop type information onto and from the VCS. The list_init function inserts a new entry int add(int start, ...) { into the VLM, using the top element on the stack as the /* ...*/ entry’s type information and initializing the counter for va_start(list, start); consumed arguments to 0. list_init(&list); check arg looks up the type information for the do{ va_list being accessed in the VLM and checks if the check_arg(&list, typeid(int)); requested argument exists (based on the counter of con- total += va_arg(list, int); sumed arguments), and if its type matches the one pro- } while (next != 0); vided by the caller. If either of these checks fails, exe- va_end(list); list_free(&list); cution is aborted, and the runtime will generate an error /* ...*/ message such as the one shown in Listing 3. As a con- } sequence, the pointer to the argument is never read or written, since the pointer to it is never dereferenced. const vcsd_t main_add_vcsd = { .count = 3, Error: Type Mismatch .args = {typeid(int), typeid(int), typeid(int)} Index is 1 }; Callee Type : 43 (32-bit Integer) Caller Type : 15 (Pointer) int main(int argc, const char *argv[]) { Backtrace: /* ...*/ [0] 0x4019ff <__vasan_backtrace+0x1f> at pre_call(&main_add_vcsd); [1] 0x401837 <__vasan_check_arg+0x187> at test int result = add(5, 1, 2, 0); [2] 0x8011b3afa <__vfprintf+0x20fa> at libc.so.7 post_call(); [3] 0x8011b1816 at libc.so.7 printf("%d\n", result); [4] 0x801200e50 at libc.so.7 /* ...*/ [5] 0x4024ae at test } [6] 0x4012ff <_start+0x17f> at test

Listing 2: Simplified C++ representation of the Listing 3: Error message reported by HexVASAN instrumented code for Listing 1.

190 26th USENIX Security Symposium USENIX Association 6 Evaluation of the variadic arguments. HexVASAN detects both of these attacks when the callee attempts to retrieve the In this section we present a case study on variadic func- variadic arguments using the va_arg macro described tion based attacks against state-of-the-art CFI implemen- in Section 2.1. Checking and enforcing the correct tations. Next, we evaluate the effectiveness of Hex- types for variadic functions is only possible at runtime VASAN as an exploit mitigation technique. Then, we and any sanitizer must resort to run-time checks to do evaluate the overhead introduced by our HexVASAN so. CFI mechanisms must therefore be extended with prototype implementation on the SPEC CPU2006 in- a HexVASAN-like mechanism to detect violations. To teger (CINT2006) benchmarks, on Firefox using stan- show that our tool can complement CFI, we create test dard JavaScript benchmarks, and on micro-benchmarks. programs containing several variadic functions and one We also evaluate how widespread the usage of variadic non-variadic function. The definitions of these functions functions is in SPEC CPU2006 and in Firefox 51.0.1, are shown below. Chromium 58.0.3007.0, Apache 2.4.23, CPython 3.7.0, nginx 1.11.5, OpenSSL 1.1.1, Wireshark 2.2.1, and the int sum_ints(int n, ...); int avg_longs(int n, ...); FreeBSD 11.0 base system. int avg_doubles(int n, ...); Note that, along with testing the aforementioned soft- void print_longs(int n, ...); ware, we also developed an internal set of regression void print_doubles(int n, ...); tests. Our regression tests allow us to verify that our int square(int n); sanitizer correctly catches problematic variadic function calls, and does not raise false alarms for benign calls. This program contains one indirect call site from The test suite explores corner cases, including trying to which only the sum_ints function can be called legally, access arguments that have not been passed and trying to and one indirect call site from which only the square access them using a type different from the one used at function can be legally called. We also introduce a mem- the call site. ory corruption vulnerability which allows us to override the target of both indirect calls. We constructed the program such that sum_ints, 6.1 Case study: CFI effectiveness avg_longs, print_longs, and square are all address-taken One of the attack scenarios we envision is that an at- functions. The avg_doubles and print_doubles functions tacker controls the target of an indirect call site. If the are not address-taken. intended target of the call site was a variadic function, Functions avg_longs, avg_doubles, print_longs, and the attacker could illegally call a different variadic func- print_doubles all expect different variadic argument tion that expects different variadic arguments than the in- types than function sum_ints. Functions sum_ints, tended target (yet shares the types for all non-variadic avg_longs, avg_doubles, and square do, however, all arguments). If the intended target of the call site was a have the same non-variadic prototype (int (*)(int)). non-variadic function, the attacker could call a variadic We compiled six versions of the test program, function that interprets some of the intended target’s ar- instrumenting them with, respectively, HexVASAN, guments as variadic arguments. LLVM 3.9 Forward-Edge CFI [59], Per-Input CFI [44], All existing CFI mechanisms allow such attacks to CCFI [35], GCC 6.2’s VTV [59] and Visual C++ Control some extent. The most precise CFI mechanisms, which Flow Guard [37]. In each version, we first built an attack rely on function prototypes to classify target sets (e.g., involving a variadic function, by overriding the indirect LLVM-CFI, piCFI, or VTV) will allow all targets with call sites with a call to each of the variadic functions de- the same prototype, possibly restricting to the subset scribed above. We then also tested overwriting the argu- of functions whose addresses are taken in the program. ments of the sum_ints function, without overwriting the This is problematic for variadic functions, as only non- indirect call target. Table 1 shows the detection results. variadic types are known statically. For example, if LLVM Forward-Edge CFI allows calls to avg_longs a function of type int (*)(int, ...) is expected to and avg_doubles from the sum_ints indirect call site be- be called from an indirect call site, then precise CFI cause these functions have the same static type signa- schemes allow calls to all other variadic functions of that ture as the intended call target. This implementation of type, even if those other functions expect different types CFI does not allow calls to variadic functions from non- for the variadic arguments. variadic call sites, however. A second way to attack variadic functions is to over- CCFI only detects calls to print_doubles, a function write their arguments directly. This happens, for ex- that is not address-taken and has a different non-variadic ample, in format string attacks, where an attacker can prototype than square, from the square call site. It allows overwrite the format string to cause misinterpretation all of the other illegal calls.

USENIX Association 26th USENIX Security Symposium 191 Actual target Intended target Prototype A.T.? LLVM-CFI pi-CFI CCFI VTV CFG HexVASAN      Same X Variadic No  X    X Yes    Different XX X No XX    X Yes    Same XX X Non-variadic No XX    X Yes    Different XX X No XXX   X Original Overwritten Arguments      X

Table 1: Detection coverage for several types of illegal calls to variadic functions. X indicates detection,  indicates non-detection. “A.T.” stands for address taken.

GCC VTV, and Visual C++ CFG allow all of the ille- Although it is sudo itself that calls the format string gal calls, even if the non-variadic type signature does not function (fprintf), HexVASAN can only detect the vio- match that of the intended call target. lation on the callee side. We therefore had to build hard- pi-CFI allows calls to the avg_longs function from the ened versions of not just the sudo binary itself, but also sum_ints indirect call site. avg_longs is address-taken the C library. We chose to do this on the FreeBSD plat- and it has the same static type signature as the intended form, as its standard C library can be easily built using call target. pi-CFI does not allow illegal calls to non- LLVM, and HexVASAN therefore readily fits into the address-taken functions or functions with different static FreeBSD build process. As expected, HexVASAN does type signatures. pi-CFI also does not allow calls to vari- detect any exploit that triggers the vulnerability, produc- adic functions from non-variadic call sites. ing the error message shown in Listing 4. All implementations of CFI allow direct overwrites of variadic arguments, as long as the original control flow $ -s /usr/bin/sudo %x%x%x%x $ ./%x%x%x%x -D9 -A of the program is not violated. ------Error: Index greater than Argument Count 6.2 Exploit Detection Index is 1 Backtrace: To evaluate the effectiveness of our tool as a real-world [0] 0x4053bf <__vasan_backtrace+0x1f> at sudo exploit detector, we built a HexVASAN-hardened ver- [1] 0x405094 <__vasan_check_index+0xf4> at sudo sion of sudo 1.8.3. sudo allows authorized users to ex- [2] 0x8015dce24 <__vfprintf+0x2174> at libc.so ecute shell commands as another user, often one with [3] 0x8015dac52 at libc.so a high privilege level on the system. If compromised, [4] 0x8015daab3 at libc.so sudo can escalate the privileges of non-authorized users, [5] 0x40bdaf at sudo [6] 0x40ada3 at sudo making it a popular target for attackers. Versions 1.8.0 [7] 0x40494f <_start+0x17f> at sudo through 1.8.3p1 of sudo contained a format string vul- nerability (CVE-2012-0809) that allowed exactly such a Listing 4: Exploit detection in sudo. compromise. This vulnerability could be exploited by passing a format string as the first argument (argv[0]) of the sudo program. One such exploit was shown to by- pass ASLR, DEP, and glibc’s FORTIFY SOURCE pro- 6.3 Prevalence of variadic functions tection [20]. In addition, we were able to verify that GCC 5.4.0 and clang 3.8.0 fail to catch this exploit, even when To collect variadic function usage in real software, annotating the vulnerable function with the format func- we extended our instrumentation mechanism to collect tion attribute [5] and setting the compiler’s format string statistics about variadic functions and their calls. As checking (-Wformat) to the highest level. shown in Table 2, for each program, we collect:

192 26th USENIX Security Symposium USENIX Association Call sites Func. Ratio prevent functions with their address taken from being Program Tot. Ind. % Tot. A.T. Proto Tot. A.T. reachable from indirect call sites. Firefox 30225 1664 5.5 421 18 241 1.75 0.07 Variadic prototypes. The number of distinct variadic Chromium 83792 1728 2.1 794 44 396 2.01 0.11 function prototypes in the program. FreeBSD 189908 7508 3.9 1368 197 367 3.73 0.53 Apache 7121 0 0 94 29 41 2.29 0.71 Functions-per-prototype. The average number of vari- CPython 4183 0 0 382 0 38 10.05 0.00 adic functions sharing the same prototype. This mea- nginx 1085 0 0 26 0 14 1.86 0.00 sures how many targets are available, on average, for each indirect call sites targeting a specific prototype. OpenSSL 4072 1 0.02 23 0 15 1.53 0.00 In practice, this the average number of permitted des- Wireshark 37717 0 0 469 1 110 4.26 0.01 tinations for an indirect call site in the case of a perfect perlbench 1460 1 0.07 60 2 18 3.33 0.11 CFI implementation. We report this value both consid- bzip2 85 0 0 3 0 3 1.00 0.00 ering all the variadic functions and only those whose address is taken. gcc 3615 55 1.5 125 0 31 4.03 0.00

mcf 29 0 0 3 0 3 1.00 0.00 Interestingly, each benchmark we analyzed contains milc 424 0 0 21 0 8 2.63 0.00 calls to variadic functions and several programs (Fire- namd 485 0 0 24 2 8 3.00 0.25 fox, OpenSSL, perlbench, gcc, povray, and hmmer) even gobmk 2911 0 0 35 0 8 4.38 0.00 contain indirect calls to variadic functions. In addition to calling variadic functions, each benchmark also defines soplex 6 0 0 2 1 2 1.00 0.50 numerous variadic functions (421 for Firefox, 794 for povray 1042 40 3.8 45 10 16 2.81 0.63 Chromium, 1368 for FreeBSD, 469 for Wireshark, and hmmer 671 7 1 9 1 5 1.80 0.20 382 for CPython). Variadic functions are therefore preva- sjeng 253 0 0 4 0 3 1.33 0.00 lent and used ubiquitously in software. Adversaries have libquantum 74 0 0 91 0 7 13.00 0.00 plenty of opportunities to modify these calls and to at- tack the implicit contract between caller and callee. The h264ref 432 0 0 85 5 13 6.54 0.38 compiler is unable to enforce any static safety guaran- lbm 11 0 0 3 0 2 1.50 0.00 tees when calling these functions, either for the number omnetpp 340 0 0 48 23 19 2.53 1.21 of arguments, nor their types. In addition, many of the astar 42 0 0 4 1 4 1.00 0.25 benchmarks have variadic functions that are called indi- rectly, often with their address being taken. Looking at sphinx3 731 0 0 20 0 5 4.00 0.00 Firefox, a large piece of software, the numbers are even xalancbmk 19 0 0 4 2 4 1.00 0.50 more staggering with several thousand indirect call sites that target variadic functions and 241 different variadic prototypes. Table 2: Statistics of Variadic Functions for Different Benchmarks. The second and third columns are vari- The prevalence of variadic functions leaves both a adic call sites broken into “Tot.” (total) and “Ind.” (indi- large attack surface for attackers to either redirect vari- rect); % shows the percentage of variadic call sites. The adic calls to alternate locations (even if defense mecha- fifth and sixth columns are for variadic functions. “A.T.” nisms like CFI are present) or to modify the arguments so stands for address taken. “Proto.” is the number of dis- that callees misinterpret the supplied arguments (similar tinct variadic function prototypes. “Ratio” indicates the to extended format string attacks). function-per-prototypes ratio for variadic functions. In addition, the compiler has no insight into these functions and cannot statically check if the programmer supplied the correct parameters. Our sanitizer identi- Call sites. The number of function calls targeting vari- fied three interesting cases in omnetpp, one of the SPEC adic functions. We report the total number and how CPU2006 benchmarks that implements a discrete event many of them are indirect, since they are of particular simulator. The benchmark calls a variadic functions with interest for an attack scenario where the adversary can a mismatched type, where it expects a char * but re- override a function pointer. ceives a NULL, which has type void *. Listing 5 shows the offending code. Variadic functions. The number of variadic functions. We also identified a bug in SPEC CPU2006’s We report their total number and how many of them perlbench. This benchmark passes the result of a sub- have their address taken, since CFI mechanism cannot traction of two character pointers as an argument to a

USENIX Association 26th USENIX Security Symposium 193 1.1 static sEnumBuilder _EtherMessageKind(

"EtherMessageKind", 1.05 JAM_SIGNAL,"JAM_SIGNAL",

ETH_FRAME,"ETH_FRAME", 1 ETH_PAUSE,"ETH_PAUSE", ETHCTRL_DATA,"ETHCTRL_DATA", 0.95

ETHCTRL_REGISTER_DSAP, Native HexVASAN "ETHCTRL_REGISTER_DSAP", 0.9 ETHCTRL_DEREGISTER_DSAP, "ETHCTRL_DEREGISTER_DSAP", ETHCTRL_SENDPAUSE,"ETHCTRL_SENDPAUSE", 0, NULL Figure 2: Run-time overhead of HexVASAN in the ); SPECint CPU2006 benchmarks, compared to baseline Listing 5: Variadic violation in omnetpp. LLVM 3.9.1 performance.

the speed of test cases gathered from different real-world variadic function. At the call site, this argument is a ma- applications and libraries. chine word-sized integer (i.e., 64-bits integer on our test platform). The callee truncates this argument to a 32- bit integer by calling va arg(list, int). HexVASAN 6.5 SPEC CPU2006 reports this (likely unintended) truncation as a violation. We measured HexVASAN’s run-time overhead by run- 6.4 Firefox ning the SPEC CPU2006 integer (CINT2006) bench- marks on an Ubuntu 14.04.5 LTS machine with an Intel We evaluate the performance of HexVASAN by in- Xeon E5-2660 CPU and 64 GiB of RAM. We ran each strumenting Firefox (51.0.1) and using three differ- benchmark program on its reference inputs and measured ent browser benchmark suites: Octane, JetStream, and the average run-time over three runs. Figure 2 shows the Kraken. Table 3 shows the comparison between the Hex- results of these tests. We compiled each benchmark with VASAN instrumented Firefox and native Firefox. To re- a vanilla clang/LLVM 3.9.1 compiler and optimization duce variance between individual runs, we averaged fif- level -O3 to establish a baseline. We then compiled the teen runs for each benchmark (after one warmup run). benchmarks with our modified clang/LLVM 3.9.1 com- For each run we started Firefox, ran the benchmark, and piler to generate the HexVASAN results. closed the browser. HexVASAN incurs only 1.08% and The geometric mean overhead in these benchmarks 1.01% overhead for Octane and JetStream respectively was just 0.45%, indistinguishable from measurement and speeds up around 0.01% for Kraken. These num- noise. The only individual benchmark result that stands bers are indistinguishable from measurement noise. Oc- out is that of libquantum. This benchmark program per- tane [4] and JetStream measure the time a test takes to formed 880M variadic function calls in a run of just 433 complete and then assign a score that is inversely pro- seconds. portional to the runtime, whereas Kraken [3] measures

Benchmark Native HexVASAN 6.6 Micro-benchmarks AVERAGE 31241.80 30907.73 Besides evaluating large benchmarks, we have also Octane STDDEV 2449.82 2442.82 OVERHEAD -1.08% measured HexVASAN’s runtime overhead on a set of AVERAGE 200.76 198.75 micro-benchmarks. We have written test cases for vari- JetStream STDDEV 0.66 1.68 adic functions with different number of arguments, in OVERHEAD -1.01% which we repeatedly invoke the variadic functions. Ta- AVERAGE [ms] 832.48 832.41 ble 4 shows the comparison between the native and Kraken STDDEV [ms] 7.41 12.71 HexVASAN-instrumented micro-benchmarks. Overall, OVERHEAD 0.01% HexVASAN incurs runtime overheads of 4-6x for vari- adic function calls due to the additional security checks. Table 3: Performance overhead on Firefox benchmarks. In real-world programs, however, variadic functions are For Octane and JetStream higher is better, while for invoked rarely, so HexVASAN has little impact on the Kraken lower is better. overall runtime performance.

194 26th USENIX Security Symposium USENIX Association # calls Native [µs] HexVASAN [µs] imate executions of the program. Among these miti- Variadic function gations, we find Code Pointer Integrity (CPI) [30], a argument count: 3 1 0 0 mitigation that prevents attackers from overwriting code 100 2 12 pointers in the program, and various implementations of 1000 20 125 Control-Flow Integrity (CFI), a technique that does not Variadic function prevent code pointer overwrites, but rather verifies the in- argument count: 12 1 0 0 100 6 22 tegrity of control-flow transfers in the program [6, 7, 11, 1000 55 198 14–16,21,22,28,35,37,38,41–44,46,49–51,59,61–66].

Table 4: Performance overhead in micro-benchmarks. Control-flow hijacking mitigations cannot prevent at- tackers from overwriting variadic arguments directly. At best, they can prevent variadic functions from be- 7 Related work ing called through control-flow edges that do not ap- pear in legitimate executions of the program. We there- HexVASAN can either be used as an always-on runtime fore argue that HexVASAN and these mitigations are monitor to mitigate exploits or as a sanitizer to detect orthogonal. Moreover, prior research has shown that bugs, sharing similarities with the sanitizers that exist many of the aforementioned implementations fail to fully primarily in the LLVM compiler. Similar to HexVASAN, prevent control-flow hijacking as they are too impre- these sanitizers embed run-time checks into a program cise [8, 17, 19, 23], too limited in scope [53, 57], vulner- by instrumenting potentially dangerous program instruc- able to information leakage attacks [18], or vulnerable tions. to spraying attacks [24, 45]. We further showed in Sec- AddressSanitizer [54] (ASan), instruments memory tion 6.1 that variadic functions exacerbate CFI’s impre- accesses and allocation sites to detect spatial memory cision problems, allowing additional leeway for adver- errors, such as out-of-bounds accesses, as well as tem- saries to attack variadic functions. poral memory errors, such as use-after-free bugs. Unde- fined Behavior Sanitizer [52] (UBSan) instruments vari- Defenses that protect against direct overwrites or mis- ous types of instructions to detect operations whose se- use of variadic arguments have thus far only focused on mantics are not strictly defined by the C and C++ stan- format string attacks, which are a subset of the possible dards, e.g., increments that cause signed integers to over- attacks on variadic functions. LibSafe detects potentially flow, or null-pointer dereferences. Thread Sanitizer [55] dangerous calls to known format string functions such (TSAN) instruments memory accesses and atomic opera- as printf and sprintf [60]. A call is considered dan- tions to detect data races, deadlocks, and various misuses gerous if a %n specifier is used to overwrite the frame of synchronization primitives. Memory Sanitizer [58] pointer or return address, or if the argument list for the (MSAN) detects uses of uninitialized memory. printf function is not contained within a single stack CaVer [32] is a sanitizer targeted at verifying correct- frame. FormatGuard [12] instruments calls to printf ness of downcasts in C++. Downcasting converts a base and checks if the number of arguments passed to printf class pointer to a derived class pointer. This operation matches the number of format specifiers used in the for- may be unsafe as it cannot be statically determined, in mat string. general, if the pointed-to object is of the derived class type. TypeSan [25] is a refinement of CaVer that reduces Shankar et al. proposed to use static taint analysis to overhead and improves the sanitizer coverage. detect calls to format string functions where the format UniSan [34] sanitizes information leaks from the ker- string originates from an untrustworthy source [56]. This nel. It ensures that data is initialized before leaving the approach was later refined by Chen and Wagner [10] and kernel, preventing reads of uninitialized memory. used to analyze thousands of packages in the Debian 3.1 All of these sanitizers are highly effective at finding Linux distribution. TaintCheck [39] also detects untrust- specific types of bugs, but, unlike HexVASAN, they do worthy format strings, but relies on dynamic taint analy- not address misuses of variadic functions. The aforemen- sis to do so. tioned sanitizers also differ from HexVASAN in that they typically incur significant run-time and memory over- FORTIFY SOURCE of glibc provides some lightweight . checks to ensure all the arguments are consumed. How- Different control-flow hijacking mitigations offer par- ever, it can be bypassed [2] and does not check for type- tial protection against variadic function attacks by mismatch. Hence, none of these aforementioned solu- preventing adversaries from calling variadic functions tions provide comprehensive protection against variadic through control-flow edges that do not appear in legit- argument overwrites or misuse.

USENIX Association 26th USENIX Security Symposium 195 8 Conclusions [7]B OUNOV,D.,KICI,., AND LERNER, S. Protecting C++ dy- namic dispatch through vtable interleaving. In Symposium on Variadic functions introduce an implicitly defined con- Network and Distributed System Security (NDSS) (2016). tract between the caller and callee. When the program- [8]C ARLINI,N.,BARRESI,A.,PAYER,M.,WAGNER,D., AND mer fails to enforce this contract correctly, the violation GROSS, T. R. Control-flow bending: On the effectiveness of control-flow integrity. In USENIX Security Symposium (2015). leads to runtime crashes or opens up a vulnerability to [9]C ASTRO,M.,COSTA,M.,MARTIN, J.-P., PEINADO,M., an attacker. Current tools, including static type check- AKRITIDIS, P., DONNELLY,A.,BARHAM, P., AND BLACK,R. ers and CFI implementations, do not find variadic func- Fast byte-granularity software fault isolation. In ACM Symposium tion type errors or prevent attackers from exploiting calls on Operating Systems Principles (SOSP) (2009). to variadic functions. Unfortunately, variadic functions [10]C HEN,K., AND WAGNER, D. Large-scale analysis of format are prevalent. Programs such as SPEC CPU2006, Fire- string vulnerabilities in debian linux. In Proceedings of the 2007 workshop on Programming languages and analysis for security fox, Apache, CPython, nginx, wireshark and libraries (2007). frequently leverage variadic functions to offer flexibility [11]C HENG, Y., ZHOU,Z.,MIAO, Y., DING,X., AND DENG, and abundantly call these functions. R. H. ROPecker: A generic and practical approach for defending We have designed a sanitizer, HexVASAN, that ad- against ROP attacks. In Symposium on Network and Distributed dresses this attack vector. HexVASAN is a light weight System Security (NDSS) (2014). runtime monitor that detects bugs in variadic functions [12]C OWAN,C.,BARRINGER,M.,BEATTIE,S.,KROAH- and prevents the bugs from being exploited. It imposes HARTMAN,G.,FRANTZEN,M., AND LOKIER, J. Formatguard: Automatic protection from vulnerabilities. In negligible overhead (0.45%) on the SPEC CPU2006 USENIX Security Symposium (2001). benchmarks and is effective at detecting type viola- [13]C OWAN,C.,PU,C.,MAIER,D.,WALPOLE,J.,BAKKE, P., tions when calling variadic arguments. Download Hex- BEATTIE,S.,GRIER,A.,WAGLE, P., ZHANG,Q., AND HIN- VASAN at https://github.com/HexHive/HexVASAN. TON, H. Stackguard: Automatic adaptive detection and preven- tion of buffer-overflow attacks. In USENIX Security Symposium (1998). 9 Acknowledgments [14]C RISWELL,J.,DAUTENHAHN,N., AND ADVE, V. KCoFI: Complete control-flow integrity for commodity operating system We thank the anonymous reviewers for their insightful kernels. In IEEE Symposium on Security and Privacy (S&P) comments. We also thank our shepherd Adam Doupe´ (2014). for his informative feedback. This material is based [15]D AVI,L.,DMITRIENKO,A.,EGELE,M.,FISCHER, T., HOLZ, T., HUND,R.,NURNBERGER¨ ,S., AND SADEGHI,A.-R. in part upon work supported by the National Science MoCFI: A framework to mitigate control-flow attacks on smart- Foundation under awards CNS-1513783, CNS-1657711, phones. In Symposium on Network and Distributed System Secu- and CNS-1619211, by the Defense Advanced Research rity (NDSS) (2012). Projects Agency (DARPA) under contracts FA8750-15- [16]D AVI,L.,KOEBERL, P., AND SADEGHI, A.-R. Hardware- C-0124 and FA8750-15-C-0085, and by Intel Corpora- assisted fine-grained control-flow integrity: Towards efficient protection of embedded systems against software exploitation. In tion. We also gratefully acknowledge a gift from Oracle Annual Design Automation Conference (DAC) (2014). Corporation. Any opinions, findings, and conclusions or [17]D AVI,L.,SADEGHI,A.-R.,LEHMANN,D., AND MONROSE, recommendations expressed in this material are those of F. Stitching the gadgets: On the ineffectiveness of coarse-grained the authors and do not necessarily reflect the views of control-flow integrity protection. In USENIX Security Symposium the National Science Foundation, the Defense Advanced (2014). Research Projects Agency (DARPA) and its Contracting [18]E VANS,I.,FINGERET,S.,GONZALEZ´ ,J.,OTGONBAATAR,U., Agents, or any other agency of the U.S. Government. TANG, T., SHROBE,H.,SIDIROGLOU-DOUSKOS,S.,RINARD, M., AND OKHRAVI, H. Missing the point (er): On the effective- ness of code pointer integrity. In IEEE Symposium on Security References and Privacy (S&P) (2015). [19]E VANS,I.,LONG, F., OTGONBAATAR,U.,SHROBE,H.,RI- [1] http://www.cve.mitre.org/cgi-bin/cvename.cgi?name= NARD,M.,OKHRAVI,H., AND SIDIROGLOU-DOUSKOS,S. CVE-2015-8617. Control jujutsu: On the weaknesses of fine-grained control flow [2] A eulogy for format strings. http://phrack.org/issues/67/ integrity. In ACM Conference on Computer and Communications 9.html. Security (CCS) (2015). [3] Kraken benchmark. https://wiki.mozilla.org/Kraken. [20]E XPLOIT DATABASE. sudo debug privilege escalation. https: [4] Octane benchmark. https://developers.google.com/ //www.exploit-db.com/exploits/25134/, 2013. octane/faq. [21]G AWLIK,R., AND HOLZ, T. Towards Automated Integrity [5] Using the (gcc) - function at- Protection of C++ Virtual Function Tables in Binary Programs. tributes. https://gcc.gnu.org/onlinedocs/gcc-3.2/gcc/ In Annual Computer Security Applications Conference (ACSAC) Function-Attributes.html. (2014). [6]A BADI,M.,BUDIU,M.,ERLINGSSON,U., AND LIGATTI,J. [22]G E,X.,TALELE,N.,PAYER,M., AND JAEGER, T. Fine- Control-flow integrity. In ACM Conference on Computer and Grained Control-Flow Integrity for Kernel Software. In IEEE Communications Security (CCS) (2005). European Symp. on Security and Privacy (2016).

196 26th USENIX Security Symposium USENIX Association [23]G OKTAS¨ ,E.,ATHANASOPOULOS,E.,BOS,H., AND POR- [41]N IU,B., AND TAN, G. Monitor integrity protection with space TOKALIDIS, G. Out of control: Overcoming control-flow in- efficiency and separate compilation. In ACM Conference on Com- tegrity. In IEEE Symposium on Security and Privacy (S&P) puter and Communications Security (CCS) (2013). (2014). [42]N IU,B., AND TAN, G. Modular control-flow integrity. In ACM [24]G OKTAS¨ ,E.,GAWLIK,R.,KOLLENDA,B.,ATHANASOPOU- SIGPLAN Conference on Design and LOS,E.,PORTOKALIDIS,G.,GIUFFRIDA,C., AND BOS,H. Implementation (PLDI) (2014). Undermining information hiding (and what to do about it). In [43]N IU,B., AND TAN, G. RockJIT: Securing just-in-time compila- USENIX Security Symposium (2016). tion using modular control-flow integrity. In ACM Conference on [25]H ALLER,I.,JEON, Y., PENG,H.,PAYER,M.,GIUFFRIDA, Computer and Communications Security (CCS) (2014). C.,BOS,H., AND VAN DER KOUWE, E. Typesan: Practical [44]N IU,B., AND TAN, G. Per-input control-flow integrity. In ACM type confusion detection. In ACM Conference on Computer and Conference on Computer and Communications Security (CCS) Communications Security (CCS) (2016). (2015). [26] Information technology – Programming languages – C++. Stan- [45]O IKONOMOPOULOS,A.,ATHANASOPOULOS,E.,BOS,H., dard, International Organization for Standardization, Geneva, AND GIUFFRIDA, C. Poking holes in information hiding. In CH, Dec. 2014. USENIX Security Symposium (2016). [27] Information technology – Programming languages – C. Standard, [46]P APPAS, V., POLYCHRONAKIS,M., AND KEROMYTIS,A.D. International Organization for Standardization, Geneva, CH, Dec. Transparent ROP exploit mitigation using indirect branch tracing. 2011. In USENIX Security Symposium (2013).

[28]J ANG,D.,TATLOCK,Z., AND LERNER, S. SAFEDISPATCH: [47]P AXTEAM. address space layout randomization (aslr). Securing C++ virtual calls from memory corruption attacks. In Symposium on Network and Distributed System Security (NDSS) [48]P AXTEAM. PaX non-executable pages design & implementa- (2014). tion. http://pax.grsecurity.net/docs/noexec.txt, 2004.

[29]J ELINEK, J. FORTIFY SOURCE. https://gcc.gnu.org/ml/ [49]P AYER,M.,BARRESI,A., AND GROSS, T. R. Fine-grained gcc-patches/2004-09/msg02055.html, 2004. control-flow integrity through binary hardening. In Conference on Detection of Intrusions and Malware & Vulnerability Assess- [30]K UZNETSOV, V., SZEKERES,L.,PAYER,M.,CANDEA,G., ment (DIMVA) (2015). SEKAR,R., AND SONG, D. Code-pointer integrity. In USENIX Symposium on Operating Systems Design and Implementation [50]P EWNY,J., AND HOLZ, T. Control-flow restrictor: Compiler- (OSDI) (2014). based CFI for iOS. In Annual Computer Security Applications Conference (ACSAC) (2013). [31]L ATTNER,C., AND ADVE, V. Llvm: A compilation framework for lifelong program analysis & transformation. In IEEE/ACM [51]P RAKASH,A.,HU,X., AND YIN, H. vfGuard: Strict Protection International Symposium on Code Generation and Optimization for Virtual Function Calls in COTS C++ Binaries. In Symposium (CGO) (2004). on Network and Distributed System Security (NDSS) (2015).

[32]L EE,B.,SONG,C.,KIM, T., AND LEE, W. Type casting verifi- [52]P ROJECT, G. C. Undefined behavior sanitizer. cation: Stopping an emerging attack vector. In USENIX Security https://www.chromium.org/developers/testing/ Symposium (2015). undefinedbehaviorsanitizer. CHUSTER ENDYCK IEBCHEN AVI [33]L INUX PROGRAMMER’S MANUAL. va start (3) - Linux Manual [53]S , F., T , T., L ,C.,D ,L., Page. SADEGHI,A.-R., AND HOLZ, T. Counterfeit object-oriented programming: On the difficulty of preventing code reuse attacks [34]L U,K.,SONG,C.,KIM, T., AND LEE, W. Unisan: Proactive in c++ applications. In IEEE Symposium on Security and Privacy kernel memory initialization to eliminate data leakages. In ACM (S&P) (2015). Conference on Computer and Communications Security (CCS) (2016). [54]S EREBRYANY,K.,BRUENING,D.,POTAPENKO,A., AND VYUKOV, D. Addresssanitizer: a fast address sanity checker. [35]M ASHTIZADEH,A.J.,BITTAU,A.,BONEH,D., AND In USENIX Annual Technical Conference (2012). MAZIERES` , D. Ccfi: cryptographically enforced control flow integrity. In ACM Conference on Computer and Communications [55]S EREBRYANY,K., AND ISKHODZHANOV, T. Threadsanitizer: Security (CCS) (2015). Data race detection in practice. In Workshop on Binary Instru- mentation and Applications (2009). [36]M ATZ,M.,HUBICKA,J.,JAEGER,A., AND MITCHELL,M. [56]S HANKAR,U.,TALWAR,K.,FOSTER,J.S., AND WAGNER, System v application binary interface. AMD64 Architecture Pro- D. Detecting format string vulnerabilities with type qualifiers. In cessor Supplement, Draft v0.99 (2013). USENIX Security Symposium (2001). [37]M ICROSOFT CORPORATION. Control Flow Guard (Windows). https://msdn.microsoft.com/en-us/library/windows/ [57]S NOW,K.Z.,MONROSE, F., DAVI,L.,DMITRIENKO,A., LIEBCHEN,C., AND SADEGHI, A. Just-in-time code reuse: On desktop/mt637065(v=vs.85).aspx, 2016. the effectiveness of fine-grained address space layout randomiza- [38]M OHAN, V., LARSEN, P., BRUNTHALER,S.,HAMLEN,K., tion. In IEEE Symposium on Security and Privacy (S&P) (2013). AND FRANZ, M. Opaque control-flow integrity. In Symposium [58]S TEPANOV,E., AND SEREBRYANY, K. Memorysanitizer: Fast on Network and Distributed System Security (NDSS) (2015). detector of uninitialized memory use in c++. In IEEE/ACM In- [39]N EWSOME,J., AND SONG, D. Dynamic taint analysis for auto- ternational Symposium on Code Generation and Optimization matic detection, analysis, and signature generation of exploits on (CGO) (2015). commodity software. In Symposium on Network and Distributed [59]T ICE,C.,ROEDER, T., COLLINGBOURNE, P., CHECKOWAY, System Security (NDSS) (2005). S.,ERLINGSSON, U.,L´ OZANO,L., AND PIKE, G. Enforcing [40]NISSIL,R. http://cve.mitre.org/cgi-bin/cvename.cgi? forward-edge control-flow integrity in gcc & llvm. In USENIX name=CVE-2009-1886. Security Symposium (2014).

USENIX Association 26th USENIX Security Symposium 197 [60]T SAI, T., AND SINGH, N. Libsafe 2.0: Detection of format string vulnerability exploits. white paper, Avaya Labs (2001).

[61] VAN DER VEEN, V., ANDRIESSE,D.,GOKTAS¨ ¸, E., GRAS,B., SAMBUC,L.,SLOWINSKA,A.,BOS,H., AND GIUFFRIDA, C. PathArmor: Practical ROP protection using context-sensitive CFI. In ACM Conference on Computer and Communications Se- curity (CCS) (2015).

[62]W ANG,Z., AND JIANG, X. Hypersafe: A lightweight approach to provide lifetime hypervisor control-flow integrity. In IEEE Symposium on Security and Privacy (S&P) (2010).

[63]Y UAN, P., ZENG,Q., AND DING, X. Hardware-assisted fine- grained code-reuse attack detection. In International Symposium on Research in Attacks, Intrusions and Defenses (RAID) (2015).

[64]Z HANG,C.,SONG,C.,CHEN,K.Z.,CHEN,Z., AND SONG, D. VTint: Defending virtual function tables’ integrity. In Sympo- sium on Network and Distributed System Security (NDSS) (2015).

[65]Z HANG,C.,WEI, T., CHEN,Z.,DUAN,L.,SZEKERES,L., MCCAMANT,S.,SONG,D., AND ZOU, W. Practical control flow integrity and randomization for binary executables. In IEEE Symposium on Security and Privacy (S&P) (2013).

[66]Z HANG,M., AND SEKAR, R. Control flow integrity for cots binaries. In USENIX Security Symposium (2013).

198 26th USENIX Security Symposium USENIX Association