focussoftware development tools Automating Software Testing Using Program Analysis

Patrice Godefroid, Peli de Halleux, Aditya V. Nori, Sriram K. Rajamani, Wolfram Schulte, and Nikolai Tillmann, Microsoft Research

Michael Y. Levin, Microsoft Center for Software Excellence

uring the last decade, code inspection for standard programming errors has Three new tools largely been automated with static code analysis. Commercial static program- combine techniques analysis tools are now routinely used in many software development organiza- from static program tions.1 These tools are popular because they find many software bugs, thanks to analysis, dynamic D three main ingredients: they’re automatic, they’re scalable, and they check many properties. analysis, model Intuitively, any tool that can automatically check millions of lines of code against hundreds checking, and of coding rules is bound to find on average, say, one bug every thousand lines of code. automated constraint Our long-term goal is to automate, as much as say, even half the code of a million-line C program solving to automate possible, an even more expensive part of the soft- would have tremendous value. test generation in ware development process, namely software testing. Such a tool doesn’t exist today, but in this arti- varied application Testing usually accounts for about half the R&D cle, we report on some recent significant progress budget of software development organizations. In toward that goal. Although automating test genera- domains. particular, we want to automate test generation by tion using program analysis is an old idea,2 practi- leveraging recent advances in program analysis, au- cal tools have only started to emerge during the last tomated constraint solving, and modern comput- few years. This recent progress was partly enabled ers’ increasing computational power. To replicate by advances in dynamic test generation,3 which gen­ the success of static program analysis, we need the eralizes and is more powerful than traditional static same three key ingredients found in those tools. test generation. A key technical challenge is automatic code- At Microsoft, we are developing three new tools driven test generation: given a program with a set for automatic code-driven test generation. These of input parameters, automatically generate a set tools all combine techniques from static program of input values that, upon execution, will exercise analysis (symbolic execution), dynamic analysis (test- as many program statements as possible. An op- ing and runtime instrumentation), model checking timal solution is theoretically impossible because (systematic state-space exploration), and automated this problem is generally undecidable. In practice, constraint solving. However, they target differ- however, approximate solutions suffice: a tool that ent application domains and include other original could automatically generate a test suite covering, techniques.

30 IEEE Softwar e Published by the IEEE Computer Society 0740-7459/08/$25.00 © 2008 IEEE Static versus dynamic are hard or impossible to reason about symbolically test generation with good enough precision. Work on automatic code-driven test generation can Dynamic test generation,4 on the other hand, DART blends roughly be partitioned into two groups: static and consists of dynamic. dynamic test Static test generation consists of analyzing a ■ executing the program P, starting with some generation program P statically by reading the program code given or random inputs; with model and using symbolic execution techniques to simu- ■ gathering symbolic constraints on inputs at con- late abstract program executions to attempt to com- ditional statements along the execution; and checking to pute inputs to drive P along specific execution paths ■ using a constraint solver to infer variants of the systematically or branches, without ever executing the program.2 previous inputs to steer the program’s next ex- The idea is to symbolically explore the tree of all ecution toward an alternative program branch. execute the computations that the program exhibits with all a program’s possible value assignments to input parameters. For This process is repeated until a specific program each control path p (that is, a sequence of the pro- statement is reached. feasible gram’s control locations), symbolic execution con- Directed Automated Random Testing (DART)3 program structs a path constraint that characterizes the input is a recent variant of dynamic test generation that paths. assignments for which the program executes along blends it with model-checking techniques to system- p. A path constraint is thus a conjunction of con- atically execute all of a program’s feasible program straints on input values. If a path constraint is satisfi- paths, while checking each execution using runtime able, then the corresponding control path is feasible. checking tools (such as Purify) for detecting various We can enumerate all the control paths by consider- types of errors. In a DART directed search, each ing all possible branches at conditional statements. new input vector tries to force the program’s ex- Assuming that the constraint solver used to check ecution through some new path. By repeating this the satisfiability of all path constraints is sound and process, such a directed search attempts to force the complete, this use of static analysis amounts to a program to sweep through all its feasible execution kind of symbolic testing. paths, similarly to systematic testing and dynamic Unfortunately, this approach doesn’t work when­ software model checking.5 ever the program contains statements involving In practice, a directed search typically can’t ex- constraints outside the constraint solver’s scope of plore all the feasible paths of large programs in a reasoning. The following example illustrates this reasonable amount of time. However, it usually limitation: does achieve much better coverage than pure ran- dom testing and, hence, can find new program int obscure(int x, int y) { bugs. Moreover, it can alleviate imprecision in sym- if (x == hash(y)) return -1; // error bolic execution by using concrete values and ran- return 0; // ok domization: whenever symbolic execution doesn’t } know how to generate a constraint for a program statement depending on some inputs, we can always Assume the constraint solver can’t “symbolically simplify this constraint using those inputs’ concrete reason” about the function hash. This means that the values.3 constraint solver can’t generate two values for inputs Let’s illustrate this point with our previous pro- x and y that are guaranteed to satisfy (or violate) the gram. Even though it’s statically impossible to gen- constraint x == hash(y). (For instance, if hash is a hash erate two values for inputs x and y such that the con- or cryptographic function, it has been mathemati- straint x == hash(y) is satisfied (or violated), it’s easy cally designed to prevent such reasoning.) In this to generate, for a fixed value ofy , a value of x that is case, static test generation can’t generate test inputs equal to hash(y) because the latter is known dynami- to drive this program’s execution through either cally at runtime. branch of its conditional statement; static test gen- A directed search would proceed as follows. For eration is helpless for a program like this. In other a first program run, pick random values for inputs words, static test generation is doomed to perform x and y: for example, x = 33, y = 42. Assuming the poorly whenever perfect symbolic execution is im- concrete value of hash(42) is 567, the first concrete possible. Unfortunately, this is frequent in practice run takes the else branch of the conditional state- owing to complex program statements (pointer ma- ment (since 33 ≠ 567), and the path constraint for nipulations, arithmetic operations, and so on) and this first run isx ≠ 567 because the expression hash(y) calls to operating-system and library functions that isn’t representable and is therefore simplified with

September/October 2008 IEEE Softwar e 31 its concrete value 567. Next, the negation of the coverage is used as a heuristic to favor the ex- simplified constraint x = 567 can easily be solved pansion of executions with high new coverage. Fuzz testing, and lead to a new input assignment x = 567, y = 42. 3. The tracer records a complete instruction-level Next, running the program a second time with in- trace of the run using the iDNA framework.8 a black- puts x = 567, y = 42 leads to the error. 4. Lastly, the symbolic executor replays the re- box testing Therefore, static test generation is unable to gen- corded execution, collects input-related con- technique, is erate test inputs to control this program’s execu- straints, and generates new inputs using the tion, but dynamic test generation can easily drive constraint solver Disolver.9 a quick and that same program’s executions through all its fea- cost-effective sible program paths. In realistic programs, impreci- The symbolic executor is implemented on top sion in symbolic execution typically arises in many of the trace replay infrastructure TruScan,10 which method for places, and dynamic test generation can recover consumes trace files generated by iDNA and vir- uncovering from that imprecision. Dynamic test generation ex- tually re-executes the recorded runs. TruScan of- tends static test generation with additional runtime fers several features that substantially simplify security bugs. information, so it’s more general and powerful. It is symbolic execution. These include instruction de- the basis of the three tools we present in the remain- coding, providing an interface to program symbol der of this article, which all implement variants and information, monitoring various I/O system calls, extensions of a directed search. keeping track of heap and stack frame allocations, and tracking the data flow through the program SAGE: White-box fuzz structures. testing for security The constraint generation approach SAGE uses Security vulnerabilities (like buffer overflows) are a differs from previous dynamic test generation im- class of dangerous software defects that can let an plementations in two main ways. First, instead of attacker cause unintended behavior in a software a source-based instrumentation, SAGE adopts a component by sending it particularly crafted in- machine-code-based approach. This lets us use puts. Fixing security vulnerabilities after a product SAGE on any target program regardless of its source release is expensive because it might involve deploy- language or build process with little up-front cost. ing hundreds of millions of patches and tying up the This is important in a large company that uses mul- resources of many engineers. A single vulnerability tiple source languages—both managed and unman- can cost millions of dollars. aged—and various incompatible build processes Fuzz testing is a black-box testing technique that that make source-based instrumentation difficult. has recently leapt to prominence as a quick and cost- Second, SAGE deviates from previous approaches effective method for uncovering security bugs. This by using offline trace-based, rather than online, con- approach involves randomly mutating well-formed straint generation. Indeed, hard-to-control nonde- inputs and testing the program on the resulting terminism in large target programs makes debug- data.6 Although fuzz-testing tools can be remark- ging online constraint generation difficult. Thanks ably effective, their ability to discover bugs on low- to offline tracing, constraint generation in SAGE probability program paths is inherently limited. is completely deterministic because it works with We’ve recently proposed a conceptually simple an execution trace that captures the outcome of all but different approach of white-box fuzz testing nondeterministic events encountered during the re- that extends systematic dynamic test generation.7 corded run. We implemented this approach in SAGE (scalable, automated, guided execution), a new tool using in- Generational path exploration struction-level tracing and emulation for white-box Because SAGE targets large applications where a fuzzing of Windows applications. single execution might contain hundreds of mil- lions of instructions, symbolic execution is its SAGE architecture slowest component. Therefore, SAGE implements SAGE repeatedly performs four main tasks. a novel directed search algorithm, dubbed genera- tional search, that maximizes the number of new 1. The tester executes the test program on a given input tests generated from each symbolic execution. input under a runtime checker looking for vari- Given a path constraint, all the constraints in that ous kinds of runtime exceptions, such as hangs path are systematically negated one by one, placed and memory access violations. in a conjunction with the prefix of the path con- 2. The coverage collector collects instruction ad- straint leading to it, and attempted to be solved by dresses executed during the run; instruction a constraint solver. (In contrast, a standard depth-

32 IEEE Softwar e www.computer.org/software void top(char input[4]) { int cnt=0; if (input[0] == ’b’) cnt++; if (input[1] == ’a’) cnt++; if (input[2] == ’d’) cnt++; if (input[3] == ’!’) cnt++; if (cnt >= 3) abort(); // error 0 1 1 2 1 2 2 3 1 2 2 3 2 3 3 4 } good goo! godd god! gaod gao! gadd gad! bood boo! bodd bod! baod bao! badd bad! (a) (b)

Figure 1. Example of or breadth-first search would negate only the last or branch constraints after parsing 1,279,939 instruc- program (a) and its first constraint in each path constraint.) tions over 10,072 symbolic input bytes. SAGE then search space (b) with Consider the program in Figure 1. This program created a crashing ANI file after 7 hours 36 min- the value of cnt at the takes 4 bytes as input and contains an error when utes and 7,706 test cases, using one core of a 2-GHz end of each run. the value of the variable cnt is greater than or equal AMD Opteron 270 dual-core processor running to three. Starting with some initial input good, SAGE 32-bit Windows Vista with 4 Gbytes of RAM. executes its four main tasks. The path constraint for SAGE is currently being used internally at Mi- this initial run is i0 ≠ b, i1 ≠ a, i2 ≠ d, and i3 ≠ !. crosoft and has already found tens of previously un- Figure 1 also shows the set of all feasible pro- known security-related bugs in various products.7 gram paths for the function top. The left-most path represents the program’s initial run and is labeled Pex: Automating with 0 for Generation 0. Four Generation 1 inputs unit testing for .NET are obtained by systematically negating and solving Although it’s important to analyze existing pro- each constraint in the Generation 0 path constraint. grams to find and remove security vulnerabilities, By repeating this process, all paths are eventually automatic-analysis tools can help avoid such pro- enumerated. (See related work7 for other benefits gramming errors to begin with when developing of a generational search as well as several optimi- new programs. zations that are key to handling long execution Unit testing is a popular way to ensure early and traces.) frequent testing while developing software. Unit tests are written to document customer require- Experience ments at the API level, reflect design decisions, pro- On 3 April 2007, Microsoft released an out-of- tect against observable changes of implementation band security patch for code that parses ANI- details, and as part of the testing process, achieve format animated cursors. The Microsoft SDL Pol- certain code coverage. A unit test, as opposed to an icy weblog states that extensive black-box fuzz test- integration test, should only exercise a single feature ing of this code failed to uncover the bug and that in isolation. This way, unit tests don’t take long to existing static-analysis tools aren’t capable of find- run, so developers can run them often while writing ing the bug without excessive false positives. new code. Because unit tests target only individual In contrast, SAGE can generate a crash exhibit- features, it’s usually easy to locate errors from fail- ing this bug starting from a well-formed ANI input ing unit tests. file, despite having no knowledge of the ANI for- Many tools, such as JUnit and NUnit, support mat. We arbitrarily picked a seed file from a library unit testing. These tools manage a set of unit tests of well-formed ANI files and analyzed a small test and provide a way to run them and inspect the program that called user32.dll to parse ANI files. results. However, they don’t automate the task of The initial run generated a path constraint with 341 creating unit tests. Writing unit tests by hand is a

September/October 2008 IEEE Softwar e 33 Figure 2. A glimpse of Pex in Visual Studio.

laborious undertaking. In many projects at Micro- method preconditions and postconditions in the de- soft, there are more lines of code for the unit tests sign-by-contract paradigm.) than for the implementation being tested. Pex (for program exploration; http://research. On the other hand, most fully automatic test- microsoft.com/Pex) is a tool developed at Micro- generation tools suffer from a common problem: soft Research that helps developers write PUTs in they don’t know when a test fails (except for obvi- a .NET language. For each PUT, Pex uses dynamic ous errors, such as a division-by-zero exception). test-generation techniques to compute a set of input To combine the advantages of automatic test gen- values that exercise all the statements and assertions eration with unit tests’ error-detecting capabilities, in the analyzed program. For example, for our sam- we’ve developed a new testing methodology: the pa- ple PUT, Pex generates two test cases that cover all rameterized unit test (PUT).11 the reachable branches: A PUT is simply a method that takes param- eters. Developers write PUTs, just like traditional void TestAdd_Generated1() { unit tests, at the level of the actual software APIs in TestAdd(new ArrayList(0), new object()); the software project’s programming language. The } purpose of a PUT is to express an API’s intended behavior. For example, the following PUT asserts void TestAdd_Generated2() { that after adding an element to a non-null list, the TestAdd(new ArrayList(1), new object()); element is indeed contained in the list: }

void TestAdd(ArrayList list, object element) { The first test executes code (not shown here) in the Assume.IsNotNull(list); array list that allocates more memory because the list.Add(element); initial capacity 0 isn’t sufficient to hold the added el- Assert.IsTrue(list.Contains(element)); ement. The second test initializes the array list with } capacity 1, which is sufficient to add one element. Pex comes with an add-in for Visual Studio that This PUT states assumptions on test inputs, per- enables developers to perform most frequent tasks forms a sequence of method calls, and asserts prop- with a few mouse clicks. Also, when Pex generates erties that should hold in the final state. (The ini- a test that fails, Pex performs a root cause analysis tial assumptions and final assertions are similar to and suggests a code change to fix the bug.

34 IEEE Softwar e www.computer.org/software Figure 2 illustrates Pex in Visual Studio. The up- strengths. Because testing executes a program con- per code window shows a PUT; the lower window cretely, it precludes false alarms, but it might not shows the results of Pex’s analysis of this test com- achieve high coverage. On the other hand, static bined with the code-under-test. On the lower left analysis can cover all program behaviors at the side is a table of generated input parameter values. cost of potential false alarms, because the analysis The selected row indicates an assertion violation ignores several details about the program’s state. when the data argument is an array with two ele- For example, the SLAM project has successfully ments, each of which contain a zero. This violation applied static analysis to check the properties of is associated with a stack trace shown on the lower Windows device drivers.12 Thus, it’s natural to try right side and the highlighted line in the upper code to combine testing and static analysis. For combin- window. In SAGE, the test input is usually just a ing these, the Yogi tool implements a novel algo- sequence of bytes (a file); in contrast, Pex generates rithm, Dash13 (initially called Synergy), which was inputs that are typed according to the .NET type recently enhanced to handle pointers and proce- system. Besides primitive types (Boolean, integer, dures. The Yogi tool verifies properties specified by and so on), the inputs can be arrays, arrays of ar- finite-state machines representing invalid program rays, value types (struct in C#), or instances of other behaviors. For example, we might want to check classes. When the formal type of an input is an in- that along all paths in a program, for a mutex m, terface or an abstract class, Pex can generate mock the calls acquire(m) and release(m) are called in strict al- classes with methods that return different values ternation. Figure 3a shows a program that follows that the tested code distinguishes. The result is simi- this rule. There are an unbounded number of paths lar to the behavior of mock objects that developers for the loop in lines 2 through 6 if the loop count would write by hand today. The following example c is an unbounded input to the program. Thus, it’s shows a method of a mock object that can choose problematic to exercise all the feasible paths of the to return any value: program using testing. Yet Yogi can prove that acquire(m) and release(m) are class MFormatProvider : IFormatProvider { correctly called along all paths by constructing a fi- public object GetFormat(Type type) { nite abstraction of the program that includes (that return PexOracle.ChooseResult