TO APPEAR IN Proceedings Automated Software Engineering 2001 Conference (ASE 2001),NOV. 2001. 1 A Technique for Mutation of Java Objects James M. Bieman Sudipto Ghosh Roger T. Alexander Computer Science Department Colorado State University Fort Collins CO 80523, USA g fbieman, ghosh, rta @cs.colostate.edu Abstract to model any real fault, software fault models are hard to de- velop. Mutation of scalar values is straightforward, because Mutation analysis inserts faults into a program to create their semantics are well understood. For example, we can test sets that distinguish the mutant from the original pro- add one to an integer value. Mutating objects that are in- gram. Inserted faults must represent plausible errors. Stan- stances of user de®ned types is more dif®cult. There is no dard transformations can mutate scalar values such as in- obvious way to modify such objects in a manner consistent tegers, ¯oats, and character data. Mutating objects is an with realistic faults, without writing custom mutation meth- open problem, because object semantics are de®ned by the ods for each object class. Our approach is to inject faults into programmer and can vary widely. We develop mutation op- objects that instantiate items from common Java libraries. erators and support tools that can mutate Java library items We address the following issues in the paper: that are heavily used in commercial software. Our mutation engine can support reusable libraries of mutation compo- 1. Derivation of mutation operators for Java: Prior work nents to inject faults into objects that instantiate items from de®nes operators to mutate statements, operators, con- these common Java libraries. Our technique should be ef- stants and variables for FORTRAN [11], C [1, 2, 3], fective for evaluating real-world software testing suites. Ada [10] and Java [6]. We describe mutation faults that can be injected into Java objects. 2. Generation of mutants by applying the operators to 1. Introduction programs: Prior work focuses on compile-time or static mutation [9]. We show how to generate Java pro- gram mutants during program execution. Mutation analysis aids in the assessment of the adequacy of tests [9]. It involves the modi®cation of programs to see if existing tests can distinguish the original program from 2. Mutation Operators for Java the modi®ed program. Traditionally, syntactic modi®ca- tions have been used. A set of mutation operators gener- Mutation operators de®ned for procedural languages are ate the syntactic modi®cations, which are determined by the also applicable to Java. Java includes additional features language of the program being tested, and the mutation sys- related to the object-oriented paradigm. Offutt et al. de- tem used for testing. Mutation operators are created with ®ned mutation operators for Ada [10]; these operators ad- one of two goals: to induce simple syntax changes based dress some object-oriented features; they do not address in- on errors that programmers typically make (such as using heritance and are limited to properties within a class. the wrong variable name), or to force common testing goals Kim et al [6] propose mutation operators for Java based (such as executing each branch). on deviations of Java language constructs. The mutation op- One way to conduct program modi®cation is to change erators do not take object semantics into account. Several data values while a program is running. We are developing operators create mutants that do not compile. a mechanism to conduct mutation analysis of objects rather Objects have state and methods implement transitions than just scalar data values. We inject faults into the objects from one state to another. Doong and Frankl's [4] AS- at runtime. Others have used runtime fault injection, for ex- TOOT uses algebraic speci®cations to generate method test ample, Voas uses fault injection to measure testability [12]. sequences. Kirani and Tsai [7] describe a method sequence Injected faults must represent plausible errors. Unlike testing method: testers select sequences of methods in vary- hardware, where a combination of simple faults can be used ing orders and length. Kung et al. [8] describe testing based on state transition diagrams. Mutation operators that are ap- before it is used by inserting, just after line 3, the following plied to program code are not suf®cient to ensure that objects statement: will go through different states. Interface mutation identi®es errors that programmers f = (Foo) ObjectMutationEngine.mutate(f); may make in de®ning, implementing and using interfaces. We can also mutate an object returned by a method, as Integration mutation operators test the connections between shown in Figure 2, placing the mutate calljustpriortothe two modules by mutating module interfaces [3]. return statement. The ObjectMutationEngine implements Ghosh and Mathur [5] use interface mutation on dis- the mutate methods. Inserting calls to the ObjectMutatio- tributed object systems that use CORBA, DCOM and Java- nEngine is relatively easy using a code instrumenter that RMI. Interface mutation operators change parameter values builds a parse tree and inserts calls to mutate into certain in method calls de®ned in an interface. nodes in the tree. The ObjectMutationEngine is described The above mutation operators can be easily applied when in Section 4. the parameters are scalars. A simple object mutation is to make an object reference null. A more plausible approach 1. class C { is to modify the state of an object. State mutation operators 2. ... cannot be applied statically to a program, because the state 3. public Bar m (Foo f) { 4. ... of the object depends on program execution. Interface muta- 5. Bar b; tion operators will not reveal state errors caused by speci®c 6. ... method sequences. These errors depend on the order of op- 7. return b; eration and not the values of the arguments. 8. } 9. ... 10. } 3. Mutation Faults for Java Objects Figure 2. Mutation of Return Statement Instead of de®ning mutation operators for each class in the Java API, we de®ne operators that apply to a whole group of classes that implement a certain interface. We ex- 3.1. Default mutation operators amine mutation operators that apply to the following inter- faces: Mutating an instance of an arbitrary user-de®ned type Container types de®ned in the interfaces, Collection modi®es the ®elds of the object. The Java re¯ection API en- and List in the package java.util. ables us to identify the types of objects. If the ®eld is a scalar, Iterators de®ned in the interface Iterator in the package we apply traditional scalar mutation operators: java.util. 1. Increment the value by 1 InputStream de®ned in the abstract class InputStream 2. Decrement the value by 1 in the package java.io. 3. Set the value to a constant We also de®ne default mutation operators. Our new mutation operators apply to ®elds that are ob- 1. class C { jects. If the semantics of the objects are known, it is easier 2. ... to select the operators. Default operators apply when the se- 3. public void m (Foo f) { mantics are not known. A default operator might make an 4. ... object reference null. Such a mutation will probably raise a 5. ... 6. } NullPointerException, limiting its utility. Moreover this is 7. ... an operation that is applied to the reference, not the object. 8. } This mutation treats a variable that refers to an object as hav- ing an underlying type of ªreference to objectº and takes ad- vantage of knowledge of the semantics of variables of this Figure 1. Code of Class C sort. It can only mutate an object reference by assigning it a value, testing for equality between two such variables, and dereferencing its value. Injecting The Faults. Figure 1 shows a class C containing Another mutation operator for arbitrary object types is a method m() with parameter f which references an object one that recursively applies mutation operators to the nested of type Foo. Code inside m (not shown) uses the parame- ®elds until the scalar ®elds are reached. The following op- ter f for computation. We can mutate the object bound to f erators can also be applied: 2 1. Cloning the object referred to by the variable and as- Operators for container implementations. The speci®c signing the reference to the clone to the variable. This semantics of implementations can be used to mutate con- tests the sensitivity of a program to the object's identity tainer objects. For example, a binary tree has a notion of rather than its state. ordering. This notion can be used to implement a reorder 2. Creating a new object whose type is compatible with mutation operator. the declared type T of the object reference Ð we in- stantiate a new object whose type is a descendant of T . 3.3. Operator for the Iterator interface 3.2. Mutation operators for containers The Iterator interface in the Java API generates the next element in the iteration using the method next().Askip op- Operators for the Collection Interface. erator makes the iterator skip elements. It is implemented by a mutate method that calls the next() method one or more 1. Make the Collection empty: times. In Java every Collection needs to implement the The skip operator does not affect all instances of the mu- method clear(). The mutate method for making the tated type but just the instance held by some client. Since Collection empty just invokes the clear() method on we are only mutating one iterator and not the original con- this object. tainer, any other client using the container or a different it- erator over the container will not see a difference.