Automatically Testing Interacting Software Components¤ Leonard Gallagher Jeff Offutt Information Technology Laboratory Information and Software Engineering National Institute of Standards and Technology George Mason University Gaithersburg, MD 20899, USA Fairfax, VA 22030, USA [email protected] [email protected] ABSTRACT General Terms One goal of integration testing for object-oriented software is Veri¯cation to ensure high object interoperability. Sent messages should have the intended e®ects on the states and subsequent ac- 1. INTRODUCTION tions of the receiving objects. This is especially di±cult Many object-oriented applications are constructed from a when software is comprised of components developed by combination of previously written, externally obtained com- di®erent vendors, with di®erent languages, and the imple- ponents with some new components added for specialization. mentation sources are not all available. A previous paper Source is often not available for the previously written com- presented a model of inter-operating OO classes based on ¯- ponents, yet new objects must interoperate via messages nite state machines. It addresses methods for identifying the with objects in the existing components. This research is relevant actions of a test component to be integrated into concerned with ensuring that objects inter-operate correctly, the system, transforms the ¯nite state speci¯cation into a particularly when new objects are added to existing compo- control and data flow graph, labels the graph with all defs nents. and uses of class variables, and presents an algorithm to In this paper, a class is the basic unit of semantic abstrac- generate test speci¯cations as speci¯c paths through the tion and each class is assumed to have state and behavior [2, directed graph. It also presents empirical results from an 11]. A component is a closely related collection of classes automatic tool that was built to support this test method. (possibly even a single class), and components inter-operate This paper presents additional details about the tool itself, to provide needed functionality. Each component is assumed including how several di±cult problems were solved, and to be a separate executable, thereby allowing asynchronous adds new capabilities to help automate the transformation behavior. An object is an instance of a class. Each object has of test speci¯cations into executable test cases. The result is state and behavior, where state is determined by the values a fresh approach to automated testing. It follows accepted of variables de¯ned in the class, and behavior is determined theoretical procedures while operating directly on an object- by methods de¯ned in the class that operate on one or more oriented software speci¯cation. This yields a data flow graph objects to read and modify their state variables. The be- and executable test cases that adequately cover the graph havior of an object is modeled as the e®ect the method has according to classical graph coverage criteria. The tool sup- on the variables of that object (the state), together with the ports speci¯cation-based testing and helps to bridge the gap messages it sends to other objects. Variables declared by between theory and practice. the class that have one instance for each object are called instance variables, and variables that are shared among all objects of the class (static in Java) are class variables. This Categories and Subject Descriptors research is independent of programming language and the D.2.5 [Software Engineering]: Testing and Debugging| paper uses a mix of Java and C++ terminology. Testing tools Behavior of objects is captured as a set of transition rules for each method and described as ¯nite state machines [3, 4, 12]. A transition is triggered by a call to a method with ¤The second author is sponsored in part by National Insti- a particular signature, and is comprised of a source state, tute of Standards and Technology (NIST), Software Diag- a target state, an event, a guard, and a sequence of ac- nostics and Conformance Testing Division (SDCT). tions. Events are represented as calls to member functions of a class. A guard is a predicate that must be true for the transition to be taken; guards are expressed in terms of predicates over state variables (possibly from multiple Permission to make digital or hard copies of all or part of this work for classes) and input parameters to the method. An action is personal or classroom use is granted without fee provided that copies are performed when the transition occurs; actions are usually not made or distributed for profit or commercial advantage and that copies expressed as assignments to class member variables, calls bear this notice and the full citation on the first page. To copy otherwise, to sent to other objects, and values that are returned from the republish, to post on servers or to redistribute to lists, requires prior specific event method. A sequence of actions is assumed to be a permission and/or a fee. AST’06, May 23, 2006, Shanghai, China. block of statements in which all operations are executed if Copyright 2006 ACM 1-59593-085-X/06/0005 ...$5.00. any one is executed. Pre-conditions and post-conditions of methods can be de- rived directly from the transitions. The pre-condition is a Sss Gt Tt Sts combination of the predicates of the source state and the guard; the post-condition is the predicate of the target state. Note that the post-condition derived from a transition is Figure 1: Path Derived From a Relevant Transition not the strongest post-condition. If the tester desired, state de¯nitions could be more re¯ned, allowing stronger post- conditions, which would yield larger graphs and more tests. Whether to do so is a choice of granularity that results in a ² There is a directed edge from every transition node to cost versus potential bene¯t tradeo®. its target state A state transition speci¯cation for a class is the set of ² There is a directed edge from the source state node of state transition rules for each method of the class. Given a a transition to either the guard node of that transition state transition speci¯cation for each class, the goal of this or to the transition node itself if the guard is trivially research is to construct test speci¯cations that are used to true construct an executable test suite. Hong et al. [8] developed a class-level flow graph to represent control and data flow In general, every relevant transition t produces a path within a single class. Our previous paper [6] extended their in the component flow graph from its source state node, ideas to integration testing of multiple interacting classes. through the guard and transition nodes, to the target state The state transition speci¯cation is stored in a relational node. This is shown in Figure 1. database. Transitions that are relevant to the class under The path in Figure 1 represents control flow from the state test are used to create a component flow graph, which in- that an object is in when a method event is invoked, to the cludes control and data flow information. Classical data guard node that evaluates to true, to the action of the tran- flow test criteria are applied to this graph and converted to sition with that guard, to the target state of the transition. test speci¯cations in the form of candidate test paths, and Actions on variables are also represented by transitions with then to executable test cases. associated get and set methods. Transitions with get (or ac- In traditional data flow testing [5], the tester is provided tor) methods typically do not have non-trivial guards, and with pairs of de¯nitions and uses of variables (def-use pairs), never change the state of an object, so usually produce paths and then attempts to ¯nd tests to cover those def-use pairs. in the component flow graph only from state nodes where This research stores information about the speci¯cation in the method is de¯ned through the transition node and back the database, represents object behavior as branch choices to the same state node. in a directed graph, provides the tester with full def-use In multi-class systems, state and guard predicates are al- paths instead of just def-use pairs, and provides control lowed to call actor functions from other classes and the ac- mechanisms to construct calls of external methods that force tion of a transition is allowed to invoke actor or mutator traversal of the identi¯ed paths. functions in other classes. Control and data flow result- ing from these actions is represented by edges that connect 2. BEHAVIOR IN A DIRECTED GRAPH nodes from di®erent classes, as follows: A combined class state machine represents the variables, ² If a state or guard predicate, or the action of a transi- methods, parameters, states and transitions of all state tran- tion, calls an actor method in another class, there will sition speci¯cations for all classes in a system of interoper- be an edge from every transition node of the called ating components. If a speci¯c component is identi¯ed as method back to the node representing the calling state, the test component in this system, then transitions from the guard or transition. classes in the test component form the basis of transitions that are relevant to that component. Relevant transitions ² If the action of a transition calls a mutator function in that call mutator methods in other components of the sys- another class, there will be an edge from the transition tem represent outward data flow, whereas transitions that node that represents that transition to the source state call actor functions in other components represent inward node of every transition of the called method in the data flow.