A Practical Approach to UML-Based Derivation of Integration Tests
A Practical approach to UML-based derivation of integration tests
F. Basanieri, A. Bertolino IEI-CNR, Pisa, Italy
Abstract: We present an on-going project for developing a We present here our practical approach for UML- tool supported test methodology based on UML descriptions. based integration testing, that is called Use- The leading criteria of this investigation are that no Interaction testing, as it mainly uses the UML Use additional language or expertise is required more than UML, Case and Interaction diagrams (specifically, the and that the methodology can be easily transferred to Sequence diagram). Indeed, integration testing is industrial contexts. aimed at verifying that the (hopefully pre-tested) system components interact correctly, and UML Keywords: Interaction diagrams can provide the information Category-partition, Integration test strategy, UML, Use- of how the system components should interact. Interaction test. The proposed approach is very simple, and is inspired at large by the well-known Category 1. INTRODUCTION Partition method [OB88]: we first look at the Use In recent years, the object oriented (OO) Case diagram to identify the suitable steps of an paradigm has got widespread use. Software incremental test strategy. For each identified sub- developers need appropriate OO models and Use Case, we then look at the Sequence diagram support tools for describing and analysing the to identify the relevant components, or "Test characteristics of complex distributed systems. OO Units". For each Test Unit, we derive the relevant system models usually involve graphical notations, "Settings" (could be parameters, variables, so that the relationships between the many environmental states) and the relevant elements (objects) forming the system can be "Interactions" (essentially, messages from other easily visualized. Test Units), and identify for them the significant UML, the Unified Modeling Language, is the "Choices" (with the same meaning of the Category emerging graphical notation to model, document Partition method). To do this, we can analyse the and specify OO systems along all the phases of related Class Diagrams, as well as other design the software process. Indeed, there exist now documentation. Hence, by following the many studies about using UML for design, and sequences of messages between the components many tools are available. However, only a little over the Sequence Diagram, we construct the Test part of these studies so far has addressed the usage Cases, whereby each Test Case is characterized by of UML for testing, and none of the available a combination of all suitable choices of the commercial tools provide specific assistance for involved Settings and Interactions. test planning and generation from the UML We have applied the method to a case study, descriptions. Argo/UML [Argo]. Argo/UML is an open source In this paper we address UML-based testing. tool that provide cognitive support to system Specifically, we want to address this task design. We have developed quite a few Test Cases, according to the following two guidelines: (i) we and we have been able to identify a few bugs in aim at a test method that is entirely based on the case study. However, the method is currently UML, so that it can be easily adopted by manual, and still under evaluation. Our short term industries already using UML; and, (ii) we refer to plan is the realization of a tool that support it, by high level descriptions of the system: indeed, we smoothly expanding an existing UML design want to address test planning for the integration tool. test phase starting from the very first stages of The paper is organized in the following way. In system design. the next section, we provide some introductory material to UML and Integration testing. In
1 Section 3, we describe the method in more detail, Sequence diagram: it shows the dynamic explaining it step by step. Then, in Section 4 the collaborations between a certain number of case study Argo/UML is briefly introduced, so objects, highlighting the way in which a particular that in Section 5 some examples of application of scenario1 is realized using the interactions of a Use-Interaction to it are shown. In Section 6 we (sub)set of these objects. More precisely, a also provide a very brief sketch of a UML scenario is described by the set of messages definition of the Use Interaction method itself by exchanged between objects. A Sequence diagram means of the UML extension mechanism of expresses the same information of a Collaboration stereotypes. diagram, whereby the former describes the interactions between the objects during their life- 2. BACKGROUND cycle, while the latter shows the relative UML is a graphical modeling language to distribution of these objects links in the space. visualize, specify, design and document all the phases of a software development process. Born 2.2 Integration Testing and UML-based Testing by the unification of I. Jacobson, J. Rumbaugh The testing goal is to execute the system to verify and G. Booch methods, in 1997 it was declared by its behaviour and to reveal possible failures. OMT the standard for analysis and design of Testing is an important piece of the software Object-Oriented systems. development process, because of its cost and impact on the reliability of final product. In our 2.1 UML diagrams methodology we consider the Integration Testing A UML design consists of an integrated meta- phase, performed to find errors in unit interfaces model composed of many elements representing and to build up the whole structure of software OO common world concepts. Through this meta- system in a systematic way. model we define Views showing different aspects To do this, one could use a non-incremental of the system to be modelled (Logical View, approach (big-bang), where all the modules are Component View, Deployment View, linked together and tested all at once, or Concurrency View and Use Case View). As a preferably, incremental approaches like top-down, whole, these views provide a complete picture of where modules are integrated from the main the system to be built. UML diagrams are graphs, program downto the subordinated ones, or, each describing the content of a view; they can be bottom-up, where tests are constructed from arranged in different combinations to provide modules at the lowest hierarchical level and then several system’s views. are linked together upwards, to construct the We only provide in this section a brief description whole system. of the UML diagrams mainly used in our But, when we consider an object-oriented system, methodology (for more details see [UML97a], the described techniques are not always usable, [UML97b]): because, for example, we cannot identify the Use Case diagram: a use case is the hierarchical structure of control by which it is representation of a functionality (a specific use) possible to define top-down or bottom-up provided by the system. This diagram shows a strategies. In an object-oriented environment we number of external users (actors) and their can test the class interactions by, for example, relationships with the system, when this is used to integrating together those classes used in reply to satisfy a specific functional requirement. a particular input or system event (thread-based Class diagram: it shows the static structure of the testing) or by testing together those classes that system through the representation of its classes contribute to a particular use of the system. In the with their attributes and methods. Moreover it proposed methodology, the classes to be specifies the relations between the classes using integration tested (modules, subsystem, processes) different types of associations. A system can have are those that realize a system functionality more than one Class diagram; in subsequent identified from a Use Case diagram. phases of the software development process, the
Class diagram represents objects at different levels 1 A scenario is defined as a specific sequence of actions that of abstraction. illustrates behaviour and may be used to show an interaction. [UML97b] 2 As a matter of fact, even though UML is a Use Cases, since, to obtain a complete system powerful mechanism of description, we have functionality, it is generally necessary to execute found few studies about its use to guide the testing several actions realizing lower level functionalities. phases. In our methodology, Use Case diagrams drive Some researchers have proposed methods to Integration test according to an incremental translate a UML description into another formal strategy. We start analysing low-level description, and then derive the tests from the functionalities that represent a subset of actions latter. For instance, in [JGP98] the authors present described by a sub-Use Case, and then we a tool, UMLAUT, that is used to manipulate the progressively put them together, until the whole UML representation of the system and system described in the main Use Case is automatically transforms it into an intermediate obtained. form, suitable for validation. Another interesting For each selected Use Case we analyse the paper is [OA99], where a method is proposed to corresponding Sequence diagram, composed by generate test data from UML State diagrams. objects and the messages they exchange. The They translate the UML State diagram into formal objects involved in the diagram are those that SRC specifications, from which input data for unit realize and execute the functionality described in testing are automatically generated. Finally, in a the Use Case through elaborations and message recent paper from Siemens Corporate Research exchanges and so they are precisely the [HIM00] UML diagrams are used to components to be tested. In this phase we consider automatically construct test cases as follows. The the Class diagram too, and particularly a Class developers first define the dynamic behaviour of diagram at high level of abstraction. This diagram each system component using a State diagram; the becomes important to define operations (or interactions between components are specified by abstract operations) and attributes required by annotating the State diagrams, and then the global classes for the interactions of their objects. Also FSM that corresponds to the integrated system the Collaboration diagram could be used to behavior is used to generate the tests. This analyse the object interactions, but it is not as approach is being automated to execute the tests expressive, since it underlines links and in an environment compatible with the UML dependencies among objects, but not their modelling tool Rational Rose. dynamic interactions along a temporal sequence. All the mentioned studies are interesting, and we Therefore, for our methodology, the most see them as complementary to ours, as we do not important diagram is Sequence diagram, that is use State diagrams, but system descriptions at the basis for generating integration tests. In coarser granularity. [JBR98], in fact, we can find a suggestion to use this diagram in integration testing phase. They 3. AN INTEGRATION TEST APPROACH suggest to study different sequences found in this diagram from a possible input state, or from a 3.1 Overview of the Use Interaction Testing system input done by actors. approach. To analyse the message sequences we have used a As said, Integration Testing progressively verifies systematic methodology inspired by the well- the interactions between software components known Category Partition Method [OB88]. (modules, packages, subsystem, processes, classes) in order to realize the final integrated system. 3.2. The Category Partition method The Use Interaction methodology for Integration Category Partition (CP) [OB88] is a well-known testing uses as a reference model the UML method to systematically derive functional tests diagrams to systematically construct and define from the specifications. tests. Generally speaking, the partitioning of the input The Use Case diagrams, by visualizing the various domain is a standard approach to functional system functionalities, help the tester to decide the testing, based on the idea that, for the classes of way in which the system can be decomposed for equivalence defined by the identified partitions, testing each of its parts (representing a specific one or few tests can be selected as representative functionality of a Use Case) and then the whole of the whole class behaviour. system. Each Use Case can contain in turn other
3 The first step of the CP method is to analyse the called TSL. The CP method has encountered wide functional requirements to divide the analysed interest, and has inspired the development of a system in functional units to be separately tested. large number of test methodologies, also using A functional unit can be a high-level function or a formal languages such as Z. procedure of the implemented system. For each defined functional unit, the environment 3.3 Steps for Use-Interaction testing conditions (system characteristic of a certain We now describe in more detail the proposed functional unit) and the parameters (explicit input approach. It can be logically subdivided into of the same unit) relevant for testing must be seven steps. identified. Test Cases are then derived by finding Step 1: UML Design analysis and search of significant values of environment conditions and different Use Cases. We analyse UML design and, parameters; this can be done dividing them into particularly, Use Case diagram. This diagram can categories representing relevant system properties lead test construction by defining different or particular characteristics of parameters or integration test stages with respect to an environment conditions. Then, for each category, incremental strategy from lower-level abstraction we identify different choices, that are different levels upto higher ones. Inside this diagram we significant values for these categories. The CP can thus find different Use Cases representing method uses a tool to automatically construct test (sub)functionalities used to construct the main cases from specifications expressed into a one (see, for example, Fig.1) dedicated semi-formal specification language,
Fig 1: Use Case diagram (of uci.argo.Kernel)
Step 2: Analysis of Sequence and Class diagrams from top to bottom along the objects life time, of involved in the selected Use Case. After selecting messages involved in object interactions. one of the Use Cases, we analyse the relative Moreover, horizontal axis is useful for tracing the Sequence (Fig. 2) and Class (Fig. 3) (at a high message sequences, that is at the basis of test cases level of abstraction) diagrams. The focus is construction. mainly on the Sequence diagram, analysed along Step 3: Test Units definition. Each object inside a two orthogonal directions corresponding to its Sequence diagram is considered a Test Unit, since axes. The horizontal axis (Arrow a) shows a set of it can be separately tested and it represents and objects that interact through messages; this axis is defines a possible use of system. Test Units search used to verify the object interactions and their can be done automatically because it descends correct use with respect to the Use Case directly from a Sequence diagram. requirements. The vertical axis (Arrow b) is Step 4: Research of Settings and Interactions studied because it shows the temporal sequence, Categories. Interactions categories are the 4 interactions that an object has with the others the messages involved in a Sequence diagram, it is involved in a same Sequence diagram and they possible to find some Messages Sequences, i.e., a are represented by all the messages to the set of messages used by objects to define and considered object. In this type of diagram, in fact, elaborate particular functionalities. Several the interactions are defined with a sender who categories can correspond to each found Message sends a message to a receiver asking for a service Sequence, of the two types Interaction and provided by the latter. So, following vertically the Settings: precisely, messages/methods in the life time of the analysed object, we can find all its considered sequence identify the Interaction Interaction Categories represented by the entering Categories, and the attributes that affect these arrows. We could also study these methods in the messages identify the Settings Categories. For Class diagram description, but it is for searching each category (of either types) within a Message the Settings Categories that the Class diagram Sequence, we consider each possible choice, becomes very important. Settings categories are taking them from the Test Specification (Step 5). attributes (or a subset of them) of a class (and the Then, we derive as many Test Cases as necessary, corresponding Sequence diagram’s object), like by considering all potential combinations of input parameters used in messages or data compatible choices. The construction of Test structures. Cases could be performed by a tool, after Step 5: Test Specification construction. In this Messages Sequences and Test Specification have step we define a Test Specification involved in a been appropriately defined. Test Unit, that is: for each found category we find Step 7: Definition of Use Case Test Suite and all its possible values and constraints. For this aim Incremental Construction of Test Frame. Finally, we use the Class diagram where we can find a all Test Cases built for a same Use Case are preliminary description of a method collected together into a Use Case Test Suite. If implementation, its possible input values or the there exist more than one Use Cases to be description of an attribute used and its significant analysed at the same level of abstraction, we repeat values. the process from step 2, constructing other Use Step 6: Search of Messages Sequences and Test Case Test Suites. Cases definition. Following the temporal order of
Fig 2: Sequence diagram analysis
After all the (sub)-Use Cases, at a given level, have Cases at higher levels of abstraction, based on the been analysed, we consider other possible Use Use Case diagrams, until we reach the main Use 5 Case. The same seven-steps process is applied at system components than those considered for the the identified (higher) Use Cases, using again the (sub)-Use Cases. Sequence and Class diagrams. As we move Finally, we define a Test Frame, that is, the set of towards the main Use Case, the Test Units in the all Use Case Test Suites built for the analysed Sequence diagram will generally be more abstract (sub)system.
Designer
Decision Goal
Decision(name:String;priority:int ) Goal(name:String;priority:int) GetName() GetName() GetPriority() GetPriority()
* *
contains contains
DecisionModel GoalModel _decisions _goals DecisionModel() GoalModel() GetDecisions() GetGoals()
Fig 3: Class diagram of the selected Use case for uci.argo.UserModel
4. THE CASE STUDY based on: (a) Reflection-in-action; (b) The analysed case study is Argo/UML [Argo], a Opportunistic Design. tool supporting Object Oriented design with (a)Reflection-in-action is divided into: Design Critics : UML; this is an open source project that was simple agents that continuously execute design analysis in a launched by a research group of the Institute of background thread of control, while the designer is working, Computer Science of Irvine, University of and suggest possible improvements. The suggestions can be, California. Argo/UML is based on the UML 1.1 for instance, the signalling of syntax errors or reminders to specification and uses a Java version of UML return to parts of the design that needed to be finishing; meta-model with support for OCL and XMI Corrective automations : done by a Wizard for critics (XML Model Interchange format). identifying specific problems in the design; To Do List : We selected this case study because it follows items, organized in a To Do List, helping designer about exactly the UML specifications, and it is many details of his work. They are, for example, suggestions completely UML designed with good from arisen critics, personal notes, suggestion for documentation on Java code, even if only a part improving and completing some parts and so on; User of its UML design is available on the web. This Model : maintains information about designer’s choices tool is designed to provide very interesting about project and its features. This is done, for example, features, but most of them are not yet developed suggesting only critics that are relevant for designer’s tasks. and only three diagrams are supported so far: Use User model consists on a decision model, a list of Case, Class and State Machine diagrams. Its major decisions that must be made when doing object-oriented feature is to provide “cognitive support for design and goal model, a list of goals that designer must design”, that is an intelligent support for reach for the design project. (b) Opportunistic Design divided designers who have to build a complex software into: To Do List : a list of To Do Item; CheckList : used to product. This support, as the authors describe, is remind designers to cover all design details and avoid
6 common errors. These lists are specific to the selected design items, where you can keep track, every time, of active critics element (association, classes, attributes..). and their priority values. To Do List: manages To Do For our case study we have chosen the part of the tool Items, Fig. 5; it shows all active critics descriptions and realizing cognitive support for design, the Argo package, steps to eliminate them. User Model: manages decisions divided into two other packages: uci.argo.Kernel, that (Decision Model) and goals (Goal Model) made by users for manages Critics, User Model and To Do List and project. Decision Model, Fig. 6, is used to specify which uci.argo.Checklist, that manages Checklist. In this paper, requirements are important for designer’s aim; they have a for reason of space, we analyse only uci.argo.Kernel. Its priority value from 0 to 5 and if a decision priority is 0 the functionalities are divided into: Design Critics: that corresponding critics are made inactive. Goal Model manages critics, their definition and use during design. shows which goals designer would achieve, Fig. 6. Now only Usable critics are visible in Critic Browser, Fig.4, that keeps a goal type (Unspecified) is usable; like decisions, goals track of all information like critic state (active or inactive), have a priority from 0 to 5 and, if a goal has priority 0 the priority, general description and so on. All critics properties related critic is made inactive. can be changed by users but it implies changes in To Do
Fig 4: Argo/UML Critic Browse Windows
Fig 5: Argo/UML To Do Pane
7 Fig 6: Argo/UML Design Issues Panel and Design Goals Panel
5. APPLICATION OF USE INTERACTION GoalModel METHODOLOGY TO THE ARGO CASE settings:_goals STUDY interactions:GoalModel(), getGoals() Step 1: UML Design analysis and search of Goal different Use Cases. The Use Case diagram of interactions: Goal(name:String, priority:int), uci.argo.Kernel is shown in Fig. 1. We can see that GetName(),GetPriority() the main functionality of Kernel is divided into Step 5: Test Specification construction. We three sub-functionalities, each of which can describe the Decision Model Test Unit. represent a single sub-Use Case. These Use Cases Decision Model: is a part of Design state, it describes which can be analysed separately to construct the Test type of decisions are useful for designer; critics relevant for Units and Test Cases of a Use Case Test Suite, and these decisions become active. then linked together to build one main Test Settings: _decision Frame. In this section we analyse only the User Naming, Storage, Stereotypes, Inheritance Model functionality. Relationship, Modularity, …. Step 2: Analysis of Sequence and Class diagrams Interactions: involved in the selected Use Case. Fig. 2 and 3 GetDecisions() show Sequence and Class diagram for Opening a new file uci.argo.Kernel.UserModel. Opening a saved file Step 3: Test Units definition. After a modification and before saving TestUnits : After a modification and after saving DecisionModel, Decision, GoalModel, Goal. DecisionModel() Step 4: Research of Settings and Interactions Constructor of class. Categories. Step 6: Search of Messages Sequences and Test Categories : Cases definition. One of the possible Messages DecisionModel Sequences, highlighted in Fig.2 by Arrow c, is: setting: _decisions getDecisions() -> getName / getPriority(), interactions: DecisionModel(), getDecisions() and the corresponding Test Case for a possible Decision choice involved categories is: interactions: Decision(name:String,priority:int) , TEST CASE GetName(), GetPriority() getDecisions() 8 getName()/getPriority() 6. UML DEFINITION OF METHODOLOGY Opening a saved file. The Use Interaction testing methodology follows _decisions = Naming a precise process, and therefore can itself be Action to perform test:Visualizing Design Issues Panel for designed using the UML language. We show, in the considering decision and priority opening a saved Fig. 7, a Use Case diagram in which the actor is document. Tester who interacts with system generating test Instructions for checking the test: opening the Design Issues cases. Panel we see decision with Name=Naming and priority like Moreover, all the new introduced concepts, like priorità of the same file previously saved. Test Unit or Setting and Interactions Categories, Note 1: This Test must be repeated for all possibile values of have been defined using exthension mechanism _decisions. of stereotypes. A stereotype [UML97a] is a new Step 7: Definition of Use Case Test Suite and type of modeling element that extends the Incremental Construction of Test Frame. After defining Test Cases we build a Use Case Test Suite for User Model, Design Critics, To Do List and then, following an incremental integration test strategy, we analyse the main functionality of Kernel constructing its Use Case Test Suite and, at the end, the entire Test Frame. Results from the case study: when the method has been applied to case study Argo/UML some bugs were discovered. We show two Test Results: the first one, without bugs, is referred to previous Test Case (Step 6); the second is an example of a test with fail result. TEST CASE 1 Fig. 7: Use Case diagram of Use Interaction Test method. getDecisions() getName()/getPriority() semantic of the existing metamodel; all our new Opening a saved document. concepts are (sub)classes of the existing metaclass _decisions = Naming Class with new additional characteristics and Test Result: Passed. constraints. For instance, we show a stereotyped TEST CASE 2 class, Test Unit, with its properties and constraints SetDecisionPriority (priority:int) defined using the OCL language [UML97c]. From a >0 value to 0 before saving. BeInactive() Test Unit: Critic is active. <
9 [2] The attributes of a TestUnit are SettingsCategories and InteractionsCategories. self.Attribute.ocltype = enum{SettingsCategories, InteractionsCategories} References: [3] A SettingsCategory represents one of the class [Argo] "Argo/UML", available from attributes and it is involved in one iteration. http://argouml.tigris.org. self.SettingsCategory_exists(a Attribute| a.isused=self and a [BCIM00] A. Bertolino, F. Corradini, P. Inverardi and H. IsInvolvedIn Interactions) Muccini, "Deriving Test Plans from Architectural [4] InteractionsCategories are the only associations of Descriptions"”, Proc. ACM/IEEE 22nd Int. Conf. on CollaborationDiagram. Soft. Eng. (ICSE 2000), Limerick, 4-11 June 2000, pp self.InteractionsCategory_forAll(i|self.ocltype.Collaboratio 220-229. n.Association_exists(ass| ass.name = i)) [HIM00] J. Hartmann, C. Imoberdof, M. Meisenger, "UML-Based Integration Testing", Proceedings of ISSTA
7. CONCLUSIONS 2000, Portland, Oregon, 22-25 August 2000. In this paper we have presented our methodology, Use Interaction Test, that generates Integration [JBR98] I. Jacobson, G. Booch, J.Rumbaugh, The Tests from UML diagrams like Use Case and Unified Software Development Process, Addison-Wesley, Interactions diagrams. 1998. This testing approach is placed inside a larger [JGP98] J.M Jézéquel, A. Le Guennec, F. Pennanech, research project trying to develop methods and "Validating Distributed Software Modeled with UML", tools for design-based integration test of complex Proc. UML98, in LNCS 1618, pp. 365-376. system. The approach used in this work is not based on formal methods for specification, like [MLB99] F. Mercier, P. Le Gall, A. Bertolino, the related [BCIM00], [MLB99] works, but only Formalizing integration test strategies for distributed on exclusive use of UML diagrams. systems”, First Int. ICSE Workshop Testing Distributed Component-Based System, Los Angeles, May 1999. For this reason, we think it may be a viable method for its simplicity and easy portability to [OA99] J. Offutt, A. Abdurazik, "Generating Test from industrial contexts; moreover, since we use only UML Specifications", Second International Conference on UML diagrams, the methods does not require the Unified Modeling Language, UML 99, Fort Collins, specialised expertise and analysis, and so CO, October 1999. generation of test cases can be done [OB88] T.J. Ostrand, M.J. Balcer, "The Category contemporary with project development, at no or Partition Method For Specifying and Generating little extra cost. Functional Tests", Communication of the ACM, 31(6), On the other hand, the exclusive use of UML p. 676-686, June 1988. diagrams can be considered also a limit of our method, because as known UML semantic is not [UML97a] UML Notation Guide, v. 1.1, Sept. 1997, very precise, and therefore the diagrams can have www.rational.com/uml/resources/documentation/formats.j tmpl different interpretations from different users, and also different designers could provide different [UML97b] UML Semantics, v. 1.1, Sept. 1997, diagram specifications. www.rational.com/uml/resources/documentation/formats.j In future, we aim at constructing a tool tmpl automatizing part of this process in collaboration with industrial partners. The tool would help the [UML97c] Object Constraint Language Specification, version 1.1, Sept. 1997, Tester in Test Cases generation for the tasks not www.rational.com/uml/resources/documentation/formats.j requiring human judgement (for example, tmpl identification of Test Unit and Messages Sequences), thus enhancing the cost effectiveness of the approach.
Acknowledgements This work is supported in part by the Italian Murst Project "Saladin: Software Architecture and Languages to coordinate Distributed Mobile Components". 10