UNIVERSAL UNIT TESTS GENERATOR BASED ON SOFTWARE MODELS Sarunas Packevicius, Andrej Usaniov, Eduardas Bareisa Kaunas University of Technology, Department of Software Engineering, Studentų 50, Kaunas, Lithuania, [email protected], [email protected], [email protected] Abstract. Unit tests are viewed as a coding result of software developers. Unit tests are usually created by developers and implemented directly using specific language and unit testing framework. Unit test generation tools usually does the same thing – generates tests for specific language using specific unit testing framework. Thus such generator is suitable for only one situation. Another drawback of these generators – they mainly use software code as a source for generation. In this paper we are presenting tests generator model which could be able to generate unit tests for any language using any unit testing framework. It will be able to use not only software under test code, but and other artifacts: models, specifications. Keywords: unit testing, tests generator, model based testing 1 Introduction Software testing automation is seen as a mean for reducing software construction costs by eliminating or reducing manual software testing phase. Software tests generators are used for software testing automation. Software tests generators have to fulfill such goals: 1. Create repeatable tests. 2. Create tests for bug detection. 3. Create self-checking tests. Current unit tests generators (for example Parasoft JTest) fulfill only some of these goals. They usually generate repeatable tests, but a tester has to specify the tests oracle (a generator generates test inputs, but provides no way to verify if test execution outcomes are correct) – tests are not self-checking. Tests generators usually target only one programming language and are able to generate tests using only one specific unit testing framework. These generators usually use a source code of software under test as an input for tests generation. For example, Parasoft JTest tests generator generates tests for software which are implemented using Java and unit tests use JUnit testing framework only. In this paper, we propose a modified unit tests generator model which will allow generating a test in any programming language using any unit testing framework and will be able to use software under test implemented or modeled in any language. The remaining part of this paper is organized as follows: The tests generator architecture is presented in Chapter 3. A generator example is presented in Chapter 4. Finally conclusions and the future work are given in Section 5. 2 Related works Usually software is firstly modelled and only in the later phase its code is written. If the developers have chosen to use agile software development methodologies [1, 2] , they have to overcome another obstacle. Many of agile methodologies state that software tests have to be prepared before writing software code. But majority of tests generators uses software code as an input for test generation. These generators create test directly in the same programming language as the software implementation is. Thus, if during software design phase the implementation language is not yet chosen, tests can not be created. Also tests can not be generated by tests generator then software implementation is not yet available. Duo to the fact that test code is tightly coupled with software implementation code (uses the same programming language, and usually resides in the same compilation unit), it’s almost impossible to reuse tests between several different projects. Also the fact that the tests generator is only able to create tests for one programming language, limits developers possibility to select the best available tests generator. They have to choose the tool based not on the generated tests quality but on the supported programming language by that tool. A. Krass et al. [3] proposed a way to generate tests from UML models. They were using UML models as an input for the tests generator and have generated tests which are stored as XML documents. These XML documents could be transformed later into a unit test code using a specific unit testing framework (for example, JUnit [4] or TTCN-3). The drawback of this approach is that the tests generator abstracts the generated tests from test frameworks only, and is able to generate tests using only UML models as an input. - 201 - Other authors have proposed generators which take UML models directly [5, 6] or the program code in specific programming language [7, 8] and generate the testing code using a specific unit testing framework directly. Thus their generator can produce tests for one specific unit testing framework only. 3 Platform Independent Unit Tests Generator We are proposing a model of the unit tests generator which would be able to generate tests using any unit testing framework and will take not only software’s under test code as an input, but also can use other artifacts as an input for tests generation. Artifacts could be: UML models, OCL constraints, Business Rules. The generators model is based on Model Driven Engineering (MDE) ideas [9]. 3.1 Tests Generation Using MDE Ideas The MDE idea is based on the fact that a developer does not write the software code. He or she only models it. Models are transformed to software implementation later using any chosen programming language, platform and/or framework. MDE targets implementation code development. But its principles can be used for tests generation also. Tests could be created not only as a code using a selected unit testing framework; they can be created as tests models and transformed later into any selected programming language and unit testing framework. For example, we could model tests as UML diagrams and OCL constraints and transform the modeled tests later into the test code which uses JUnit testing framework; the code is generated in Java programming language. Such tests generator can be roughly depicted in Figure 1. Figure 1. Tests generator model The tests generator takes software under test (SUT - software under test in Figure 1) as an input and generates tests which are represented as a model. For example, the UML diagram using Testing profile stereotype could be such a model. After that the tests model is transformed into the platform specific test model. The platform specific test model represents tests as a model also, but this model uses platform specific elements. For example, test classes extend the required unit testing framework base classes, implement the required methods, sequence diagrams show calls to specific unit testing framework methods. The final phase is to transform the platform specific model into tests implementation (test code). This transformation could be no more different than the ones used today for code generation from a model (usually such transformations are used in many UML diagramming tools). The benefit of this generator is that tests can be transformed to any selected unit testing framework and implementation language. We just have to select a different transformation if we want to generate tests for another programming language or/and unit testing framework. 3.2 Software Under Test Meta-Model The tests generator does not take software under test code or model as tests generation input directly. The tests generator transforms SUT into a software model firstly (Figure 2). That model represents software independently from its implementation language or modeling language. For example, if SUT is implemented using Java programming language, the generator does reverse code engineering. If the program is modeled using OMT notation its model is converted into our model (Similar to UML). - 202 - Figure 2. SUT Meta-model Figure 2 presents a meta-model of software under test. This model is similar to UML static structure meta-model. We have extended UML meta-model by adding the elements “Constraints” into meta-model. This addition allows us to store software under test model, code, OCL constrains or any combination of them into one generic model. SUT meta-model has all information about software under test; in order to generate tests it contains information about classes, methods, fields in this software under test. If software was modeled using OCL language, the meta-model links OCL constraints to associated classes, methods, fields, method parameters. 3.3 Tests Meta-Model The tests meta-model represents the generated tests in an abstracted from implementation form. This meta-model is presented in Figure 3. Figure 3. Tests Meta-model This model stores unit tests. The meta-model has classes to store the generated input values; it references classes, methods, fields from software under test meta-model. It carries the expected test execution - 203 - values and/or references to OCL constraints which are used as a test oracle [10]. When implementing our unit tests generator we can store the generated tests in a database. And then the required tests from database can be transformed into a test code using the selected programming language and unit testing framework. Transformations into the test code can be performed using usual code generation techniques. 4 Generator Example We have an implementation of the unit tests generator based on our idea. Its source code is available at http://atf.sourceforge.net/ web page. The generator’s structure is represented in Figure 4. Business Rules 1. Transform 1. Transform Software Code OCL UML (C++, Java,..) 2. Transform2. Reverse2. Transform SUT Meta Model 3. Generate Tests Meta Model 4.Transform 4. Transform 4. Transform Tests Model C++, Tests Model C#, NUnit Tests Model JUnit, Java CppUnit 5. Generate 5. Generate 5. Generate Tests Code C# Test Code Java Tests Code C++ Figure 4. Tests generator structure Our generator is able to generate tests for software which is modeled in UML and/or has OCL constraints, requirements expressed as Business Rules (We have presented how to transform Business Rules into UML model and OCL constraints for this generator [11]).