UNIVERSAL UNIT TESTS GENERATOR BASED ON SOFTWARE MODELS

Sarunas Packevicius, Andrej Usaniov, Eduardas Bareisa Kaunas University of Technology, Department of Software Engineering, Studentų 50, Kaunas, Lithuania, [email protected], [email protected], [email protected] Abstract. Unit tests are viewed as a coding result of software developers. Unit tests are usually created by developers and implemented directly using specific language and framework. Unit test generation tools usually does the same thing – generates tests for specific language using specific unit testing framework. Thus such generator is suitable for only one situation. Another drawback of these generators – they mainly use software code as a source for generation. In this paper we are presenting tests generator model which could be able to generate unit tests for any language using any unit testing framework. It will be able to use not only software under test code, but and other artifacts: models, specifications.

Keywords: unit testing, tests generator, model based testing

1 Introduction automation is seen as a mean for reducing software construction costs by eliminating or reducing manual software testing phase. Software tests generators are used for software testing automation. Software tests generators have to fulfill such goals: 1. Create repeatable tests. 2. Create tests for bug detection. 3. Create self-checking tests. Current unit tests generators (for example JTest) fulfill only some of these goals. They usually generate repeatable tests, but a tester has to specify the tests oracle (a generator generates test inputs, but provides no way to verify if test execution outcomes are correct) – tests are not self-checking. Tests generators usually target only one programming language and are able to generate tests using only one specific unit testing framework. These generators usually use a source code of software under test as an input for tests generation. For example, Parasoft JTest tests generator generates tests for software which are implemented using and unit tests use JUnit testing framework only. In this paper, we propose a modified unit tests generator model which will allow generating a test in any programming language using any unit testing framework and will be able to use software under test implemented or modeled in any language. The remaining part of this paper is organized as follows: The tests generator architecture is presented in Chapter 3. A generator example is presented in Chapter 4. Finally conclusions and the future work are given in Section 5.

2 Related works Usually software is firstly modelled and only in the later phase its code is written. If the developers have chosen to use agile software development methodologies [1, 2] , they have to overcome another obstacle. Many of agile methodologies state that software tests have to be prepared before writing software code. But majority of tests generators uses software code as an input for test generation. These generators create test directly in the same programming language as the software implementation is. Thus, if during software design phase the implementation language is not yet chosen, tests can not be created. Also tests can not be generated by tests generator then software implementation is not yet available. Duo to the fact that test code is tightly coupled with software implementation code (uses the same programming language, and usually resides in the same compilation unit), it’s almost impossible to reuse tests between several different projects. Also the fact that the tests generator is only able to create tests for one programming language, limits developers possibility to select the best available tests generator. They have to choose the tool based not on the generated tests quality but on the supported programming language by that tool. A. Krass et al. [3] proposed a way to generate tests from UML models. They were using UML models as an input for the tests generator and have generated tests which are stored as XML documents. These XML documents could be transformed later into a unit test code using a specific unit testing framework (for example, JUnit [4] or TTCN-3). The drawback of this approach is that the tests generator abstracts the generated tests from test frameworks only, and is able to generate tests using only UML models as an input.

- 201 - Other authors have proposed generators which take UML models directly [5, 6] or the program code in specific programming language [7, 8] and generate the testing code using a specific unit testing framework directly. Thus their generator can produce tests for one specific unit testing framework only.

3 Platform Independent Unit Tests Generator We are proposing a model of the unit tests generator which would be able to generate tests using any unit testing framework and will take not only software’s under test code as an input, but also can use other artifacts as an input for tests generation. Artifacts could be: UML models, OCL constraints, Business Rules. The generators model is based on Model Driven Engineering (MDE) ideas [9].

3.1 Tests Generation Using MDE Ideas The MDE idea is based on the fact that a developer does not write the software code. He or she only models it. Models are transformed to software implementation later using any chosen programming language, platform and/or framework. MDE targets implementation code development. But its principles can be used for tests generation also. Tests could be created not only as a code using a selected unit testing framework; they can be created as tests models and transformed later into any selected programming language and unit testing framework. For example, we could model tests as UML diagrams and OCL constraints and transform the modeled tests later into the test code which uses JUnit testing framework; the code is generated in Java programming language. Such tests generator can be roughly depicted in Figure 1.

Figure 1. Tests generator model The tests generator takes software under test (SUT - software under test in Figure 1) as an input and generates tests which are represented as a model. For example, the UML diagram using Testing profile stereotype could be such a model. After that the tests model is transformed into the platform specific test model. The platform specific test model represents tests as a model also, but this model uses platform specific elements. For example, test classes extend the required unit testing framework base classes, implement the required methods, sequence diagrams show calls to specific unit testing framework methods. The final phase is to transform the platform specific model into tests implementation (test code). This transformation could be no more different than the ones used today for code generation from a model (usually such transformations are used in many UML diagramming tools). The benefit of this generator is that tests can be transformed to any selected unit testing framework and implementation language. We just have to select a different transformation if we want to generate tests for another programming language or/and unit testing framework.

3.2 Software Under Test Meta-Model The tests generator does not take software under test code or model as tests generation input directly. The tests generator transforms SUT into a software model firstly (Figure 2). That model represents software independently from its implementation language or modeling language. For example, if SUT is implemented using Java programming language, the generator does reverse code engineering. If the program is modeled using OMT notation its model is converted into our model (Similar to UML).

- 202 -

Figure 2. SUT Meta-model Figure 2 presents a meta-model of software under test. This model is similar to UML static structure meta-model. We have extended UML meta-model by adding the elements “Constraints” into meta-model. This addition allows us to store software under test model, code, OCL constrains or any combination of them into one generic model. SUT meta-model has all information about software under test; in order to generate tests it contains information about classes, methods, fields in this software under test. If software was modeled using OCL language, the meta-model links OCL constraints to associated classes, methods, fields, method parameters.

3.3 Tests Meta-Model The tests meta-model represents the generated tests in an abstracted from implementation form. This meta-model is presented in Figure 3.

Figure 3. Tests Meta-model

This model stores unit tests. The meta-model has classes to store the generated input values; it references classes, methods, fields from software under test meta-model. It carries the expected test execution

- 203 - values and/or references to OCL constraints which are used as a test oracle [10]. When implementing our unit tests generator we can store the generated tests in a database. And then the required tests from database can be transformed into a test code using the selected programming language and unit testing framework. Transformations into the test code can be performed using usual code generation techniques.

4 Generator Example We have an implementation of the unit tests generator based on our idea. Its source code is available at http://atf.sourceforge.net/ web page. The generator’s structure is represented in Figure 4.

Business Rules

1. Transform 1. Transform

Software Code OCL UML (C++, Java,..)

2. Transform2. Reverse2. Transform

SUT Meta Model

3. Generate

Tests Meta Model

4.Transform 4. Transform 4. Transform

Tests Model C++, Tests Model C#, NUnit Tests Model JUnit, Java CppUnit

5. Generate 5. Generate 5. Generate

Tests Code C# Test Code Java Tests Code C++

Figure 4. Tests generator structure

Our generator is able to generate tests for software which is modeled in UML and/or has OCL constraints, requirements expressed as Business Rules (We have presented how to transform Business Rules into UML model and OCL constraints for this generator [11]). The generator is composed of three main parts: 1. The meta-model extractor. 2. The tests generator. 3. The tests transformer. It also has an additional part: tests runner. The model extractor extracts the software under test meta- model from various sources: implementation code and/or UML, OCL models and stores it in its meta-model data structures. The tests generator takes the data stored in the meta-model structures and generates test cases. Test cases are stored in an abstract test meta-model. The tests transformer takes the tests stored in test meta-model and transforms it into an actual test code for a specified programming language and unit testing framework. The tests runner is than able to execute the generated tests. The generator is able to extract software model and store it into its internal meta-model from java code. It can also extract model from UML diagrams and/or OCL constraints and store it into its internal meta-model. The model extractor is designed by using factory design pattern [12] what allows to easily to integrate another additional source of model to be extracted from. The generator also uses the factory design pattern for test data generation algorithm. It defines the interface which has to be implemented in order to use the selected test data generation algorithm. The tests stored in the tests model data structures are transformed into actual test code by the tests transformer component. This component is also implemented using a factory design pattern. The tests transformer defines an interface which takes tests stored in a tests model data structures and generates unit tests code stored in code files. Using the defined interface the test code transformer can be implemented for creating tests in any programming language using any unit testing framework. The current implementation is able to transform tests from tests model to test code implemented in java programming language. The transformed unit tests code uses JUnit 4.0 [4] testing framework. The generated test code can finally be executed by tests runner and the actual unit testing can be

- 204 - performed. The tests executor module invokes a unit testing framework (JUnit testing framework) for generated unit tests execution. The tests executor invokes a java compiler to compile test classes into a binary code and after the compilation it invokes the JUnit tests runner for the generated tests execution. The test runner is implemented using a factory design pattern. The class implementing this interface has to execute all test files stored in the specified folder. The tests runner transfers execution to the JUnit testing framework main class (org.junit.TestsRunner). The JUnit testing framework itself enumerates generated tests and executes them. The tests generator tool has been implemented as an [13] plug-in. By being an eclipse plug-in it can be easily integrated into various integrated development environments which are based on eclipse platform. Some of these environments are: IBM Rational Software Architect [14], Oracle WebLogic development environment. Then this tool is integrated into IBM Rational Software Architect, it can easily access software under test model using the libraries provided by the integrated development environment. In this case exporting and parsing XMI file can be omitted. The tests generator seamlessly integrates into an eclipse development application. Tests can be generated by a single mouse clip. Then the software under test is availabe as only a software model test generator performs the following steps: 1. Transform the software model into its SUT model. (Step 2) 2. Generate tests using the software SUT model and store tests as a tests model. (Step 3) 3. Generate the test code for the selected unit testing framework using the tests model. (Steps 4, 5) If software under test is represented as a code, the test generating procedure is such: 1. Reverse the code into the SUT model. (Step 2) 2. Generate tests using the software SUT model and store the tests as a tests model. (Step 3) 3. Generate the test code for the selected unit testing framework using the tests model. (Steps 4, 5) If we have software specified as Business Rules, the tests generation procedure is: 1. Transform Business Rules into UML, OCL models. (Step 1) 2. Transform the software model into its SUT model. (Step 2) 3. Generate tests using the software SUT model and store the tests as a tests model. (Step 3) 4. Generate the test code for the selected testing framework using the tests model. (Steps 4, 5)

The generator has to generate tests using always the same type of data and has to produce the same type of results (Step 3). All we have to do is just to implement different transformations, if we want to target another unit testing framework or we want to generate tests for programs which are implemented in other programming languages or modeled using other modeling languages.

5 Conclusion The suggested abstract unit tests generator is able to generate tests for software implemented in any programming language and/or modeled using any modeling language. It can generate the test code in any programming language using any unit testing framework. The tests generator mechanism is independent from the unit testing framework and software under test implementation or modeling language. This generator model can be used for benchmarking tests generators when we have benchmarks implemented in various programming languages.

References [1] Highsmith J., Cockburn A. Agile software development: the business of innovation. Computer, 2001. 34(9): p. 120. [2] Meszaros, G. Agile using record & playback. In Companion of the 18th annual ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications. 2003, ACM: Anaheim, CA, USA. [3] Kraas, A. et al. A Generic toolchain for model-based test generation and selection. In TESTCOM / FATES 2007. Tallinn, Estonia. [4] Louridas P. JUnit: unit testing and coding in tandem. Software, IEEE, 2005. 22(4): p. 12-15. [5] Kim S.K., Wildman L., Duke R. A UML approach to the generation of test sequences for Java-based concurrent systems. In 2005 Australian Software Engineering Conference (ASWEC'05) 2005.

- 205 - [6] Kim Y.G. et al. Test cases generation from UML state diagrams. IEE Proceedings on Software Engineering, 1999. 146(4): p. 187-192. [7] Boyapati C., Khurshid S., Marinov D. Korat: automated testing based on Java predicates. In Proceedings of the 2002 ACM SIGSOFT international symposium on Software testing and analysis. 2002, ACM Press: Roma, Italy. [8] Visser W., Pasareanu C.S., Khurshid S. Test input generation with java PathFinder. In Proceedings of the 2004 ACM SIGSOFT international symposium on Software testing and analysis. 2004, ACM Press: Boston, Massachusetts, USA. [9] Schmidt D.C. Guest Editor's Introduction: Model-Driven Engineering. Computer, 2006. 39(2): p. 25-31. [10 Packevičius Š., Ušaniov A., Bareiša E. Using Models Constraints as Imprecise Software Test Oracles. Information Technology and Control, 2007. [11] Packevičius, Š., Ušaniov A., Bareiša E. Creating unit tests using business rules. In Information Technologies' 2008 : 14th International Conference on Information and Software Technologies. 2008. Kaunas, Lithuania. [12] Gamma E. et al. Design patterns : elements of reusable object-oriented software. 2003, Boston: Addison- Wesley professional computing series, 395. [13] Carlson D. Eclipse Distilled. 2005: Addison-Wesley Professional. [14] Quatrani T., Palistrant J. Visual Modeling with IBM Rational Software Architect and UML. 2006: IBM Press.

- 206 -