A Lightweight Test-Driven Approach to Test Indie Software Products Yong Lai, and Hassan Reza School of Aerospace Sciences Department of Computer Science University of North Dakota Grand Forks, ND 58201, USA [email protected] Abstract In this work, we propose a lightweight testing method In this work, we discuss the application of testing and guidelines for indie software developers aiming to method to a typical independent (or indie) software improve the software quality, simplify the testing process, development. Toward this goal, an experimental indie and reducing the cost of testing. Toward this goal, we are software ShoppingList is developed and tested using the applying our proposed method to a typical indie software proposed techniques. Some testing frameworks and tools application to examine its feasibility. are also utilized to facilitate the process. Based on our In what follows, we first describe the test-driven results we concluded that the test-driven development development method [1]. This discussion will assist us to does have a positive effect on reducing the cost of indie effectively incorporate the methodology into our proposed software development.. software testing approach. Next, our lightweight testing management process is discussed. We then discuss our decision on programming language, testing framework Key Words: Test-driven Development, Extreme and tool support. A model-view-controller [3] testing Programming (XP), Lightweight testing, JUnit, model is developed to depict the feasibility of our Jemmy, Indie Software Development and Testing. proposed testing process. More specifically, the feasibility of our method is examined using a simple indie software application known as ShoppingList that manages a list of 1. Introduction groceries the user wants to shop. Comparing to well-established software testing 2. Background and Related Works method (e.g., white-box, black-box, etc.) used in the traditional software development methodologies (e.g., Test-driven development (TDD) refers to a testing waterfall model, spiral model, etc.) indie software testing method that requires testing a requirement (or code) first is a new concept and it is at its primitive stage. and implementing that requirement (or code) when the Informally, the indie software development refers to the test fails [1]. More specifically, TDD is an evolutionary independent development of software systems by very and specification development method in which writing small team of talented but market savvy software code and testing the corresponding code are carried out in developers having limited human and financial resources. interleaved fashion [1]. The approach originally Examples of indie software developers include introduced as part of agile development such as Extreme independent game and mobile applications (or mobile Programming [4] that often works with automated apps) developers. software testing such as Junit [10]. Test-driven software The front-end and back-end developments of these development process heavily relies on the repetition of a types of software require agilities and speed due to market very short development cycle: first developer writes a short windows of opportunities. Traditional heavy failing test case describing the new functionality, then weighted and plan driven software development process s/he writes the code to pass that test, and finally, s/he such as waterfall model are very expensive and hence are refactors (or improve) the new code according to ill-suited for development of these types of systems for acceptable standards [1,4]. Testing-driven development the following reasons: demands complete, consistent, and (TDD) is especially efficient in solving the prevailing correct requirements, testing process takes place only problem by indie software developers consisting of small after the software has been written, lack of systematic test number of talented programmers with limited resources. case design and testing plan, combined debugging and A test-driven development cycle includes the testing approach, etc [4]. following steps [4]: 1) Identify new functionality and generating the the following steps: 1) test planning, 2) test design, 3) test corresponding test cases. setup, 4) test execution, 5) test result analysis, and 6) test 2) Write the code: Initially, the test case will fail and reporting. In our experiment, we simplified the task of corresponding code must be written and implemented each stage to create a light-weight testing management to pass subsequent test case. process. To further simplify the testing process, we have 3) Refactor the code: After all test cases passed integrated the elements of lightweight testing process with successfully, we need to clean up the code along with the steps of test-driven development (TDD) as follows: the test cases as necessary. 1) Test planning and test design. 4) Repeat: The previous three steps are repeated to push These two stages are integrated in the first step of forward the whole development process until the test-driven development. Different from the test whole program is written. planning and test design stages in traditional testing management process, which usually requires the Figure 1 shows a typical test-driven development cycle. requirements to be complete into consideration. In addition, test planning and design at each cycle of test-driven development focuses on a small chunk of requirements. Since the test planning and design only consider a small part of the functionality of the software, a detailed test schedule is minimized or becomes unnecessary. Also, because of the small scale of indie software, other steps such as deciding test strategy, test requirements and items can be skipped in order to keep the testing management process simple and agile. 2) Test setup and test execution. For the test-driven indie software development, test setup and execution are related to writing product code to pass the newly added test case. Again, because of the small scale of indie software, test setup is trivial. After the product code is written, the new test cases and old test cases Figure 1. Test-driven development cycle. all need to be executed to ensure the new code doesn’t introduce errors (regression testing). With the advancement of the development, test execution will 3. A light-weight testing method be a tedious task as the number of test cases keeps growing. Normally, automated testing tools [10] are The well managed testing processes such as waterfall used to ease the task of test execution. The test models have been proven to be very effective but very entrance criterion is easy to define: when the new test expensive in the development of commercial software cases are written. So as the test exit criterion: the new product because the approach demands complete, product code passes both the new test cases and old consistent, and correct requirements before the validation test cases. and verification (V&V) begin [4]. The testing (static and 3) Test result analysis and reporting. The task of this dynamic) activities usually are normally performed by stage is to write test report, analyze defects, make independent testing teams aiming at showing both correct changes, and prevent defects from reoccurring. In and incorrect behaviors of the systems under test. Because test-driven indie software development, this stage of the cost and human resource attributed to these should happen before or after the third step. The most approaches, it is impractical to directly apply them in the important task for indie developers at this stage is to indie software development. Therefore, it makes sense to document the test case design and test results adopt a lightweight testing process that shall be cheaper including the effectiveness of test cases, whether new and yet effective for indie software developers with product code passes the old test cases, the choice of limited resources. test data, etc. The output of this stage is very A lightweight methodology shall be a flexible important as well since it provides insightful development process using a handful of rules to manage guidance for future software development. the unstable requirements and environments. Examples of light-weight approach include adaptive software A modified test-driven development incorporating a development [5] and extreme programming (XP) [6], light-weight testing management process is shown in feature driven development [7], etc. figure 2. In general, the testing management process includes 4. Testing tools and techniques popularity of Android platform development and mobile apps. Besides testing methodology, testing tools also play an important role in the indie software development. 4.2 Unit testing framework Appropriate choice of testing tools can greatly enhance the productivity of indie developers because they can We choose JUnit [10] as our testing framework. JUnit is a minimize the test cost by providing sophisticated testing unit and regression testing framework; it is commonly frameworks and saving programmers’ effort in creating, considered as de facto tool for test-driven development organizing, and executing test cases. For different that works with java language. The key justifications for programming languages and platforms, there are many using Junit include providing a complete assertion tools to different testing tools available for indie developers to validate actual result against expected results; generating choose. Some of them are open source, while