Practical Approach to Developing an Automation Testing Tool
Total Page：16
File Type：pdf, Size：1020Kb
Linköping University | Department of Computer and Information Science Bachelor thesis, 16 ECTS | Computer Science 2018 | LIU-IDA/LITH-EX-G--18/033--SE Practical Approach to Developing an Automation Testing Tool Hannes Persson and Povel Ståhlberg Supervisor : Ivan Ukhov Examiner : Ahmed Rezine Linköping University SE–581 83 Linköping +46 13 28 10 00 , www.liu.se Upphovsrätt Detta dokument hålls tillgängligt på Internet – eller dess framtida ersättare – under 25 år från publiceringsdatum under förutsättning att inga extraordinära omständigheter uppstår. Tillgång till dokumentet innebär tillstånd för var och en att läsa, ladda ner, skriva ut enstaka kopior för enskilt bruk och att använda det oförändrat för ickekommersiell forskning och för undervisning. Överföring av upphovsrätten vid en senare tidpunkt kan inte upphäva detta tillstånd. All annan användning av dokumentet kräver upphovsmannens medgivande. För att garantera äktheten, säkerheten och tillgängligheten finns lösningar av teknisk och admin- istrativ art. Upphovsmannens ideella rätt innefattar rätt att bli nämnd som upphovsman i den omfattning som god sed kräver vid användning av dokumentet på ovan beskrivna sätt samt skydd mot att dokumentet ändras eller presenteras i sådan form eller i sådant sam- manhang som är kränkande för upphovsmannenslitterära eller konstnärliga anseende eller egenart. För ytterligare information om Linköping University Electronic Press se förlagets hemsida http://www.ep.liu.se/. Copyright The publishers will keep this document online on the Internet – or its possible replacement – for a period of 25 years starting from the date of publication barring exceptional circum- stances. The online availability of the document implies permanent permission for anyone to read, to download, or to print out single copies for his/hers own use and to use it unchanged for non-commercial research and educational purpose. Subsequent transfers of copyright cannot revoke this permission. All other uses of the document are conditional upon the con- sent of the copyright owner. The publisher has taken technical and administrative measures to assure authenticity, security and accessibility. According to intellectual property law the author has the right to be mentioned when his/her work is accessed as described above and to be protected against infringement. For additional information about the Linköping Uni- versity Electronic Press and its procedures for publication and for assurance of document integrity, please refer to its www home page: http://www.ep.liu.se/. c 2017 Hannes Persson and Povel Ståhlberg Abstract Manually verifying a software under development can be time consuming because of its complexity, but also because of frequent updates to different parts of the system. As the software grows larger, a way of verifying the software automatically without user-interaction is a good approach. Verifying a software systematically and automatically will both save time for the developers; and assure that the updated version functions as before. This report will present a starting point for automatic testing. This is done in cooperation with XperDi, a company developing a plug-in for CAD-software that currently verifies their functionality manually. This was achieved by developing a testing tool that support communication between Windows applications used by the plug-in; this was needed to automate the testing process. The reached conclusions during this thesis are promising as a starting point for XperDi; to move from manually to automatic verification. There are however several improvements that this report presents for further development of testing tool. Keywords: Software Testing, Box Approach Testing, Testing Tool, VB.NET Acknowledgments We would like to thank Albin Mannerfelt and Manokar Munisamy at XperDi in Linköping. We also want to thank Ahmed Rezine at IDA, institution of computer science at Linköping University, for being our examiner. A special thank to Creative Science Park for providing a working environment. iv Contents 1 Introduction 2 1.1 Background . 2 1.2 Motivation . 2 1.3 Purpose . 3 1.4 Research questions . 3 1.5 Thesis outline . 3 2 Theory 4 2.1 Development Environment . 4 2.1.1 VB.NET . 4 2.1.2 Windows API . 4 2.1.3 Component Object Model . 5 2.2 CAD configurator . 5 2.2.1 SQLite . 5 2.3 Continuous Integration . 5 2.4 Documentation and Testing . 5 2.4.1 Log4Net . 6 2.4.2 Software Testing . 6 2.4.3 Pairwise Testing . 9 3 Method 10 3.1 Pre-studies of the CAD Configurator . 10 3.1.1 Parts . 11 3.1.2 Features . 11 3.2 Testing strategy . 12 3.2.1 Execution time . 12 3.3 Test Structure . 12 3.3.1 Permutations . 12 3.3.2 Pairwise Test Structure . 12 3.4 Box Testing Approach . 13 3.5 Implementation . 13 3.5.1 Automated Testing Tool . 13 3.5.2 Testing Features . 17 3.5.3 Logging with Log4Net . 23 4 Results 24 4.1 Testing Tool Implementation and Tests . 24 4.1.1 Tests . 24 4.1.2 Testing Tool . 24 5 Discussion 26 5.1 Results . 26 v CONTENTS 5.2 Method . 26 5.3 Maintenance and future improvements . 27 6 Conclusion 29 6.1 Future Work . 29 Bibliography 30 1 1 Introduction Software development is a challenging process. It is hard to predict how the user will operate applications. Therefore, the developer needs a systematic way to gain information about po- tential issues before the application is deployed. By using testing methods, the developer can minimise potential errors and therefore improve the software quality. This makes testing a fundamental part of the application development cycle. Having a test tool allows one to cap- ture the desired behaviour and improve with confidence later on. This thesis will investigate and evaluate how an automatic test system can be designed and implemented for a system that assists with 3D-modeling. 1.1 Background XperDi is a company located in Linköping, Sweden that provides a software where when used can achieve a more effective 3D-modeling prototyping process. This is provided by XperDi’s own add-on that functions together with a number of 3D-modeling software. This application is used by drafters in order to design complex 3D models in a straightforward manner. Usually this process is a tedious and time-consuming practice, especially at later stages of a design where small changes can have a large impact on related parts. The appli- cation developed by XperDi assist in the assembly and alteration process at the design stage of a 3D model in order to ensure that time consumption is kept to a minimum. 1.2 Motivation Today, when the developers at XperDi provides new functionality, the system is evaluated and tested manually to verify that the application is working properly. This is not a testing process that scales well with an increasing set of functionality, and it necessitates redundant work. Because of this there is no systematic way to gain confidence that the updated func- tionality and other related parts of the system works as intended. In order to avoid manual testing and redundant work, automatic testing systems could provide a systematic approach to prevent bugs in deployed code. 2 1.3. Purpose 1.3 Purpose The goal of this thesis is to develop an automated testing prototype that exercises a defined set of the application functionality. The prototype that is to be developed has the following set of requirements to follow: • The test tool shall conduct fully automatic testing without supervision and interaction. • The test tool shall handle all types of errors the test might throw without breaking the queued test execution. • The test tool shall be able to detect communication errors and faultily waiting states. • The test tool shall have the possibility to be integrated in a continuous integration envi- ronment. • The test tool shall output information regarding the test execution to a log. • The test tool shall function on a Windows 10 system. By analysing a range of testing and automation types, a suitable testing prototype is to be de- veloped in order to test the functions and provide an entry point for an automatic integrated testing system. 1.4 Research questions In this thesis, we are set out to investigate the following questions: • Is it possible to apply system automation testing on the application? • Can automated testing be conducted with an existing testing framework? The first research question aims to investigate if it is possible to implement a set of tests to validate each of the features users can access from the application. The second research question aims to answer if there is an existing testing framework that can be used for the purpose of testing the applications features correctly. Even though there exists many testing frameworks it might be a possibility that none satisfies our needs, one reason to this could be user-interaction between windows applications that should be automated. This particular communication is between the application and the 3D-modeling software CATIA. 1.5 Thesis outline In chapter 2 we explain the relevant concepts of software testing and tools of importance when designing a testing tool. In chapter 3 we describe how the application first were analysed in order to get a better understanding of how each part in the system correlates with the other parts. We will also present the development of the automated testing system. In chapter 4 we present the testing prototype and the results from the test suite. In chapter 5 a discussion will be conducted where the system is evaluated in a larger context, design decisions are analysed and re-evaluated. Chapter 6 will present the conclusions of this thesis together with suggestions for im- provements in future work. 3 2 Theory This chapter will go through the tools and techniques that will be of importance for this thesis.