Monika Sharma et al, / (IJCSIT) International Journal of Computer Science and Information Technologies, Vol. 5 (1) , 2014, 908-912

Web based Automation Testing and Tools

Monika Sharma, Rigzin Angmo Department of Information Technology, University Institute of Engineering & Technology, Panjab University Chandigarh (U.T.), India

Abstract— testing is an essential phase of software Method development life cycle. Today many software applications are written as web based application that runs in an Internet Browser. The economic relevance of web based application increases the importance of controlling and improving its Static & Dynamic Manual & White-box, quality. The quality assurance of a system depends upon Testing Automation Black- box & automation testing that decreases the test cost and increases testing Gray-box testing work efficiency. In this paper we have discussed various web automation testing tools which will help us to understand the Fig. 1.1(a) automation testing as well as the tools available for automation A. Static and Dynamic Testing testing. A variety of web based systems and applications are tested by automation testing tools. The automation testing This type of testing method can be illustrated with the help script is used in test automation. To choose the best tool for a of difference (TABLE I below):- task, various issues like ease of integration should be TABLE I. Static vs. Dynamic Testing S. considered and weighed against the cost and performance. Static Testing Dynamic testing Also the tool needs to be compatible with the design and No. Static Testing is done without Dynamic Testing is done by implementation of an application. 1. executing the program executing the program It is performed before It is performed after 2. Keywords—Web automation testing, automation testing tools, compilation compilation web application, software testing type, software quality. Static testing is about Dynamic testing is about 3. prevention of defects finding and fixing the defects 4. It s less expensive It is more expensive NTRODUCTION I. I This testing requires loads of This testing comparatively 5. Software testing is one of the most important and crucial meetings requires lesser meetings phase in the software development life cycle process, Static Testing is a verification 6. It involves validation process consuming an average of 40% to 70% of software process Static testing involves checklist Dynamic testing involves test development process [1]. Software testing is a method, 7. and process to be followed cases for execution which is used for assessing the functionality of a software B. The Box Approach: program. Software testing is also a process through which Traditionally software testing is divided into white- and software item is evaluated to detect the defects and then black-box testing. These two approaches are used to correct them. It is also used to assess the feature of a describe the point of view that a test engineer takes when software system and as well as the quality of the product. designing test cases. The software testing should be done during the development

process. In other words software testing is stated as TABLE II. Compares various features of black Box Testing And white verification and validation process that a computer box Testing S. program/application/product should:- Black Box Testing White Box Testing  Meets the requirement that is used in software No. Also known as closed box Also known as clear box testing, designing and development. testing, data driven 1. structural testing or code based  Can be implemented with the same characteristics. testing and functional testing  Works as expected and satisfies the needs of testing stakeholders. The Internal Workings of In this testing Tester has full 2. an application are not knowledge of the Internal workings Software Testing is very important because software bugs required to be known of the application can potentially cause monetary as well as human losses. In Testing is performed by Normally done by testers and 3. end users, testers and simple words, software testing is an activity to check developers whether the actual results match the expected results and to developers Testing is based on Internal workings are fully known ensure that the software system is defect free. external expectations as 4. and the tester can design test data Internal behaviour of the accordingly II. TYPE OF TESTING application is unknown Not suited to algorithm The software testing is a very wide topic. It includes varies 5. Suited for algorithm testing methods, types and levels or stages of testing. The basic testing This is the least time This type of testing is the most testing methods are elaborated below (in fig. 1.1(a)):- 6. consuming and exhaustive and time consuming  Static testing and Dynamic testing exhaustive.  The box approach method This testing can only be Data domains and Internal 7. done by trial and error boundaries can be better tested  Manual and Automation Testing method with the help of this testing

www.ijcsit.com 908 Monika Sharma et al, / (IJCSIT) International Journal of Computer Science and Information Technologies, Vol. 5 (1) , 2014, 908-912

Also there is another box approach is available known as industry. In 40’s Programmers used to test the software and Grey Box Testing. This type of testing is also known as not the testers and testing was often not distinguished from translucent testing as the tester has limited knowledge of debugging as well. Throughout the sixties, papers about inside the application. It has the mixed features of white testing, such as the proceedings of the IFIPS conferences, and black box testing approach. The testing is performed by almost assumed that programmers tested the software they end user, testers and developers. Testing is done on the built. As larger and more complex systems arose, the idea basis of high level data diagrams and data flow diagrams. It of software testers came into trend. The first conference on is partly time consuming and exhaustive. testing software was organized in 1972 at Chapel Hill where the proceedings of the conference presented that the C. Manual And Automated Testing discipline of testing is distinct from programming. The : In manual testing type the software are automation of technology allows us to share information tested manually i.e. without using any tool. limitlessly. Several researchers used the web automation Automation testing: It is also known as test automation. In testing tools in their research work. The researchers also this the tester writes scripts and uses another software or enhanced some of tool for better performance and results. tool to test the software. It is basically a process that These tools are applied independently as well as combined involves automation of a manual process. The difference to solve the challenges in software testing. Here, we discuss between manual and automation testing can be clearly work done on web automation testing with the use of specified with help of following table III: automation testing tools. Leott et al. [2] presents an Industrial case study about test TABLE III. Compares Manual and Automation Testing automation & test suite maintenance in context of Learning S. No. Manual Testing Automation Testing Content management System. In this author proposed The execution speed of Manual testing is slow as realignment effort, which conclude that ID based method Automation testing executes test is executed by human for locating web page elements that is better than Xpath 1. test cases significantly faster resources so it is Time than human resources. methods. The goal of author work is to understand which consuming and tedious as Selenium WebDriver method is suitable for locating web well Test cases are executed by page element which reduces the maintenance effort needed using automation tool so less to realign the test cases to a new release. Automation testing executes testers are required in 2. test cases significantly faster Leott et al. [3] also presents another report on an industrial automation testing. As a than human resources. case study which is done to investigate the potential benefit result Less investment is required in human resources of adopting the page object pattern to improve In manual testing no Automation testing maintainability of selenium webDriver test cases. The paper programming can be done to programmable in this testers compares the two equivalent test suits, one is built using the 3. write complicated tests can program sophisticated page object pattern and one is built without it. The results which fetch the hidden tests to bring out hidden will indicate a strong reduction of time and number of information. information. Manual testing is less modified LOCs to repair the test suite when the page object reliable as tests may not be pattern is used. Automation testing is more 4. performed with precision reliable and less error prone Cheluvaraju et al. [4] proposed a novel method to each time because of human investigating relationship between the files that are errors. committed together by applying advance SNA (social

network Analysis) to a network. In this paper they There are many others type of software testing are available, performed the empirical analyses on revision histories of in fact any type of software testing type can be executed [1] well known open source web application testing system, both manually as well using an automation tool . The other selenium and results parameters like extracting cross – software testing types under above two testing methods are: language change dependencies, change propagation, impact , , , Load analysis are reported. testing etc. As illustrated in fig. 1.1(b) below: Wong et al. [5] presented an approach to mining an

executable specification from given Selenium IDE test suite. Manual Testing Automation Testing There approach evaluate by number of test suits for real

world system. There work also demonstrates integration of

the two opposite processes of model mining and model

based test generation in one framework.

Nagowah et al. [6] proposed a design of an automated

Unit testing,System Testing, testing tool, Kishanium tool by analyzing the existing tools.

User , Kishanium tool is an innovative tool with improvement on

Web Testing,Etc. the record and playback approach. The tool manages the set

of test data and also reuses existing test data even if there Figure 1.1(b) are number of changes in the user interface. The tool

Fig. 1.1(b) provides additional features like Data generator, spylink, III. EVOLUTION OF WEB AUTOMATION TESTING TOOL Snapshot. The test automation is not at all new but the automation Wang et al. [7] introduced a new automation framework testing has been around the day one of the computing integrated by selenium and Jmeter. This automation

www.ijcsit.com 909 Monika Sharma et al, / (IJCSIT) International Journal of Computer Science and Information Technologies, Vol. 5 (1) , 2014, 908-912 framework shares the test steps and test data which is and an open source automated testing suite that is built for convenient to switch in various types of testing for web web applications supporting different browser and application. It supports multiple browser and operating application. Selenium was developed by Jason Huggins in System. With use of this software framework one can 2004 who working in ThroughtWorks which is a privately efficiently improve the extensibility and reusability of owned global software delivery and products company. automation test. Ever since then Selenium has been making a tremendous improvement and has come out as a strong tool for web IV. WEB AUTOMATION TESTING AND TOOLS based automation testing. A. Web testing 2.) HP –QTP: QTP is the part of HP quality centre tool 2 Web testing is the name given to the software testing that suite . It provides regression and functional testing focuses entirely on web applications. This helps to cut automation for major software environment and down the cost, minimize the efforts required to test web applications. It was originally written by Mercury 4 applications as well as web sites, increase software quality, Interactive , which was acquired by HP (Hewlett Packard) reduces time-to-market and uses reusable test cases. There in 2006. are various web testing available like: Functionality 3.) FitNesse: Fitnesse is an automation testing tool to write, Testing, , Localized Testing, Stress or organize and execute table based tests. Fitnesse is wiki Testing, , Web Services Functional Testing, wrapper over the fit sever that provides a web interface to Web Services Performance Testing, etc. the test suite. It is based on Ward Cunningham’s B. Web automation testing Framework for Integrated Test. FitNesse is a tool for 3 In the fast changing and highly competitive web-based enhancing collaboration in software development . Fitnesse business environment, it is critical for organizations to test enables customers, testers, and programmers to discover their web sites and web applications using a manual testing what their software should do, and to automatically tool. So an automated web testing should ensure that the compare that to what it actually does. Fitnesse compares web applications/web sites/web services usual functionality customers' expectation to actual result. It is also compatible works correctly, provides the ability to reuse and extend the with Junit. tests across multiple browsers / platforms/ languages / 4.) Watir: Watir is pronounced as water. It is acronym for databases / servers and ensure that all the users accessing web application testing in ruby. Watir is an open source the web applications get results in an acceptable time. family that uses ruby libraries for automation web 4 C. Features of web automation testing:- browsers . Watir allows testers to write tests that are easy to  Automated Software Testing Saves Time and read and maintain. Watir is simple and flexible too. Money 5.) TestComplete: TestComplete is an automated testing  Testing Improves Accuracy tool which allows testers to create, manage and run tests for any Windows, Web or Rich Client software. Testcomplete  Increase Test Coverage 5  Automation Does What Manual Testing Cannot is developed by SmartBear Software . The aim of this automated tool is to create software quality tests.  Automated QA Testing Helps Developers and 6.) LoadRunner: HP LoadRunner is an industry standard Testers based an automated performance and test automation  Team Moral Improves product from HP for load testing of application: that D. Web automation testing tool examines system performance and behaviour. HP To choose the best tool for the task, there are a lot of things LoadRunner works by using the virtual users. It also that need to be considered. First thing is the ease of simulates thousands of concurrent users to put the integration and should be weighed against the cost and application through various real life user loads and analyses performance i.e. how it performs in the environment with the results in detail to discover the particular behaviour. the network traffic and hardware. Also the tool has to be 7.) TestNg: TestNG refers to “Testing, the Next compatible with the design and implementation of Generation”, is a testing framework that is inspired from application. There is different type of tool available in the JUnit6 and NUnit7. It has also introduced some new market. We will discuss here at least 10 best web functionality that makes it more powerful and easier for automation testing tool and their purpose ( TABLE IV):- testers.  Selenium 8.) TOSCA: Tosca tool is a testsuite for the automated  HP –QTP execution of functional and regression software testing.  FitNesse TOSCA testsuite includes integrated test management,  Watir design, execution and data generation toolset for functional  testComplete and regression tests.  LoadRunner 9.) SilkTest: SilkTest is an automation tool that is  TestNg specifically designed for regression and function testing.  TOSCA Originally SilkTest is developed by Segue Software Inc.  SilkTest which was acquired by Borland in 2006 and later Borland  WinRunner was acquired by Micro Focus International in 2009. The 1.) Selenium: Selenium is not just a single tool but a suite SilkTest provides the flexible and robust test scripting that comprises of 4 tools1: selenium IDE, selenium RC, language that is a built in recovery system for unattended selenium Webdrive, selenium grid. Selenium is a portable testing, and silkTest has the ability to test across multiple

www.ijcsit.com 910 Monika Sharma et al, / (IJCSIT) International Journal of Computer Science and Information Technologies, Vol. 5 (1) , 2014, 908-912 platforms, browsers and technologies. Silk Test offers test capture, verified and replayed UI interaction as test script. planning, validation, management, and direct database WinRunner is originally written by Mercury Interactive. access. 10.) WinRunner: HP WinRunner software is an automation Functional GUI testing tool. This tool allowed User to

TABLE IV. Web Automation Testing Tools Stable / Name of Lan- Operat- initial Develo- S. Language Browser tool guage ing Type release / Lice-nse pment Developer no supported supports Use system development status year Jason Software huggins & testing later joined Domain Cross - 2.33/May 22, All major 1 Selenium Java framework Apache 2.0 Active other specific platform 2013, (2004) browser for web programmer language application and tester of thoughtwork Originally Software only for testing 4.0/ Bret Internet Cross - Java, .NET, 2 Watir Ruby framework September BSD Active Pettichard & Explorer, later platform c# for web 30, 2012 Paul Rogers supports application multiply browser IE 6,7,8,10, 11.0/ HP Originally by Firefox 3.0 acquire by Mercury and later, QTP Test Mercury Microsoft Interactive 11.51 provides 3 HP-QTP VB Script Auto - Interactive in Pro-prietary Active VBscript Windows and later by reply support mation 2006, (now HP Software for Google 11.51 & Division chrome 11.52) browser. 9.31/ September VBscript, Test Auto- 3,13; initial Jscript, C++, IE, Firefox, Test Microsoft SmartBear 4 Java mation release in Proprietary Active Delphi Google Complete windows Software In. Tool 1999 by Script, chrome automated c#Script. QA Test Auto- Rober C, C++, Python, Cross - V20130530/ Platform 5 FitNesse Java mation CPL Active Martin & Ruby, platform May 30, 2013 independent Tool other Delphi, c# etc Microsoft windows Hewlett- VB, Load 11.52/ first HP Load & Linux Packard, HP VBscript, 6 C Testing vesion in Pro-prietary Active Any browser Runner (load Software java, Tool 1989 generator Division javaScript, c# only) All versions Java, also are stable, Window, include more Testing current IE, FireFox, 7 TestNG Java Linux, Apache 2.0 Active Cedric Beust object- Framework version-6.8.1, chrome MAC oriented /April 27th, feature 2004 Delphi, .NET including Pro-prietary TRICENTIS C#, java, Microsoft Test Auto- 8.1/October WPF, java 8 TOSCA commercial Active Technology IE, FireFox VB6 windows mation 2013 swing/SWT/ Software & Consulting AWT visual Basic Microfocus International, 4Test Java, 4Test, Microsoft Test Auto- 13.5/ October originally 9 Silktest Scripting Pro-prietary Active VB, c#, IE, FireFox Windows mation 10, 2012 developed by Language VB.net Segue Software Test Microsoft Load Win HP Software Scripting 10 C Windows Testing 11.52 Pro-prietary Active IE, Netscape Runner Division Language and Linux Tool (TSL)

www.ijcsit.com 911 Monika Sharma et al, / (IJCSIT) International Journal of Computer Science and Information Technologies, Vol. 5 (1) , 2014, 908-912

th V. CONCLUSION of the 19 Asia-Pacific Software Engineering Conference. IEEE In this paper we have analysed different automated tools for Computer Society Press, 2012, pp. 717-720. [4] D. Xu, W. Xu, B. K. Bavikati, W. E. Wong, “Mining Executable web testing. Web based automation testing is concerned Specifications of Web Applications from Selenium IDE Tests,” in with testing web based applications in an automated way. A Proceedings of the Sixth International Conference on Software variety of automation tools are feasible for testing web Security and Reliability. IEEE Computer Society, 2012, pp. 263- applications. A popular way of testing web sites is 272. [5] L. Nagowah and K. Doorgah, “Improving Test Data Management in using Selenium tool, which can record your behaviour as Record and Playback Testing Tools,” in Proceedings of the you use the web site and then play back the steps International Conference on Computer & Information Science automatically in your web browser. The main benefit of (ICCIS). IEEE Computer Society, 2012, pp. 931-937. using automated tools is that you can avoid the manual [6] L. Nagowah and K. Doorgah, “Improving Test Data Management in Record and Playback Testing Tools,” in Proceedings of the effort required to test each feature of your web site by International Conference on Computer & Information Science automating the tests. From the study on web based (ICCIS). IEEE Computer Society, 2012, pp. 931-937. automation testing and the automated tools for this type of [7] F. Wang, W. Du, “A Test Automation Framework Based on WEB,” th testing, we conclude that Selenium is the best available in Proceedings of the 11 International Conference on Computer and Information Science. IEEE/ACIS, 2012, pp. 683-687. automation tool for web applications till now. [8] R. A. Razak, F. R. Fahrurazi, “Agile Testing with Selenium”, in Proceedings of the 5th Malaysian Conference in Software Engineering (MySEC), IEEE Computer Society Press, 2011, pp. 217- REFERENCES 219. [1] A. Gargantini and E. Riccobene, “ASM-Based Testing: Coverage [9] X. Wang, P. Xu, “Build an Auto Testing Framework Based on Criteria and Automatic Test Sequence,” J.UCS: Journal of Universal Selenium and FitNesse”, in Proceedings of the International Computer Science, pp. 1050–1067, 2001. Conference on Information Technology and Computer Science, IEEE [1] M. Leotta, D. Clerissi, F. Ricca, C. Spadaro, “Repairing Selenium Computer Society, 2009, pp. 436-439. Test Cases: An Industrial Case Study about Web Page Element [10] F. Cannizzo, G. Marcionetti, P. Moser, ”Evolution of the Tools and Localization,” in Proceedings of the Sixth International Conference Practices of a Large Distributed Agile Team”, in Proceedings of the on Software Testing, Verification and Validation. IEEE Computer Agile 2008 Conference, IEEE Computer Society, 2008, pp. 513-518. Society Press, 2013, pp. 487-488. [11] B. Haugset, G. K. Hanssen,“Automated Acceptance Testing: a [2] M. Leotta, D. Clerissi, F. Ricca, C. Spadaro, “Improving Test Suites Literature Review and an Industrial Case Study”, in Proceedings of Maintainability with the Page Object Pattern: An Industrial Case the Agile 2008 Conference, IEEE Computer Society, 2008, pp. 27-32. Study,” in Proceedings of the Sixth International Conference on [12] A. Holmes, M. Kellogg,” Automating Functional Tests Using Software Testing, Verification and Validation Workshops. IEEE Selenium”, in Proceedings of the AGILE 2006 Conference Computer Society Press, 2013, pp. 108-113. (AGILE'06), IEEE Computer Society, 2006. [3] B. Cheluvaraju, K. Nagal, A. Pasala, “Mining Software Revision History using Advanced Social Network Analysis,” in Proceedings

www.ijcsit.com 912