CALIFORNIA STATE UNIVERSITY SAN MARCOS PROJECT SIGNATURE PAGE PROJECT SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE MASTER OF SCIENCE IN COMPUTER SCIENCE PROJECT TITLE: AN EXPLORATORY STUDY ON WEB APPLICATION AUTOMATION TESTING AUTHOR: Nishtha Sharma DATE OF SUCCESSFUL DEFENSE: 07/31/2020 THE PROJECT HAS BEEN ACCEPTED BY THE PROJECT COMMITTEE IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE IN COMPUTER SCIENCE. Simon Fan 7/31/2020 PROJECT COMMITTEE CHAIR SIGNATURE DATE Xin Ye 7/31/2020 PROJECT COMMITTEE MEMBER SIGNATURE DATE An exploratory Study on Web Application Automation Testing Submitted in Partial fulfilment of the requirements for the degree of Master of Science California State University San Marcos Department of Computer Science 2020 By Nishtha Sharma Under the guidance of Dr. Simon Fan Project Report Nishtha Sharma Abstract Large organizations like a university typically have many web applications deployed and used by a larger number of users every day. As such an application changes, it should be thoroughly tested to avoid unexpected service failures or interruptions. The purpose of this project is to evaluate the pros and cons of the Page-Object design pattern in Selenium-based testing of web applications. We have experimented with two web applications deployed at CSUSM and our conclusion is two-fold. On the one hand the Page-Object design pattern can largely improve readability due to clean separation between page-specific test code and production code. On the other hand, it improves the change management of large web applications: any UI change can easily be implemented, updated and maintained into the Page Objects. 1 Project Report Nishtha Sharma Table of contents: Abstract…………………………………………………………………………………….………………………………………..1 1. Introduction……………………………………………………………………………………………………………………….3 1.1 Promises and Problems of Test Automation…………………………………………………………….4 1.2 Different Test Automation Approaches…………………………………………………………………….6 1.2.1 Dynamic vs. Static Testing………………………………………………………………………………6 1.2.2 Functional vs. Non-Functional Testing………………………………………………………….6 1.2.3 Granularity of the Tested System………………………………………………………………….7 1.2.4 Testing Activities……………………………………………………………………………………………..8 1.2.5 Small Scale vs. Large Scale Test Automation………………………………………………..9 1.3 Scope……………………………………………………………………………………………………………………………10 2. Background and Related Work……………………….…………………………………………………………….11 2.1 Selenium…………………………………………………………………………………………………………………….. 11 2.2 Page Object Pattern……………………………………………………………………………………………………14 2.3 TestNG…………………………………………………………………………………………………………………………15 2.4 Extent Reports…………………………………………………………………………………………………………….18 2.5 Related Work………………………………………………………………………………………………………………..19 3. Design and Implementation…………………………………………………………………………………… 21 3.1 Design Diagram………………………………………………………………………………………………….21 3.2 Components required to get started with automation………………………………………22 3.3 Basic steps for Selenium test case………………………………………………………………………26 3.4 Case Study…………………………………………………………………………………………………………28 3.5 Test Execution Report………………………………………………………………………………………..31 4. Lessons Learned………………………………………………………………………………………………………36 5. Conclusion……………………………………………………………………………………………………………… 39 References…..………………………………………………………………………………………………………………41 2 Project Report Nishtha Sharma 1. Introduction: With the development of each web application or software there is a very important need to verify that the developed software’s functionality is as per the requirements of the stakeholders. Software testing is a process to evaluate the functionality of a software application with an intent to find whether the developed software has met the specified requirements or not and to identify the potential defects in order to produce the quality product. Automation testing uses software tools, separate from the software being tested, to control the execution of tests and the comparison of actual outcomes with predicted outcomes. Test automation process can automate repetitive but necessary tasks in a formalized testing process, or perform additional testing that would be difficult to do manually. Test automation is critical for continuous delivery and continuous testing [3]. The key benefit of automating any web application is it saves time and effort to test the application. Software tests have to be repeated often during development cycles to ensure quality. Every time source code is modified software tests should be repeated. For each release of the software it may be tested on all supported operating systems and hardware configurations. Manually repeating these tests is costly and time consuming. Once created, automated tests can be run over and over again at no additional cost and they are much faster than manual tests. Automated software testing can reduce the time to run repetitive tests from days to hours [3]. Once automated tests have been developed, they can be run quickly and repeatedly. Many times, this can be a cost-effective method for regression testing of software products that have a long maintenance life. Even minor patches over the lifetime of the application can cause existing features to break which were working at an earlier point in time. Test automation reduces the effort associated with manual testing. Manual effort is needed to develop and maintain automated checks, as well as reviewing test results [3]. Software systems are getting more and more important for organizations and individuals alike and at the same time they are growing bigger and more complex. It is thus only logical that importance of software quality is also rising. If quality does not get better as systems grow in size, complexity and importance, then software system may face more issues in future. The need for better quality means more pressure for software testing and for test engineers taking care of it. Test automation, i.e. giving some testing tasks to computers, is an obvious way to ease their workload. Computers are relatively cheap, they are faster than humans, they do not get tired or bored, and they work over weekends without extra pay. Giving computers all the needed details is not easy and takes time. Test automation can be used in multiple ways. It can and should be used differently in different contexts and no single automation approach works everywhere. Test automation is no silver bullet either but it has a lot of potential and when done well it can significantly help test engineers to get their work done faster and better [12]. 3 Project Report Nishtha Sharma Automation can offer huge improvements in test efficiency and effectiveness but may require substantial investment. Software Test automation makes use of specialized tools to control the execution of tests and compares the actual results against the expected result. The test execution environment configured for testing. Test bed consists of specific hardware, Operating system, network configuration, the product under test, other system software and application software [12]. 1.1 Promises and Problems of Test Automation Test automation is expected to help run lots of test cases consistently again and again on different versions of the system under test. Automation can also ease test engineers’ workload and release them from repeating tasks. All these promises has the potential to increase software quality and shorten testing time. All this makes test automation look really attractive but achieving them in real life requires plenty of hard work. If automation is not done well it will be abandoned and promises will never be realized. The general problem with test automation seems to be forgetting that any larger test automation project is a software project on its own right [17]. Software projects fail if they do not follow processes and are not managed adequately, and test automation projects are not different. Of all people, engineers ought to realize how important it is to have a disciplined approach to software development. Manual Testing of all workflows, all fields, negative scenarios is time and money consuming. It is difficult to test for multilingual sites manually. Automation does not require human intervention. Automation increases the speed of test execution, Automation helps increase Test Coverage. Below table describes some pros and cons of automation testing: Automation Pros Description Run existing regression tests on Being able to run previously created tests without extra effort a new version of a program [16] clearly makes testing more efficient. Run more tests more often [16] Automation means faster test execution which means more test rounds. Automation should also make creating new test cases easy and fast. Perform tests which would be For example performance and stress tests are nearly impossible difficult or impossible to do to conduct without automation. manually [19] Better use of resources Automation repeating and boring tasks releases test engineers for more demanding and rewarding work. Consistency and repeatability of Tests are always run the same way so test results can be tests [19] consistently compared to previous results from previous testing rounds. Tests can also be easily repeated on different environments. 4 Project Report Nishtha Sharma Automation Pros Description Reuse of tests [19] Reusing tests from earlier projects gives a kick start to a new project. Earlier time to market [17] Reusing tests and shortening test execution time fastens feedback cycle to developers. In the end that shortens