IJISET - International Journal of Innovative Science, Engineering & Technology, Vol. 1 Issue 4, June 2014. www.ijiset.com ISSN 2348 – 7968

Performance Testing Tools: A Comparative Study

Dr. Manju Kaushik1, Pratibha Fageria2

1Associate Professor (Computer Science and Engineering),JECRC University Jaipur, India 2M.tech (Computer science), JECRC University Jaipur, India Email: [email protected]

Abstract- types of performance testing: Stress test, The objective of this research paper Load test, Strength test. Performance testing is to analyze the performance of some tools is used to determine the time required specific testing tools and compared in terms performing a task by the system and it of their different performance parameters. provides stability of system under different Performance parameters results generated by load conditions. Testing tools simulate these performance testing tools have been multiple user access of web server. Through evaluated and analyzed. The same web site analysis of data tools, performance has been tested for performance under these parameters of web server are obtained. performance testing tools then differences in Several issues have been observed related to results of various performance parameters tools when conducting performance testing like throughput, response time, number of such as tools compatibility with the hit pages, error rate, memory and CPU under test, tools installation, tools setup, utilization etc. are obtained. The same tools flexibility in doing test both for client website has been put under load test for a and server side. With the help of test tools, number of virtual users and results have testers can create, manage and execute tests been analyzed. for a specific test environment for a Keywords: Performance testing tools, particular application. The test results are Throughput, Response Time, Virtual Users, compared with the expected results to Load generation. evaluate the quality of the product. Different testing tools are used to test the load of server. Tools have many issues to conduct 1. INTRODUCTION performance testing such as tools compatibility with system under test, tools Now days, various business critical installation, tools setup and flexibility in applications are tested for performance doing test both for client and server side. In before launching satisfy their customer this paper, three different performance needs. Performance testing is a non- testing tools are used for performance functional type of testing to determine the testing for same web site and results are speed, stability, reliability and scalability of generated for different performance the system. Various types of performance parameters in different browsers. The tests are done to check the system’s research discussion has been organized into behavior and to determine the part that different sections. Section II discusses prior performs poorly. Ignoring performance test related works. In section III, overview of means that your system is not fully tested, features of three tools that are used for especially from the risk and operational comparison is given. In section IV, profile perspectives. There are different concludes the overall work.

264 IJISET - International Journal of Innovative Science, Engineering & Technology, Vol. 1 Issue 4, June 2014. www.ijiset.com ISSN 2348 – 7968

2. RELATED WORKS simulation mechanism has been given. In the paper “Open Source & Commercial Most previous work on performance testing process and emulation of multiple tools comparison ignored different results concurrent users. Performance Testing reported by each tool. In the research paper Tools” by Vinod P [6], performance testing “A Comparative Study of Performance tools have been compared based on factors Testing Tools” by Rina and Sanjay Tyagi including accuracy, cost and other features. [1], some specific performance testing tools In the paper “Performance Testing Tool are used for performance testing by using Comparison” by Smith[6], HP, Load Runner, different performance parameters. NeoLoad, Load Test, loadUI and Grinder performance LoadUI and WAPT generate different testing tools have been compared in terms of parameters results in different browsers. In cost, market place skill set, scalability, result the research paper “Performance evaluation reporting etc. In the paper “Stress, Load, and comparison of Software Testing Tools” Volume, Performance, Baseline Testing by Sneha Khoria and Pragati Upadhyay [2], Tool Evaluation and Comparison” by VCAA some specific performance testing tools [7], stress, load, volume, performance, have been compared for their usability and benchmarking and base line testing tools effectiveness. WAPT and RANOXEX have been compared in terms of all features performance testing tools inferences, and price. Testingrefeclections.com [8] implications and results have been presented concludes that accuracy of load and and discussed. Different attributes, their response time is something we need to ability to compare the results, test cases evaluate against our particular application documentation ability and regression testing and not something to compare when performance ability have been compared. In determining the tool to use or buy. the research paper “Web services testing tools: A Comparative Study” by Shariq In our research paper, differences in Hussain, Zhaoshun Wang, Ibrahima Kalil performance parameters resulting in Toure and Abdoulaye Diop [3], three popular different browsers of different performance open source web service testing tools have testing tools have been analyzed. been compared in terms of features, usability, performance and software requirements. In “Web Application–A Study on Comparing Software Testing Tools” by Dr. S. M. Afroz, 3. OVERVIEW OF N. Elezabeth Rani and N. Indira Priyadarshini PERFORMANCE TESTING [4], Dart and Apollo software web tools TOOLS have been compared in terms of their dynamic test generation ability. A survey Performance tools are used for different has been presented on static and dynamic types of performance testing including load testing analysis. In the paper “Performance test, stress test, volume test and endurance Testing: Analyzing Differences of Response test. These tools are either open source or Time between Performance Testing Tools” proprietary tools. For this research, three by Muhammad Dhiauddin Mohamed performance testing tools Neoload, WAPT Suffiani, Fairul Rizal Fahrurazi [5], different and LoadUI, have been selected. performance testing tools response time Neoload have been compared and justification of Neoload [9] used for measuring and these differences include architecture and analyzing the performance of the website.

265 IJISET - International Journal of Innovative Science, Engineering & Technology, Vol. 1 Issue 4, June 2014. www.ijiset.com ISSN 2348 – 7968

The performance and the end result can be creating, configuring and updating tests evaluated by using this tool. This tool while the application is being tested. It also analyzes the performance of the web gives a visual Aid for the user with a drag application by increasing the traffic to the and drop experience. The advanced analysis website and determines the performance and report generating features allows under heavy load. This tool provides all the examining the actual performance by features that are needed for load testing and pumping in new data even while the to analyze the results. A large number of application is being tested. This tool is users are simulated simultaneously. It allows compatible with operating systems like you to analyze both the user response time , . and Infrastructure’s statistics (database, web server, network components etc). It 4. RESULT AND DISCUSSION performs testing more quickly, efficiently and frequently. This tool tests rich internet These performance testing tools NeoLoad, applications such as AJAX, FLEX&AIR, WAPT and LoadUI generate different GWT, RTMP, and Java Serialization. This parameters results in different browsers. By tool is compatible with operating systems using comparison result tables it concludes like Microsoft Windows, Linux and Solaris. that WAPT takes less average response time WAPT than LoadUI and NeoLoad. Errors are zero WAPT [9] provides load, stress and in all browsers. Total hits are higher in performance testing of web sites and web WAPT than other two tools. CPU utilization applications with web interface. It consists is less than NeoLoad. In NeoLoad average of the workplace component and multiple response time is less, Average hits value is load agents that can be installed anywhere high and average throughput is higher than and managed remotely. This tool provides LoadUI in all browsers. NeoLoad is better detailed information about the virtual users than LoadUI. From this discussion it is and its output to users during the load found that WAPT performance testing tool testing. This tool is considered to be the best is best in all these three tools. cost effective tool for analyzing the performance of the web services. It uses WMI and SNMP interfaces to collect the 5. CONCLUSION AND FUTURE performance information directly from each WORK server and database. It provides custom java script code that provides dynamic request In this paper performance parameters results parameterization. It uses GUI approach for of different performance testing tools test creation and execution. This tool is NeoLoad, WAPT and LoadUI in different having modules for ASP.NET testing, browsers have been analyzed. The same ADOBE FLASH tests and JSDN format Web site has been tested for performance testing. This tool is compatible with under these performance testing tools and Microsoft Windows XP/2003/Vista performance results of different tools have /2008/Win7. been compared. These comparisons provide information to select the better tool for performance testing of web applications LoadUI – according to performance requirement. It is LoadUI [9] is an open source testing tool, difficult to compare tools because many targeted mainly at web services. This allows parameters values are not considers in all

266 IJISET - International Journal of Innovative Science, Engineering & Technology, Vol. 1 Issue 4, June 2014. www.ijiset.com ISSN 2348 – 7968 tools. This research work can be extended to http://www.scribd.com/doc/2776593 more experiments with more tools and 9/Performance-Test-Tools- different comparison parameters to provide Comparison. more realistic results. 7. Kualitatem (2012), Comparison between HP, IBM and APACHE – Performance Testing Tools 6. REFERENCES [Online].Avialable:http://www.kualit atem.com/comparison-between- 1. Rina, Sanjay Tyagi “A Comparative performance-testing-tool/. Study of Performance Testing Tools” 8. VCAA (2005), Stress, Load, International Journal of Advanced Volume, Performance, Baseline Research in Computer Science and Testing Tool Evaluation and Software Engineering (IJARCSSE), comparison. 3(5), Pages 1300-1307, (2013). [Online].Available:http://www.vcaa. 2. Sneha Khoria, Pragati Upadhyay com/tools/loadtesttoolsevaluationcha “Performance Evaluation and rt-023.pdf. Comparison of Software Testing 9. Barber, S. (2006). Choosing Tools” VSRD International Journal Performance Testing of Computer Science and [Online].Testingreflections.com. Information Technology, 2(10), Available: Pages 801-808, (2012). http://www.testingreflections.com/

3. Shariq Hussain, Zhaoshun Wang, Node/view/3939 Ibrahima Kalil Toure and Abdoulaye 10. http://www.softwaretestinghelp.co Diop, “Web Service Testing Tools: m/performance-testing-tools-load- A Comparative Study”, International testing-tools/ Journal of Computer Science Issues (IJCSI), 10(1), Pages 641-647, (2013). 4. Dr. S. M. Afroz, N. Elezabeth Rani and N. Indira Priyadarshini, “Web Application– A Study on Comparing Software Testing Tools”, International Journal of Computer Science and Telecommunications, 2(3), Pages 1-6 (2011). 5. Muhammad Dhiauddin Muhammad Suffian, Fairul Rizal Fahrurazi “Performance Testing: Analyzing Differences of Response Time between Performance Testing Tools” International Conference on Computer & Information Science (ICCIS), Malaysia, 919 (2012) 6. Vinod, P. (2008), Open Source & Commercial Performance Testing Tools [Online].Available:

267