STRESS, LOAD, VOLUME, PERFORMANCE, BENCHMARK AND BASE LINE TESTING TOOL EVALUATION AND COMPARISON Cordell Vail Copyright 2005 by Cordell Vail - All rights reserved www.vcaa.com NOTE: The information contained in this document or on the handout CD at the seminars is for use only by the participants who attend one of our seminars. Distribution of this information to anyone other than those attending one of the seminars is not authorized by the authors. It is for educational purposes of the seminar attendees only. It is our intention that by putting this information here from the vendor web pages and from other testers evaluations, that you will have a tool that will let you do your own evaluation of the different tool features. Hopefully this will let you find tools that will best meet your needs. The authors are not recommending any of these tools as one being better than another. All the information has been taken from reviews we have found on the Internet, from their own web pages, or from correspondence with the vendor. They have been grouped here according to our presentation outline. For some of the tools, we were not able to determine the cost or type (ie open source) from their web page or correspondence with the vendor. Users are cautioned that they will be installing any downloaded tools at their own risk. TABLE OF CONTENTS ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Table Of Contents Key: EVALUATION SECTION Page VENDOR (and foreign country if known) Tool Name [type and price if known] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ TOOLS WORTH LOOKING AT (the listing by price does not imply prefereance)............................12 MICROSOFT ...................................................................................................................................................................12 Web App Stress Tool (WAS) [free]...................................................................................................................................................................... 12 APPPERFECT ................................................................................................................................................................37 LoadTester [Free]................................................................................................................................................................................................ 37 PUSHTOTEST ................................................................................................................................................................40 Performance Kit [Free] ........................................................................................................................................................................................ 40 PILOT SOFTWARE LTD (TURKEY)...............................................................................................................................43 SiteTester [$29.00] .............................................................................................................................................................................................. 43 SOFTLOGICA LLC (Russia) ..........................................................................................................................................47 2 WAPT [$299] ....................................................................................................................................................................................................... 47 LOADTESTING.COM......................................................................................................................................................49 Portent Supreme [$279.95] ................................................................................................................................................................................. 49 PAESSLER (Germany) ..................................................................................................................................................53 Webserver Stress Tool 7 [$624.95]..................................................................................................................................................................... 53 SIS –Florida Tech...........................................................................................................................................................60 Holodeck [$1,500]................................................................................................................................................................................................ 60 MAJOR TOOLS (in the 93% of Market Share)...................................................................................... 67 MERCURY INTERACTIVE..............................................................................................................................................67 LoadRunner [Over $5,000] 54%.......................................................................................................................................................................... 67 IBM RATIONAL...............................................................................................................................................................74 Performance Tester [Over $5,000] 11%.............................................................................................................................................................. 74 COMPUWARE ................................................................................................................................................................75 QALoad [Over $5,000] 9% .................................................................................................................................................................................. 75 SEGUE ............................................................................................................................................................................78 SilkPerformer [Over $5,000] 7%.......................................................................................................................................................................... 78 EMPIRIX..........................................................................................................................................................................85 e-TEST suite [Over $5,000] 6% .......................................................................................................................................................................... 85 3 RADVIEW........................................................................................................................................................................88 WebLoad [[Over $5,000] 3% ............................................................................................................................................................................... 88 OTHER LEADING VENDORS................................................................................................................. 91 MICROSOFT ...................................................................................................................................................................91 Application Center Test [$2,999]......................................................................................................................................................................... 91 REDGATE SOFTWARE..................................................................................................................................................92 ANTS Load Professional Edition [$2,490]........................................................................................................................................................... 92 OPENDEMAND...............................................................................................................................................................94 OpenLoad [$2,195].............................................................................................................................................................................................. 94 TEVRON..........................................................................................................................................................................95 CitraTest [Over $5,000] ....................................................................................................................................................................................... 95 HOSTEDLABS................................................................................................................................................................98 HostedToolBox [$20 per month].......................................................................................................................................................................... 98 AUTOMATEDQA ..........................................................................................................................................................101 AQtime [$599]...................................................................................................................................................................................................