Load Testing GUI and Middleware Web Applications Requires a Different Approach

Tom Igielski The Thomas Consulting Group, Inc. [email protected]

Solving the Software Quality Puzzle Page 1 www.psqtconference.com D Characteristics of All Web Applications

Extremely heterogeneous Use complex technologies Incorporate a wide variety of standards Extraordinary demands at both predictable and unpredictable times Potentially millions of users….none of which have been trained on the application May or may not have a user interface

Solving the Software Quality Puzzle Page 2 www.psqtconference.com D Characteristics of GUI Web Applications

User navigation • Predictable…..or NOT…effect on performance? Image and text validation General page content validation Client-side scripting Network bandwidth considerations Site abandonment Does not affect performance….or does it? Application usage patterns Simulating different browser characteristics Thin clients vs. thick clients

Solving the Software Quality Puzzle Page 3 www.psqtconference.com D Characteristics of Middleware Environments

Importance of message content vs. visual cues Web service interactions Concurrent message requests Dependence of messages on the back-end Message to message interaction No GUI client available for testing May be harder to debug Stronger dependency on developers during testing

Solving the Software Quality Puzzle Page 4 www.psqtconference.com D Common Performance Problems

Long response time measured by the servers Long response time from a client’s point of view Memory “leaks” High CPU usage Lengthy queues Too many database table scans Database deadlocks Erroneous data returned

Solving the Software Quality Puzzle Page 5 www.psqtconference.com D Typical Performance Problem Areas

ISP2 ISP1

Hub Router Router Hub Web Database Server Server Load Application Balancer Web Server Server

Firewall

External Internet Datafeed bottlenecks Network vs. Server issues Load balancer Server HW, device problems and OS issues ISP performance and peering Errors Application server issues Faulty web server End-user Connection issues within a cluster Content issues

Solving the Software Quality Puzzle Page 6 www.psqtconference.com D Typical Performance Testing Environment

Load Database Balancer Server

Web Application Servers Servers Agents

Solving the Software Quality Puzzle Page 7 www.psqtconference.com D Performance Test Planning

• (Common) Questions to Ask Before You Start What are the characteristics of the web application? How scaleable does the web environment need to be? What are the data sources? Which database(s) will be used? What are the security considerations? What are the main objectives of the test? What are the goals for the number of virtual users under load? Is this intended to be a one time or repeatable test?

Solving the Software Quality Puzzle Page 8 www.psqtconference.com D Performance Test Planning

• (Common) Questions to Ask Before You Start What is the business function of the system? What and where are the users? Who says there is a performance problem, and what is slow? What is the system configuration? What application software is to be used? What will the busy processes on the system doing? What are the anticipated CPU and disk utilization levels? What will make the disks “busy”? How much network activity is there expected to be? How much memory will be required?

Solving the Software Quality Puzzle Page 9 www.psqtconference.com D Performance Test Planning

GUI Specific Questions • What browser configurations need to be supported? • How is the system typically navigated? • How much activity is generated on the server vs. the client? • What visual cues can be used to verify functional correctness during load testing? Non-GUI Specific Questions • How are requests initiated and what initiates them? • What kind of message-to-message interaction exists and how might it affect performance? • How do we verify functional correctness of messages / applications? • e.g. HTTP Error Codes, message return values

Solving the Software Quality Puzzle Page 10 www.psqtconference.com D Common Approach Activities

• Plan the Testing Activities Develop a Strategy Clearly State Your Objectives Define the Scope of the Performance Test Build the Test Team Identify the Performance Requirements Develop Necessary Test Scripts / Cases Line Up the Necessary Testing Resources Define Needed Test Data and Process to Maintain Identify any Necessary Tools / Environments

Solving the Software Quality Puzzle Page 11 www.psqtconference.com D Creating A GUI Virtual User Record User Actions

BrowserBrowser WebWeb Application Application Web server LoadLoad ToolTool Internet sendsend URL1 URL1 receivereceive Web Web page page •• LoadLoad Test Test Tool Tool launches launches the the browser browser and and taps taps intointo its its HTTP HTTP communication communication with with the the server server •• LoadLoad Tool Tool captures captures the the HTTP HTTP communication communication whilewhile you you work work in in your your application application •• TThehe captured captured communication communication becomes becomes part part ofof a a Vuser Vuser script script whic whichh can can be be played played back back

Solving the Software Quality Puzzle Page 12 www.psqtconference.com D Creating A Non-GUI Virtual User Script User Actions

HandHand ScriptingScripting WebWeb Application Application Web server LoadLoad ToolTool Internet sendsend URL1 URL1 receivereceive Web Web page page •• LoadLoad Tester Tester generates generates the the script script by by hand hand or or byby using using the the WSDL WSDL definitions definitions •• LoadLoad Tool Tool sends sends and and receiv receiveses the the HTTP HTTP communicationcommunication to/from to/from you you application application •• TThehe captured captured communication communication becomes becomes part part ofof a a Vuser Vuser script script whic whichh can can be be played played back back

Solving the Software Quality Puzzle Page 13 www.psqtconference.com D Which Transactions to Measure?

Choose meaningful transactions – What is a valid response time from the user’s perspective? – Solicit input from business analysts, end-users, developers Few transactions will provide little data Suggestions for Transactions – Insert into, or update the DB – Search or Sort – Heavy graphic manipulation

Solving the Software Quality Puzzle Page 14 www.psqtconference.com D Ideal Business Scenarios

• Identify critical business functions – 80 / 20 rule • 80% of the load is created by 20% of the transactions – Determine scenario information • User profile (types of users) • # Users (# business processes / user tasks) • User volume (# processes per user per hour) – Determine data needs • Input data (vuser parameters) • Identify dependencies • Minimize exceptions

Solving the Software Quality Puzzle Page 15 www.psqtconference.com D Ideal Business Scenarios

Hours to be emulated Business Processes Night Night Create new account 40 30 Day50 100 50 50 25

Search for title 2000 5000 1500 1000 500

700 1600 500 300 150 View shopping cart 500 200 170 140 Purchase book

Administrative Processes *Invoice processing 50 50 50 50

*System backup 1 1

12 2244668810 12 10 12 am pm *Not a Web task ONLINE BOOKSTORE (typical work day)

Solving the Software Quality Puzzle Page 16 www.psqtconference.com D GUI Load Test Cases

Generally 3 Categories – End-to-end – Modular – Single or low number of pages Modular tests of single / low number of pages work best for isolating bottlenecks Create short tests that focus narrowly on problem pages or functions Run scripts one at a time, with either fixed delays or no delays

Solving the Software Quality Puzzle Page 17 www.psqtconference.com D End-to-End Scripts

Home Access Move Login Logout Page Accounts Funds

Solving the Software Quality Puzzle Page 18 www.psqtconference.com D Modular Scripts

Home Home Home Home Page Page Page Page

Access Move Logout Accounts Funds

Logout Logout

Solving the Software Quality Puzzle Page 19 www.psqtconference.com D Business Process Is Completed

texttext visual visual Expected cuecue Expected resultresult appears;appears; verificationverification passespasses

Solving the Software Quality Puzzle Page 20 www.psqtconference.com D Business Process Not Completed

texttext visualvisual cuecue ExpectedExpected resultresult does does notnotappear;appear; verificationverification failsfails

Solving the Software Quality Puzzle Page 21 www.psqtconference.com D Determine Visual Cues to Verify

Same image Use whenever image page is loaded check

Solving the Software Quality Puzzle Page 22 www.psqtconference.com D Successful Load Tests

Metrics – Hits / Sec – # Virtual Users – Transaction Response Time – Transactions / Second – Pages / Second – % CPU Utilization

Solving the Software Quality Puzzle Page 23 www.psqtconference.com D Performance Testing Considerations

Changes in the Architecture Long running queries Load / not load images # expected users; concurrent users Common user actions Predicted behavior patterns Think times and wait times Growth levels and peak periods Acceptable response time per critical function

Solving the Software Quality Puzzle Page 24 www.psqtconference.com D Common Performance Scripting Issues

• Database errors – Duplicate data – Non-existent data • Unusually fast response time – Data is cached • Script branching due to data – If purchase < $500, go to shipping screen – If purchase >= $500, go to approve credit screen • Data driven exceptions – Cannot ship on weekend

Solving the Software Quality Puzzle Page 25 www.psqtconference.com D What Load Testing Transactions Measure

A Load Testing transaction measures the system performance resulting from one or more user actions

Transaction 1 - Transaction 2 - measure measure the everything that happens login actions after the login

Transaction 3 - measure User steps only the save order action

Solving the Software Quality Puzzle Page 26 www.psqtconference.com D Transaction Analysis

TransactTransactiionon ResponseResponse TimesTimes

TransactTransactionion NameName

Solving the Software Quality Puzzle Page 27 www.psqtconference.com D Look for System Bottlenecks

Solving the Software Quality Puzzle Page 28 www.psqtconference.com D Typical Load Testing Scenarios

OBJECTIVE PURPOSE # VUSERS

1. Verify that Vusers Verify that parameterized data 3-5 run concurrently works under concurrent usage

2. Isolate Top Time Isolate transactions that have 20% transactions performance problems irrespective of load

3. Run full load test Verify that the system functions 100% under the expected load

4. Run "overdrive test" Run more than 100% load to 100% + determine scalability of system

Solving the Software Quality Puzzle Page 29 www.psqtconference.com D Cross Result Graphs

heavier load

heavier load lighter load

lighter load

Solving the Software Quality Puzzle Page 30 www.psqtconference.com D Load Testing Results

What happened to cause response times to spike after 20 minutes?

Solving the Software Quality Puzzle Page 31 www.psqtconference.com D Questions?

Solving the Software Quality Puzzle Page 32 www.psqtconference.com D