Continuous Load Testing: Reinventing Load Testing for DevOps tricentis.com Executive Overview Transformational initiatives such as Agile and DevOps are compelling software development teams to change their approach to load testing. This new approach involves extending beyond late-cycle load testing by performance testing specialists and empowering developers and testers to perform early, continuous load testing. Flood IO is built to enable load testing for DevOps, helping DevTest teams gain performance feedback rapidly and continuously. In addition to supporting industry-leading load testing tools, Flood integrates with Tricentis Tosca—allowing users to perform load testing with the same scriptless tests they use for functional testing. For DevOps, Load Testing Shifts Left Over 91% of organizations already have Agile or DevOps initiatives or plan to adopt them within the immediate future. As part of the process transformation required to adopt these more iterative and rapid methods of delivering software, organizations need to reassess ALL practices associated with software development and test—including load and performance testing. Traditionally, load testing has been deferred until the late phases of waterfall release cycles. Specialized performance testers applied legacy load testing tools to validate application performance prior to release. Why is this no longer sufficient? • With teams moving to more componentized applications (including cloud-native apps, microservices, etc.), an application involves many highly-distributed components—and a performance issue in any of them could have a ripple effect across the entire application • Now that new functionality is being released weekly, daily, or hourly, each team needs instant insight into whether their incremental changes negatively impact performance • The later you start load testing, the more difficult, time-consuming, and costly it is to expose, debug, and resolve performance problems In-depth load testing by performance testing specialists remains critical—but it doesn’t provide the level of fast, on-demand load testing that’s critical for Agile and DevOps. Developers and testers need a way to expose critical performance issues before new functionality progresses through the delivery pipeline. To achieve this, they must be able to easily: • Create load tests that provide fast feedback on the functionality they’re evolving 1 • Execute and scale those load tests as needed—without the exorbitant costs and efforts traditionally required to establish, configure, and maintain a performance test lab This is exactly where Flood IO excels. Flood IO: Load Testing for Modern Software Development Flood is a cloud-based load testing platform that helps teams see how their applications respond to globally-distributed load. By reducing the complexity traditionally associated with load testing, it extends load testing to developers and testers. Teams can start load testing their application components as they’re evolving them and continuously monitor performance impacts by integrating load testing into their CI/CD pipelines. Flood takes test plans from open source tools such as JMeter, Gatling, and Selenium and distributes them across readily available servers in the cloud. It also allows users to “flood” any HTTP endpoint on-the-fly. Teams can start load testing new functionality as soon as it’s developed. A developer or tester can instantly generate millions of globally-distributed requests per second from his or her local machine—without dealing with the hassles of requesting, configuring, or managing performance resources on-prem or in the cloud. 2 Thanks to Flood’s powerful scaling, the same core set of load tests can be leveraged to meet different performance testing demands throughout the delivery process. Load can instantly be scaled up or down, making it simple for any team member or automated process to generate the exact load needed for a particular goal or phase. Flood covers all load testing needs—from interactive developer performance tuning with real-time results, to automated CI-driven Continuous Load Testing under a typical load, to nightly testing under a heavier load, to comparing major feature updates versus previous performance benchmarks. The Flood API enables seamless integration into CI tools and a best-of-breed DevOps toolchain. Flood simplifies the following challenges associated with load testing: • Provisioning of load test infrastructure on demand • Distribution and execution of load tests at scale • Collection and aggregation of distributed results • Visualization and analysis of results in real time A New Era of Load Testing: Load Testing with Tricentis Tosca In addition to supporting traditional “protocol-level” level approaches to load testing (e.g., with JMeter and Gatling), Flood helps testers load test at the browser level. Without diving into technical details, there are two primary advantages of testing at this level. First, tests deliver more consistent results and are more resistant to change. Second, tests assess the application at the business (and user) level, which makes them more realistic. Until now, the main challenges with load testing at the browser level have been: • The level of work required to create effective browser-level tests • The level of resources required to run browser-level tests at scale The barriers to testing at this level have been removed now that Tricentis Tosca functional tests can be used for load testing with the cloud. Tricentis Tosca tests are not only simple to create, but also stable and easy to maintain. This makes them perfect candidates for the Continuous Load Testing that’s become critical for today’s DevTest teams. 3 For a concrete idea of how this plays out, consider this scenario: 1. Teams start off with load testing smoke tests—using their functional tests to assess how well “happy paths” through the application (also known as straight throughs) respond to a relatively light load. 2. As the application progresses through the pipeline, they extend these load testing efforts to include more test scenarios and larger loads. The varying degrees of load can be generated on-demand without any infrastructure hassles (Flood handles all the setup, configuration, and reporting). This way, the team can instantly generate load when they need it—without the costs and resources associated with traditional performance test lab. As with all Flood load tests, these tests can be seamlessly integrated into any CI ecosystem to enable Continuous Load Testing. 4 Wrap Up To conclude, we wanted to leave you with an idea of what’s so special about Flood. But we won’t bore you with details about the scalability of our shared nothing architecture, the advantage of our test-friendly pricing, our commitment to being a tool-agnostic solution, and so on. Instead, we’ll let a few of our customers explain why they’ve made Flood an integral part of their software delivery process. “Previously, we had very specialized tools for load and performance testing that were quite expensive and very richly featured, but completely disconnected from our everyday development tools. The main outcome of this was that we ended up with a couple of engineers who were quite good at load & performance testing with our enterprise tools while the majority of engineers found the barriers too great. With Flood, we have moved to an approach which is far more inclusive and utilizes many of the tools our engineers are working with on a daily basis.” - REA (Digital Advertising) “Rally Health began the new year with a bang. On January 1, we added more than 350 new organizations as clients, which meant that we suddenly had 5 million new users eligible to register. Add that to our current users, and it’s a bit like trying to expand your airplane in midair. With Flood load testing, it was smooth sailing — despite this massive influx of new users, our engineers report that we set up all these clients with no errors whatsoever, which surpassed even our most optimistic predictions.” - Rally Health “Flood helped us get started with load/stress testing quickly and without hassle. We are able to produce massive load tests with ease. We can easily test and monitor the resilience of all our applications and tune our instance sizes and auto scaling groups accordingly.” - Online ticket marketplace 5 Reinvent load testing for DevOps Start a free trial of Flood or get first access to Tricentis Tosca Flood About Tricentis Tricentis, the Continuous Testing Company, specializes in market leading agile software testing tools for enterprises. We help Global 2000 companies adopt DevOps and gain success by achieving automation rates of over 90%. Top analysts recognize Tricentis as a leader in Software Test Automation, with Model-based Test Automation and Test Case Design as standout features. Our 400+ customers include global names from the Top 500 brands such as A&E, Allianz, Deutsche Bank, HBO, JetBlue, Orange, Swiss Re, Telstra, Toyota, UBS, Vantiv, Virgin Airlines, and Zurich Insurance. Tricentis has offices in Austria, United States, Germany, Switzerland, UK, Netherlands, Poland, and Australia. 6 .