Neotys White Paper - Load Testing with the Cloud Load Testing with the Cloud A Neotys Whitepaper Neotys White Paper - Load Testing with the Cloud Table of Contents Introduction ........................................................................................................................................................................................ 3 Benefits of Load Testing in the Cloud ....................................................................................................................... 4 Load Testing Inside vs. Outside the Firewall ....................................................................................................... 5 Best Practices .................................................................................................................................................................................... 6 How to Choose a Cloud Testing Solution ............................................................................................................. 8 Getting Started with Load Testing in the Cloud ......................................................................................... 12 Conclusion ......................................................................................................................................................................................... 13 About Neotys .................................................................................................................................................................................. 14 Neotys White Paper - Load Testing with the Cloud Introduction Businesses worldwide rely on the cloud to keep pace with the speed of innovation. As organizations expand, they need to support growth with computing architecture able to scale as needed, and they look to the public cloud to address this need. In the age of the Internet of Things, most enterprise apps are delivered and consumed as a service. Application performance is now a competitive differentiator that determines whether users will consume the app based on the provided user experience. In October 2017, Gartner forecasted that Worldwide Public Cloud Services Revenue would total $260 Billion in 2017 and estimated that it would attain $411 Billion by 2020. Q3 2017 data from Synergy Research Group indicated that the cloud market continues to grow by over 40% per year, and AWS’s domination of the Infrastructure-as-a-Service market continues unabated. While Microsoft, Google and Alibaba are all gaining market share, and growing their revenues much more rapidly than Amazon, Amazon remains bigger than its next five largest competitors combined. Amazon Web Services (AWS) hosts applications for every imaginable vertical business segment. These applications fulfill a plethora of needs –they allow end users to access documents and applications remotely from anywhere and from any device. They help IT managers rapidly adapt and scale their infrastructures on-demand via virtualization and a pay-per use-model. The ability to harness scalability on-demand through cloud infrastructure-as-a-service and platform-as-a-service offerings provides enterprises with the business and operational agility required to meet ever-changing market demands at the speed of business. But how does testing in the cloud impact the work of load testing engineers? To reap the benefits of cloud, testers require a hosted, paid SaaS solution for load testing, one that deploys into the public cloud, such as AWS, uses a hosting service, such as Rackspace to set up a private cloud, or can support test in a private cloud hosted on-premises. How can testers seize the opportunities afforded by the cloud to test the performance of their web applications better? This white paper describes how the cloud has revolutionized load testing and the advantages it provides for ensuring that web applications perform well in production and meet user experience thresholds. It also covers the limitations of a test strategy that relies solely on cloud-based testing, highlighting the need for a complementary internal load testing solution. After describing several best practices for load testing in the cloud, this white paper outlines key capabilities to look for in a load testing solution. Without the right tools in place, merely moving testing activities to the cloud will likely 3 Neotys White Paper - Load Testing with the Cloud not deliver the results necessary to justify the move. Understanding how to apply the right tools and practices to make the most of the cloud is fundamental to cloud-based testing and vital to ultimately going live with total peace of mind. Benefits of Load Testing in the Cloud Load testing in the cloud is beneficial for many reasons. First, by leveraging the cloud, enterprises decrease infrastructure spend by eliminating the substantial up-front costs of purchasing and maintaining test environments that may be used only infrequently. By employing a pay-as-you-go model, cloud vendors enable access to infrastructure that companies need, when they need it, for as long as they need it. From a business standpoint, the cloud lowers total cost of ownership, while increasing flexibility. Second, load testing in the cloud eliminates the time-consuming set up and configuration of complex testing environments. Testers can create and configure the machine image they need once and then replicate it in the cloud as many times as required. Often, the cloud testing provider will automate this process, saving even more time. Third, the on-demand nature of the cloud supports distributed development teams and promotes the collaboration that ensures that performance testing is supported and understood throughout the organization—before "go live" decisions are approved. Lastly, load testing in the cloud enables testing teams to conduct more efficient and realistic large-scale tests. Robust and comprehensive cloud-enabled testing solutions can support test for both unsecured HTTP and encrypted https protocols, as well as Docker containers, ensuring that complete and efficient test is supported throughout the application testing process. Perform Testing at Scale Today's web applications can experience rapid and sporadic surges in traffic based on a variety of factors, such as the launch of new products, the initiation of sales campaigns, periodic seasonal discounting, or even the use of vouchers that erroneously enable discounting across multiple product lines rather than for only the intended product. Whenever the application is unable to handle the increased load, businesses risk lost revenue, customer service spikes and damage to corporate brand and reputation. Generating the load for large-scale tests to mimic these unanticipated spikes in production traffic, however, typically requires tens or even hundreds of machines. Purchasing and configuring these systems requires a significant investment of time and money—an untenable option in today's business environment. By using the cloud, enterprises can rapidly set up as many load generating machines as required, on demand. Perform More Realistic Tests When testing a web application using infrastructure inside the firewall, testers cannot verify the entire delivery chain. Unless all end users will also be within the firewall, such tests are inherently limited and may fail to reveal all performance issues. 4 Neotys White Paper - Load Testing with the Cloud With the cloud, testers can execute load tests that access web applications as the users will—from outside the firewall—and validate all components of the delivery chain, including the firewall, DNS, network equipment, and ISP. Validating delivery chain components makes these tests more realistic. Further, they enable testers to evaluate the real-world effects of third-party components, such as content delivery networks, analytics servers, and ad servers. Because all users won't access the app from the same fixed location across the same network, a realistic load test cannot be completed from a single location. That's why it is important to test the application and its components from different locations and geographic regions and assess its performance as network bandwidth and latency changes. Load Testing Inside vs. Outside the Firewall Although the advantages of load testing in the cloud are clear, internal testing still has its place in the overall test plan, particularly when testing from outside the firewall is not feasible. Internal testing helps isolate effects that manifest from both the application and infrastructure by distinguishing them from those that are outside the firewall and potentially beyond the tester's control. Testing Inside the Firewall Some testing use cases preclude the use of the cloud. For example, an enterprise web application not designed to be accessed from the Internet will not benefit from load testing in the cloud. Likewise, a pre-production environment not set up for Internet access is best tested using an internal testing solution applied within the firewall. By using the same load testing solution for both internal testing and testing in the cloud, scripts developed for internal pre-production testing can be reused in production cloud-based testing. Using different tools for internal and cloud testing would not only require a rewrite of the scripts, but it would also increase licensing and training costs. Conducting Small Scale Tests Not all load testing requires hundreds of load generators. Even applications that anticipate many thousands of concurrent