3 Why Software Testing Is TESTING IN DEVOPS Key to DevOps eGuide 4 Test Everywhere: A In order to fully realize the potential of DevOps and a team’s ability to deliver higher-quality software faster and more securely, testing Journey into DevOps and Continuous Testing must be done continuously throughout the software development lifecycle. But continuous testing isn’t just about automation and tools; 7 there are cultural and process shifts that must happen, and everyone must be on the same page. This eGuide collects methodologies, processes, and tips to help your organization venture into continuous testing in DevOps. Shifting Your Testing: When to Switch Gears 11 In this Testing in DevOps eGuide Continuous Testing, Why Software Testing Is Key to DevOps Why You Need Continuous Testing in DevOps Shifting Left, and Test One of the major reasons organizations adopt DevOps practices is to accelerate DevOps is more than adopting the right set of tools; it’s a cultural shift that Automation: Getting It delivery of software to production. However, many fail to include quality compo- incorporates testing at each stage of the agile project lifecycle. Continuous test- Straight nents in their practices. Continuous deployment without quality is just delivering ing is key to unlocking this culture change because it weaves testing activities 12 continuous bugs. Here’s why software testing is an essential part of DevOps. into every part of the software design, development, and deployment process- es, which helps everyone involved communicate more, collaborate better, and Demystifying DevOps: Test Everywhere: A Journey into DevOps and innovate faster. A Day in the Life of a Continuous Testing DevOps Tester A move to DevOps creates an opportunity to shift the testing process to the Continuous Delivery Is Not a Pipeline left. But what if you went further? DevOps supports continuous testing, so you Pretty much everything you hear about DevOps mentions “the pipeline.” Con- 13 can advocate for a constant focus on quality, with testing permeating the entire tinuous delivery is not really about the pipeline, however. Continuous delivery is Why You Need Continuous software development process. Here’s how you can actually have a faster testing about two things: testing strategy and branching strategy. The pipeline is import- Testing in DevOps process when the software is tested throughout the lifecycle, by developers, ant; it is an integral part of DevOps. However, the central element is the practice testers, and automation alike. of testing continually using automated tests. 17 Continuous Delivery Is Shifting Your Testing: When to Switch Gears 5 Key Factors to Achieve Agile Testing in DevOps Not a Pipeline Shifting your testing either left or right can meet different needs and improve Part of the path to DevOps requires adoption of agile methodologies. What does different aspects. How do you know whether to make a change? Let your test cy- it mean for testing when you switch from the traditional waterfall model, with 19 cles be your guide. Just like when driving a car with a manual transmission, if the a few long release cycles per year, to the agile model, with changes occurring 5 Key Factors to Achieve engine starts to whine or you’re afraid you’re about to stall out, switching gears every two weeks? Here are five key factors to achieve the agile software testing Agile Testing in DevOps may be just what you need. necessary in DevOps. 23 Continuous Testing, Shifting Left, and Test Key Enablers for Continuous Testing Automation: Getting It Straight Continuous testing means testing before, during, and after each software change Key Enablers for is made. Testers have long advocated for this, but DevOps has made it more pop- Continuous Testing Continuous testing can help you achieve the optimal balance between speed and risk and deliver high-quality products faster. But what exactly does continuous ular by pushing for rapid feedback and shifting testing left in the lifecycle. Here 25 testing entail? Is it just shifting testing left in a DevOps environment? And where are three practices your company should embrace to enable continuous testing. does automation fit in? Here’s a breakdown of all these testing concepts. Insight from the Industry Insight from the Industry Demystifying DevOps: A Day in the Life of a DevOps 26 Additional Resources Tester Additional Resources The idea of working as a test specialist on a team using DevOps can be intimidat- ing. There are at least two technology stacks, containerization and continuous integration, that you need to be familiar with. But few people need to be able to start from scratch. Here’s what a normal day of testing in DevOps looks like. COPYRIGHT 2019 2 3 Why Software Testing Is TESTING IN DEVOPS Key to DevOps eGuide 4 Test Everywhere: A Journey into DevOps and Continuous Testing Why Software Testing Is Key to DevOps 7 By Alan Crouch Shifting Your Testing: When to Switch Gears One of the major reasons organizations adopt DevOps prac- into high-risk areas of the application, including interfac- 11 tices is to accelerate delivery of software to production. This es, misuse cases, and the most important assets to your Continuous Testing, includes deploying more frequently and reducing lead time. system. While not every test can (or should) be automated, Shifting Left, and Test spend your critical resources wisely by only focusing manual Automation: Getting It However, many organizations fail to include quality compo- efforts on the things that really require their attention. Straight nents in their practices. This leads to organizations deliv- ering code faster, but unfortunately, that code is just poor. Most organizations today use some level of automation when 12 Continuous deployment without quality is just delivering looking at performance, load, and security. Leveraging those Demystifying DevOps: continuous bugs to your customers. existing capabilities into your DevOps pipelines ensures that A Day in the Life of a your deployments are not held back by independent groups DevOps Tester If this sounds like your organization, software testing may late in the software lifecycle and they aren’t an afterthought 13 be the missing component to your DevOps program. Top when the application is already in production. This has im- Why You Need Continuous performing DevOps organizations, like Netflix, Amazon, and mediate benefits, such as reducing cyber security impacts on Testing in DevOps Etsy, utilize automated regression, performance, load, and your applications and their data by ensuring critical vulner- security testing to ensure software quality is built into their abilities are identified earlier in your software development 17 DevOps pipelines and ensured by being forced to be run on lifecycle and aren’t deployed to production in error. Continuous Delivery Is every build. For Netflix, this allows software to be commit- Not a Pipeline ted, tested, and fully deployed to production within sixteen minutes! Pulling software testing into your DevOps practices doesn’t have to 19 be difficult. You can start by bringing your existing testers and secu- If your organization isn’t as large or doesn’t require as rapid de- rity engineers into your planning sessions. Next, ensure each phase 5 Key Factors to Achieve ployments as those companies, automated testing in your DevOps of your pipeline has a quality gate and software quality criteria that Agile Testing in DevOps pipelines still provides significant benefits. Even a small amount of should be met in order to move to the next phase of your pipeline. 23 automated regression testing can ensure basic tests are always run Last, identify gaps between your quality goals and reality, and en- Key Enablers for on a build. Typically, organizations start by building an automated sure you prioritize efforts into the activities that provide the greatest Continuous Testing smoke test. This smoke test provides a sanity check on a developer’s return on investment—those that are either run with the highest code. This can not only reduce some of the overall manual testing frequency, reduce the greatest software quality risk, or reduce the 25 effort by the testing team, but also ensure effort isn’t being expend- biggest bottlenecks. Insight from the Industry ed on builds that don’t meet minimal quality standards. 26 These simple steps will lead to not just delivering code faster, but Additional Resources More robust automated testing suites can result in even less man- delivering better code faster. ual testing, in addition to better-focused exploratory testing efforts COPYRIGHT 2019 3 3 Why Software Testing Is TESTING IN DEVOPS Key to DevOps eGuide 4 Test Everywhere: A Journey into DevOps and Continuous Testing Insight from the Industry 7 Shifting Your Testing: When to Switch Gears “In shifting left, teams see “We’re not going faster, 11 huge benefits such as finding we’re not coding any Continuous Testing, Shifting Left, and Test bugs much earlier, sometimes even faster, we’re not typing any Automation: Getting It before code is ever written. This results faster. We’re learning good ways Straight in a higher quality of code and a far less to slice these features down, so 12 expensive way in achieving such.” we can deliver small increments.” Demystifying DevOps: A Day in the Life of a —Angie Jones —Lisa Crispin DevOps Tester 13 Why You Need Continuous READ THE FULL INTERVIEW LISTEN TO THE FULL INTERVIEW Testing in DevOps 17 Continuous Delivery Is Not a Pipeline GAIN MORE INSIGHT WITH THESE VIDEO INTERVIEWS 19 5 Key Factors to Achieve Agile Testing in DevOps 23 Key Enablers for DevOps Understanding Shift Your Deciding Continuous Continuous Testing Misconceptions the Role of QA Mindset with if You’re Testing and 25 and Testing in DevOps: An Your Testing: Technically Open Source Insight from the Industry Confidence: An Interview with An Interview Ready for Integration: An Interview with Gene Gotimer with Isabel DevOps: An Interview with 26 Hans Buwalda Evans Interview with Alex Martins Additional Resources Sunil Sehgal COPYRIGHT 2019 25.