White Paper DevOps and Testers DevOps is part of an overall approach that organizations use to deliver software frequently and with high quality. The most obvious outcome of successful DevOps implementations is the reduction in the time it takes for software changes to transition from an idea to production. What Does DevOps Mean If you are an experienced DevOps Automated tools and processes practitioner, we hope that you are used in system configuration, for Testers might still find the article useful. If the build process, testing, the Background you are not a tester, we hope that deployment to test, staging and The DevOps movement (for want you will at least see the tester’s production environments, post- of a better label) is progressing perspective. deployment monitoring, evaluation, rapidly. Like many other and operations. movements in the industry, the What Is DevOps? speed of adoption accelerates Is DevOps Just About faster than the definition of the Simplistically, DevOps is a label movement itself. DevOps is still to describe an ecosystem in Tools? not well defined and the nuances which development teams and At one level, the goal of DevOps of culture, the emergent capability systems operations teams work is to eliminate bottlenecks in of new technologies, and range of more closely together. In the the delivery pipeline through (mostly successful) case studies so-called delivery pipeline, from automation. But the automation means that the issues at hand are committing source code to putting of staged processes still still widely debated.1 code into production, developers requires governance. Most accommodate and automate some automated processes are not Depending on who you talk to, of operations activities. Operations really autonomous; they cannot DevOps can be a solution to a has more visibility into, and some complete their tasks without problem or a goal in itself. In some influence over, the activities of human intervention in maintenance businesses, the goal is going developers. The motivation for or in handling exceptions. A fully digital, and DevOps is part of the this is primarily to speed up the automated DevOps process is overall approach to delivering deployment and implementation meaningless without consideration frequently and with high quality. of software. Bringing Dev and of the human factor. Although tools But in the marketing of DevOps- Ops closer together in an agile do a lot of heavy-lifting, it is the related technologies and services, team effectively implements agile people running the process that this goal can be obscured. The operations. make it work or fail. challenge of making the cultural change (or more concretely, The most obvious outcome of Is DevOps Just Dev and Ops making the behavioral change) successful DevOps implementations Working More Closely with the Aid required for success is frequently is the reduction in the time it takes of Tools? underestimated. for software changes to transition No, DevOps is not that either. from an idea to production. When a Often, the testers involved in and developer says a software change is The hand-offs between automated affected by DevOps are new to done, the transition to production is processes often involve other the whole idea. This article is an performed with the aid of pervasive processes, usually testing of one introduction to DevOps for these automation. kind or another. testers and is a discussion of its impact on test practices. 1 “What is DevOps,” The Agile Admin, http://theagileadmin. com/what-is-devops/ DevOps and Testers White Paper If there is one mantra that best If there are manual interventions, It might be painful describes the drive towards typically between the automated DevOps it is, “If it hurts, do it stages in the DevOps process, we for you, as a tester, to more.” It might be a bit of a cliché, will see these as the pain points. trust developers and but use that as the context for They are bottlenecks, the causes implementing and improving of delays, and the potentially less automation to do the DevOps test practices. reliable and error-prone aspects of testing job properly. If the process. it hurts, you must do it If It Hurts, Do It More Manual testing is painful. Yes, you more often. The difficulty or pain we experience might love exploratory testing; when doing a particular job and you might fear that only you, influences us adversely. If we do not as a human, can find those gnarly like to do a task, we tend to put it bugs that automation will never Is DevOps Just About off. When we finally take the task find, that you as the tester are the on, it is more painful. This is true only person trustworthy enough to Tools? (cont.) for visiting the dentist, cleaning the prevent disaster happening. Automated tests are created garage, integrating software, doing by developers and testers. The testing, and so on. It might be painful for you, as a output of these tests is focused on tester, to trust developers and providing sufficient information to Our experience is commonly that automation to do the testing job other processes, or just as often the less frequently we perform a properly. If it hurts, you must do it to people, to make transitions task, the more traumatic the task more often. between stages in the pipeline. is when we actually do it. Martin Fowler suggests three reasons why Tests, Automation, and Testers and developers who the frequent or even continuous test provide the assurance that execution of certain tasks reduces Trust the DevOps process delivers the pain.3 There is much debate around the successfully and reliably. meaning of checking and testing4, • First, larger, more complex tasks and the reliance we can place are hard to plan, manage, and What is DevOps, Really? on testers, on checks, and on control. Breaking up large tasks automation5, 6. The question is posed and makes them easier to do, less discussed at length in an excellent risky to do and, if something Can we place all our faith in post on the Agile Admin, “What is does go wrong, easier to unwind. automated checks? 2 DevOps” . DevOps is an evolving, • Second, many tasks (and testing We certainly need more emergent discipline. So the is the shining example) provide sophistication than that. But definition of DevOps is still not feedback. Feedback, if received we can, for the purpose of this settled. Perhaps it never will be. early and often, helps us to article, at least separate tests and rapidly address problems before What does that mean for testers? It test execution activity into four any further time is wasted. means that there is still no one true components. way and that your role in a DevOps • Thirdly, if we practice an activity regime that is evolving (and every more frequently, we get better • Checks that can be automated regime is evolving) is not yet fixed. at it. We learn how to do it by developers as part of their There are two main contributions efficiently. We might also see component-level check-in and you can make: opportunities to automate it in continuous integration processes. some way. • Checks that can be automated • You need to pay attention to From the tester’s perspective, the (typically by system testers) to the things that hurt and work to mantra, “It it hurts, do it more,” exercise API-level, link, or end-to- make them less painful. forces us to take the notion of end transactions. • You need to identify the automation in the testing process opportunities and interventions much more seriously. that will add value to the DevOps 4 “Testing and Checking Refined,” James Bach, Michael process. Bolton, http://www.satisfice.com/blog/archives/856 5 “A New Model for Testing,” Paul Gerrard, http://dev.sp.qa/ download/newModel 2 “What is DevOps,” The Agile Admin, http://theagileadmin. 3 “Frequency Reduces Difficulty,” Martin Fowler, http:// 6 “The New Model and Testing v Checking,” Paul Gerrard, com/what-is-devops/ martinfowler.com/bliki/FrequencyReducesDifficulty.html http://blog.gerrardconsulting.com/?q=node/659 DevOps and Testers White Paper Tests, Automation, and • Which checks need only be run once, manually, as opposed to Certainly, you have to Trust (cont.) checks that need to be retained • Tests that can perform for regression purposes, and are stop believing you are compatibility checks to candidates for automation? the gatekeeper of quality, demonstrate compatibility across • We mentioned the notion of browsers, operating systems, and the last defense, the only trust previously. Another way of platforms. looking at this is to speculate on person who cares. You • Tests that can only be performed how a system could be reliably have to think more like a by a human. tested if there were no late In this article, we offer only a manual testing at all. visionary, risk-identifier, few suggestions on how to make • Imagine an environment where risk manager, pathfinder, these distinctions because every all of the testing was done by environment is different, of course. tools. Would your concerns facilitator, and as a coach be dominated by the fact that and mentor. The more germane question to ask you simply do not trust the is, how does the tester let go of developers to do a good job of 7 late, manual checking? Letting go testing? Moving testing thinking One goal of monitoring is to raise requires proactive effort and trust. to the left should reduce the alerts on failure before users doubt. experience the impact of them. This These areas will be the main focus is rather ambitious, but this is the of your efforts: • If, as a tester, you act more as a pathfinder to identify risks ultimate goal.