In This Testing in Devops Eguide

Total Page:16

File Type:pdf, Size:1020Kb

In This Testing in Devops Eguide 3 Why Software Testing Is TESTING IN DEVOPS Key to DevOps eGuide 4 Test Everywhere: A In order to fully realize the potential of DevOps and a team’s ability to deliver higher-quality software faster and more securely, testing Journey into DevOps and Continuous Testing must be done continuously throughout the software development lifecycle. But continuous testing isn’t just about automation and tools; 7 there are cultural and process shifts that must happen, and everyone must be on the same page. This eGuide collects methodologies, processes, and tips to help your organization venture into continuous testing in DevOps. Shifting Your Testing: When to Switch Gears 11 In this Testing in DevOps eGuide Continuous Testing, Why Software Testing Is Key to DevOps Why You Need Continuous Testing in DevOps Shifting Left, and Test One of the major reasons organizations adopt DevOps practices is to accelerate DevOps is more than adopting the right set of tools; it’s a cultural shift that Automation: Getting It delivery of software to production. However, many fail to include quality compo- incorporates testing at each stage of the agile project lifecycle. Continuous test- Straight nents in their practices. Continuous deployment without quality is just delivering ing is key to unlocking this culture change because it weaves testing activities 12 continuous bugs. Here’s why software testing is an essential part of DevOps. into every part of the software design, development, and deployment process- es, which helps everyone involved communicate more, collaborate better, and Demystifying DevOps: Test Everywhere: A Journey into DevOps and innovate faster. A Day in the Life of a Continuous Testing DevOps Tester A move to DevOps creates an opportunity to shift the testing process to the Continuous Delivery Is Not a Pipeline left. But what if you went further? DevOps supports continuous testing, so you Pretty much everything you hear about DevOps mentions “the pipeline.” Con- 13 can advocate for a constant focus on quality, with testing permeating the entire tinuous delivery is not really about the pipeline, however. Continuous delivery is Why You Need Continuous software development process. Here’s how you can actually have a faster testing about two things: testing strategy and branching strategy. The pipeline is import- Testing in DevOps process when the software is tested throughout the lifecycle, by developers, ant; it is an integral part of DevOps. However, the central element is the practice testers, and automation alike. of testing continually using automated tests. 17 Continuous Delivery Is Shifting Your Testing: When to Switch Gears 5 Key Factors to Achieve Agile Testing in DevOps Not a Pipeline Shifting your testing either left or right can meet different needs and improve Part of the path to DevOps requires adoption of agile methodologies. What does different aspects. How do you know whether to make a change? Let your test cy- it mean for testing when you switch from the traditional waterfall model, with 19 cles be your guide. Just like when driving a car with a manual transmission, if the a few long release cycles per year, to the agile model, with changes occurring 5 Key Factors to Achieve engine starts to whine or you’re afraid you’re about to stall out, switching gears every two weeks? Here are five key factors to achieve the agile software testing Agile Testing in DevOps may be just what you need. necessary in DevOps. 23 Continuous Testing, Shifting Left, and Test Key Enablers for Continuous Testing Automation: Getting It Straight Continuous testing means testing before, during, and after each software change Key Enablers for is made. Testers have long advocated for this, but DevOps has made it more pop- Continuous Testing Continuous testing can help you achieve the optimal balance between speed and risk and deliver high-quality products faster. But what exactly does continuous ular by pushing for rapid feedback and shifting testing left in the lifecycle. Here 25 testing entail? Is it just shifting testing left in a DevOps environment? And where are three practices your company should embrace to enable continuous testing. does automation fit in? Here’s a breakdown of all these testing concepts. Insight from the Industry Insight from the Industry Demystifying DevOps: A Day in the Life of a DevOps 26 Additional Resources Tester Additional Resources The idea of working as a test specialist on a team using DevOps can be intimidat- ing. There are at least two technology stacks, containerization and continuous integration, that you need to be familiar with. But few people need to be able to start from scratch. Here’s what a normal day of testing in DevOps looks like. COPYRIGHT 2019 2 3 Why Software Testing Is TESTING IN DEVOPS Key to DevOps eGuide 4 Test Everywhere: A Journey into DevOps and Continuous Testing Why Software Testing Is Key to DevOps 7 By Alan Crouch Shifting Your Testing: When to Switch Gears One of the major reasons organizations adopt DevOps prac- into high-risk areas of the application, including interfac- 11 tices is to accelerate delivery of software to production. This es, misuse cases, and the most important assets to your Continuous Testing, includes deploying more frequently and reducing lead time. system. While not every test can (or should) be automated, Shifting Left, and Test spend your critical resources wisely by only focusing manual Automation: Getting It However, many organizations fail to include quality compo- efforts on the things that really require their attention. Straight nents in their practices. This leads to organizations deliv- ering code faster, but unfortunately, that code is just poor. Most organizations today use some level of automation when 12 Continuous deployment without quality is just delivering looking at performance, load, and security. Leveraging those Demystifying DevOps: continuous bugs to your customers. existing capabilities into your DevOps pipelines ensures that A Day in the Life of a your deployments are not held back by independent groups DevOps Tester If this sounds like your organization, software testing may late in the software lifecycle and they aren’t an afterthought 13 be the missing component to your DevOps program. Top when the application is already in production. This has im- Why You Need Continuous performing DevOps organizations, like Netflix, Amazon, and mediate benefits, such as reducing cyber security impacts on Testing in DevOps Etsy, utilize automated regression, performance, load, and your applications and their data by ensuring critical vulner- security testing to ensure software quality is built into their abilities are identified earlier in your software development 17 DevOps pipelines and ensured by being forced to be run on lifecycle and aren’t deployed to production in error. Continuous Delivery Is every build. For Netflix, this allows software to be commit- Not a Pipeline ted, tested, and fully deployed to production within sixteen minutes! Pulling software testing into your DevOps practices doesn’t have to 19 be difficult. You can start by bringing your existing testers and secu- If your organization isn’t as large or doesn’t require as rapid de- rity engineers into your planning sessions. Next, ensure each phase 5 Key Factors to Achieve ployments as those companies, automated testing in your DevOps of your pipeline has a quality gate and software quality criteria that Agile Testing in DevOps pipelines still provides significant benefits. Even a small amount of should be met in order to move to the next phase of your pipeline. 23 automated regression testing can ensure basic tests are always run Last, identify gaps between your quality goals and reality, and en- Key Enablers for on a build. Typically, organizations start by building an automated sure you prioritize efforts into the activities that provide the greatest Continuous Testing smoke test. This smoke test provides a sanity check on a developer’s return on investment—those that are either run with the highest code. This can not only reduce some of the overall manual testing frequency, reduce the greatest software quality risk, or reduce the 25 effort by the testing team, but also ensure effort isn’t being expend- biggest bottlenecks. Insight from the Industry ed on builds that don’t meet minimal quality standards. 26 These simple steps will lead to not just delivering code faster, but Additional Resources More robust automated testing suites can result in even less man- delivering better code faster. ual testing, in addition to better-focused exploratory testing efforts COPYRIGHT 2019 3 3 Why Software Testing Is TESTING IN DEVOPS Key to DevOps eGuide 4 Test Everywhere: A Journey into DevOps and Continuous Testing Insight from the Industry 7 Shifting Your Testing: When to Switch Gears “In shifting left, teams see “We’re not going faster, 11 huge benefits such as finding we’re not coding any Continuous Testing, Shifting Left, and Test bugs much earlier, sometimes even faster, we’re not typing any Automation: Getting It before code is ever written. This results faster. We’re learning good ways Straight in a higher quality of code and a far less to slice these features down, so 12 expensive way in achieving such.” we can deliver small increments.” Demystifying DevOps: A Day in the Life of a —Angie Jones —Lisa Crispin DevOps Tester 13 Why You Need Continuous READ THE FULL INTERVIEW LISTEN TO THE FULL INTERVIEW Testing in DevOps 17 Continuous Delivery Is Not a Pipeline GAIN MORE INSIGHT WITH THESE VIDEO INTERVIEWS 19 5 Key Factors to Achieve Agile Testing in DevOps 23 Key Enablers for DevOps Understanding Shift Your Deciding Continuous Continuous Testing Misconceptions the Role of QA Mindset with if You’re Testing and 25 and Testing in DevOps: An Your Testing: Technically Open Source Insight from the Industry Confidence: An Interview with An Interview Ready for Integration: An Interview with Gene Gotimer with Isabel DevOps: An Interview with 26 Hans Buwalda Evans Interview with Alex Martins Additional Resources Sunil Sehgal COPYRIGHT 2019 25.
Recommended publications
  • Devops and Testers White Paper
    White Paper DevOps and Testers DevOps is part of an overall approach that organizations use to deliver software frequently and with high quality. The most obvious outcome of successful DevOps implementations is the reduction in the time it takes for software changes to transition from an idea to production. What Does DevOps Mean If you are an experienced DevOps Automated tools and processes practitioner, we hope that you are used in system configuration, for Testers might still find the article useful. If the build process, testing, the Background you are not a tester, we hope that deployment to test, staging and The DevOps movement (for want you will at least see the tester’s production environments, post- of a better label) is progressing perspective. deployment monitoring, evaluation, rapidly. Like many other and operations. movements in the industry, the What Is DevOps? speed of adoption accelerates Is DevOps Just About faster than the definition of the Simplistically, DevOps is a label movement itself. DevOps is still to describe an ecosystem in Tools? not well defined and the nuances which development teams and At one level, the goal of DevOps of culture, the emergent capability systems operations teams work is to eliminate bottlenecks in of new technologies, and range of more closely together. In the the delivery pipeline through (mostly successful) case studies so-called delivery pipeline, from automation. But the automation means that the issues at hand are committing source code to putting of staged processes still still widely debated.1 code into production, developers requires governance. Most accommodate and automate some automated processes are not Depending on who you talk to, of operations activities.
    [Show full text]
  • IBM Developer for Z/OS Enterprise Edition
    Solution Brief IBM Developer for z/OS Enterprise Edition A comprehensive, robust toolset for developing z/OS applications using DevOps software delivery practices Companies must be agile to respond to market demands. The digital transformation is a continuous process, embracing hybrid cloud and the Application Program Interface (API) economy. To capitalize on opportunities, businesses must modernize existing applications and build new cloud native applications without disrupting services. This transformation is led by software delivery teams employing DevOps practices that include continuous integration and continuous delivery to a shared pipeline. For z/OS Developers, this transformation starts with modern tools that empower them to deliver more, faster, with better quality and agility. IBM Developer for z/OS Enterprise Edition is a modern, robust solution that offers the program analysis, edit, user build, debug, and test capabilities z/OS developers need, plus easy integration with the shared pipeline. The challenge IBM z/OS application development and software delivery teams have unique challenges with applications, tools, and skills. Adoption of agile practices Application modernization “DevOps and agile • Mainframe applications • Applications require development on the platform require frequent updates access to digital services have jumped from the early adopter stage in 2016 to • Development teams must with controlled APIs becoming common among adopt DevOps practices to • The journey to cloud mainframe businesses”1 improve their
    [Show full text]
  • Security Testing
    ISSN 1866-5705 www.testingexperience.com free digital version print version 8,00 € printed in Germany 6 The Magazine for Professional Testers The MagazineforProfessional Security Testing © iStockphoto/alexandru_lamba June, 2009 © iStockphoto/Manu1174 ISTQB® Certified Tester Foundation Level for only 499,- € plus VAT ONLINE TRAINING English & German www.testingexperience.learntesting.com Certified Tester Advanced Level coming soon Editorial Dear readers, One of my professors at the university said once to all of us: Computer scientists are at some point criminals. What he meant was that we or some of us – computer scientists – at some point like to try things that are not that “legal”. The most of us are “clean”, but some of us are “free time hackers”! Nowadays the hackers are almost away from the 17 years old guy, trying to pen- etrate in some website and so on. They are now adults, with families, cars, pets, holidays and a job. They are professionals earning money for acting as such. Application Security is not only important and essential for the companies and their businesses, technology and employees. Application Security is a macroeco- nomic aspect for the countries. There are a lot of secret services or governments agencies working on getting technology or information by advance hacking the server and databases of top companies or governments worldwide. When we hear that some countries could be behind the penetration of the USA electricity net- work, you can imagine what is going on outside. Are we testers prepared for that job? I’m not! Last year we had the first tutorial by Manu Cohen about Application Security Testing.
    [Show full text]
  • Designing Software Architecture to Support Continuous Delivery and Devops: a Systematic Literature Review
    Designing Software Architecture to Support Continuous Delivery and DevOps: A Systematic Literature Review Robin Bolscher and Maya Daneva University of Twente, Drienerlolaan 5, Enschede, The Netherlands [email protected], [email protected] Keywords: Software Architecture, Continuous Delivery, Continuous Integration, DevOps, Deployability, Systematic Literature Review, Micro-services. Abstract: This paper presents a systematic literature review of software architecture approaches that support the implementation of Continuous Delivery (CD) and DevOps. Its goal is to provide an understanding of the state- of-the-art on the topic, which is informative for both researchers and practitioners. We found 17 characteristics of a software architecture that are beneficial for CD and DevOps adoption and identified ten potential software architecture obstacles in adopting CD and DevOps in the case of an existing software system. Moreover, our review indicated that micro-services are a dominant architectural style in this context. Our literature review has some implications: for researchers, it provides a map of the recent research efforts on software architecture in the CD and DevOps domain. For practitioners, it describes a set of software architecture principles that possibly can guide the process of creating or adapting software systems to fit in the CD and DevOps context. 1 INTRODUCTION designing new software architectures tailored for CD and DevOps practices. The practice of releasing software early and often has For clarity, before elaborating on the subject of been increasingly more adopted by software this SLR, we present the definitions of the concepts organizations (Fox et al., 2014) in order to stay that we will address: Software architecture of a competitive in the software market.
    [Show full text]
  • Devops Point of View an Enterprise Architecture Perspective
    DevOps Point of View An Enterprise Architecture perspective Amsterdam, 2020 Management summary “It is not the strongest of the species that survive, nor the most intelligent, but the one most responsive to change.”1 Setting the scene Goal of this Point of View In the current world of IT and the development of This point of view aims to create awareness around the IT-related products or services, companies from transformation towards the DevOps way of working, to enterprise level to smaller sizes are starting to help gain understanding what DevOps is, why you need it use the DevOps processes and methods as a part and what is needed to implement DevOps. of their day-to-day organization process. The goal is to reduce the time involved in all the An Enterprise Architecture perspective software development phases, to achieve greater Even though it is DevOps from an Enterprise Architecture application stability and faster development service line perspective, this material has been gathered cycles. from our experiences with customers, combined with However not only on the technical side of the knowledge from subject matter experts and theory from organization is DevOps changing the playing within and outside Deloitte. field, also an organizational change that involves merging development and operations teams is Targeted audience required with an hint of cultural changes. And last but not least the skillset of all people It is specifically for the people within Deloitte that want to involved is changing. use this as an accelerator for conversations and proposals & to get in contact with the people who have performed these type of projects.
    [Show full text]
  • Continuous Testing for Devops Evolving Beyond Simple Automation
    Technical Whitepaper 1 Continuous Testing for DevOps Evolving Beyond Simple Automation INTRODUCTION DevOps represents a cultural shift that stresses collaboration be- on acceleration. Moreover, adopting a bona fide Continuous Testing tween the business, developers, and IT professionals. Software test process (more than just automated tests running regularly) helps automation can enhance these connections and help organizations promote all of the core pillars of DevOps: Culture, Automation, Lean, achieve desired SDLC acceleration, but it’s important to recognize Metrics, and Sharing. that automation is just one piece of the DevOps puzzle. In this paper, we’ll explore why and how Continuous Testing’s real- Since testing is often one of the greatest constraints in the SDLC, time objective assessment of an application’s business risks is a optimizing quality processes to allow testing to begin earlier, as well critical component of DevOps. as shrink the amount of testing required, can have a marked impact DEVOPS PRINCIPLES There are several key pieces to understanding DevOps revolutions and they are often brought about by a compelling event at an organization, such as a shift to agile. As organizations start to move into an agile development methodology, they start to uncover other processes that can be accelerated, such as delivery by DevOps and testing by Continuous Testing. The acceleration that is set in motion via agile makes it necessary to accelerate the release schedule. In order to ensure a successful release, an organization must adopt continuous testing to make sure the conveyer belt does not break down. The modernization maturity model has these three distinct phases: AGILE Agile software development is a different way of thinking about approaching the challenge of development time.
    [Show full text]
  • Software Engineering Using Devops - a Silver Bullet?
    UPTEC IT 19 002 Examensarbete 30 hp Januari 2019 Software Engineering using DevOps - a Silver Bullet? Mikaela Eriksson Institutionen för informationsteknologi Department of Information Technology Abstract Software Engineering using DevOps - a Silver Bullet? Mikaela Eriksson Teknisk- naturvetenskaplig fakultet UTH-enheten Today we have technology that help us scan millions of medical databases in a glimpse of an eye and self-driving cars that are outperforming humans at driving. Besöksadress: Technology is developing so fast that new updates in the technology world are Ångströmlaboratoriet Lägerhyddsvägen 1 commonplace to us and we are more often frustrated in case something is not up Hus 4, Plan 0 to speed. Technology is moving so quickly and in order for humans to keep up with the development needed in the tech business, different methodologies for how to Postadress: optimise the development process have been applied, some that work better than Box 536 751 21 Uppsala others. But just as fast as the technology changes, the methodologies used change with them. Recently a new term has entered the methodologies field. This Telefon: term is said to bring faster deployment, decreased failures and improved the 018 – 471 30 03 loyalties within the teams. The term in question, is called DevOps. Telefax: 018 – 471 30 00 This study is about uncovering the world of DevOps. This thesis is exploring the term in real teams in order to find out whether or not DevOps is the silver bullet it Hemsida: makes out to be. The study is based on ten interviews with people at different http://www.teknat.uu.se/student organisations, using DevOps, and will find out how these interviewees use and feel about DevOps.
    [Show full text]
  • Software Development a Practical Approach!
    Software Development A Practical Approach! Hans-Petter Halvorsen https://www.halvorsen.blog https://halvorsen.blog Software Development A Practical Approach! Hans-Petter Halvorsen Software Development A Practical Approach! Hans-Petter Halvorsen Copyright © 2020 ISBN: 978-82-691106-0-9 Publisher Identifier: 978-82-691106 https://halvorsen.blog ii Preface The main goal with this document: • To give you an overview of what software engineering is • To take you beyond programming to engineering software What is Software Development? It is a complex process to develop modern and professional software today. This document tries to give a brief overview of Software Development. This document tries to focus on a practical approach regarding Software Development. So why do we need System Engineering? Here are some key factors: • Understand Customer Requirements o What does the customer needs (because they may not know it!) o Transform Customer requirements into working software • Planning o How do we reach our goals? o Will we finish within deadline? o Resources o What can go wrong? • Implementation o What kind of platforms and architecture should be used? o Split your work into manageable pieces iii • Quality and Performance o Make sure the software fulfills the customers’ needs We will learn how to build good (i.e. high quality) software, which includes: • Requirements Specification • Technical Design • Good User Experience (UX) • Improved Code Quality and Implementation • Testing • System Documentation • User Documentation • etc. You will find additional resources on this web page: http://www.halvorsen.blog/documents/programming/software_engineering/ iv Information about the author: Hans-Petter Halvorsen The author currently works at the University of South-Eastern Norway.
    [Show full text]
  • Devops Engineer
    DevOps Engineer The IHMC Robotics group is looking to fill a DevOps position. The ideal candidate is somebody who is excited about bringing our lab up to speed on how to best use modern solutions for making our software development and release process super repeatable, reliable, and painless. This is a full-time position. We are huge proponents of strong software engineering practices, leveraging the philosophies of test-driven development, continuous delivery/integration, agile programming, repeatable deployments, and many others that make managing diverse teams and complex software easier. Specifically, the DevOps engineer we’re looking for is awesome at: • Automation. Of everything. An obsession with making every aspect of the development and deployment pipeline painless, testable, repeatable, and reliable is a must-have. • Provisioning and configuration management using tools like Puppet, Chef, Ansible, Salt, etc. The tool of choice is open to discussion, but being comfortable with a provisioning oriented workflow is a must. • Automated Continuous Integration and Continuous Delivery. As a Java shop, we use Gradle as our build system. We are just now beginning to migrate to a full continuous delivery workflow; our builds are becoming more automated with regards to artifact generation, artifact hosting, and integration with tools like Docker. The ideal candidate would be willing to grow a system like this in to something with high reliability so it can be a cornerstone of our open source software workflow. Being test-obsessed is desired, both for making sure our CI stack is doing its job and making sure that your infrastructure automation is just as well tested.
    [Show full text]
  • Agile Automated Software Testing Into Automotive V-Model Process
    Agile automated software testing into automotive V-Model process: A practical case Xavier Martin Artal Software QA Manager [email protected] es.linkedin.com/pub/xavier-martin/6/a89/723/ Agenda • Introduction • Automotive Trends: Car Connectivity • Car Telematics project Challenges • Use Case Solution: From V-Model to Agile Testing • Results and Conclusions Introduction What is this presentation about? • Expose a practical case of adoption of Agile techniques in automotive testing • Converge Spice automotive V-Model to Agile Spice V-Model Agile • Present Technical Solution adopted: Automation Test Framework • Discuss results and Agile adequacy to Automotive industry Automotive Trends: Vehicle Connectivity Car Telematics • Car Manufacturers start to add 3G/4G capabilities • Connectivity opens new opportunities to develop services for both clients and manufacturers Connectivity Services – Emergency Call – Fleet Management – Car Sharing – Remote Car Diagnostics – Stolen Vehicle Tracking (SVT) – WOTA Update – Dealer Services – User Premium Services Car telematics: eCall • Emergency Call Service for Europe • U.E Council proposes eCall obligatory in European Cars for end 2017 • Automatic call in case of accident or emergency will force car manufacturers to add IVTU to every new car for European Service • Similar regulations for Russia, USA, BRA and PRC Car Telematics Project Challenges What is an iVTU? iVTU = in Vehicle Telematics Unit - Electronic Unit in charge of granting 2G/3G/LTE connectivity to vehicles - Two Main processors architecture:
    [Show full text]
  • Introduction to Devops on AWS
    Introduction to DevOps on AWS October 2020 Notices Customers are responsible for making their own independent assessment of the information in this document. This document: (a) is for informational purposes only, (b) represents current AWS product offerings and practices, which are subject to change without notice, and (c) does not create any commitments or assurances from AWS and its affiliates, suppliers or licensors. AWS products or services are provided “as is” without warranties, representations, or conditions of any kind, whether express or implied. The responsibilities and liabilities of AWS to its customers are controlled by AWS agreements, and this document is not part of, nor does it modify, any agreement between AWS and its customers. © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved. Contents Introduction .......................................................................................................................... 1 Continuous Integration ........................................................................................................ 2 AWS CodeCommit ........................................................................................................... 2 AWS CodeBuild ................................................................................................................ 3 AWS CodeArtifact ............................................................................................................ 3 Continuous Delivery ...........................................................................................................
    [Show full text]
  • Contracting for Agile and Devops
    2019 AN EVEREST GROUP VIEWPOINT R Contracting for Agile and DevOps Recommended Practice for Sourcing Yugal Joshi, Vice President Executives Achint Arora, Practice Director Abhishek Mundra, Senior Analyst Copyright © 2019, Everest Global, Inc. All rights reserved. www.everestgrp.com EGR-2019-32-V-3071 2019 CONTRACTING FOR AGILE AND DEVOPS The “as-is” state of Agile contracting Only 27% of firms have Existing adoption reported more than 50% It has been nearly 20 years since the Manifesto for Agile Software Development was of Agile adoption across released; since that time, four key values and 12 principles that form its bedrock have software teams been instrumental in revolutionizing product delivery across industries and organizations. However, even after almost two decades, many enterprises struggle to scale up Agile adoption – in fact, it is only the last two to three years that we have begin to see contracts customized to an Agile delivery model. In an Everest Group research with over 150 C-level executives, we found: ⚫ 46% of the respondents reported that fewer than 30% of their software development teams leverage Agile methodology, which clearly highlights scalability challenges ⚫ 31% of respondents reported challenges in contracting for something that is difficult to define ⚫ 22% indicated that they had trouble aligning on the appropriate performance metrics Our research also suggests that more than 80% of organizations consider themselves to have low maturity in Agile delivery. To address these challenges, enterprises are looking to third-party service providers to help them move toward enterprise Agile delivery. EXHIBIT 1 Agile adoption across enterprises Source: Everest Group Over 90% of organizations are using Agile development methods in some shape or form However, less than 20% organizations consider themselves to have high levels of maturity in the use of Agile In its current form, an Agile contract is shaped in alignment with the delivery methodology and generally consists of the components described ahead.
    [Show full text]