QA in Devops Why a Great Development Team Isn’T Enough?

Home , Functional testing, Regression testing

Reimagine technology to accelerate your business

QA in DevOps Why a great development team isn’t enough?

Vinai Gangireddy, Director & Practice Leader for QA & Testing DevOps Introduction DevOps is a set of practices that combines development, QA, and IT operations to reduce time to market. It leverages processes and tools that automate the software development life cycle and support accelerated project delivery goals. The popularity of a DevOps approach is due to its ability to build and improve products at a faster pace, compared to traditional software development methodologies.

Key outcomes of DevOps include:

• Shorter development cycles with continuous delivery and innovation • Reduced implementation failures • Tighter feedback loops and quicker recovery time • Less complexity to manage change • Improved communication and collaboration

Traditional QA vs DevOps QA Traditionally, the QA team receives a product build from the development team. The build is placed in the designated testing environment where the QA team conducts functional and regression testing. When the build has been veri ed by the QA team, it is released to production. This process typically takes between a few days and a few weeks to complete.

In the DevOps methodology, throughout the life cycle, developers and quality analysts share responsibility for properly functioning software and timely delivery. Quality functions are integrated in a collaborative approach to development, augmented by tools and infrastructure that ensure a fast and seamless process. Builds are delivered continuously to the testing environment, where developers and testers work collaboratively to increase product quality.

Critical rules for QA in a DevOps process:

• QA analysts must validate requirements during the design process and before coding is started as the clari cation process will almost certainly change the code and may also change the architecture. • QA tools, templates, and processes must be customized to align to the DevOps methodology. • Automation of both unit and functional test cases must be employed to the maximum possible code coverage, minimizing manual testing. • Deployment to all environments is automated • Automated test cases should be properly aligned with the Continuous Integration Cycle • The continuous planning aspect of DevOps requires continuous engagement with product owners

QA in DevOps | White Paper 2 Shift-Left QA in DevOps

DEV QA UAT

Code Progression Through Environments PROD

Intake Requirements Process Functional/Integration/ Backlog/Intake Development/ Business/User Continuous Testing Process Continuous Integration Acceptance

Defects/ Change Requests

Automated Acceptance Test Functional & Regression Solution Design Use Case Review End -to-End Validation Design Test Estimations Build Automation Suite Code Reviews Architecture Test Planning Performance Testing Code Analysis Test Execution Process Flow Acceptance Criteria HLD Code Coverage Defect Logging Tech Specs Test Data Prep Defect Reviews Use Cases Automated Unit Tests User Acceptance Testing Defect Triage Test Plans Test Framework Defect Reviews Root Cause Analysis Test Automation Scripts Defect Triage & Metrics Test Reports Reports Early Performance Tests Interface/API Root Cause Analysis Test Sign Off Integration Tests CR Details Metrics Test Reports & Sign Off Test Sign Off

Team Utilizing Tools and Frameworks

A key aspect of DevOps is the ability to automate the build, and limitations of various frameworks and know intuitively deploy, and smoke test process. This eliminates human errors which one to choose based on different application factors. we frequently see in these steps, while also signi cantly They also understand how to maximize the value of increasing the speed. Specialized automation and continuous automation and minimize future maintenance, what should integration tools help achieve this goal. be automated, and where automation should be avoided. • A big risk in fast-paced DevOps development is that In addition, QA functions are integrated throughout the releases which should focus on new functionality may development life cycle, from design to deployment. QA analysts include rework from previously missed requirements and validate the clarity of requirements by decomposing them into defect xes, which will ultimately slow the pace of overall test scenarios, collaborate with developers to create automated development. Employing the skills of quality professionals unit tests, prepare functional and regression automated and throughout the life cycle not only protects against this manual tests, and coordinate with developers to include smoke possibility, but also supports the concept of ongoing tests in automated build/deploy processes. improvements so that a DevOps team can become increasingly more ef cient and more effective in producing The specialized training, experience, and mindset of QA high quality software at a very rapid pace. analysts/testers enable critical success factors:

• Understanding exactly where requirements may be Ciber’s Approach for QA in DevOps misinterpreted or assumptions made by development or if The key QA aspect to consider in DevOps is that the testers are there are any ambiguities that need clari cation teamed with the developers. In this way, the QA team ensures • Since it is impossible to test every business scenario, quality throughout the development life cycle, de ning tangible knowing where to focus, what to validate, and what test aspects of quality that are expected from the application. This data provides the highest risk coverage are some of the helps to achieve timely results. key skills leveraged by quality professionals. Another important action is de ning quality metrics, which • There can be no DevOps without automation, and great helps teams identify defects earlier. The metrics can be automation goes far beyond knowing how to write code. integrated with the organizational culture, ensuring quality at all Professional automation testers understand the advantages levels.

QA in DevOps | White Paper 3 While testers can see the developer's perspective, they maintain their tester mindset that supports objective testing and avoids the limitations of author bias. Working directly with the developer, the tester can x the code or assist the developer with technical insights.

By working together, both developers and testers are able to start end-to-end testing long before all of the code is written. The developers create special drivers that simulate missing code, while the testers are mentored to simulate real-life business scenarios in pre-production environments. This enables bug detection much earlier in the development process — long before the full product development is completed.

Performance, load, and security testing is also started early in the development cycle. The non-functional testing team operates as a shared service, integrating with the various application teams to identify performance, load, and security issues early in the development process.

Members of Ciber's Enterprise Quality Assurance and Testing Practice are cross trained with automation skills and DevOps tools, which are extensively implemented in all our DevOps projects. Our library of CI/CD frameworks enables fast setup for our customers and quick onboarding of team members. In addition, Ciber has partnerships with leading vendors of quality assurance analysis and automation software.

Conclusion Ciber embraces the idea that ‘DevOps is a culture, not a team.’ While addressing customer demands and shortening lead time between design and deploy, DevOps QA has a larger role to play. DevOps can speed up development but having a solid QA strategy helps prevent defect creation and creates a better production product.

At the same time, building a self-suf cient QA team with strong skills to support the DevOps process at all levels is an important objective. Automation is used throughout the life cycle and a proven framework is required to support continuous integration and a continuous delivery model.

In one case study, Ciber partnered with a leading hospitality provider in an enterprise initiative to migrate all their applications to a micro services platform. Ciber implemented leading CI/CD tools and customized CI/CD ow involving development, code repository and builds, and deployments. Today, they have executed over 15 production releases, selectively activating code with feature ags depending on application readiness. Bug introduction is very low and virtually all identi ed bugs are xed the same day they are found, signi cantly reducing development costs.

QA in DevOps | White Paper 4 About The Author

Vinai Gangireddy Director & Practice Leader for QA & Testing

Vinai Gangireddy leads the Enterprise Quality Assurance and Testing Practice (EQAT) at Ciber Global with over 20 years of testing experience. He has effectively executed multiple software testing projects. Working in different technology stacks and domains, he has assisted numerous clients in implementing quality best practices and testing solutions that have resulted in high-quality software.

QA in DevOps | White Paper 5 3270 West Big Beaver Road | Troy MI 48084