Importance of Continuous Testing in Devops

Importance of Continuous testing in DevOps

Agenda

1. History and current status of DevOps

1.1 What is DevOps? 1.2 How DevOps works: 2. Benefits of DevOps for Enterprise IT 3. Continuous Testing 3.1What is Continuous Testing? 3.2. Continuous Testing in DevOps 4. DevOps at a Glance 5. References & Appendix

Abstract

DevOps –It’s All about Continuous Testing and monitoring the feedback loop from dev to testing to production. Is has become apparent that the benefits of having a Continuous Testing cycle helps improvise the quality & TAT while your business solutions are in production.

 DevOps QA Is About Preventing Defects, Not Finding Them

 QA owns continuous improvement and quality tracking

 Tests are code

 Automation rules

 Testers are the quality advocates

 Mature automation testing framework

 Changed Role of QA in DevOps

 Traditional build which is deployed on their designated environment and QA would then commence their functional & regression testing.

 QA align their efforts in the DevOps cycle

 All their test cases are automated and achieve near 100% code coverage

 QA Environments are standardized and deployment on their QA boxes is automated

 All their pre-testing tasks, cleanups, post-testing tasks, etc. are automated and aligned with the Continuous Integration cycle.

 QA Engineers can configure their automation test cases into the DevOps chain

 Beyond Functional Testing: Automation for Load Testing, Stress Testing, and Performance Testing

1. History and current status of DevOps At the Agile 2008 conference, Andrew Shafer and Patrick Debois presented on "Agile Infrastructure". The term DevOps was popularized through a series of

"devopsdays" starting in 2009 in Belgium. Since then, there have been devopsdays conferences, held in many countries, worldwide.

Over the last five years, we’ve surveyed more than 25,000 technical professionals worldwide to better understand how DevOps practices impact IT and organizational performance.

We've looked closely at lean management practices, application architecture, the role of IT managers in a DevOps transformation, diversity, deployment pain and burnout.

And we've confirmed that there’s a lot more to IT performance than technical practices; organizations need to invest just as much in their people as they do in their technology.

This year, we look at the role of experimentation, how to integrate security into DevOps, the ROI of DevOps and more.

1.1 What is DevOps?

DevOps is the combination of cultural philosophies, practices, and tools that increases an organization’s ability to deliver applications and services at high velocity: evolving and improving products at a faster pace than organizations using traditional software development and infrastructure management processes. This speed enables organizations to better serve their customers and compete more effectively in the market.

Under a DevOps model, development and operations teams are no longer “siloed.” Sometimes, these two teams are merged into a single team where the engineers work across the entire application lifecycle, from development and test to deployment to operations, and develop a range of skills not limited to a single function. Quality assurance and security teams may also become more tightly integrated with development and operations and throughout the application lifecycle. These teams use practices to automate processes that historically have been manual and slow. They use a technology stack and tooling which help them operate and evolve applications quickly and reliably. These tools also help engineers independently accomplish tasks (for example, deploying code or provisioning infrastructure) that normally would have required help from other teams, and this further increases a team’s velocity.

1.2 How DevOps works: As DevOps is a truly cross-functional mode of working, there is no single "DevOps tool": it is rather a set (or "toolchain") of multiple tools. DevOps tools end to fit into one or more of these categories, reflective of key aspects of the development and delivery process:  Code — code development and review, source code management tools, code merging  Build — continuous integration tools, build status  Test — continuous testing tools that provide feedback on business risks  Package — artifact repository, application pre-deployment staging  Release — change management, release approvals, release automation  Configure — infrastructure configuration and management, Infrastructure as Code tools  Monitor — applications performance monitoring, end–user experience While there are many tools available, certain categories are essential in a DevOps toolchain; especially containerization (e.g. Docker), continuous integration (e.g. Jenkins), infrastructure as code (e.g. Puppet) and platform virtualization (e.g. Vagrant).

Illustration showing stages in a DevOps toolchain

2. Benefits of DevOps for Enterprise IT When implemented properly, DevOps can transform your software engineering process and create value for both employees and customers, producing business performance and value.

A fully working DevOps methodology will bring the following benefits:

1. Increased service quality

Perceived service quality and reliability depend on both the availability of a service in the absence of error (MTTF: Mean-Time-To-Failure) and the ability to restore the service to an error-free-state (MTTR: Mean-Time-To-Repair).

Due to fast feedback loops and high release velocity, service deficiencies are removed much faster than in the past (shortened MTTR), which leads to an improvement of perceived service quality and reliability.

2. Increased reliability of service delivery The perceived timeliness of delivery is an extremely important satisfaction factor for your internal or external customers Usable results are delivered early and frequently, which is critical in uncertain and changing business environments.

With agile and DevOps, big projects are cut into small chunks, which are then continuously delivered. Both initial and successive requirements and changes are delivered with short lead times, while changes are possible at any time in the process.

3. Increased customer value through responsiveness to change To stay relevant, businesses need to constantly adopt to a rapidly changing market environment, respond to competitors and stay ahead in the innovation cycle. Your business needs to act in a world of uncertainty.

This means that at the time a new idea is brought up, most of the project details cannot be known. Even once they are defined later on, they are subject to constant change. Complex upfront specifications – a necessary deliverable in the absence of short release cycles – often become obsolete.

4. Better usability increases customer satisfaction Shorter release cycles lead to shorter feedback cycles. Your customers are receiving frequent and timely updates, which gives you the ability to evaluate and test their satisfaction and response early on.

You can even test different versions of your product at the same time with different customer groups by A/B testing. This gives you the ability to compare different new features or capabilities of your products to each other.

5. Efficient operations Agile and DevOps are built on lean principles. The reduction of waste, such as unnecessary waiting times or process overhead through continuous improvement, is a core principle.

This not only helps agility, but also has a positive impact on costs by helps to limit their growth.

6. Reduce bottlenecks through cross-pollination of skills Blurring the line between dev and ops involves each side learning about the other’s trade. This enables staff to understand the whole process end-to-end and see where it can be tweaked and improved.

If ops know the basics of coding and vice-versa, bottlenecks can be removed by not having to rely on highly specialized individuals. Automation also helps here, for instance in the way a dev can use a service catalogue within Service Now to spin up environments on demand, without waiting for an ops member to have free time.

7. Holistic value creation increases employee satisfaction In a conventional IT organization, both dev and ops are often limited to executing specialized tasks. The value these tasks create for customers is often difficult for customers to measure, as it is one small aspect of a complex process.

Whether internal or external customers are served, the impact created by agile dev and ops teams is far more direct and holistic and can thus be more easily observed and “felt” by employees.

8. Collaboration improves employee attitude Whereas previously dev would hand code over to ops then switch to the next project, DevOps encourages collaboration throughout the team along the entire process. Not just exchanging emails, but face-to-face and through collaborative devices such as Kanban boards and Scrum meetings.

In the past, the two teams could have been working in different offices. DevOps brings IT together, and this shift in culture is likely to spread outside the department. This reduces waste through misaligned incentives causing political behaviour and mutual blame and increases employee satisfaction.

9. Constant learning and improvement DevOps promotes a culture of continuous learning and improvement through lean principles. In a fast-moving and unpredictable environment, continuous improvement is the single most important source of sustained competitive advantage and superior economic value.

A continuous learning environment also attracts and sustains outstanding talent. It is an important ingredient to employee satisfaction.

10.Reinvigorate your employees Apart from the new technical practices DevOps brings, perhaps one of the main advantages for IT is the shake-up it gives working lives. Like a new sports coach can bring a fresh perspective and increased work-rate amongst the team, so DevOps can reinvigorate software engineering teams.

New people to interact with, perhaps even a new office and a new way of working – DevOps can be just what IT needs to breathe life into stagnating staff who have been following the same routine for far too long.

3. Continuous Testing 3.1What is Continuous Testing? Continuous Testing is the process of executing automated tests as part of the software delivery pipeline in order to obtain feedback on the business risks associated with a software release candidate as rapidly as possible. It evolves and extends test automation to address the increased complexity and pace of modern application development and delivery. 3.2.Continuous Testing in DevOps:

Continuous Testing, when implemented diligently, allows flawless continuity in the delivery cycle. Integrating a fully automated Continuous Testing process into the SDLC is the most effective solution for a successful Continuous Delivery process.

DevOps at a Glance:

Set up flexible DevOps Quality Automation Framework:

The enhanced DevOps process requires maximum automation of all parts of the process. From User Story gathering to Continuous Development, Continuous Build, Continuous Deployment and Continuous Monitoring.

The commonly used tools for managing and realizing quality in DevOps environments are:

Use a virtualized Environment for DevOps Quality Validation:

A flexible and instantaneous DevOps Quality validation Environment is required to enable right coverage of validation checks in a repeatable, and cost –effective manner. Such an environment is based on three principles:

• Virtualization of Services, Network and Devices

• Cloud Based Environment Provisioning

• Test Data Management and Generation.

Proposed Roadmap to achieve DevOps with Quality:

For organizations that are implementing the DevOps philosophies, we recommend using our DevOps QBP approach to establish the baseline maturity, roadmap and continuous process for improvement in DevOps maturity.

The roadmap to DevOps with Quality will have to cover all the critical elements

 Integrated QA approach covering Continuous Testing to Continuous Quality Monitoring

 QA awareness and expertise among all disciplines

 Quality Engineering role in DevOps teams

 QA automation

 QA environment and test data provisioning

References & Appendix https://devops.com/continuous-testing-what-exactly-is-it/ https://en.wikipedia.org/wiki/Continuous_testing http://www.cigniti.com/blog/top-10-open-source-tools-for-testing-devops/

Author Biography

Gayathri Malladi is having 4 years of experience in Capgemini India Pvt ltd as a Quality Analyst.

Metta Murali Mohan is having 4 years of experience in Capgemini India Pvt ltd as a Quality Analyst.

THANK YOU!