NTT DATA welcomed Dell Services into the family in 2016. Together, Testing in DevOps: we offer one of the industry’s most comprehensive services portfolios An NTT DATA Services Point designed to modernize business and technology to deliver the outcomes of View that matter most to our clients.

Introduction Our approach to agile testing in DevOps In this evolving IT world, innovation and adaptability is the key Testing and QA follows a collaborative and continuous to remaining competitive. Testing in DevOps is a practice that approach. In our practice, we use shift-left testing for test follows next-generation, smart automation methodologies automation with integrated processes and continuous using proven processes, tools and subject matter experts feedback to DevOps agile project teams. This not only helps that help companies achieve their quality assurance (QA) improve early testing but also ensures increased test coverage objectives. At NTT DATA, we’re focused on reducing delays in general. By automating the testing solution, we’re able to and redundancy, and utilizing automation to implement utilize while also supporting integration and transformation like rapid development across the entire non-functional testing like performance, database, security value chain — a discipline of DevOps. We believe that test and mobility testing. The solution also takes care of other automation solutions must provide continuous feedback to dependencies like , test data management development and operations teams and enable rapid and and dynamic environment provisioning, ultimately meeting , ensuring bug-free build deployment at DevOps testing objectives. a push of a button. Table of contents

Introduction 1

Enabling continuous feedback inside the agile team 3

Solving the test environment dilemma 4

The NTT DATA way of automation 5

Client success story 6

What we offer 6

Written by: Saurabh Pandey

About the author: Saurabh is the center of excellence capability lead, ensuring organization-wide competency, and developing frameworks and best practices as well as supporting next-generation process and technology transfer/adoption for our customers’ IT ecosystem. He is an MBA and engineer with more than 15 years of rich experience. As an agile and DevOps evangelist, he provides consulting for automation, QA testing, independent verification and evaluation, and has helped organizations on transform their testing processes for the digital age. Saurabh Pandey

Saurabh developed the SDET Competency Development framework and a certification program widely accepted not only within Dell Services but also by our customers to help them jump-start the DevOps transformation journey.

2 Product owner The team Scrum master The team Everyone Gathers input from Prepares the release Measures progress • Enable executives, the backlog, selecting with burn down/up continuous team, stakeholders the highest priority charts delivery of and users stories for the sprint highest priority business value • Allow changes throughout Sprint Sprint review lifecycle, as planning 1–4 prioritized by week sprint cycle Finished customers Sprint end date work Automate labor- Product backlog and deliverables • Release intensive and Ranked list of planning do not change error-prone what is required Daily scrum Focus on small, activities, such meeting valuable, vertical as builds, chunks of work regression and to release value deployments earlier in the process

Figure 1: Inside the agile team — from sprint planning to the finished product

Accelerate delivery for Balance speed, cost, quality and risk Reduce time to customer feedback faster time to value for increased capacity to innovate for improved customer satisfaction experience

Continuous business planning

Continuous Steer customer Steer Collaborative feedback development and optimization

Automation AgileAgile Develop/ /regression continuous / continuous Develop/ feedback regression feedback test

Deploy Continuous Continuous Deploy monitoring testing

Continuous release and deployment

Figure 2: Enabling continuous feedback and processes in a testing environment with agile

3 The NTT DATA way of automation we use single-click deployment, which return on investment with fast, easy-to- SDETs commonly develop frameworks reduces manual effort and increases use automation testing processes. More that support DevOps principles ensuring value across , functional importantly, automation is planned within rapid and . They testing, and user the sprint, and the speed is maintained design these frameworks and scripts . Gone are the days across the release. before the development builds are where return on investment was the only created to streamline the entire software measurable parameter for automation. development lifecycle. At NTT DATA, We can help organizations maintain their

Technology

• Automation framework/tools: for functional and non-functional testing Process • Service virtualization: when dependent applications are • Automation across value chain unavailable or even before they have • Behavior-driven development: been built collaboration between developers, QA • Environment provisioning view: a People and non-technical participants centralized, automated system that • Test-driven development: technique triggers application deployment • engineers in process test (SDETs): qualified resources that using automated unit tests to drive the design of software and force • Mobility test framework: guidelines hold agile certifications and have for testing on various commercially expertise in development decoupling of dependencies Acceptance test-driven available mobile testing frameworks • Cross-functional experts: resources • development: methodology based like pCloudy, Perfecto Mobile and with multiple areas of expertise DeviceAnywhere working toward a common goal to on communication between the business customers, the developers • Test data management: Leveraging achieve increased performance and IPs and industry-leading tools produce a faster, higher quality and the testers product • Continuous Performance

In-house tools: smart performance testing framework, test data Third-party tools: SOAtest, Delphix, TestNow, management, environment provisioning view, agnostic BuildNow, Maven, Gradle, TestNG, Jenkins, Cucumber, Geb, automation accelerator Selenium

Figure 3: Integrating people, process and technology for end-to-end testing

4 Best practices in DevOps and agile testing Include testers and QA managers in the agile development team • Promote the importance of testers and encourage continuous feedback with the programmers and the customer • Ensures SDETs actively participate in meetings to define the main business flows and work on short iteration activities alongside developers • Utilize the specialized skills of test- driven development, including unit testing and continuous integration at the unit level • Leverage automation testing as a key method of

Benefits of agile testing in DevOps • SDET engagement: SDETs Agile challenges for testing Agile and can participate in the overview sprint engineers in DevOps improve and reduce risk planning, ensuring automated tests Since and code flux by taking advantage of cloud computing are fully leveraged to implement steadily during a sprint, ensuring full and automated deployment. Throughput regression testing. test coverage is not always straight can also be improved while increasing • SDET and developer sync: Because forward. Requirements churns effect test testing coverage and ensuring high the SDET is working with the coverage and can also lead to support quality. Typical agile testing activities developers on the same team, they of daily build tests. When there’s a tight include: are able to give immediate feedback, turnaround, a tendency to defer certain • Collaborative meetings: Testing which lets the developer quickly and types of testing to later sprints can result requirements are discussed and easily fix the bug in larger problems that are harder to fix refined as a team (during stand- • Implementation of continuous when so close to final delivery. Pushing up meetings/scrums) allowing the integration: Because testing is done performance, API, compatibility and combined team to better address continuously throughout the software to later in the software the business and technical aspects development lifecycle, it can’t be cut. development lifecycle will create a high- of the . This enables In waterfall projects, testing is often stress environment for the test team. overall alignment and helps prevent cut in order to meet deadlines. This misunderstandings. results in defective software being

Test data management Service virtualization Deployment automation

In DevOps, the agile test organization • Simulates the behavior of an entire • Orchestrates and automates the and testers need continuous access to application or system during testing deployment of applications, test data. With agile development, • Runs on commodity hardware, middleware configuration and continuous integration and delivery is private cloud and public cloud database changes into development, test and production also critical as frequent testing is • Provides each developer with their environments required. Our test data management own test environment • Enables automated, consistent software supports the agile method by • Allows developer and testers to deployments and rollbacks of streamlining access to test data and continue to use their existing testing applications providing up-to-date insights. tools (such as manual, web, • Ensures clear visibility performance and user interface test • Integrates with middleware, automation) provisioning and service virtualization

Figure 4: You can enable a dynamic, test-driven infrastructure by automating deployment, virtualizing test environments and integrating agile into test data management.

5 Client success story:

Business need Solution Benefits

One of the largest global hospitality • Shifted testing left by performing early • Increased productivity and defect- companies in the world managing a stage testing and adapting lean finding capacity portfolio of 3,897 hotels and resorts functional testing across six continents. In order to • Reduced effort by 50% • Focused on areas that could be improve their testing efforts, they wanted • Delivered release on time to automate regression packs and automated using Selenium and in- reduce time and effort. house utilities • Enabled reusable business components to streamline during • Implemented component-based test automation approach using Selenium development and maintenance lean framework Awards Winner of “Digital Accelerator Award” in • Created lean test abilities to reduce the manual repetitive effort the “Top API program” contest from Apigee. • Automated API Regression Test Suite 100% using SOAP UI (Groovy Scripts)

What we offer QA/test team transformation consulting: We can help you transform your organization to fit an agile/DevOps model and adopt DevOps principles and processes in-house. We offer a wide range of services including defining test/QA transformation strategies, helping you plan your DevOps implementation, training sessions for your in-house staff and suggesting best-in-class professional services to help get you started. QA/testing and automation services: If you’re ready to implement a DevOps approach, our experienced subject matter experts and engineers can deploy, manage and support all of your environments 24x7x365 — so your team can focus on the activities that grow your business.

Further reading “Part Dev, Part Ops = DevOps,” infographic by Dustin Whittle. App Dynamics. July 8, 2014. https://blog.appdynamics.com/devops/part-dev-part-ops/ http://searchsoftwarequality.techtarget.com/definition/Scrum-sprint

Visit nttdataservices.com to learn more.

NTT DATA partners with clients to navigate the modern complexities of business and technology, delivering the insights, solutions and outcomes that matter most. We’re a top 10 global IT services and consulting provider that wraps deep industry expertise around a comprehensive portfolio of infrastructure, applications and business process services. 6 © 2016 NTT DATA, Inc. All rights reserved. November 2016 | 1108_Agile_Testing_Point_of_view_paper.indd | Rev. 1.0